Commit graph

487 commits

Author SHA1 Message Date
Brian Goff
05ec12b082 Fix make shell
a3292263a3 broke `make shell` which is
trying to build the `dev` Dockerfile stage which no longer exists after
the change.

This adds the stage back.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2020-03-05 23:39:30 -08:00
Akihiro Suda
e0c9966a51
Merge pull request #40607 from AkihiroSuda/shfmt2
validate: add shfmt
2020-03-04 13:55:40 +09:00
Akihiro Suda
a3292263a3 integration: use systemd when $DOCKER_SYSTEMD is set
Fix #40492

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-03-04 03:03:38 +09:00
Akihiro Suda
eb484fcb67 validate: add shfmt
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-03-03 12:25:51 +09:00
Brian Goff
7a76104b2e
Merge pull request #40583 from thaJeztah/criu_3.13
Update CRIU to v3.13 "Silicon Willet"
2020-02-27 11:22:19 -08:00
Sebastiaan van Stijn
f508db4833
Update CRIU to v3.13 "Silicon Willet"
full diff: https://github.com/checkpoint-restore/criu/compare/v3.12...v3.13

Here we have some bugfixes, huuuge *.py patch for coding style
and nice set of new features like 32bit for ARM, TLS for page
server and new mode for CGroups.

New features

- VDSO: arm32 support
- Add TLS support for page server communications
- "Ignore" mode for --manage-cgroups
- Restore SO_BROADCAST option for inet sockets

Bugfixes

- Auxiliary events were left in inotify queues
- Lazy-pages daemon didn't detect stack pages and surrounders properly and marked them as "lazy"
- Memory and resource leakage were detected by coverity, cppcheck and clang

Improvements

- Use gettimeofday() directly from vdso for restore timings
- Reformat all .py code into pep8 style

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-02-26 14:17:25 +01:00
Tianon Gravi
0ce3ab679c Add a TODO in the Dockerfile regarding nftables
Someday, we'll hopefully support nftables directly and will likely then need some kind of in-container runtime detection (perhaps based on loaded modules or something similar).  This updates the related `Dockerfile` comment accordingly (linking to the appropriate `nftables` tracking issue).

Signed-off-by: Tianon Gravi <admwiggin@gmail.com>
2020-02-25 15:31:07 -08:00
Sebastiaan van Stijn
3f7503f98a
Update Golang 1.13.8
full diff: https://github.com/golang/go/compare/go1.13.7...go1.13.8

go1.13.8 (released 2020/02/12) includes fixes to the runtime, the crypto/x509,
and net/http packages. See the Go 1.13.8 milestone on the issue tracker for details.

https://github.com/golang/go/issues?q=milestone%3AGo1.13.8+label%3ACherryPickApproved

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-02-17 11:52:43 +01:00
Sebastiaan van Stijn
addc0c809f
Fix go-swagger to work with Go 1.14
The template used was broken with Go 1.14, which has some stricter
rules, introduced in https://golang.org/cl/206124

```
root@b5beaed18589:/go/src/github.com/docker/docker# ./hack/generate-swagger-api.sh
2020/02/07 15:12:01 trying to read config from /go/src/github.com/docker/docker/api/swagger-gen.yaml
2020/02/07 15:12:02 rendering 1 templates for model ErrorResponse
2020/02/07 15:12:02 name field ErrorResponse
2020/02/07 15:12:02 package field types
2020/02/07 15:12:02 creating "error_response.go" in "api/types" as definition
model: template: schematype:2:18: executing "schemaType" at <(len .AllOf) gt 0>: can't give argument to non-function len .AllOf
```

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-02-14 09:29:20 +01:00
Brian Goff
82879e9a9c Add vpnkit back to binary targets
This was removed after refactoring the Dockerfile in #40180

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2020-02-05 23:23:00 -08:00
Akihiro Suda
46ec8731fb
Merge pull request #40429 from thaJeztah/bump_golang_1.13.7
Update Golang 1.13.7, golang.org/x/crypto (CVE-2020-0601, CVE-2020-7919)
2020-02-03 18:09:20 +01:00
Brian Goff
5dffe74df2 Fix windows cross builds.
Windows still writes to the autogen directory, but the source code is
mounted in as read-only.
In order to do enable this without taking a massive hit in doing an rw
mount (for the source code) we mount a tmpfs into the build at the
autogen dir.
In order for this to work the directory must alreay exist, so we create
it before entering the build.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2020-01-30 15:43:23 -08:00
Sebastiaan van Stijn
878db479be
Update Golang 1.13.7 (CVE-2020-0601, CVE-2020-7919)
full diff: https://github.com/golang/go/compare/go1.13.6...go1.13.7

go1.13.7 (released 2020/01/28) includes two security fixes. One mitigates
the CVE-2020-0601 certificate verification bypass on Windows. The other affects
only 32-bit architectures.

https://github.com/golang/go/issues?q=milestone%3AGo1.13.7+label%3ACherryPickApproved

- X.509 certificate validation bypass on Windows 10
  A Windows vulnerability allows attackers to spoof valid certificate chains when
  the system root store is in use. These releases include a mitigation for Go
  applications, but it’s strongly recommended that affected users install the
  Windows security update to protect their system.
  This issue is CVE-2020-0601 and Go issue golang.org/issue/36834.
- Panic in crypto/x509 certificate parsing and golang.org/x/crypto/cryptobyte
  On 32-bit architectures, a malformed input to crypto/x509 or the ASN.1 parsing
  functions of golang.org/x/crypto/cryptobyte can lead to a panic.
  The malformed certificate can be delivered via a crypto/tls connection to a
  client, or to a server that accepts client certificates. net/http clients can
  be made to crash by an HTTPS server, while net/http servers that accept client
  certificates will recover the panic and are unaffected.
  Thanks to Project Wycheproof for providing the test cases that led to the
  discovery of this issue. The issue is CVE-2020-7919 and Go issue golang.org/issue/36837.
  This is also fixed in version v0.0.0-20200124225646-8b5121be2f68 of golang.org/x/crypto/cryptobyte.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-01-29 18:13:13 +01:00
Sebastiaan van Stijn
d68385b861
Update Golang 1.13.6
full diff: https://github.com/golang/go/compare/go1.13.5...go1.13.6

go1.13.6 (released 2020/01/09) includes fixes to the runtime and the net/http
package. See the Go 1.13.6 milestone on the issue tracker for details.

https://github.com/golang/go/issues?q=milestone%3AGo1.13.6+label%3ACherryPickApproved

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-01-17 15:18:44 +01:00
Sebastiaan van Stijn
50f1e205e4
Dockerfile: use tmpfs for building registry, go-swagger
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-01-10 15:31:14 +01:00
Sebastiaan van Stijn
95a4702943
Dockerfile: also build old registry on armhf
Building the old version of the registry is not supported
on arm64, but building on armhf is possible.

Before using a multi-arch Dockerfile, the registry was
built on armhf as well, but this got lost in the transition
in commit f1701a741d.

This commit re-enables building the old registry on armhf.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-01-10 15:31:12 +01:00
Sebastiaan van Stijn
bb0472bd23
Dockerfile: switch to iptables-legacy to match the host
CI runs on Ubuntu 16.04 machines, which use iptables (legacy), but
Debian buster uses nftables. Because of this, DNS resolution does not
work if the daemon configures iptables.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-01-09 18:52:07 +01:00
Sebastiaan van Stijn
4e3ab9e9fb
Dockerfile: switch golang image to "buster" variant, and update btrfs packages
The btrfs-tools was a transitional package, and no longer exists:

> Package btrfs-tools
> stretch (oldstable) (admin): transitional dummy package
> 4.7.3-1: amd64 arm64 armel armhf i386 mips mips64el mipsel ppc64el s390x

It must be replaced either by `btrfs-progs` or `libbtrfs-dev` (which has just the development headers)

> Package: libbtrfs-dev (4.20.1-2)
> Checksumming Copy on Write Filesystem utilities (development headers)

Note that the `libbtrfs-dev` package is not available on Debian stretch
(only in stretch-backports)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-01-09 18:52:05 +01:00
Akihiro Suda
e6f6c35b79
Merge pull request #40331 from cpuguy83/fix_bundle_output_dir
Fix output targets getting double nested.
2020-01-01 01:56:39 +09:00
Brian Goff
9177ac322a Fix output targets getting double nested.
Targets are going to bundles/bundles instead of just bundles/. This is
because there is `bundles` in the actual built binaries as well as the
output dir being set to bundles.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2019-12-30 13:20:11 -08:00
Sebastiaan van Stijn
a218e9b7b0
Update Golang 1.13.5
go1.13.5 (released 2019/12/04) includes fixes to the go command, the runtime, the
linker, and the net/http package. See the Go 1.13.5 milestone on our issue tracker
for details:

https://github.com/golang/go/issues?q=milestone%3AGo1.13.5+label%3ACherryPickApproved

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-12-23 17:32:32 +01:00
Brian Goff
5f314c24fc Bind-mount context for Dockerfile build targets
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2019-12-02 10:21:46 -08:00
Brian Goff
ccb0b0a7cc binary targets do not need the dev environment
This makes the binary build targets use a minimal build env instead of
having to build all the stuff needed for the full dev enviornment.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2019-12-02 09:52:16 -08:00
Sebastiaan van Stijn
44d3901386
Merge pull request #40163 from tao12345666333/update-golang-1.13.4
Bump Golang 1.13.4
2019-11-13 15:37:03 -08:00
Jintao Zhang
cf86eeaf96 Bump Golang 1.13.4
Signed-off-by: Jintao Zhang <zhangjintao9020@gmail.com>
2019-11-01 10:42:19 +08:00
Brian Goff
c057825cf5 Pass VERSION variable to binary build targets.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2019-10-30 10:07:45 -07:00
Jintao Zhang
635584280b Bump Golang 1.13.3 (CVE-2019-17596)
Signed-off-by: Jintao Zhang <zhangjintao9020@gmail.com>
2019-10-18 10:23:09 +08:00
Sebastiaan van Stijn
93edf327dc
Dockerfile: align consecutive COPY lines
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-11 15:51:47 +02:00
Sebastiaan van Stijn
8edbe5dec2
Dockerfile: order COPY lines by change frequency
Ordering the COPY lines to optimize for layer sharing
when these dependencies are updated.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-11 15:51:45 +02:00
Sebastiaan van Stijn
ee0ef6c535
Dockerfile: sort packages alphabetically
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-11 15:51:43 +02:00
Sebastiaan van Stijn
1cfcce5e21
Dockerfile: use build-arg for vpnkit
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-11 15:49:20 +02:00
Sebastiaan van Stijn
a42b4144bc
Dockerfile: use spaces for indentation
Indenting with tabs can cause the formatting to go wonky,
because the first line of any command is "indented" with spaces,
but following lines are not, therefore they can be mis-aligned with
the first line.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-11 15:49:17 +02:00
Sebastiaan van Stijn
f5eb39f75a
Merge pull request #39919 from jmartin84/36412-build-arg-override-binary-commits
Use build args to override binary commits in dockerfile
2019-10-11 15:43:27 +02:00
Tibor Vass
d56adcf0ec
Merge pull request #40059 from cpuguy83/move_source_copy
Fix trick for `make BINDDIR=. shell` and `COPY .`
2019-10-10 13:38:23 -07:00
Justen Martin
095ca77f48
Use build args to override binary commits in dockerfile
Signed-off-by: Justen Martin <jmart@the-coder.com>
2019-10-10 14:52:57 -05:00
Brian Goff
1c82d11908 Fix trick for make BINDDIR=. shell and COPY .
This was inadvertently removed when updating the Dockerfile for buildkit
specific features.

Trick selects a different build target depending on if the source is
going to be bind-mounted in anyway, which prevents the need to copy the
whole source tree to the builder.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2019-10-10 11:47:23 -07:00
Tibor Vass
6f069f14e6
Merge pull request #40058 from cpuguy83/buildkit_dockerfile
Fix `make cross` target
2019-10-10 11:29:19 -07:00
Kir Kolyshkin
93f9b902af go-swagger: fix panic
This is an attempt to fix go-swagger panic under Golang 1.13.

Details:
 * https://github.com/go-openapi/jsonpointer/pull/4
 * https://github.com/go-swagger/go-swagger/pull/2059

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2019-10-08 11:29:59 -07:00
Brian Goff
e5bfaf19b1 Fix make cross target
When changing the various cross targets in the Dockerfile I neglected
some `;`.
Instead of dealing with that now this just sets `--platform` on the
cross specific targets which only work on linux/amd64 anyway.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2019-10-08 10:54:39 -07:00
Brian Goff
82c56755ea Buildkit-optimized dockerfile
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2019-10-03 14:07:00 -07:00
Sebastiaan van Stijn
8eb23cde95
bump golang 1.13.1 (CVE-2019-16276)
full diff: https://github.com/golang/go/compare/go1.13...go1.13.1

```
Hi gophers,

We have just released Go 1.13.1 and Go 1.12.10 to address a recently reported security issue. We recommend that all affected users update to one of these releases (if you're not sure which, choose Go 1.13.1).

net/http (through net/textproto) used to accept and normalize invalid HTTP/1.1 headers with a space before the colon, in violation of RFC 7230. If a Go server is used behind an uncommon reverse proxy that accepts and forwards but doesn't normalize such invalid headers, the reverse proxy and the server can interpret the headers differently. This can lead to filter bypasses or request smuggling, the latter if requests from separate clients are multiplexed onto the same upstream connection by the proxy. Such invalid headers are now rejected by Go servers, and passed without normalization to Go client applications.

The issue is CVE-2019-16276 and Go issue golang.org/issue/34540.

Thanks to Andrew Stucki, Adam Scarr (99designs.com), and Jan Masarik (masarik.sh) for discovering and reporting this issue.

Downloads are available at https://golang.org/dl for all supported platforms.

Alla prossima,
Filippo on behalf of the Go team
```

From the patch: 6e6f4aaf70

```
net/textproto: don't normalize headers with spaces before the colon

RFC 7230 is clear about headers with a space before the colon, like

X-Answer : 42

being invalid, but we've been accepting and normalizing them for compatibility
purposes since CL 5690059 in 2012.

On the client side, this is harmless and indeed most browsers behave the same
to this day. On the server side, this becomes a security issue when the
behavior doesn't match that of a reverse proxy sitting in front of the server.

For example, if a WAF accepts them without normalizing them, it might be
possible to bypass its filters, because the Go server would interpret the
header differently. Worse, if the reverse proxy coalesces requests onto a
single HTTP/1.1 connection to a Go server, the understanding of the request
boundaries can get out of sync between them, allowing an attacker to tack an
arbitrary method and path onto a request by other clients, including
authentication headers unknown to the attacker.

This was recently presented at multiple security conferences:
https://portswigger.net/blog/http-desync-attacks-request-smuggling-reborn

net/http servers already reject header keys with invalid characters.
Simply stop normalizing extra spaces in net/textproto, let it return them
unchanged like it does for other invalid headers, and let net/http enforce
RFC 7230, which is HTTP specific. This loses us normalization on the client
side, but there's no right answer on the client side anyway, and hiding the
issue sounds worse than letting the application decide.
```

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-27 14:26:12 +02:00
Tibor Vass
d49fa5354c
Merge pull request #39549 from thaJeztah/bump_go_1.13
Bump go 1.13.0
2019-09-19 16:10:07 -07:00
Sebastiaan van Stijn
85b89c9e50
Replace gometalinter with golangci-lint
Replacing gometalinter, because it has been deprecated,
and the repository was archived.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-18 12:57:57 +02:00
Sebastiaan van Stijn
38e4ae3bca
Bump Golang version 1.13.0
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-12 00:19:29 +02:00
Sebastiaan van Stijn
961119db21
Dockerfile: set GO111MODULE=off
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-12 00:19:26 +02:00
Sebastiaan van Stijn
a464a3d51f
Dockerfile: remove GOMETALINTER_OPTS
This `ENV` was added to the Dockerfile in b96093fa56,
when the repository used per-architecture Dockerfiles, and some architectures needed
a different configuration.

Now that we use a multi-arch Dockerfile, and CI uses a Jenkinsfile, we can remove
this `ENV` from the Dockerfile, and set it in CI instead if needed.

Also updated the wording and fixed linting issues in hack/validate/gometalinter

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-11 08:58:38 +02:00
Sebastiaan van Stijn
d185ca78ec
Replace libprotobuf-c0-dev with libprotobuf-c-dev
The `libprotobuf-c0-dev` virtual package is no longer available
in Debian Buster, but is provided by `libprotobuf-c-dev`, which
is available.

https://packages.debian.org/stretch/libprotobuf-c0-dev

> Virtual Package: libprotobuf-c0-dev
>
> This is a virtual package. See the Debian policy for a definition of virtual packages.
>
> Packages providing libprotobuf-c0-dev
> libprotobuf-c-dev
> Protocol Buffers C static library and headers (protobuf-c)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-08-26 14:37:38 +02:00
Sebastiaan van Stijn
ad70bf6866
Dockerfile: add python3-wheel back again (for yamllint)
Although the Dockerfile builds without it, adding wheel back
should save some time

```
00:45:28  #14 10.70 Building wheels for collected packages: pathspec, pyyaml
00:45:28  #14 10.70   Running setup.py bdist_wheel for pathspec: started
00:45:28  #14 10.88   Running setup.py bdist_wheel for pathspec: finished with status 'error'
00:45:28  #14 10.88   Complete output from command /usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-mbotnxes/pathspec/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/tmpg9pl4u6kpip-wheel- --python-tag cp35:
00:45:28  #14 10.88   usage: -c [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
00:45:28  #14 10.88      or: -c --help [cmd1 cmd2 ...]
00:45:28  #14 10.88      or: -c --help-commands
00:45:28  #14 10.88      or: -c cmd --help
00:45:28  #14 10.88
00:45:28  #14 10.88   error: invalid command 'bdist_wheel'
00:45:28  #14 10.88
00:45:28  #14 10.88   ----------------------------------------
00:45:28  #14 10.88   Failed building wheel for pathspec
00:45:28  #14 10.88   Running setup.py clean for pathspec
00:45:28  #14 11.05   Running setup.py bdist_wheel for pyyaml: started
00:45:28  #14 11.25   Running setup.py bdist_wheel for pyyaml: finished with status 'error'
00:45:28  #14 11.25   Complete output from command /usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-mbotnxes/pyyaml/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/tmpyci_xi0bpip-wheel- --python-tag cp35:
00:45:28  #14 11.25   usage: -c [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
00:45:28  #14 11.25      or: -c --help [cmd1 cmd2 ...]
00:45:28  #14 11.25      or: -c --help-commands
00:45:28  #14 11.25      or: -c cmd --help
00:45:28  #14 11.25
00:45:28  #14 11.25   error: invalid command 'bdist_wheel'
00:45:28  #14 11.25
00:45:28  #14 11.25   ----------------------------------------
00:45:28  #14 11.25   Failed building wheel for pyyaml
00:45:28  #14 11.25   Running setup.py clean for pyyaml
00:45:28  #14 11.44 Failed to build pathspec pyyaml
00:45:28  #14 11.45 Installing collected packages: pathspec, pyyaml, yamllint
00:45:28  #14 11.45   Running setup.py install for pathspec: started
00:45:29  #14 11.73     Running setup.py install for pathspec: finished with status 'done'
00:45:29  #14 11.73   Running setup.py install for pyyaml: started
00:45:29  #14 12.05     Running setup.py install for pyyaml: finished with status 'done'
00:45:29  #14 12.12 Successfully installed pathspec-0.5.9 pyyaml-5.1.2 yamllint-1.16.0
```

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-08-19 13:24:01 +02:00
Jintao Zhang
01d6a56699 Bump Golang 1.12.9
Signed-off-by: Jintao Zhang <zhangjintao9020@gmail.com>
2019-08-16 10:02:22 +08:00
Sebastiaan van Stijn
00ad0222ce
Dockerfile: update CRIU to v3.12
New features

- build CRIU with Android NDK
- C/R of
  - IP RAW sockets
  - lsm: dump and restore any SELinux process label
  - support restoring ghost files on readonly mounts

Bugfixes

 - Do not lock network if running in the host network namespace
- Fix RPC configuration file handling
- util: don't leak file descriptors to third-party tools
- small fixes here and there

Improvements

- travis: switch to the Ubuntu Xenial
- travis-ci: Enable ia32 tests
- Many improvements and bug fixes in the libcriu
  - Changes in the API and ABI (SONAME increased from 1 to 2)

full diff: https://github.com/checkpoint-restore/criu/compare/v3.11...v3.12

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-08-14 20:43:52 +02:00