Fixes#41704
The latest released versions of the static binaries (20.10.3) are still unable
to use faccessat2 with musl-1.2.2 even though this was addressed in #41353 and
related issues. The underlying cause seems to be that the build system
here still uses the default version of libseccomp shipped with buster.
An updated version is available in buster backports:
https://packages.debian.org/buster-backports/libseccomp-dev
Signed-off-by: Jeremy Huntwork <jhuntwork@lightcubesolutions.com>
(cherry picked from commit 1600e851b5)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit 088e6ee790)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The `RUN --mount` options have been promoted to the stable channel,
so we can switch from "experimental" to "stable".
Note that the syntax directive should no longer be needed now, but
it's good practice to add a syntax-directive, to allow building on
older versions of docker.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 083dbe9fcd)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This fixes a regression based on expectations of the runtime:
```
docker pull arm32v7/alpine
docker run arm32v7/alpine
```
Without this change, the `docker run` will fail due to platform
matching on non-arm32v7 systems, even though the image could run
(assuming the system is setup correctly).
This also emits a warning to make sure that the user is aware that a
platform that does not match the default platform of the system is being
run, for the cases like:
```
docker pull --platform armhf busybox
docker run busybox
```
Not typically an issue if the requests are done together like that, but
if the image was already there and someone did `docker run` without an
explicit `--platform`, they may very well be expecting to run a native
version of the image instead of the armhf one.
This warning does add some extra noise in the case of platform specific
images being run, such as `arm32v7/alpine`, but this can be supressed by
explicitly setting the platform.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Currently we use the `go` command to read GOARCH and use the value for
matching manifests.
This change allows:
1. Specifying the arch through `TARGETARCH`
2. Falling back to `dpkg` if `go` is not available
3. Falling back to `uname -m` if `dpkg` is not available
4. A default value (amd64) if none of these commands is available.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
In order to run tests at mips64el device.
Now official-images has supported the following images for mips64el.
buildpack-deps:stretch
buildpack-deps:buster
debian:stretch
debian:buster
But official-images does not support the following images for mips64el.
debian:jessie
buildpack-deps:jessie
Signed-off-by: wanghuaiqing <wanghuaiqing@loongson.cn>
full diff: https://github.com/checkpoint-restore/criu/compare/v3.13...v3.14
New features
- C/R of memfd memory mappings and file descriptors.
- Add time namespace support.
- Add the read pre-dump mode which uses process_vm_readv.
- Add --cgroup-yard option
- Add support of the cgroup v2 freezer.
- Add support of opened O_PATH fds.
Bugfixes
- Fix C/R ia32 processes on AMD
- Fix cross-compilation
- Many fixes here and there
Improvements
- Use clone3() with set_tid to restore processes
- Clean up compel headers.
- Use the new mount API
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Usage: `dockerd-rootless-setuptool.sh install --force` .
See `--help` for further information.
Co-authored-by: Tianon Gravi <admwiggin@gmail.com>
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
These scripts are only used to build the binaries, and
the scripts themselves don't have to be persisted in an
image, so we can mount them instead of copying to the stage
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This version avoids doing name lookups on creating tarball that
should be avoided in to not hit loading glibc shared libraries.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit aa6a9891b0)
Signed-off-by: Tibor Vass <tibor@docker.com>
Issue: https://github.com/moby/moby/issues/35729
- Dockerfile now has an ARG for fully specifying GOLANG base image used for build
- Base image defaults to current patter
* This allows testing of alternate GOLANG environments
* Required downstream (EE) for things like FIPS testing
Signed-off-by: James Nesbitt <jnesbitt@mirantis.com>
a3292263a3 broke `make shell` which is
trying to build the `dev` Dockerfile stage which no longer exists after
the change.
This adds the stage back.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
full diff: https://github.com/checkpoint-restore/criu/compare/v3.12...v3.13
Here we have some bugfixes, huuuge *.py patch for coding style
and nice set of new features like 32bit for ARM, TLS for page
server and new mode for CGroups.
New features
- VDSO: arm32 support
- Add TLS support for page server communications
- "Ignore" mode for --manage-cgroups
- Restore SO_BROADCAST option for inet sockets
Bugfixes
- Auxiliary events were left in inotify queues
- Lazy-pages daemon didn't detect stack pages and surrounders properly and marked them as "lazy"
- Memory and resource leakage were detected by coverity, cppcheck and clang
Improvements
- Use gettimeofday() directly from vdso for restore timings
- Reformat all .py code into pep8 style
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Someday, we'll hopefully support nftables directly and will likely then need some kind of in-container runtime detection (perhaps based on loaded modules or something similar). This updates the related `Dockerfile` comment accordingly (linking to the appropriate `nftables` tracking issue).
Signed-off-by: Tianon Gravi <admwiggin@gmail.com>
The template used was broken with Go 1.14, which has some stricter
rules, introduced in https://golang.org/cl/206124
```
root@b5beaed18589:/go/src/github.com/docker/docker# ./hack/generate-swagger-api.sh
2020/02/07 15:12:01 trying to read config from /go/src/github.com/docker/docker/api/swagger-gen.yaml
2020/02/07 15:12:02 rendering 1 templates for model ErrorResponse
2020/02/07 15:12:02 name field ErrorResponse
2020/02/07 15:12:02 package field types
2020/02/07 15:12:02 creating "error_response.go" in "api/types" as definition
model: template: schematype:2:18: executing "schemaType" at <(len .AllOf) gt 0>: can't give argument to non-function len .AllOf
```
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Windows still writes to the autogen directory, but the source code is
mounted in as read-only.
In order to do enable this without taking a massive hit in doing an rw
mount (for the source code) we mount a tmpfs into the build at the
autogen dir.
In order for this to work the directory must alreay exist, so we create
it before entering the build.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
full diff: https://github.com/golang/go/compare/go1.13.6...go1.13.7
go1.13.7 (released 2020/01/28) includes two security fixes. One mitigates
the CVE-2020-0601 certificate verification bypass on Windows. The other affects
only 32-bit architectures.
https://github.com/golang/go/issues?q=milestone%3AGo1.13.7+label%3ACherryPickApproved
- X.509 certificate validation bypass on Windows 10
A Windows vulnerability allows attackers to spoof valid certificate chains when
the system root store is in use. These releases include a mitigation for Go
applications, but it’s strongly recommended that affected users install the
Windows security update to protect their system.
This issue is CVE-2020-0601 and Go issue golang.org/issue/36834.
- Panic in crypto/x509 certificate parsing and golang.org/x/crypto/cryptobyte
On 32-bit architectures, a malformed input to crypto/x509 or the ASN.1 parsing
functions of golang.org/x/crypto/cryptobyte can lead to a panic.
The malformed certificate can be delivered via a crypto/tls connection to a
client, or to a server that accepts client certificates. net/http clients can
be made to crash by an HTTPS server, while net/http servers that accept client
certificates will recover the panic and are unaffected.
Thanks to Project Wycheproof for providing the test cases that led to the
discovery of this issue. The issue is CVE-2020-7919 and Go issue golang.org/issue/36837.
This is also fixed in version v0.0.0-20200124225646-8b5121be2f68 of golang.org/x/crypto/cryptobyte.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Building the old version of the registry is not supported
on arm64, but building on armhf is possible.
Before using a multi-arch Dockerfile, the registry was
built on armhf as well, but this got lost in the transition
in commit f1701a741d.
This commit re-enables building the old registry on armhf.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
CI runs on Ubuntu 16.04 machines, which use iptables (legacy), but
Debian buster uses nftables. Because of this, DNS resolution does not
work if the daemon configures iptables.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The btrfs-tools was a transitional package, and no longer exists:
> Package btrfs-tools
> stretch (oldstable) (admin): transitional dummy package
> 4.7.3-1: amd64 arm64 armel armhf i386 mips mips64el mipsel ppc64el s390x
It must be replaced either by `btrfs-progs` or `libbtrfs-dev` (which has just the development headers)
> Package: libbtrfs-dev (4.20.1-2)
> Checksumming Copy on Write Filesystem utilities (development headers)
Note that the `libbtrfs-dev` package is not available on Debian stretch
(only in stretch-backports)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Targets are going to bundles/bundles instead of just bundles/. This is
because there is `bundles` in the actual built binaries as well as the
output dir being set to bundles.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
This makes the binary build targets use a minimal build env instead of
having to build all the stuff needed for the full dev enviornment.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Indenting with tabs can cause the formatting to go wonky,
because the first line of any command is "indented" with spaces,
but following lines are not, therefore they can be mis-aligned with
the first line.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This was inadvertently removed when updating the Dockerfile for buildkit
specific features.
Trick selects a different build target depending on if the source is
going to be bind-mounted in anyway, which prevents the need to copy the
whole source tree to the builder.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
When changing the various cross targets in the Dockerfile I neglected
some `;`.
Instead of dealing with that now this just sets `--platform` on the
cross specific targets which only work on linux/amd64 anyway.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
full diff: https://github.com/golang/go/compare/go1.13...go1.13.1
```
Hi gophers,
We have just released Go 1.13.1 and Go 1.12.10 to address a recently reported security issue. We recommend that all affected users update to one of these releases (if you're not sure which, choose Go 1.13.1).
net/http (through net/textproto) used to accept and normalize invalid HTTP/1.1 headers with a space before the colon, in violation of RFC 7230. If a Go server is used behind an uncommon reverse proxy that accepts and forwards but doesn't normalize such invalid headers, the reverse proxy and the server can interpret the headers differently. This can lead to filter bypasses or request smuggling, the latter if requests from separate clients are multiplexed onto the same upstream connection by the proxy. Such invalid headers are now rejected by Go servers, and passed without normalization to Go client applications.
The issue is CVE-2019-16276 and Go issue golang.org/issue/34540.
Thanks to Andrew Stucki, Adam Scarr (99designs.com), and Jan Masarik (masarik.sh) for discovering and reporting this issue.
Downloads are available at https://golang.org/dl for all supported platforms.
Alla prossima,
Filippo on behalf of the Go team
```
From the patch: 6e6f4aaf70
```
net/textproto: don't normalize headers with spaces before the colon
RFC 7230 is clear about headers with a space before the colon, like
X-Answer : 42
being invalid, but we've been accepting and normalizing them for compatibility
purposes since CL 5690059 in 2012.
On the client side, this is harmless and indeed most browsers behave the same
to this day. On the server side, this becomes a security issue when the
behavior doesn't match that of a reverse proxy sitting in front of the server.
For example, if a WAF accepts them without normalizing them, it might be
possible to bypass its filters, because the Go server would interpret the
header differently. Worse, if the reverse proxy coalesces requests onto a
single HTTP/1.1 connection to a Go server, the understanding of the request
boundaries can get out of sync between them, allowing an attacker to tack an
arbitrary method and path onto a request by other clients, including
authentication headers unknown to the attacker.
This was recently presented at multiple security conferences:
https://portswigger.net/blog/http-desync-attacks-request-smuggling-reborn
net/http servers already reject header keys with invalid characters.
Simply stop normalizing extra spaces in net/textproto, let it return them
unchanged like it does for other invalid headers, and let net/http enforce
RFC 7230, which is HTTP specific. This loses us normalization on the client
side, but there's no right answer on the client side anyway, and hiding the
issue sounds worse than letting the application decide.
```
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This `ENV` was added to the Dockerfile in b96093fa56,
when the repository used per-architecture Dockerfiles, and some architectures needed
a different configuration.
Now that we use a multi-arch Dockerfile, and CI uses a Jenkinsfile, we can remove
this `ENV` from the Dockerfile, and set it in CI instead if needed.
Also updated the wording and fixed linting issues in hack/validate/gometalinter
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The `libprotobuf-c0-dev` virtual package is no longer available
in Debian Buster, but is provided by `libprotobuf-c-dev`, which
is available.
https://packages.debian.org/stretch/libprotobuf-c0-dev
> Virtual Package: libprotobuf-c0-dev
>
> This is a virtual package. See the Debian policy for a definition of virtual packages.
>
> Packages providing libprotobuf-c0-dev
> libprotobuf-c-dev
> Protocol Buffers C static library and headers (protobuf-c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
New features
- build CRIU with Android NDK
- C/R of
- IP RAW sockets
- lsm: dump and restore any SELinux process label
- support restoring ghost files on readonly mounts
Bugfixes
- Do not lock network if running in the host network namespace
- Fix RPC configuration file handling
- util: don't leak file descriptors to third-party tools
- small fixes here and there
Improvements
- travis: switch to the Ubuntu Xenial
- travis-ci: Enable ia32 tests
- Many improvements and bug fixes in the libcriu
- Changes in the API and ABI (SONAME increased from 1 to 2)
full diff: https://github.com/checkpoint-restore/criu/compare/v3.11...v3.12
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
go1.12.8 (released 2019/08/13) includes security fixes to the net/http and net/url packages.
See the Go 1.12.8 milestone on our issue tracker for details:
https://github.com/golang/go/issues?q=milestone%3AGo1.12.8
- net/http: Denial of Service vulnerabilities in the HTTP/2 implementation
net/http and golang.org/x/net/http2 servers that accept direct connections from untrusted
clients could be remotely made to allocate an unlimited amount of memory, until the program
crashes. Servers will now close connections if the send queue accumulates too many control
messages.
The issues are CVE-2019-9512 and CVE-2019-9514, and Go issue golang.org/issue/33606.
Thanks to Jonathan Looney from Netflix for discovering and reporting these issues.
This is also fixed in version v0.0.0-20190813141303-74dc4d7220e7 of golang.org/x/net/http2.
net/url: parsing validation issue
- url.Parse would accept URLs with malformed hosts, such that the Host field could have arbitrary
suffixes that would appear in neither Hostname() nor Port(), allowing authorization bypasses
in certain applications. Note that URLs with invalid, not numeric ports will now return an error
from url.Parse.
The issue is CVE-2019-14809 and Go issue golang.org/issue/29098.
Thanks to Julian Hector and Nikolai Krein from Cure53, and Adi Cohen (adico.me) for discovering
and reporting this issue.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Using a build-arg so that we don't have to specify it for each
`apt-get install`, and to preserve that the `DEBIAN_FRONTEND` is
preserved in the image itself (which changes the default behavior,
and can be surprising if the image is run interactively).`
With this patch, some (harmless, but possibly confusing) errors
are no longer printed during build, for example:
```patch
Unpacking libgcc1:armhf (1:6.3.0-18+deb9u1) ...
Selecting previously unselected package libc6:armhf.
Preparing to unpack .../04-libc6_2.24-11+deb9u4_armhf.deb ...
-debconf: unable to initialize frontend: Dialog
-debconf: (TERM is not set, so the dialog frontend is not usable.)
-debconf: falling back to frontend: Readline
Unpacking libc6:armhf (2.24-11+deb9u4) ...
Selecting previously unselected package libgcc1:arm64.
Preparing to unpack .../05-libgcc1_1%3a6.3.0-18+deb9u1_arm64.deb ...
Unpacking libgcc1:arm64 (1:6.3.0-18+deb9u1) ...
Selecting previously unselected package libc6:arm64.
Preparing to unpack .../06-libc6_2.24-11+deb9u4_arm64.deb ...
-debconf: unable to initialize frontend: Dialog
-debconf: (TERM is not set, so the dialog frontend is not usable.)
-debconf: falling back to frontend: Readline
```
Looks like some output is now also printed on stdout instead of stderr
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This was inadvertedly removed in 7bfe48cc00,
because it was documented as a dependency for docker-py, but
actually used to validate the swagger file.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The Golang base images switch to buster, which causes some breakage
in networking and packages that are no longer available; (`btrfs-tools`
is now an empty package, and `libprotobuf-c0-dev` is gone).
Some of out tests also start faiilng on stretch, and will have to be
investigated further;
```
15:13:06 --- FAIL: TestRenameAnonymousContainer (3.37s)
15:13:06 rename_test.go:168: assertion failed: 0 (int) != 1 (inspect.State.ExitCode int): container a7fe866d588d65f353f42ffc5ea5288e52700384e1d90850e9c3d4dce8657666 exited with the wrong exitcode:
15:13:38 --- FAIL: TestHostnameDnsResolution (2.23s)
15:13:38 run_linux_test.go:128: assertion failed:
15:13:38 --- ←
15:13:38 +++ →
15:13:38 @@ -1 +1,2 @@
15:13:38 +ping: bad address 'foobar'
15:13:38
15:13:38
15:13:38 run_linux_test.go:129: assertion failed: 0 (int) != 1 (res.ExitCode int)
```
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This allows overriding the version of Go without making modifications in the
source code, which can be useful to test against multiple versions.
For example:
make GO_VERSION=1.13beta1 shell
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This removes all the installation steps for docker-py from the
Dockerfile, and instead builds the upstream Dockerfile, and runs
docker-py tests in a container.
To test;
```
make test-docker-py
...
Removing bundles/
---> Making bundle: dynbinary (in bundles/dynbinary)
Building: bundles/dynbinary-daemon/dockerd-dev
Created binary: bundles/dynbinary-daemon/dockerd-dev
---> Making bundle: test-docker-py (in bundles/test-docker-py)
---> Making bundle: .integration-daemon-start (in bundles/test-docker-py)
Using test binary docker
Starting dockerd
INFO: Waiting for daemon to start...
.
INFO: Building docker-sdk-python3:3.7.0...
sha256:686428ae28479e9b5c8fdad1cadc9b7a39b462e66bd13a7e35bd79c6a152a402
INFO: Starting docker-py tests...
============================= test session starts ==============================
platform linux -- Python 3.6.8, pytest-4.1.0, py-1.8.0, pluggy-0.9.0
rootdir: /src, inifile: pytest.ini
plugins: timeout-1.3.3, cov-2.6.1
collected 359 items
tests/integration/api_build_test.py .......s....
....
```
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The fastly cdn mirror we're using also mirrors the debian security
repository;
```
Welcome to deb.debian.org (fastly instance)!
This is deb.debian.org. This service provides mirrors for the following Debian archive repositories:
/debian/
/debian-debug/
/debian-ports/
/debian-security/
The server deb.debian.org does not have packages itself, but the name has SRV records in DNS that let apt in stretch and later find places.
```
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This cross-compiling other binaries simpler.
It would be nice if the cross Makefile target built all the required
bins, but at least this is a first step.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
