Commit graph

530 commits

Author SHA1 Message Date
Frédéric Dalleau
3a208c0ff9 Add ppc64le to cross platforms target
debian uses ppc64el
go uses ppc64le
the cross compiler is powerpc64le
Signed-off-by: Frédéric Dalleau <frederic.dalleau@docker.com>
2021-06-24 18:51:49 +02:00
Frédéric Dalleau
7168d98c43 Remove libseccomp dependency in runtime-dev-cross-true image
It turns out libseccomp is not used for building docker at all.
It is only used for building runc (and needs libseccomp > 2.4)

Signed-off-by: Frédéric Dalleau <frederic.dalleau@docker.com>
2021-06-22 22:18:03 +02:00
Sebastiaan van Stijn
ae5ddd257c
Bump go 1.16.5
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-06-10 13:04:45 +02:00
Brian Goff
a77317882d
Merge pull request #42262 from cpuguy83/move_libnetwork
Move libnetwork
2021-06-03 12:06:31 -07:00
Brian Goff
86c8b8f1a0 Move proxy build into hack/make
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2021-06-01 22:14:06 +00:00
Kir Kolyshkin
c8e6ad1adc
Dockerfile: install criu from binary repo
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-06-01 14:38:20 +02:00
moby
31b2c3bbd9
hack/vendor: add check for vendored archive/tar
Also allow re-vendoring using `./hack/vendor.sh archive/tar`

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-05-20 18:57:52 +02:00
Sebastiaan van Stijn
c35cefb489
Dockerfile: update yamllint to v1.26.1 to fix build
Installation of yamllint started failing, on non-amd64 builds, which could
be if the version we were using wasn't specific enough about a dependency
to install.

    copying Cython/Utility/CppSupport.cpp -> build/lib.linux-aarch64-3.7/Cython/Utility
    running build_ext
    building 'Cython.Plex.Scanners' extension
    creating build/temp.linux-aarch64-3.7
    creating build/temp.linux-aarch64-3.7/tmp
    creating build/temp.linux-aarch64-3.7/tmp/pip-install-jasgbmp7
    creating build/temp.linux-aarch64-3.7/tmp/pip-install-jasgbmp7/Cython
    creating build/temp.linux-aarch64-3.7/tmp/pip-install-jasgbmp7/Cython/Cython
    creating build/temp.linux-aarch64-3.7/tmp/pip-install-jasgbmp7/Cython/Cython/Plex
    aarch64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/usr/include/python3.7m -c /tmp/pip-install-jasgbmp7/Cython/Cython/Plex/Scanners.c -o build/temp.linux-aarch64-3.7/tmp/pip-install-jasgbmp7/Cython/Cython/Plex/Scanners.o
         /tmp/pip-install-jasgbmp7/Cython/Cython/Plex/Scanners.c:21:10: fatal error: Python.h: No such file or directory
          #include "Python.h"
                   ^~~~~~~~~~
         compilation terminated.
         error: command 'aarch64-linux-gnu-gcc' failed with exit status 1

         ----------------------------------------
     Command "/usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-jasgbmp7/Cython/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-if5qclwe/install-record.txt --single-version-externally-managed --prefix /tmp/pip-build-env-_dtiuyfw --compile" failed with error code 1 in /tmp/pip-install-jasgbmp7/Cython/

      ----------------------------------------
    Command "/usr/bin/python3 -m pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-_dtiuyfw --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- setuptools wheel Cython" failed with error code 1 in None
    #22 ERROR: executor failed running [/bin/sh -c pip3 install yamllint==1.16.0]: exit code: 1

Trying if updating to the latest version fixes this.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-04-15 21:30:10 +02:00
Sebastiaan van Stijn
1d4a273487
validate/toml: switch to github.com/pelletier/go-toml
The github.com/BurntSushi/toml project is no longer maintained,
and containerd is switching to this project instead, so start
moving our code as well.

This patch only changes the binary used during validation (tbh,
we could probably remove this validation step, but leaving that
for now).

I manually verified that the hack/verify/toml still works by adding a commit
that makes the MAINTAINERS file invalid;

        diff --git a/MAINTAINERS b/MAINTAINERS
        index b739e7e20c..81ababd8de 100644
        --- a/MAINTAINERS
        +++ b/MAINTAINERS
        @@ -23,7 +23,7 @@
                # a subsystem, they are responsible for doing so and holding the
                # subsystem maintainers accountable. If ownership is unclear, they are the de facto owners.

        -               people = [
        +               people =
                                "akihirosuda",
                                "anusha",
                                "coolljt0725",

Running `hack/verify/toml` was able to detect the broken format;

        hack/validate/toml
        (27, 4): keys cannot contain , characterThese files are not valid TOML:
         - MAINTAINERS

        Please reformat the above files as valid TOML

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-04-02 16:08:17 +02:00
Akihiro Suda
4735a0c84f
Merge pull request #42054 from jhuntwork/41704-update-libseccomp 2021-03-15 05:54:43 +09:00
Sebastiaan van Stijn
4648e8bc6f
Merge pull request #42067 from AkihiroSuda/vpnkit-arm64
Include VPNkit binary for arm64
2021-03-12 09:02:28 +01:00
Jeremy Huntwork
1600e851b5 Use buster backports to build with libseccomp-2.4.4
Fixes #41704

The latest released versions of the static binaries (20.10.3) are still unable
to use faccessat2 with musl-1.2.2 even though this was addressed in #41353 and
related issues. The underlying cause seems to be that the build system
here still uses the default version of libseccomp shipped with buster.
An updated version is available in buster backports:
https://packages.debian.org/buster-backports/libseccomp-dev

Signed-off-by: Jeremy Huntwork <jhuntwork@lightcubesolutions.com>
2021-03-10 10:01:28 -05:00
Akihiro Suda
088e6ee790
Include VPNkit binary for arm64
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-02-25 18:16:00 +09:00
Sebastiaan van Stijn
083dbe9fcd
Dockerfile: switch to "stable" dockerfile front-end
The `RUN --mount` options have been promoted to the stable channel,
so we can switch from "experimental" to "stable".

Note that the syntax directive should no longer be needed now, but
it's good practice to add a syntax-directive, to allow building on
older versions of docker.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-02-23 09:48:14 +01:00
Sebastiaan van Stijn
1c0af18c6c
vendor: opencontainers/selinux v1.8.0, and remove selinux build-tag and stubs
full diff: https://github.com/opencontainers/selinux/compare/v1.7.0...v1.8.0

Remove "selinux" build tag

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-12-24 00:47:16 +01:00
Eric Mountain
1c5806cf57
Move use of debian:buster frozen image to debian:bullseye
Signed-off-by: Eric Mountain <eric.mountain@datadoghq.com>
2020-12-19 17:59:36 +01:00
Eric Mountain
58a4c03fed
Add debian:bullseye to frozen images
Signed-off-by: Eric Mountain <eric.mountain@datadoghq.com>
2020-12-18 07:51:40 +01:00
Sebastiaan van Stijn
8a4671fb1f
Merge pull request #41556 from cpuguy83/41552_platform_regress
Don't set default platform on container create
2020-10-21 01:06:15 +02:00
Brian Goff
88c0271605 Don't set default platform on container create
This fixes a regression based on expectations of the runtime:

```
docker pull arm32v7/alpine
docker run arm32v7/alpine
```

Without this change, the `docker run` will fail due to platform
matching on non-arm32v7 systems, even though the image could run
(assuming the system is setup correctly).

This also emits a warning to make sure that the user is aware that a
platform that does not match the default platform of the system is being
run, for the cases like:

```
docker pull --platform armhf busybox
docker run busybox
```

Not typically an issue if the requests are done together like that, but
if the image was already there and someone did `docker run` without an
explicit `--platform`, they may very well be expecting to run a native
version of the image instead of the armhf one.

This warning does add some extra noise in the case of platform specific
images being run, such as `arm32v7/alpine`, but this can be supressed by
explicitly setting the platform.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2020-10-20 20:17:23 +00:00
Brian Goff
2036b277a5 Add TARGETARCH var to frozen images stage
This allows the arch that buildkit passes through to be used for the
frozen image downloader.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2020-09-29 15:42:03 -07:00
Brian Goff
dedf8528a5 Allow download-frozen-images to work without go
Currently we use the `go` command to read GOARCH and use the value for
matching manifests.

This change allows:

1. Specifying the arch through `TARGETARCH`
2. Falling back to `dpkg` if `go` is not available
3. Falling back to `uname -m` if `dpkg` is not available
4. A default value (amd64) if none of these commands is available.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2020-09-28 19:10:16 +00:00
Sebastiaan van Stijn
2a6325e310
Bump Golang 1.13.15
full diff: https://github.com/golang/go/compare/go1.13.14...go1.13.15

go1.13.15 (released 2020/08/06) includes security fixes to the encoding/binary
package. See the Go 1.13.15 milestone on the issue tracker for details.

https://github.com/golang/go/issues?q=milestone%3AGo1.13.15+label%3ACherryPickApproved

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-08-10 12:13:38 +02:00
Sebastiaan van Stijn
9c66a2f4e1
Bump Golang 1.13.14
full diff: https://github.com/golang/go/compare/go1.13.13...go1.13.14

go1.13.14 (released 2020/07/16) includes fixes to the compiler, vet, and the
database/sql, net/http, and reflect packages. See the Go 1.13.14 milestone on
the issue tracker for details:

https://github.com/golang/go/issues?q=milestone%3AGo1.13.14+label%3ACherryPickApproved

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-07-17 15:17:15 +02:00
Akihiro Suda
bc4f242e79
Bump Go 1.13.13
Includes security fixes to the `crypto/x509` and `net/http` packages.

https://github.com/golang/go/issues?q=milestone%3AGo1.13.13+label%3ACherryPickApproved

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-07-15 14:21:41 +09:00
wanghuaiqing
228d74842f Upgrading the versions of images in Dockerfile.
In order to run tests at mips64el device.
Now official-images has supported the following images for mips64el.
buildpack-deps:stretch
buildpack-deps:buster
debian:stretch
debian:buster

But official-images does not support the following images for mips64el.
debian:jessie
buildpack-deps:jessie

Signed-off-by: wanghuaiqing <wanghuaiqing@loongson.cn>
2020-06-30 12:24:06 +08:00
Jintao Zhang
004fd7be92 Bump Golang 1.13.12
Signed-off-by: Jintao Zhang <zhangjintao9020@gmail.com>
2020-06-07 20:04:59 +08:00
Tibor Vass
171dcbbcf2
Merge pull request #40895 from thaJeztah/bump_criu
Dockerfile: bump CRIU 3.14
2020-05-28 11:22:09 -07:00
Brian Goff
07e6b84359
Merge pull request #40950 from AkihiroSuda/dockerd-rootless-setuptool.sh
add dockerd-rootless-setuptool.sh
2020-05-21 11:52:07 -07:00
Sebastiaan van Stijn
a342010823
Dockerfile: bump CRIU 3.14
full diff: https://github.com/checkpoint-restore/criu/compare/v3.13...v3.14

New features

- C/R of memfd memory mappings and file descriptors.
- Add time namespace support.
- Add the read pre-dump mode which uses process_vm_readv.
- Add --cgroup-yard option
- Add support of the cgroup v2 freezer.
- Add support of opened O_PATH fds.

Bugfixes

- Fix C/R ia32 processes on AMD
- Fix cross-compilation
- Many fixes here and there

Improvements

- Use clone3() with set_tid to restore processes
- Clean up compel headers.
- Use the new mount API

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-05-18 12:13:13 +02:00
Sebastiaan van Stijn
90758fb028
Bump Golang 1.13.11
full diff: https://github.com/golang/go/compare/go1.13.10...go1.13.11

go1.13.11 (released 2020/05/14) includes fixes to the compiler. See the Go 1.13.11
milestone on the issue tracker for details:

https://github.com/golang/go/issues?q=milestone%3AGo1.13.11+label%3ACherryPickApproved

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-05-17 22:57:18 +02:00
Akihiro Suda
c696b95269 add dockerd-rootless-setuptool.sh
Usage: `dockerd-rootless-setuptool.sh install --force` .
See `--help` for further information.

Co-authored-by: Tianon Gravi <admwiggin@gmail.com>
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-05-15 08:46:36 +09:00
Sebastiaan van Stijn
e184eac73a
Merge pull request #40380 from thaJeztah/dockerfile_improvements
Dockerfile: improve binary build stages
2020-05-08 15:00:35 +02:00
Akihiro Suda
cdf49f37ee
Merge pull request #40672 from tiborvass/19.03.8-forwardport-to-master
vendor: add local copy of archive/tar
2020-05-08 08:11:03 +09:00
Sebastiaan van Stijn
de7f0120fd
Dockerfile: mount install scripts instead of copying
These scripts are only used to build the binaries, and
the scripts themselves don't have to be persisted in an
image, so we can mount them instead of copying to the stage

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-05-07 21:36:37 +02:00
Sebastiaan van Stijn
d1a3a17c67
Dockerfile: remove "INSTALL_BINARY_NAME" envs
Using a variable here doesn't seem to add much, so inlining
the value instead.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-05-07 21:35:54 +02:00
Sebastiaan van Stijn
84d7b57eb7
Dockerfile: update syntax to 1.1.7-experimental
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-04-29 14:30:23 +02:00
Tonis Tiigi
7260adfff9 vendor: add local copy of archive/tar
This version avoids doing name lookups on creating tarball that
should be avoided in to not hit loading glibc shared libraries.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit aa6a9891b0)
Signed-off-by: Tibor Vass <tibor@docker.com>
2020-04-24 11:22:24 -07:00
Sebastiaan van Stijn
7cb13d4d85
Bump Golang 1.13.10
go1.13.10 (released 2020/04/08) includes fixes to the go command, the runtime,
os/exec, and time packages. See the Go 1.13.10 milestone on the issue tracker
for details:

https://github.com/golang/go/issues?q=milestone%3AGo1.13.10+label%3ACherryPickApproved

full diff: https://github.com/golang/go/compare/go1.13.9...go1.13.10

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-04-09 21:58:09 +02:00
Sebastiaan van Stijn
6ee9a1ad29
Bump Golang 1.13.9
go1.13.9 (released 2020/03/19) includes fixes to the go command, tools, the
runtime, the toolchain, and the crypto/cypher package. See the Go 1.13.9
milestone on the issue tracker for details:

https://github.com/golang/go/issues?q=milestone%3AGo1.13.9+label%3ACherryPickApproved

full diff: https://github.com/golang/go/compare/go1.13.8...go1.13.9

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-04-04 00:44:25 +02:00
James Nesbitt
3dd0262773 build Dockerfile allows arg for base build FROM
Issue: https://github.com/moby/moby/issues/35729

- Dockerfile now has an ARG for fully specifying GOLANG base image used for build
- Base image defaults to current patter

* This allows testing of alternate GOLANG environments
* Required downstream (EE) for things like FIPS testing

Signed-off-by: James Nesbitt <jnesbitt@mirantis.com>
2020-03-30 12:27:52 -04:00
Brian Goff
37defbfd9b
Merge pull request #40633 from AkihiroSuda/update-vpnkit
update VPNKit
2020-03-13 15:45:19 -07:00
Akihiro Suda
93f3532f64 update VPNKit
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-03-12 17:20:58 +09:00
Akihiro Suda
5bf33adba2 hack: support $DOCKER_ROOTLESS for testing rootless
```
$ DOCKER_EXPERIMENTAL=1 DOCKER_ROOTLESS=1 TEST_SKIP_INTEGRATION_CLI=1 \
 make test-integration
```

test-integration-cli is unsupported currently.
Also, tests that spawn custom daemon (testutil/daemon) are skipped.

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-03-10 05:49:45 +09:00
Brian Goff
05ec12b082 Fix make shell
a3292263a3 broke `make shell` which is
trying to build the `dev` Dockerfile stage which no longer exists after
the change.

This adds the stage back.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2020-03-05 23:39:30 -08:00
Akihiro Suda
e0c9966a51
Merge pull request #40607 from AkihiroSuda/shfmt2
validate: add shfmt
2020-03-04 13:55:40 +09:00
Akihiro Suda
a3292263a3 integration: use systemd when $DOCKER_SYSTEMD is set
Fix #40492

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-03-04 03:03:38 +09:00
Akihiro Suda
eb484fcb67 validate: add shfmt
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-03-03 12:25:51 +09:00
Brian Goff
7a76104b2e
Merge pull request #40583 from thaJeztah/criu_3.13
Update CRIU to v3.13 "Silicon Willet"
2020-02-27 11:22:19 -08:00
Sebastiaan van Stijn
f508db4833
Update CRIU to v3.13 "Silicon Willet"
full diff: https://github.com/checkpoint-restore/criu/compare/v3.12...v3.13

Here we have some bugfixes, huuuge *.py patch for coding style
and nice set of new features like 32bit for ARM, TLS for page
server and new mode for CGroups.

New features

- VDSO: arm32 support
- Add TLS support for page server communications
- "Ignore" mode for --manage-cgroups
- Restore SO_BROADCAST option for inet sockets

Bugfixes

- Auxiliary events were left in inotify queues
- Lazy-pages daemon didn't detect stack pages and surrounders properly and marked them as "lazy"
- Memory and resource leakage were detected by coverity, cppcheck and clang

Improvements

- Use gettimeofday() directly from vdso for restore timings
- Reformat all .py code into pep8 style

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-02-26 14:17:25 +01:00
Tianon Gravi
0ce3ab679c Add a TODO in the Dockerfile regarding nftables
Someday, we'll hopefully support nftables directly and will likely then need some kind of in-container runtime detection (perhaps based on loaded modules or something similar).  This updates the related `Dockerfile` comment accordingly (linking to the appropriate `nftables` tracking issue).

Signed-off-by: Tianon Gravi <admwiggin@gmail.com>
2020-02-25 15:31:07 -08:00