Commit graph

3011 commits

Author SHA1 Message Date
Akihiro Suda
5ca47f5179 rootless: graduate from experimental
Close #40484

Note that the support for cgroup v2 isn't ready for production yet,
regardless to rootful or rootless.

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-04-07 00:59:15 +09:00
Akihiro Suda
846b557cb8
Merge pull request #40664 from carlosedp/riscv64-build
Add riscv64 support to the build scripts
2020-04-04 07:04:28 +09:00
Carlos de Paula
7ac638f86a Add support to riscv64 to the build scripts
Added riscv64 architecture support to the scripts used to build Docker
and it's dependencies.

Signed-off-by: Carlos de Paula <me@carlosedp.com>
2020-04-03 14:33:32 -03:00
Sebastiaan van Stijn
97b742676b
hack/ci/windows.ps1: build busybox from local Dockerfile
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-04-03 09:32:30 +02:00
Sebastiaan van Stijn
7f8b4b621b
Merge pull request #40689 from AkihiroSuda/test-rootless2
test-integration: support more rootless tests
2020-03-30 14:13:34 +02:00
Jintao Zhang
ea44be4b8f vendor: bump libnetwork ef149a924dfde2e506ea3cb3f617d7d0fa96b8ee
fix https://github.com/moby/moby/issues/40715

Signed-off-by: Jintao Zhang <zhangjintao9020@gmail.com>
2020-03-19 17:15:58 +08:00
Akihiro Suda
f310bd29bd rootless: support forwarding signals from RootlessKit to dockerd
See https://github.com/rootless-containers/rootlesskit/pull/127

RootlessKit changes: https://github.com/rootless-containers/rootlesskit/compare/v0.9.1...v0.9.2

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-03-15 12:24:23 +09:00
Sebastiaan van Stijn
e26e1cc5c1
vendor: vishvananda/netlink v1.1.0
- vishvananda/netlink: https://github.com/vishvananda/netlink/compare/v1.0.0...v1.1.0
- vishvananda/netns: 7109fa855b...0a2b9b5464
- libnetwork: bf2bd42abc...beab24292c
- github.com/moby/ipvs: new dependency (was previously part of libnetwork)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-03-12 22:57:23 +01:00
Tianon Gravi
87c2e65259
Merge pull request #40661 from thaJeztah/update_golangci_lint
ci: update golangci-lint v1.23.8
2020-03-10 14:09:41 -07:00
Sebastiaan van Stijn
51ea39c592
Merge pull request #40658 from AkihiroSuda/install-shim-v2
containerd.installer: install containerd-shim-runc-v2
2020-03-10 16:06:28 +01:00
Sebastiaan van Stijn
e36d222666
ci: update golangci-lint v1.23.8
full diff: https://github.com/golangci/golangci-lint/compare/v1.20.0...v1.23.8

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-03-10 11:52:40 +01:00
Akihiro Suda
d949fddc6c containerd.installer: install containerd-shim-runc-v2
cgroup2 mode requires containerd-shim-runc-v2 (containerd v1.4).

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-03-10 10:35:21 +09:00
Akihiro Suda
5bf33adba2 hack: support $DOCKER_ROOTLESS for testing rootless
```
$ DOCKER_EXPERIMENTAL=1 DOCKER_ROOTLESS=1 TEST_SKIP_INTEGRATION_CLI=1 \
 make test-integration
```

test-integration-cli is unsupported currently.
Also, tests that spawn custom daemon (testutil/daemon) are skipped.

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-03-10 05:49:45 +09:00
Sebastiaan van Stijn
d5595a5845
Merge pull request #40596 from thaJeztah/bump_libnetwork
vendor: bump libnetwork bf2bd42abc0a3734f12b5ec724e571434e42c669
2020-03-09 17:23:06 +01:00
Akihiro Suda
1ea3a2b7f5 rootless: launch rootlesskit with --propagation=rslave
The propagation was previously set to rprivate and didn't propagate
mounts from the host mount namespace into the daemon's mount namespace.

Further information about --propagation: https://github.com/rootless-containers/rootlesskit/tree/v0.9.1#mount-propagation
RootlessKit changes: https://github.com/rootless-containers/rootlesskit/compare/v0.8.0...v0.9.1

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-03-07 21:16:29 +09:00
Sebastiaan van Stijn
e1710b42d3
vendor: bump libnetwork bf2bd42abc0a3734f12b5ec724e571434e42c669
full diff: 264bffcb88...bf2bd42abc

relevant changes:

- docker/libnetwork#2407 Macvlan internal network should not change default gateway
    - fixes docker/libnetwork#2406 Internal macvlan network overrides default gateway
- vendor godbus/dbus v5
- Fix InhibitIPv4 nil panic
- Cleanup VFP during overlay network removal
    - fixes VFP leak in windows overlay network deletion

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-03-07 00:35:47 +01:00
Akihiro Suda
abf2404642 hack/make.sh: remove extra empty lines
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-03-03 12:36:06 +09:00
Akihiro Suda
3cf82748dd run shfmt
git grep --name-only '^#!' | egrep -v '(vendor|\.go|Jenkinsfile)' | xargs shfmt -w -bn -ci -sr

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-03-03 12:27:49 +09:00
Akihiro Suda
eb484fcb67 validate: add shfmt
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-03-03 12:25:51 +09:00
Brian Goff
76e3a49933
Merge pull request #40486 from AkihiroSuda/rootless-cgroup2-systemd
rootless: support `--exec-opt native.cgroupdriver=systemd`
2020-03-02 16:11:21 -08:00
Stefan Scherer
fdf42f7a59
Update Windows ico
Signed-off-by: Stefan Scherer <stefan.scherer@docker.com>
2020-03-02 12:52:10 +01:00
Sam Whited
8c0f6142a8 Update DNS library
This makes sure that we don't become vulnerable to CVE-2018-17419 or
CVE-2019-19794 in the future.

Signed-off-by: Sam Whited <sam@samwhited.com>
2020-02-25 17:05:57 -05:00
Sebastiaan van Stijn
79b130ac31
Revert "Fix containerutility compilation on gcc-mingw-w64 8.3"
This reverts commit 25a1bf53d2.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-02-14 23:31:42 +01:00
Sebastiaan van Stijn
5125f8b304
bump windows-container-utility aa1ba87e99b68e0113bd27ec26c60b88f9d4ccd9
full diff: e004a1415a...aa1ba87e99

changes:

- Use standard include paths instead of hard-coding

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-02-14 23:31:14 +01:00
Sebastiaan van Stijn
d8772509d1
Merge pull request #40489 from pricec/fix-license-check
Vendor check to succeed even if copyright information is not found
2020-02-14 23:19:43 +01:00
Akihiro Suda
ca4b51868a rootless: support --exec-opt native.cgroupdriver=systemd
Support cgroup as in Rootless Podman.

Requires cgroup v2 host with crun.
Tested with Ubuntu 19.10 (kernel 5.3, systemd 242), crun v0.12.1.

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-02-14 15:32:31 +09:00
Chris Price
9f94cfdc57 Vendor check to succeed even if copyright information is not found
The documentation for validate_vendor_used in hack/validate/vendor states
that a warning will be emitted if license information cannot be found in
a vendored package. However, because the script is run with pipefail set
(owing to the inclusion of the common validation script .validate) and
`grep -c` is used, the entire script will fail whenever license information
cannot be found in a vendored package.

Signed-off-by: Chris Price <cprice@mirantis.com>
2020-02-13 06:08:44 -08:00
Sebastiaan van Stijn
da6c1429d0
docker-py: skip flaky AttachContainerTest::test_attach_no_stream (again)
This test was disabled in the past, but re-enabled when we upgraded
docker-py to 4.2.0.

The test looks to be still flaky though, so skipping it again:

```
[2020-02-10T23:40:44.429Z] =================================== FAILURES ===================================
[2020-02-10T23:40:44.429Z] __________________ AttachContainerTest.test_attach_no_stream ___________________
[2020-02-10T23:40:44.429Z] tests/integration/api_container_test.py:1250: in test_attach_no_stream
[2020-02-10T23:40:44.429Z]     assert output == 'hello\n'.encode(encoding='ascii')
[2020-02-10T23:40:44.429Z] E   AssertionError: assert b'' == b'hello\n'
[2020-02-10T23:40:44.429Z] E     Right contains more items, first extra item: 104
[2020-02-10T23:40:44.429Z] E     Use -v to get the full diff
[2020-02-10T23:40:44.429Z] ------- generated xml file: /src/bundles/test-docker-py/junit-report.xml -------
````

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-02-12 18:37:58 +01:00
Sebastiaan van Stijn
5a028d2736
docker-py: override default API version to run all tests
Otherwise some tests are skipped with the default API version
used:

    SKIPPED [1] tests/integration/api_service_test.py:882: API version is too low (< 1.38)
    SKIPPED [1] tests/integration/api_swarm_test.py:59: API version is too low (< 1.39)
    SKIPPED [1] tests/integration/api_swarm_test.py:38: API version is too low (< 1.39)
    SKIPPED [1] tests/integration/api_swarm_test.py:45: API version is too low (< 1.39)
    SKIPPED [1] tests/integration/api_swarm_test.py:52: API version is too low (< 1.39)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-02-11 15:50:49 +01:00
Brian Goff
3473ff2893
Merge pull request #40469 from thaJeztah/containerd_1.3.3
update containerd binary to v1.3.3
2020-02-07 12:16:40 -08:00
Sebastiaan van Stijn
486161a63a
bump vndr v0.1.1
full diff: https:/github.com/LK4D4/vndr/compare/v0.1.0...v0.1.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-02-07 13:37:45 +01:00
Sebastiaan van Stijn
27649ee44f
update containerd binary to v1.3.3
full diff: https://github.com/containerd/containerd/compare/v1.3.2...v1.3.3
release notes: https://github.com/containerd/containerd/releases/tag/v1.3.3

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-02-07 10:21:01 +01:00
Sebastiaan van Stijn
1cb4a60950
testing: bump docker-py to 4.2.0
full diff: https://github.com/docker/docker-py/compare/4.1.0...4.2.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-02-06 11:55:18 +01:00
Brian Goff
5dffe74df2 Fix windows cross builds.
Windows still writes to the autogen directory, but the source code is
mounted in as read-only.
In order to do enable this without taking a massive hit in doing an rw
mount (for the source code) we mount a tmpfs into the build at the
autogen dir.
In order for this to work the directory must alreay exist, so we create
it before entering the build.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2020-01-30 15:43:23 -08:00
Akihiro Suda
cd43c1d1ac update runc binary to v1.0.0-rc10 (CVE-2019-19921)
Notable changes:
* Fix CVE-2019-19921 (Volume mount race condition with shared mounts): https://github.com/opencontainers/runc/pull/2207
* Fix exec FIFO race: https://github.com/opencontainers/runc/pull/2185
* Basic support for cgroup v2.  Almost feature-complete, but still missing support for systemd mode in rootless.
  See also https://github.com/opencontainers/runc/issues/2209 for the known issues.

Full changes: https://github.com/opencontainers/runc/compare/v1.0.0-rc9...v1.0.0-rc10

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-01-25 03:55:39 +09:00
Sebastiaan van Stijn
572cb55c0e
hack/make/.go-autogen: fix LDFLAGS to accept values with spaces
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-01-18 00:40:06 +01:00
Sebastiaan van Stijn
e8122ed724
hack/make/.go-autogen: fix typo in LDFLAGS variable
Introduced in 675b414f56

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-01-17 12:39:56 +01:00
Arko Dasgupta
4c407caada Bump libnetwork to latest refpoint
Commits:
feeff4f0 Merge pull request #2380 from liskin/bridge-atomic-hwaddr
fec6476d Merge pull request #2489 from suwang48404/doc
8757597e Added document describing libnetwork traffic flow.
eaea5722 Merge pull request #2445 from kdomanski/ipv6-addr-in-hosts
1680ce71 Merge pull request #2462 from arkodg/fix-key-spi-panic
4420ee92 Fix panic in drivers/overlay/encryption.go
57178323 Merge pull request #2472 from thaJeztah/bump_golang_1.12.12
f741dc9c Update Golang 1.12.12 (CVE-2019-17596)
79c19d09 Merge pull request #2461 from suwang48404/master
94facacc Added API to set ephemeral port allocator range.

Signed-off-by: Arko Dasgupta <arko.dasgupta@docker.com>
2020-01-16 16:28:23 -08:00
Brian Goff
1d699d40f4
Merge pull request #40359 from thaJeztah/bump_vndr
bump vndr v0.1.0 to support versioned import paths
2020-01-10 09:21:39 -08:00
Sebastiaan van Stijn
d4f05c168d
bump vndr v0.1.0 to support versioned import paths
With this change, go packages/modules that use versioned
import paths (github.com/foo/bar/v2), but don't use a directory
in the repository, can now be supported.

For example:

```
github.com/coreos/go-systemd/v22 v22.0.0
```

will vendor the github.com/coreos/go-systemd repository
into `vendor/github.com/coreos/go-systemd/v22`.

full diff: f5ab8fc5fb...v0.1.0

- LK4D4/vndr#83 migrate bitbucket to api 2.0
    - fixes LK4D4/vndr#82 https://api.bitbucket.org/1.0/repositories/ww/goautoneg: 410 Gone
- LK4D4/vndr#86 Replace sort.Sort with sort.Strings
- LK4D4/vndr#87 support `github.com/coreos/go-systemd/v22`

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-01-09 21:56:20 +01:00
Sebastiaan van Stijn
25a1bf53d2
Fix containerutility compilation on gcc-mingw-w64 8.3
The makefile for this binary has version 6.3 hardcoded,
which causes compilation on 8.3 to fail:

```
Building: bundles/cross/windows/amd64/containerutility.exe
In file included from /usr/x86_64-w64-mingw32/include/minwindef.h:163,
                 from /usr/x86_64-w64-mingw32/include/windef.h:8,
                 from /usr/x86_64-w64-mingw32/include/windows.h:69,
                 from containerutility.h:3,
                 from argumentstream.cpp:1:
/usr/x86_64-w64-mingw32/include/winnt.h:1554:11: fatal error: x86intrin.h: No such file or directory
 # include <x86intrin.h>
           ^~~~~~~~~~~~~
compilation terminated.
```

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-01-09 18:52:09 +01:00
Sebastiaan van Stijn
5dbfae6949
hack/make: ignore failure to stop apparmor
```
 ---> Making bundle: .integration-daemon-stop (in bundles/test-integration)
 ++++ cat bundles/test-integration/docker.pid
 +++ kill 13137
 +++ /etc/init.d/apparmor stop
 Leaving: AppArmorNo profiles have been unloaded.

 Unloading profiles will leave already running processes permanently
 unconfined, which can lead to unexpected situations.

 To set a process to complain mode, use the command line tool
 'aa-complain'. To really tear down all profiles, run 'aa-teardown'."

script returned exit code 255
```

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-01-09 18:52:03 +01:00
Sebastiaan van Stijn
f2b25e498f
docker-py: skip broken ImageCollectionTest::test_pull_multiple
The ImageCollectionTest.test_pull_multiple test performs a `docker pull` without
a `:tag` specified) to pull all tags of the given repository (image).

After pulling the image, the image(s) pulled are checked to verify if the list
of images contains the `:latest` tag.

However, the test assumes that all tags of the image are tags for the same
version of the image (same digest), and thus a *single* image is returned, which
is not always the case.

Currently, the `hello-world:latest` and `hello-world:linux` tags point to a
different digest, therefore the `client.images.pull()` returns multiple images:
one image for digest, making the test fail:

    =================================== FAILURES ===================================
    ____________________ ImageCollectionTest.test_pull_multiple ____________________
    tests/integration/models_images_test.py:90: in test_pull_multiple
        assert len(images) == 1
    E   AssertionError: assert 2 == 1
    E    +  where 2 = len([<Image: 'hello-world:linux'>, <Image: 'hello-world:latest'>])

This patch temporarily skips the broken test until it is fixed upstream.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-01-03 15:47:39 +01:00
Sebastiaan van Stijn
6bc45b09e7
docker-py: re-enable tests that were fixed in v4.1.0
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-01-03 15:46:28 +01:00
Sebastiaan van Stijn
bbcabf69c8
Merge pull request #40050 from thaJeztah/ci_force_validate
Jenkinsfile: force validate on master, and some fixes/improvements
2019-12-23 13:39:43 +01:00
Vikram bir Singh
8d2e1ee546
Build and use gotestsum for running all tests
1. Dockerfile.Windows modified to build gotestsum.exe

2. Use gotestsum.exe in invoking the execution of:

   (a) Unit tests (run in containers),
   (b) Integration tests (run outside containers)
   (c) Integration-cli (run outside containers)

No changes made to other categories of tests (e.g.
LCOW).

3. Copy .xml files produced by gotestsum in
   appropriate paths where Jenkins can ingest them

4. Modify Jenkinsfile to mark results output as
   being jUnit "type" as well as to archive the
   .xml test result files as artifacts.

Signed-off-by: Vikram bir Singh <vikrambir.singh@docker.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-12-19 20:31:22 +01:00
Sebastiaan van Stijn
d3c0261876
hack/validate/deprecate-integration-cli: add integration-cli/*_api_*.go
The intent of this check is to not add integration tests that use the
CLI to test API functionality, so check for changes in "API" tests as
well, now that the `integration/` suite has matured enough to develop
new tests, or migrate existing ones.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-12-19 18:07:12 +01:00
Sebastiaan van Stijn
1f0fdcc6f5
hack/validate: clean-up output, move golang-ci-lint to end
The golang-ci-lint takes longest to run, so do the other
checks first; this also makes the output a bit more readable.

While at it, fix some consistency issues in the output of the
other checks.

Before this change:

    TEST_FORCE_VALIDATE=1 hack/validate/default
    Congratulations!  All commits are properly signed with the DCO!
    Congratulations! Seccomp profile generation is done correctly.
    INFO [config_reader] Used config file hack/validate/golangci-lint.yml
    INFO [lintersdb] Active 15 linters: [deadcode gofmt goimports golint gosec gosimple govet ineffassign misspell staticcheck structcheck typecheck unconvert unused varcheck]
    ...
    INFO Execution took 2m12.4287302s
    Congratulations!  "./pkg/..." is safely isolated from internal code.
    The swagger spec at "api/swagger.yaml" is valid against swagger specification 2.0
    Congratulations! All api changes are done the right way.
    Congratulations!  All toml source files changed here have valid syntax.
    Congratulations!  Changelog CHANGELOG.md is well-formed.
    Congratulations!  Changelog CHANGELOG.md dates are in descending order.

After this change:

    TEST_FORCE_VALIDATE=1 hack/validate/default
    Congratulations!  All commits are properly signed with the DCO!
    Congratulations!  Seccomp profile generation is done correctly.
    Congratulations!  Packages in "./pkg/..." are safely isolated from internal code.
    Congratulations!  The swagger spec at "api/swagger.yaml" is valid against swagger specification 2.0
    Congratulations!  All API changes are done the right way.
    Congratulations!  All TOML source files changed here have valid syntax.
    Congratulations!  Changelog CHANGELOG.md is well-formed.
    Congratulations!  Changelog CHANGELOG.md dates are in descending order.
    Congratulations!  No new tests were added to integration-cli.

    INFO Start validation with golang-ci-lint
    INFO [config_reader] Used config file hack/validate/golangci-lint.yml
    INFO [lintersdb] Active 15 linters: [deadcode gofmt goimports golint gosec gosimple govet ineffassign misspell staticcheck structcheck typecheck unconvert unused varcheck]
    ...
    INFO Execution took 2m12.4287302s

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-12-19 18:07:07 +01:00
Sebastiaan van Stijn
545e81714b
hack/validate: allow forcing validation
This introduces a FORCE_VALIDATE environment variable, which allows
forcing some validation steps, even if no changes were detected.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-12-19 18:03:55 +01:00
Sebastiaan van Stijn
9dde8f2b46
hack/validate/deprecate-integration-cli: don't exit on success
The `hack/validate/deprecate-integration-cli` script exited on
success. As a result, validation steps to run afterwards would
not be executed.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-12-19 17:59:23 +01:00