Commit graph

2423 commits

Author SHA1 Message Date
Tibor Vass
eee3f67571
Jenkinsfile: reduce time of integration tests by dividing tests into 3 parallel runs
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit e554fb23c8)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-08-19 18:58:00 +02:00
Tibor Vass
78207d5380
hack: unmount leftover daemon root folders
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 13df617d4c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-08-19 18:57:58 +02:00
Sebastiaan van Stijn
3dd11dd0b5
docker-py: skip PullImageTest::test_pull_invalid_platform
and remove `PullImageTest::test_build_invalid_platform` from the list,
which was a copy/paste error in f8cde0b32d

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 6f5c377ddc)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-08-19 18:57:35 +02:00
Sebastiaan van Stijn
94d56428a6
Consistently use DOCKER_EXPERIMENTAL=1 instead or =y
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit a43123cab1)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-08-19 18:57:24 +02:00
Sebastiaan van Stijn
52ec660936
docker-py: deselect broken experimental tests
These tests are fixed upstream, but those fixes are not yet in a
released version.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit f8cde0b32d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-08-19 18:57:22 +02:00
Andrew Hsu
24144dbdc9
run unit tests and generate junit report
Signed-off-by: Andrew Hsu <andrewhsu@docker.com>
(cherry picked from commit 42f0a0db75)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-08-19 18:56:04 +02:00
Brian Goff
da9289fb54
Improvements to the test runners
1. Use `go list` to get list of integration dirs to build. This means we
   do not need to have a valid `.go` in every subdirectory and also
   filters out other dirs like "bundles" which may have been created.
2. Add option to specify custom flags for integration and
   integration-cli. This is needed so both suites can be run AND set
   custom flags... since the cli suite does not support standard go
   flags.
3. Add options to skip an entire integration suite.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit abece9b562)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-08-19 18:55:57 +02:00
Michael Zhao
0b274cf18f
Set TIMEOUT according to os/arch.
Signed-off-by: Michael Zhao <michael.zhao@arm.com>
(cherry picked from commit 790da6c223)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-08-19 18:55:52 +02:00
Andrew Hsu
55fc016efc
allow running of single integration test
Signed-off-by: Andrew Hsu <andrewhsu@docker.com>
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit c222c5ac6f)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-08-19 18:55:45 +02:00
Sebastiaan van Stijn
63e03b155f
Remove Codecov
Codecov has shown to be flaky, and calculate the wrong diff, in
addition, it doesn't show coverage for integration tests, which
makes the coverage report not useful.

Removing it for now, while we look at alternatives.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit bd5c5373f1)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-08-19 18:55:21 +02:00
Sebastiaan van Stijn
19fa3ab213
WIP Move docker-py tests first again
See if networking works if we run it first

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 6aafe0fd9e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-08-19 18:54:55 +02:00
Sebastiaan van Stijn
ab68b5dd9a
docker-py: skip flaky tests
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 980f2813b4)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-08-19 18:54:53 +02:00
Sebastiaan van Stijn
9ad75d26fc
docker-py: use host-network for nested build of docker-py
When building this image docker-in-docker, the DNS in the environment
may not be usable for the build-container, causing resolution to fail:

```
02:35:31 W: Failed to fetch http://deb.debian.org/debian/dists/jessie/Release.gpg  Temporary failure resolving 'deb.debian.org'
```

This patch detects if we're building from within a container, and if
so, skips creating a networking namespace for the build by using
`--network=host`.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 3c15cea650)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-08-19 18:54:51 +02:00
Sebastiaan van Stijn
c3a7556f73
docker-py: don't build --quiet is TESTDEBUG is set
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit ba8f4c7994)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-08-19 18:54:49 +02:00
Sebastiaan van Stijn
e092ff3f74
Bump docker-py to 4.0.2, and run tests from upstream repository
This removes all the installation steps for docker-py from the
Dockerfile, and instead builds the upstream Dockerfile, and runs
docker-py tests in a container.

To test;

```
make test-docker-py

...

Removing bundles/

---> Making bundle: dynbinary (in bundles/dynbinary)
Building: bundles/dynbinary-daemon/dockerd-dev
Created binary: bundles/dynbinary-daemon/dockerd-dev

---> Making bundle: test-docker-py (in bundles/test-docker-py)
---> Making bundle: .integration-daemon-start (in bundles/test-docker-py)
Using test binary docker
Starting dockerd
INFO: Waiting for daemon to start...
.
INFO: Building docker-sdk-python3:3.7.0...
sha256:686428ae28479e9b5c8fdad1cadc9b7a39b462e66bd13a7e35bd79c6a152a402
INFO: Starting docker-py tests...
============================= test session starts ==============================
platform linux -- Python 3.6.8, pytest-4.1.0, py-1.8.0, pluggy-0.9.0
rootdir: /src, inifile: pytest.ini
plugins: timeout-1.3.3, cov-2.6.1
collected 359 items

tests/integration/api_build_test.py .......s....
....
```

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 7bfe48cc00)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-08-19 18:54:45 +02:00
Akihiro Suda
2b33fe3512
hack: remove integration-cli-on-swarm
integration-on-swarm had unnecessary complexity and was too hard to
maintain. Also, it didn't support the new non-CLI integration test suite.

I'm now doing some experiments out of the repo using Kubernetes:
https://github.com/AkihiroSuda/kube-moby-integration

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit e7fbe8e457)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-08-19 18:54:38 +02:00
Andrew Hsu
305b2416ea
Merge pull request #316 from thaJeztah/19.03_backport_buildkit_userns_remap_take2
[19.03 backport] builder-next: userns remap support and honor daemon's DNS config
2019-08-08 13:35:34 -07:00
Sebastiaan van Stijn
1fbed3ffc9
bump vndr to f5ab8fc5f, and revendor
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 0a3c9b935c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-08-08 18:49:07 +02:00
Sebastiaan van Stijn
791aa3c338
make.ps1: Run-IntegrationTests(): set working directory for test suite
This function changed to the correct working directory before starting the tests
(which is the same as on Linux), however the `ProcessStartInfo` process does
not inherit this working directory, which caused Windows tests to be running
with a different working directory as Linux (causing files used in tests to not
be found).

From the documentation; https://docs.microsoft.com/en-us/dotnet/api/system.diagnostics.processstartinfo.workingdirectory?view=netframework-4.8

> When `UseShellExecute` is `true`, the fully qualified name of the directory that contains
> the process to be started. When the `UseShellExecute` property is `false`, the working
> directory for the process to be started. The default is an empty string (`""`).

This patch sets the `ProcessStartInfo.WorkingDirectory` to the correct working
directory before starting the process.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 6ae46aeabf)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-08-08 17:05:08 +02:00
Sebastiaan van Stijn
640193b2bb
Windows: fix Golang version checks for GO_VERSION build-arg
This check was used to make sure we don't bump Go versions independently
(Linux/Windows). The Dockerfile switched to using a build-arg to allow
overriding the Go version, which rendered this check non-functional.

It also fails if Linux versions use a specific variant of the image;

08:41:31 ERROR: Failed 'ERROR: Mismatched GO versions between Dockerfile and Dockerfile.windows. Update your PR to ensure that both files are updated and in sync. ${GO_VERSION}-stretch ${GO_VERSION}' at 07/20/2019 08:41:31
08:41:31 At C:\gopath\src\github.com\docker\docker\hack\ci\windows.ps1:448 char:9
08:41:31 +         Throw "ERROR: Mismatched GO versions between Dockerfile and D ...
08:41:31 +         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

This patch fixes the check by looking for the value of `GO_VERSION` instead
of looking at the `FROM` line (which is harder to parse).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 4fa57a8191)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-08-08 03:21:12 +02:00
Sebastiaan van Stijn
c364e5d1ba
Dockerfile: use GO_VERSION build-arg for overriding Go version
This allows overriding the version of Go without making modifications in the
source code, which can be useful to test against multiple versions.

For example:

    make GO_VERSION=1.13beta1 shell

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit c6281bc438)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-08-08 03:20:49 +02:00
Akihiro Suda
509a793378 rootless: allow exposing dockerd TCP socket easily
eg.

  $ DOCKERD_ROOTLESS_ROOTLESSKIT_FLAGS="-p 0.0.0.0:2376:2376/tcp" \
   dockerd-rootless.sh --experimental \
   -H tcp://0.0.0.0:2376 \
   --tlsverify --tlscacert=ca.pem --tlscert=cert.pem --tlskey=key.pem

This commit bumps up RootlessKit from v0.4.1 to v0.6.0:
27a0c7a248...2fcff6ceae

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit 34f4729bc0)
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2019-07-18 13:03:33 +09:00
Tibor Vass
e61e107040 validate: temporarily disable deprecate-integration-cli as part of a revert
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 3f1cdd5364)
Signed-off-by: Tibor Vass <tibor@docker.com>
2019-06-18 18:55:04 +00:00
Olli Janatuinen
de45ce73eb
Enable integrations API tests for Windows CI
Signed-off-by: Olli Janatuinen <olli.janatuinen@gmail.com>
(cherry picked from commit 2f22247cad)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-06-12 10:16:04 +02:00
Sebastiaan van Stijn
88374fa982
Re-align proxy commit with libnetwork vendor
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 35069de3fd)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-06-09 11:18:39 +02:00
Olli Janatuinen
d59fb97c5b
Windows CI - Corrected LOCALAPPDATA location
Signed-off-by: Olli Janatuinen <olli.janatuinen@gmail.com>
(cherry picked from commit 61815f6763)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-06-03 11:25:45 +02:00
Andrew Hsu
a7e03f69be
Merge pull request #216 from AkihiroSuda/rootless-fix-kill-1903
[19.03 backport] rootless: fix killing daemon
2019-05-24 09:46:01 -07:00
Sebastiaan van Stijn
939aa52465
bump libnetwork 5ac07abef4eee176423fdc1b870d435258e2d381
full diff: 9ff9b57c34...5ac07abef4

brings in:

- docker/libnetwork#2376 Forcing a nil IP specified in PortBindings to IPv4zero (0.0.0.0)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit a66ddd8ab8)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-05-21 14:49:34 +02:00
Akihiro Suda
8f61032ec4 bump up rootlesskit to v0.4.1
Now the child process is killed when the parent dies (rootless-containers/rootlesskit#66)

e92d5e7...27a0c7a

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit 00c92a6719)
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2019-05-16 22:05:48 +09:00
Andrew Hsu
3f4657f6db
Merge pull request #210 from thaJeztah/19.03_backport_bump_runc_1.0.0-rc.8
[19.03 backport] Bump runc 1.0.0-rc8, opencontainers/selinux v1.2.2
2019-05-13 19:06:00 -07:00
Sebastiaan van Stijn
04c51495da
bump runc binary v1.0.0-rc8
full diff: 029124da7a...425e105d5a

- opencontainers/runc#2043 Vendor in latest selinux code for keycreate errors

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 4bc310c11b)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-05-13 15:39:54 -07:00
Akihiro Suda
e8b9a752d3
rootless: optional support for lxc-user-nic SUID binary
lxc-user-nic can eliminate slirp overhead but needs /etc/lxc/lxc-usernet to be configured for the current user.

To use lxc-user-nic, $DOCKERD_ROOTLESS_ROOTLESSKIT_NET=lxc-user-nic also needs to be set.

This commit also bumps up RootlessKit from v0.3.0 to v0.4.0:
70e0502f32...e92d5e772e

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit 63a66b0eb0)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-05-13 15:31:11 -07:00
Sebastiaan van Stijn
2bfa24b393
Merge pull request #39129 from thaJeztah/bump_libnetwork
bump libnetwork 9ff9b57c344df5cd47443ad9e65702ec85c5aeb0
2019-04-23 23:27:07 +02:00
Sebastiaan van Stijn
7f2c564b23
bump libnetwork 9ff9b57c344df5cd47443ad9e65702ec85c5aeb0
full diff: 48f846327b...9ff9b57c34

- docker/libnetwork#2368 (windows) Pick a random host port if the user does not specify a host port

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-04-23 18:37:36 +02:00
corbin-coleman
edc639e99f Add ability to override the version in make.ps1
Checks for environment variable VERSION if it exists then it sets dockerVersion to VERSION

Signed-off-by: corbin-coleman <corbin.coleman@docker.com>
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
2019-04-19 18:39:57 +00:00
Brian Goff
fbb001d1f9 Add support for setting GOARM in cross target.
This adds to the existing format of `<GOOS>/<GOARCH>` to allow for
`<GOOS>/arm/v<GOARM>`

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2019-04-17 13:19:14 -07:00
Sebastiaan van Stijn
23416c1b36
re-align userland-proxy with vendored version
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-04-15 18:15:04 +02:00
Brian Goff
61a3285864
Support cross-compile for arm
Pretty much cross-compile doesn't work because  of this:

> profiles/seccomp/seccomp.go:13:2: build constraints exclude all Go files in /go/src/github.com/docker/docker/vendor/github.com/seccomp/libseccomp-golang

This changes adds a new Dockerfile target for cross compilation with the
neccesary arch specific libseccomp packages and CC toolchains.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-04-10 14:13:32 +02:00
Jintao Zhang
d43a41d7af Update runc 029124da7af7360afa781a0234d1b083550f797c
Signed-off-by: Jintao Zhang <zhangjintao9020@gmail.com>
2019-04-06 16:53:33 +00:00
Jintao Zhang
8092cfb6e7 Update containerd v1.2.6
Signed-off-by: Jintao Zhang <zhangjintao9020@gmail.com>
2019-04-06 16:51:56 +00:00
Michael Crosby
265b605118 Move docker-py tests to the end
These tests have bad output and its hard to figure out what went wrong
when one of them fails.  Move them to the end to atleast get the real
test output first and better debug things.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-04-05 10:15:08 -04:00
Tõnis Tiigi
bcaa613d82
Merge pull request #38983 from thaJeztah/bump_libnetwork
bump libnetwork to ebcade70ad1059b070d0040d798ecca359bc5fed
2019-04-03 10:26:47 -07:00
Akihiro Suda
c458822887 bump up rootlesskit
Changes: ed26714429...70e0502f32

Contains the fix for running RootlessKit+VPNKit instances simultaneously with multiple users: https://github.com/rootless-containers/rootlesskit/issues/56

Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2019-04-03 20:08:50 +09:00
Tibor Vass
a0d64a3093
Merge pull request #38913 from AkihiroSuda/rootlesskit-docker-proxy
rootless: expose ports automatically
2019-04-02 19:51:15 -07:00
Sebastiaan van Stijn
3ab093d567
bump libnetwork to ebcade70ad1059b070d0040d798ecca359bc5fed
full diff: 1a06131fb8...ebcade70ad

relevant changes:

- docker/libnetwork#2349 IPVS: Add support for GetConfig/SetConfig
- docker/libnetwork#2343 Revert "debian has iptables-legacy and iptables-nft now"
- docker/libnetwork#2230 Moving IPVLAN driver out of experimental
- docker/libnetwork#2307 Fix for problem where agent is stopped and does not restart
- docker/libnetwork#2303 Touch-up error-message and godoc for ConfigVXLANUDPPort
- docker/libnetwork#2325 Fix possible nil pointer exception
- docker/libnetwork#2302 Use sync.RWMutex for VXLANUDPPort
- docker/libnetwork#2306 Improve error if auto-selecting IP-range failed

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-04-01 19:40:09 +02:00
Vincent Demeester
da823cf3a5
Merge pull request #38853 from cyphar/integration-cli-ensureImage
integration-cli: don't build -test images if they already exist
2019-03-27 07:32:23 +01:00
Akihiro Suda
f0b405fbda rootless: expose ports automatically
Now `docker run -p` ports can be exposed to the host namespace automatically when `dockerd-rootless.sh` is launched with
`--userland-proxy --userland-proxy-path $(which rootlesskit-docker-proxy)`.
This is akin to how Docker for Mac/Win works with `--userland-proxy-path=/path/to/vpnkit-expose-port`.

The port number on the host namespace needs to be set to >= 1024.
SCTP ports are currently unsupported.

RootlessKit changes: 7bbbc48a6f...ed26714429

Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2019-03-21 02:44:08 +09:00
Yong Tang
87d593639c
Merge pull request #38860 from xichengliudui/fixtypos
Delete duplicate words
2019-03-15 10:01:54 -07:00
xichengliudui
1d62807db3 Delete duplicate words
update pull request

Signed-off-by: “xichengliudui” <“liuduidui@beyondcent.com”>
2019-03-14 07:53:38 -04:00
Sebastiaan van Stijn
25cdae293f
Update containerd v1.2.5, runc 2b18fe1d885ee5083ef9f0838fee39b62d653e30
Notable Updates

- Fix an issue that non-existent parent directory in image layers is created with permission 0700. containerd#3017
- Fix an issue that snapshots of the base image can be deleted by mistake, when images built on top of it are deleted. containerd#3087
- Support for GC references to content from snapshot and container objects. containerd#3080
- cgroups updated to dbea6f2bd41658b84b00417ceefa416b97 to fix issues for systemd 420 and non-existent cgroups. containerd#3079
- runc updated to 2b18fe1d885ee5083ef9f0838fee39b62d653e30 to include the improved fix for CVE-2019-5736. containerd#3082
- cri: Fix a bug that pod can't get started when the same volume is defined differently in the image and the pod spec. cri#1059
- cri: Fix a bug that causes container start failure after in-place upgrade containerd to 1.2.4+ or 1.1.6+. cri#1082
- cri updated to a92c40017473cbe0239ce180125f12669757e44f. containerd#3084

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-03-13 21:00:50 +01:00