Commit graph

37849 commits

Author SHA1 Message Date
Sebastiaan van Stijn
de10c7d013
client: reduce string-matching in tests
These checks were redundant, as we were not expecting
a specific string, just that a server-error or authentication
error was returned.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-13 17:00:40 +02:00
Akihiro Suda
d1e837d2a8
Merge pull request #39992 from thaJeztah/apparmor_fix_network_paths
AppArmor: add missing rules for running in userns
2019-10-13 03:14:45 +09:00
Brian Goff
28b645755a
Merge pull request #40062 from thaJeztah/testutil_improvements
Various testutil improvements
2019-10-11 15:37:48 -07:00
Sebastiaan van Stijn
f5eb39f75a
Merge pull request #39919 from jmartin84/36412-build-arg-override-binary-commits
Use build args to override binary commits in dockerfile
2019-10-11 15:43:27 +02:00
Akihiro Suda
97c084032a
Merge pull request #40053 from thaJeztah/harden_testdaemonevents
integration-cli: rely less on CLI output format
2019-10-11 18:20:45 +09:00
Sebastiaan van Stijn
293c1a27a2
testutil/daemon: remove redundant d.cmd.Wait()
`daemon.StartWithLogFile()` already creates a goroutine that
calls `d.cmd.Waits()` and sends its return to the channel, `d.Wait`.

This code called `d.cmd.Wait()` one more time, and returns the
error, which may produce an error _because_ it's called a second
time, and potentially cause an incorrect test-result.

(thanks to Kir Kolyshkin for spotting this)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-11 00:38:08 +02:00
Sebastiaan van Stijn
c56bfdf10a
testutil/daemon: always remove pidfile after daemon is stopped
If the daemon was stopped successfully in one of the retry-loops,
the function would return early;

```go
for {
	select {
	case err := <-d.Wait:
---> the function returns here, both on "success" and on "fail"
		return err
	case <-time.After(20 * time.Second):
...
```

In that case, the pidfile would not be cleaned up. This patch changes
the function to clean-up the pidfile in a defer, so that it will
always be removed after succesfully stopping the daemon.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-11 00:38:06 +02:00
Sebastiaan van Stijn
f6842327b0
testutil/daemon: print all arguments when failing to start daemon
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-11 00:38:04 +02:00
Sebastiaan van Stijn
b843b1ffe3
testutil/daemon: store pidfile-path, and ignore errors when removing
This patch stores the location of the pidfile, so that we can use the
same path that was set to create it. If no pidfile was created, we'll
not try to remove it.

We're now also ignoring errors when removing the pidfile, as they should
not fail the test (especialy if no pidfile was created in the first place,
as that could potentially hide the actual failure).

This may help with "failures" such as the one below:

```
FAIL: check_test.go:347: DockerSwarmSuite.TearDownTest

check_test.go:352:
    d.Stop(c)
/go/src/github.com/docker/docker/internal/test/daemon/daemon.go:414:
    t.Fatalf("Error while stopping the daemon %s : %v", d.id, err)
... Error: Error while stopping the daemon d1512c423813a : remove /go/src/github.com/docker/docker/bundles/test-integration/DockerSwarmSuite.TestServiceLogs/d1512c423813a/docker.pid: no such file or directory
```

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-11 00:38:02 +02:00
Sebastiaan van Stijn
22662cac57
testutil/daemon: wrap errors
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-11 00:38:00 +02:00
Sebastiaan van Stijn
2b3957d0b1
testutil/daemon: prefix all logs with daemon-id
This makes it easier to debug issues with tests that
start multiple daemons.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-11 00:37:49 +02:00
Sebastiaan van Stijn
1fe7a9552c
testutil/daemon: daemon.Cleanup(): unmount daemon root-dir as part of cleanup
test-daemons remove their docker.pid when stopped, so the `.integration-daemon-stop`
script did not find the mounts for those daemons, and therefore was not unmounting
them.

As a result, cleaning up the bundles directory on consecutive runs of the tests would fail;

    rm: cannot remove 'bundles/test-integration/TestDockerSwarmSuite/TestSwarmInit/d1f188f3f5472/root': Device or resource busy

This patch unmounts the root directory of the daemon as part of the cleanup step.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-11 00:34:55 +02:00
Sebastiaan van Stijn
9407a57522
hack/make: don't attempt to unmount non-existing daemon root-dir
Before:

    DONE 2 tests in 12.272s
    ---> Making bundle: .integration-daemon-stop (in bundles/test-integration)
    umount: bundles/test-integration/root: mountpoint not found

After:

    DONE 2 tests in 14.650s
    ---> Making bundle: .integration-daemon-stop (in bundles/test-integration)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-11 00:34:53 +02:00
Tibor Vass
d56adcf0ec
Merge pull request #40059 from cpuguy83/move_source_copy
Fix trick for `make BINDDIR=. shell` and `COPY .`
2019-10-10 13:38:23 -07:00
Justen Martin
095ca77f48
Use build args to override binary commits in dockerfile
Signed-off-by: Justen Martin <jmart@the-coder.com>
2019-10-10 14:52:57 -05:00
Brian Goff
1c82d11908 Fix trick for make BINDDIR=. shell and COPY .
This was inadvertently removed when updating the Dockerfile for buildkit
specific features.

Trick selects a different build target depending on if the source is
going to be bind-mounted in anyway, which prevents the need to copy the
whole source tree to the builder.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2019-10-10 11:47:23 -07:00
Tibor Vass
6f069f14e6
Merge pull request #40058 from cpuguy83/buildkit_dockerfile
Fix `make cross` target
2019-10-10 11:29:19 -07:00
Tibor Vass
d31633be7d
Merge pull request #39678 from thaJeztah/carry_39595_tailor_arm_ci
[Carry 39595] Tailor CI for ARM, skip legacy integration test.
2019-10-10 11:22:02 -07:00
Tibor Vass
b4e912b70e
Merge pull request #40038 from kolyshkin/go-swagger
go-swagger: fix panic
2019-10-10 11:13:49 -07:00
Sebastiaan van Stijn
8a3e8ac017
TestDaemonEvents: use is.Contains() for easier debugging
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-10 00:13:24 +02:00
Tibor Vass
0e0ab58a96
integration-cli: in TestDaemonEvents* use ioutil.WriteFile
Signed-off-by: Tibor Vass <tibor@docker.com>
2019-10-10 00:13:22 +02:00
Sebastiaan van Stijn
24a419221d
integration-cli: TestDaemonEvents*: don't rely on CLI output format
Running these tests with a different version of the CLI caused
some failures because the tests relied on the CLI's output format.

Although these tests should be rewritten to use the API directly,
in the meantime this makes them slightly more reliable.

Signed-off-by: Tibor Vass <tibor@docker.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-10 00:13:03 +02:00
Brian Goff
588d0e6133
Merge pull request #40064 from thaJeztah/requirement_helper
integration-cli: make testRequires() a Helper
2019-10-09 14:11:47 -07:00
Sebastiaan van Stijn
6afe0f38f6
integration-cli: make testRequires() a Helper
Make this utility a helper, so that the "skip" message is printing
the location of the test, instead of the location of the helper,
which is what it's printing now:

    requirement.go:26: unmatched requirement bridgeNfIptables

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-09 21:44:22 +02:00
Tibor Vass
448db5a783
Merge pull request #40060 from thaJeztah/require_buildkit
Makefile: force using buildkit if USE_BUILDX is not set
2019-10-08 16:51:15 -07:00
Sebastiaan van Stijn
1be2cc2568
Makefile: force using buildkit if USE_BUILDX is not set
Before this change:

```
unset DOCKER_BUILDKIT

make build
docker build  --build-arg=CROSS=false  -t "docker-dev:require-buildkit" -f "Dockerfile" .
Sending build context to Docker daemon  50.01MB
Error response from daemon: Dockerfile parse error line 17: Unknown flag: mount
make: *** [build] Error 1
```

After this change:

```
unset DOCKER_BUILDKIT

make build
docker build  --build-arg=CROSS=false  -t "docker-dev:require-buildkit" -f "Dockerfile" .
[+] Building 5.2s (71/71) FINISHED
 => [internal] load .dockerignore                                                                     0.1s
...
...
 => => exporting layers                                                                               0.9s
 => => writing image sha256:1ea4128a0e7f3bdee47de1675252609d9d6071e32da24a2aafee9fba96b2404b          0.0s
 => => naming to docker.io/library/docker-dev:require-buildkit                                        0.0s
...

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-08 21:47:07 +02:00
Kir Kolyshkin
93f9b902af go-swagger: fix panic
This is an attempt to fix go-swagger panic under Golang 1.13.

Details:
 * https://github.com/go-openapi/jsonpointer/pull/4
 * https://github.com/go-swagger/go-swagger/pull/2059

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2019-10-08 11:29:59 -07:00
Brian Goff
e5bfaf19b1 Fix make cross target
When changing the various cross targets in the Dockerfile I neglected
some `;`.
Instead of dealing with that now this just sets `--platform` on the
cross specific targets which only work on linux/amd64 anyway.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2019-10-08 10:54:39 -07:00
Tibor Vass
b3be2802d4
Merge pull request #39713 from thaJeztah/containerd_1.3
bump containerd and dependencies to v1.3.0
2019-10-08 10:43:56 -07:00
Sebastiaan van Stijn
eda98ad00f
Jenkinsfile: aarch64: use new labels to select agents
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-08 18:47:59 +02:00
Sebastiaan van Stijn
a0d670e516
Jenkinsfile: aarch64: sync with latest changes
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-08 18:47:57 +02:00
Sebastiaan van Stijn
14ea1f62eb
Jenkinsfile: aarch64: don't restrict to packet workers only
Pick whatever is available; packet worker, or auto-scaling
a1.xlarge arm64 machines on AWS

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-08 18:47:55 +02:00
Sebastiaan van Stijn
9d5361de3f
Jenkinsfile: rename aarch64 to arm64
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-08 18:47:52 +02:00
Sebastiaan van Stijn
58d57c76b5
Jenkinsfile: aarch64: split into stages, add "print info" unit-tests
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-08 18:47:40 +02:00
Sebastiaan van Stijn
86e0c5a0d4
Jenkinsfile: aarch64: sync stage with other stages
Also switch aarch64 to use overlay2

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-08 18:37:17 +02:00
Sebastiaan van Stijn
402c7b1b27
Jenkinsfile: aarch64: move stage inside parallel group
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-08 18:37:15 +02:00
Michael Zhao
48b06a2561
Tailor CI for ARM, skip legacy integration test.
Signed-off-by: Michael Zhao <michael.zhao@arm.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-08 18:37:13 +02:00
Michael Zhao
af86580000
Test to enable CI on aarch64.
Signed-off-by: Michael Zhao <michael.zhao@arm.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-08 18:37:10 +02:00
Tibor Vass
dba8da8158
Merge pull request #40057 from andrewhsu/truetop
integration-cli: TestDockerNetworkConnectLinkLocalIP return on failure
2019-10-07 17:51:28 -07:00
Tibor Vass
72befc2218
Merge pull request #40043 from andrewhsu/true
integration-cli: TestUserDefinedNetworkConnectDisconnectAlias return on failure
2019-10-07 16:42:05 -07:00
Brian Goff
4faf65f250
Merge pull request #40056 from kolyshkin/dm-unit
devmapper: fix unit test
2019-10-07 15:43:08 -07:00
Andrew Hsu
318e279fd8 integration-cli: TestDockerNetworkConnectLinkLocalIP return on failure
Signed-off-by: Andrew Hsu <andrewhsu@docker.com>
2019-10-07 22:27:24 +00:00
Andrew Hsu
7450f89f6c integration-cli: TestUserDefinedNetworkConnectDisconnectAlias return on failure
Have the test return immediately if the test does not pass instead of
stuck in `top`.

Signed-off-by: Andrew Hsu <andrewhsu@docker.com>
2019-10-07 22:26:45 +00:00
Brian Goff
9425487117
Merge pull request #40054 from thaJeztah/swarmtests_more_debugging
integration-cli: DockerSwarmSuite: show output on failures
2019-10-07 13:50:05 -07:00
Kir Kolyshkin
8663d09334 devmapper: fix unit test
It has been pointed out that sometimes device mapper unit tests
fail with the following diagnostics:

> --- FAIL: TestDevmapperSetup (0.02s)
>    graphtest_unix.go:44: graphdriver: loopback attach failed
>    graphtest_unix.go:48: loopback attach failed

The root cause is the absence of udev inside the container used
for testing, which causes device nodes (/dev/loop*) to not be
created.

The test suite itself already has a workaround, but it only
creates 8 devices (loop0 till loop7). It might very well be
the case that the first few devices are already used by the
system (on my laptop 15 devices are busy).

The fix is to raise the number of devices being manually created.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2019-10-07 13:21:14 -07:00
Sebastiaan van Stijn
d5f07220fc
integration-cli: DockerSwarmSuite: show output on failures
Unfortunately quite some of these tests do output-matching, which
may be CLI dependent; this patch prints the output string, to help
debugging failures that may be related to the output having changed
between CLI versions.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-07 21:28:49 +02:00
Sebastiaan van Stijn
31da04af21
Merge pull request #40045 from thaJeztah/bump_deps
Bump logrus 1.4.2, go-shellwords, mergo, flock, creack/pty, golang/gddo, gorilla/mux
2019-10-07 18:38:07 +02:00
Brian Goff
0abbb9e4eb
Merge pull request #40049 from CoRfr/buidx-typo
Jenkinsfile: fix typo buidx -> buildx
2019-10-06 10:39:54 -07:00
Bertrand Roussel
bef0aca449 Jenkinsfile: fix typo buidx -> buildx
Addressing a small typo noticed in commit c04ea1133d

Signed-off-by: Bertrand Roussel <broussel@sierrawireless.com>
2019-10-06 10:26:55 -07:00
Sebastiaan van Stijn
bb5650619e
bump gorilla/mux v1.7.3
full diff: ed099d4238...00bdffe0f3

changes included:

- gorilla/mux#477 Improve CORS Method Middleware
    - implements gorilla/mux#477 Make CORSMethodMiddleware actually make sense
- gorilla/mux#489 Fix nil panic in authentication middleware example

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-06 11:38:53 +02:00