Commit graph

37207 commits

Author SHA1 Message Date
Vincent Demeester
4e83c90ae8
Merge pull request #39638 from tiborvass/unit-junit
added entry for running unit tests with junit report
2019-08-01 23:03:16 -04:00
Brian Goff
4fb5e9e696
Merge pull request #39628 from cpuguy83/test_setup_improvements
Improvements to the test runners
2019-08-01 15:45:16 -07:00
Brian Goff
41040b7998
Merge pull request #39644 from kolyshkin/quota-map
projectquota: protect concurrent map access
2019-08-01 13:02:09 -07:00
Kir Kolyshkin
1ac0a66a64 projectquota: protect concurrent map access
Protect access to q.quotas map, and lock around changing nextProjectID.

Techinically, the lock in findNextProjectID() is not needed as it is
only called during initialization, but one can never be too careful.

Fixes: 52897d1c09 ("projectquota: utility class for project quota controls")
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2019-08-01 09:58:34 -07:00
Andrew Hsu
42f0a0db75 run unit tests and generate junit report
Signed-off-by: Andrew Hsu <andrewhsu@docker.com>
2019-08-01 06:08:35 +00:00
Brian Goff
abece9b562 Improvements to the test runners
1. Use `go list` to get list of integration dirs to build. This means we
   do not need to have a valid `.go` in every subdirectory and also
   filters out other dirs like "bundles" which may have been created.
2. Add option to specify custom flags for integration and
   integration-cli. This is needed so both suites can be run AND set
   custom flags... since the cli suite does not support standard go
   flags.
3. Add options to skip an entire integration suite.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2019-07-31 16:37:17 -07:00
Sebastiaan van Stijn
928381b221
Merge pull request #39633 from thaJeztah/bump_libnetwork
bump libnetwork to 09cdcc8c0eab3946c2d70e8f6225b05baf1e90d1
2019-07-31 08:03:26 -07:00
Kirill Kolyshkin
589f1dad8d
Merge pull request #39636 from thaJeztah/add_back_yamllint
Dockerfile: add back yamllint
2019-07-30 17:17:54 -07:00
Sebastiaan van Stijn
b1723b3721
Dockerfile: add back yamllint
This was inadvertedly removed in 7bfe48cc00,
because it was documented as a dependency for docker-py, but
actually used to validate the swagger file.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-07-30 17:04:26 -07:00
Sebastiaan van Stijn
6f234db9fe
bump libnetwork to 09cdcc8c0eab3946c2d70e8f6225b05baf1e90d1
full diff: 83d30db536...09cdcc8c0e

changes included:

- docker/libnetwork#2416 Fix hardcoded AF_INET for IPv6 address handling
- docker/libnetwork#2411 Macvlan network handles netlabel.Internal wrong
  - fixes docker/libnetwork#2410 Macvlan network handles netlabel.Internal wrong
- docker/libnetwork#2414 Allow network with --config-from to be --internal
  - fixes docker/libnetwork#2413 Network with --config-from does not honor --internal
- docker/libnetwork#2351 Use fewer modprobes
  - relates to moby/moby#38930 Use fewer modprobes
- docker/libnetwork#2415 Support dockerd and system restarts for ipvlan and macvlan networks
  - carry of docker/libnetwork#2295 phantom ip/mac vlan network after a powercycle
  - fixes docker/libnetwork#1743 Phantom docker network

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-07-30 10:37:48 -07:00
Brian Goff
7cfd8146dc
Merge pull request #39629 from thaJeztah/switch_creack_pty
switch kr/pty to creack/pty v1.1.7
2019-07-30 09:41:51 -07:00
Sebastiaan van Stijn
0595c01718
switch kr/pty to creack/pty v1.1.7
kr/pty was moved to creak/pty and the old location was
archived.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-07-29 16:59:08 -07:00
Brian Goff
582591d56b
Merge pull request #39569 from thaJeztah/remove_redundant_checks
integration-cli: remove redundant "testrequires"
2019-07-29 10:36:09 -07:00
Yong Tang
9c92080b13
Merge pull request #39589 from thaJeztah/prevent_network_attach_panic
Prevent panic on network attach
2019-07-29 07:10:59 -07:00
Tibor Vass
917a8b4259
Merge pull request #39616 from dperny/fix-cluster-rotate-unlock-key
Fix TestSwarmClusterRotateUnlockKey
2019-07-26 16:09:22 -07:00
Sebastiaan van Stijn
bd983a60bb
Merge pull request #39592 from zelahi/integrate-with-jsp-jenkinsfile
Added changes to integrate with the new Jenkins ci
2019-07-26 14:16:21 -07:00
Drew Erny
b79adac339 Fix TestSwarmClusterRotateUnlockKey
TestSwarmClusterRotateUnlockKey had been identified as a flaky test. It
turns out that the test code was wrong: where we should have been
checking the string output of a command, we were instead checking the
value of the error. This means that the error case we were expecting was
not being matched, and the test was failing when it should have just
retried.

Signed-off-by: Drew Erny <drew.erny@docker.com>
2019-07-26 15:51:56 -05:00
zelahi
0ecd6ab30f
ADDED changes to integrate with our new Jenkins ci
Signed-off-by: zelahi <elahi.zuhayr@gmail.com>
2019-07-26 11:53:36 -07:00
Sebastiaan van Stijn
9f122f0d2e
Merge pull request #39591 from crosbymichael/perm-test
Add extra permission check in removal test
2019-07-26 09:41:51 -07:00
Akihiro Suda
11e48badcb
Merge pull request #39612 from tiborvass/cve-2019-14271
Fix CVE-2019-14271 loading of nsswitch based config inside chroot under Glibc
2019-07-26 13:31:28 +09:00
Justin Cormack
a316b10dab Initialize nss libraries in Glibc so that the dynamic libraries are loaded in the host
environment not in the chroot from untrusted files.

See also OpenVZ a3f732ef75/src/enter.c (L227-L234)

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
(cherry picked from commit cea6dca993c2b4cfa99b1e7a19ca134c8ebc236b)
Signed-off-by: Tibor Vass <tibor@docker.com>
2019-07-26 01:27:57 +00:00
Brian Goff
17fe47d6c9
Merge pull request #39594 from andrewhsu/o2
use overlay2 for janky and experimental checks
2019-07-25 14:53:52 -07:00
Andrew Hsu
ccfaf1ed92 use overlay2 for janky and experimental checks
instead of vfs

Signed-off-by: Andrew Hsu <andrewhsu@docker.com>
2019-07-25 21:51:53 +00:00
Brian Goff
8297dc26e6
Merge pull request #39606 from andrewhsu/execdriver
remove DOCKER_EXECDRIVER from Jenkinsfile
2019-07-25 12:59:28 -07:00
Brian Goff
a1885bd655
Merge pull request #39603 from cpuguy83/test_daemon_logging
Better logging for swarm tests
2019-07-25 12:58:53 -07:00
Vincent Demeester
ecaf0b479f
Merge pull request #39602 from cpuguy83/env_from_client
Add `FromClient` to test env execution
2019-07-25 10:34:47 +02:00
Andrew Hsu
9d98458fb7 remove DOCKER_EXECDRIVER from Jenkinsfile
Signed-off-by: Andrew Hsu <andrewhsu@docker.com>
2019-07-25 01:25:16 +00:00
Brian Goff
b0fe0dff7a Better logging for swarm tests
Call helper for starting swarm agents and add some logging with daemon
id's when joining the swarm.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2019-07-24 16:38:11 -07:00
Brian Goff
1381956499 Add FromClient to test env execution
While working on other tests I noticed that environment.Execution cannot
be used for anything but the pre-configured daemon, however this can
come in handy for being able share daemons across multiple tests that
currently spin up a new daemon.
The execution env also seems to be misused in some of these cases.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2019-07-24 12:27:41 -07:00
Sebastiaan van Stijn
a9dc697fd2
Merge pull request #39535 from michael2012z/fix_timeout_setting
Set TIMEOUT according to architecture.
2019-07-23 08:46:12 +02:00
Michael Zhao
790da6c223 Set TIMEOUT according to os/arch.
Signed-off-by: Michael Zhao <michael.zhao@arm.com>
2019-07-23 10:44:50 +08:00
Sebastiaan van Stijn
2cdaceb2a0
Merge pull request #39580 from andrewhsu/ci-master
added hack/ci/master as entry point for master codeline checks
2019-07-23 01:06:28 +02:00
Michael Crosby
d6cbeee470 Add extra permission check in removal test
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-07-22 20:39:22 +00:00
Andrew Hsu
aac6e62209
added hack/ci/master as entry point for master codeline checks
Signed-off-by: Andrew Hsu <andrewhsu@docker.com>
2019-07-22 20:00:07 +02:00
Brian Goff
b00666366b
Merge pull request #39585 from thaJeztah/trim_the_zero
Dockerfile.windows: trim .0 from Go versions
2019-07-22 10:47:21 -07:00
Sebastiaan van Stijn
651e694508
Prevent panic on network attach
In situations where `container.NetworkSettings` was not nil, but
`container.NetworkSettings.Networks` was, a panic could occur:

```
2019-06-10 15:26:50.548309 I | http: panic serving @: assignment to entry in nil map
goroutine 1376 [running]:
net/http.(*conn).serve.func1(0xc4211068c0)
	/usr/local/go/src/net/http/server.go:1726 +0xd2
panic(0x558939d7e1e0, 0x55893a0c4410)
	/usr/local/go/src/runtime/panic.go:502 +0x22d
github.com/docker/docker/daemon.(*Daemon).updateNetworkSettings(0xc42090c5a0, 0xc420fb6fc0, 0x55893a101140, 0xc4210e0540, 0xc42112aa80, 0xc4217d77a0, 0x0)
	/go/src/github.com/docker/docker/daemon/container_operations.go:275 +0x40e
github.com/docker/docker/daemon.(*Daemon).updateNetworkConfig(0xc42090c5a0, 0xc420fb6fc0, 0x55893a101140, 0xc4210e0540, 0xc42112aa80, 0x55893a101101, 0xc4210e0540, 0x0)
	/go/src/github.com/docker/docker/daemon/container_operations.go:683 +0x219
github.com/docker/docker/daemon.(*Daemon).connectToNetwork(0xc42090c5a0, 0xc420fb6fc0, 0xc420e8290f, 0x40, 0xc42112aa80, 0x558937eabd01, 0x0, 0x0)
	/go/src/github.com/docker/docker/daemon/container_operations.go:728 +0x1cb
github.com/docker/docker/daemon.(*Daemon).ConnectToNetwork(0xc42090c5a0, 0xc420fb6fc0, 0xc420e8290f, 0x40, 0xc42112aa80, 0x0, 0x0)
	/go/src/github.com/docker/docker/daemon/container_operations.go:1046 +0x2b3
github.com/docker/docker/daemon.(*Daemon).ConnectContainerToNetwork(0xc42090c5a0, 0xc4214ca580, 0x40, 0xc420e8290f, 0x40, 0xc42112aa80, 0x2, 0xe600000000000001)
	/go/src/github.com/docker/docker/daemon/network.go:450 +0xa1
github.com/docker/docker/api/server/router/network.(*networkRouter).postNetworkConnect(0xc42121bbc0, 0x55893a0edee0, 0xc420de7cb0, 0x55893a0ec2e0, 0xc4207f0e00, 0xc420173600, 0xc420de7980, 0x5589394707cc, 0x5)
	/go/src/github.com/docker/docker/api/server/router/network/network_routes.go:278 +0x330
github.com/docker/docker/api/server/router/network.(*networkRouter).(github.com/docker/docker/api/server/router/network.postNetworkConnect)-fm(0x55893a0edee0, 0xc420de7cb0, 0x55893a0ec2e0, 0xc4207f0e00, 0xc420173600, 0xc420de7980, 0x558937fd89dc, 0x558939f2cec0)
	/go/src/github.com/docker/docker/api/server/router/network/network.go:37 +0x6b
github.com/docker/docker/api/server/middleware.ExperimentalMiddleware.WrapHandler.func1(0x55893a0edee0, 0xc420de7cb0, 0x55893a0ec2e0, 0xc4207f0e00, 0xc420173600, 0xc420de7980, 0x55893a0edee0, 0xc420de7cb0)
	/go/src/github.com/docker/docker/api/server/middleware/experimental.go:26 +0xda
github.com/docker/docker/api/server/middleware.VersionMiddleware.WrapHandler.func1(0x55893a0edee0, 0xc420de7a70, 0x55893a0ec2e0, 0xc4207f0e00, 0xc420173600, 0xc420de7980, 0x0, 0x0)
	/go/src/github.com/docker/docker/api/server/middleware/version.go:62 +0x401
github.com/docker/docker/pkg/authorization.(*Middleware).WrapHandler.func1(0x55893a0edee0, 0xc420de7a70, 0x55893a0ec2e0, 0xc4207f0e00, 0xc420173600, 0xc420de7980, 0x0, 0x558939640868)
	/go/src/github.com/docker/docker/pkg/authorization/middleware.go:59 +0x7ab
github.com/docker/docker/api/server/middleware.DebugRequestMiddleware.func1(0x55893a0edee0, 0xc420de7a70, 0x55893a0ec2e0, 0xc4207f0e00, 0xc420173600, 0xc420de7980, 0x55893a0edee0, 0xc420de7a70)
	/go/src/github.com/docker/docker/api/server/middleware/debug.go:53 +0x4b8
github.com/docker/docker/api/server.(*Server).makeHTTPHandler.func1(0x55893a0ec2e0, 0xc4207f0e00, 0xc420173600)
	/go/src/github.com/docker/docker/api/server/server.go:141 +0x19a
net/http.HandlerFunc.ServeHTTP(0xc420e0c0e0, 0x55893a0ec2e0, 0xc4207f0e00, 0xc420173600)
	/usr/local/go/src/net/http/server.go:1947 +0x46
github.com/docker/docker/vendor/github.com/gorilla/mux.(*Router).ServeHTTP(0xc420ce5950, 0x55893a0ec2e0, 0xc4207f0e00, 0xc420173600)
	/go/src/github.com/docker/docker/vendor/github.com/gorilla/mux/mux.go:103 +0x228
github.com/docker/docker/api/server.(*routerSwapper).ServeHTTP(0xc421078330, 0x55893a0ec2e0, 0xc4207f0e00, 0xc420173600)
	/go/src/github.com/docker/docker/api/server/router_swapper.go:29 +0x72
net/http.serverHandler.ServeHTTP(0xc420902f70, 0x55893a0ec2e0, 0xc4207f0e00, 0xc420173600)
	/usr/local/go/src/net/http/server.go:2697 +0xbe
net/http.(*conn).serve(0xc4211068c0, 0x55893a0ede20, 0xc420d81440)
	/usr/local/go/src/net/http/server.go:1830 +0x653
created by net/http.(*Server).Serve
	/usr/local/go/src/net/http/server.go:2798 +0x27d
```

I have not been able to reproduce the situation, but preventing a panic should
not hurt.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-07-22 19:44:14 +02:00
Brian Goff
e6091b28c5
Merge pull request #39422 from lemrouch/35876-workaround
Don't try to load plugin without name
2019-07-22 10:21:50 -07:00
Michael Crosby
2101bfa7e7
Merge pull request #39582 from thaJeztah/its_a_stretch_but_it_was_busted
Pin Dockerfile to -stretch variant because it's bust'd
2019-07-22 12:55:48 -04:00
Sebastiaan van Stijn
c5bd6e3dc7
Dockerfile.windows: trim .0 from Go versions
This was an oversight when changing the Dockerfile to use a build-arg;
the Windows Dockerfile downloads the Go binaries, which never have a
trailing `.0`.

This patch makes sure that the trailing zero (if any) is removed.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-07-20 13:00:56 +02:00
Sebastiaan van Stijn
ed672bb523
Pin Dockerfile to -stretch variant
The Golang base images switch to buster, which causes some breakage
in networking and packages that are no  longer available; (`btrfs-tools`
is now an empty package, and `libprotobuf-c0-dev` is gone).

Some of out tests also start faiilng on stretch, and will have to be
investigated further;

```
15:13:06 --- FAIL: TestRenameAnonymousContainer (3.37s)
15:13:06     rename_test.go:168: assertion failed: 0 (int) != 1 (inspect.State.ExitCode int): container a7fe866d588d65f353f42ffc5ea5288e52700384e1d90850e9c3d4dce8657666 exited with the wrong exitcode:

15:13:38 --- FAIL: TestHostnameDnsResolution (2.23s)
15:13:38     run_linux_test.go:128: assertion failed:
15:13:38         --- ←
15:13:38         +++ →
15:13:38         @@ -1 +1,2 @@
15:13:38         +ping: bad address 'foobar'
15:13:38
15:13:38
15:13:38     run_linux_test.go:129: assertion failed: 0 (int) != 1 (res.ExitCode int)
```

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-07-20 12:58:20 +02:00
Sebastiaan van Stijn
4fa57a8191
Windows: fix Golang version checks for GO_VERSION build-arg
This check was used to make sure we don't bump Go versions independently
(Linux/Windows). The Dockerfile switched to using a build-arg to allow
overriding the Go version, which rendered this check non-functional.

It also fails if Linux versions use a specific variant of the image;

08:41:31 ERROR: Failed 'ERROR: Mismatched GO versions between Dockerfile and Dockerfile.windows. Update your PR to ensure that both files are updated and in sync. ${GO_VERSION}-stretch ${GO_VERSION}' at 07/20/2019 08:41:31
08:41:31 At C:\gopath\src\github.com\docker\docker\hack\ci\windows.ps1:448 char:9
08:41:31 +         Throw "ERROR: Mismatched GO versions between Dockerfile and D ...
08:41:31 +         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

This patch fixes the check by looking for the value of `GO_VERSION` instead
of looking at the `FROM` line (which is harder to parse).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-07-20 12:54:50 +02:00
Sebastiaan van Stijn
48da116486
Merge pull request #39076 from thaJeztah/remove_hack_malformed_host_header
Remove hack MalformedHostHeaderOverride
2019-07-20 02:15:10 +02:00
Michael Crosby
943129ed86
Merge pull request #39575 from thaJeztah/fix_TestHealthKillContainer
Skip TestHealthKillContainer on Windows
2019-07-19 14:52:24 -04:00
Sebastiaan van Stijn
eeaa0b30d4
Skip TestHealthKillContainer on Windows
This test is failing on Windows currently:

```
11:59:47 --- FAIL: TestHealthKillContainer (8.12s)
11:59:47     health_test.go:57: assertion failed: error is not nil: Error response from daemon: Invalid signal: SIGUSR1
``

That test was added recently in https://github.com/moby/moby/pull/39454, but
rewritten in a commit in the same PR:
f8aef6a92f

In that rewrite, there were some changes:

- originally it was skipped on Windows, but the rewritten test doesn't have that skip:

    ```go
    testRequires(c, DaemonIsLinux) // busybox doesn't work on Windows
    ```

- the original test used `SIGINT`, but the new one uses `SIGUSR1`

Analysis:

- The Error bubbles up from: 8e610b2b55/pkg/signal/signal.go (L29-L44)
- Interestingly; `ContainerKill` should validate if a signal is valid for the given platform, but somehow we don't hit that part; f1b5612f20/daemon/kill.go (L40-L48)
- Windows only looks to support 2 signals currently 8e610b2b55/pkg/signal/signal_windows.go (L17-L26)
- Upstream Golang looks to define `SIGINT` as well; 77f9b2728e/src/runtime/defs_windows.go (L44)
- This looks like the current list of Signals upstream in Go; 3b58ed4ad3/windows/types_windows.go (L52-L67)

```go
const (
	// More invented values for signals
	SIGHUP  = Signal(0x1)
	SIGINT  = Signal(0x2)
	SIGQUIT = Signal(0x3)
	SIGILL  = Signal(0x4)
	SIGTRAP = Signal(0x5)
	SIGABRT = Signal(0x6)
	SIGBUS  = Signal(0x7)
	SIGFPE  = Signal(0x8)
	SIGKILL = Signal(0x9)
	SIGSEGV = Signal(0xb)
	SIGPIPE = Signal(0xd)
	SIGALRM = Signal(0xe)
	SIGTERM = Signal(0xf)
)
```

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-07-19 16:09:50 +02:00
Sebastiaan van Stijn
7f37d99ef5
integration-cli: remove redundant "testrequires"
The `DockerDaemonSuite.SetUpTest` already checks for Linux and a local daemon;

```
func (s *DockerDaemonSuite) SetUpTest(c *check.C) {
	testRequires(c, DaemonIsLinux, testEnv.IsLocalDaemon)
	s.d = daemon.New(c, dockerBinary, dockerdBinary, testdaemon.WithEnvironment(testEnv.Execution))
}
```

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-07-19 10:53:42 +02:00
Akihiro Suda
88f6b5e746
Merge pull request #39554 from thaJeztah/session_not_experimental
integration: run build session tests on non-experimental
2019-07-19 16:33:09 +09:00
Sebastiaan van Stijn
0949fe8283
Merge pull request #39548 from thaJeztah/golang_arg
Dockerfile: use GO_VERSION build-arg for overriding Go version
2019-07-19 09:25:10 +02:00
Kir Kolyshkin
a557538770 TESTING.md: document GO_VERSION
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2019-07-18 17:36:09 -07:00
Sebastiaan van Stijn
c6281bc438 Dockerfile: use GO_VERSION build-arg for overriding Go version
This allows overriding the version of Go without making modifications in the
source code, which can be useful to test against multiple versions.

For example:

    make GO_VERSION=1.13beta1 shell

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-07-18 17:36:09 -07:00
Sebastiaan van Stijn
b4058a6c91
Merge pull request #39258 from thaJeztah/revert_remove_TestSearchCmdOptions
Revert "Remove TestSearchCmdOptions test"
2019-07-19 01:30:22 +02:00