Commit graph

37350 commits

Author SHA1 Message Date
Sebastiaan van Stijn
b1723b3721
Dockerfile: add back yamllint
This was inadvertedly removed in 7bfe48cc00,
because it was documented as a dependency for docker-py, but
actually used to validate the swagger file.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-07-30 17:04:26 -07:00
Zuhayr Elahi
438c7eb606
Added information regarding our new Jenkins ci on moby/moby
Added link to header

Signed-off-by: Zuhayr Elahi <elahi.zuhayr@gmail.com>
2019-07-30 13:48:00 -07:00
Sebastiaan van Stijn
6f234db9fe
bump libnetwork to 09cdcc8c0eab3946c2d70e8f6225b05baf1e90d1
full diff: 83d30db536...09cdcc8c0e

changes included:

- docker/libnetwork#2416 Fix hardcoded AF_INET for IPv6 address handling
- docker/libnetwork#2411 Macvlan network handles netlabel.Internal wrong
  - fixes docker/libnetwork#2410 Macvlan network handles netlabel.Internal wrong
- docker/libnetwork#2414 Allow network with --config-from to be --internal
  - fixes docker/libnetwork#2413 Network with --config-from does not honor --internal
- docker/libnetwork#2351 Use fewer modprobes
  - relates to moby/moby#38930 Use fewer modprobes
- docker/libnetwork#2415 Support dockerd and system restarts for ipvlan and macvlan networks
  - carry of docker/libnetwork#2295 phantom ip/mac vlan network after a powercycle
  - fixes docker/libnetwork#1743 Phantom docker network

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-07-30 10:37:48 -07:00
Brian Goff
7cfd8146dc
Merge pull request #39629 from thaJeztah/switch_creack_pty
switch kr/pty to creack/pty v1.1.7
2019-07-30 09:41:51 -07:00
Sebastiaan van Stijn
0595c01718
switch kr/pty to creack/pty v1.1.7
kr/pty was moved to creak/pty and the old location was
archived.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-07-29 16:59:08 -07:00
Brian Goff
582591d56b
Merge pull request #39569 from thaJeztah/remove_redundant_checks
integration-cli: remove redundant "testrequires"
2019-07-29 10:36:09 -07:00
Yong Tang
9c92080b13
Merge pull request #39589 from thaJeztah/prevent_network_attach_panic
Prevent panic on network attach
2019-07-29 07:10:59 -07:00
Tibor Vass
917a8b4259
Merge pull request #39616 from dperny/fix-cluster-rotate-unlock-key
Fix TestSwarmClusterRotateUnlockKey
2019-07-26 16:09:22 -07:00
Sebastiaan van Stijn
bd983a60bb
Merge pull request #39592 from zelahi/integrate-with-jsp-jenkinsfile
Added changes to integrate with the new Jenkins ci
2019-07-26 14:16:21 -07:00
Drew Erny
b79adac339 Fix TestSwarmClusterRotateUnlockKey
TestSwarmClusterRotateUnlockKey had been identified as a flaky test. It
turns out that the test code was wrong: where we should have been
checking the string output of a command, we were instead checking the
value of the error. This means that the error case we were expecting was
not being matched, and the test was failing when it should have just
retried.

Signed-off-by: Drew Erny <drew.erny@docker.com>
2019-07-26 15:51:56 -05:00
zelahi
0ecd6ab30f
ADDED changes to integrate with our new Jenkins ci
Signed-off-by: zelahi <elahi.zuhayr@gmail.com>
2019-07-26 11:53:36 -07:00
Sebastiaan van Stijn
9f122f0d2e
Merge pull request #39591 from crosbymichael/perm-test
Add extra permission check in removal test
2019-07-26 09:41:51 -07:00
Akihiro Suda
11e48badcb
Merge pull request #39612 from tiborvass/cve-2019-14271
Fix CVE-2019-14271 loading of nsswitch based config inside chroot under Glibc
2019-07-26 13:31:28 +09:00
Justin Cormack
a316b10dab Initialize nss libraries in Glibc so that the dynamic libraries are loaded in the host
environment not in the chroot from untrusted files.

See also OpenVZ a3f732ef75/src/enter.c (L227-L234)

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
(cherry picked from commit cea6dca993c2b4cfa99b1e7a19ca134c8ebc236b)
Signed-off-by: Tibor Vass <tibor@docker.com>
2019-07-26 01:27:57 +00:00
Brian Goff
17fe47d6c9
Merge pull request #39594 from andrewhsu/o2
use overlay2 for janky and experimental checks
2019-07-25 14:53:52 -07:00
Andrew Hsu
ccfaf1ed92 use overlay2 for janky and experimental checks
instead of vfs

Signed-off-by: Andrew Hsu <andrewhsu@docker.com>
2019-07-25 21:51:53 +00:00
Brian Goff
8297dc26e6
Merge pull request #39606 from andrewhsu/execdriver
remove DOCKER_EXECDRIVER from Jenkinsfile
2019-07-25 12:59:28 -07:00
Brian Goff
a1885bd655
Merge pull request #39603 from cpuguy83/test_daemon_logging
Better logging for swarm tests
2019-07-25 12:58:53 -07:00
Vincent Demeester
ecaf0b479f
Merge pull request #39602 from cpuguy83/env_from_client
Add `FromClient` to test env execution
2019-07-25 10:34:47 +02:00
Andrew Hsu
9d98458fb7 remove DOCKER_EXECDRIVER from Jenkinsfile
Signed-off-by: Andrew Hsu <andrewhsu@docker.com>
2019-07-25 01:25:16 +00:00
Brian Goff
b0fe0dff7a Better logging for swarm tests
Call helper for starting swarm agents and add some logging with daemon
id's when joining the swarm.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2019-07-24 16:38:11 -07:00
Brian Goff
1381956499 Add FromClient to test env execution
While working on other tests I noticed that environment.Execution cannot
be used for anything but the pre-configured daemon, however this can
come in handy for being able share daemons across multiple tests that
currently spin up a new daemon.
The execution env also seems to be misused in some of these cases.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2019-07-24 12:27:41 -07:00
Sebastiaan van Stijn
a9dc697fd2
Merge pull request #39535 from michael2012z/fix_timeout_setting
Set TIMEOUT according to architecture.
2019-07-23 08:46:12 +02:00
Michael Zhao
790da6c223 Set TIMEOUT according to os/arch.
Signed-off-by: Michael Zhao <michael.zhao@arm.com>
2019-07-23 10:44:50 +08:00
Sebastiaan van Stijn
2cdaceb2a0
Merge pull request #39580 from andrewhsu/ci-master
added hack/ci/master as entry point for master codeline checks
2019-07-23 01:06:28 +02:00
Michael Crosby
d6cbeee470 Add extra permission check in removal test
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-07-22 20:39:22 +00:00
Andrew Hsu
aac6e62209
added hack/ci/master as entry point for master codeline checks
Signed-off-by: Andrew Hsu <andrewhsu@docker.com>
2019-07-22 20:00:07 +02:00
Brian Goff
b00666366b
Merge pull request #39585 from thaJeztah/trim_the_zero
Dockerfile.windows: trim .0 from Go versions
2019-07-22 10:47:21 -07:00
Sebastiaan van Stijn
651e694508
Prevent panic on network attach
In situations where `container.NetworkSettings` was not nil, but
`container.NetworkSettings.Networks` was, a panic could occur:

```
2019-06-10 15:26:50.548309 I | http: panic serving @: assignment to entry in nil map
goroutine 1376 [running]:
net/http.(*conn).serve.func1(0xc4211068c0)
	/usr/local/go/src/net/http/server.go:1726 +0xd2
panic(0x558939d7e1e0, 0x55893a0c4410)
	/usr/local/go/src/runtime/panic.go:502 +0x22d
github.com/docker/docker/daemon.(*Daemon).updateNetworkSettings(0xc42090c5a0, 0xc420fb6fc0, 0x55893a101140, 0xc4210e0540, 0xc42112aa80, 0xc4217d77a0, 0x0)
	/go/src/github.com/docker/docker/daemon/container_operations.go:275 +0x40e
github.com/docker/docker/daemon.(*Daemon).updateNetworkConfig(0xc42090c5a0, 0xc420fb6fc0, 0x55893a101140, 0xc4210e0540, 0xc42112aa80, 0x55893a101101, 0xc4210e0540, 0x0)
	/go/src/github.com/docker/docker/daemon/container_operations.go:683 +0x219
github.com/docker/docker/daemon.(*Daemon).connectToNetwork(0xc42090c5a0, 0xc420fb6fc0, 0xc420e8290f, 0x40, 0xc42112aa80, 0x558937eabd01, 0x0, 0x0)
	/go/src/github.com/docker/docker/daemon/container_operations.go:728 +0x1cb
github.com/docker/docker/daemon.(*Daemon).ConnectToNetwork(0xc42090c5a0, 0xc420fb6fc0, 0xc420e8290f, 0x40, 0xc42112aa80, 0x0, 0x0)
	/go/src/github.com/docker/docker/daemon/container_operations.go:1046 +0x2b3
github.com/docker/docker/daemon.(*Daemon).ConnectContainerToNetwork(0xc42090c5a0, 0xc4214ca580, 0x40, 0xc420e8290f, 0x40, 0xc42112aa80, 0x2, 0xe600000000000001)
	/go/src/github.com/docker/docker/daemon/network.go:450 +0xa1
github.com/docker/docker/api/server/router/network.(*networkRouter).postNetworkConnect(0xc42121bbc0, 0x55893a0edee0, 0xc420de7cb0, 0x55893a0ec2e0, 0xc4207f0e00, 0xc420173600, 0xc420de7980, 0x5589394707cc, 0x5)
	/go/src/github.com/docker/docker/api/server/router/network/network_routes.go:278 +0x330
github.com/docker/docker/api/server/router/network.(*networkRouter).(github.com/docker/docker/api/server/router/network.postNetworkConnect)-fm(0x55893a0edee0, 0xc420de7cb0, 0x55893a0ec2e0, 0xc4207f0e00, 0xc420173600, 0xc420de7980, 0x558937fd89dc, 0x558939f2cec0)
	/go/src/github.com/docker/docker/api/server/router/network/network.go:37 +0x6b
github.com/docker/docker/api/server/middleware.ExperimentalMiddleware.WrapHandler.func1(0x55893a0edee0, 0xc420de7cb0, 0x55893a0ec2e0, 0xc4207f0e00, 0xc420173600, 0xc420de7980, 0x55893a0edee0, 0xc420de7cb0)
	/go/src/github.com/docker/docker/api/server/middleware/experimental.go:26 +0xda
github.com/docker/docker/api/server/middleware.VersionMiddleware.WrapHandler.func1(0x55893a0edee0, 0xc420de7a70, 0x55893a0ec2e0, 0xc4207f0e00, 0xc420173600, 0xc420de7980, 0x0, 0x0)
	/go/src/github.com/docker/docker/api/server/middleware/version.go:62 +0x401
github.com/docker/docker/pkg/authorization.(*Middleware).WrapHandler.func1(0x55893a0edee0, 0xc420de7a70, 0x55893a0ec2e0, 0xc4207f0e00, 0xc420173600, 0xc420de7980, 0x0, 0x558939640868)
	/go/src/github.com/docker/docker/pkg/authorization/middleware.go:59 +0x7ab
github.com/docker/docker/api/server/middleware.DebugRequestMiddleware.func1(0x55893a0edee0, 0xc420de7a70, 0x55893a0ec2e0, 0xc4207f0e00, 0xc420173600, 0xc420de7980, 0x55893a0edee0, 0xc420de7a70)
	/go/src/github.com/docker/docker/api/server/middleware/debug.go:53 +0x4b8
github.com/docker/docker/api/server.(*Server).makeHTTPHandler.func1(0x55893a0ec2e0, 0xc4207f0e00, 0xc420173600)
	/go/src/github.com/docker/docker/api/server/server.go:141 +0x19a
net/http.HandlerFunc.ServeHTTP(0xc420e0c0e0, 0x55893a0ec2e0, 0xc4207f0e00, 0xc420173600)
	/usr/local/go/src/net/http/server.go:1947 +0x46
github.com/docker/docker/vendor/github.com/gorilla/mux.(*Router).ServeHTTP(0xc420ce5950, 0x55893a0ec2e0, 0xc4207f0e00, 0xc420173600)
	/go/src/github.com/docker/docker/vendor/github.com/gorilla/mux/mux.go:103 +0x228
github.com/docker/docker/api/server.(*routerSwapper).ServeHTTP(0xc421078330, 0x55893a0ec2e0, 0xc4207f0e00, 0xc420173600)
	/go/src/github.com/docker/docker/api/server/router_swapper.go:29 +0x72
net/http.serverHandler.ServeHTTP(0xc420902f70, 0x55893a0ec2e0, 0xc4207f0e00, 0xc420173600)
	/usr/local/go/src/net/http/server.go:2697 +0xbe
net/http.(*conn).serve(0xc4211068c0, 0x55893a0ede20, 0xc420d81440)
	/usr/local/go/src/net/http/server.go:1830 +0x653
created by net/http.(*Server).Serve
	/usr/local/go/src/net/http/server.go:2798 +0x27d
```

I have not been able to reproduce the situation, but preventing a panic should
not hurt.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-07-22 19:44:14 +02:00
Brian Goff
e6091b28c5
Merge pull request #39422 from lemrouch/35876-workaround
Don't try to load plugin without name
2019-07-22 10:21:50 -07:00
Michael Crosby
2101bfa7e7
Merge pull request #39582 from thaJeztah/its_a_stretch_but_it_was_busted
Pin Dockerfile to -stretch variant because it's bust'd
2019-07-22 12:55:48 -04:00
Sebastiaan van Stijn
c5bd6e3dc7
Dockerfile.windows: trim .0 from Go versions
This was an oversight when changing the Dockerfile to use a build-arg;
the Windows Dockerfile downloads the Go binaries, which never have a
trailing `.0`.

This patch makes sure that the trailing zero (if any) is removed.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-07-20 13:00:56 +02:00
Sebastiaan van Stijn
ed672bb523
Pin Dockerfile to -stretch variant
The Golang base images switch to buster, which causes some breakage
in networking and packages that are no  longer available; (`btrfs-tools`
is now an empty package, and `libprotobuf-c0-dev` is gone).

Some of out tests also start faiilng on stretch, and will have to be
investigated further;

```
15:13:06 --- FAIL: TestRenameAnonymousContainer (3.37s)
15:13:06     rename_test.go:168: assertion failed: 0 (int) != 1 (inspect.State.ExitCode int): container a7fe866d588d65f353f42ffc5ea5288e52700384e1d90850e9c3d4dce8657666 exited with the wrong exitcode:

15:13:38 --- FAIL: TestHostnameDnsResolution (2.23s)
15:13:38     run_linux_test.go:128: assertion failed:
15:13:38         --- ←
15:13:38         +++ →
15:13:38         @@ -1 +1,2 @@
15:13:38         +ping: bad address 'foobar'
15:13:38
15:13:38
15:13:38     run_linux_test.go:129: assertion failed: 0 (int) != 1 (res.ExitCode int)
```

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-07-20 12:58:20 +02:00
Sebastiaan van Stijn
4fa57a8191
Windows: fix Golang version checks for GO_VERSION build-arg
This check was used to make sure we don't bump Go versions independently
(Linux/Windows). The Dockerfile switched to using a build-arg to allow
overriding the Go version, which rendered this check non-functional.

It also fails if Linux versions use a specific variant of the image;

08:41:31 ERROR: Failed 'ERROR: Mismatched GO versions between Dockerfile and Dockerfile.windows. Update your PR to ensure that both files are updated and in sync. ${GO_VERSION}-stretch ${GO_VERSION}' at 07/20/2019 08:41:31
08:41:31 At C:\gopath\src\github.com\docker\docker\hack\ci\windows.ps1:448 char:9
08:41:31 +         Throw "ERROR: Mismatched GO versions between Dockerfile and D ...
08:41:31 +         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

This patch fixes the check by looking for the value of `GO_VERSION` instead
of looking at the `FROM` line (which is harder to parse).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-07-20 12:54:50 +02:00
Sebastiaan van Stijn
48da116486
Merge pull request #39076 from thaJeztah/remove_hack_malformed_host_header
Remove hack MalformedHostHeaderOverride
2019-07-20 02:15:10 +02:00
Michael Crosby
943129ed86
Merge pull request #39575 from thaJeztah/fix_TestHealthKillContainer
Skip TestHealthKillContainer on Windows
2019-07-19 14:52:24 -04:00
Sebastiaan van Stijn
eeaa0b30d4
Skip TestHealthKillContainer on Windows
This test is failing on Windows currently:

```
11:59:47 --- FAIL: TestHealthKillContainer (8.12s)
11:59:47     health_test.go:57: assertion failed: error is not nil: Error response from daemon: Invalid signal: SIGUSR1
``

That test was added recently in https://github.com/moby/moby/pull/39454, but
rewritten in a commit in the same PR:
f8aef6a92f

In that rewrite, there were some changes:

- originally it was skipped on Windows, but the rewritten test doesn't have that skip:

    ```go
    testRequires(c, DaemonIsLinux) // busybox doesn't work on Windows
    ```

- the original test used `SIGINT`, but the new one uses `SIGUSR1`

Analysis:

- The Error bubbles up from: 8e610b2b55/pkg/signal/signal.go (L29-L44)
- Interestingly; `ContainerKill` should validate if a signal is valid for the given platform, but somehow we don't hit that part; f1b5612f20/daemon/kill.go (L40-L48)
- Windows only looks to support 2 signals currently 8e610b2b55/pkg/signal/signal_windows.go (L17-L26)
- Upstream Golang looks to define `SIGINT` as well; 77f9b2728e/src/runtime/defs_windows.go (L44)
- This looks like the current list of Signals upstream in Go; 3b58ed4ad3/windows/types_windows.go (L52-L67)

```go
const (
	// More invented values for signals
	SIGHUP  = Signal(0x1)
	SIGINT  = Signal(0x2)
	SIGQUIT = Signal(0x3)
	SIGILL  = Signal(0x4)
	SIGTRAP = Signal(0x5)
	SIGABRT = Signal(0x6)
	SIGBUS  = Signal(0x7)
	SIGFPE  = Signal(0x8)
	SIGKILL = Signal(0x9)
	SIGSEGV = Signal(0xb)
	SIGPIPE = Signal(0xd)
	SIGALRM = Signal(0xe)
	SIGTERM = Signal(0xf)
)
```

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-07-19 16:09:50 +02:00
Sebastiaan van Stijn
7f37d99ef5
integration-cli: remove redundant "testrequires"
The `DockerDaemonSuite.SetUpTest` already checks for Linux and a local daemon;

```
func (s *DockerDaemonSuite) SetUpTest(c *check.C) {
	testRequires(c, DaemonIsLinux, testEnv.IsLocalDaemon)
	s.d = daemon.New(c, dockerBinary, dockerdBinary, testdaemon.WithEnvironment(testEnv.Execution))
}
```

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-07-19 10:53:42 +02:00
Akihiro Suda
88f6b5e746
Merge pull request #39554 from thaJeztah/session_not_experimental
integration: run build session tests on non-experimental
2019-07-19 16:33:09 +09:00
Sebastiaan van Stijn
0949fe8283
Merge pull request #39548 from thaJeztah/golang_arg
Dockerfile: use GO_VERSION build-arg for overriding Go version
2019-07-19 09:25:10 +02:00
Kir Kolyshkin
a557538770 TESTING.md: document GO_VERSION
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2019-07-18 17:36:09 -07:00
Sebastiaan van Stijn
c6281bc438 Dockerfile: use GO_VERSION build-arg for overriding Go version
This allows overriding the version of Go without making modifications in the
source code, which can be useful to test against multiple versions.

For example:

    make GO_VERSION=1.13beta1 shell

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-07-18 17:36:09 -07:00
Sebastiaan van Stijn
b4058a6c91
Merge pull request #39258 from thaJeztah/revert_remove_TestSearchCmdOptions
Revert "Remove TestSearchCmdOptions test"
2019-07-19 01:30:22 +02:00
Sebastiaan van Stijn
cff71f84d1
Merge pull request #39560 from dperny/fix-flaky-swarm-tests-2
Retry service updates on out of sequence errors
2019-07-19 01:24:11 +02:00
Sebastiaan van Stijn
f6b1f01de3
Remove hack MalformedHostHeaderOverride
This hack was added to fix a compatibility with clients
that were built using Go 1.5 and older (added in 3d6f5984f5)

This hack causes some problems with current clients; with Go 1.5 and older
no longer being supported for some time, and being several years old, it
should now be ok to remove this hack altogether.

People using tools that are built with those versions of Go wouldn't have
updated those for years, and are probably out of date anyway; that's not
something we can continue taking into account.

This will affect docker clients (the docker cli) for docker 1.12 and older.
Those versions have reached EOL a long time ago (and have known unpatched
vulnerabilities), so should no longer be used anyway, but We should add
a nebtuib in the release notes, just in case someone, somewhere, still
has such old tools.

For those affected, using a more recent client (and if needed, setting
the DOCKER_API_VERSION environment variable to the needed API version)
should provide a way out.

This reverts the changes originally made in; #22000 and #22888,
which were to address #20865.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-07-18 21:25:04 +02:00
Sebastiaan van Stijn
9b67b40e1a
Merge pull request #39552 from ddebroy/fix-39544
Be more conservative for Windows in TestFrequency for Splunk
2019-07-18 21:02:27 +02:00
Tibor Vass
764160031e
Merge pull request #39068 from thaJeztah/separate_docker_py
Bump docker-py to 4.0.2, and run tests from upstream repository
2019-07-18 11:54:16 -07:00
Drew Erny
1de914695b Retry service updates on out of sequence errors
Code retrying service update operations when receiving "update out of
sequence" errors was removed because of a misunderstanding, which has
made tests flaky. This re-adds the "CmdRetryOutOfSequence" method, and
uses it in TestSwarmPublishAdd to avoid flaky behavior.

Signed-off-by: Drew Erny <drew.erny@docker.com>
2019-07-18 12:58:21 -05:00
Michael Crosby
fd6f0b1cab
Merge pull request #39531 from tonistiigi/swarm-ci-check
integration-cli: fix swarm tests flakiness
2019-07-18 10:09:34 -04:00
Sebastiaan van Stijn
de784d47c4
Merge pull request #39557 from tonistiigi/test-health-timeout
integration-cli: increase healthcheck timeout
2019-07-18 12:38:16 +02:00