Commit graph

37170 commits

Author SHA1 Message Date
Sebastiaan van Stijn
651e694508
Prevent panic on network attach
In situations where `container.NetworkSettings` was not nil, but
`container.NetworkSettings.Networks` was, a panic could occur:

```
2019-06-10 15:26:50.548309 I | http: panic serving @: assignment to entry in nil map
goroutine 1376 [running]:
net/http.(*conn).serve.func1(0xc4211068c0)
	/usr/local/go/src/net/http/server.go:1726 +0xd2
panic(0x558939d7e1e0, 0x55893a0c4410)
	/usr/local/go/src/runtime/panic.go:502 +0x22d
github.com/docker/docker/daemon.(*Daemon).updateNetworkSettings(0xc42090c5a0, 0xc420fb6fc0, 0x55893a101140, 0xc4210e0540, 0xc42112aa80, 0xc4217d77a0, 0x0)
	/go/src/github.com/docker/docker/daemon/container_operations.go:275 +0x40e
github.com/docker/docker/daemon.(*Daemon).updateNetworkConfig(0xc42090c5a0, 0xc420fb6fc0, 0x55893a101140, 0xc4210e0540, 0xc42112aa80, 0x55893a101101, 0xc4210e0540, 0x0)
	/go/src/github.com/docker/docker/daemon/container_operations.go:683 +0x219
github.com/docker/docker/daemon.(*Daemon).connectToNetwork(0xc42090c5a0, 0xc420fb6fc0, 0xc420e8290f, 0x40, 0xc42112aa80, 0x558937eabd01, 0x0, 0x0)
	/go/src/github.com/docker/docker/daemon/container_operations.go:728 +0x1cb
github.com/docker/docker/daemon.(*Daemon).ConnectToNetwork(0xc42090c5a0, 0xc420fb6fc0, 0xc420e8290f, 0x40, 0xc42112aa80, 0x0, 0x0)
	/go/src/github.com/docker/docker/daemon/container_operations.go:1046 +0x2b3
github.com/docker/docker/daemon.(*Daemon).ConnectContainerToNetwork(0xc42090c5a0, 0xc4214ca580, 0x40, 0xc420e8290f, 0x40, 0xc42112aa80, 0x2, 0xe600000000000001)
	/go/src/github.com/docker/docker/daemon/network.go:450 +0xa1
github.com/docker/docker/api/server/router/network.(*networkRouter).postNetworkConnect(0xc42121bbc0, 0x55893a0edee0, 0xc420de7cb0, 0x55893a0ec2e0, 0xc4207f0e00, 0xc420173600, 0xc420de7980, 0x5589394707cc, 0x5)
	/go/src/github.com/docker/docker/api/server/router/network/network_routes.go:278 +0x330
github.com/docker/docker/api/server/router/network.(*networkRouter).(github.com/docker/docker/api/server/router/network.postNetworkConnect)-fm(0x55893a0edee0, 0xc420de7cb0, 0x55893a0ec2e0, 0xc4207f0e00, 0xc420173600, 0xc420de7980, 0x558937fd89dc, 0x558939f2cec0)
	/go/src/github.com/docker/docker/api/server/router/network/network.go:37 +0x6b
github.com/docker/docker/api/server/middleware.ExperimentalMiddleware.WrapHandler.func1(0x55893a0edee0, 0xc420de7cb0, 0x55893a0ec2e0, 0xc4207f0e00, 0xc420173600, 0xc420de7980, 0x55893a0edee0, 0xc420de7cb0)
	/go/src/github.com/docker/docker/api/server/middleware/experimental.go:26 +0xda
github.com/docker/docker/api/server/middleware.VersionMiddleware.WrapHandler.func1(0x55893a0edee0, 0xc420de7a70, 0x55893a0ec2e0, 0xc4207f0e00, 0xc420173600, 0xc420de7980, 0x0, 0x0)
	/go/src/github.com/docker/docker/api/server/middleware/version.go:62 +0x401
github.com/docker/docker/pkg/authorization.(*Middleware).WrapHandler.func1(0x55893a0edee0, 0xc420de7a70, 0x55893a0ec2e0, 0xc4207f0e00, 0xc420173600, 0xc420de7980, 0x0, 0x558939640868)
	/go/src/github.com/docker/docker/pkg/authorization/middleware.go:59 +0x7ab
github.com/docker/docker/api/server/middleware.DebugRequestMiddleware.func1(0x55893a0edee0, 0xc420de7a70, 0x55893a0ec2e0, 0xc4207f0e00, 0xc420173600, 0xc420de7980, 0x55893a0edee0, 0xc420de7a70)
	/go/src/github.com/docker/docker/api/server/middleware/debug.go:53 +0x4b8
github.com/docker/docker/api/server.(*Server).makeHTTPHandler.func1(0x55893a0ec2e0, 0xc4207f0e00, 0xc420173600)
	/go/src/github.com/docker/docker/api/server/server.go:141 +0x19a
net/http.HandlerFunc.ServeHTTP(0xc420e0c0e0, 0x55893a0ec2e0, 0xc4207f0e00, 0xc420173600)
	/usr/local/go/src/net/http/server.go:1947 +0x46
github.com/docker/docker/vendor/github.com/gorilla/mux.(*Router).ServeHTTP(0xc420ce5950, 0x55893a0ec2e0, 0xc4207f0e00, 0xc420173600)
	/go/src/github.com/docker/docker/vendor/github.com/gorilla/mux/mux.go:103 +0x228
github.com/docker/docker/api/server.(*routerSwapper).ServeHTTP(0xc421078330, 0x55893a0ec2e0, 0xc4207f0e00, 0xc420173600)
	/go/src/github.com/docker/docker/api/server/router_swapper.go:29 +0x72
net/http.serverHandler.ServeHTTP(0xc420902f70, 0x55893a0ec2e0, 0xc4207f0e00, 0xc420173600)
	/usr/local/go/src/net/http/server.go:2697 +0xbe
net/http.(*conn).serve(0xc4211068c0, 0x55893a0ede20, 0xc420d81440)
	/usr/local/go/src/net/http/server.go:1830 +0x653
created by net/http.(*Server).Serve
	/usr/local/go/src/net/http/server.go:2798 +0x27d
```

I have not been able to reproduce the situation, but preventing a panic should
not hurt.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-07-22 19:44:14 +02:00
Brian Goff
e6091b28c5
Merge pull request #39422 from lemrouch/35876-workaround
Don't try to load plugin without name
2019-07-22 10:21:50 -07:00
Michael Crosby
2101bfa7e7
Merge pull request #39582 from thaJeztah/its_a_stretch_but_it_was_busted
Pin Dockerfile to -stretch variant because it's bust'd
2019-07-22 12:55:48 -04:00
Sebastiaan van Stijn
ed672bb523
Pin Dockerfile to -stretch variant
The Golang base images switch to buster, which causes some breakage
in networking and packages that are no  longer available; (`btrfs-tools`
is now an empty package, and `libprotobuf-c0-dev` is gone).

Some of out tests also start faiilng on stretch, and will have to be
investigated further;

```
15:13:06 --- FAIL: TestRenameAnonymousContainer (3.37s)
15:13:06     rename_test.go:168: assertion failed: 0 (int) != 1 (inspect.State.ExitCode int): container a7fe866d588d65f353f42ffc5ea5288e52700384e1d90850e9c3d4dce8657666 exited with the wrong exitcode:

15:13:38 --- FAIL: TestHostnameDnsResolution (2.23s)
15:13:38     run_linux_test.go:128: assertion failed:
15:13:38         --- ←
15:13:38         +++ →
15:13:38         @@ -1 +1,2 @@
15:13:38         +ping: bad address 'foobar'
15:13:38
15:13:38
15:13:38     run_linux_test.go:129: assertion failed: 0 (int) != 1 (res.ExitCode int)
```

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-07-20 12:58:20 +02:00
Sebastiaan van Stijn
4fa57a8191
Windows: fix Golang version checks for GO_VERSION build-arg
This check was used to make sure we don't bump Go versions independently
(Linux/Windows). The Dockerfile switched to using a build-arg to allow
overriding the Go version, which rendered this check non-functional.

It also fails if Linux versions use a specific variant of the image;

08:41:31 ERROR: Failed 'ERROR: Mismatched GO versions between Dockerfile and Dockerfile.windows. Update your PR to ensure that both files are updated and in sync. ${GO_VERSION}-stretch ${GO_VERSION}' at 07/20/2019 08:41:31
08:41:31 At C:\gopath\src\github.com\docker\docker\hack\ci\windows.ps1:448 char:9
08:41:31 +         Throw "ERROR: Mismatched GO versions between Dockerfile and D ...
08:41:31 +         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

This patch fixes the check by looking for the value of `GO_VERSION` instead
of looking at the `FROM` line (which is harder to parse).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-07-20 12:54:50 +02:00
Sebastiaan van Stijn
48da116486
Merge pull request #39076 from thaJeztah/remove_hack_malformed_host_header
Remove hack MalformedHostHeaderOverride
2019-07-20 02:15:10 +02:00
Michael Crosby
943129ed86
Merge pull request #39575 from thaJeztah/fix_TestHealthKillContainer
Skip TestHealthKillContainer on Windows
2019-07-19 14:52:24 -04:00
Sebastiaan van Stijn
eeaa0b30d4
Skip TestHealthKillContainer on Windows
This test is failing on Windows currently:

```
11:59:47 --- FAIL: TestHealthKillContainer (8.12s)
11:59:47     health_test.go:57: assertion failed: error is not nil: Error response from daemon: Invalid signal: SIGUSR1
``

That test was added recently in https://github.com/moby/moby/pull/39454, but
rewritten in a commit in the same PR:
f8aef6a92f

In that rewrite, there were some changes:

- originally it was skipped on Windows, but the rewritten test doesn't have that skip:

    ```go
    testRequires(c, DaemonIsLinux) // busybox doesn't work on Windows
    ```

- the original test used `SIGINT`, but the new one uses `SIGUSR1`

Analysis:

- The Error bubbles up from: 8e610b2b55/pkg/signal/signal.go (L29-L44)
- Interestingly; `ContainerKill` should validate if a signal is valid for the given platform, but somehow we don't hit that part; f1b5612f20/daemon/kill.go (L40-L48)
- Windows only looks to support 2 signals currently 8e610b2b55/pkg/signal/signal_windows.go (L17-L26)
- Upstream Golang looks to define `SIGINT` as well; 77f9b2728e/src/runtime/defs_windows.go (L44)
- This looks like the current list of Signals upstream in Go; 3b58ed4ad3/windows/types_windows.go (L52-L67)

```go
const (
	// More invented values for signals
	SIGHUP  = Signal(0x1)
	SIGINT  = Signal(0x2)
	SIGQUIT = Signal(0x3)
	SIGILL  = Signal(0x4)
	SIGTRAP = Signal(0x5)
	SIGABRT = Signal(0x6)
	SIGBUS  = Signal(0x7)
	SIGFPE  = Signal(0x8)
	SIGKILL = Signal(0x9)
	SIGSEGV = Signal(0xb)
	SIGPIPE = Signal(0xd)
	SIGALRM = Signal(0xe)
	SIGTERM = Signal(0xf)
)
```

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-07-19 16:09:50 +02:00
Akihiro Suda
88f6b5e746
Merge pull request #39554 from thaJeztah/session_not_experimental
integration: run build session tests on non-experimental
2019-07-19 16:33:09 +09:00
Sebastiaan van Stijn
0949fe8283
Merge pull request #39548 from thaJeztah/golang_arg
Dockerfile: use GO_VERSION build-arg for overriding Go version
2019-07-19 09:25:10 +02:00
Kir Kolyshkin
a557538770 TESTING.md: document GO_VERSION
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2019-07-18 17:36:09 -07:00
Sebastiaan van Stijn
c6281bc438 Dockerfile: use GO_VERSION build-arg for overriding Go version
This allows overriding the version of Go without making modifications in the
source code, which can be useful to test against multiple versions.

For example:

    make GO_VERSION=1.13beta1 shell

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-07-18 17:36:09 -07:00
Sebastiaan van Stijn
b4058a6c91
Merge pull request #39258 from thaJeztah/revert_remove_TestSearchCmdOptions
Revert "Remove TestSearchCmdOptions test"
2019-07-19 01:30:22 +02:00
Sebastiaan van Stijn
cff71f84d1
Merge pull request #39560 from dperny/fix-flaky-swarm-tests-2
Retry service updates on out of sequence errors
2019-07-19 01:24:11 +02:00
Sebastiaan van Stijn
f6b1f01de3
Remove hack MalformedHostHeaderOverride
This hack was added to fix a compatibility with clients
that were built using Go 1.5 and older (added in 3d6f5984f5)

This hack causes some problems with current clients; with Go 1.5 and older
no longer being supported for some time, and being several years old, it
should now be ok to remove this hack altogether.

People using tools that are built with those versions of Go wouldn't have
updated those for years, and are probably out of date anyway; that's not
something we can continue taking into account.

This will affect docker clients (the docker cli) for docker 1.12 and older.
Those versions have reached EOL a long time ago (and have known unpatched
vulnerabilities), so should no longer be used anyway, but We should add
a nebtuib in the release notes, just in case someone, somewhere, still
has such old tools.

For those affected, using a more recent client (and if needed, setting
the DOCKER_API_VERSION environment variable to the needed API version)
should provide a way out.

This reverts the changes originally made in; #22000 and #22888,
which were to address #20865.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-07-18 21:25:04 +02:00
Sebastiaan van Stijn
9b67b40e1a
Merge pull request #39552 from ddebroy/fix-39544
Be more conservative for Windows in TestFrequency for Splunk
2019-07-18 21:02:27 +02:00
Tibor Vass
764160031e
Merge pull request #39068 from thaJeztah/separate_docker_py
Bump docker-py to 4.0.2, and run tests from upstream repository
2019-07-18 11:54:16 -07:00
Drew Erny
1de914695b Retry service updates on out of sequence errors
Code retrying service update operations when receiving "update out of
sequence" errors was removed because of a misunderstanding, which has
made tests flaky. This re-adds the "CmdRetryOutOfSequence" method, and
uses it in TestSwarmPublishAdd to avoid flaky behavior.

Signed-off-by: Drew Erny <drew.erny@docker.com>
2019-07-18 12:58:21 -05:00
Michael Crosby
fd6f0b1cab
Merge pull request #39531 from tonistiigi/swarm-ci-check
integration-cli: fix swarm tests flakiness
2019-07-18 10:09:34 -04:00
Sebastiaan van Stijn
de784d47c4
Merge pull request #39557 from tonistiigi/test-health-timeout
integration-cli: increase healthcheck timeout
2019-07-18 12:38:16 +02:00
Tonis Tiigi
8c9362857f integration-cli: increase healthcheck timeout
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-07-17 18:28:34 -07:00
Sebastiaan van Stijn
08191c3b90
Merge pull request #39165 from stafwag/master
Updated mkimage-arch.sh
2019-07-18 00:44:28 +02:00
Sebastiaan van Stijn
becd29c665
integration: run build session tests on non-experimental
The session endpoint is no longer experimental since
01c9e7082e, so we don't
need to start an experimental daemon.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-07-17 23:56:04 +02:00
Deep Debroy
a5c420ac54 Be more conservative for Windows in TestFrequency for Splunk
Signed-off-by: Deep Debroy <ddebroy@docker.com>
2019-07-17 13:13:50 -07:00
Tibor Vass
c4aadee761
Merge pull request #39493 from AkihiroSuda/rootlesskit-060
rootless: allow exposing dockerd TCP socket easily
2019-07-17 12:34:42 -07:00
Tonis Tiigi
096a7afd37 vendor: update swarmkit to 7dded76ec
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-07-17 11:52:58 -07:00
Tonis Tiigi
52e0dfef90 integration-cli: allow temporary no-leader error
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-07-17 11:52:58 -07:00
Tonis Tiigi
3df1095bbd integration-cli: allow temporary errors on leader switch
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-07-17 11:52:58 -07:00
Staf Wagemakers
0a1947cb9f spaces to tabs
Signed-off-by: Staf Wagemakers <staf@wagemakers.be>
2019-07-17 19:48:13 +02:00
Sebastiaan van Stijn
42a054473b
Merge pull request #39546 from jiahongxujia/master
imporve hardcoded CC on cross compile
2019-07-17 17:43:44 +02:00
Sebastiaan van Stijn
e4b9edd31f
Merge pull request from GHSA-99qr-h2rp-65rc
DebugRequestMiddleware: unconditionally scrub data field
2019-07-17 17:19:48 +02:00
Sebastiaan van Stijn
3d1850d10d
integration-cli: also run Docker Hub search tests on Windows
The API does not filter images on platform, so searching on
Windows should work as well.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-07-17 13:10:27 +02:00
Sebastiaan van Stijn
2ac55d5c9a
TestSearchWithLimit: slight refactor and improve boundary checks
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-07-17 13:09:17 +02:00
Sebastiaan van Stijn
85d6fb888c
TestSearchStarsOptionWithWrongParameter: remove checks for deprecated flags
The `--stars` flag was deprecated, and was replaced by `--filter stars=xx`

Integration tests run with a fixed version of the CLI, and the new
(`--filter`) option is already tested in this test, so there's no need
to verify the old flags.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-07-17 13:07:17 +02:00
Sebastiaan van Stijn
b38c71bfe0
TestSearchCmdOptions: remove checks for deprecated flags
The `--stars` and `--automated` flags have been deprecated, and were
replaced by `--filter stars=xx` and `--filter is-automated=true`.

Integration tests run with a fixed version of the CLI, and the new
(`--filter`) option is already tested in this test, so there's no need
to verify the old flags.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-07-17 12:47:57 +02:00
Sebastiaan van Stijn
a78b9a3726
TestSearchCmdOptions: remove cli-only checks
Both `--help` and `--no-trunc` are implemented in the CLI. There's
no need to verify them here because the integration tests use a
fixed version of the CLI.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-07-17 12:41:30 +02:00
Sebastiaan van Stijn
1be7065e99
Revert "Remove TestSearchCmdOptions test"
This reverts commit 21e662c774.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-07-17 12:26:35 +02:00
Hongxu Jia
3c701e4db1 imporve hardcoded CC on cross compile
Since commit applied in moby [61a3285 Support cross-compile for arm]
it hardcoded var-CC to support cross-compile for arm

Correct it with "${parameter:-word}" format, it is helpful for user
define toolchains

(Use Default Values.  If parameter is unset or null, the expansion of
word is substituted.  Otherwise, the value of parameter is substituted.)

Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
2019-07-17 17:42:14 +08:00
Tibor Vass
880feeb3a8
Merge pull request #39543 from tonistiigi/64align
atomic: patch 64bit alignment on 32bit systems
2019-07-16 22:34:21 -07:00
Tonis Tiigi
af2e82d054 atomic: patch 64bit alignment on 32bit systems
causes panic on armv7

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-07-16 16:57:58 -07:00
Sebastiaan van Stijn
5d04e0adce
Merge pull request #39540 from andrewhsu/run-single-integration
allow running of single integration test
2019-07-17 00:47:15 +02:00
Andrew Hsu
c222c5ac6f allow running of single integration test
Signed-off-by: Andrew Hsu <andrewhsu@docker.com>
Signed-off-by: Tibor Vass <tibor@docker.com>
2019-07-16 21:09:25 +00:00
Sebastiaan van Stijn
618be06424
Merge pull request #37147 from bachp/patch-1
Check for BRIDGE_VLAN_FILTERING in overlay section
2019-07-16 21:06:36 +02:00
Sebastiaan van Stijn
6aafe0fd9e
WIP Move docker-py tests first again
See if networking works if we run it first

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-07-16 19:53:17 +02:00
Pascal Bach
78405559cf
Check for BRIDGE_VLAN_FILTERING in overlay section
Overlay networking in docker stack does not work correctly if this option is missing, docker will output the following error:

```
enabling default vlan on bridge br0 failed open /sys/class/net/br0/bridge/default_pvdi: permission denied
```

This because `default_pvdi` does not exist without this option.

Signed-off-by: Pascal Bach <pascal.bach@siemens.com>
Signed-off-by: Simon Ausserlechner <simon.ausserlechner@siemens.com>
2019-07-16 17:32:51 +02:00
Sebastiaan van Stijn
7169b2328c
Merge pull request #39537 from thaJeztah/cdn_mirror_deb_security
Dockerfile: Use APT_MIRROR for security.debian.org as well
2019-07-16 16:51:38 +02:00
Sebastiaan van Stijn
c7bbb1c5a1
Merge pull request #39329 from tiborvass/buildkit-honor-daemon-dnsconfig
build: buildkit now honors daemon's DNS config
2019-07-16 16:19:20 +02:00
Sebastiaan van Stijn
980f2813b4
docker-py: skip flaky tests
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-07-16 14:35:00 +02:00
Sebastiaan van Stijn
3c15cea650
docker-py: use host-network for nested build of docker-py
When building this image docker-in-docker, the DNS in the environment
may not be usable for the build-container, causing resolution to fail:

```
02:35:31 W: Failed to fetch http://deb.debian.org/debian/dists/jessie/Release.gpg  Temporary failure resolving 'deb.debian.org'
```

This patch detects if we're building from within a container, and if
so, skips creating a networking namespace for the build by using
`--network=host`.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-07-16 14:34:54 +02:00
Sebastiaan van Stijn
ba8f4c7994
docker-py: don't build --quiet is TESTDEBUG is set
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-07-16 14:34:52 +02:00