Commit graph

1452 commits

Author SHA1 Message Date
Sebastiaan van Stijn
5858a99267
Builder: fix "COPY --from" to non-existing directory on Windows
This fixes a regression introduced in 6d87f19142,
causing `COPY --from` to fail if the target directory does not exist:

```
FROM mcr.microsoft.com/windows/servercore:ltsc2019 as s1
RUN echo "Hello World" > /hello

FROM mcr.microsoft.com/windows/servercore:ltsc2019
COPY --from=s1 /hello /hello/another/world
```

Would produce an error:

```
Step 4/4 : COPY --from=s1 /hello /hello/another/world
failed to copy files: mkdir \\?: The filename, directory name, or volume label syntax is incorrect.
```

The cause for this was that Go's `os.MkdirAll()` does not support/detect volume GUID paths
(`\\?\Volume{dae8d3ac-b9a1-11e9-88eb-e8554b2ba1db}\hello\another}`), and as a result
attempted to create the volume as a directory (`\\?`), causing it to fail.

This patch replaces `os.MkdirAll()` with our own `system.MkdirAll()` function, which
is capable of detecting GUID volumes.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-08-08 16:45:10 +02:00
Brian Goff
abece9b562 Improvements to the test runners
1. Use `go list` to get list of integration dirs to build. This means we
   do not need to have a valid `.go` in every subdirectory and also
   filters out other dirs like "bundles" which may have been created.
2. Add option to specify custom flags for integration and
   integration-cli. This is needed so both suites can be run AND set
   custom flags... since the cli suite does not support standard go
   flags.
3. Add options to skip an entire integration suite.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2019-07-31 16:37:17 -07:00
Michael Crosby
d6cbeee470 Add extra permission check in removal test
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-07-22 20:39:22 +00:00
Sebastiaan van Stijn
eeaa0b30d4
Skip TestHealthKillContainer on Windows
This test is failing on Windows currently:

```
11:59:47 --- FAIL: TestHealthKillContainer (8.12s)
11:59:47     health_test.go:57: assertion failed: error is not nil: Error response from daemon: Invalid signal: SIGUSR1
``

That test was added recently in https://github.com/moby/moby/pull/39454, but
rewritten in a commit in the same PR:
f8aef6a92f

In that rewrite, there were some changes:

- originally it was skipped on Windows, but the rewritten test doesn't have that skip:

    ```go
    testRequires(c, DaemonIsLinux) // busybox doesn't work on Windows
    ```

- the original test used `SIGINT`, but the new one uses `SIGUSR1`

Analysis:

- The Error bubbles up from: 8e610b2b55/pkg/signal/signal.go (L29-L44)
- Interestingly; `ContainerKill` should validate if a signal is valid for the given platform, but somehow we don't hit that part; f1b5612f20/daemon/kill.go (L40-L48)
- Windows only looks to support 2 signals currently 8e610b2b55/pkg/signal/signal_windows.go (L17-L26)
- Upstream Golang looks to define `SIGINT` as well; 77f9b2728e/src/runtime/defs_windows.go (L44)
- This looks like the current list of Signals upstream in Go; 3b58ed4ad3/windows/types_windows.go (L52-L67)

```go
const (
	// More invented values for signals
	SIGHUP  = Signal(0x1)
	SIGINT  = Signal(0x2)
	SIGQUIT = Signal(0x3)
	SIGILL  = Signal(0x4)
	SIGTRAP = Signal(0x5)
	SIGABRT = Signal(0x6)
	SIGBUS  = Signal(0x7)
	SIGFPE  = Signal(0x8)
	SIGKILL = Signal(0x9)
	SIGSEGV = Signal(0xb)
	SIGPIPE = Signal(0xd)
	SIGALRM = Signal(0xe)
	SIGTERM = Signal(0xf)
)
```

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-07-19 16:09:50 +02:00
Sebastiaan van Stijn
becd29c665
integration: run build session tests on non-experimental
The session endpoint is no longer experimental since
01c9e7082e, so we don't
need to start an experimental daemon.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-07-17 23:56:04 +02:00
Sebastiaan van Stijn
9d1b4f5fc3
Add regression tests for invalid platform status codes
Before we handled containerd errors, using an invalid platform produced a 500 status:

```bash
curl -v \
  -X POST \
  --unix-socket /var/run/docker.sock \
  "http://localhost:2375/v1.40/images/create?fromImage=hello-world&platform=foobar&tag=latest" \
  -H "Content-Type: application/json"
```

```
* Connected to localhost (docker.sock) port 80 (#0)
> POST /v1.40/images/create?fromImage=hello-world&platform=foobar&tag=latest HTTP/1.1
> Host: localhost:2375
> User-Agent: curl/7.54.0
> Accept: */*
> Content-Type: application/json
>
< HTTP/1.1 500 Internal Server Error
< Api-Version: 1.40
< Content-Length: 85
< Content-Type: application/json
< Date: Mon, 15 Jul 2019 15:25:44 GMT
< Docker-Experimental: true
< Ostype: linux
< Server: Docker/19.03.0-rc2 (linux)
<
{"message":"\"foobar\": unknown operating system or architecture: invalid argument"}
```

That problem is now fixed, and the API correctly returns a 4xx status:

```bash
curl -v \
  -X POST \
  --unix-socket /var/run/docker.sock \
  "http://localhost:2375/v1.40/images/create?fromImage=hello-world&platform=foobar&tag=latest" \
  -H "Content-Type: application/json"
```

```
* Connected to localhost (/var/run/docker.sock) port 80 (#0)
> POST /v1.40/images/create?fromImage=hello-world&platform=foobar&tag=latest HTTP/1.1
> Host: localhost:2375
> User-Agent: curl/7.52.1
> Accept: */*
> Content-Type: application/json
>
< HTTP/1.1 400 Bad Request
< Api-Version: 1.41
< Content-Type: application/json
< Docker-Experimental: true
< Ostype: linux
< Server: Docker/dev (linux)
< Date: Mon, 15 Jul 2019 15:13:42 GMT
< Content-Length: 85
<
{"message":"\"foobar\": unknown operating system or architecture: invalid argument"}
* Curl_http_done: called premature == 0
```

This patch adds tests to validate the behaviour

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-07-15 20:37:00 +02:00
Sebastiaan van Stijn
6359da4afa
Merge pull request #39431 from thaJeztah/macvlan_redundant_checks
Integration: remove redundant kernel version check for MACVlan
2019-07-15 11:33:15 +02:00
Sebastiaan van Stijn
5ce4394486
Merge pull request #37836 from selansen/test_initpool
TestServiceWithDefaultAddressPoolInit
2019-07-15 11:25:01 +02:00
selansen
88578aa9e9
TestServiceWithDefaultAddressPoolInit
Looks like TestServiceWithDefaultAddressPoolInit is failing
randomly in CI. I am not able to reproduce the issue locally
but this has been reported few times.  So I tried to modify
code and see if I can fix the random failure.

Signed-off-by: selansen <elango.siva@docker.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-07-14 20:02:46 +02:00
Brian Goff
f8aef6a92f
Move kill health test to integration
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2019-07-14 11:53:21 +02:00
Sebastiaan van Stijn
316e16618f
Integration: MACVlan add missing import comment and build-tag
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-07-13 13:35:21 +02:00
Sebastiaan van Stijn
691eb14256
Integration: remove redundant kernel version check for MACVlan
The daemon requires kernel 3.10 or up to start, so there's no need
to check if the daemon is kernel 3.8 or up.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-07-13 13:35:18 +02:00
Sebastiaan van Stijn
4060a7026c
Integration: exclude IPVlan test-suite on Windows
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-07-11 10:39:04 +02:00
Sebastiaan van Stijn
93b28677bf
Integration: IPVlan add missing import comment
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-07-11 10:09:52 +02:00
Sebastiaan van Stijn
dae9bac675
Integration: remove "experimental" option for IPVLAN test-daemons
IPVLAN no longer is experimental since 3ab093d567,
so there's no need to set this option.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-07-11 10:04:04 +02:00
Sebastiaan van Stijn
1e4bd2623a
Integration: remove unneeded platform check for IPVLAN tests
These tests require a local daemon, and are not built on Windows

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-07-11 10:04:00 +02:00
Michael Crosby
fb459f6671
Merge pull request #38441 from sirlatrom/swarm_plugin_env
Allow specifying environment variables when installing an engine plugin as a Swarm service
2019-07-08 15:26:55 -04:00
Michael Crosby
b94218560e Skip TestServiceRemoveKeepsIngressNetwork
Ref: #39426

This is a common flaky test that I have seen on multiple PRs.  It is not
consistent and should be skipped until it is fixed to be robust.  A
simple fix for the swarm tests is not easy as they all poll and have 1
billion timeouts in all the tests so a skip is valid here.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-07-02 16:33:25 -04:00
Kirill Kolyshkin
67be90dcec
Merge pull request #39358 from jim-docker/testForIpvlan
For ipvlan tests, check that the ipvlan module is there, rather than just testing the kernel version.
2019-06-24 15:33:19 -07:00
Tibor Vass
f923321aae Add test for keeping same daemon ID on upgrade
Signed-off-by: Tibor Vass <tibor@docker.com>
2019-06-18 01:40:25 +00:00
Tibor Vass
f23a51a860 Revert "Remove Schema1 integration test suite"
This reverts commit 13b7d11be1.

Signed-off-by: Tibor Vass <tibor@docker.com>
2019-06-18 00:23:04 +00:00
Kir Kolyshkin
a77e147d32 For ipvlan tests check that the ipvlan module is enabled (instead of just ensuring the kernel version is greater than 4.2)
Co-Authored-By: Jim Ehrismann <jim-docker@users.noreply.github.com>
Co-Authored-By: Sebastiaan van Stijn <thaJeztah@users.noreply.github.com>
Signed-off-by: Jim Ehrismann <jim.ehrismann@docker.com>
2019-06-14 10:49:25 -04:00
Tibor Vass
02f1eb89a4 add more tests
Signed-off-by: Tibor Vass <tibor@docker.com>
2019-06-13 06:31:05 +00:00
Brian Goff
6db9f1c3d6 Add test for copying entire container rootfs
CID=$(docker create alpine)
docker cp $CID:/ out

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Signed-off-by: Tibor Vass <tibor@docker.com>
2019-06-12 22:17:18 +00:00
Tibor Vass
a281289515 integration: get tests to compile again
Signed-off-by: Tibor Vass <tibor@docker.com>
2019-06-12 20:41:36 +00:00
Sebastiaan van Stijn
04ff4a2ba4
Merge pull request #39137 from arkodg/attach-to-existing-network-error
Handle the error case when a container reattaches to the same network
2019-06-12 19:58:04 +02:00
Brian Goff
02dceeaa7b
Merge pull request #39240 from olljanat/enable-new-integration-tests-for-win
Enable integrations API tests for Windows CI
2019-06-11 14:24:03 -07:00
Sebastiaan van Stijn
e511b3be89
Merge pull request #39336 from justincormack/entropy-cannot-be-saved
Entropy cannot be saved
2019-06-11 18:40:19 +02:00
Olli Janatuinen
2f22247cad Enable integrations API tests for Windows CI
Signed-off-by: Olli Janatuinen <olli.janatuinen@gmail.com>
2019-06-10 21:32:36 +03:00
Yong Tang
b10cd5c0e6
Merge pull request #39332 from thaJeztah/fix_golint_again
Integration: change signatures to fix golint warnings
2019-06-07 06:46:58 -07:00
Sebastiaan van Stijn
66f8f2b87c
Merge pull request #39193 from olljanat/38488-layer-garbage-collector
Added garbage collector for image layers
2019-06-07 14:08:04 +02:00
Sebastiaan van Stijn
dac5710b68
integration: change testGraphDriver signature to fix linting
Line 441: warning: context.Context should be the first parameter of a function (golint)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-06-07 13:05:51 +02:00
Sebastiaan van Stijn
123e29f44a
integration: change createAmbiguousNetworks signature to fix linting
Line 30: warning: context.Context should be the first parameter of a function (golint)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-06-07 13:05:43 +02:00
Sebastiaan van Stijn
9f9b4290b9
integration: change container.Run signature to fix linting
Line 59: warning: context.Context should be the first parameter of a function (golint)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-06-07 13:05:26 +02:00
Sebastiaan van Stijn
b4c46b0dac
integration: change container.Create signature to fix linting
```
Line 25: warning: context.Context should be the first parameter of a function (golint)
Line 44: warning: context.Context should be the first parameter of a function (golint)
Line 52: warning: context.Context should be the first parameter of a function (golint)
```

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-06-07 13:04:44 +02:00
Sebastiaan van Stijn
caec45a37f
integration: change network.CreateNoError signature to fix linting
Line 30: warning: context.Context should be the first parameter of a function (golint)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-06-07 13:03:24 +02:00
Justin Cormack
2df693e533
Entropy cannot be saved
Remove non cryptographic randomness.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2019-06-07 11:54:45 +01:00
Kirill Kolyshkin
1d5748d975
Merge pull request #39173 from olljanat/25885-capabilities-swarm
Add support for capabilities options in services
2019-06-06 15:03:46 -07:00
Olli Janatuinen
213681b66a First step to implement full garbage collector for image layers
Refactored exiting logic on way that layers are first marked to be under
removal so if actual removal fails they can be found from disk and
cleaned up.

Full garbage collector will be implemented as part of containerd
migration.

Signed-off-by: Olli Janatuinen <olli.janatuinen@gmail.com>
2019-06-05 22:02:42 +03:00
Tibor Vass
54043d2f6f
Merge pull request #39281 from dperny/vendor-swarmkit
Revert docker/swarmkit#2804
2019-06-04 10:28:42 -07:00
Drew Erny
c7d9599e3d Revert docker/swarmkit#2804
Reverts the change to swarmkit that made all updates set UpdateStatus to
Completed

Signed-off-by: Drew Erny <drew.erny@docker.com>
2019-05-29 12:54:39 -05:00
Olli Janatuinen
f787b235de Add support capabilities list on services
Signed-off-by: Olli Janatuinen <olli.janatuinen@gmail.com>
2019-05-28 19:52:36 +03:00
Olli Janatuinen
a3fcd4b82a Add alias for hostname if hostname != container
name which happens if user manually specify hostname

Signed-off-by: Olli Janatuinen <olli.janatuinen@gmail.com>
2019-05-22 20:20:43 +03:00
Yong Tang
3042254a87
Merge pull request #38377 from rgulewich/38332-cgroup-ns
Start containers in their own cgroup namespaces
2019-05-11 20:18:31 -07:00
Rob Gulewich
072400fc4b Make cgroup namespaces configurable
This adds both a daemon-wide flag and a container creation property:
- Set the `CgroupnsMode: "host|private"` HostConfig property at
  container creation time to control what cgroup namespace the container
  is created in
- Set the `--default-cgroupns-mode=host|private` daemon flag to control
  what cgroup namespace containers are created in by default
- Set the default if the daemon flag is unset to "host", for backward
  compatibility
- Default to CgroupnsMode: "host" for client versions < 1.40

Signed-off-by: Rob Gulewich <rgulewich@netflix.com>
2019-05-07 10:22:16 -07:00
Rob Gulewich
256eb04d69 Start containers in their own cgroup namespaces
This is enabled for all containers that are not run with --privileged,
if the kernel supports it.

Fixes #38332

Signed-off-by: Rob Gulewich <rgulewich@netflix.com>
2019-05-07 10:22:16 -07:00
Arko Dasgupta
680d0ba4ab Remove a network during task SHUTDOWN instead of REMOVE to
make sure the LB sandbox is removed when a service is updated
with a --network-rm option

Signed-off-by: Arko Dasgupta <arko.dasgupta@docker.com>
2019-05-06 20:26:59 -07:00
Arko Dasgupta
871acb1c86 Gracefully take care of the error case when a container
retries to attach to a network, it is already connected to

Fixes - https://github.com/docker/for-linux/issues/632

Signed-off-by: Arko Dasgupta <arko.dasgupta@docker.com>
2019-04-26 15:58:58 -07:00
Kir Kolyshkin
1ada1c8391 TestIpcModeOlderClient: skip if client < 1.40
This test case requires not just daemon >= 1.40, but also
client API >= 1.40. In case older client is used, we'll
get failure from the very first check:

> ipcmode_linux_test.go:313: assertion failed: shareable (string) != private (string)

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2019-04-25 19:19:44 -07:00
Sebastiaan van Stijn
83ac2b4c13
Skip TestImagesFilterMultiReference on API < v1.40
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-04-24 17:52:21 -07:00