Commit graph

37190 commits

Author SHA1 Message Date
Stefan Scherer
93b38b8008
Fix docker inspect for dutimgVersion
Signed-off-by: Stefan Scherer <stefan.scherer@docker.com>
(cherry picked from commit 52a53e2587)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-25 21:35:58 +02:00
Sebastiaan van Stijn
e8e2666705
integration-cli: getContainerCount() fix trimming prefix
caught by staticcheck:

```
integration-cli/docker_utils_test.go:66:29: SA1024: cutset contains duplicate characters (staticcheck)
```

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 02c9b0674f)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-25 21:35:57 +02:00
Sebastiaan van Stijn
45f49fe5c3
TestDispatch: refactor to use subtests again, and fix linting (structcheck)
Instead of using a `initDispatchTestCases()` function, declare the test-table
inside `TestDispatch` itself, and run the tests as subtests.

```
[2019-08-27T15:14:51.072Z] builder/dockerfile/evaluator_test.go:18:2: `name` is unused (structcheck)
[2019-08-27T15:14:51.072Z] 	name, expectedError string
```

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit a3f9cb5b63)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-25 21:35:56 +02:00
Sebastiaan van Stijn
1d91898ca6
integration: windows.ps1: turn defender error into a warning
Some integration tests are known to fail if Windows Defender is
enabled. On the machines that run our CI, defender is disabled
for that reason.

Contributors likely will have defender enabled, and because of
that are currently not able to run the integration tests.

This patch changes the ERROR into a WARNING, so that contributors
can still run (a limited set of) the integration tests, but get
informed that some may fail.

We should make this requirement more specific, and only skip
tests that are known to require defender to be disabled, but
while that's not yet in place, let's print a warning instead.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 31885181fc)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-25 21:35:56 +02:00
Sebastiaan van Stijn
79e5950b2f
pkg/term: refactor TestEscapeProxyRead
- use subtests to make it clearer what the individual test-cases
  are, and to prevent tests from depending on values set by the
  previous test(s).
- remove redundant messages in assert (gotest.tools already prints
  a useful message if assertions fail).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 556d26c07d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-25 21:35:55 +02:00
Vitaly Ostrosablin
6d7d877c73
Fix testcase name
TestBuildMulitStageResetScratch testcase was actually meant to be
TestBuildMulitStageResetScratch

Signed-off-by: Vitaly Ostrosablin <tmp6154@yandex.ru>
(cherry picked from commit c266d8fe56)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-25 21:35:54 +02:00
Sebastiaan van Stijn
b8b8bcb8bf
Dockerfile: update CRIU to v3.12
New features

- build CRIU with Android NDK
- C/R of
  - IP RAW sockets
  - lsm: dump and restore any SELinux process label
  - support restoring ghost files on readonly mounts

Bugfixes

 - Do not lock network if running in the host network namespace
- Fix RPC configuration file handling
- util: don't leak file descriptors to third-party tools
- small fixes here and there

Improvements

- travis: switch to the Ubuntu Xenial
- travis-ci: Enable ia32 tests
- Many improvements and bug fixes in the libcriu
  - Changes in the API and ABI (SONAME increased from 1 to 2)

full diff: https://github.com/checkpoint-restore/criu/compare/v3.11...v3.12

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 00ad0222ce)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-25 21:35:53 +02:00
Sebastiaan van Stijn
08573e2920
Jenkinsfile: add TESTDEBUG
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit d723643dc3)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-25 21:35:53 +02:00
Sebastiaan van Stijn
5d4f5db76c
integration: improve package- and filename for junit.xml
Generate more unique names, based on architecture and test-suite name.

Clean up the path to this integration test to create a useful package name.
"$dir" can be either absolute (/go/src/github.com/docker/docker/integration/foo)
or relative (./integration/foo). To account for both, first we strip the
absolute path, then any leading periods and slashes.

For the package-name, we use periods as separator instead of slashes, to be more
in-line with Java package names (which is what junit.xml was originally designed
for).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit f007b0150a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-25 21:35:52 +02:00
Sebastiaan van Stijn
48e8f18495
integration: test2json: enable timestamps to fix zero-time test durations
Without these options set, test2json does not include a `Time`
field in the generated JSON;

    {"Action":"run","Test":"TestCgroupNamespacesBuild"}
    {"Action":"output","Test":"TestCgroupNamespacesBuild","Output":"=== RUN   TestCgroupNamespacesBuild\n"}
    {"Action":"output","Test":"TestCgroupNamespacesBuild","Output":"--- PASS: TestCgroupNamespacesBuild (1.70s)\n"}
    ...
    {"Action":"pass","Test":"TestCgroupNamespacesBuild"}

As a result, `gotestsum` was not able to calculate test-duration, and
reported `time="0.000000"` for all tests;

    <testcase classname="amd64.integration.build" name="TestCgroupNamespacesBuild" time="0.000000"></testcase>

With this patch applied:

    {"Time":"2019-08-23T22:42:41.644361357Z","Action":"run","Package":"amd64.integration.build","Test":"TestCgroupNamespacesBuild"}
    {"Time":"2019-08-23T22:42:41.644367647Z","Action":"output","Package":"amd64.integration.build","Test":"TestCgroupNamespacesBuild","Output":"=== RUN   TestCgroupNamespacesBuild\n"}
    {"Time":"2019-08-23T22:42:44.926933252Z","Action":"output","Package":"amd64.integration.build","Test":"TestCgroupNamespacesBuild","Output":"--- PASS: TestCgroupNamespacesBuild (3.28s)\n"}
    ...
    {"Time":"2019-08-23T22:42:44.927003836Z","Action":"pass","Package":"amd64.integration.build","Test":"TestCgroupNamespacesBuild","Elapsed":3.28}

Which now correctly reports the test's duration:

    <testcase classname="amd64.integration.build" name="TestCgroupNamespacesBuild" time="3.280000"></testcase>

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit d2e00d62e2)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-25 21:35:52 +02:00
Sebastiaan van Stijn
517ebe626c
integration: use gotestsum to generate junit.xml and go-test-report.json
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit f3be6b346f)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-25 21:35:51 +02:00
Sebastiaan van Stijn
14d561eb1c
integration: simplify parallel run destination
'Namespace' parallel runs by bind-mounting a different directory
in the container, instead of making the tests running inside
the container aware of the namespaced location.

This makes it transparent to the tests, and slightly reduces
complexity.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 3262a69be6)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-25 21:35:50 +02:00
SataQiu
1da2e90b56
fix some spelling mistakes
Signed-off-by: SataQiu <qiushida@beyondcent.com>
(cherry picked from commit f6226a2a56)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-25 21:35:49 +02:00
Eli Uriegas
316390891c
hack: Remove inContainer check, it wasn't useful
The inContainer check isn't really useful anymore.

Even though it was said that we shouldn't rely on its existence back in
2016, we're now in 2019 and this thing still exists so we should just
rely on it now to check whether or not we're in a container.

Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
(cherry picked from commit f5cd8fdd44)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-25 21:35:48 +02:00
Brian Goff
24395d55fc
Better logging for swarm tests
Call helper for starting swarm agents and add some logging with daemon
id's when joining the swarm.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit b0fe0dff7a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-25 21:35:29 +02:00
Sebastiaan van Stijn
a768bf8673
integration-cli: remove redundant "testrequires"
The `DockerDaemonSuite.SetUpTest` already checks for Linux and a local daemon;

```
func (s *DockerDaemonSuite) SetUpTest(c *check.C) {
	testRequires(c, DaemonIsLinux, testEnv.IsLocalDaemon)
	s.d = daemon.New(c, dockerBinary, dockerdBinary, testdaemon.WithEnvironment(testEnv.Execution))
}
```

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 7f37d99ef5)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-25 17:33:51 +02:00
Drew Erny
2ebfdfd66c
Retry service updates on out of sequence errors
Code retrying service update operations when receiving "update out of
sequence" errors was removed because of a misunderstanding, which has
made tests flaky. This re-adds the "CmdRetryOutOfSequence" method, and
uses it in TestSwarmPublishAdd to avoid flaky behavior.

Signed-off-by: Drew Erny <drew.erny@docker.com>
(cherry picked from commit 1de914695b)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-25 17:33:17 +02:00
Tonis Tiigi
1204f3a77c
integration-cli: increase healthcheck timeout
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit 8c9362857f)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-25 17:32:45 +02:00
Sebastiaan van Stijn
1d795b53d3
integration: run build session tests on non-experimental
The session endpoint is no longer experimental since
01c9e7082e, so we don't
need to start an experimental daemon.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit becd29c665)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-25 17:32:13 +02:00
Deep Debroy
7e76438537
Be more conservative for Windows in TestFrequency for Splunk
Signed-off-by: Deep Debroy <ddebroy@docker.com>
(cherry picked from commit a5c420ac54)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-25 17:31:30 +02:00
Sebastiaan van Stijn
bf212c5b33
DockerSwarmSuite lock portIndex to work around race
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit c096225e8e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-25 17:29:48 +02:00
Brian Goff
eeeb2e941d
Fix Microsecond -> Milisecond.
A bit too quick on the trigger on some text completion I think...

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit 5d818213ff)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-25 17:29:20 +02:00
Brian Goff
05c096a1ac
Don't log test initial test daemon ping failures
This is just noise due to timing. I picked `> 2` just based on
logs from tests I've seen there's always 1 or 2.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit 15675e28f1)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-25 17:28:27 +02:00
Sebastiaan van Stijn
ad8327f2ce
integration: fix cleanup of raft data
The directory used for storage was either changed or new directories
were added.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 6a64a4deec)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-25 17:27:49 +02:00
Brian Goff
34418110ec
Add (hidden) flags to set containerd namespaces
This allows our tests, which all share a containerd instance, to be a
bit more isolated by setting the containerd namespaces to the generated
daemon ID's rather than the default namespaces.

This came about because I found in some cases we had test daemons
failing to start (really very slow to start) because it was (seemingly)
processing events from other tests.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit 24ad2f486d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-25 17:26:26 +02:00
Sebastiaan van Stijn
e286096089
integration-cli: remove unused requirements utils
Removes some test functions that were unused:

- bridgeNfIP6tables
- ambientCapabilities (added to support #26979, which was reverted in #27737)
- overlay2Supported

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit c887b09abc)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-25 17:25:53 +02:00
Sebastiaan van Stijn
a63a02fefd
integration-cli: remove defaultSleepImage constant
Both Linux and Windows now use busybox, so no need to keep a
constant for this.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 27f432ca57)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-25 17:25:20 +02:00
Sebastiaan van Stijn
f76cb3e6d5
integration-cli: remove ExecSupport check
All current versions of Docker support exec, so no need
to check for this.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 7204341950)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-25 17:24:48 +02:00
Sebastiaan van Stijn
edeff03134
Integration: MACVlan add missing import comment and build-tag
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 316e16618f)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-25 17:23:14 +02:00
Sebastiaan van Stijn
8c8de170d2
Integration: remove redundant kernel version check for MACVlan
The daemon requires kernel 3.10 or up to start, so there's no need
to check if the daemon is kernel 3.8 or up.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 691eb14256)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-25 17:23:08 +02:00
Sebastiaan van Stijn
1710bba5c3
Integration: exclude IPVlan test-suite on Windows
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 4060a7026c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-25 17:22:16 +02:00
Sebastiaan van Stijn
0378afaf5f
Integration: IPVlan add missing import comment
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 93b28677bf)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-25 17:22:11 +02:00
Sebastiaan van Stijn
0c8bc0b57a
Integration: remove "experimental" option for IPVLAN test-daemons
IPVLAN no longer is experimental since 3ab093d567,
so there's no need to set this option.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit dae9bac675)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-25 17:22:05 +02:00
Sebastiaan van Stijn
c95330420c
Integration: remove unneeded platform check for IPVLAN tests
These tests require a local daemon, and are not built on Windows

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 1e4bd2623a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-25 17:21:57 +02:00
Andrew Hsu
dda2b4454f
Merge pull request #293 from thaJeztah/19.03_backport_test_restart
[19.03 backport] Improve select for daemon restart tests
2019-09-24 11:40:18 -07:00
Andrew Hsu
0ff52c285d
Merge pull request #289 from thaJeztah/19.03_backport_bump_gorilla_mux
[19.03 backport] bump gorilla/mux v1.7.2
2019-09-24 11:36:24 -07:00
Andrew Hsu
09f8810272
Merge pull request #356 from thaJeztah/19.03_backport_fix_windows_errortype
[19.03 backport] Windows: fix error-type for starting a running container
2019-09-24 11:34:21 -07:00
Andrew Hsu
f4f8feafe7
Merge pull request #361 from thaJeztah/19.03_backport_libcontainerd_events_wait
[19.03 backport] Sleep before restarting event processing
2019-09-24 11:32:35 -07:00
Andrew Hsu
29db7cb98b
Merge pull request #376 from thaJeztah/19.03_backport_fix_flaky_addr_pool_init_test
[19.03 backport] Fix flaky TestServiceWithDefaultAddressPoolInit
2019-09-24 11:27:30 -07:00
Sebastiaan van Stijn
cf298d3073
Merge pull request #378 from kolyshkin/19.03-backport-log-max-file-1-follow
[19.03 backport] logger: fix follow logs for max-file=1
2019-09-24 20:25:22 +02:00
Andrew Hsu
53b9d440b8
Merge pull request #373 from tonistiigi/19.03-buildkit
[19.03] vendor: update buildkit for 19.03
2019-09-23 15:43:25 -07:00
Kir Kolyshkin
1e13f66fbb logger: fix follow logs for max-file=1
In case jsonlogfile is used with max-file=1 and max-size set,
the log rotation is not perfomed; instead, the log file is closed
and re-open with O_TRUNC.

This situation is not handled by the log reader in follow mode,
leading to an issue of log reader being stuck forever.

This situation (file close/reopen) could be handled in waitRead(),
but fsnotify library chose to not listen to or deliver this event
(IN_CLOSE_WRITE in inotify lingo).

So, we have to handle this by checking the file size upon receiving
io.EOF from the log reader, and comparing the size with the one received
earlier. In case the new size is less than the old one, the file was
truncated and we need to seek to its beginning.

Fixes #39235.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 9cd24ba605)
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2019-09-23 11:29:52 -07:00
Andrew Hsu
9a9ff44418
Merge pull request #296 from thaJeztah/19.03_backport_exec_hang
[19.03 backport] Handle blocked I/O of exec'd processes
2019-09-23 10:05:26 -07:00
Arko Dasgupta
218af8c7bd
Move defer method to the top right after New is called
Signed-off-by: Arko Dasgupta <arko.dasgupta@docker.com>
(cherry picked from commit a65dee30fc)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-23 18:40:51 +02:00
Arko Dasgupta
a24fddc2ad
Fix flaky TestServiceWithDefaultAddressPoolInit
1.This commit replaces serviceRunningCount with
swarm.RunningTasksCount to accurately check if the
service is running with the accurate number of instances
or not. serviceRunningCount was only checking the ServiceList
and was not checking if the tasks were running or not

This adds a safe barrier to execute docker network inspect
commands for overlay networks which get created
asynchronously via Swarm

2. Make sure client connections are closed

3. Make sure every service and network name is unique

4. Make sure services and networks are cleaned up

Signed-off-by: Arko Dasgupta <arko.dasgupta@docker.com>
(cherry picked from commit f3a3ea0d3c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-23 18:40:48 +02:00
selansen
6d9666c8a0
TestServiceWithDefaultAddressPoolInit
Looks like TestServiceWithDefaultAddressPoolInit is failing
randomly in CI. I am not able to reproduce the issue locally
but this has been reported few times.  So I tried to modify
code and see if I can fix the random failure.

Signed-off-by: selansen <elango.siva@docker.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 88578aa9e9)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-23 18:40:46 +02:00
Andrew Hsu
c5754a7329
Merge pull request #281 from thaJeztah/19.03_backport_fix_golint_again
[19.03 backport] Integration: change signatures to fix golint warnings
2019-09-23 09:37:43 -07:00
Andrew Hsu
c27f11fa2e
Merge pull request #340 from thaJeztah/19.03_backport_bump_grpc
[19.03 backport] bump google.golang.org/grpc v1.23.0 (CVE-2019-9512, CVE-2019-9514, CVE-2019-9515)
2019-09-23 09:32:43 -07:00
Andrew Hsu
09b72e0be4
Merge pull request #335 from thaJeztah/19.03_backport_dev190815
[19.03 backport] fix docker rmi stucking
2019-09-23 09:30:56 -07:00
Tonis Tiigi
b71e1008a5 vendor: update buildkit for 19.03
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-09-23 09:23:35 -07:00