Now that we marked these utilities as helpers, it should be
possible to find which test-case failed (if any), and we
can skip logging in the "happy path".
This makes these tests less noisy, which makes it easier
to find actually important information in the output:
--- PASS: TestDockerSuite/TestCpFromCaseC (0.96s)
docker_cli_cp_utils_test.go:244: checking that file "/tmp/test-cp-from-case-c450122079/file2" contains "file2\n"
docker_cli_cp_utils_test.go:192: running `docker cp 962b1f3311e742b0842e13b2ad350214cea25883999fd26e87e8c9ddf40d5eb4:/root/file1 /tmp/test-cp-from-case-c450122079/file2`
docker_cli_cp_utils_test.go:244: checking that file "/tmp/test-cp-from-case-c450122079/file2" contains "file1\n"
Some of these tests should probably be rewritten to use subtests,
but that's something for a follow-up.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- Updated TestInfoSecurityOptions to not rely on CLI output. Note that this
test should be migrated to the integration suite, but that suite does not yet
have checks for "Seccomp" and "AppArmor"
- TestInfoAPIWarnings: don't start with busybox because we're not running containers in this test
- Migrate TestInfoDebug to integration suite
- Migrate TestInsecureRegistries to integration suite (renamed to TestInfoInsecureRegistries)
- Migrate TestRegistryMirrors to integration suite (renamed to TestInfoRegistryMirrors)
- Migrate TestInfoDiscoveryBackend to integration suite
- Migrate TestInfoDiscoveryInvalidAdvertise to integration suite
- Migrate TestInfoDiscoveryAdvertiseInterfaceName to integration suite
- Remove TestInfoFormat, which is testing the CLI functionality, and there is an
existing test in docker/cli (TestFormatInfo) covering this
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This makes the test less noisy, and won't print the `failed: Error ...` messages,
which were confusing.
Also, running as a subtest allows tracking failures individually through the
junit.xml files.
Before:
=== RUN TestDockerSwarmSuite/TestSwarmNetworkCreateDup
--- PASS: TestDockerSwarmSuite/TestSwarmNetworkCreateDup (3.00s)
daemon.go:26: Creating a new daemon at: "/go/src/github.com/docker/docker/bundles/test-integration/TestDockerSwarmSuite/TestSwarmNetworkCreateDup"
docker_cli_swarm_test.go:1527: Creating a network named "network-test-0" with "bridge", then "bridge"
docker_cli_swarm_test.go:1534: As expected, the attempt to network "network-test-0" with "bridge" failed: Error response from daemon: network with name network-test-0 already exists
docker_cli_swarm_test.go:1527: Creating a network named "network-test-0" with "bridge", then "overlay"
docker_cli_swarm_test.go:1534: As expected, the attempt to network "network-test-0" with "overlay" failed: Error response from daemon: network with name network-test-0 already exists
docker_cli_swarm_test.go:1527: Creating a network named "network-test-1" with "overlay", then "bridge"
docker_cli_swarm_test.go:1534: As expected, the attempt to network "network-test-1" with "bridge" failed: Error response from daemon: network with name network-test-1 already exists
docker_cli_swarm_test.go:1527: Creating a network named "network-test-1" with "overlay", then "overlay"
docker_cli_swarm_test.go:1534: As expected, the attempt to network "network-test-1" with "overlay" failed: Error response from daemon: network with name network-test-1 already exists
After:
=== RUN TestDockerSwarmSuite
=== RUN TestDockerSwarmSuite/TestSwarmNetworkCreateDup
=== RUN TestDockerSwarmSuite/TestSwarmNetworkCreateDup/driver_bridge_then_bridge
=== RUN TestDockerSwarmSuite/TestSwarmNetworkCreateDup/driver_bridge_then_overlay
=== RUN TestDockerSwarmSuite/TestSwarmNetworkCreateDup/driver_overlay_then_bridge
=== RUN TestDockerSwarmSuite/TestSwarmNetworkCreateDup/driver_overlay_then_overlay
--- PASS: TestDockerSwarmSuite (8.12s)
--- PASS: TestDockerSwarmSuite/TestSwarmNetworkCreateDup (8.12s)
daemon.go:26: Creating a new daemon at: "/go/src/github.com/docker/docker/bundles/test-integration/TestDockerSwarmSuite/TestSwarmNetworkCreateDup"
--- PASS: TestDockerSwarmSuite/TestSwarmNetworkCreateDup/driver_bridge_then_bridge (0.52s)
--- PASS: TestDockerSwarmSuite/TestSwarmNetworkCreateDup/driver_bridge_then_overlay (0.31s)
--- PASS: TestDockerSwarmSuite/TestSwarmNetworkCreateDup/driver_overlay_then_bridge (0.17s)
--- PASS: TestDockerSwarmSuite/TestSwarmNetworkCreateDup/driver_overlay_then_overlay (0.12s)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Before:
daemon.go:26: Creating a new daemon at: "/go/src/github.com/docker/docker/bundles/test-integration/TestDockerSwarmSuite/TestSwarmNetworkCreateDup"
After:
docker_cli_swarm_test.go:1522: Creating a new daemon at: "/go/src/github.com/docker/docker/bundles/test-integration/TestDockerSwarmSuite/TestSwarmNetworkCreateDup"
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Running these tests with a different version of the CLI caused
some failures because the tests relied on the CLI's output format.
Although these tests should be rewritten to use the API directly,
in the meantime this makes them slightly more reliable.
Signed-off-by: Tibor Vass <tibor@docker.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Make this utility a helper, so that the "skip" message is printing
the location of the test, instead of the location of the helper,
which is what it's printing now:
requirement.go:26: unmatched requirement bridgeNfIptables
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Unfortunately quite some of these tests do output-matching, which
may be CLI dependent; this patch prints the output string, to help
debugging failures that may be related to the output having changed
between CLI versions.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
These were accidentally wrong due to a sloppy copy/paste issue. Interestingly,
CI passed on the PR that added it (6397dd4d31),
possibly because of this issue, it stopped linting?
WARN [runner/golint] Golint: can't lint 4 files: no file name for file &{Doc:<nil> Package:23044677 Name:quota Decls:[0xc02cc3fa40 0xc02cc3fac0 0xc02cc3fb40 0xc02cc3fbc0 0xc02cc62ab0 0xc02cc62c00] Scope:scope 0xc02cc5c340 {
var ErrQuotaNotSupported
type errQuotaNotSupported
}
Imports:[0xc02cc62930] Unresolved:[errdefs nil string] Comments:[0xc02cbc9ae0 0xc02cbc9c60]}
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
On account of being flaky on both RS1 and RS5.
Co-Authored-By: Sebastiaan van Stijn <thaJeztah@users.noreply.github.com>
Signed-off-by: Vikram bir Singh <vikrambir.singh@docker.com>
This test was updated in b79adac339, but is still flaky;
```
20:24:13 FAIL: docker_cli_swarm_test.go:1333: DockerSwarmSuite.TestSwarmClusterRotateUnlockKey
20:24:13
20:24:13 Creating a new daemon at: /go/src/github.com/docker/docker/bundles/test-integration/3/DockerSwarmSuite.TestSwarmClusterRotateUnlockKey
20:24:13 [d6f95e679cb65] waiting for daemon to start
20:24:13 [d6f95e679cb65] waiting for daemon to start
20:24:13 [d6f95e679cb65] daemon started
20:24:13
20:24:13 Creating a new daemon at: /go/src/github.com/docker/docker/bundles/test-integration/3/DockerSwarmSuite.TestSwarmClusterRotateUnlockKey
20:24:13 [d204a02ba4780] waiting for daemon to start
20:24:13 [d204a02ba4780] waiting for daemon to start
20:24:13 [d204a02ba4780] daemon started
20:24:13
20:24:13 [d204a02ba4780] joining swarm manager [d6f95e679cb65]@0.0.0.0:2477, swarm listen addr 0.0.0.0:2478
20:24:13 Creating a new daemon at: /go/src/github.com/docker/docker/bundles/test-integration/3/DockerSwarmSuite.TestSwarmClusterRotateUnlockKey
20:24:13 [d873d6a842829] waiting for daemon to start
20:24:13 [d873d6a842829] waiting for daemon to start
20:24:13 [d873d6a842829] daemon started
20:24:13
20:24:13 [d873d6a842829] joining swarm manager [d6f95e679cb65]@0.0.0.0:2477, swarm listen addr 0.0.0.0:2479
20:24:13 [d204a02ba4780] Stopping daemon
20:24:13 [d204a02ba4780] exiting daemon
20:24:13 [d204a02ba4780] Daemon stopped
20:24:13 [d204a02ba4780] waiting for daemon to start
20:24:13 [d204a02ba4780] waiting for daemon to start
20:24:13 [d204a02ba4780] daemon started
20:24:13
20:24:13 [d873d6a842829] Stopping daemon
20:24:13 [d873d6a842829] exiting daemon
20:24:13 [d873d6a842829] Daemon stopped
20:24:13 [d873d6a842829] waiting for daemon to start
20:24:13 [d873d6a842829] waiting for daemon to start
20:24:13 [d873d6a842829] daemon started
20:24:13
20:24:13 docker_cli_swarm_test.go:1413:
20:24:13 c.Assert(err, checker.IsNil, check.Commentf("%s", outs))
20:24:13 ... value *exec.ExitError = &exec.ExitError{ProcessState:(*os.ProcessState)(0xc000934240), Stderr:[]uint8(nil)} ("exit status 1")
20:24:13 ... Error response from daemon: rpc error: code = Unknown desc = The swarm does not have a leader. It's possible that too few managers are online. Make sure more than half of the managers are online.
20:24:13
20:24:13
20:24:13 [d6f95e679cb65] Stopping daemon
20:24:13 [d6f95e679cb65] exiting daemon
20:24:13 [d6f95e679cb65] Daemon stopped
20:24:13 [d204a02ba4780] Stopping daemon
20:24:13 [d204a02ba4780] exiting daemon
20:24:13 [d204a02ba4780] Daemon stopped
20:24:13 [d873d6a842829] Stopping daemon
20:24:13 [d873d6a842829] exiting daemon
20:24:13 [d873d6a842829] Daemon stopped
```
The interesting bit there is that the retry loop should have a 3 second sleep before retrying,
but looking at the failure above, the test started (and failed) within a second, which means that
a different error / output was returned.
This patch adds some additional debugging to that test to see if we can catch the reason
this test is still flaky.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
```
docker/integration-cli/checker/checker.go
Line 12: warning: exported type Compare should have comment or be unexported (golint)
Line 14: warning: exported function False should have comment or be unexported (golint)
Line 20: warning: exported function True should have comment or be unexported (golint)
Line 26: warning: exported function Equals should have comment or be unexported (golint)
Line 32: warning: exported function Contains should have comment or be unexported (golint)
Line 38: warning: exported function Not should have comment or be unexported (golint)
Line 52: warning: exported function DeepEquals should have comment or be unexported (golint)
Line 58: warning: exported function HasLen should have comment or be unexported (golint)
Line 64: warning: exported function IsNil should have comment or be unexported (golint)
Line 70: warning: exported function GreaterThan should have comment or be unexported (golint)
Line 76: warning: exported function NotNil should have comment or be unexported (golint)
```
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
```
internal/test/environment/environment.go:37:23: `useing` is a misspelling of `using`(misspell)
integration/container/wait_test.go:49:9: `waitres` is a misspelling of `waiters`(misspell)
integration/container/wait_test.go:95:9: `waitres` is a misspelling of `waiters`(misspell)
integration-cli/docker_api_containers_test.go:1042:7: `waitres` is a misspelling of `waiters`(misspell)
```
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Fixes linter warnings like this one:
> distribution/pull_v2.go:229:39: SA1019: os.SEEK_SET is deprecated: Use io.SeekStart, io.SeekCurrent, and io.SeekEnd. (staticcheck)
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
```
integration-cli/docker_cli_attach_test.go:44:: SA2002: the goroutine calls T.Fatal, which must be called in the same goroutine as the test (staticcheck)
```
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
```
integration-cli/docker_cli_daemon_test.go:1753:32: S1025: the argument is already a string, there's no need to use fmt.Sprintf (gosimple)
integration-cli/docker_cli_daemon_test.go:1783:31: S1025: the argument is already a string, there's no need to use fmt.Sprintf (gosimple)
integration-cli/docker_cli_daemon_test.go:1893:92: S1025: the argument is already a string, there's no need to use fmt.Sprintf (gosimple)
integration-cli/docker_cli_external_volume_driver_test.go:444:34: S1025: the argument is already a string, there's no need to use fmt.Sprintf (gosimple)
integration-cli/docker_cli_external_volume_driver_test.go:600:36: S1025: the argument is already a string, there's no need to use fmt.Sprintf (gosimple)
integration-cli/docker_cli_external_volume_driver_test.go:602:36: S1025: the argument is already a string, there's no need to use fmt.Sprintf (gosimple)
integration-cli/docker_cli_external_volume_driver_test.go:610:34: S1025: the argument is already a string, there's no need to use fmt.Sprintf (gosimple)
integration-cli/docker_cli_external_volume_driver_test.go:613:34: S1025: the argument is already a string, there's no need to use fmt.Sprintf (gosimple)
integration-cli/docker_cli_external_volume_driver_test.go:614:36: S1025: the argument is already a string, there's no need to use fmt.Sprintf (gosimple)
integration-cli/docker_cli_external_volume_driver_test.go:617:36: S1025: the argument is already a string, there's no need to use fmt.Sprintf (gosimple)
integration-cli/docker_cli_plugins_test.go:431:39: S1025: the argument is already a string, there's no need to use fmt.Sprintf (gosimple)
integration-cli/docker_cli_swarm_test.go:174:31: S1025: the argument is already a string, there's no need to use fmt.Sprintf (gosimple)
integration-cli/docker_cli_swarm_test.go:1046:31: S1025: the argument is already a string, there's no need to use fmt.Sprintf (gosimple)
integration-cli/docker_cli_swarm_test.go:1071:31: S1025: the argument is already a string, there's no need to use fmt.Sprintf (gosimple)
integration-cli/docker_cli_swarm_test.go:1074:31: S1025: the argument is already a string, there's no need to use fmt.Sprintf (gosimple)
integration-cli/docker_cli_swarm_test.go:1079:31: S1025: the argument is already a string, there's no need to use fmt.Sprintf (gosimple)
integration-cli/docker_cli_swarm_test.go:1087:31: S1025: the argument is already a string, there's no need to use fmt.Sprintf (gosimple)
integration-cli/docker_cli_swarm_test.go:1102:31: S1025: the argument is already a string, there's no need to use fmt.Sprintf (gosimple)
integration-cli/docker_cli_swarm_test.go:1108:31: S1025: the argument is already a string, there's no need to use fmt.Sprintf (gosimple)
integration-cli/docker_cli_swarm_test.go:1128:31: S1025: the argument is already a string, there's no need to use fmt.Sprintf (gosimple)
integration-cli/docker_cli_swarm_test.go:1323:31: S1025: the argument is already a string, there's no need to use fmt.Sprintf (gosimple)
integration-cli/docker_cli_swarm_test.go:1329:32: S1025: the argument is already a string, there's no need to use fmt.Sprintf (gosimple)
integration-cli/docker_cli_swarm_test.go:1388:34: S1025: the argument is already a string, there's no need to use fmt.Sprintf (gosimple)
integration-cli/docker_cli_swarm_test.go:1985:31: S1025: the argument is already a string, there's no need to use fmt.Sprintf (gosimple)
```
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
```
integration-cli/docker_cli_registry_user_agent_test.go:78:2: SA5001: should check returned error before deferring reg.Close() (staticcheck)
integration-cli/docker_cli_v2_only_test.go:30:2: SA5001: should check returned error before deferring reg.Close() (staticcheck)
integration-cli/docker_api_containers_test.go:392:3: SA5001: should check returned error before deferring resp.Body.Close() (staticcheck)
```
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
```
14:26:43 integration-cli/docker_cli_build_test.go:3430:15: S1007: should use raw string (`...`) with regexp.MustCompile to avoid having to escape twice (gosimple)
14:26:43 outRegexp := regexp.MustCompile("^(sha256:|)[a-z0-9]{64}\\n$")
14:26:43 ^
14:26:43 integration-cli/docker_cli_by_digest_test.go:26:20: S1007: should use raw string (`...`) with regexp.MustCompile to avoid having to escape twice (gosimple)
14:26:43 pushDigestRegex = regexp.MustCompile("[\\S]+: digest: ([\\S]+) size: [0-9]+")
14:26:43 ^
14:26:43 integration-cli/docker_cli_by_digest_test.go:27:20: S1007: should use raw string (`...`) with regexp.MustCompile to avoid having to escape twice (gosimple)
14:26:43 digestRegex = regexp.MustCompile("Digest: ([\\S]+)")
```
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Format the source according to latest goimports.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Hyper-V isolated containers do not allow file-operations on a
running container. This test currently uses `docker cp` to verify
that the WORKDIR was automatically created, which cannot be done
while the container is running.
```
FAIL: docker_cli_create_test.go:302: DockerSuite.TestCreateWithWorkdir
assertion failed:
Command: d:\CI-7\CI-f3768a669\binary\docker.exe cp foo:c:\home\foo\bar c:\tmp
ExitCode: 1
Error: exit status 1
Stdout:
Stderr: Error response from daemon: filesystem operations against a running Hyper-V container are not supported
Failures:
ExitCode was 1 expected 0
Expected no error
```
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
After the commit faaffd5d6d ("Windows:Disable 2 restart test when
Hyper-V") some tests became skipped on linux:
SKIP: docker_cli_restart_test.go:167: DockerSuite.TestRestartContainerSuccess (unmatched requirement IsolationIsProcess)
SKIP: docker_cli_restart_test.go:240: DockerSuite.TestRestartPolicyAfterRestart (unmatched requirement IsolationIsProcess)
But AFAIU it is highly unlikely that we actually meant to skip them on linux.
https://github.com/moby/moby/issues/39625
Signed-off-by: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>
`TestBuildBuildTimeArgEnv` and `TestBuildBuildTimeArgEmptyValVariants` were
using non-standard comparisons.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Found these by doing a `grep -R 'using the force'` on a full test run.
There's still a few more which are running against the main test daemon,
so it is difficult to find which test they belong to.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
I've seen this test fail a number of times recently on RS1
Looking at failures, the test is taking a long time ro run (491.77s, which is
more than 8 minutes), so perhaps it's just too slow on RS1, which may be
because we switch to a different base image, or because we're now running
on different machines.
Compared to RS5 (still slow, but a lot faster);
```
--- PASS: Test/DockerSuite/TestAPIImagesSaveAndLoad (146.25s)
```
```
--- FAIL: Test/DockerSuite/TestAPIImagesSaveAndLoad (491.77s)
cli.go:45: assertion failed:
Command: d:\CI-5\CI-93d2cf881\binary\docker.exe inspect --format {{.Id}} sha256:69e7c1ff23be5648c494294a3808c0ea3f78616fad67bfe3b10d3a7e2be5ff02
ExitCode: 1
Error: exit status 1
Stdout:
Stderr: Error: No such object: sha256:69e7c1ff23be5648c494294a3808c0ea3f78616fad67bfe3b10d3a7e2be5ff02
Failures:
ExitCode was 1 expected 0
Expected no error
```
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Replaced using a bit of grep-ing;
```
find . -name "*_test.go" -exec sed -E -i 's#assert.Assert\((.*), fmt.Sprintf\((.*)\)\)$#assert.Assert\(\1, \2\)#g' '{}' \;
```
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Starting the daemon should not load the busybox image again
in most cases, so add a new `StartNodeWithBusybox` function
to be clear that this one loads the busybox image, and use
`StartNode()` for cases where loading the busybox image is
not needed.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
go run rm-gocheck.go redress '[^/]\bcheck\.Suite\(.*\{\s*$' \
"integration-cli/check_test.go" "integration-cli/docker_cli_external_volume_driver_unix_test.go" "integration-cli/docker_cli_network_unix_test.go"
Signed-off-by: Tibor Vass <tibor@docker.com>
