Commit graph

70 commits

Author SHA1 Message Date
Sebastiaan van Stijn
75fd873fe6
Merge pull request #47009 from cpuguy83/swarm_rotate_key_flake
De-flake TestSwarmClusterRotateUnlockKey... again... maybe?
2024-01-19 23:13:34 +01:00
Akihiro Suda
ae9fbcb82c
testutil/daemon: shorten XDG_RUNTIME_DIR
XDG_RUNTIME_DIR will contain sockets so its path mustn't be too long.
Prior to this commit, it was set to very long path like
`/go/src/github.com/docker/docker/bundles/test-integration/TestDiskUsage/de4fb36576d7d/xdgrun`

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2024-01-17 22:59:18 +09:00
Brian Goff
fbdc02534a De-flake TestSwarmClusterRotateUnlockKey... again... maybe?
This hopefully makes the test less flakey (or removes any flake that
would be caused by the test itself).

1. Adds tail of cluster daemon logs when there is a test failure so we
   can more easily see what may be happening
2. Scans the daemon logs to check if the key is rotated before
   restarting the daemon. This is a little hacky but a little better
   than assuming it is done after a hard-coded 3 seconds.
3. Cleans up the `node ls` check such that it uses a poll function

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2024-01-04 00:18:58 +00:00
Djordje Lukic
a7acfffea6
test: Add a test for userns remapped daemon commit
The files in an image created by a commit should have the right uid and gid set

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-11-08 12:33:03 +01:00
Sebastiaan van Stijn
9670d9364d
api/types: move ContainerListOptions to api/types/container
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-12 11:29:24 +02:00
Brian Goff
e8dc902781 Wire up tests to support otel tracing
Integration tests will now configure clients to propagate traces as well
as create spans for all tests.

Some extra changes were needed (or desired for trace propagation) in the
test helpers to pass through tracing spans via context.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2023-09-07 18:38:22 +00:00
Sebastiaan van Stijn
0f871f8cb7
api/types/events: define "Action" type and consts
Define consts for the Actions we use for events, instead of "ad-hoc" strings.
Having these consts makes it easier to find where specific events are triggered,
makes the events less error-prone, and allows documenting each Action (if needed).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-29 00:38:08 +02:00
Sebastiaan van Stijn
263c00f27c
testutil: format code with gofumpt
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-24 17:57:54 +02:00
Brian Goff
1a51898d2e
TestDaemonProxy: use new scanners to check logs
Also fixes up some cleanup issues.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-01 00:13:50 +02:00
Brian Goff
476e788090
Fix daemon proxy test for "reload sanitized"
I noticed this was always being skipped because of race conditions
checking the logs.

This change adds a log scanner which will look through the logs line by
line rather than allocating a big buffer.
Additionally it adds a `poll.Check` which we can use to actually wait
for the desired log entry.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-01 00:11:28 +02:00
Sebastiaan van Stijn
462d6ef826
Merge pull request #45901 from thaJeztah/api_move_system_info
api/types: move system info types to api/types/system
2023-07-10 17:45:13 +02:00
Sebastiaan van Stijn
c90229ed9a
api/types: move system info types to api/types/system
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-07 13:01:36 +02:00
Brian Goff
914888cf8b Improve test daemon logging
1. On failed start tail the daemon logs
2. Exposes generic tailing functions to make test debugging simpler

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2023-07-05 22:23:25 +00:00
Sebastiaan van Stijn
a31411c679
testutil: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:05:27 +02:00
Cory Snider
165dfd6c3e daemon: fix restoring container with missing task
Before 4bafaa00aa, if the daemon was
killed while a container was running and the container shim is killed
before the daemon is restarted, such as if the host system is
hard-rebooted, the daemon would restore the container to the stopped
state and set the exit code to 255. The aforementioned commit introduced
a regression where the container's exit code would instead be set to 0.
Fix the regression so that the exit code is once against set to 255 on
restore.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-06-23 11:28:45 -04:00
Sebastiaan van Stijn
ab35df454d
remove pre-go1.17 build-tags
Removed pre-go1.17 build-tags with go fix;

    go mod init
    go fix -mod=readonly ./...
    rm go.mod

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-19 20:38:51 +02:00
Sebastiaan van Stijn
8d76acfe6c
testutil: replace uses of client.IsErrNotFound
The client no longer returns the old error-type, so we can use errdefs
instead.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-10 22:13:50 +02:00
Sebastiaan van Stijn
c0c4a16053
testutil: inline filters in tests
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-25 15:16:39 +02:00
Paweł Gronowski
88992de283
StartWithLogFile: Fix d.cmd race
Use `exec.Command` created by this function instead of obtaining it from
daemon struct. This prevents a race condition where `daemon.Kill` is
called before the goroutine has the chance to call `cmd.Wait`.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-03-20 16:16:44 +01:00
Sebastiaan van Stijn
402e0b58ca
testutil: use filepath.WalkDir instead of filepath.Walk
WalkDir is more performant as it doesn't perform an os.Lstat on every visited
file or directory.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-09 17:26:31 +02:00
Sebastiaan van Stijn
e9f1b83a4a
testutil: fix empty-lines (revive)
testutil/fixtures/load/frozen.go:141:99: empty-lines: extra empty line at the end of a block (revive)
    testutil/daemon/plugin.go:56:129: empty-lines: extra empty line at the end of a block (revive)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-28 01:58:50 +02:00
Cory Snider
547da0d575 daemon: support other containerd runtimes (MVP)
Contrary to popular belief, the OCI Runtime specification does not
specify the command-line API for runtimes. Looking at containerd's
architecture from the lens of the OCI Runtime spec, the _shim_ is the
OCI Runtime and runC is "just" an implementation detail of the
io.containerd.runc.v2 runtime. When one configures a non-default runtime
in Docker, what they're really doing is instructing Docker to create
containers using the io.containerd.runc.v2 runtime with a configuration
option telling the runtime that the runC binary is at some non-default
path. Consequently, only OCI runtimes which are compatible with the
io.containerd.runc.v2 shim, such as crun, can be used in this manner.
Other OCI runtimes, including kata-containers v2, come with their own
containerd shim and are not compatible with io.containerd.runc.v2.
As Docker has not historically provided a way to select a non-default
runtime which requires its own shim, runtimes such as kata-containers v2
could not be used with Docker.

Allow other containerd shims to be used with Docker; no daemon
configuration required. If the daemon is instructed to create a
container with a runtime name which does not match any of the configured
or stock runtimes, it passes the name along to containerd verbatim. A
user can start a container with the kata-containers runtime, for
example, simply by calling

    docker run --runtime io.containerd.kata.v2

Runtime names which containerd would interpret as a path to an arbitrary
binary are disallowed. While handy for development and testing it is not
strictly necessary and would allow anyone with Engine API access to
trivially execute any binary on the host as root, so we have decided it
would be safest for our users if it was not allowed.

It is not yet possible to set an alternative containerd shim as the
default runtime; it can only be configured per-container.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-07-27 14:22:49 -04:00
Sebastiaan van Stijn
7e06add3c8
testutil/daemon: remove redundant fmt.Sprintf()'s
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-02-15 12:49:35 +01:00
Sebastiaan van Stijn
1a15a1a061
testutil: daemon.Cleanup(): cleanup more directories
The storage-driver directory caused Jenkins cleanup to fail. While at it, also
removing other directories that we do not include in the "bundles" that are
stored as Jenkins artifacts.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-11-10 12:09:58 +01:00
Sebastiaan van Stijn
e7583ab859
testutil/daemon: ReadLogFile() trigger os.Sync() before reading
Make sure it's written to disk before we try reading the logs.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-10-27 12:39:06 +02:00
Artem Khramov
8f1b2a0fd3
Port testutil/daemon to FreeBSD
testutil/daemon uses a generic unix implementation that assumes that
the host OS supports cgroups & network namespaces, which is not the
case for FreeBSD.

This change adds a FreeBSD-specific implementation for
`testutil/daemon`, namely for `cleanupNetworkNamespace` and
`CgroupNamespace` functions.

Signed-off-by: Artem Khramov <akhramov@pm.me>
2021-10-16 09:51:55 +03:00
Eng Zer Jun
c55a4ac779
refactor: move from io/ioutil to io and os package
The io/ioutil package has been deprecated in Go 1.16. This commit
replaces the existing io/ioutil functions with their new definitions in
io and os packages.

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2021-08-27 14:56:57 +08:00
Sebastiaan van Stijn
686be57d0a
Update to Go 1.17.0, and gofmt with Go 1.17
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-08-24 23:33:27 +02:00
Sebastiaan van Stijn
b90dd66bdd
testutil: don't import "opts" package
It was only used for two consts, which are unlikely to change, but the
"opts" package currently also imports libnetwork/ipamutils, which has
an `init()` function that does some heavy lifting, and not needed for
this utility's purpose.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-08-18 14:28:00 +02:00
Rich Horwood
8f80e55111 Add configuration validation option and tests.
Fixes #36911

If config file is invalid we'll exit anyhow, so this just prevents
the daemon from starting if the configuration is fine.

Mainly useful for making config changes and restarting the daemon
iff the config is valid.

Signed-off-by: Rich Horwood <rjhorwood@apple.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Anca Iordache <anca.iordache@docker.com>
2021-06-23 09:54:55 +00:00
Brian Goff
4a175fd050 Cleanup container shutdown check and add test
Adds a test case for the case where dockerd gets stuck on startup due to
hanging `daemon.shutdownContainer`

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2020-12-23 16:59:03 +00:00
Sebastiaan van Stijn
6d243cdf27
testutil: remove remaining use of deprecated pkg/mount
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-11-10 14:46:59 +01:00
Sebastiaan van Stijn
182795cff6
Do not call mount.RecursiveUnmount() on Windows
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-10-29 23:00:16 +01:00
Sebastiaan van Stijn
561b8014c0
testutil: set -500 OOM score for test daemons
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-10-16 18:49:55 +02:00
Sebastiaan van Stijn
0fc914de5c
Merge pull request #40595 from sfzhu93/GL_2_25
[integration] add buffer to prevent goroutine leak
2020-05-08 16:00:52 +02:00
Akihiro Suda
5ca47f5179 rootless: graduate from experimental
Close #40484

Note that the support for cgroup v2 isn't ready for production yet,
regardless to rootful or rootless.

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-04-07 00:59:15 +09:00
Sebastiaan van Stijn
af0415257e
Merge pull request #40694 from kolyshkin/moby-sys-mount-part-II
switch to moby/sys/{mount,mountinfo} part II
2020-04-02 21:52:21 +02:00
Kir Kolyshkin
39048cf656 Really switch to moby/sys/mount*
Switch to moby/sys/mount and mountinfo. Keep the pkg/mount for potential
outside users.

This commit was generated by the following bash script:

```
set -e -u -o pipefail

for file in $(git grep -l 'docker/docker/pkg/mount"' | grep -v ^pkg/mount); do
	sed -i -e 's#/docker/docker/pkg/mount"#/moby/sys/mount"#' \
		-e 's#mount\.\(GetMounts\|Mounted\|Info\|[A-Za-z]*Filter\)#mountinfo.\1#g' \
		$file
	goimports -w $file
done
```

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2020-03-20 09:46:25 -07:00
Akihiro Suda
5e1b246b9a test-integration: support more rootless tests
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-03-16 17:37:38 +09:00
Akihiro Suda
5bf33adba2 hack: support $DOCKER_ROOTLESS for testing rootless
```
$ DOCKER_EXPERIMENTAL=1 DOCKER_ROOTLESS=1 TEST_SKIP_INTEGRATION_CLI=1 \
 make test-integration
```

test-integration-cli is unsupported currently.
Also, tests that spawn custom daemon (testutil/daemon) are skipped.

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-03-10 05:49:45 +09:00
Ziheng Liu
c322af8019 test: add buffer to prevent goroutine leak
Signed-off-by: Ziheng Liu <lzhfromustc@gmail.com>
2020-02-27 21:37:26 -05:00
Sebastiaan van Stijn
9f0b3f5609
bump gotest.tools v3.0.1 for compatibility with Go 1.14
full diff: https://github.com/gotestyourself/gotest.tools/compare/v2.3.0...v3.0.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-02-11 00:06:42 +01:00
Sam Whited
0c9b232bf5 Remove unused GlobalFlags
Signed-off-by: Sam Whited <sam@samwhited.com>
2019-11-06 18:30:20 -05:00
Sam Whited
b96a0c7754 Add daemon options required by buildkit tests
Signed-off-by: Sam Whited <sam@samwhited.com>
2019-11-06 17:13:24 -05:00
Tibor Vass
2cf86a03a9
Merge pull request #40112 from thaJeztah/cut_the_noise
integration-cli: don't log "Creating new daemon" logs, and print correct line numbers
2019-10-24 11:24:24 -07:00
Sebastiaan van Stijn
56230f4d37
testutil: no more "Creating a new daemon at"
Instead of logging on the "happy path", add more details when
we fail to create a daemon. Now that we base the path of the
daemon on the test-name, it should still be easy to find.

Before:

    make TEST_FILTER=TestSwarmNetworkCreateIssue27866 test-integration
    ...
    === RUN   TestDockerSwarmSuite
    === RUN   TestDockerSwarmSuite/TestSwarmNetworkCreateIssue27866
    --- PASS: TestDockerSwarmSuite (7.47s)
        --- PASS: TestDockerSwarmSuite/TestSwarmNetworkCreateIssue27866 (7.47s)
            docker_cli_swarm_test.go:1499: Creating a new daemon at: "/go/src/github.com/docker/docker/bundles/test-integration/TestDockerSwarmSuite/TestSwarmNetworkCreateIssue27866"

After:

    make TEST_FILTER=TestSwarmNetworkCreateIssue27866 test-integration
    ...
    === RUN   TestDockerSwarmSuite
    === RUN   TestDockerSwarmSuite/TestSwarmNetworkCreateIssue27866
    --- PASS: TestDockerSwarmSuite (8.67s)
        --- PASS: TestDockerSwarmSuite/TestSwarmNetworkCreateIssue27866 (8.67s)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-21 12:51:27 +02:00
Sebastiaan van Stijn
bbf82b2e53
testutil: daemon.Info() close client after request
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-18 18:09:41 +02:00
Justin Cormack
f681590a25
Merge pull request #40081 from thaJeztah/http_constants
Use http constants for HTTP methods and status codes
2019-10-17 11:30:26 -07:00
Sam Whited
95ea6e7069 Export the LogT type and use it in the options
Signed-off-by: Sam Whited <sam@samwhited.com>
2019-10-16 16:27:37 -04:00
Sebastiaan van Stijn
a617809fe7
testutil: use constants for http methods
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-13 17:30:17 +02:00