Commit graph

44213 commits

Author SHA1 Message Date
Sebastiaan van Stijn
90fce781d9
Merge pull request #43732 from thaJeztah/daemon_fix_hosts_validation_step1c1
daemon: refactor config loading
2022-06-30 01:15:30 +02:00
Sebastiaan van Stijn
6cd0171868
Merge pull request #43744 from crazy-max/fix-copy-link
build: force unlazy of refs by calling extract
2022-06-29 20:29:50 +02:00
Sebastiaan van Stijn
1f187e640c
daemon/config: use more assertions in tests
Removes some custom handling, some of which were giving the wrong
error on failure ("expected no error" when we were checking for an
error).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-06-29 19:59:23 +02:00
Sebastiaan van Stijn
10e42f599a
daemon/config: TestUnixValidateConfigurationErrors: use subtests
Use sub-tests and make sure we get the expected error

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-06-29 19:59:21 +02:00
Sebastiaan van Stijn
751222d907
daemon/config: verify that flags were set correctly in tests
To prevent (e.g.) introducing a typo in the flag-name and invalidating
the tests because of that.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-06-29 19:59:20 +02:00
Sebastiaan van Stijn
f73aadb230
daemon/config: New(): set more defaults
Set the defaults when constructing the config, instead of setting them
indirectly through the command-line flags.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-06-29 19:59:18 +02:00
Sebastiaan van Stijn
a0d0db126c
daemon/config: set default MTU when initializing config
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-06-29 19:59:16 +02:00
Sebastiaan van Stijn
62f71c4505
daemon/config: fix TestDaemonConfigurationMerge
This test was validating that the config file would not overwrite the
log-opt, but the test did not set up the flags correctly; as the flags
were not marked as "changed", it would not detect a conflict between
the config-file and daemon-flags.

This patch:

- removes the incorrect fields from the JSON file
- initializes the Config using config.New(), so that any defaults are also set
- sets flag values by actually setting them through the flags

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-06-29 19:59:14 +02:00
Sebastiaan van Stijn
9b39cab510
daemon/config: improve some tests
- TestReloadWithDuplicateLabels() also check value
- TestReloadDefaultConfigNotExist, TestReloadBadDefaultConfig,
  TestReloadWithConflictingLabels: verify that config is not
  reloaded.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-06-29 19:59:08 +02:00
Sebastiaan van Stijn
f8231c62f4
daemon/config: Validate() also validate default MTU
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-06-29 19:55:08 +02:00
Sebastiaan van Stijn
fce7ebdaa5
cmd/dockerd: loadDaemonCliConfig() safeguard for unparsed flags
This function depends on flags having been parsed before it's used;
add a safety-net in case this function would be called before that.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-06-29 19:55:06 +02:00
Sebastiaan van Stijn
0281f50efa
cmd/dockerd: move setting defaults to loadDaemonCliConfig()
Move changes to follow, but this moves the code to a more logical place.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-06-29 19:55:05 +02:00
Sebastiaan van Stijn
717a503590
api: add Hosts to API Config
This makes the API configuration more self-contained.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-06-29 19:55:03 +02:00
Sebastiaan van Stijn
fee8a6a5c4
cmd/dockerd: make newAPIServerConfig() more idiomatic
Construct the TLSConfig if needed, before constructing and returning the whole config.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-06-29 19:55:01 +02:00
Sebastiaan van Stijn
b053376741
cmd/dockerd: reserve port before creating sockets/listeners
This prevents creating a socket and touching the filesystem before
trying to use a port that was already in use by a container.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-06-29 19:55:00 +02:00
Sebastiaan van Stijn
986725b466
daemon: improve some errors
use pkg/errors for all errors in this file, and wrap some errors
to provide context.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-06-29 19:54:57 +02:00
Akihiro Suda
8a147a8c80
Merge pull request #43745 from thaJeztah/bump_moby_sys
vendor: github.com/moby/sys/mountinfo v0.6.2, mount v0.3.3
2022-06-29 13:43:50 +09:00
Tianon Gravi
b8b1e16fed
Merge pull request #43748 from thaJeztah/fix_typo
fix typo (cluser -> cluster)
2022-06-27 16:10:28 -07:00
Tianon Gravi
cf8b057100
Merge pull request #43747 from thaJeztah/fix_container_wait
api: POST /containers/{id}/wait: fix validation for "condition" parameter
2022-06-27 09:15:57 -07:00
Sebastiaan van Stijn
db977355b0
fix typo (cluser -> cluster)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-06-27 15:12:14 +02:00
Sebastiaan van Stijn
0bce64f08d
api: POST /containers/{id}/wait: fix validation for "condition" parameter
commit 737e8c6ab8 added validation for the wait
condition parameter, however, the default ("not-running") option was not part
of the list of valid options, resulting in a regression if the default value
was explicitly passed;

    docker scan --accept-license --version
    Error response from daemon: invalid condition: "not-running"

This patch adds the missing option, and adds a test to verify.

With this patch;

    make BIND_DIR=. DOCKER_GRAPHDRIVER=vfs TEST_FILTER=TestWaitConditions test-integration
    ...
    --- PASS: TestWaitConditions (0.04s)
    --- PASS: TestWaitConditions/removed (1.79s)
    --- PASS: TestWaitConditions/default (1.91s)
    --- PASS: TestWaitConditions/next-exit (1.97s)
    --- PASS: TestWaitConditions/not-running (1.99s)
    PASS

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-06-27 14:45:32 +02:00
Sebastiaan van Stijn
cf543a0bd5
vendor: github.com/moby/sys/mount v0.3.3
full diff: https://github.com/moby/sys/compare/mount/v0.3.1...v0.3.3

- bump x/sys/unix
- ci: add go mod tidy check
- mount: bump mountinfo to v0.6.2

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-06-27 10:27:51 +02:00
Sebastiaan van Stijn
4178caade6
vendor: github.com/moby/sys/mountinfo v0.6.2
full diff: https://github.com/moby/sys/compare/mountinfo/v0.6.0...mountinfo/v0.6.2

- update golang.org/x/sys v0.0.0-20220412211240-33da011f77ad
- mountinfo: BSDs no longer need cgo nor reflect
- mountinfo: update doc to use fs.ErrNotExist
- Bump x/sys/unix

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-06-27 10:26:53 +02:00
Sebastiaan van Stijn
53aefba7f3
vendor: golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a
full diff: 33da011f77...bc2c85ada1

notable changes;

- unix: use ByteSliceFromString in (*Ifreq).Name
- unix: update openbsd Statfs_t fields

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-06-27 10:21:35 +02:00
Sebastiaan van Stijn
4eb1c5bd52
Merge pull request #43622 from vvoland/3554-exec-size
container/exec: Support ConsoleSize
2022-06-24 14:31:39 +02:00
Erik Sipsma
88916949ef
build: force unlazy of refs by calling extract
Signed-off-by: Erik Sipsma <erik@sipsma.dev>
2022-06-24 11:54:41 +02:00
Paweł Gronowski
56a20dbc19 container/exec: Support ConsoleSize
Now client have the possibility to set the console size of the executed
process immediately at the creation. This makes a difference for example
when executing commands that output some kind of text user interface
which is bounded by the console dimensions.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2022-06-24 11:54:25 +02:00
Tianon Gravi
9c4987ee6b
Merge pull request #43731 from thaJeztah/minor_error_and_logging_improvements
minor error and logging improvements
2022-06-23 11:29:52 -07:00
Sebastiaan van Stijn
5f8ff7245c
Merge pull request #43741 from thaJeztah/fresh_curators
Add new people to the curators list
2022-06-23 20:23:13 +02:00
Sebastiaan van Stijn
04c51cba84
Add new people to the curators list
This adds Bjorn, Cory, Nicolas and Djordje to the list of curators
to enable them to help out with triage and other tasks.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-06-23 16:14:04 +02:00
Sebastiaan van Stijn
a347f79711
Merge pull request #43718 from s4ke/feature/bump-netlink#main
Feature/bump netlink#main
2022-06-23 00:03:27 +02:00
Sebastiaan van Stijn
58e1f8d0b5
Merge pull request #43682 from crazy-max/split-test-suites
ci(integration-cli): split test suites in a matrix
2022-06-22 23:22:43 +02:00
Sebastiaan van Stijn
0861539571
Merge pull request #43680 from rumpl/move-image-inspect
Move the inspect code away from the image service
2022-06-22 20:12:15 +02:00
Djordje Lukic
b4ffe3a9fb Move the inspect code away from the image service
The LoopkupImage method is only used by the inspect image route and
returns an api/type struct. The depenency to api/types of the
daemon/images package is wrong, the daemon doesn't need to know about
the api types.

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2022-06-22 15:08:55 +02:00
Sebastiaan van Stijn
c17a566b0d
Merge pull request #43733 from thaJeztah/fix_flaky_TestRunContainerWithRmFlag
fix flaky TestRunContainerWithRmFlag tests (take 2)
2022-06-22 13:16:31 +02:00
Sebastiaan van Stijn
f1c111b176
fix flaky TestRunContainerWithRmFlag tests (take 2)
This is a new attempt on making these tests less flaky. The previous attempt in
commit 585c147b7a assumed that the test was failing
if the test-daemon still had unrelated containers present from other tests, but
it appears that the actual reason for the tests to be flaky may be that the `--rm`
option was moved to the daemon side and an asynchronous operation. As a result,
the container may not yet be removed once the `docker run` completes, which happens
frequently on Windows (likely be- cause removing containers is somewhat slower
on Windows).

This patch adds a retry-loop (using `poll.WaitOn()`) to wait for the container
to be removed.

    make DOCKER_GRAPHDRIVER=vfs TEST_FILTER='TestRunContainerWithRmFlag' test-integration

    INFO: Testing against a local daemon
    === RUN   TestDockerSuite
    === RUN   TestDockerSuite/TestRunContainerWithRmFlagCannotStartContainer
    === RUN   TestDockerSuite/TestRunContainerWithRmFlagExitCodeNotEqualToZero
    --- PASS: TestDockerSuite (1.00s)
    --- PASS: TestDockerSuite/TestRunContainerWithRmFlagCannotStartContainer (0.50s)
    --- PASS: TestDockerSuite/TestRunContainerWithRmFlagExitCodeNotEqualToZero (0.49s)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-06-21 16:24:52 +02:00
Sebastiaan van Stijn
068ab51442
layer: layerstore.Cleanup(): improve some logging
Improve consistency for the logs, and remove a redundant log:

    time="2022-06-07T15:37:24.418470152Z" level=debug msg="found 0 orphan layers"

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-06-21 13:34:03 +02:00
Sebastiaan van Stijn
c2e32c144c
volumes/service: NewStore: add more context to error
Adding some more context to errors to debug a failure in TestDaemonEvents

    === RUN   TestDockerDaemonSuite/TestDaemonEvents
    docker_cli_events_unix_test.go:399: [dd34383dd9b63] failed to start daemon with arguments [--data-root /go/src/github.com/docker/docker/bundles/test-integration/TestDockerDaemonSuite/TestDaemonEvents/dd34383dd9b63/root --exec-root /tmp/dxr/dd34383dd9b63 --pidfile /go/src/github.com/docker/docker/bundles/test-integration/TestDockerDaemonSuite/TestDaemonEvents/dd34383dd9b63/docker.pid --userland-proxy=true --containerd-namespace dd34383dd9b63 --containerd-plugins-namespace dd34383dd9b63p --containerd /var/run/docker/containerd/containerd.sock --host unix:///tmp/docker-integration/dd34383dd9b63.sock --debug --storage-driver overlay2 --config-file=test.json] : [dd34383dd9b63] daemon exited during startup: exit status 1
    check_test.go:307: [dd34383dd9b63] daemon is not started
    --- FAIL: TestDockerDaemonSuite/TestDaemonEvents (1.59s)

daemon logs:

    level=info  msg="Starting up"
    level=debug msg="Listener created for HTTP on unix (/tmp/docker-integration/dd34383dd9b63.sock)"
    level=debug msg="Golang's threads limit set to 55530"
    level=info  msg="[core] original dial target is: \"unix:///var/run/docker/containerd/containerd.sock\"" module=grpc
    level=info  msg="[core] parsed dial target is: {Scheme:unix Authority: Endpoint:var/run/docker/containerd/containerd.sock URL:{Scheme:unix Opaque: User: Host: Path:/var/run/docker/containerd/containerd.sock RawPath: ForceQuery:false RawQuery: Fragment: RawFragment:}}" module=grpc
    level=info  msg="[core] Channel authority set to \"localhost\"" module=grpc
    level=info  msg="[core] ccResolverWrapper: sending update to cc: {[{/var/run/docker/containerd/containerd.sock  0xc000010898 <nil> 0 <nil>}] <nil> <nil>}" module=grpc
    level=info  msg="[core] ClientConn switching balancer to \"pick_first\"" module=grpc
    level=info  msg="[core] Channel switches to new LB policy \"pick_first\"" module=grpc
    level=info  msg="[core] Subchannel Connectivity change to CONNECTING" module=grpc
    level=info  msg="[core] Subchannel picks a new address \"/var/run/docker/containerd/containerd.sock\" to connect" module=grpc
    level=debug msg="metrics API listening on /var/run/docker/metrics.sock"
    level=info  msg="[core] Channel Connectivity change to CONNECTING" module=grpc
    level=info  msg="[core] Subchannel Connectivity change to READY" module=grpc
    level=info  msg="[core] Channel Connectivity change to READY" module=grpc
    level=info  msg="[core] original dial target is: \"unix:///var/run/docker/containerd/containerd.sock\"" module=grpc
    level=info  msg="[core] parsed dial target is: {Scheme:unix Authority: Endpoint:var/run/docker/containerd/containerd.sock URL:{Scheme:unix Opaque: User: Host: Path:/var/run/docker/containerd/containerd.sock RawPath: ForceQuery:false RawQuery: Fragment: RawFragment:}}" module=grpc
    level=info  msg="[core] Channel authority set to \"localhost\"" module=grpc
    level=info  msg="[core] ccResolverWrapper: sending update to cc: {[{/var/run/docker/containerd/containerd.sock  0xc00045ece8 <nil> 0 <nil>}] <nil> <nil>}" module=grpc
    level=info  msg="[core] ClientConn switching balancer to \"pick_first\"" module=grpc
    level=info  msg="[core] Channel switches to new LB policy \"pick_first\"" module=grpc
    level=info  msg="[core] Subchannel Connectivity change to CONNECTING" module=grpc
    level=info  msg="[core] Subchannel picks a new address \"/var/run/docker/containerd/containerd.sock\" to connect" module=grpc
    level=info  msg="[core] Channel Connectivity change to CONNECTING" module=grpc
    level=info  msg="[core] Subchannel Connectivity change to READY" module=grpc
    level=info  msg="[core] Channel Connectivity change to READY" module=grpc
    level=debug msg="Using default logging driver json-file"
    level=info  msg="[graphdriver] trying configured driver: overlay2"
    level=debug msg="processing event stream" module=libcontainerd namespace=plugins.moby
    level=debug msg="successfully detected metacopy status" storage-driver=overlay2 usingMetacopy=false
    level=debug msg="backingFs=extfs, projectQuotaSupported=false, usingMetacopy=false, indexOff=\"index=off,\", userxattr=\"\"" storage-driver=overlay2
    level=debug msg="Initialized graph driver overlay2"
    level=debug msg="No quota support for local volumes in /var/lib/docker/volumes: Filesystem does not support, or has not enabled quotas"
    level=info  msg="[core] Channel Connectivity change to SHUTDOWN" module=grpc
    level=info  msg="[core] Subchannel Connectivity change to SHUTDOWN" module=grpc
    level=debug msg="Cleaning up old mountid : start."
    failed to start daemon: error while opening volume store metadata database: timeout

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-06-21 13:34:00 +02:00
CrazyMax
15716fc9ae
ci(windows): test report summary output
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-06-17 10:59:05 +02:00
CrazyMax
ca8b659a06
ci(integration-cli): move integration-cli run to make script
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-06-17 10:59:05 +02:00
CrazyMax
1887d85e21
ci(integration-cli): dynamically split tests in matrix
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-06-17 10:59:04 +02:00
CrazyMax
0e6a1b9596
integration-cli: split DockerSuite into subsequent build suites
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-06-17 10:59:04 +02:00
CrazyMax
7ed823ead9
validation: temporarily allows changes in integration-cli
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-06-17 10:58:55 +02:00
Martin Braun
5edfd6d081 bump netlink library
bump netlink to 1.2.1
change usages of netlink handle .Delete() to Close()
remove superfluous replace in vendor.mod
make requires of github.com/Azure/go-ansiterm direct

Signed-off-by: Martin Braun <braun@neuroforge.de>
2022-06-16 22:25:33 +02:00
Sebastiaan van Stijn
74286cba8c
Merge pull request #43669 from vvoland/test-container-logs2
test: Add tests for logging
2022-06-16 13:00:31 +02:00
Sebastiaan van Stijn
799df725b7
Merge pull request #43715 from corhere/install-crun
Dockerfile: add crun to dev image
2022-06-16 12:44:29 +02:00
Cory Snider
cf6058941c Dockerfile: add crun to dev image
Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-06-10 17:25:10 -04:00
Paweł Gronowski
2ec3e14c0f test: Add tests for logging
1. Add integration tests for the ContainerLogs API call
Each test handle a distinct case of ContainerLogs output.
- Muxed stream, when container is started without tty
- Single stream, when container is started with tty

2. Add unit test for LogReader suite that tests concurrent logging
It checks that there are no race conditions when logging concurrently
from multiple goroutines.

Co-authored-by: Cory Snider <csnider@mirantis.com>
Signed-off-by: Cory Snider <csnider@mirantis.com>
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2022-06-10 09:26:17 +02:00
Sebastiaan van Stijn
20d6b5c1bd
Merge pull request #43702 from thaJeztah/daemon_event_simplify
daemon: LogDaemonEventWithAttributes: don't call SystemInfo()
2022-06-08 02:25:23 +02:00
Sebastiaan van Stijn
9959eceb9a
Merge pull request #42626 from mfeit-internet2/small-ipv4-networks
Support small ipv4 networks
2022-06-07 22:15:19 +02:00