Commit graph

44235 commits

Author SHA1 Message Date
Sebastiaan van Stijn
04c51cba84
Add new people to the curators list
This adds Bjorn, Cory, Nicolas and Djordje to the list of curators
to enable them to help out with triage and other tasks.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-06-23 16:14:04 +02:00
Sebastiaan van Stijn
a347f79711
Merge pull request #43718 from s4ke/feature/bump-netlink#main
Feature/bump netlink#main
2022-06-23 00:03:27 +02:00
Sebastiaan van Stijn
58e1f8d0b5
Merge pull request #43682 from crazy-max/split-test-suites
ci(integration-cli): split test suites in a matrix
2022-06-22 23:22:43 +02:00
Sebastiaan van Stijn
0861539571
Merge pull request #43680 from rumpl/move-image-inspect
Move the inspect code away from the image service
2022-06-22 20:12:15 +02:00
Djordje Lukic
b4ffe3a9fb Move the inspect code away from the image service
The LoopkupImage method is only used by the inspect image route and
returns an api/type struct. The depenency to api/types of the
daemon/images package is wrong, the daemon doesn't need to know about
the api types.

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2022-06-22 15:08:55 +02:00
Sebastiaan van Stijn
c17a566b0d
Merge pull request #43733 from thaJeztah/fix_flaky_TestRunContainerWithRmFlag
fix flaky TestRunContainerWithRmFlag tests (take 2)
2022-06-22 13:16:31 +02:00
Sebastiaan van Stijn
f1c111b176
fix flaky TestRunContainerWithRmFlag tests (take 2)
This is a new attempt on making these tests less flaky. The previous attempt in
commit 585c147b7a assumed that the test was failing
if the test-daemon still had unrelated containers present from other tests, but
it appears that the actual reason for the tests to be flaky may be that the `--rm`
option was moved to the daemon side and an asynchronous operation. As a result,
the container may not yet be removed once the `docker run` completes, which happens
frequently on Windows (likely be- cause removing containers is somewhat slower
on Windows).

This patch adds a retry-loop (using `poll.WaitOn()`) to wait for the container
to be removed.

    make DOCKER_GRAPHDRIVER=vfs TEST_FILTER='TestRunContainerWithRmFlag' test-integration

    INFO: Testing against a local daemon
    === RUN   TestDockerSuite
    === RUN   TestDockerSuite/TestRunContainerWithRmFlagCannotStartContainer
    === RUN   TestDockerSuite/TestRunContainerWithRmFlagExitCodeNotEqualToZero
    --- PASS: TestDockerSuite (1.00s)
    --- PASS: TestDockerSuite/TestRunContainerWithRmFlagCannotStartContainer (0.50s)
    --- PASS: TestDockerSuite/TestRunContainerWithRmFlagExitCodeNotEqualToZero (0.49s)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-06-21 16:24:52 +02:00
Sebastiaan van Stijn
068ab51442
layer: layerstore.Cleanup(): improve some logging
Improve consistency for the logs, and remove a redundant log:

    time="2022-06-07T15:37:24.418470152Z" level=debug msg="found 0 orphan layers"

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-06-21 13:34:03 +02:00
Sebastiaan van Stijn
c2e32c144c
volumes/service: NewStore: add more context to error
Adding some more context to errors to debug a failure in TestDaemonEvents

    === RUN   TestDockerDaemonSuite/TestDaemonEvents
    docker_cli_events_unix_test.go:399: [dd34383dd9b63] failed to start daemon with arguments [--data-root /go/src/github.com/docker/docker/bundles/test-integration/TestDockerDaemonSuite/TestDaemonEvents/dd34383dd9b63/root --exec-root /tmp/dxr/dd34383dd9b63 --pidfile /go/src/github.com/docker/docker/bundles/test-integration/TestDockerDaemonSuite/TestDaemonEvents/dd34383dd9b63/docker.pid --userland-proxy=true --containerd-namespace dd34383dd9b63 --containerd-plugins-namespace dd34383dd9b63p --containerd /var/run/docker/containerd/containerd.sock --host unix:///tmp/docker-integration/dd34383dd9b63.sock --debug --storage-driver overlay2 --config-file=test.json] : [dd34383dd9b63] daemon exited during startup: exit status 1
    check_test.go:307: [dd34383dd9b63] daemon is not started
    --- FAIL: TestDockerDaemonSuite/TestDaemonEvents (1.59s)

daemon logs:

    level=info  msg="Starting up"
    level=debug msg="Listener created for HTTP on unix (/tmp/docker-integration/dd34383dd9b63.sock)"
    level=debug msg="Golang's threads limit set to 55530"
    level=info  msg="[core] original dial target is: \"unix:///var/run/docker/containerd/containerd.sock\"" module=grpc
    level=info  msg="[core] parsed dial target is: {Scheme:unix Authority: Endpoint:var/run/docker/containerd/containerd.sock URL:{Scheme:unix Opaque: User: Host: Path:/var/run/docker/containerd/containerd.sock RawPath: ForceQuery:false RawQuery: Fragment: RawFragment:}}" module=grpc
    level=info  msg="[core] Channel authority set to \"localhost\"" module=grpc
    level=info  msg="[core] ccResolverWrapper: sending update to cc: {[{/var/run/docker/containerd/containerd.sock  0xc000010898 <nil> 0 <nil>}] <nil> <nil>}" module=grpc
    level=info  msg="[core] ClientConn switching balancer to \"pick_first\"" module=grpc
    level=info  msg="[core] Channel switches to new LB policy \"pick_first\"" module=grpc
    level=info  msg="[core] Subchannel Connectivity change to CONNECTING" module=grpc
    level=info  msg="[core] Subchannel picks a new address \"/var/run/docker/containerd/containerd.sock\" to connect" module=grpc
    level=debug msg="metrics API listening on /var/run/docker/metrics.sock"
    level=info  msg="[core] Channel Connectivity change to CONNECTING" module=grpc
    level=info  msg="[core] Subchannel Connectivity change to READY" module=grpc
    level=info  msg="[core] Channel Connectivity change to READY" module=grpc
    level=info  msg="[core] original dial target is: \"unix:///var/run/docker/containerd/containerd.sock\"" module=grpc
    level=info  msg="[core] parsed dial target is: {Scheme:unix Authority: Endpoint:var/run/docker/containerd/containerd.sock URL:{Scheme:unix Opaque: User: Host: Path:/var/run/docker/containerd/containerd.sock RawPath: ForceQuery:false RawQuery: Fragment: RawFragment:}}" module=grpc
    level=info  msg="[core] Channel authority set to \"localhost\"" module=grpc
    level=info  msg="[core] ccResolverWrapper: sending update to cc: {[{/var/run/docker/containerd/containerd.sock  0xc00045ece8 <nil> 0 <nil>}] <nil> <nil>}" module=grpc
    level=info  msg="[core] ClientConn switching balancer to \"pick_first\"" module=grpc
    level=info  msg="[core] Channel switches to new LB policy \"pick_first\"" module=grpc
    level=info  msg="[core] Subchannel Connectivity change to CONNECTING" module=grpc
    level=info  msg="[core] Subchannel picks a new address \"/var/run/docker/containerd/containerd.sock\" to connect" module=grpc
    level=info  msg="[core] Channel Connectivity change to CONNECTING" module=grpc
    level=info  msg="[core] Subchannel Connectivity change to READY" module=grpc
    level=info  msg="[core] Channel Connectivity change to READY" module=grpc
    level=debug msg="Using default logging driver json-file"
    level=info  msg="[graphdriver] trying configured driver: overlay2"
    level=debug msg="processing event stream" module=libcontainerd namespace=plugins.moby
    level=debug msg="successfully detected metacopy status" storage-driver=overlay2 usingMetacopy=false
    level=debug msg="backingFs=extfs, projectQuotaSupported=false, usingMetacopy=false, indexOff=\"index=off,\", userxattr=\"\"" storage-driver=overlay2
    level=debug msg="Initialized graph driver overlay2"
    level=debug msg="No quota support for local volumes in /var/lib/docker/volumes: Filesystem does not support, or has not enabled quotas"
    level=info  msg="[core] Channel Connectivity change to SHUTDOWN" module=grpc
    level=info  msg="[core] Subchannel Connectivity change to SHUTDOWN" module=grpc
    level=debug msg="Cleaning up old mountid : start."
    failed to start daemon: error while opening volume store metadata database: timeout

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-06-21 13:34:00 +02:00
CrazyMax
15716fc9ae
ci(windows): test report summary output
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-06-17 10:59:05 +02:00
CrazyMax
ca8b659a06
ci(integration-cli): move integration-cli run to make script
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-06-17 10:59:05 +02:00
CrazyMax
1887d85e21
ci(integration-cli): dynamically split tests in matrix
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-06-17 10:59:04 +02:00
CrazyMax
0e6a1b9596
integration-cli: split DockerSuite into subsequent build suites
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-06-17 10:59:04 +02:00
CrazyMax
7ed823ead9
validation: temporarily allows changes in integration-cli
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-06-17 10:58:55 +02:00
Martin Braun
5edfd6d081 bump netlink library
bump netlink to 1.2.1
change usages of netlink handle .Delete() to Close()
remove superfluous replace in vendor.mod
make requires of github.com/Azure/go-ansiterm direct

Signed-off-by: Martin Braun <braun@neuroforge.de>
2022-06-16 22:25:33 +02:00
Sebastiaan van Stijn
74286cba8c
Merge pull request #43669 from vvoland/test-container-logs2
test: Add tests for logging
2022-06-16 13:00:31 +02:00
Sebastiaan van Stijn
799df725b7
Merge pull request #43715 from corhere/install-crun
Dockerfile: add crun to dev image
2022-06-16 12:44:29 +02:00
Cory Snider
cf6058941c Dockerfile: add crun to dev image
Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-06-10 17:25:10 -04:00
Paweł Gronowski
2ec3e14c0f test: Add tests for logging
1. Add integration tests for the ContainerLogs API call
Each test handle a distinct case of ContainerLogs output.
- Muxed stream, when container is started without tty
- Single stream, when container is started with tty

2. Add unit test for LogReader suite that tests concurrent logging
It checks that there are no race conditions when logging concurrently
from multiple goroutines.

Co-authored-by: Cory Snider <csnider@mirantis.com>
Signed-off-by: Cory Snider <csnider@mirantis.com>
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2022-06-10 09:26:17 +02:00
Sebastiaan van Stijn
20d6b5c1bd
Merge pull request #43702 from thaJeztah/daemon_event_simplify
daemon: LogDaemonEventWithAttributes: don't call SystemInfo()
2022-06-08 02:25:23 +02:00
Sebastiaan van Stijn
9959eceb9a
Merge pull request #42626 from mfeit-internet2/small-ipv4-networks
Support small ipv4 networks
2022-06-07 22:15:19 +02:00
Sebastiaan van Stijn
3b94561db2
Merge pull request #43662 from vvoland/fix-logs-regression2
daemon/logger: Driver-scope buffer pools, bigger buffers
2022-06-07 22:04:14 +02:00
Sebastiaan van Stijn
f90056a79d
daemon: LogDaemonEventWithAttributes: don't call SystemInfo()
This function was calling SystemInfo() only to get the daemon's name
to add to the event that's generated.

SystemInfo() is quite heavy, and no info other than the Name was used.
The name returned is just looking up the hostname, so instead, call
`hostName()` directly.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-06-07 22:01:12 +02:00
Sebastiaan van Stijn
237c30dacf
Merge pull request #43704 from thaJeztah/codecov_report_only
codecov: disable patch status
2022-06-07 20:21:07 +02:00
Sebastiaan van Stijn
9ccedde970
Merge pull request #43695 from AkihiroSuda/remove-io.containerd.runtime.v1.linux
daemon: remove support for deprecated `io.containerd.runtime.v1.linux`
2022-06-07 17:29:54 +02:00
Sebastiaan van Stijn
b2795564dc
codecov: disable "patch" status
CodeCov has been very hit-and-miss recently; it looks like we
may need some additional settings to make it compare with the
correct parent commit (perhaps it doesn't work well with rebasing),

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-06-07 17:24:46 +02:00
Sebastiaan van Stijn
aca813e135
Merge pull request #43700 from thaJeztah/update_containerd_1.6.6
update containerd binary to v1.6.6
2022-06-07 01:50:53 +02:00
Tianon Gravi
9948e1425d
Merge pull request #43699 from thaJeztah/bump_containerd_1.6.6
vendor: github.com/containerd/containerd v1.6.6
2022-06-06 16:04:46 -07:00
Sebastiaan van Stijn
73b4b07c4d
vendor: github.com/containerd/containerd v1.6.6
full diff: https://github.com/containerd/containerd/compare/v1.6.5...v1.6.6

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-06-06 22:42:00 +02:00
Sebastiaan van Stijn
a7e3182757
update containerd binary to v1.6.6
Welcome to the v1.6.6 release of containerd!

The sixth patch release for containerd 1.6 includes a fix for
[CVE-2022-31030](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-31030).

Notable Updates

- Fix ExecSync handler to cap console output size ([GHSA-5ffw-gxpp-mxpf](https://github.com/containerd/containerd/security/advisories/GHSA-5ffw-gxpp-mxpf))

full diff: https://github.com/containerd/containerd/compare/v1.6.5...v1.6.6

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-06-06 22:38:36 +02:00
Akihiro Suda
c3eefab550
hack: remove error_on_leaked_containerd_shims
The function does not support containerd-shim-runc-v2 (io.containerd.runc.v2)

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2022-06-05 18:41:44 +09:00
Akihiro Suda
1fcd9f73d4
hack: stop installing containerd-shim (io.containerd.runtime.v1.linux)
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2022-06-05 18:41:44 +09:00
Akihiro Suda
658a4b0fec
libcontainerd: remove support for runtime v1 API
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2022-06-05 18:41:44 +09:00
Akihiro Suda
2c7a6d7bb1
daemon: remove support for deprecated io.containerd.runtime.v1.linux
This has been deprecated in Docker 20.10.0 (f63f73a4a8)

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2022-06-05 18:41:30 +09:00
Akihiro Suda
4734f1e8c9
Merge pull request #43693 from thaJeztah/bump_containerd_1.6.5
vendor: github.com/containerd/containerd v1.6.5
2022-06-05 13:39:28 +09:00
Akihiro Suda
4f1e3516f1
Merge pull request #43691 from thaJeztah/bump_containerd_binary_1.6.5
update containerd binary to v1.6.5
2022-06-05 13:39:08 +09:00
Sebastiaan van Stijn
89db5a6fd4
vendor: github.com/containerd/containerd v1.6.5
full diff: https://github.com/containerd/containerd/compare/v1.6.4...v1.6.5

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-06-04 22:39:32 +02:00
Sebastiaan van Stijn
a747cd3702
update containerd binary to v1.6.5
Welcome to the v1.6.5 release of containerd!

The fifth patch release for containerd 1.6 includes a few fixes and updated
version of runc.

Notable Updates

- Fix for older CNI plugins not reporting version
- Fix mount path handling for CRI plugin on Windows

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-06-04 22:28:42 +02:00
Samuel Karp
a586d536be
Merge pull request #43688 from ferozsalam/fix-names-alphabetisation 2022-06-03 16:12:03 -07:00
Sebastiaan van Stijn
38633e7971
Merge pull request #43689 from thaJeztah/fix_incorrect_warnings
daemon.NewDaemon(): fix network feature detection on first start
2022-06-03 19:30:24 +02:00
Sebastiaan van Stijn
b241e2008e
daemon.NewDaemon(): fix network feature detection on first start
Commit 483aa6294b introduced a regression, causing
spurious warnings to be shown when starting a daemon for the first time after
a fresh install:

    docker info
    ...
    WARNING: IPv4 forwarding is disabled
    WARNING: bridge-nf-call-iptables is disabled
    WARNING: bridge-nf-call-ip6tables is disabled

The information shown is incorrect, as checking the corresponding options on
the system, shows that these options are available:

    cat /proc/sys/net/ipv4/ip_forward
    1
    cat /proc/sys/net/bridge/bridge-nf-call-iptables
    1
    cat /proc/sys/net/bridge/bridge-nf-call-ip6tables
    1

The reason this is failing is because the daemon itself reconfigures those
options during networking initialization in `configureIPForwarding()`;
cf4595265e/libnetwork/drivers/bridge/setup_ip_forwarding.go (L14-L25)

Network initialization happens in the `daemon.restore()` function within `daemon.NewDaemon()`:
cf4595265e/daemon/daemon.go (L475-L478)

However, 483aa6294b moved detection of features
earlier in the `daemon.NewDaemon()` function, and collects the system information
(`d.RawSysInfo()`) before we enter `daemon.restore()`;
cf4595265e/daemon/daemon.go (L1008-L1011)

For optimization (collecting the system information comes at a cost), those
results are cached on the daemon, and will only be performed once (using a
`sync.Once`).

This patch:

- introduces a `getSysInfo()` utility, which collects system information without
  caching the results
- uses `getSysInfo()` to collect the preliminary information needed at that
  point in the daemon's lifecycle.
- moves printing warnings to the end of `daemon.NewDaemon()`, after all information
  can be read correctly.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-06-03 17:54:43 +02:00
Feroz Salam
1db286c5e8 Fix alphabetisation of possible names
Signed-off-by: Feroz Salam <feroz.salam@sourcegraph.com>
2022-06-03 15:07:52 +01:00
Sebastiaan van Stijn
cf4595265e
Merge pull request #43687 from thaJeztah/update_authors_mailmap
update AUTHORS and mailmap
2022-06-03 13:15:27 +02:00
Sebastiaan van Stijn
ca6e94f994
update AUTHORS and mailmap
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-06-03 12:29:40 +02:00
Sebastiaan van Stijn
6e80c027c5
Merge pull request #43409 from vincentbernat/fix/udp-conntrack
bridge: also flush conntrack entries when setting up endpoints
2022-06-03 11:29:25 +02:00
Sebastiaan van Stijn
82e58a0011
Merge pull request #43066 from akerouanton/fix-43054
Revert "Added API to set ephemeral port allocator range."
2022-06-03 10:13:03 +02:00
Sebastiaan van Stijn
9cace7da9e
Merge pull request #43632 from thaJeztah/volumes_fixup_part2
volume/local: refactor validation to allow invalidating early
2022-06-03 09:37:57 +02:00
Sebastiaan van Stijn
cd58d11b2a
volume/local.New(): extract loading options to a function
Note that Windows does not support options, so strictly doesn't need
to have this code, but keeping it in case we're adding support.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-06-03 00:34:29 +02:00
Sebastiaan van Stijn
7e907e29a3
volume/local.New() always unmount existing mounts
Unmounting does not depend on wether or not loading options failed.

This code-path seemed to be used as a "hack" to prevent hitting the
unmount on Windows (which does not support unmounting).

Moving it outside of the "if" to make more clear that it's independent
of loading the options.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-06-03 00:34:27 +02:00
Sebastiaan van Stijn
c0f0cf6c19
volume/local: extract saving options to a separate method
Differentiate between Windows and Linux, as Windows doesn't support
options, so there's no need to save options to disk,

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-06-03 00:34:25 +02:00