Commit graph

8178 commits

Author SHA1 Message Date
Albin Kerouanton
6fd893bd3a
Merge pull request #46768 from akerouanton/dedicated-default-nw-config-struct
daemon/config: Put params for the default network into a dedicated struct
2023-11-03 17:03:05 +01:00
Albin Kerouanton
d5d41c2849
daemon/config: Put params for the default network into a dedicated struct
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-11-03 14:10:41 +01:00
Brian Goff
02a932d63f Fix case where health start interval is 0 uses default
When the start interval is 0 we should treat that as unset.
This is especially important for older API versions where we reset the
value to 0.

Instead of using the default probe value we should be using the
configured `interval` value (which may be a default as well) which gives
us back the old behavior before support for start interval was added.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2023-11-02 20:02:16 +00:00
Brian Goff
2f138d860e
Merge pull request #46685 from rumpl/c8d-tag-does-not-exist
c8d: Return the "tag does not exist error"
2023-11-02 12:33:36 -07:00
Sebastiaan van Stijn
d9dce8e0d0
Merge pull request #46618 from vvoland/c8d-pull-all-tags-2
c8d/pull: Handle pull all tags (2nd approach)
2023-10-31 16:45:18 +01:00
Sebastiaan van Stijn
1c50526092
daemon: improve daemon start informational log message
When starting a daemon in debug mode (such as used in CI), many log-messages
are printed during startup. As a result, the log message indicating whether
graph-drivers or snapshotters are used may appear far separate from the
informational log about the daemon (and selected storage-driver).

The existing log-driver also unconditionally uses the legacy "graph-driver"
terminology, instead of the more generic "storage-driver".

This patch changes the log message shown during startup to use the generic
"graph-driver" as field, and adds a new field that indicates wheter we're
using snapshotters or graph-drivers.

Given that snapshotters will be the default at some point, an alternative
could be to include the _type_ of driver used, for example;
`io.containerd.snapshotter.v1`, which may continue to be relevant after
snapshotters become the default, and at which point (potentially) the
type of snapshotter becomes more relevant.

Before this change:

    TEST_INTEGRATION_USE_SNAPSHOTTER=1 DOCKER_GRAPHDRIVER=overlayfs dockerd
    ...
    INFO[2023-10-31T09:12:33.586269801Z] Starting daemon with containerd snapshotter integration enabled
    INFO[2023-10-31T09:12:33.586322176Z] Loading containers: start.
    INFO[2023-10-31T09:12:33.640514759Z] Loading containers: done.
    INFO[2023-10-31T09:12:33.646498134Z] Docker daemon                                 commit=dcf7287d647bcb515015e389df46ccf1e09855b7 graphdriver=overlayfs version=dev
    INFO[2023-10-31T09:12:33.646706551Z] Daemon has completed initialization
    INFO[2023-10-31T09:12:33.658840592Z] API listen on /var/run/docker.sock

With this change;

    TEST_INTEGRATION_USE_SNAPSHOTTER=1 DOCKER_GRAPHDRIVER=overlayfs dockerd
    ...
    INFO[2023-10-31T08:41:38.841155928Z] Starting daemon with containerd snapshotter integration enabled
    INFO[2023-10-31T08:41:38.841207512Z] Loading containers: start.
    INFO[2023-10-31T08:41:38.902461053Z] Loading containers: done.
    INFO[2023-10-31T08:41:38.910535137Z] Docker daemon                                 commit=dcf7287d647bcb515015e389df46ccf1e09855b7 containerd-snapshotter=true storage-driver=overlayfs version=dev
    INFO[2023-10-31T08:41:38.910936803Z] Daemon has completed initialization

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-31 10:20:27 +01:00
Sebastiaan van Stijn
e9efc0a361
Merge pull request #46741 from vvoland/c8d-push-notags
c8d/push: Return error when repository has no tags
2023-10-30 20:08:31 +01:00
Sebastiaan van Stijn
dcf7287d64
Merge pull request #46662 from vvoland/c8d-pull-access-denied-msg-2
c8d/pull: Don't wrap `no basic auth` error
2023-10-30 12:43:41 +01:00
Paweł Gronowski
cd95cd0671
c8d/push: Return error when repository has no tags
In case of `docker push -a`, we need to return an error if there is no
image for the given repository.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-10-30 10:55:08 +01:00
Paweł Gronowski
df34db1158
c8d/pull: Don't wrap no basic auth error
Don't wrap the `no basic auth credentials` error from containerd and
return it as-is.

The error will look like:
```
failed to resolve reference "docker.io/library/aodkoakds:latest": pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed
```

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-10-30 09:40:08 +01:00
Paweł Gronowski
27e064e7e9
c8d/history: Fix nil dereference
Check if `Created` is not nil before dereferencing.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-10-26 15:37:45 +02:00
Sebastiaan van Stijn
1b9411ef77
Merge pull request #46723 from laurazard/cleanup-redundant-calls
daemon/exec_linux: assorted cleanups
2023-10-26 09:54:51 +02:00
Sebastiaan van Stijn
e0476beb78
Merge pull request #46688 from thaJeztah/restart_nocancel
daemon: daemon.containerRestart: don't cancel restart on context cancel
2023-10-26 09:50:20 +02:00
Laura Brehm
9d578a2e6d
Cleanup duplicate imports
Kept `coci` import alias since we use it elsewhere,
maybe to prevent confusion with our own `oci` package.

Signed-off-by: Laura Brehm <laurabrehm@hey.com>
2023-10-26 00:10:23 +01:00
Albin Kerouanton
61c78d9438
daemon: Validate MacAddress on container create
This was previously checked only on ContainerStart.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-10-25 22:55:59 +02:00
Albin Kerouanton
ee9f0ed895
api: Deprecate ContainerConfig.MacAddress
Having a sandbox/container-wide MacAddress field makes little sense
since a container can be connected to multiple networks at the same
time. This field is an artefact of old times where a container could be
connected to a single network only.

As we now have a way to specify per-endpoint mac address, this field is
now deprecated.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-25 22:55:59 +02:00
Albin Kerouanton
052562ffd5
api: Add a field MacAddress to EndpointSettings
Prior to this commit, only container.Config had a MacAddress field and
it's used only for the first network the container connects to. It's a
relic of old times where custom networks were not supported.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-10-25 22:52:26 +02:00
Sebastiaan van Stijn
4776a9e6fb
Merge pull request #46722 from thaJeztah/dedup_additionalgroups
daemon: remove redundant withResetAdditionalGIDs option
2023-10-25 22:48:19 +02:00
Sebastiaan van Stijn
460e1b3600
Merge pull request #46720 from akerouanton/container-create-init-structs
api: ContainerCreate: clean up BC conditions
2023-10-25 22:47:29 +02:00
Albin Kerouanton
91eee33f62
api: ContainerCreate: return an error when config is nil
The same error is already returned by `(*Daemon).containerCreate()` but
since this function is also called by the cluster executor, the error
has to be duplicated.

Doing that allows to remove a nil check on container config in
`postContainersCreate`.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-10-25 21:25:17 +02:00
Sebastiaan van Stijn
1f096174d6
Merge pull request #46717 from rumpl/c8d-image-children
c8d: Use the labels to get the children of an image
2023-10-25 18:00:46 +02:00
Sebastiaan van Stijn
a54a34ef0e
Merge pull request #46718 from rumpl/c8d-tag-label
c8d: Preserve image labels when tagging
2023-10-25 17:55:43 +02:00
Djordje Lukic
7d5445e312
Merge pull request #46719 from rumpl/c8d-image-created-date
c8d: show the real image creation date when listing images
2023-10-25 17:38:01 +02:00
Sebastiaan van Stijn
417018659b
daemon: remove redundant withResetAdditionalGIDs option
containerd's `WithUser` function now resets this property, starting with
[3eda46af12b1deedab3d0802adb2e81cb3521950][1] (v1.7.0-beta.4), so we no
longer need this function.

[1]: 3eda46af12

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-25 17:15:05 +02:00
Sebastiaan van Stijn
e36260f39a
Merge pull request #46715 from laurazard/c8d-fix-swarm-failures
c8d/exec: Add additional groups on exec
2023-10-25 17:10:45 +02:00
Djordje Lukic
8f756fe679
c8d: show the real image creation date when listing images
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-10-25 16:43:24 +02:00
Djordje Lukic
7e0cb4c46f
c8d: Use the labels to get the children of an image
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-10-25 15:12:42 +02:00
Djordje Lukic
f5535d516f
c8d: Preserve image labels when tagging
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-10-25 14:56:36 +02:00
Laura Brehm
d7266afc7e
c8d/exec: Add additional groups on exec
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
2023-10-25 02:28:39 +01:00
Sebastiaan van Stijn
df3a321164
migrate to github.com/moby/sys/user
The github.com/opencontainers/runc/libcontainer/user package was moved
to a separate module. While there's still uses of the old module in
our code-base, runc itself is migrating to the new module, and deprecated
the old package (for runc 1.2).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-24 15:45:02 +02:00
Sebastiaan van Stijn
aeb8972281
daemon: daemon.containerRestart: don't cancel restart on context cancel
commit def549c8f6 passed through the context
to the daemon.ContainerStart function. As a result, restarting containers
no longer is an atomic operation, because a context cancellation could
interrupt the restart (between "stopping" and "(re)starting"), resulting
in the container being stopped, but not restarted.

Restarting a container, or more factually; making a successful request on
the `/containers/{id]/restart` endpoint, should be an atomic operation.

This patch uses a context.WithoutCancel for restart requests.

It's worth noting that daemon.containerStop already uses context.WithoutCancel,
so in that function, we'll be wrapping the context twice, but this should
likely not cause issues (just redundant for this code-path).

Before this patch, starting a container that bind-mounts the docker socket,
then restarting itself from within the container would cancel the restart
operation. The container would be stopped, but not started after that:

    docker run -dit --name myself -v /var/run/docker.sock:/var/run/docker.sock docker:cli sh
    docker exec myself sh -c 'docker restart myself'

    docker ps -a
    CONTAINER ID   IMAGE         COMMAND                  CREATED          STATUS                       PORTS     NAMES
    3a2a741c65ff   docker:cli    "docker-entrypoint.s…"   26 seconds ago   Exited (128) 7 seconds ago             myself

With this patch: the stop still cancels the exec, but does not cancel the
restart operation, and the container is started again:

    docker run -dit --name myself -v /var/run/docker.sock:/var/run/docker.sock docker:cli sh
    docker exec myself sh -c 'docker restart myself'
    docker ps
    CONTAINER ID   IMAGE        COMMAND                  CREATED              STATUS         PORTS     NAMES
    4393a01f7c75   docker:cli   "docker-entrypoint.s…"   About a minute ago   Up 4 seconds             myself

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-24 12:56:45 +02:00
Sebastiaan van Stijn
164167ea91
Merge pull request #46687 from thaJeztah/more_nocancel
daemon: use context.WithoutCancel in more places
2023-10-20 19:36:44 +02:00
Sebastiaan van Stijn
aad51c0b4e
daemon: daemon.shutdownContainer: use context.WithoutCancel
Use context.WithoutCancel so that both the containerStop and
container.Wait can share the same parent context. This context is still
a "TODO", but can be wired up in future.

It's worth noting that daemon.containerStop already uses context.WithoutCancel,
so in that function, we'll be wrapping the context twice, but this should
likely not cause issues (just redundant for this code-path).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-20 17:50:06 +02:00
Sebastiaan van Stijn
dca89c67ef
daemon: daemon.containerStop: use context.WithoutCancel
Follow-up to fc94ed0a86. Now that
f6e44bc0e8 added the compatcontext
package, we can start using context.WithoutCancel.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-20 17:44:30 +02:00
Djordje Lukic
f800215946
Merge pull request #46570 from dmcgowan/fix-disk-usage-test
integration: fix disk usage test for c8d
2023-10-20 17:11:31 +02:00
Djordje Lukic
8166818791
c8d: Return the "tag does not exist error"
In the tagged case the error message when the image/tag is not found
should be "tag does not exist: ref"

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-10-20 16:55:20 +02:00
Paweł Gronowski
cdb44010c9
c8d/history: Fill ID for parents without the label
When choosing the next image, don't reject images without the classic
builder parent label. The intention was to *prefer* images them instead
of making that a condition.
This fixes the ID not being filled for parent images that weren't built
with the classic builder.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-10-19 16:41:45 +02:00
Djordje Lukic
29d863d451
Merge pull request #46653 from vvoland/c8d-history-fix-tags
c8d/history: Fix Tags assigned from parents
2023-10-19 14:53:09 +02:00
Djordje Lukic
0253fedf03
Merge pull request #46655 from vvoland/c8d-push-propagate-source-children
c8d/push: Set distribution source recursively
2023-10-19 10:30:47 +02:00
Paweł Gronowski
d28bd54f2a
c8d/history: Fix Tags assigned from parents
The `Tags` slice of each history entry was filled with tags of parent
image. Change it to correctly assign the current image tags.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-10-19 10:19:04 +02:00
Derek McGowan
e97716a95a
Fix disk usage test
Check for accurate values that may contain content sizes unknown to the
usage test in the calculation. Avoid asserting using deep equals when
only the expected value range is known to the test.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-10-18 21:20:54 -07:00
Bjorn Neergaard
fd3066c168
Merge pull request #46646 from thaJeztah/start_interval_validation
HealthCheck: add validation for minimum accepted start-interval (1ms)
2023-10-16 11:07:14 -07:00
Sebastiaan van Stijn
a7dd561c4a
Merge pull request #46651 from akerouanton/releaseNetwork-NetworkDisabled
daemon: release sandbox even when NetworkDisabled
2023-10-16 19:33:08 +02:00
Paweł Gronowski
9357bcb582
c8d/pull: Return same access denied error as graphdrivers
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-10-16 17:54:01 +02:00
Paweł Gronowski
488559a330
c8d/push: Set distribution source recursively
After a successful push, all pushed blobs should have a
distribution.source label pointing to the new registry.

Before this commit, the label was only appended to the top-level blob
(manifest or manifest list). Adjust this to also do that recursively to
its children.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-10-16 17:07:39 +02:00
Paweł Gronowski
d9b5445f39
c8d/pull: Handle pull all tags
Use the distribution code to query the remote repository for tags and
pull them sequentially just like the non-c8d pull.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-10-16 14:19:29 +02:00
Paweł Gronowski
6040283f23
Merge pull request #46581 from vvoland/c8d-push-hide-jsons
c8d/push: Show progress only on blobs
2023-10-16 13:39:25 +02:00
payall4u
9664f33e0d
daemon: release sandbox even when NetworkDisabled
When the default bridge is disabled by setting dockerd's `--bridge=none`
option, the daemon still creates a sandbox for containers with no
network attachment specified. In that case `NetworkDisabled` will be set
to true.

However, currently the `releaseNetwork` call will early return if
NetworkDisabled is true. Thus, these sandboxes won't be deleted until
the daemon is restarted. If a high number of such containers are
created, the daemon would then take few minutes to start.

See https://github.com/moby/moby/issues/42461.

Signed-off-by: payall4u <payall4u@qq.com>
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-10-16 12:08:01 +02:00
Sebastiaan van Stijn
2df698025c
HealthCheck: add validation for minimum accepted start-interval
This is a follow-up to 2216d3ca8d, which
implemented the StartInterval for health-checks, but did not add validation
for the minimum accepted interval;

> The time to wait between checks in nanoseconds during the start period.
> It should be 0 or at least 1000000 (1 ms). 0 means inherit.

This patch adds validation for the minimum accepted interval (1ms).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-16 10:46:18 +02:00
Sebastiaan van Stijn
80a9fc6d36
Merge pull request #46565 from vvoland/c8d-mirrors-fix
daemon/RegistryHosts: Don't lose mirrors
2023-10-13 22:31:24 +02:00