Commit graph

1428 commits

Author SHA1 Message Date
Albin Kerouanton
30545de83e
Merge pull request #47393 from vvoland/rro-backwards-compatible-25
[25.0 backport] api/pre-1.44: Default `ReadOnlyNonRecursive` to true
2024-03-01 12:15:21 +01:00
Sebastiaan van Stijn
06767446fe
Merge pull request #47481 from akerouanton/25.0-internal-bridge
[25.0 backport] Make 'internal' bridge networks accessible from host
2024-03-01 11:57:38 +01:00
Paweł Gronowski
f7be6dcba6 integration: Reset OTEL_EXPORTER_OTLP_ENDPOINT for sub-daemons
When creating a new daemon in the `TestDaemonProxy`, reset the
`OTEL_EXPORTER_OTLP_ENDPOINT` to an empty value to disable OTEL
collection to avoid it hitting the proxy.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 5fe96e234d)
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-03-01 10:30:13 +01:00
Rob Murray
c761353e7c Make 'internal' bridge networks accessible from host
Prior to release 25.0.0, the bridge in an internal network was assigned
an IP address - making the internal network accessible from the host,
giving containers on the network access to anything listening on the
bridge's address (or INADDR_ANY on the host).

This change restores that behaviour. It does not restore the default
route that was configured in the container, because packets sent outside
the internal network's subnet have always been dropped. So, a 'connect()'
to an address outside the subnet will still fail fast.

Signed-off-by: Rob Murray <rob.murray@docker.com>
(cherry picked from commit 419f5a6372)
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-03-01 09:29:41 +01:00
Paweł Gronowski
e85cef89fa
api/pre-1.44: Default ReadOnlyNonRecursive to true
Don't change the behavior for older clients and keep the same behavior.
Otherwise client can't opt-out (because `ReadOnlyNonRecursive` is
unsupported before 1.44).

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 432390320e)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-28 10:00:22 +01:00
Paweł Gronowski
9ee331235a
integration: Add container.Output utility
Extracted from bfb810445c

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-27 23:31:35 +01:00
Sebastiaan van Stijn
ab29279200
Merge pull request #47294 from vvoland/fix-save-manifests-25
[25.0 backport] image/save: Fix untagged images not present in index.json
2024-02-05 19:59:57 +01:00
Paweł Gronowski
147b5388dd
integration/save: Add tests checking OCI archive output
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 2ef0b53e51)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-05 14:28:02 +01:00
Laura Brehm
51133117fb
tests: add plugin install test w/ digest
Adds a test case for installing a plugin from a remote in the form
of `plugin-content-trust@sha256:d98f2f8061...`, which is currently
causing the daemon to panic, as we found while running the CLI e2e
tests:

```
docker plugin install registry:5000/plugin-content-trust@sha256:d98f2f806144bf4ba62d4ecaf78fec2f2fe350df5a001f6e3b491c393326aedb
```

Signed-off-by: Laura Brehm <laurabrehm@hey.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-05 09:40:54 +01:00
Rob Murray
bbe6f09afc No inspect 'Config.MacAddress' unless configured.
Do not set 'Config.MacAddress' in inspect output unless the MAC address
is configured.

Also, make sure it is filled in for a configured address on the default
network before the container is started (by translating the network name
from 'default' to 'config' so that the address lookup works).

Signed-off-by: Rob Murray <rob.murray@docker.com>
(cherry picked from commit 8c64b85fb9)
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-02-02 09:10:04 +01:00
Rob Murray
5b13a38144 Only restore a configured MAC addr on restart.
The API's EndpointConfig struct has a MacAddress field that's used for
both the configured address, and the current address (which may be generated).

A configured address must be restored when a container is restarted, but a
generated address must not.

The previous attempt to differentiate between the two, without adding a field
to the API's EndpointConfig that would show up in 'inspect' output, was a
field in the daemon's version of EndpointSettings, MACOperational. It did
not work, MACOperational was set to true when a configured address was
used. So, while it ensured addresses were regenerated, it failed to preserve
a configured address.

So, this change removes that code, and adds DesiredMacAddress to the wrapped
version of EndpointSettings, where it is persisted but does not appear in
'inspect' results. Its value is copied from MacAddress (the API field) when
a container is created.

Signed-off-by: Rob Murray <rob.murray@docker.com>
(cherry picked from commit dae33031e0)
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-02-02 09:10:04 +01:00
Albin Kerouanton
5295e88ceb daemon: rename: don't reload endpoint from datastore
Commit 8b7af1d0f added some code to update the DNSNames of all
endpoints attached to a sandbox by loading a new instance of each
affected endpoints from the datastore through a call to
`Network.EndpointByID()`.

This method then calls `Network.getEndpointFromStore()`, that in
turn calls `store.GetObject()`, which then calls `cache.get()`,
which calls `o.CopyTo(kvObject)`. This effectively creates a fresh
new instance of an Endpoint. However, endpoints are already kept in
memory by Sandbox, meaning we now have two in-memory instances of
the same Endpoint.

As it turns out, libnetwork is built around the idea that no two objects
representing the same thing should leave in-memory, otherwise breaking
mutex locking and optimistic locking (as both instances will have a drifting
version tracking ID -- dbIndex in libnetwork parliance).

In this specific case, this bug materializes by container rename failing
when applied a second time for a given container. An integration test is
added to make sure this won't happen again.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
(cherry picked from commit 80c44b4b2e)
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-01-23 22:53:43 +01:00
Albin Kerouanton
e8801fbe26 daemon: only add short cid to aliases for custom networks
Prior to 7a9b680a, the container short ID was added to the network
aliases only for custom networks. However, this logic wasn't preserved
in 6a2542d and now the cid is always added to the list of network
aliases.

This commit reintroduces the old logic.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
(cherry picked from commit 9f37672ca8)
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-01-23 17:09:11 +01:00
Rob Murray
2b036fb1da
Remove generated MAC addresses on restart.
The MAC address of a running container was stored in the same place as
the configured address for a container.

When starting a stopped container, a generated address was treated as a
configured address. If that generated address (based on an IPAM-assigned
IP address) had been reused, the containers ended up with duplicate MAC
addresses.

So, remember whether the MAC address was explicitly configured, and
clear it if not.

Signed-off-by: Rob Murray <rob.murray@docker.com>
(cherry picked from commit cd53b7380c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-22 19:51:54 +01:00
Cory Snider
13964248f1
libnetwork: loosen container IPAM validation
Permit container network attachments to set any static IP address within
the network's IPAM master pool, including when a subpool is configured.
Users have come to depend on being able to statically assign container
IP addresses which are guaranteed not to collide with automatically-
assigned container addresses.

Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit 058b30023f)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-20 11:33:04 +01:00
Evan Lezar
f7065ab207
Add testutil.TempDir function
This change adds a TempDir function that ensures the correct permissions for
the fake-root user in rootless mode.

Signed-off-by: Evan Lezar <elezar@nvidia.com>
2024-01-17 15:44:47 +01:00
Sebastiaan van Stijn
af4f6c124d
Merge pull request #47088 from vvoland/testing-setupTest-parallel
environment: Error when t.Parallel was called before Protect
2024-01-17 14:57:19 +01:00
Sebastiaan van Stijn
d3e08fe3cf
Merge pull request #47087 from neersighted/cdi_feature
cdi: use separate feature-flag
2024-01-17 14:11:41 +01:00
Paweł Gronowski
24da5233dd
integration: Fix Parallel before setupTest
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-17 13:06:05 +01:00
Bjorn Neergaard
d22c775e04
cdi: use separate feature-flag
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-17 11:31:37 +01:00
Brian Goff
353bccdf22
Merge pull request #47051 from corhere/reinit-health-on-live-restore 2024-01-16 10:08:20 -08:00
Paweł Gronowski
9de132dbf6
integration/image: Move Parallel after setupTest
setupTest should be called before Parallel as it modifies the test
environment which might produce:

```
fatal error: concurrent map writes

goroutine 143 [running]:
github.com/docker/docker/testutil/environment.(*Execution).ProtectContainer(...)
	/go/src/github.com/docker/docker/testutil/environment/protect.go:59
github.com/docker/docker/testutil/environment.ProtectContainers({0x12e8d98, 0xc00040e420}, {0x12f2878?, 0xc0004fc340}, 0xc0001fac00)
	/go/src/github.com/docker/docker/testutil/environment/protect.go:68 +0xb1
github.com/docker/docker/testutil/environment.ProtectAll({0x12e8d98, 0xc00040e210}, {0x12f2878, 0xc0004fc340}, 0xc0001fac00)
	/go/src/github.com/docker/docker/testutil/environment/protect.go:45 +0xf3
github.com/docker/docker/integration/image.setupTest(0xc0004fc340)
	/go/src/github.com/docker/docker/integration/image/main_test.go:46 +0x59
```

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-16 14:45:26 +01:00
Cory Snider
312450d079 integration: test container healthcheck is reset
Update the TestDaemonRestartKilContainers integration test to assert
that a container's healthcheck status is always reset to the Starting
state after a daemon restart, even when the container is live-restored.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-01-15 15:30:22 -05:00
Paweł Gronowski
c58acf06c2 image/save: Derive Descriptor from actual tar archive
Layer size is the sum of the individual files count, not the tar
archive. Use the total bytes read returned by `io.Copy` to populate the
`Size` field.

Also set the digest to the actual digest of the tar archive.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-15 16:35:54 +01:00
Paweł Gronowski
2c45df2ab4 image/save: Fix missing layers in manifest
The new OCI-compatible archive export relies on the Descriptors returned
by the layer (`distribution.Describable` interface implementation).

The issue with that is that the `roLayer` and the `referencedCacheLayer`
types don't implement this interface. Implementing that interface for
them based on their `descriptor` doesn't work though, because that
descriptor is empty.

To workaround this issue, just create a new descriptor if the one
provided by the layer is empty.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-12 17:10:23 +01:00
Sebastiaan van Stijn
7bc56c5365
Merge pull request #46853 from akerouanton/libnet-ep-dns-names
libnet: Endpoint: remove isAnonymous & myAliases
2023-12-20 19:53:16 +01:00
Albin Kerouanton
6a2542dacf
libnet: remove Endpoint.anonymous
No more concept of "anonymous endpoints". The equivalent is now an
endpoint with no DNSNames set.

Some of the code removed by this commit was mutating user-supplied
endpoint's Aliases to add container's short ID to that list. In order to
preserve backward compatibility for the ContainerInspect endpoint, this
commit also takes care of adding that short ID (and the container
hostname) to `EndpointSettings.Aliases` before returning the response.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-12-20 19:04:37 +01:00
Sebastiaan van Stijn
388216fc45
Merge pull request #46850 from robmry/46829-allow_ipv6_subnet_change
Allow overlapping change in bridge's IPv6 network.
2023-12-19 18:35:13 +01:00
Derek McGowan
35a51fd697
Update authz plugin test to not use httputil
Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-12-18 13:16:36 -08:00
Rob Murray
27f3abd893 Allow overlapping change in bridge's IPv6 network.
Calculate the IPv6 addreesses needed on a bridge, then reconcile them
with the addresses on an existing bridge by deleting then adding as
required.

(Previously, required addresses were added one-by-one, then unwanted
addresses were removed. This meant the daemon failed to start if, for
example, an existing bridge had address '2000:db8::/64' and the config
was changed to '2000:db8::/80'.)

IPv6 addresses are now calculated and applied in one go, so there's no
need for setupVerifyAndReconcile() to check the set of IPv6 addresses on
the bridge. And, it was guarded by !config.InhibitIPv4, which can't have
been right. So, removed its IPv6 parts, and added IPv4 to its name.

Link local addresses, the example given in the original ticket, are now
released when containers are stopped. Not releasing them meant that
when using an LL subnet on the default bridge, no container could be
started after a container was stopped (because the calculated address
could not be re-allocated). In non-default bridge networks using an
LL subnet, addresses leaked.

Linux always uses the standard 'fe80::/64' LL network. So, if a bridge
is configured with an LL subnet prefix that overlaps with it, a config
error is reported. Non-overlapping LL subnet prefixes are allowed.

Signed-off-by: Rob Murray <rob.murray@docker.com>
2023-12-18 16:10:41 +00:00
Paweł Gronowski
eaaf1ea96d
integration/prune: Run in a separate daemon
Isolate the prune effects by running the test in a separate daemon.
This minimizes the impact of/on other integration tests.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-12-07 17:33:26 +01:00
Sebastiaan van Stijn
82a318db5f
Merge pull request #46894 from vvoland/hack-load-specialimages
hack: Load special images on demand
2023-12-07 11:28:38 +01:00
Paweł Gronowski
bc94dfc7d2
hack: Load special images on demand
Rewrite `.build-empty-images` shell script that produced special images
(emptyfs with no layers, and empty danglign image) to a Go functions
that construct the same archives in a temporary directory.

Use them to load these images on demand only in the tests that need
them.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-12-06 17:16:37 +01:00
Paweł Gronowski
d5eba1bfe5
integration/build: Use setupTest
These tests build new images, setupTest sets up the test cleanup
function that clears the test environment from created images,
containers, etc.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-12-05 12:49:30 +01:00
Sebastiaan van Stijn
713c7d49a1
integration(-cli): remove skips for old daemon versions (<20.10)
This removes various skips that accounted for running the integration tests
against older versions of the daemon before 20.10 (API version v1.41). Those
versions are EOL, and we don't run tests against them.

This reverts most of e440831802, and similar
PRs.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-05 01:03:50 +01:00
Sebastiaan van Stijn
97549d923c
integration/container: TestInfoAPIVersioned: adjust API version
This test was using API version 1.20 to test old behavior, but the actual change
in behavior was API v1.25; see commit 6d98e344c7
and 63b5a37203.
This updates the test to use API v1.24 to test the old behavior.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-04 22:38:55 +01:00
Sebastiaan van Stijn
410feab962
integration/container: TestKillDifferentUserContainer: adjust API version
TestKillDifferentUserContainer was migrated from integration-cli in
commit 0855922cd3. Before migration, it
was not using a specific API version, so we can assume "current"
API version.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-04 22:38:52 +01:00
Sebastiaan van Stijn
d3533eee68
Merge pull request #46849 from robmry/37937_explicit_bridge_mtu
Explicitly set MTU on bridge devices.
2023-11-30 22:30:11 +01:00
Tianon Gravi
488fe76eb8
Merge pull request #45789 from thaJeztah/update_debian
Dockerfile: update to Debian "bookworm", and update frozen images to debian:bookworm-slim
2023-11-30 12:33:48 -08:00
Sebastiaan van Stijn
3bfb6a9420
frozen images: update to debian:bookworm-slim
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-11-30 12:01:06 +01:00
Paweł Gronowski
3a0af5ad30
integration/TestLiveRestore: Wait for process to exit
Replace `time.Sleep` with a poll that checks if process no longer exists
to avoid possible race condition.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-11-30 10:15:58 +01:00
Paweł Gronowski
c5ea3d595c
liverestore: Don't remove --rm containers on restart
When live-restore is enabled, containers with autoremove enabled
shouldn't be forcibly killed when engine restarts.
They still should be removed if they exited while the engine was down
though.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-11-28 12:59:38 +01:00
Rob Murray
964ab7158c Explicitly set MTU on bridge devices.
This is purely cosmetic - if a non-default MTU is configured, the bridge
will have the default MTU=1500 until a container's 'veth' is connected
and an MTU is set on the veth. That's a disconcerting, it looks like the
config has been ignored - so, set the bridge's MTU explicitly.

Fixes #37937

Signed-off-by: Rob Murray <rob.murray@docker.com>
2023-11-27 11:18:54 +00:00
Paweł Gronowski
05523e289b
c8d/integration: Adjust TestSaveCheckTimes
The graphdriver implementation sets the ModTime of all image content to
match the `Created` time from the image config, whereas the containerd's
archive export code just leaves it empty (zero).

Adjust the test in the case where containerd integration is enabled to
check if config file ModTime is equal to zero (UNIX epoch) instead.

This behaviour is not a part of the Docker Image Specification and the
intention behind introducing it was to make the `docker save` produce
the same archive regardless of the time it was performed.

It would also be a bit problematic with the OCI archive layout which can
contain multiple images referencing the same content.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-11-24 14:49:02 +01:00
Sebastiaan van Stijn
9b02a84a5a
Merge pull request #46785 from rumpl/c8d-userns-commit
c8d: handle user namespace remapping on commit
2023-11-13 16:31:32 +01:00
Djordje Lukic
a7acfffea6
test: Add a test for userns remapped daemon commit
The files in an image created by a commit should have the right uid and gid set

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-11-08 12:33:03 +01:00
Rachit Sharma
7995e3288f
Add until filter to docker image ls
Signed-off-by: Rachit Sharma <rachitsharma613@gmail.com>

add handling for multiple filters

Signed-off-by: Rachit Sharma <rachitsharma613@gmail.com>

Update integration/image/list_test.go

Co-authored-by: Cory Snider <corhere@gmail.com>
Signed-off-by: Rachit Sharma <rachitsharma613@gmail.com>

Add documentation of filter

Signed-off-by: Rachit Sharma <rachitsharma613@gmail.com>

Update integration/image/list_test.go

Co-authored-by: Cory Snider <corhere@gmail.com>
Signed-off-by: Rachit Sharma <rachitsharma613@gmail.com>

Fix bug with CommitOptions

Signed-off-by: Rachit Sharma <rachitsharma613@gmail.com>

add wrapping of text to 80 chars

Signed-off-by: Rachit Sharma <rachitsharma613@gmail.com>
2023-11-08 12:12:23 +01:00
Sebastiaan van Stijn
58785c2932
integration/networking: fix TestBridgeICC
This test broke in 98323ac114.

This commit renamed WithMacAddress into WithContainerWideMacAddress.
This helper sets the MacAddress field in container.Config. However, API
v1.44 now ignores this field if the NetworkMode has no matching entry in
EndpointsConfig.

This fix uses the helper WithMacAddress and specify for which
EndpointConfig the MacAddress is specified.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-11-08 10:23:24 +01:00
Sebastiaan van Stijn
49cea49cfa
Merge pull request #45905 from akerouanton/endpoint-specific-mac-address
api: Add a field MacAddress to EndpointSettings
2023-11-07 16:37:27 +01:00
Sebastiaan van Stijn
26c054edbf
Merge pull request #46531 from akerouanton/networking-suite-bridge-tests
integration: Add a new networking integration test suite
2023-11-03 12:33:44 +01:00