Commit graph

1441 commits

Author SHA1 Message Date
Rob Murray
9faf4855d5 Simplify macvlan/ipvlan integration test structure
Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-02-14 14:09:45 +00:00
Rob Murray
4eb95d01bc Run the macvlan/ipvlan integration tests
The problem was accidentally introduced in:
  e8dc902781

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-02-14 14:08:10 +00:00
Sebastiaan van Stijn
8758d08bb4
api: remove handling of HostConfig on POST /containers/{id}/start (api < v1.24)
API v1.20 (Docker Engine v1.11.0) and older allowed a HostConfig to be passed
when starting a container. This feature was deprecated in API v1.21 (Docker
Engine v1.10.0) in 3e7405aea8, and removed in
API v1.23 (Docker Engine v1.12.0) in commit 0a8386c8be.

API v1.23 and older are deprecated, and this patch removes the feature.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-06 18:44:44 +01:00
Sebastiaan van Stijn
b3a0ff9944
api: remove POST /containers/{id}/copy endpoint (api < v1.23)
This endpoint was deprecated in API v1.20 (Docker Engine v1.8.0) in
commit db9cc91a9e, in favor of the
`PUT /containers/{id}/archive` and `HEAD /containers/{id}/archive`
endpoints, and disabled in API v1.24 (Docker Engine v1.12.0) through
commit 428328908d.

This patch removes the endpoint, and the associated `daemon.ContainerCopy`
method in the backend.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-06 18:44:44 +01:00
Sebastiaan van Stijn
ed93110e11
api: update test to reflect reality on Windows
The TestInspectAPIContainerResponse mentioned that Windows does not
support API versions before v1.25.

While technically, no stable release existed for Windows with API versions
before that (see f811d5b128), API version
v1.24 was enabled in e4af39aeb3, to have
a consistend fallback version for API version negotiation.

This patch updates the test to reflect that change.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-06 18:44:43 +01:00
Sebastiaan van Stijn
f0dd554e3c
api: remove code for ContainerInspect on api < v1.20
API v1.23 and older are deprecated, so we can remove the code to adjust
responses for API v1.19 and lower.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-06 18:44:43 +01:00
Sebastiaan van Stijn
dfdf2adf0c
api: POST /containers/{id}/kill: remove handling for api < 1.20
API v1.20 and up produces an error when signalling / killing a non-running
container (see c92377e300). Older API versions
allowed this, and an exception was added in 621e3d8587.

API v1.23 and older are deprecated, so we can remove this handling.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-06 18:44:42 +01:00
Sebastiaan van Stijn
2156635843
Merge pull request #47232 from vvoland/fix-save-manifests
image/save: Fix untagged images not present in index.json
2024-02-05 19:06:54 +01:00
Paweł Gronowski
2ef0b53e51
integration/save: Add tests checking OCI archive output
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-05 11:17:58 +01:00
Sebastiaan van Stijn
6b83319773
Merge pull request #47299 from laurazard/plugin-install-digest
plugins: Fix panic when fetching by digest
2024-02-05 09:39:05 +01:00
Albin Kerouanton
ca683c1c77
Merge pull request #47233 from robmry/47146-duplicate_mac_addrs2
Only restore a configured MAC addr on restart.
2024-02-02 09:08:17 +01:00
Laura Brehm
82dda18898
tests: add plugin install test w/ digest
Adds a test case for installing a plugin from a remote in the form
of `plugin-content-trust@sha256:d98f2f8061...`, which is currently
causing the daemon to panic, as we found while running the CLI e2e
tests:

```
docker plugin install registry:5000/plugin-content-trust@sha256:d98f2f806144bf4ba62d4ecaf78fec2f2fe350df5a001f6e3b491c393326aedb
```

Signed-off-by: Laura Brehm <laurabrehm@hey.com>
2024-02-01 23:00:38 +00:00
Rob Murray
8c64b85fb9 No inspect 'Config.MacAddress' unless configured.
Do not set 'Config.MacAddress' in inspect output unless the MAC address
is configured.

Also, make sure it is filled in for a configured address on the default
network before the container is started (by translating the network name
from 'default' to 'config' so that the address lookup works).

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-02-01 09:57:35 +00:00
Rob Murray
dae33031e0 Only restore a configured MAC addr on restart.
The API's EndpointConfig struct has a MacAddress field that's used for
both the configured address, and the current address (which may be generated).

A configured address must be restored when a container is restarted, but a
generated address must not.

The previous attempt to differentiate between the two, without adding a field
to the API's EndpointConfig that would show up in 'inspect' output, was a
field in the daemon's version of EndpointSettings, MACOperational. It did
not work, MACOperational was set to true when a configured address was
used. So, while it ensured addresses were regenerated, it failed to preserve
a configured address.

So, this change removes that code, and adds DesiredMacAddress to the wrapped
version of EndpointSettings, where it is persisted but does not appear in
'inspect' results. Its value is copied from MacAddress (the API field) when
a container is created.

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-02-01 09:55:54 +00:00
Albin Kerouanton
794f7127ef
Merge pull request #47062 from robmry/35954-default_ipv6_enabled
Detect IPv6 support in containers, generate '/etc/hosts' accordingly.
2024-01-29 16:31:35 +01:00
Sebastiaan van Stijn
ac2a028dcc
api/types: move image options to api/types/image
To prevent a circular import between api/types and api/types image,
the RequestPrivilegeFunc reference was not moved, but defined as
part of the PullOptions / PushOptions.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-24 00:10:33 +01:00
Sebastiaan van Stijn
0bb84f5cef
Merge pull request #47195 from akerouanton/fix-multiple-rename-error
daemon: rename: don't reload endpoint from datastore
2024-01-23 23:41:07 +01:00
Albin Kerouanton
80c44b4b2e daemon: rename: don't reload endpoint from datastore
Commit 8b7af1d0f added some code to update the DNSNames of all
endpoints attached to a sandbox by loading a new instance of each
affected endpoints from the datastore through a call to
`Network.EndpointByID()`.

This method then calls `Network.getEndpointFromStore()`, that in
turn calls `store.GetObject()`, which then calls `cache.get()`,
which calls `o.CopyTo(kvObject)`. This effectively creates a fresh
new instance of an Endpoint. However, endpoints are already kept in
memory by Sandbox, meaning we now have two in-memory instances of
the same Endpoint.

As it turns out, libnetwork is built around the idea that no two objects
representing the same thing should leave in-memory, otherwise breaking
mutex locking and optimistic locking (as both instances will have a drifting
version tracking ID -- dbIndex in libnetwork parliance).

In this specific case, this bug materializes by container rename failing
when applied a second time for a given container. An integration test is
added to make sure this won't happen again.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-01-23 22:53:21 +01:00
Sebastiaan van Stijn
9763709c05
Merge pull request #47181 from akerouanton/fix-aliases-on-default-bridge
daemon: only add short cid to aliases for custom networks
2024-01-23 18:28:33 +01:00
Albin Kerouanton
9f37672ca8 daemon: only add short cid to aliases for custom networks
Prior to 7a9b680a, the container short ID was added to the network
aliases only for custom networks. However, this logic wasn't preserved
in 6a2542d and now the cid is always added to the list of network
aliases.

This commit reintroduces the old logic.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-01-23 17:07:40 +01:00
Sebastiaan van Stijn
c87e0ad209
Merge pull request #47168 from robmry/47146-duplicate_mac_addrs
Remove generated MAC addresses on restart.
2024-01-22 19:48:24 +01:00
Rob Murray
cd53b7380c Remove generated MAC addresses on restart.
The MAC address of a running container was stored in the same place as
the configured address for a container.

When starting a stopped container, a generated address was treated as a
configured address. If that generated address (based on an IPAM-assigned
IP address) had been reused, the containers ended up with duplicate MAC
addresses.

So, remember whether the MAC address was explicitly configured, and
clear it if not.

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-01-22 17:52:20 +00:00
Sebastiaan van Stijn
64a6cc3afd
integration/build: improve some asserts, and add asserts for unhandled errs
- add some asserts for unhandled errors
- use consts for fixed values, and slightly re-format Dockerfile contentt
- inline one-line Dockerfiles
- fix some vars to be properly camel-cased
- improve assert for error-types;

Before:

    === RUN   TestBuildPlatformInvalid
        build_test.go:685: assertion failed: expression is false: errdefs.IsInvalidParameter(err)
    --- FAIL: TestBuildPlatformInvalid (0.01s)
    FAIL

After:

    === RUN   TestBuildPlatformInvalid
        build_test.go:689: assertion failed: error is Error response from daemon: "foobar": unknown operating system or architecture: invalid argument (errdefs.errSystem), not errdefs.IsInvalidParameter
    --- FAIL: TestBuildPlatformInvalid (0.01s)
    FAIL

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-21 13:45:06 +01:00
Sebastiaan van Stijn
a88cd68d3e
integration/images: improve some asserts, and add asserts for unhandled errs
Before:

    === FAIL: amd64.integration.image TestImagePullPlatformInvalid (0.01s)
        pull_test.go:37: assertion failed: expression is false: errdefs.IsInvalidParameter(err)

After:

    === RUN   TestImagePullPlatformInvalid
        pull_test.go:37: assertion failed: error is Error response from daemon: "foobar": unknown operating system or architecture: invalid argument (errdefs.errSystem), not errdefs.IsInvalidParameter

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-21 13:45:06 +01:00
Cory Snider
058b30023f libnetwork: loosen container IPAM validation
Permit container network attachments to set any static IP address within
the network's IPAM master pool, including when a subpool is configured.
Users have come to depend on being able to statically assign container
IP addresses which are guaranteed not to collide with automatically-
assigned container addresses.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-01-19 20:18:15 -05:00
Rob Murray
a8f7c5ee48 Detect IPv6 support in containers.
Some configuration in a container depends on whether it has support for
IPv6 (including default entries for '::1' etc in '/etc/hosts').

Before this change, the container's support for IPv6 was determined by
whether it was connected to any IPv6-enabled networks. But, that can
change over time, it isn't a property of the container itself.

So, instead, detect IPv6 support by looking for '::1' on the container's
loopback interface. It will not be present if the kernel does not have
IPv6 support, or the user has disabled it in new namespaces by other
means.

Once IPv6 support has been determined for the container, its '/etc/hosts'
is re-generated accordingly.

The daemon no longer disables IPv6 on all interfaces during initialisation.
It now disables IPv6 only for interfaces that have not been assigned an
IPv6 address. (But, even if IPv6 is disabled for the container using the
sysctl 'net.ipv6.conf.all.disable_ipv6=1', interfaces connected to IPv6
networks still get IPv6 addresses that appear in the internal DNS. There's
more to-do!)

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-01-19 20:24:07 +00:00
Paweł Gronowski
bfb810445c
volumes: Implement subpath mount
`VolumeOptions` now has a `Subpath` field which allows to specify a path
relative to the volume that should be mounted as a destination.

Symlinks are supported, but they cannot escape the base volume
directory.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-19 17:32:10 +01:00
Sebastiaan van Stijn
66cf6e3a7a
rename "image" vars to prevent conflicts with imports
We have many "image" packages, so these vars easily conflict/shadow
imports. Let's rename them (and in some cases use a const) to
prevent that.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-19 12:49:53 +01:00
Evan Lezar
f7065ab207
Add testutil.TempDir function
This change adds a TempDir function that ensures the correct permissions for
the fake-root user in rootless mode.

Signed-off-by: Evan Lezar <elezar@nvidia.com>
2024-01-17 15:44:47 +01:00
Sebastiaan van Stijn
af4f6c124d
Merge pull request #47088 from vvoland/testing-setupTest-parallel
environment: Error when t.Parallel was called before Protect
2024-01-17 14:57:19 +01:00
Sebastiaan van Stijn
d3e08fe3cf
Merge pull request #47087 from neersighted/cdi_feature
cdi: use separate feature-flag
2024-01-17 14:11:41 +01:00
Paweł Gronowski
24da5233dd
integration: Fix Parallel before setupTest
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-17 13:06:05 +01:00
Bjorn Neergaard
d22c775e04
cdi: use separate feature-flag
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-17 11:31:37 +01:00
Brian Goff
353bccdf22
Merge pull request #47051 from corhere/reinit-health-on-live-restore 2024-01-16 10:08:20 -08:00
Paweł Gronowski
9de132dbf6
integration/image: Move Parallel after setupTest
setupTest should be called before Parallel as it modifies the test
environment which might produce:

```
fatal error: concurrent map writes

goroutine 143 [running]:
github.com/docker/docker/testutil/environment.(*Execution).ProtectContainer(...)
	/go/src/github.com/docker/docker/testutil/environment/protect.go:59
github.com/docker/docker/testutil/environment.ProtectContainers({0x12e8d98, 0xc00040e420}, {0x12f2878?, 0xc0004fc340}, 0xc0001fac00)
	/go/src/github.com/docker/docker/testutil/environment/protect.go:68 +0xb1
github.com/docker/docker/testutil/environment.ProtectAll({0x12e8d98, 0xc00040e210}, {0x12f2878, 0xc0004fc340}, 0xc0001fac00)
	/go/src/github.com/docker/docker/testutil/environment/protect.go:45 +0xf3
github.com/docker/docker/integration/image.setupTest(0xc0004fc340)
	/go/src/github.com/docker/docker/integration/image/main_test.go:46 +0x59
```

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-16 14:45:26 +01:00
Cory Snider
312450d079 integration: test container healthcheck is reset
Update the TestDaemonRestartKilContainers integration test to assert
that a container's healthcheck status is always reset to the Starting
state after a daemon restart, even when the container is live-restored.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-01-15 15:30:22 -05:00
Paweł Gronowski
c58acf06c2 image/save: Derive Descriptor from actual tar archive
Layer size is the sum of the individual files count, not the tar
archive. Use the total bytes read returned by `io.Copy` to populate the
`Size` field.

Also set the digest to the actual digest of the tar archive.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-15 16:35:54 +01:00
Paweł Gronowski
2c45df2ab4 image/save: Fix missing layers in manifest
The new OCI-compatible archive export relies on the Descriptors returned
by the layer (`distribution.Describable` interface implementation).

The issue with that is that the `roLayer` and the `referencedCacheLayer`
types don't implement this interface. Implementing that interface for
them based on their `descriptor` doesn't work though, because that
descriptor is empty.

To workaround this issue, just create a new descriptor if the one
provided by the layer is empty.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-12 17:10:23 +01:00
Sebastiaan van Stijn
7bc56c5365
Merge pull request #46853 from akerouanton/libnet-ep-dns-names
libnet: Endpoint: remove isAnonymous & myAliases
2023-12-20 19:53:16 +01:00
Albin Kerouanton
6a2542dacf
libnet: remove Endpoint.anonymous
No more concept of "anonymous endpoints". The equivalent is now an
endpoint with no DNSNames set.

Some of the code removed by this commit was mutating user-supplied
endpoint's Aliases to add container's short ID to that list. In order to
preserve backward compatibility for the ContainerInspect endpoint, this
commit also takes care of adding that short ID (and the container
hostname) to `EndpointSettings.Aliases` before returning the response.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-12-20 19:04:37 +01:00
Sebastiaan van Stijn
388216fc45
Merge pull request #46850 from robmry/46829-allow_ipv6_subnet_change
Allow overlapping change in bridge's IPv6 network.
2023-12-19 18:35:13 +01:00
Derek McGowan
35a51fd697
Update authz plugin test to not use httputil
Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-12-18 13:16:36 -08:00
Rob Murray
27f3abd893 Allow overlapping change in bridge's IPv6 network.
Calculate the IPv6 addreesses needed on a bridge, then reconcile them
with the addresses on an existing bridge by deleting then adding as
required.

(Previously, required addresses were added one-by-one, then unwanted
addresses were removed. This meant the daemon failed to start if, for
example, an existing bridge had address '2000:db8::/64' and the config
was changed to '2000:db8::/80'.)

IPv6 addresses are now calculated and applied in one go, so there's no
need for setupVerifyAndReconcile() to check the set of IPv6 addresses on
the bridge. And, it was guarded by !config.InhibitIPv4, which can't have
been right. So, removed its IPv6 parts, and added IPv4 to its name.

Link local addresses, the example given in the original ticket, are now
released when containers are stopped. Not releasing them meant that
when using an LL subnet on the default bridge, no container could be
started after a container was stopped (because the calculated address
could not be re-allocated). In non-default bridge networks using an
LL subnet, addresses leaked.

Linux always uses the standard 'fe80::/64' LL network. So, if a bridge
is configured with an LL subnet prefix that overlaps with it, a config
error is reported. Non-overlapping LL subnet prefixes are allowed.

Signed-off-by: Rob Murray <rob.murray@docker.com>
2023-12-18 16:10:41 +00:00
Paweł Gronowski
eaaf1ea96d
integration/prune: Run in a separate daemon
Isolate the prune effects by running the test in a separate daemon.
This minimizes the impact of/on other integration tests.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-12-07 17:33:26 +01:00
Sebastiaan van Stijn
82a318db5f
Merge pull request #46894 from vvoland/hack-load-specialimages
hack: Load special images on demand
2023-12-07 11:28:38 +01:00
Paweł Gronowski
bc94dfc7d2
hack: Load special images on demand
Rewrite `.build-empty-images` shell script that produced special images
(emptyfs with no layers, and empty danglign image) to a Go functions
that construct the same archives in a temporary directory.

Use them to load these images on demand only in the tests that need
them.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-12-06 17:16:37 +01:00
Paweł Gronowski
d5eba1bfe5
integration/build: Use setupTest
These tests build new images, setupTest sets up the test cleanup
function that clears the test environment from created images,
containers, etc.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-12-05 12:49:30 +01:00
Sebastiaan van Stijn
713c7d49a1
integration(-cli): remove skips for old daemon versions (<20.10)
This removes various skips that accounted for running the integration tests
against older versions of the daemon before 20.10 (API version v1.41). Those
versions are EOL, and we don't run tests against them.

This reverts most of e440831802, and similar
PRs.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-05 01:03:50 +01:00
Sebastiaan van Stijn
97549d923c
integration/container: TestInfoAPIVersioned: adjust API version
This test was using API version 1.20 to test old behavior, but the actual change
in behavior was API v1.25; see commit 6d98e344c7
and 63b5a37203.
This updates the test to use API v1.24 to test the old behavior.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-04 22:38:55 +01:00
Sebastiaan van Stijn
410feab962
integration/container: TestKillDifferentUserContainer: adjust API version
TestKillDifferentUserContainer was migrated from integration-cli in
commit 0855922cd3. Before migration, it
was not using a specific API version, so we can assume "current"
API version.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-04 22:38:52 +01:00