Commit graph

48522 commits

Author SHA1 Message Date
Rob Murray
dae33031e0 Only restore a configured MAC addr on restart.
The API's EndpointConfig struct has a MacAddress field that's used for
both the configured address, and the current address (which may be generated).

A configured address must be restored when a container is restarted, but a
generated address must not.

The previous attempt to differentiate between the two, without adding a field
to the API's EndpointConfig that would show up in 'inspect' output, was a
field in the daemon's version of EndpointSettings, MACOperational. It did
not work, MACOperational was set to true when a configured address was
used. So, while it ensured addresses were regenerated, it failed to preserve
a configured address.

So, this change removes that code, and adds DesiredMacAddress to the wrapped
version of EndpointSettings, where it is persisted but does not appear in
'inspect' results. Its value is copied from MacAddress (the API field) when
a container is created.

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-02-01 09:55:54 +00:00
Sebastiaan van Stijn
86198815a2
Merge pull request #47209 from corhere/sliceutil-map
internal/sliceutil: add utilities to map values
2024-01-25 11:51:50 +01:00
Sebastiaan van Stijn
d864df5a1d
Merge pull request #47208 from akerouanton/libnet-ds-remove-unused-key-params
libnet/ds: remove unused param `key` from `GetObject` and `List`
2024-01-25 11:46:20 +01:00
Cory Snider
e245fb76de internal/sliceutil: add utilities to map values
Functional programming for the win! Add a utility function to map the
values of a slice, along with a curried variant, to tide us over until
equivalent functionality gets added to the standard library
(https://go.dev/issue/61898)

Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-01-24 17:56:29 -05:00
Albin Kerouanton
3147a013fb libnet/ds: remove unused param key from List
Since 43dccc6 the `key` param is never used and can be safely
removed.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-01-24 22:42:18 +01:00
Albin Kerouanton
f7ef0e9fc7 libnet/ds: remove unused param key from GetObject
Since 43dccc6 the `key` param is never used and can be safely
removed.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-01-24 22:42:18 +01:00
Sebastiaan van Stijn
e8346c53d9
Merge pull request #46786 from rumpl/c8d-userns-namespace
c8d: Use a specific containerd namespace when userns are remapped
2024-01-24 20:36:40 +01:00
Djordje Lukic
3a617e5463
c8d: Use a specific containerd namespace when userns are remapped
We need to isolate the images that we are remapping to a userns, we
can't mix them with "normal" images. In the graph driver case this means
we create a new root directory where we store the images and everything
else, in the containerd case we can use a new namespace.

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2024-01-24 15:46:16 +01:00
Sebastiaan van Stijn
43ffb1ee9d
Merge pull request #47148 from thaJeztah/api_remove_deprecated_types
api/types: remove deprecated type-aliases
2024-01-24 12:40:27 +01:00
Sebastiaan van Stijn
115c7673dc
Merge pull request #47198 from thaJeztah/image_remove_IDFromDigest
image: remove deprecated IDFromDigest
2024-01-24 12:36:42 +01:00
Sebastiaan van Stijn
f7e2357745
image: remove deprecated IDFromDigest
This function was deprecated in 456ea1bb1d
(Docker v24.0), and is no longer used.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-24 01:54:59 +01:00
Sebastiaan van Stijn
314ea05f8d
Merge pull request #47197 from corhere/libn/carry-typo-fixes
libnetwork: carry typo fixes from moby/libnetwork repo
2024-01-24 01:31:08 +01:00
Sebastiaan van Stijn
13f46948dd
api/types: remove deprecated container-types
These types were deprecated in v25.0, and moved to api/types/container;

This patch removes the aliases for;

- api/types.ResizeOptions (deprecated in 95b92b1f97)
- api/types.ContainerAttachOptions (deprecated in 30f09b4a1a)
- api/types.ContainerCommitOptions (deprecated in 9498d897ab)
- api/types.ContainerRemoveOptions (deprecated in 0f77875220)
- api/types.ContainerStartOptions (deprecated in 7bce33eb0f)
- api/types.ContainerListOptions (deprecated in 9670d9364d)
- api/types.ContainerLogsOptions (deprecated in ebef4efb88)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-24 01:27:27 +01:00
Sebastiaan van Stijn
4b09bc2145
api/types: remove deprecated service-types
These types were deprecated in v25.0, and moved to api/types/swarm;

This patch removes the aliases for;

- api/types.ServiceUpdateResponse (deprecated in 5b3e6555a3)
- api/types.ServiceCreateResponse (deprecated in ec69501e94)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-24 01:27:27 +01:00
Sebastiaan van Stijn
49637d0206
api/types: remove deprecated image-types
These types were deprecated in 48cacbca24
(v25.0), and moved to api/types/image.

This patch removes the aliases for;

- api/types.ImageDeleteResponseItem
- api/types.ImageSummary
- api/types.ImageMetadata

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-24 01:27:27 +01:00
Sebastiaan van Stijn
eccb1a3eb8
api/types: remove deprecated checkpoint-types
These types were deprecated in b688af2226
(v25.0), and moved to api/types/checkpoint.

This patch removes the aliases for;

- api/types.CheckpointCreateOptions
- api/types.CheckpointListOptions
- api/types.CheckpointDeleteOptions
- api/types.Checkpoint

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-24 01:27:27 +01:00
Sebastiaan van Stijn
0b1921649f
api/types: remove deprecated system info types and functions
These types were deprecated in c90229ed9a
(v25.0), and moved to api/types/system.

This patch removes the aliases for;

- api/types.Info
- api/types.Commit
- api/types.PluginsInfo
- api/types.NetworkAddressPool
- api/types.Runtime
- api/types.SecurityOpt
- api/types.KeyValue
- api/types.DecodeSecurityOptions

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-24 01:27:24 +01:00
Sebastiaan van Stijn
4544bedea3
Merge pull request #47139 from thaJeztah/api_move_image_options
api/types: move image options to api/types/image
2024-01-24 01:26:55 +01:00
Cory Snider
6f44138269 libnetwork: fix tiny grammar mistake on design.md
Co-authored-by: Farhim Ferdous <37705070+AluBhorta@users.noreply.github.com>
Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-01-23 18:26:45 -05:00
Cory Snider
9a41cc58d9 libnetwork: fix typo in iptables.go
Co-authored-by: Ikko Ashimine <eltociear@gmail.com>
Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-01-23 18:25:08 -05:00
Sebastiaan van Stijn
ac2a028dcc
api/types: move image options to api/types/image
To prevent a circular import between api/types and api/types image,
the RequestPrivilegeFunc reference was not moved, but defined as
part of the PullOptions / PushOptions.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-24 00:10:33 +01:00
Sebastiaan van Stijn
8906adc8d4
Merge pull request #47138 from thaJeztah/move_image_backend_opt
api/types/image: move GetImageOpts to api/types/backend
2024-01-23 23:41:38 +01:00
Sebastiaan van Stijn
0bb84f5cef
Merge pull request #47195 from akerouanton/fix-multiple-rename-error
daemon: rename: don't reload endpoint from datastore
2024-01-23 23:41:07 +01:00
Albin Kerouanton
80c44b4b2e daemon: rename: don't reload endpoint from datastore
Commit 8b7af1d0f added some code to update the DNSNames of all
endpoints attached to a sandbox by loading a new instance of each
affected endpoints from the datastore through a call to
`Network.EndpointByID()`.

This method then calls `Network.getEndpointFromStore()`, that in
turn calls `store.GetObject()`, which then calls `cache.get()`,
which calls `o.CopyTo(kvObject)`. This effectively creates a fresh
new instance of an Endpoint. However, endpoints are already kept in
memory by Sandbox, meaning we now have two in-memory instances of
the same Endpoint.

As it turns out, libnetwork is built around the idea that no two objects
representing the same thing should leave in-memory, otherwise breaking
mutex locking and optimistic locking (as both instances will have a drifting
version tracking ID -- dbIndex in libnetwork parliance).

In this specific case, this bug materializes by container rename failing
when applied a second time for a given container. An integration test is
added to make sure this won't happen again.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-01-23 22:53:21 +01:00
Sebastiaan van Stijn
2da96d6c12
Merge pull request #47121 from voloder/master
Make sure that make doesn't rm -rf the system out of existence
2024-01-23 19:03:06 +01:00
Paweł Gronowski
0b64499a24
Merge pull request #47194 from vvoland/volume-cifs-resolve-optout-2
volume/local: Fix CIFS urls with spaces, add tests
2024-01-23 18:58:52 +01:00
Sebastiaan van Stijn
9763709c05
Merge pull request #47181 from akerouanton/fix-aliases-on-default-bridge
daemon: only add short cid to aliases for custom networks
2024-01-23 18:28:33 +01:00
Paweł Gronowski
250886741b
volume/local: Fix cifs url containing spaces
Unescapes the URL to avoid passing an URL encoded address to the kernel.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-23 17:42:11 +01:00
Paweł Gronowski
f4beb130b0
volume/local: Add tests for parsing nfs/cifs mounts
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-23 17:42:09 +01:00
Paweł Gronowski
df43311f3d
volume/local: Break early if addr was specified
I made a mistake in the last commit - after resolving the IP from the
passed `addr` for CIFS it would still resolve the `device` part.

Apply only one name resolution

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-23 17:17:53 +01:00
Albin Kerouanton
9f37672ca8 daemon: only add short cid to aliases for custom networks
Prior to 7a9b680a, the container short ID was added to the network
aliases only for custom networks. However, this logic wasn't preserved
in 6a2542d and now the cid is always added to the list of network
aliases.

This commit reintroduces the old logic.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-01-23 17:07:40 +01:00
Sebastiaan van Stijn
f19f233ca5
Merge pull request #47187 from thaJeztah/fix_gateway_ip
fix "host-gateway-ip" label not set for builder workers
2024-01-23 16:52:35 +01:00
Sebastiaan van Stijn
ca67dbd12c
Merge pull request #47185 from vvoland/volume-cifs-resolve-optout
volume/local: Make host resolution backwards compatible
2024-01-23 16:23:40 +01:00
Paweł Gronowski
cac52f7173
Merge pull request #47167 from vvoland/c8d-prefer-default-platform-snapshot
c8d/snapshot: Create any platform if not specified
2024-01-23 15:25:17 +01:00
Sebastiaan van Stijn
00c9785e2e
fix "host-gateway-ip" label not set for builder workers
Commit 21e50b89c9 added a label on the buildkit
worker to advertise the host-gateway-ip. This option can be either set by the
user in the daemon config, or otherwise defaults to the gateway-ip.

If no value is set by the user, discovery of the gateway-ip happens when
initializing the network-controller (`NewDaemon`, `daemon.restore()`).

However d222bf097c changed how we handle the
daemon config. As a result, the `cli.Config` used when initializing the
builder only holds configuration information form the daemon config
(user-specified or defaults), but is not updated with information set
by `NewDaemon`.

This patch adds an accessor on the daemon to get the current daemon config.
An alternative could be to return the config by `NewDaemon` (which should
likely be a _copy_ of the config).

Before this patch:

    docker buildx inspect default
    Name:   default
    Driver: docker

    Nodes:
    Name:      default
    Endpoint:  default
    Status:    running
    Buildkit:  v0.12.4+3b6880d2a00f
    Platforms: linux/arm64, linux/amd64, linux/amd64/v2, linux/riscv64, linux/ppc64le, linux/s390x, linux/386, linux/mips64le, linux/mips64, linux/arm/v7, linux/arm/v6
    Labels:
     org.mobyproject.buildkit.worker.moby.host-gateway-ip: <nil>

After this patch:

    docker buildx inspect default
    Name:   default
    Driver: docker

    Nodes:
    Name:      default
    Endpoint:  default
    Status:    running
    Buildkit:  v0.12.4+3b6880d2a00f
    Platforms: linux/arm64, linux/amd64, linux/amd64/v2, linux/riscv64, linux/ppc64le, linux/s390x, linux/386, linux/mips64le, linux/mips64, linux/arm/v7, linux/arm/v6
    Labels:
     org.mobyproject.buildkit.worker.moby.host-gateway-ip: 172.18.0.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-23 14:58:01 +01:00
Paweł Gronowski
0d51cf9db8
volume/local: Make host resolution backwards compatible
Commit 8ae94cafa5 added a DNS resolution
of the `device` part of the volume option.

The previous way to resolve the passed hostname was to use `addr`
option, which was handled by the same code path as the `nfs` mount type.

The issue is that `addr` is also an SMB module option handled by kernel
and passing a hostname as `addr` produces an invalid argument error.

To fix that, restore the old behavior to handle `addr` the same way as
before, and only perform the new DNS resolution of `device` if there is
no `addr` passed.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-23 14:49:05 +01:00
Sebastiaan van Stijn
1786517338
Merge pull request #47179 from thaJeztah/update_compose
Dockerfile: update docker compose to v2.24.2
2024-01-23 11:25:58 +01:00
Sebastiaan van Stijn
22a504935f
Merge pull request #45474 from thaJeztah/testing_cleanups
assorted test fixes and cleanups
2024-01-23 10:01:27 +01:00
Sebastiaan van Stijn
05d952b246
Dockerfile: update docker compose to v2.24.2
Update the version of compose used in CI to the latest version.

- full diff: docker/compose@v2.24.1...v2.24.2
- release notes: https://github.com/docker/compose/releases/tag/v2.24.2

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-23 09:51:41 +01:00
Sebastiaan van Stijn
d86d24de35
Merge pull request #47174 from corhere/richer-xattr-errors
pkg/system: return even richer xattr errors
2024-01-23 09:46:12 +01:00
Sebastiaan van Stijn
20bd690844
integration-cli: simplify test-file creation
Also fixes some potentially unclosed file-handles,
inlines some variables, and use consts for fixed
values.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-22 21:43:30 +01:00
Sebastiaan van Stijn
34668a5945
pkg/archive: fixe some unclosed file-handles in tests
Also fixing a "defer in loop" warning, instead changing to use
sub-tests, and simplifying some code, using os.WriteFile() instead.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-22 21:43:29 +01:00
Sebastiaan van Stijn
1090aaaedd
libnetwork: fix some unclosed file-handles in tests
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-22 21:43:29 +01:00
Sebastiaan van Stijn
c482383458
fix some leaking mounts in tests
This should help with errors such as:

    === RUN   TestSysctlOverride
        testing.go:1090: TempDir RemoveAll cleanup: unlinkat /tmp/TestSysctlOverride3702360633/001/mounts/shm: device or resource busy
    --- FAIL: TestSysctlOverride (0.00s)

    === RUN   TestSysctlOverrideHost
        testing.go:1090: TempDir RemoveAll cleanup: unlinkat /tmp/TestSysctlOverrideHost226485533/001/mounts/shm: device or resource busy
    --- FAIL: TestSysctlOverrideHost (0.00s)

    === RUN   TestDockerSuite/TestRunWithVolumesIsRecursive
        testing.go:1090: TempDir RemoveAll cleanup: unlinkat /tmp/TestDockerSuiteTestRunWithVolumesIsRecursive1156692230/001/tmpfs: device or resource busy
        --- FAIL: TestDockerSuite/TestRunWithVolumesIsRecursive (0.49s)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-22 21:43:23 +01:00
Cory Snider
43bf65c174 pkg/system: return even richer xattr errors
The names of extended attributes are not completely freeform. Attributes
are namespaced, and the kernel enforces (among other things) that only
attributes whose names are prefixed with a valid namespace are
permitted. The name of the attribute therefore needs to be known in
order to diagnose issues with lsetxattr. Include the name of the
extended attribute in the errors returned from the Lsetxattr and
Lgetxattr so users and us can more easily troubleshoot xattr-related
issues. Include the name in a separate rich-error field to provide code
handling the error enough information to determine whether or not the
failure can be ignored.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-01-22 15:25:10 -05:00
Sebastiaan van Stijn
a3a42c459e
api/types/image: move GetImageOpts to api/types/backend
The `GetImageOpts` struct is used for options to be passed to the backend,
and are not used in client code. This struct currently is intended for internal
use only.

This patch moves the `GetImageOpts` struct to the backend package to prevent
it being imported in the client, and to make it more clear that this is part
of internal APIs, and not public-facing.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-22 20:45:21 +01:00
Sebastiaan van Stijn
c87e0ad209
Merge pull request #47168 from robmry/47146-duplicate_mac_addrs
Remove generated MAC addresses on restart.
2024-01-22 19:48:24 +01:00
Rob Murray
cd53b7380c Remove generated MAC addresses on restart.
The MAC address of a running container was stored in the same place as
the configured address for a container.

When starting a stopped container, a generated address was treated as a
configured address. If that generated address (based on an IPAM-assigned
IP address) had been reused, the containers ended up with duplicate MAC
addresses.

So, remember whether the MAC address was explicitly configured, and
clear it if not.

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-01-22 17:52:20 +00:00
Paweł Gronowski
fb19f1fc20
c8d/snapshot: Create any platform if not specified
With containerd snapshotters enabled `docker run` currently fails when
creating a container from an image that doesn't have the default host
platform without an explicit `--platform` selection:

```
$ docker run image:amd64
Unable to find image 'asdf:amd64' locally
docker: Error response from daemon: pull access denied for asdf, repository does not exist or may require 'docker login'.
See 'docker run --help'.
```

This is confusing and the graphdriver behavior is much better here,
because it runs whatever platform the image has, but prints a warning:

```
$ docker run image:amd64
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
```

This commits changes the containerd snapshotter behavior to be the same
as the graphdriver. This doesn't affect container creation when platform
is specified explicitly.

```
$ docker run --rm --platform linux/arm64 asdf:amd64
Unable to find image 'asdf:amd64' locally
docker: Error response from daemon: pull access denied for asdf, repository does not exist or may require 'docker login'.
See 'docker run --help'.
```

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-22 16:15:07 +01:00
Sebastiaan van Stijn
3602ba0afd
Merge pull request #47162 from vvoland/25-fix-swarm-startinterval
daemon/cluster/executer: Add missing `StartInterval`
2024-01-22 15:51:37 +01:00