Commit graph

48502 commits

Author SHA1 Message Date
Sebastiaan van Stijn
ac2a028dcc
api/types: move image options to api/types/image
To prevent a circular import between api/types and api/types image,
the RequestPrivilegeFunc reference was not moved, but defined as
part of the PullOptions / PushOptions.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-24 00:10:33 +01:00
Sebastiaan van Stijn
8906adc8d4
Merge pull request #47138 from thaJeztah/move_image_backend_opt
api/types/image: move GetImageOpts to api/types/backend
2024-01-23 23:41:38 +01:00
Sebastiaan van Stijn
0bb84f5cef
Merge pull request #47195 from akerouanton/fix-multiple-rename-error
daemon: rename: don't reload endpoint from datastore
2024-01-23 23:41:07 +01:00
Albin Kerouanton
80c44b4b2e daemon: rename: don't reload endpoint from datastore
Commit 8b7af1d0f added some code to update the DNSNames of all
endpoints attached to a sandbox by loading a new instance of each
affected endpoints from the datastore through a call to
`Network.EndpointByID()`.

This method then calls `Network.getEndpointFromStore()`, that in
turn calls `store.GetObject()`, which then calls `cache.get()`,
which calls `o.CopyTo(kvObject)`. This effectively creates a fresh
new instance of an Endpoint. However, endpoints are already kept in
memory by Sandbox, meaning we now have two in-memory instances of
the same Endpoint.

As it turns out, libnetwork is built around the idea that no two objects
representing the same thing should leave in-memory, otherwise breaking
mutex locking and optimistic locking (as both instances will have a drifting
version tracking ID -- dbIndex in libnetwork parliance).

In this specific case, this bug materializes by container rename failing
when applied a second time for a given container. An integration test is
added to make sure this won't happen again.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-01-23 22:53:21 +01:00
Sebastiaan van Stijn
2da96d6c12
Merge pull request #47121 from voloder/master
Make sure that make doesn't rm -rf the system out of existence
2024-01-23 19:03:06 +01:00
Paweł Gronowski
0b64499a24
Merge pull request #47194 from vvoland/volume-cifs-resolve-optout-2
volume/local: Fix CIFS urls with spaces, add tests
2024-01-23 18:58:52 +01:00
Sebastiaan van Stijn
9763709c05
Merge pull request #47181 from akerouanton/fix-aliases-on-default-bridge
daemon: only add short cid to aliases for custom networks
2024-01-23 18:28:33 +01:00
Paweł Gronowski
250886741b
volume/local: Fix cifs url containing spaces
Unescapes the URL to avoid passing an URL encoded address to the kernel.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-23 17:42:11 +01:00
Paweł Gronowski
f4beb130b0
volume/local: Add tests for parsing nfs/cifs mounts
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-23 17:42:09 +01:00
Paweł Gronowski
df43311f3d
volume/local: Break early if addr was specified
I made a mistake in the last commit - after resolving the IP from the
passed `addr` for CIFS it would still resolve the `device` part.

Apply only one name resolution

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-23 17:17:53 +01:00
Albin Kerouanton
9f37672ca8 daemon: only add short cid to aliases for custom networks
Prior to 7a9b680a, the container short ID was added to the network
aliases only for custom networks. However, this logic wasn't preserved
in 6a2542d and now the cid is always added to the list of network
aliases.

This commit reintroduces the old logic.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-01-23 17:07:40 +01:00
Sebastiaan van Stijn
f19f233ca5
Merge pull request #47187 from thaJeztah/fix_gateway_ip
fix "host-gateway-ip" label not set for builder workers
2024-01-23 16:52:35 +01:00
Sebastiaan van Stijn
ca67dbd12c
Merge pull request #47185 from vvoland/volume-cifs-resolve-optout
volume/local: Make host resolution backwards compatible
2024-01-23 16:23:40 +01:00
Paweł Gronowski
cac52f7173
Merge pull request #47167 from vvoland/c8d-prefer-default-platform-snapshot
c8d/snapshot: Create any platform if not specified
2024-01-23 15:25:17 +01:00
Sebastiaan van Stijn
00c9785e2e
fix "host-gateway-ip" label not set for builder workers
Commit 21e50b89c9 added a label on the buildkit
worker to advertise the host-gateway-ip. This option can be either set by the
user in the daemon config, or otherwise defaults to the gateway-ip.

If no value is set by the user, discovery of the gateway-ip happens when
initializing the network-controller (`NewDaemon`, `daemon.restore()`).

However d222bf097c changed how we handle the
daemon config. As a result, the `cli.Config` used when initializing the
builder only holds configuration information form the daemon config
(user-specified or defaults), but is not updated with information set
by `NewDaemon`.

This patch adds an accessor on the daemon to get the current daemon config.
An alternative could be to return the config by `NewDaemon` (which should
likely be a _copy_ of the config).

Before this patch:

    docker buildx inspect default
    Name:   default
    Driver: docker

    Nodes:
    Name:      default
    Endpoint:  default
    Status:    running
    Buildkit:  v0.12.4+3b6880d2a00f
    Platforms: linux/arm64, linux/amd64, linux/amd64/v2, linux/riscv64, linux/ppc64le, linux/s390x, linux/386, linux/mips64le, linux/mips64, linux/arm/v7, linux/arm/v6
    Labels:
     org.mobyproject.buildkit.worker.moby.host-gateway-ip: <nil>

After this patch:

    docker buildx inspect default
    Name:   default
    Driver: docker

    Nodes:
    Name:      default
    Endpoint:  default
    Status:    running
    Buildkit:  v0.12.4+3b6880d2a00f
    Platforms: linux/arm64, linux/amd64, linux/amd64/v2, linux/riscv64, linux/ppc64le, linux/s390x, linux/386, linux/mips64le, linux/mips64, linux/arm/v7, linux/arm/v6
    Labels:
     org.mobyproject.buildkit.worker.moby.host-gateway-ip: 172.18.0.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-23 14:58:01 +01:00
Paweł Gronowski
0d51cf9db8
volume/local: Make host resolution backwards compatible
Commit 8ae94cafa5 added a DNS resolution
of the `device` part of the volume option.

The previous way to resolve the passed hostname was to use `addr`
option, which was handled by the same code path as the `nfs` mount type.

The issue is that `addr` is also an SMB module option handled by kernel
and passing a hostname as `addr` produces an invalid argument error.

To fix that, restore the old behavior to handle `addr` the same way as
before, and only perform the new DNS resolution of `device` if there is
no `addr` passed.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-23 14:49:05 +01:00
Sebastiaan van Stijn
1786517338
Merge pull request #47179 from thaJeztah/update_compose
Dockerfile: update docker compose to v2.24.2
2024-01-23 11:25:58 +01:00
Sebastiaan van Stijn
22a504935f
Merge pull request #45474 from thaJeztah/testing_cleanups
assorted test fixes and cleanups
2024-01-23 10:01:27 +01:00
Sebastiaan van Stijn
05d952b246
Dockerfile: update docker compose to v2.24.2
Update the version of compose used in CI to the latest version.

- full diff: docker/compose@v2.24.1...v2.24.2
- release notes: https://github.com/docker/compose/releases/tag/v2.24.2

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-23 09:51:41 +01:00
Sebastiaan van Stijn
d86d24de35
Merge pull request #47174 from corhere/richer-xattr-errors
pkg/system: return even richer xattr errors
2024-01-23 09:46:12 +01:00
Sebastiaan van Stijn
20bd690844
integration-cli: simplify test-file creation
Also fixes some potentially unclosed file-handles,
inlines some variables, and use consts for fixed
values.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-22 21:43:30 +01:00
Sebastiaan van Stijn
34668a5945
pkg/archive: fixe some unclosed file-handles in tests
Also fixing a "defer in loop" warning, instead changing to use
sub-tests, and simplifying some code, using os.WriteFile() instead.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-22 21:43:29 +01:00
Sebastiaan van Stijn
1090aaaedd
libnetwork: fix some unclosed file-handles in tests
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-22 21:43:29 +01:00
Sebastiaan van Stijn
c482383458
fix some leaking mounts in tests
This should help with errors such as:

    === RUN   TestSysctlOverride
        testing.go:1090: TempDir RemoveAll cleanup: unlinkat /tmp/TestSysctlOverride3702360633/001/mounts/shm: device or resource busy
    --- FAIL: TestSysctlOverride (0.00s)

    === RUN   TestSysctlOverrideHost
        testing.go:1090: TempDir RemoveAll cleanup: unlinkat /tmp/TestSysctlOverrideHost226485533/001/mounts/shm: device or resource busy
    --- FAIL: TestSysctlOverrideHost (0.00s)

    === RUN   TestDockerSuite/TestRunWithVolumesIsRecursive
        testing.go:1090: TempDir RemoveAll cleanup: unlinkat /tmp/TestDockerSuiteTestRunWithVolumesIsRecursive1156692230/001/tmpfs: device or resource busy
        --- FAIL: TestDockerSuite/TestRunWithVolumesIsRecursive (0.49s)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-22 21:43:23 +01:00
Cory Snider
43bf65c174 pkg/system: return even richer xattr errors
The names of extended attributes are not completely freeform. Attributes
are namespaced, and the kernel enforces (among other things) that only
attributes whose names are prefixed with a valid namespace are
permitted. The name of the attribute therefore needs to be known in
order to diagnose issues with lsetxattr. Include the name of the
extended attribute in the errors returned from the Lsetxattr and
Lgetxattr so users and us can more easily troubleshoot xattr-related
issues. Include the name in a separate rich-error field to provide code
handling the error enough information to determine whether or not the
failure can be ignored.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-01-22 15:25:10 -05:00
Sebastiaan van Stijn
a3a42c459e
api/types/image: move GetImageOpts to api/types/backend
The `GetImageOpts` struct is used for options to be passed to the backend,
and are not used in client code. This struct currently is intended for internal
use only.

This patch moves the `GetImageOpts` struct to the backend package to prevent
it being imported in the client, and to make it more clear that this is part
of internal APIs, and not public-facing.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-22 20:45:21 +01:00
Sebastiaan van Stijn
c87e0ad209
Merge pull request #47168 from robmry/47146-duplicate_mac_addrs
Remove generated MAC addresses on restart.
2024-01-22 19:48:24 +01:00
Rob Murray
cd53b7380c Remove generated MAC addresses on restart.
The MAC address of a running container was stored in the same place as
the configured address for a container.

When starting a stopped container, a generated address was treated as a
configured address. If that generated address (based on an IPAM-assigned
IP address) had been reused, the containers ended up with duplicate MAC
addresses.

So, remember whether the MAC address was explicitly configured, and
clear it if not.

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-01-22 17:52:20 +00:00
Paweł Gronowski
fb19f1fc20
c8d/snapshot: Create any platform if not specified
With containerd snapshotters enabled `docker run` currently fails when
creating a container from an image that doesn't have the default host
platform without an explicit `--platform` selection:

```
$ docker run image:amd64
Unable to find image 'asdf:amd64' locally
docker: Error response from daemon: pull access denied for asdf, repository does not exist or may require 'docker login'.
See 'docker run --help'.
```

This is confusing and the graphdriver behavior is much better here,
because it runs whatever platform the image has, but prints a warning:

```
$ docker run image:amd64
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
```

This commits changes the containerd snapshotter behavior to be the same
as the graphdriver. This doesn't affect container creation when platform
is specified explicitly.

```
$ docker run --rm --platform linux/arm64 asdf:amd64
Unable to find image 'asdf:amd64' locally
docker: Error response from daemon: pull access denied for asdf, repository does not exist or may require 'docker login'.
See 'docker run --help'.
```

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-22 16:15:07 +01:00
Sebastiaan van Stijn
3602ba0afd
Merge pull request #47162 from vvoland/25-fix-swarm-startinterval
daemon/cluster/executer: Add missing `StartInterval`
2024-01-22 15:51:37 +01:00
Sebastiaan van Stijn
f0eef50273
Merge pull request #47159 from akerouanton/fix-bad-http-code
daemon: return an InvalidParameter error when ep settings are wrong
2024-01-22 15:06:06 +01:00
Sebastiaan van Stijn
b6a5c2968b
Merge pull request #47160 from vvoland/save-fix-oci-diffids
image/save: Fix layers order in OCI manifest
2024-01-22 15:05:06 +01:00
Paweł Gronowski
6100190e5c
daemon/cluster/executer: Add missing StartInterval
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-22 14:42:17 +01:00
Paweł Gronowski
17fd6562bf
image/save: Fix layers order in OCI manifest
Order the layers in OCI manifest by their actual apply order. This is
required by the OCI image spec.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-22 13:48:12 +01:00
Paweł Gronowski
4979605212
image/save: Change layers type to DiffID
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-22 13:47:57 +01:00
Albin Kerouanton
fcc651972e
daemon: return an InvalidParameter error when ep settings are wrong
Since v25.0 (commit ff50388), we validate endpoint settings when
containers are created, instead of doing so when containers are started.
However, a container created prior to that release would still trigger
validation error at start-time. In such case, the API returns a 500
status code because the Go error isn't wrapped into an InvalidParameter
error. This is now fixed.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-01-22 12:48:23 +01:00
Sebastiaan van Stijn
42a07883f7
Merge pull request #47154 from thaJeztah/integration_cli_api_versions
integration-cli: adjust inspect tests to use current API version
2024-01-22 11:07:45 +01:00
Sebastiaan van Stijn
5a3a101af2
Merge pull request #47151 from thaJeztah/fix_TestPutContainerArchiveErrSymlinkInVolumeToReadOnlyRootfs
integration-cli: TestPutContainerArchiveErrSymlinkInVolumeToReadOnlyRootfs: use current API
2024-01-22 10:23:50 +01:00
Sebastiaan van Stijn
6803dd8c90
Merge pull request #47124 from thaJeztah/remove_deprecated_api_docs
docs: remove documentation for deprecated API versions (v1.23 and before)
2024-01-22 10:23:31 +01:00
Sebastiaan van Stijn
82acb922cb
Merge pull request #47147 from thaJeztah/integration_minor_refactor
integration: improve some asserts, and add asserts for unhandled errs
2024-01-22 10:23:03 +01:00
Akihiro Suda
e5dbe10e65
Merge pull request #47123 from thaJeztah/cli_25
Dockerfile: update docker-cli to v25.0.0, docker compose v2.24.1
2024-01-22 11:05:43 +09:00
Akihiro Suda
8528ed3409
Merge pull request #47128 from thaJeztah/remove_pkg_loopback
remove deprecated pkg/loopback (utility package for devicemapper)
2024-01-22 11:05:15 +09:00
Akihiro Suda
570b8a794c
Merge pull request #47129 from thaJeztah/pkg_system_deprecated
pkg/system: remove deprecated ErrNotSupportedOperatingSystem, IsOSSupported
2024-01-22 11:04:25 +09:00
Akihiro Suda
5ad5334c2e
Merge pull request #47130 from thaJeztah/pkg_homedir_deprecated
pkg/homedir: remove deprecated Key() and GetShortcutString()
2024-01-22 11:03:54 +09:00
Akihiro Suda
417826376f
Merge pull request #47131 from thaJeztah/pkg_containerfs_deprecated
pkg/containerfs: remove deprecated ResolveScopedPath
2024-01-22 11:03:23 +09:00
Akihiro Suda
e30610aaa3
Merge pull request #47143 from thaJeztah/golang_x_updates
vendor: assorted golang.org/x/... updates
2024-01-22 11:02:48 +09:00
Sebastiaan van Stijn
a0466ca8e1
integration-cli: TestInspectAPIMultipleNetworks: use current version
This test was added in f301c5765a to test
inspect output for API > v1.21, however, it was pinned to API v1.21,
which is now deprecated.

Remove the fixed version, as the intent was to test "current" API versions
(API v1.21 and up),

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-21 20:42:36 +01:00
Sebastiaan van Stijn
13a384a6fa
integration-cli: TestInspectAPIBridgeNetworkSettings121: use current version
This test was added in f301c5765a to test
inspect output for API > v1.21, however, it was pinned to API v1.21,
which is now deprecated.

Remove the fixed version, as the intent was to test "current" API versions
(API v1.21 and up),

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-21 20:42:36 +01:00
Sebastiaan van Stijn
52e3fff828
integration-cli: TestPutContainerArchiveErrSymlinkInVolumeToReadOnlyRootfs: use current API
This test was added in 75f6929b44, but pinned
to the API version that was current at the time (v1.20), which is now
deprecated.

Update the test to use the current API version.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-21 15:37:31 +01:00
Sebastiaan van Stijn
521123944a
docs/api: remove version matrices from swagger files
These tables linked to deprecated API versions, and an up-to-date version of
the matrix is already included at https://docs.docker.com/engine/api/#api-version-matrix

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-21 15:17:53 +01:00