Commit graph

48484 commits

Author SHA1 Message Date
Sebastiaan van Stijn
00c9785e2e
fix "host-gateway-ip" label not set for builder workers
Commit 21e50b89c9 added a label on the buildkit
worker to advertise the host-gateway-ip. This option can be either set by the
user in the daemon config, or otherwise defaults to the gateway-ip.

If no value is set by the user, discovery of the gateway-ip happens when
initializing the network-controller (`NewDaemon`, `daemon.restore()`).

However d222bf097c changed how we handle the
daemon config. As a result, the `cli.Config` used when initializing the
builder only holds configuration information form the daemon config
(user-specified or defaults), but is not updated with information set
by `NewDaemon`.

This patch adds an accessor on the daemon to get the current daemon config.
An alternative could be to return the config by `NewDaemon` (which should
likely be a _copy_ of the config).

Before this patch:

    docker buildx inspect default
    Name:   default
    Driver: docker

    Nodes:
    Name:      default
    Endpoint:  default
    Status:    running
    Buildkit:  v0.12.4+3b6880d2a00f
    Platforms: linux/arm64, linux/amd64, linux/amd64/v2, linux/riscv64, linux/ppc64le, linux/s390x, linux/386, linux/mips64le, linux/mips64, linux/arm/v7, linux/arm/v6
    Labels:
     org.mobyproject.buildkit.worker.moby.host-gateway-ip: <nil>

After this patch:

    docker buildx inspect default
    Name:   default
    Driver: docker

    Nodes:
    Name:      default
    Endpoint:  default
    Status:    running
    Buildkit:  v0.12.4+3b6880d2a00f
    Platforms: linux/arm64, linux/amd64, linux/amd64/v2, linux/riscv64, linux/ppc64le, linux/s390x, linux/386, linux/mips64le, linux/mips64, linux/arm/v7, linux/arm/v6
    Labels:
     org.mobyproject.buildkit.worker.moby.host-gateway-ip: 172.18.0.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-23 14:58:01 +01:00
Sebastiaan van Stijn
1786517338
Merge pull request #47179 from thaJeztah/update_compose
Dockerfile: update docker compose to v2.24.2
2024-01-23 11:25:58 +01:00
Sebastiaan van Stijn
22a504935f
Merge pull request #45474 from thaJeztah/testing_cleanups
assorted test fixes and cleanups
2024-01-23 10:01:27 +01:00
Sebastiaan van Stijn
05d952b246
Dockerfile: update docker compose to v2.24.2
Update the version of compose used in CI to the latest version.

- full diff: docker/compose@v2.24.1...v2.24.2
- release notes: https://github.com/docker/compose/releases/tag/v2.24.2

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-23 09:51:41 +01:00
Sebastiaan van Stijn
d86d24de35
Merge pull request #47174 from corhere/richer-xattr-errors
pkg/system: return even richer xattr errors
2024-01-23 09:46:12 +01:00
Sebastiaan van Stijn
20bd690844
integration-cli: simplify test-file creation
Also fixes some potentially unclosed file-handles,
inlines some variables, and use consts for fixed
values.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-22 21:43:30 +01:00
Sebastiaan van Stijn
34668a5945
pkg/archive: fixe some unclosed file-handles in tests
Also fixing a "defer in loop" warning, instead changing to use
sub-tests, and simplifying some code, using os.WriteFile() instead.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-22 21:43:29 +01:00
Sebastiaan van Stijn
1090aaaedd
libnetwork: fix some unclosed file-handles in tests
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-22 21:43:29 +01:00
Sebastiaan van Stijn
c482383458
fix some leaking mounts in tests
This should help with errors such as:

    === RUN   TestSysctlOverride
        testing.go:1090: TempDir RemoveAll cleanup: unlinkat /tmp/TestSysctlOverride3702360633/001/mounts/shm: device or resource busy
    --- FAIL: TestSysctlOverride (0.00s)

    === RUN   TestSysctlOverrideHost
        testing.go:1090: TempDir RemoveAll cleanup: unlinkat /tmp/TestSysctlOverrideHost226485533/001/mounts/shm: device or resource busy
    --- FAIL: TestSysctlOverrideHost (0.00s)

    === RUN   TestDockerSuite/TestRunWithVolumesIsRecursive
        testing.go:1090: TempDir RemoveAll cleanup: unlinkat /tmp/TestDockerSuiteTestRunWithVolumesIsRecursive1156692230/001/tmpfs: device or resource busy
        --- FAIL: TestDockerSuite/TestRunWithVolumesIsRecursive (0.49s)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-22 21:43:23 +01:00
Cory Snider
43bf65c174 pkg/system: return even richer xattr errors
The names of extended attributes are not completely freeform. Attributes
are namespaced, and the kernel enforces (among other things) that only
attributes whose names are prefixed with a valid namespace are
permitted. The name of the attribute therefore needs to be known in
order to diagnose issues with lsetxattr. Include the name of the
extended attribute in the errors returned from the Lsetxattr and
Lgetxattr so users and us can more easily troubleshoot xattr-related
issues. Include the name in a separate rich-error field to provide code
handling the error enough information to determine whether or not the
failure can be ignored.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-01-22 15:25:10 -05:00
Sebastiaan van Stijn
c87e0ad209
Merge pull request #47168 from robmry/47146-duplicate_mac_addrs
Remove generated MAC addresses on restart.
2024-01-22 19:48:24 +01:00
Rob Murray
cd53b7380c Remove generated MAC addresses on restart.
The MAC address of a running container was stored in the same place as
the configured address for a container.

When starting a stopped container, a generated address was treated as a
configured address. If that generated address (based on an IPAM-assigned
IP address) had been reused, the containers ended up with duplicate MAC
addresses.

So, remember whether the MAC address was explicitly configured, and
clear it if not.

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-01-22 17:52:20 +00:00
Sebastiaan van Stijn
3602ba0afd
Merge pull request #47162 from vvoland/25-fix-swarm-startinterval
daemon/cluster/executer: Add missing `StartInterval`
2024-01-22 15:51:37 +01:00
Sebastiaan van Stijn
f0eef50273
Merge pull request #47159 from akerouanton/fix-bad-http-code
daemon: return an InvalidParameter error when ep settings are wrong
2024-01-22 15:06:06 +01:00
Sebastiaan van Stijn
b6a5c2968b
Merge pull request #47160 from vvoland/save-fix-oci-diffids
image/save: Fix layers order in OCI manifest
2024-01-22 15:05:06 +01:00
Paweł Gronowski
6100190e5c
daemon/cluster/executer: Add missing StartInterval
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-22 14:42:17 +01:00
Paweł Gronowski
17fd6562bf
image/save: Fix layers order in OCI manifest
Order the layers in OCI manifest by their actual apply order. This is
required by the OCI image spec.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-22 13:48:12 +01:00
Paweł Gronowski
4979605212
image/save: Change layers type to DiffID
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-22 13:47:57 +01:00
Albin Kerouanton
fcc651972e
daemon: return an InvalidParameter error when ep settings are wrong
Since v25.0 (commit ff50388), we validate endpoint settings when
containers are created, instead of doing so when containers are started.
However, a container created prior to that release would still trigger
validation error at start-time. In such case, the API returns a 500
status code because the Go error isn't wrapped into an InvalidParameter
error. This is now fixed.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-01-22 12:48:23 +01:00
Sebastiaan van Stijn
42a07883f7
Merge pull request #47154 from thaJeztah/integration_cli_api_versions
integration-cli: adjust inspect tests to use current API version
2024-01-22 11:07:45 +01:00
Sebastiaan van Stijn
5a3a101af2
Merge pull request #47151 from thaJeztah/fix_TestPutContainerArchiveErrSymlinkInVolumeToReadOnlyRootfs
integration-cli: TestPutContainerArchiveErrSymlinkInVolumeToReadOnlyRootfs: use current API
2024-01-22 10:23:50 +01:00
Sebastiaan van Stijn
6803dd8c90
Merge pull request #47124 from thaJeztah/remove_deprecated_api_docs
docs: remove documentation for deprecated API versions (v1.23 and before)
2024-01-22 10:23:31 +01:00
Sebastiaan van Stijn
82acb922cb
Merge pull request #47147 from thaJeztah/integration_minor_refactor
integration: improve some asserts, and add asserts for unhandled errs
2024-01-22 10:23:03 +01:00
Akihiro Suda
e5dbe10e65
Merge pull request #47123 from thaJeztah/cli_25
Dockerfile: update docker-cli to v25.0.0, docker compose v2.24.1
2024-01-22 11:05:43 +09:00
Akihiro Suda
8528ed3409
Merge pull request #47128 from thaJeztah/remove_pkg_loopback
remove deprecated pkg/loopback (utility package for devicemapper)
2024-01-22 11:05:15 +09:00
Akihiro Suda
570b8a794c
Merge pull request #47129 from thaJeztah/pkg_system_deprecated
pkg/system: remove deprecated ErrNotSupportedOperatingSystem, IsOSSupported
2024-01-22 11:04:25 +09:00
Akihiro Suda
5ad5334c2e
Merge pull request #47130 from thaJeztah/pkg_homedir_deprecated
pkg/homedir: remove deprecated Key() and GetShortcutString()
2024-01-22 11:03:54 +09:00
Akihiro Suda
417826376f
Merge pull request #47131 from thaJeztah/pkg_containerfs_deprecated
pkg/containerfs: remove deprecated ResolveScopedPath
2024-01-22 11:03:23 +09:00
Akihiro Suda
e30610aaa3
Merge pull request #47143 from thaJeztah/golang_x_updates
vendor: assorted golang.org/x/... updates
2024-01-22 11:02:48 +09:00
Sebastiaan van Stijn
a0466ca8e1
integration-cli: TestInspectAPIMultipleNetworks: use current version
This test was added in f301c5765a to test
inspect output for API > v1.21, however, it was pinned to API v1.21,
which is now deprecated.

Remove the fixed version, as the intent was to test "current" API versions
(API v1.21 and up),

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-21 20:42:36 +01:00
Sebastiaan van Stijn
13a384a6fa
integration-cli: TestInspectAPIBridgeNetworkSettings121: use current version
This test was added in f301c5765a to test
inspect output for API > v1.21, however, it was pinned to API v1.21,
which is now deprecated.

Remove the fixed version, as the intent was to test "current" API versions
(API v1.21 and up),

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-21 20:42:36 +01:00
Sebastiaan van Stijn
52e3fff828
integration-cli: TestPutContainerArchiveErrSymlinkInVolumeToReadOnlyRootfs: use current API
This test was added in 75f6929b44, but pinned
to the API version that was current at the time (v1.20), which is now
deprecated.

Update the test to use the current API version.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-21 15:37:31 +01:00
Sebastiaan van Stijn
521123944a
docs/api: remove version matrices from swagger files
These tables linked to deprecated API versions, and an up-to-date version of
the matrix is already included at https://docs.docker.com/engine/api/#api-version-matrix

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-21 15:17:53 +01:00
Sebastiaan van Stijn
d54be2ee6d
docs: remove documentation for deprecated API versions < v1.23
These versions are deprecated in v25.0.0, and disabled by default,
see 08e4e88482.

Users that need to refer to documentation for older API versions,
can use archived versions of the documentation on GitHub:

- API v1.23 and before: https://github.com/moby/moby/tree/v25.0.0/docs/api
- API v1.17 and before: https://github.com/moby/moby/tree/v1.9.1/docs/reference/api

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-21 15:12:42 +01:00
Sebastiaan van Stijn
64a6cc3afd
integration/build: improve some asserts, and add asserts for unhandled errs
- add some asserts for unhandled errors
- use consts for fixed values, and slightly re-format Dockerfile contentt
- inline one-line Dockerfiles
- fix some vars to be properly camel-cased
- improve assert for error-types;

Before:

    === RUN   TestBuildPlatformInvalid
        build_test.go:685: assertion failed: expression is false: errdefs.IsInvalidParameter(err)
    --- FAIL: TestBuildPlatformInvalid (0.01s)
    FAIL

After:

    === RUN   TestBuildPlatformInvalid
        build_test.go:689: assertion failed: error is Error response from daemon: "foobar": unknown operating system or architecture: invalid argument (errdefs.errSystem), not errdefs.IsInvalidParameter
    --- FAIL: TestBuildPlatformInvalid (0.01s)
    FAIL

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-21 13:45:06 +01:00
Sebastiaan van Stijn
a88cd68d3e
integration/images: improve some asserts, and add asserts for unhandled errs
Before:

    === FAIL: amd64.integration.image TestImagePullPlatformInvalid (0.01s)
        pull_test.go:37: assertion failed: expression is false: errdefs.IsInvalidParameter(err)

After:

    === RUN   TestImagePullPlatformInvalid
        pull_test.go:37: assertion failed: error is Error response from daemon: "foobar": unknown operating system or architecture: invalid argument (errdefs.errSystem), not errdefs.IsInvalidParameter

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-21 13:45:06 +01:00
Sebastiaan van Stijn
27e85c7b68
Merge pull request #47141 from thaJeztah/internalize_pkg_platforms
pkg/platforms: internalize in daemon/containerd
2024-01-20 23:47:48 +01:00
Sebastiaan van Stijn
a404017a86
vendor: golang.org/x/tools v0.14.0
full diff: https://github.com/golang/tools/comopare/v0.13.0...v0.14.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-20 22:58:31 +01:00
Sebastiaan van Stijn
41a2aa2ee2
vendor: golang.org/x/oauth2 v0.11.0
full diff: https://github.com/golang/oauth2/comopare/v0.10.0...v0.11.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-20 22:58:31 +01:00
Sebastiaan van Stijn
2799417da1
vendor: golang.org/x/mod v0.13.0, golang.org/x/tools v0.13.0
full diff:

- https://github.com/golang/mod/comopare/v0.11.0...v0.13.0
- https://github.com/golang/tools/comopare/v0.10.0...v0.13.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-20 22:58:30 +01:00
Sebastiaan van Stijn
407ad89ff0
vendor: golang.org/x/sync v0.5.0
full diff: https://github.com/golang/sync/comopare/v0.3.0...v0.5.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-20 22:58:30 +01:00
Sebastiaan van Stijn
94b4765363
pkg/platforms: internalize in daemon/containerd
This matcher was only used internally in the containerd implementation of
the image store. Un-export it, and make it a local utility in that package
to prevent external use.

This package was introduced in 1616a09b61
(v24.0), and there are no known external consumers of this package, so there
should be no need to deprecate / alias the old location.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-20 22:28:56 +01:00
Sebastiaan van Stijn
c5187380d8
Merge pull request #46252 from thaJeztah/libnetwork_sandbox_sorting
libnetwork: Sandbox.ResolveName: refactor ordering of endpoints
2024-01-20 17:41:35 +01:00
Sebastiaan van Stijn
0a9bc3b507
libnetwork: Sandbox.ResolveName: refactor ordering of endpoints
When resolving names in swarm mode, services with exposed ports are
connected to user overlay network, ingress network, and local (docker_gwbridge)
networks. Name resolution should prioritize returning the VIP/IPs on user
overlay network over ingress and local networks.

Sandbox.ResolveName implemented this by taking the list of endpoints,
splitting the list into 3 separate lists based on the type of network
that the endpoint was attached to (dynamic, ingress, local), and then
creating a new list, applying the networks in that order.

This patch refactors that logic to use a custom sorter (sort.Interface),
which makes the code more transparent, and prevents iterating over the
list of endpoints multiple times.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-20 12:41:33 +01:00
Sebastiaan van Stijn
17c3829528
Merge pull request #47132 from corhere/allow-container-ip-outside-subpool
libnetwork: loosen container IPAM validation
2024-01-20 11:29:51 +01:00
Cory Snider
058b30023f libnetwork: loosen container IPAM validation
Permit container network attachments to set any static IP address within
the network's IPAM master pool, including when a subpool is configured.
Users have come to depend on being able to statically assign container
IP addresses which are guaranteed not to collide with automatically-
assigned container addresses.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-01-19 20:18:15 -05:00
Sebastiaan van Stijn
dfdd5169a2
Merge pull request #46113 from akerouanton/remove-deprecated-oom-score-adjust
daemon: remove --oom-score-adjust flag
2024-01-20 01:35:37 +01:00
Sebastiaan van Stijn
844ca49743
pkg/containerfs: remove deprecated ResolveScopedPath
This function was deprecated in b8f2caa80a
(v25.0), and is no longer in use.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-20 01:27:07 +01:00
Sebastiaan van Stijn
2767d9ba05
pkg/homedir: remove deprecated Key() and GetShortcutString()
These were deprecated in ddd9665289 (v25.0),
and 3c1de2e667, and are no longer used.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-20 01:11:44 +01:00
Sebastiaan van Stijn
f16a2179a6
pkg/system: remove deprecated ErrNotSupportedOperatingSystem, IsOSSupported
These were deprecated in a3c97beee0 (v25.0.0),
and are no longer used.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-20 01:01:11 +01:00