Commit graph

48591 commits

Author SHA1 Message Date
Paweł Gronowski
5d6679345c
Merge pull request #47508 from vvoland/v25.0-47504
[25.0 backport] update RootlessKit to 2.0.2
2024-03-06 12:55:15 +01:00
Paweł Gronowski
ef1fa235cd
Merge pull request #47510 from akerouanton/25.0-47441_mac_addr_config_migration
[25.0 backport] Don't create endpoint config for MAC addr config migration
2024-03-06 12:15:50 +01:00
Rob Murray
0451b287dc Don't create endpoint config for MAC addr config migration
In a container-create API request, HostConfig.NetworkMode (the identity
of the "main" network) may be a name, id or short-id.

The configuration for that network, including preferred IP address etc,
may be keyed on network name or id - it need not match the NetworkMode.

So, when migrating the old container-wide MAC address to the new
per-endpoint field - it is not safe to create a new EndpointSettings
entry unless there is no possibility that it will duplicate settings
intended for the same network (because one of the duplicates will be
discarded later, dropping the settings it contains).

This change introduces a new API restriction, if the deprecated container
wide field is used in the new API, and EndpointsConfig is provided for
any network, the NetworkMode and key under which the EndpointsConfig is
store must be the same - no mixing of ids and names.

Signed-off-by: Rob Murray <rob.murray@docker.com>
(cherry picked from commit a580544d82)
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-03-06 11:20:10 +01:00
Akihiro Suda
d27fe2558d
dockerd-rootless-setuptool.sh: check RootlessKit functionality
RootlessKit will print hints if something is still unsatisfied.

e.g., `kernel.apparmor_restrict_unprivileged_userns` constraint
rootless-containers/rootlesskit@33c3e7ca6c

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit b32cfc3b3a)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-06 10:38:17 +01:00
Akihiro Suda
77de535364
Dockerfile: update RootlessKit to v2.0.2
https://github.com/rootless-containers/rootlesskit/compare/v2.0.1...v2.0.2

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit 49fd8df9b9)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-06 10:38:15 +01:00
Paweł Gronowski
9e526bc394
Merge pull request #47503 from vvoland/v25.0-47502
[25.0 backport] update to go1.21.8
2024-03-05 21:58:50 +01:00
Paweł Gronowski
2d347024d1 update to go1.21.8
go1.21.8 (released 2024-03-05) includes 5 security fixes

- crypto/x509: Verify panics on certificates with an unknown public key algorithm (CVE-2024-24783, https://go.dev/issue/65390)
- net/http: memory exhaustion in Request.ParseMultipartForm (CVE-2023-45290, https://go.dev/issue/65383)
- net/http, net/http/cookiejar: incorrect forwarding of sensitive headers and cookies on HTTP redirect (CVE-2023-45289, https://go.dev/issue/65065)
- html/template: errors returned from MarshalJSON methods may break template escaping (CVE-2024-24785, https://go.dev/issue/65697)
- net/mail: comments in display names are incorrectly handled (CVE-2024-24784, https://go.dev/issue/65083)

View the release notes for more information:
https://go.dev/doc/devel/release#go1.22.1

- https://github.com/golang/go/issues?q=milestone%3AGo1.21.8+label%3ACherryPickApproved
- full diff: https://github.com/golang/go/compare/go1.21.7...go1.21.8

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 57b7ffa7f6)
2024-03-05 19:23:00 +01:00
Albin Kerouanton
51e876cd96
Merge pull request #47493 from akerouanton/25.0-47370_windows_natnw_dns_test
[25.0 backport] Test DNS on Windows 'nat' networks
2024-03-01 17:02:36 +01:00
Sebastiaan van Stijn
3fa0cedce3
Merge pull request #47484 from akerouanton/25.0-c8d-pull-fslayer
[25.0 backport] c8d/pull: Progress fixes
2024-03-01 14:06:52 +01:00
Sebastiaan van Stijn
4e7d8531ed
Merge pull request #47491 from akerouanton/25.0-c8d-skip-last-windows-tests
[25.0 backport] c8d/windows: Temporarily skip two failing tests
2024-03-01 14:04:41 +01:00
Rob Murray
f66b5f642e Test DNS on Windows 'nat' networks
Signed-off-by: Rob Murray <rob.murray@docker.com>
(cherry picked from commit 9083c2f10d)
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-03-01 13:51:20 +01:00
Albin Kerouanton
41fde13f64
Merge pull request #47490 from akerouanton/25.0-47370_windows_nat_network_dns
[25.0 backport] Set up DNS names for Windows default network
2024-03-01 13:13:02 +01:00
Sebastiaan van Stijn
0db1c6d8bb
Merge pull request #47488 from akerouanton/25.0-run_macvlan_ipvlan_tests
[25.0 backport] Run the macvlan/ipvlan integration tests
2024-03-01 12:59:32 +01:00
Paweł Gronowski
33a29c0135
Merge pull request #47489 from akerouanton/25.0-ci-codecov-token
[25.0 backport] ci: set codecov token
2024-03-01 12:44:46 +01:00
Albin Kerouanton
30545de83e
Merge pull request #47393 from vvoland/rro-backwards-compatible-25
[25.0 backport] api/pre-1.44: Default `ReadOnlyNonRecursive` to true
2024-03-01 12:15:21 +01:00
Paweł Gronowski
fa4ea308f0 c8d/windows: Temporarily skip two failing tests
They're failing the CI and we have a tracking ticket: #47107

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 44167988c3)
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-03-01 12:07:21 +01:00
Rob Murray
d66e0fb7b1 Set up DNS names for Windows default network
DNS names were only set up for user-defined networks. On Linux, none
of the built-in networks (bridge/host/none) have built-in DNS, so they
don't need DNS names.

But, on Windows, the default network is "nat" and it does need the DNS
names.

Signed-off-by: Rob Murray <rob.murray@docker.com>
(cherry picked from commit 443f56efb0)
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-03-01 12:03:16 +01:00
Sebastiaan van Stijn
30ecc0ea8a
Merge pull request #47482 from akerouanton/25.0-swarm-ipam-validation
[25.0 backport] Don't enforce new validation rules for existing networks
2024-03-01 11:58:18 +01:00
Sebastiaan van Stijn
06767446fe
Merge pull request #47481 from akerouanton/25.0-internal-bridge
[25.0 backport] Make 'internal' bridge networks accessible from host
2024-03-01 11:57:38 +01:00
Sebastiaan van Stijn
7048a63686
Merge pull request #47486 from akerouanton/25.0-go-1.21.7
[25.0 backport] update to go1.21.7
2024-03-01 11:56:23 +01:00
Paweł Gronowski
81fb7f9986
Merge pull request #47487 from akerouanton/25.0-integration-testdaemonproxy-reset-otel
[25.0 backport] integration: Reset `OTEL_EXPORTER_OTLP_ENDPOINT` for sub-daemons
2024-03-01 11:46:42 +01:00
Sebastiaan van Stijn
b77bb69f87
Merge pull request #47483 from akerouanton/25.0-best-effort-xattrs-classic-builder
[25.0 backport] builder/dockerfile: ADD with best-effort xattrs
2024-03-01 11:03:34 +01:00
CrazyMax
7a4abb8c77 ci: set codecov token
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
(cherry picked from commit 38827ba290)
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-03-01 10:38:53 +01:00
Rob Murray
81a83f0544 Simplify macvlan/ipvlan integration test structure
Signed-off-by: Rob Murray <rob.murray@docker.com>
(cherry picked from commit 9faf4855d5)
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-03-01 10:34:52 +01:00
Rob Murray
abcd6f8a46 Run the macvlan/ipvlan integration tests
The problem was accidentally introduced in:
  e8dc902781

Signed-off-by: Rob Murray <rob.murray@docker.com>
(cherry picked from commit 4eb95d01bc)
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-03-01 10:34:52 +01:00
Paweł Gronowski
f7be6dcba6 integration: Reset OTEL_EXPORTER_OTLP_ENDPOINT for sub-daemons
When creating a new daemon in the `TestDaemonProxy`, reset the
`OTEL_EXPORTER_OTLP_ENDPOINT` to an empty value to disable OTEL
collection to avoid it hitting the proxy.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 5fe96e234d)
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-03-01 10:30:13 +01:00
Sebastiaan van Stijn
10609544e5 update to go1.21.7
go1.21.7 (released 2024-02-06) includes fixes to the compiler, the go command,
the runtime, and the crypto/x509 package. See the Go 1.21.7 milestone on our
issue tracker for details:

- https://github.com/golang/go/issues?q=milestone%3AGo1.21.7+label%3ACherryPickApproved
- full diff: https://github.com/golang/go/compare/go1.21.6...go1.21.7

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 7c2975d2df)
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-03-01 10:20:49 +01:00
Paweł Gronowski
be59afce2d c8d/pull: Output truncated id for Pulling fs layer
All other progress updates are emitted with truncated id.

```diff
$ docker pull --platform linux/amd64 alpine
Using default tag: latest
latest: Pulling from library/alpine
-sha256:4abcf20661432fb2d719aaf90656f55c287f8ca915dc1c92ec14ff61e67fbaf8: Pulling fs layer
+4abcf2066143: Download complete
Digest: sha256:c5b1261d6d3e43071626931fc004f70149baeba2c8ec672bd4f27761f8e1ad6b
Status: Image is up to date for alpine:latest
docker.io/library/alpine:latest
```

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 16aa7dd67f)
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-03-01 10:10:04 +01:00
Paweł Gronowski
97951c39fb c8d/pull: Don't emit Downloading with 0 progress
To align with the graphdrivers behavior and don't send unnecessary
progress messages.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 14df52b709)
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-03-01 10:09:41 +01:00
Paweł Gronowski
2001813571 c8d/pull: Emit Pulling fs layer
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit ff5f780f2b)
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-03-01 10:09:37 +01:00
Paweł Gronowski
8e3bcf1974 pkg/streamformatter: Make progressOutput concurrency safe
Sync access to the underlying `io.Writer` with a mutex.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 5689dabfb3)
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-03-01 10:09:32 +01:00
Cory Snider
27f36f42a4 builder/dockerfile: ADD with best-effort xattrs
Archives being unpacked by Dockerfiles may have been created on other
OSes with different conventions and semantics for xattrs, making them
impossible to apply when extracting. Restore the old best-effort xattr
behaviour users have come to depend on in the classic builder.

The (archive.Archiver).UntarPath function does not allow the options
passed to Untar to be customized. It also happens to be a trivial
wrapper around the Untar function. Inline the function body and add the
option.

Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit 5bcd2f6860)
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-03-01 10:01:09 +01:00
Rob Murray
1ae019fca2 Don't enforce new validation rules for existing networks
Non-swarm networks created before network-creation-time validation
was added in 25.0.0 continued working, because the checks are not
re-run.

But, swarm creates networks when needed (with 'agent=true'), to
ensure they exist on each agent - ignoring the NetworkNameError
that says the network already existed.

By ignoring validation errors on creation of a network with
agent=true, pre-existing swarm networks with IPAM config that would
fail the new checks will continue to work too.

New swarm (overlay) networks are still validated, because they are
initially created with 'agent=false'.

Signed-off-by: Rob Murray <rob.murray@docker.com>
(cherry picked from commit 571af915d5)
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-03-01 09:54:46 +01:00
Rob Murray
c761353e7c Make 'internal' bridge networks accessible from host
Prior to release 25.0.0, the bridge in an internal network was assigned
an IP address - making the internal network accessible from the host,
giving containers on the network access to anything listening on the
bridge's address (or INADDR_ANY on the host).

This change restores that behaviour. It does not restore the default
route that was configured in the container, because packets sent outside
the internal network's subnet have always been dropped. So, a 'connect()'
to an address outside the subnet will still fail fast.

Signed-off-by: Rob Murray <rob.murray@docker.com>
(cherry picked from commit 419f5a6372)
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-03-01 09:29:41 +01:00
Paweł Gronowski
00b2e1072b
Merge pull request #47476 from vvoland/ci-report-timeout-25
[25.0 backport] ci: Update `teststat` to v0.1.25
2024-02-29 20:40:06 +01:00
Paweł Gronowski
10bc347b03
ci: Update teststat to v0.1.25
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit fc0e5401f2)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-29 16:01:20 +01:00
Sebastiaan van Stijn
6d675b429e
Merge pull request #47471 from vvoland/ci-reports-better-find-25
[25.0 backport] ci: Make `find` for test reports more specific
2024-02-29 15:56:26 +01:00
Paweł Gronowski
9f1b47c597
Merge pull request #47470 from neersighted/backport/47440/25.0
[25.0 backport] client: fix connection-errors being shadowed by API version errors
2024-02-29 15:53:41 +01:00
Sebastiaan van Stijn
94137f6df5
client: fix connection-errors being shadowed by API version mismatch errors
Commit e6907243af applied a fix for situations
where the client was configured with API-version negotiation, but did not yet
negotiate a version.

However, the checkVersion() function that was implemented copied the semantics
of cli.NegotiateAPIVersion, which ignored connection failures with the
assumption that connection errors would still surface further down.

However, when using the result of a failed negotiation for NewVersionError,
an API version mismatch error would be produced, masking the actual connection
error.

This patch changes the signature of checkVersion to return unexpected errors,
including failures to connect to the API.

Before this patch:

    docker -H unix:///no/such/socket.sock secret ls
    "secret list" requires API version 1.25, but the Docker daemon API version is 1.24

With this patch applied:

    docker -H unix:///no/such/socket.sock secret ls
    Cannot connect to the Docker daemon at unix:///no/such/socket.sock. Is the docker daemon running?

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 6aea26b431)
Conflicts: client/image_list.go
    client/image_list_test.go
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2024-02-29 06:02:06 -07:00
Paweł Gronowski
dd5faa9d4f
ci: Make find for test reports more specific
Don't use all `*.json` files blindly, take only these that are likely to
be reports from go test.
Also, use `find ... -exec` instead of piping results to `xargs`.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit e4de4dea5c)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-29 10:07:44 +01:00
Sebastiaan van Stijn
012bfd33e5
client: doRequest: make sure we return a connection-error
This function has various errors that are returned when failing to make a
connection (due to permission issues, TLS mis-configuration, or failing to
resolve the TCP address).

The errConnectionFailed error is currently used as a special case when
processing Ping responses. The current code did not consistently treat
connection errors, and because of that could either absorb the error,
or process the empty response.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 913478b428)
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2024-02-29 01:17:13 -07:00
Sebastiaan van Stijn
3ec1946ce1
client: NegotiateAPIVersion: do not ignore (connection) errors from Ping
NegotiateAPIVersion was ignoring errors returned by Ping. The intent here
was to handle API responses from a daemon that may be in an unhealthy state,
however this case is already handled by Ping itself.

Ping only returns an error when either failing to connect to the API (daemon
not running or permissions errors), or when failing to parse the API response.

Neither of those should be ignored in this code, or considered a successful
"ping", so update the code to return

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 901b90593d)
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2024-02-29 01:17:13 -07:00
Sebastiaan van Stijn
200a2c3576
client: fix TestPingWithError
This test was added in 27ef09a46f, which changed
the Ping handling to ignore internal server errors. That case is tested in
TestPingFail, which verifies that we accept the Ping response if a 500
status code was received.

The TestPingWithError test was added to verify behavior if a protocol
(connection) error occurred; however the mock-client returned both a
response, and an error; the error returned would only happen if a connection
error occurred, which means that the server would not provide a reply.

Running the test also shows that returning a response is unexpected, and
ignored:

    === RUN   TestPingWithError
    2024/02/23 14:16:49 RoundTripper returned a response & error; ignoring response
    2024/02/23 14:16:49 RoundTripper returned a response & error; ignoring response
    --- PASS: TestPingWithError (0.00s)
    PASS

This patch updates the test to remove the response.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 349abc64ed)
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2024-02-29 01:17:09 -07:00
Paweł Gronowski
cb66214dfd
Merge pull request #47466 from huang-jl/25.0_backport_fix_restore_digest
[25.0 backport] libcontainerd: change the digest used when restoring
2024-02-28 17:42:18 +01:00
huang-jl
70c05fe10c libcontainerd: change the digest used when restoring
For current implementation of Checkpoint Restore (C/R) in docker, it
will write the checkpoint to content store. However, when restoring
libcontainerd uses .Digest().Encoded(), which will remove the info
of alg, leading to error.

Signed-off-by: huang-jl <1046678590@qq.com>
(cherry picked from commit da643c0b8a)
Signed-off-by: huang-jl <1046678590@qq.com>
2024-02-28 23:27:17 +08:00
Paweł Gronowski
e85cef89fa
api/pre-1.44: Default ReadOnlyNonRecursive to true
Don't change the behavior for older clients and keep the same behavior.
Otherwise client can't opt-out (because `ReadOnlyNonRecursive` is
unsupported before 1.44).

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 432390320e)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-28 10:00:22 +01:00
Paweł Gronowski
a72294a668
mounts/validate: Don't check source exists with CreateMountpoint
Don't error out when mount source doesn't exist and mounts has
`CreateMountpoint` option enabled.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 05b883bdc8)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-27 23:31:36 +01:00
Paweł Gronowski
9ee331235a
integration: Add container.Output utility
Extracted from bfb810445c

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-27 23:31:35 +01:00
Bjorn Neergaard
6fb71a9764
Merge pull request #47451 from neersighted/image_created_omitempty_25.0
[25.0 backport] api: omit missing Created field from ImageInspect response
2024-02-26 15:09:06 -07:00
Bjorn Neergaard
5d9e13bc84
api: omit missing Created field from ImageInspect response
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2024-02-26 10:44:35 -07:00