Commit graph

48599 commits

Author SHA1 Message Date
Albin Kerouanton
a987bc5ad0 libnet: Don't forward to upstream resolvers on internal nw
Commit cbc2a71c2 makes `connect` syscall fail fast when a container is
only attached to an internal network. Thanks to that, if such a
container tries to resolve an "external" domain, the embedded resolver
returns an error immediately instead of waiting for a timeout.

This commit makes sure the embedded resolver doesn't even try to forward
to upstream servers.

Co-authored-by: Albin Kerouanton <albinker@gmail.com>
Signed-off-by: Rob Murray <rob.murray@docker.com>
(cherry picked from commit 790c3039d0)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-19 11:21:46 +00:00
Rob Murray
20c205fd3a Environment variable to override resolv.conf path.
If env var DOCKER_TEST_RESOLV_CONF_PATH is set, treat it as an override
for the 'resolv.conf' path.

Added as part of resolv.conf refactoring, but needed by back-ported test
TestInternalNetworkDNS.

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-03-19 11:21:46 +00:00
Sebastiaan van Stijn
5901652edd
Merge pull request #47533 from vvoland/v25.0-47530
[25.0 backport] volume: Don't decrement refcount below 0
2024-03-08 14:00:37 +01:00
Paweł Gronowski
478f6b097d
volume: Don't decrement refcount below 0
With both rootless and live restore enabled, there's some race condition
which causes the container to be `Unmount`ed before the refcount is
restored.

This makes sure we don't underflow the refcount (uint64) when
decrementing it.

The root cause of this race condition still needs to be investigated and
fixed, but at least this unflakies the `TestLiveRestore`.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 294fc9762e)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-08 12:51:45 +01:00
Bjorn Neergaard
98b171fd4d
Merge pull request #47527 from vvoland/v25.0-47523
[25.0 backport] builder-next: fix missing lock in ensurelayer
2024-03-07 07:08:59 -07:00
Tonis Tiigi
d250e13945
builder-next: fix missing lock in ensurelayer
When this was called concurrently from the moby image
exporter there could be a data race where a layer was
written to the refs map when it was already there.

In that case the reference count got mixed up and on
release only one of these layers was actually released.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit 37545cc644)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-07 12:18:07 +01:00
Paweł Gronowski
061aa95809
Merge pull request #47513 from vvoland/v25.0-47498
[25.0 backport] daemon: overlay2: remove world writable permission from the lower file
2024-03-06 14:58:50 +01:00
Jaroslav Jindrak
d0d85f6438
daemon: overlay2: remove world writable permission from the lower file
In de2447c, the creation of the 'lower' file was changed from using
os.Create to using ioutils.AtomicWriteFile, which ignores the system's
umask. This means that even though the requested permission in the
source code was always 0666, it was 0644 on systems with default
umask of 0022 prior to de2447c, so the move to AtomicFile potentially
increased the file's permissions.

This is not a security issue because the parent directory does not
allow writes into the file, but it can confuse security scanners on
Linux-based systems into giving false positives.

Signed-off-by: Jaroslav Jindrak <dzejrou@gmail.com>
(cherry picked from commit cadb124ab6)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-06 13:11:41 +01:00
Paweł Gronowski
5d6679345c
Merge pull request #47508 from vvoland/v25.0-47504
[25.0 backport] update RootlessKit to 2.0.2
2024-03-06 12:55:15 +01:00
Paweł Gronowski
ef1fa235cd
Merge pull request #47510 from akerouanton/25.0-47441_mac_addr_config_migration
[25.0 backport] Don't create endpoint config for MAC addr config migration
2024-03-06 12:15:50 +01:00
Rob Murray
0451b287dc Don't create endpoint config for MAC addr config migration
In a container-create API request, HostConfig.NetworkMode (the identity
of the "main" network) may be a name, id or short-id.

The configuration for that network, including preferred IP address etc,
may be keyed on network name or id - it need not match the NetworkMode.

So, when migrating the old container-wide MAC address to the new
per-endpoint field - it is not safe to create a new EndpointSettings
entry unless there is no possibility that it will duplicate settings
intended for the same network (because one of the duplicates will be
discarded later, dropping the settings it contains).

This change introduces a new API restriction, if the deprecated container
wide field is used in the new API, and EndpointsConfig is provided for
any network, the NetworkMode and key under which the EndpointsConfig is
store must be the same - no mixing of ids and names.

Signed-off-by: Rob Murray <rob.murray@docker.com>
(cherry picked from commit a580544d82)
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-03-06 11:20:10 +01:00
Akihiro Suda
d27fe2558d
dockerd-rootless-setuptool.sh: check RootlessKit functionality
RootlessKit will print hints if something is still unsatisfied.

e.g., `kernel.apparmor_restrict_unprivileged_userns` constraint
rootless-containers/rootlesskit@33c3e7ca6c

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit b32cfc3b3a)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-06 10:38:17 +01:00
Akihiro Suda
77de535364
Dockerfile: update RootlessKit to v2.0.2
https://github.com/rootless-containers/rootlesskit/compare/v2.0.1...v2.0.2

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit 49fd8df9b9)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-06 10:38:15 +01:00
Paweł Gronowski
9e526bc394
Merge pull request #47503 from vvoland/v25.0-47502
[25.0 backport] update to go1.21.8
2024-03-05 21:58:50 +01:00
Paweł Gronowski
2d347024d1 update to go1.21.8
go1.21.8 (released 2024-03-05) includes 5 security fixes

- crypto/x509: Verify panics on certificates with an unknown public key algorithm (CVE-2024-24783, https://go.dev/issue/65390)
- net/http: memory exhaustion in Request.ParseMultipartForm (CVE-2023-45290, https://go.dev/issue/65383)
- net/http, net/http/cookiejar: incorrect forwarding of sensitive headers and cookies on HTTP redirect (CVE-2023-45289, https://go.dev/issue/65065)
- html/template: errors returned from MarshalJSON methods may break template escaping (CVE-2024-24785, https://go.dev/issue/65697)
- net/mail: comments in display names are incorrectly handled (CVE-2024-24784, https://go.dev/issue/65083)

View the release notes for more information:
https://go.dev/doc/devel/release#go1.22.1

- https://github.com/golang/go/issues?q=milestone%3AGo1.21.8+label%3ACherryPickApproved
- full diff: https://github.com/golang/go/compare/go1.21.7...go1.21.8

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 57b7ffa7f6)
2024-03-05 19:23:00 +01:00
Albin Kerouanton
51e876cd96
Merge pull request #47493 from akerouanton/25.0-47370_windows_natnw_dns_test
[25.0 backport] Test DNS on Windows 'nat' networks
2024-03-01 17:02:36 +01:00
Sebastiaan van Stijn
3fa0cedce3
Merge pull request #47484 from akerouanton/25.0-c8d-pull-fslayer
[25.0 backport] c8d/pull: Progress fixes
2024-03-01 14:06:52 +01:00
Sebastiaan van Stijn
4e7d8531ed
Merge pull request #47491 from akerouanton/25.0-c8d-skip-last-windows-tests
[25.0 backport] c8d/windows: Temporarily skip two failing tests
2024-03-01 14:04:41 +01:00
Rob Murray
f66b5f642e Test DNS on Windows 'nat' networks
Signed-off-by: Rob Murray <rob.murray@docker.com>
(cherry picked from commit 9083c2f10d)
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-03-01 13:51:20 +01:00
Albin Kerouanton
41fde13f64
Merge pull request #47490 from akerouanton/25.0-47370_windows_nat_network_dns
[25.0 backport] Set up DNS names for Windows default network
2024-03-01 13:13:02 +01:00
Sebastiaan van Stijn
0db1c6d8bb
Merge pull request #47488 from akerouanton/25.0-run_macvlan_ipvlan_tests
[25.0 backport] Run the macvlan/ipvlan integration tests
2024-03-01 12:59:32 +01:00
Paweł Gronowski
33a29c0135
Merge pull request #47489 from akerouanton/25.0-ci-codecov-token
[25.0 backport] ci: set codecov token
2024-03-01 12:44:46 +01:00
Albin Kerouanton
30545de83e
Merge pull request #47393 from vvoland/rro-backwards-compatible-25
[25.0 backport] api/pre-1.44: Default `ReadOnlyNonRecursive` to true
2024-03-01 12:15:21 +01:00
Paweł Gronowski
fa4ea308f0 c8d/windows: Temporarily skip two failing tests
They're failing the CI and we have a tracking ticket: #47107

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 44167988c3)
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-03-01 12:07:21 +01:00
Rob Murray
d66e0fb7b1 Set up DNS names for Windows default network
DNS names were only set up for user-defined networks. On Linux, none
of the built-in networks (bridge/host/none) have built-in DNS, so they
don't need DNS names.

But, on Windows, the default network is "nat" and it does need the DNS
names.

Signed-off-by: Rob Murray <rob.murray@docker.com>
(cherry picked from commit 443f56efb0)
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-03-01 12:03:16 +01:00
Sebastiaan van Stijn
30ecc0ea8a
Merge pull request #47482 from akerouanton/25.0-swarm-ipam-validation
[25.0 backport] Don't enforce new validation rules for existing networks
2024-03-01 11:58:18 +01:00
Sebastiaan van Stijn
06767446fe
Merge pull request #47481 from akerouanton/25.0-internal-bridge
[25.0 backport] Make 'internal' bridge networks accessible from host
2024-03-01 11:57:38 +01:00
Sebastiaan van Stijn
7048a63686
Merge pull request #47486 from akerouanton/25.0-go-1.21.7
[25.0 backport] update to go1.21.7
2024-03-01 11:56:23 +01:00
Paweł Gronowski
81fb7f9986
Merge pull request #47487 from akerouanton/25.0-integration-testdaemonproxy-reset-otel
[25.0 backport] integration: Reset `OTEL_EXPORTER_OTLP_ENDPOINT` for sub-daemons
2024-03-01 11:46:42 +01:00
Sebastiaan van Stijn
b77bb69f87
Merge pull request #47483 from akerouanton/25.0-best-effort-xattrs-classic-builder
[25.0 backport] builder/dockerfile: ADD with best-effort xattrs
2024-03-01 11:03:34 +01:00
CrazyMax
7a4abb8c77 ci: set codecov token
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
(cherry picked from commit 38827ba290)
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-03-01 10:38:53 +01:00
Rob Murray
81a83f0544 Simplify macvlan/ipvlan integration test structure
Signed-off-by: Rob Murray <rob.murray@docker.com>
(cherry picked from commit 9faf4855d5)
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-03-01 10:34:52 +01:00
Rob Murray
abcd6f8a46 Run the macvlan/ipvlan integration tests
The problem was accidentally introduced in:
  e8dc902781

Signed-off-by: Rob Murray <rob.murray@docker.com>
(cherry picked from commit 4eb95d01bc)
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-03-01 10:34:52 +01:00
Paweł Gronowski
f7be6dcba6 integration: Reset OTEL_EXPORTER_OTLP_ENDPOINT for sub-daemons
When creating a new daemon in the `TestDaemonProxy`, reset the
`OTEL_EXPORTER_OTLP_ENDPOINT` to an empty value to disable OTEL
collection to avoid it hitting the proxy.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 5fe96e234d)
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-03-01 10:30:13 +01:00
Sebastiaan van Stijn
10609544e5 update to go1.21.7
go1.21.7 (released 2024-02-06) includes fixes to the compiler, the go command,
the runtime, and the crypto/x509 package. See the Go 1.21.7 milestone on our
issue tracker for details:

- https://github.com/golang/go/issues?q=milestone%3AGo1.21.7+label%3ACherryPickApproved
- full diff: https://github.com/golang/go/compare/go1.21.6...go1.21.7

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 7c2975d2df)
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-03-01 10:20:49 +01:00
Paweł Gronowski
be59afce2d c8d/pull: Output truncated id for Pulling fs layer
All other progress updates are emitted with truncated id.

```diff
$ docker pull --platform linux/amd64 alpine
Using default tag: latest
latest: Pulling from library/alpine
-sha256:4abcf20661432fb2d719aaf90656f55c287f8ca915dc1c92ec14ff61e67fbaf8: Pulling fs layer
+4abcf2066143: Download complete
Digest: sha256:c5b1261d6d3e43071626931fc004f70149baeba2c8ec672bd4f27761f8e1ad6b
Status: Image is up to date for alpine:latest
docker.io/library/alpine:latest
```

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 16aa7dd67f)
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-03-01 10:10:04 +01:00
Paweł Gronowski
97951c39fb c8d/pull: Don't emit Downloading with 0 progress
To align with the graphdrivers behavior and don't send unnecessary
progress messages.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 14df52b709)
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-03-01 10:09:41 +01:00
Paweł Gronowski
2001813571 c8d/pull: Emit Pulling fs layer
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit ff5f780f2b)
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-03-01 10:09:37 +01:00
Paweł Gronowski
8e3bcf1974 pkg/streamformatter: Make progressOutput concurrency safe
Sync access to the underlying `io.Writer` with a mutex.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 5689dabfb3)
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-03-01 10:09:32 +01:00
Cory Snider
27f36f42a4 builder/dockerfile: ADD with best-effort xattrs
Archives being unpacked by Dockerfiles may have been created on other
OSes with different conventions and semantics for xattrs, making them
impossible to apply when extracting. Restore the old best-effort xattr
behaviour users have come to depend on in the classic builder.

The (archive.Archiver).UntarPath function does not allow the options
passed to Untar to be customized. It also happens to be a trivial
wrapper around the Untar function. Inline the function body and add the
option.

Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit 5bcd2f6860)
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-03-01 10:01:09 +01:00
Rob Murray
1ae019fca2 Don't enforce new validation rules for existing networks
Non-swarm networks created before network-creation-time validation
was added in 25.0.0 continued working, because the checks are not
re-run.

But, swarm creates networks when needed (with 'agent=true'), to
ensure they exist on each agent - ignoring the NetworkNameError
that says the network already existed.

By ignoring validation errors on creation of a network with
agent=true, pre-existing swarm networks with IPAM config that would
fail the new checks will continue to work too.

New swarm (overlay) networks are still validated, because they are
initially created with 'agent=false'.

Signed-off-by: Rob Murray <rob.murray@docker.com>
(cherry picked from commit 571af915d5)
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-03-01 09:54:46 +01:00
Rob Murray
c761353e7c Make 'internal' bridge networks accessible from host
Prior to release 25.0.0, the bridge in an internal network was assigned
an IP address - making the internal network accessible from the host,
giving containers on the network access to anything listening on the
bridge's address (or INADDR_ANY on the host).

This change restores that behaviour. It does not restore the default
route that was configured in the container, because packets sent outside
the internal network's subnet have always been dropped. So, a 'connect()'
to an address outside the subnet will still fail fast.

Signed-off-by: Rob Murray <rob.murray@docker.com>
(cherry picked from commit 419f5a6372)
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-03-01 09:29:41 +01:00
Paweł Gronowski
00b2e1072b
Merge pull request #47476 from vvoland/ci-report-timeout-25
[25.0 backport] ci: Update `teststat` to v0.1.25
2024-02-29 20:40:06 +01:00
Paweł Gronowski
10bc347b03
ci: Update teststat to v0.1.25
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit fc0e5401f2)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-29 16:01:20 +01:00
Sebastiaan van Stijn
6d675b429e
Merge pull request #47471 from vvoland/ci-reports-better-find-25
[25.0 backport] ci: Make `find` for test reports more specific
2024-02-29 15:56:26 +01:00
Paweł Gronowski
9f1b47c597
Merge pull request #47470 from neersighted/backport/47440/25.0
[25.0 backport] client: fix connection-errors being shadowed by API version errors
2024-02-29 15:53:41 +01:00
Sebastiaan van Stijn
94137f6df5
client: fix connection-errors being shadowed by API version mismatch errors
Commit e6907243af applied a fix for situations
where the client was configured with API-version negotiation, but did not yet
negotiate a version.

However, the checkVersion() function that was implemented copied the semantics
of cli.NegotiateAPIVersion, which ignored connection failures with the
assumption that connection errors would still surface further down.

However, when using the result of a failed negotiation for NewVersionError,
an API version mismatch error would be produced, masking the actual connection
error.

This patch changes the signature of checkVersion to return unexpected errors,
including failures to connect to the API.

Before this patch:

    docker -H unix:///no/such/socket.sock secret ls
    "secret list" requires API version 1.25, but the Docker daemon API version is 1.24

With this patch applied:

    docker -H unix:///no/such/socket.sock secret ls
    Cannot connect to the Docker daemon at unix:///no/such/socket.sock. Is the docker daemon running?

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 6aea26b431)
Conflicts: client/image_list.go
    client/image_list_test.go
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2024-02-29 06:02:06 -07:00
Paweł Gronowski
dd5faa9d4f
ci: Make find for test reports more specific
Don't use all `*.json` files blindly, take only these that are likely to
be reports from go test.
Also, use `find ... -exec` instead of piping results to `xargs`.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit e4de4dea5c)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-29 10:07:44 +01:00
Sebastiaan van Stijn
012bfd33e5
client: doRequest: make sure we return a connection-error
This function has various errors that are returned when failing to make a
connection (due to permission issues, TLS mis-configuration, or failing to
resolve the TCP address).

The errConnectionFailed error is currently used as a special case when
processing Ping responses. The current code did not consistently treat
connection errors, and because of that could either absorb the error,
or process the empty response.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 913478b428)
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2024-02-29 01:17:13 -07:00
Sebastiaan van Stijn
3ec1946ce1
client: NegotiateAPIVersion: do not ignore (connection) errors from Ping
NegotiateAPIVersion was ignoring errors returned by Ping. The intent here
was to handle API responses from a daemon that may be in an unhealthy state,
however this case is already handled by Ping itself.

Ping only returns an error when either failing to connect to the API (daemon
not running or permissions errors), or when failing to parse the API response.

Neither of those should be ignored in this code, or considered a successful
"ping", so update the code to return

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 901b90593d)
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2024-02-29 01:17:13 -07:00