Commit graph

48608 commits

Author SHA1 Message Date
Paweł Gronowski
50bd133ad3
update to go1.21.9
go1.21.9 (released 2024-04-03) includes a security fix to the net/http
package, as well as bug fixes to the linker, and the go/types and
net/http packages. See the [Go 1.21.9 milestone](https://github.com/golang/go/issues?q=milestone%3AGo1.21.9+label%3ACherryPickApproved)
for more details.

These minor releases include 1 security fixes following the security policy:

- http2: close connections when receiving too many headers

Maintaining HPACK state requires that we parse and process all HEADERS
and CONTINUATION frames on a connection. When a request's headers exceed
MaxHeaderBytes, we don't allocate memory to store the excess headers but
we do parse them. This permits an attacker to cause an HTTP/2 endpoint
to read arbitrary amounts of header data, all associated with a request
which is going to be rejected. These headers can include Huffman-encoded
data which is significantly more expensive for the receiver to decode
than for an attacker to send.

Set a limit on the amount of excess header frames we will process before
closing a connection.

Thanks to Bartek Nowotarski (https://nowotarski.info/) for reporting this issue.

This is CVE-2023-45288 and Go issue https://go.dev/issue/65051.

View the release notes for more information:
https://go.dev/doc/devel/release#go1.22.2

- https://github.com/golang/go/issues?q=milestone%3AGo1.21.9+label%3ACherryPickApproved
- full diff: https://github.com/golang/go/compare/go1.21.8...go1.21.9

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 329d403e20)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-04-04 10:17:28 +02:00
Paweł Gronowski
e63daec867
Merge pull request #47589 from vvoland/v25.0-47538
[25.0 backport] libnet: Don't forward to upstream resolvers on internal nw
2024-03-19 15:12:29 +01:00
Paweł Gronowski
817bccb1c6
Merge pull request #47588 from vvoland/v25.0-47558
[25.0 backport] plugin: fix mounting /etc/hosts when running in UserNS
2024-03-19 15:12:15 +01:00
Bjorn Neergaard
2a0601e84e
Merge pull request #47587 from vvoland/v25.0-47559
[25.0 backport] rootless: fix `open /etc/docker/plugins: permission denied`
2024-03-19 07:24:36 -06:00
Sebastiaan van Stijn
9df9ccc06f
Merge pull request #47586 from vvoland/v25.0-47569
[25.0 backport] Makefile: generate-files: fix check for empty TMP_OUT
2024-03-19 12:45:46 +01:00
Albin Kerouanton
a987bc5ad0 libnet: Don't forward to upstream resolvers on internal nw
Commit cbc2a71c2 makes `connect` syscall fail fast when a container is
only attached to an internal network. Thanks to that, if such a
container tries to resolve an "external" domain, the embedded resolver
returns an error immediately instead of waiting for a timeout.

This commit makes sure the embedded resolver doesn't even try to forward
to upstream servers.

Co-authored-by: Albin Kerouanton <albinker@gmail.com>
Signed-off-by: Rob Murray <rob.murray@docker.com>
(cherry picked from commit 790c3039d0)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-19 11:21:46 +00:00
Rob Murray
20c205fd3a Environment variable to override resolv.conf path.
If env var DOCKER_TEST_RESOLV_CONF_PATH is set, treat it as an override
for the 'resolv.conf' path.

Added as part of resolv.conf refactoring, but needed by back-ported test
TestInternalNetworkDNS.

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-03-19 11:21:46 +00:00
Sebastiaan van Stijn
4be97233cc
daemon: move getUnprivilegedMountFlags to internal package
This code is currently only used in the daemon, but is also needed in other
places. We should consider moving this code to github.com/moby/sys, so that
BuildKit can also use the same implementation instead of maintaining a fork;
moving it to internal allows us to reuse this code inside the repository, but
does not allow external consumers to depend on it (which we don't want as
it's not a permanent location).

As our code only uses this in linux files, I did not add a stub for other
platforms (but we may decide to do that in the moby/sys repository).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 7b414f5703)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-19 10:46:49 +01:00
Akihiro Suda
7ed7e6caf6
plugin: fix mounting /etc/hosts when running in UserNS
Fix `error mounting "/etc/hosts" to rootfs at "/etc/hosts": mount
/etc/hosts:/etc/hosts (via /proc/self/fd/6), flags: 0x5021: operation
not permitted`.

This error was introduced in 7d08d84b03
(`dockerd-rootless.sh: set rootlesskit --state-dir=DIR`) that changed
the filesystem of the state dir from /tmp to /run (in a typical setup).

Fix issue 47248

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit 762ec4b60c)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-19 10:46:48 +01:00
Akihiro Suda
81ad7062f0
rootless: fix open /etc/docker/plugins: permission denied
Fix issue 47436

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit d742659877)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-19 10:12:38 +01:00
Sebastiaan van Stijn
02d4ee3f9a
Makefile: generate-files: fix check for empty TMP_OUT
commit c655b7dc78 added a check to make sure
the TMP_OUT variable was not set to an empty value, as such a situation would
perform an `rm -rf /**` during cleanup.

However, it was a bit too eager, because Makefile conditionals (`ifeq`) are
evaluated when parsing the Makefile, which happens _before_ the make target
is executed.

As a result `$@_TMP_OUT` was always empty when the `ifeq` was evaluated,
making it not possible to execute the `generate-files` target.

This patch changes the check to use a shell command to evaluate if the var
is set to an empty value.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 25c9e6e8df)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-19 10:11:47 +01:00
Sebastiaan van Stijn
5901652edd
Merge pull request #47533 from vvoland/v25.0-47530
[25.0 backport] volume: Don't decrement refcount below 0
2024-03-08 14:00:37 +01:00
Paweł Gronowski
478f6b097d
volume: Don't decrement refcount below 0
With both rootless and live restore enabled, there's some race condition
which causes the container to be `Unmount`ed before the refcount is
restored.

This makes sure we don't underflow the refcount (uint64) when
decrementing it.

The root cause of this race condition still needs to be investigated and
fixed, but at least this unflakies the `TestLiveRestore`.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 294fc9762e)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-08 12:51:45 +01:00
Bjorn Neergaard
98b171fd4d
Merge pull request #47527 from vvoland/v25.0-47523
[25.0 backport] builder-next: fix missing lock in ensurelayer
2024-03-07 07:08:59 -07:00
Tonis Tiigi
d250e13945
builder-next: fix missing lock in ensurelayer
When this was called concurrently from the moby image
exporter there could be a data race where a layer was
written to the refs map when it was already there.

In that case the reference count got mixed up and on
release only one of these layers was actually released.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit 37545cc644)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-07 12:18:07 +01:00
Paweł Gronowski
061aa95809
Merge pull request #47513 from vvoland/v25.0-47498
[25.0 backport] daemon: overlay2: remove world writable permission from the lower file
2024-03-06 14:58:50 +01:00
Jaroslav Jindrak
d0d85f6438
daemon: overlay2: remove world writable permission from the lower file
In de2447c, the creation of the 'lower' file was changed from using
os.Create to using ioutils.AtomicWriteFile, which ignores the system's
umask. This means that even though the requested permission in the
source code was always 0666, it was 0644 on systems with default
umask of 0022 prior to de2447c, so the move to AtomicFile potentially
increased the file's permissions.

This is not a security issue because the parent directory does not
allow writes into the file, but it can confuse security scanners on
Linux-based systems into giving false positives.

Signed-off-by: Jaroslav Jindrak <dzejrou@gmail.com>
(cherry picked from commit cadb124ab6)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-06 13:11:41 +01:00
Paweł Gronowski
5d6679345c
Merge pull request #47508 from vvoland/v25.0-47504
[25.0 backport] update RootlessKit to 2.0.2
2024-03-06 12:55:15 +01:00
Paweł Gronowski
ef1fa235cd
Merge pull request #47510 from akerouanton/25.0-47441_mac_addr_config_migration
[25.0 backport] Don't create endpoint config for MAC addr config migration
2024-03-06 12:15:50 +01:00
Rob Murray
0451b287dc Don't create endpoint config for MAC addr config migration
In a container-create API request, HostConfig.NetworkMode (the identity
of the "main" network) may be a name, id or short-id.

The configuration for that network, including preferred IP address etc,
may be keyed on network name or id - it need not match the NetworkMode.

So, when migrating the old container-wide MAC address to the new
per-endpoint field - it is not safe to create a new EndpointSettings
entry unless there is no possibility that it will duplicate settings
intended for the same network (because one of the duplicates will be
discarded later, dropping the settings it contains).

This change introduces a new API restriction, if the deprecated container
wide field is used in the new API, and EndpointsConfig is provided for
any network, the NetworkMode and key under which the EndpointsConfig is
store must be the same - no mixing of ids and names.

Signed-off-by: Rob Murray <rob.murray@docker.com>
(cherry picked from commit a580544d82)
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-03-06 11:20:10 +01:00
Akihiro Suda
d27fe2558d
dockerd-rootless-setuptool.sh: check RootlessKit functionality
RootlessKit will print hints if something is still unsatisfied.

e.g., `kernel.apparmor_restrict_unprivileged_userns` constraint
rootless-containers/rootlesskit@33c3e7ca6c

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit b32cfc3b3a)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-06 10:38:17 +01:00
Akihiro Suda
77de535364
Dockerfile: update RootlessKit to v2.0.2
https://github.com/rootless-containers/rootlesskit/compare/v2.0.1...v2.0.2

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit 49fd8df9b9)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-06 10:38:15 +01:00
Paweł Gronowski
9e526bc394
Merge pull request #47503 from vvoland/v25.0-47502
[25.0 backport] update to go1.21.8
2024-03-05 21:58:50 +01:00
Paweł Gronowski
2d347024d1 update to go1.21.8
go1.21.8 (released 2024-03-05) includes 5 security fixes

- crypto/x509: Verify panics on certificates with an unknown public key algorithm (CVE-2024-24783, https://go.dev/issue/65390)
- net/http: memory exhaustion in Request.ParseMultipartForm (CVE-2023-45290, https://go.dev/issue/65383)
- net/http, net/http/cookiejar: incorrect forwarding of sensitive headers and cookies on HTTP redirect (CVE-2023-45289, https://go.dev/issue/65065)
- html/template: errors returned from MarshalJSON methods may break template escaping (CVE-2024-24785, https://go.dev/issue/65697)
- net/mail: comments in display names are incorrectly handled (CVE-2024-24784, https://go.dev/issue/65083)

View the release notes for more information:
https://go.dev/doc/devel/release#go1.22.1

- https://github.com/golang/go/issues?q=milestone%3AGo1.21.8+label%3ACherryPickApproved
- full diff: https://github.com/golang/go/compare/go1.21.7...go1.21.8

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 57b7ffa7f6)
2024-03-05 19:23:00 +01:00
Albin Kerouanton
51e876cd96
Merge pull request #47493 from akerouanton/25.0-47370_windows_natnw_dns_test
[25.0 backport] Test DNS on Windows 'nat' networks
2024-03-01 17:02:36 +01:00
Sebastiaan van Stijn
3fa0cedce3
Merge pull request #47484 from akerouanton/25.0-c8d-pull-fslayer
[25.0 backport] c8d/pull: Progress fixes
2024-03-01 14:06:52 +01:00
Sebastiaan van Stijn
4e7d8531ed
Merge pull request #47491 from akerouanton/25.0-c8d-skip-last-windows-tests
[25.0 backport] c8d/windows: Temporarily skip two failing tests
2024-03-01 14:04:41 +01:00
Rob Murray
f66b5f642e Test DNS on Windows 'nat' networks
Signed-off-by: Rob Murray <rob.murray@docker.com>
(cherry picked from commit 9083c2f10d)
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-03-01 13:51:20 +01:00
Albin Kerouanton
41fde13f64
Merge pull request #47490 from akerouanton/25.0-47370_windows_nat_network_dns
[25.0 backport] Set up DNS names for Windows default network
2024-03-01 13:13:02 +01:00
Sebastiaan van Stijn
0db1c6d8bb
Merge pull request #47488 from akerouanton/25.0-run_macvlan_ipvlan_tests
[25.0 backport] Run the macvlan/ipvlan integration tests
2024-03-01 12:59:32 +01:00
Paweł Gronowski
33a29c0135
Merge pull request #47489 from akerouanton/25.0-ci-codecov-token
[25.0 backport] ci: set codecov token
2024-03-01 12:44:46 +01:00
Albin Kerouanton
30545de83e
Merge pull request #47393 from vvoland/rro-backwards-compatible-25
[25.0 backport] api/pre-1.44: Default `ReadOnlyNonRecursive` to true
2024-03-01 12:15:21 +01:00
Paweł Gronowski
fa4ea308f0 c8d/windows: Temporarily skip two failing tests
They're failing the CI and we have a tracking ticket: #47107

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 44167988c3)
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-03-01 12:07:21 +01:00
Rob Murray
d66e0fb7b1 Set up DNS names for Windows default network
DNS names were only set up for user-defined networks. On Linux, none
of the built-in networks (bridge/host/none) have built-in DNS, so they
don't need DNS names.

But, on Windows, the default network is "nat" and it does need the DNS
names.

Signed-off-by: Rob Murray <rob.murray@docker.com>
(cherry picked from commit 443f56efb0)
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-03-01 12:03:16 +01:00
Sebastiaan van Stijn
30ecc0ea8a
Merge pull request #47482 from akerouanton/25.0-swarm-ipam-validation
[25.0 backport] Don't enforce new validation rules for existing networks
2024-03-01 11:58:18 +01:00
Sebastiaan van Stijn
06767446fe
Merge pull request #47481 from akerouanton/25.0-internal-bridge
[25.0 backport] Make 'internal' bridge networks accessible from host
2024-03-01 11:57:38 +01:00
Sebastiaan van Stijn
7048a63686
Merge pull request #47486 from akerouanton/25.0-go-1.21.7
[25.0 backport] update to go1.21.7
2024-03-01 11:56:23 +01:00
Paweł Gronowski
81fb7f9986
Merge pull request #47487 from akerouanton/25.0-integration-testdaemonproxy-reset-otel
[25.0 backport] integration: Reset `OTEL_EXPORTER_OTLP_ENDPOINT` for sub-daemons
2024-03-01 11:46:42 +01:00
Sebastiaan van Stijn
b77bb69f87
Merge pull request #47483 from akerouanton/25.0-best-effort-xattrs-classic-builder
[25.0 backport] builder/dockerfile: ADD with best-effort xattrs
2024-03-01 11:03:34 +01:00
CrazyMax
7a4abb8c77 ci: set codecov token
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
(cherry picked from commit 38827ba290)
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-03-01 10:38:53 +01:00
Rob Murray
81a83f0544 Simplify macvlan/ipvlan integration test structure
Signed-off-by: Rob Murray <rob.murray@docker.com>
(cherry picked from commit 9faf4855d5)
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-03-01 10:34:52 +01:00
Rob Murray
abcd6f8a46 Run the macvlan/ipvlan integration tests
The problem was accidentally introduced in:
  e8dc902781

Signed-off-by: Rob Murray <rob.murray@docker.com>
(cherry picked from commit 4eb95d01bc)
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-03-01 10:34:52 +01:00
Paweł Gronowski
f7be6dcba6 integration: Reset OTEL_EXPORTER_OTLP_ENDPOINT for sub-daemons
When creating a new daemon in the `TestDaemonProxy`, reset the
`OTEL_EXPORTER_OTLP_ENDPOINT` to an empty value to disable OTEL
collection to avoid it hitting the proxy.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 5fe96e234d)
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-03-01 10:30:13 +01:00
Sebastiaan van Stijn
10609544e5 update to go1.21.7
go1.21.7 (released 2024-02-06) includes fixes to the compiler, the go command,
the runtime, and the crypto/x509 package. See the Go 1.21.7 milestone on our
issue tracker for details:

- https://github.com/golang/go/issues?q=milestone%3AGo1.21.7+label%3ACherryPickApproved
- full diff: https://github.com/golang/go/compare/go1.21.6...go1.21.7

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 7c2975d2df)
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-03-01 10:20:49 +01:00
Paweł Gronowski
be59afce2d c8d/pull: Output truncated id for Pulling fs layer
All other progress updates are emitted with truncated id.

```diff
$ docker pull --platform linux/amd64 alpine
Using default tag: latest
latest: Pulling from library/alpine
-sha256:4abcf20661432fb2d719aaf90656f55c287f8ca915dc1c92ec14ff61e67fbaf8: Pulling fs layer
+4abcf2066143: Download complete
Digest: sha256:c5b1261d6d3e43071626931fc004f70149baeba2c8ec672bd4f27761f8e1ad6b
Status: Image is up to date for alpine:latest
docker.io/library/alpine:latest
```

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 16aa7dd67f)
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-03-01 10:10:04 +01:00
Paweł Gronowski
97951c39fb c8d/pull: Don't emit Downloading with 0 progress
To align with the graphdrivers behavior and don't send unnecessary
progress messages.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 14df52b709)
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-03-01 10:09:41 +01:00
Paweł Gronowski
2001813571 c8d/pull: Emit Pulling fs layer
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit ff5f780f2b)
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-03-01 10:09:37 +01:00
Paweł Gronowski
8e3bcf1974 pkg/streamformatter: Make progressOutput concurrency safe
Sync access to the underlying `io.Writer` with a mutex.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 5689dabfb3)
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-03-01 10:09:32 +01:00
Cory Snider
27f36f42a4 builder/dockerfile: ADD with best-effort xattrs
Archives being unpacked by Dockerfiles may have been created on other
OSes with different conventions and semantics for xattrs, making them
impossible to apply when extracting. Restore the old best-effort xattr
behaviour users have come to depend on in the classic builder.

The (archive.Archiver).UntarPath function does not allow the options
passed to Untar to be customized. It also happens to be a trivial
wrapper around the Untar function. Inline the function body and add the
option.

Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit 5bcd2f6860)
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-03-01 10:01:09 +01:00
Rob Murray
1ae019fca2 Don't enforce new validation rules for existing networks
Non-swarm networks created before network-creation-time validation
was added in 25.0.0 continued working, because the checks are not
re-run.

But, swarm creates networks when needed (with 'agent=true'), to
ensure they exist on each agent - ignoring the NetworkNameError
that says the network already existed.

By ignoring validation errors on creation of a network with
agent=true, pre-existing swarm networks with IPAM config that would
fail the new checks will continue to work too.

New swarm (overlay) networks are still validated, because they are
initially created with 'agent=false'.

Signed-off-by: Rob Murray <rob.murray@docker.com>
(cherry picked from commit 571af915d5)
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-03-01 09:54:46 +01:00