Commit graph

47575 commits

Author SHA1 Message Date
Paweł Gronowski
5fa011dc0c
vendor: resenje.org/singleflight v0.4.0
Fixes the context aware singleflight not preserving context values.

full diff: https://github.com/janos/singleflight/compare/v0.3.0...v0.4.0

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-12 09:34:03 +02:00
Sebastiaan van Stijn
fd15ddbc30
daemon: GetImageAndReleasableLayer: simplify "FROM scratch" case
Windows doesn't support "FROM scratch", and the platform was only used
for validation on other platforms if a platform was provided, so no need
to set defaults.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-11 23:18:13 +02:00
Sebastiaan van Stijn
4229032676
daemon/containerd: use "DiffID" type for diff fields
strong-type the fields with the expected type, to make it more explicit
what we're expecting here.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-11 23:15:04 +02:00
Sebastiaan van Stijn
cdb5947316
Merge pull request #46455 from akerouanton/enable-ipv6-automatically
daemon: automatically set network EnableIPv6 if needed
2023-09-11 22:18:22 +02:00
Albin Kerouanton
5d5eeac310
daemon: automatically set network EnableIPv6 if needed
PR 4f47013feb added a validation step to `NetworkCreate` to ensure
no IPv6 subnet could be set on a network if its `EnableIPv6` parameter
is false.

Before that, the daemon was accepting such request but was doing nothing
with the IPv6 subnet.

This validation step is now deleted, and we automatically set
`EnableIPv6` if an IPv6 subnet was specified.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-09-11 20:53:29 +02:00
Sebastiaan van Stijn
51d647122a
Merge pull request #46383 from vvoland/c8d-legacybuilder-fix-layer-parent-snapshot
c8d/legacybuilder: Assorted fixes
2023-09-11 20:41:59 +02:00
Djordje Lukic
29dcf646a3
Merge pull request #46454 from rumpl/c8d-disable-schema1 2023-09-11 19:55:43 +02:00
Djordje Lukic
d0d3ddd045
Merge pull request #46375 from rumpl/c8d-userns-remap
c8d: Handle userns properly
2023-09-11 19:04:35 +02:00
Djordje Lukic
0313544f4a
c8d: Handle userns properly
If the daemon is run with --userns-remap we need to chown the prepared
snapshot

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-09-11 16:39:29 +02:00
Paweł Gronowski
c63a952dc9
c8d/builder: Don't append empty tar layer to manifest
To match the number of layers in config created in
`images.CreateChildImage`.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-11 16:22:44 +02:00
Paweł Gronowski
9127285985
c8d/builder: Lease layer snapshots
Create a lease for the snapshot and hold it until the layer is released.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-11 16:22:39 +02:00
Paweł Gronowski
f22b112005
c8d/commit: Unpack committed image
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-11 16:20:14 +02:00
Paweł Gronowski
8832cdf6e1
c8d: Common unpack code for specific manifest
Use `ImageService.unpackImage` when we want to unpack an image and we
know the exact platform-manifest to be unpacked beforehand.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-11 16:20:10 +02:00
Paweł Gronowski
c6bded3475
c8d/builder: Don't use diffID as snapshot parent
DiffID is only a digest of the one tar layer and matches the snapshot ID
only for the first layer (DiffID = ChainID).

Instead of generating random ID as a key for rolayer, just use the
snapshot ID of the unpacked image content and use it later as a parent
for creating a new RWLayer.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-11 16:20:04 +02:00
Paweł Gronowski
8c7e19c5ff
c8d/builder: Set empty diffID for rolayer
diffID is the digest of a tar archive containing changes to the parent
layer - rolayer doesn't have any changes to the parent.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-11 16:20:02 +02:00
Paweł Gronowski
6ebfa57364
c8d/builder: Don't mount the rolayer snapshot
The view snapshot and its mounts are not used.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-11 16:19:59 +02:00
Sebastiaan van Stijn
c41121cc48
update to go1.20.8
go1.20.8 (released 2023-09-06) includes two security fixes to the html/template
package, as well as bug fixes to the compiler, the go command, the runtime,
and the crypto/tls, go/types, net/http, and path/filepath packages. See the
Go 1.20.8 milestone on our issue tracker for details:

https://github.com/golang/go/issues?q=milestone%3AGo1.20.8+label%3ACherryPickApproved

full diff: https://github.com/golang/go/compare/go1.20.7...go1.20.8

From the security mailing:

[security] Go 1.21.1 and Go 1.20.8 are released

Hello gophers,

We have just released Go versions 1.21.1 and 1.20.8, minor point releases.

These minor releases include 4 security fixes following the security policy:

- cmd/go: go.mod toolchain directive allows arbitrary execution
  The go.mod toolchain directive, introduced in Go 1.21, could be leveraged to
  execute scripts and binaries relative to the root of the module when the "go"
  command was executed within the module. This applies to modules downloaded using
  the "go" command from the module proxy, as well as modules downloaded directly
  using VCS software.

  Thanks to Juho Nurminen of Mattermost for reporting this issue.

  This is CVE-2023-39320 and Go issue https://go.dev/issue/62198.

- html/template: improper handling of HTML-like comments within script contexts
  The html/template package did not properly handle HMTL-like "<!--" and "-->"
  comment tokens, nor hashbang "#!" comment tokens, in <script> contexts. This may
  cause the template parser to improperly interpret the contents of <script>
  contexts, causing actions to be improperly escaped. This could be leveraged to
  perform an XSS attack.

  Thanks to Takeshi Kaneko (GMO Cybersecurity by Ierae, Inc.) for reporting this
  issue.

  This is CVE-2023-39318 and Go issue https://go.dev/issue/62196.

- html/template: improper handling of special tags within script contexts
  The html/template package did not apply the proper rules for handling occurrences
  of "<script", "<!--", and "</script" within JS literals in <script> contexts.
  This may cause the template parser to improperly consider script contexts to be
  terminated early, causing actions to be improperly escaped. This could be
  leveraged to perform an XSS attack.

  Thanks to Takeshi Kaneko (GMO Cybersecurity by Ierae, Inc.) for reporting this
  issue.

  This is CVE-2023-39319 and Go issue https://go.dev/issue/62197.

- crypto/tls: panic when processing post-handshake message on QUIC connections
  Processing an incomplete post-handshake message for a QUIC connection caused a panic.

  Thanks to Marten Seemann for reporting this issue.

  This is CVE-2023-39321 and CVE-2023-39322 and Go issue https://go.dev/issue/62266.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-11 15:47:00 +02:00
Djordje Lukic
4acbf7d8c3
c8d: disable schema1 registry integration tests
schema1 was deprecated a while ago, containerd fails to push to a
schema1 registry, let's just skip these tests for the containerd
integration

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-09-11 15:30:44 +02:00
Sebastiaan van Stijn
ccae064209
Merge pull request #46423 from vvoland/c8d-images-all
c8d/builder: Store parent in c8d image label
2023-09-11 14:51:20 +02:00
Djordje Lukic
833b514d98
c8d/mount: Create same directory owners as graphdrivers
Graph drivers create the parent directory with
rootPair().GID:CurrentIdentity().UID owner. This change brings these in
line

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-09-11 13:24:42 +02:00
Sebastiaan van Stijn
f8a4b6755d
Merge pull request #46442 from thaJeztah/less_polling
integration: don't poll for containers to be running
2023-09-11 13:15:38 +02:00
Richard Hansen
12e27dfd8f Fix host_ipv4 bridge option when IPv6 and ip6tables are enabled
Before this commit, setting the `com.docker.network.host_ipv4` bridge
option when `enable_ipv6` is true and the experimental `ip6tables`
option is enabled would cause Docker to fail to create the network:

> failed to create network `test-network`: Error response from daemon:
> Failed to Setup IP tables: Unable to enable NAT rule: (iptables
> failed: `ip6tables --wait -t nat -I POSTROUTING -s fd01::/64 ! -o
> br-test -j SNAT --to-source 192.168.0.2`: ip6tables
> v1.8.7 (nf_tables): Bad IP address "192.168.0.2"
>
> Try `ip6tables -h` or `ip6tables --help` for more information.
>  (exit status 2))

Fix this error by passing nil -- not the `host_ipv4` address -- when
creating the IPv6 rules.

Signed-off-by: Richard Hansen <rhansen@rhansen.org>
2023-09-10 04:03:07 -04:00
Sebastiaan van Stijn
7c0f6741eb
go mod tidy
Looks like some dependencies ended up on in the wrong list.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-09 00:53:46 +02:00
Sebastiaan van Stijn
cd204f1118
libnetwork/osl: Namespace.DeleteNeighbor: remove intermediate vars
- store linkIndex in a local variable so that it can be reused
- remove / rename some intermediate vars that shadowed existing declaration

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-08 23:46:14 +02:00
Sebastiaan van Stijn
4215a1542b
libnetwork/osl: Namespace.DeleteNeighbor: remove osDelete argument
This argument was originally added in libnetwork:
03f440667f

At the time, this argument was conditional, but currently it's always set
to "true", so let's remove it.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-08 23:46:14 +02:00
Sebastiaan van Stijn
bc77104fed
libnetwork/osl: Namespace.DeleteNeighbor: don't warn on non-existing neighbor
The code ignores these errors, but will unconditionally print a warning;

> If the kernel deletion fails for the neighbor entry still remote it
> from the namespace cache. Otherwise if the neighbor moves back to the
> same host again, kernel update can fail.

Let's reduce noise if the neighbor wasn't found, to prevent logs like:

    Aug 16 13:26:35 master1.local dockerd[4019880]: time="2023-08-16T13:26:35.186662370+02:00" level=warning msg="error while deleting neighbor entry" error="no such file or directory"
    Aug 16 13:26:35 master1.local dockerd[4019880]: time="2023-08-16T13:26:35.366585939+02:00" level=warning msg="error while deleting neighbor entry" error="no such file or directory"
    Aug 16 13:26:42 master1.local dockerd[4019880]: time="2023-08-16T13:26:42.366658513+02:00" level=warning msg="error while deleting neighbor entry" error="no such file or directory"

While changing this code, also slightly rephrase the code-comment, and
fix a typo ("remote -> remove").

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

libnetwork/osl: Namespace.DeleteNeighbor: rephrase code-comment

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-08 23:46:11 +02:00
Sebastiaan van Stijn
6ce5aa1cd5
Merge pull request #46434 from vvoland/test-ignore-cgroup-check
hack: Add TEST_IGNORE_CGROUP_CHECK
2023-09-08 23:23:18 +02:00
Sebastiaan van Stijn
c3ab3f1dde
Merge pull request #46412 from rumpl/c8d-pull-progress
c8d: Better pull progress
2023-09-08 23:10:23 +02:00
Sebastiaan van Stijn
d582a26107
Merge pull request #46439 from akerouanton/no-optimizations
hack: compile w/o optimizations & inlining when debugging
2023-09-08 23:09:33 +02:00
Sebastiaan van Stijn
5e7eade1f7
integration: don't poll for containers to be running
container.Run() should be a synchronous operation in normal circumstances;
the container is created and started, so polling after that for the
container to be in the "running" state should not be needed.

This should also prevent issues when a container (for whatever reason)
exited immediately after starting; in that case we would continue
polling for it to be running (which likely would never happen).

Let's skip the polling; if the container is not in the expected state
(i.e. exited), tests should fail as well.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-08 23:05:43 +02:00
Sebastiaan van Stijn
b7d29c1e23
registry: un-skip some tests that don't require root
These tests seem to be running fine without being root, so let's not skip them.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-08 18:35:09 +02:00
Sebastiaan van Stijn
de27973538
registry: merge validateEndpoint into newV1Endpoint
validateEndpoint was doing more than just validating; it was also implicitly
mutating the endpoint that was passed to it (by reference).

Given that validation only happend when constructing a new v1Endpoint, let's
merge these functions.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-08 18:31:17 +02:00
Albin Kerouanton
6cc6682f5f
daemon: fix under what conditions container's mac-address is applied
The daemon would pass an EndpointCreateOption to set the interface MAC
address if the network name and the provided network mode were matching.
Obviously, if the network mode is a network ID, it won't work. To make
things worse, the network mode is never normalized if it's a partial ID.

To fix that: 1. the condition under what the container's mac-address is
applied is updated to also match the full ID; 2. the network mode is
normalized to a full ID when it's only a partial one.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-09-08 18:15:00 +02:00
Sebastiaan van Stijn
97e782ee2b
Merge pull request #46422 from thaJeztah/fix_resize_tests
integration/container: fix flaky resize tests, and some cleaning up
2023-09-08 18:14:18 +02:00
Sebastiaan van Stijn
e6d9d216dd
Merge pull request #46432 from vvoland/c8d-skip-image-rm-test
integration-cli: Skip TestRmiParentImageFail when using c8d snapshotters
2023-09-08 17:44:55 +02:00
Albin Kerouanton
820f37b1a6
hack: compile w/o optimizations & inlining when debugging
Without these compile flags, Delve is unable to report the value of some
variables and it's not possible to jump into inlined code.

As the contributing docs already mention that `DOCKER_DEBUG` should
disable "build optimizations", the env var is reused here instead of
introducing a new one.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-09-08 17:16:58 +02:00
Sebastiaan van Stijn
c44f8958f6
vendor: github.com/cncf-tags/container-device-interface v0.6.1
Removes uses of the github.com/opencontainers/runc/libcontainer/devices
package.

full diff: https://github.com/cncf-tags/container-device-interface/compare/v0.6.0...v0.6.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-08 16:01:56 +02:00
Sebastiaan van Stijn
0434b653c8
Merge pull request #46435 from vvoland/c8d-inspect-prefix
c8d/image: Allow truncated id to have sha256: prefix
2023-09-08 15:59:24 +02:00
Sebastiaan van Stijn
fa3ddc5116
distribution: newRepository: remove naked return and intermediate vars
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-08 15:44:58 +02:00
Sebastiaan van Stijn
6dcefa3739
Merge pull request #46431 from rumpl/fix-ci-timeout
Use process substitution to redirect to tee
2023-09-08 15:28:09 +02:00
Sebastiaan van Stijn
78ccd5d07c
Merge pull request #46433 from rumpl/c8d-remove-panic
c8d: Remove the panic from UpdateConfig
2023-09-08 15:17:53 +02:00
Sebastiaan van Stijn
43740ab642
registry: remove intermediate vars in mock
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-08 14:22:21 +02:00
Sebastiaan van Stijn
550e628f05
registry: newV1Endpoint: make it clear we skip validation for Docker Hub
validateEndpoint uses `v1Endpoint.ping` to verify if the search API can
use a secure connection, and to fall back to basic auth. For Docker Hub,
we don't allow insecure connections, and `v1Endpoint.ping` will not connect
to Docker Hub (Docker Hub also does not implement the `_ping` endpoint,
so doing so would always fail).

Let's make it more clear that we don't do any validation, and return
early.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-08 14:19:33 +02:00
Paweł Gronowski
5728516371
c8d/image: Allow truncated id to have sha256: prefix
Fixes TestInspectByPrefix when running with c8d snapshotters enabled.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-08 14:10:10 +02:00
Paweł Gronowski
c2004fb8c4
hack: Add TEST_IGNORE_CGROUP_CHECK
Add env variable that allows to run integration-cli tests with cgroup v2 enabled.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-08 13:41:03 +02:00
Sebastiaan van Stijn
7e506f71f3
registry: v1Endpoint.ping: include URL in debug log
Also remove log from `validateEndpoint`, because we don't actually
ping the default (Docker Hub).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-08 13:21:28 +02:00
Sebastiaan van Stijn
0c6f8f9290
registry: v1Endpoint.ping: don't io.Readall the response
We have the response available, which is an io.Reader, so we don't have
to read the entire response into memory before decoding.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-08 13:17:22 +02:00
Sebastiaan van Stijn
e9ad878df6
registry: v1Endpoint.ping: add fast-path for X-Docker-Registry-Standalone
This function was making a request to the `_ping` endpoint, which (if
implemented) would return a JSON response, which we unmarshal (the only
field we use from the response is the `Standalone` field).

However, if the response had a `X-Docker-Registry-Standalone`, that header
took precedence, and would overwrite the earlier `Standalone` value we
obtained from the JSON response.

This patch adds a fast-path for situations where the header is present,
in which case we can skip handling the JSON response altogether.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-08 13:17:16 +02:00
Djordje Lukic
9e797a6db9
c8d: Remove the panic from UpdateConfig
We don't really want the daemon to panic for this so let's log a warning
about max downloads and uploads

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-09-08 12:21:16 +02:00
Paweł Gronowski
4982c70db4
integration-cli: Skip TestRmiParentImageFail when using c8d snapshotters
With containerd image store the images don't depend on each other even
if they share the same content and it's totally fine to delete the
"parent" image.

The skip is necessary because deleting the "parent" image does not
produce an error with the c8d image store and deleting the `busybox`
image breaks other tests.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-08 12:08:35 +02:00