Commit graph

48607 commits

Author SHA1 Message Date
Paweł Gronowski
fcabc48ee4
c8d/load: Handle compressed archives
Allow to load images from compressed tars.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-10-09 14:03:22 +02:00
Albin Kerouanton
cbc2a71c27
libnet/d/bridge: Don't set container's gateway when network is internal
So far, internal networks were only isolated from the host by iptables
DROP rules. As a consequence, outbound connections from containers would
timeout instead of being "rejected" through an immediate ICMP dest/port
unreachable, a TCP RST or a failing `connect` syscall.

This was visible when internal containers were trying to resolve a
domain that don't match any container on the same network (be it a truly
"external" domain, or a container that don't exist/is dead). In that
case, the embedded resolver would try to forward DNS queries for the
different values of resolv.conf `search` option, making DNS resolution
slow to return an error, and the slowness being exacerbated by some libc
implementations.

This change makes `connect` syscall to return ENETUNREACH, and thus
solves the broader issue of failing fast when external connections are
attempted.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-10-09 13:57:54 +02:00
Albin Kerouanton
491758a446
Dockerfile: upgrade Delve
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-10-09 13:13:43 +02:00
Régis Behmo
c587ba3422 registry: allow mirror path prefix in config
Path prefixes were originally disallowed in the `--registry-mirrors`
option because the /v1 endpoint was assumed to be at the root of the
URI. This is no longer the case in v2.

Close #36598

Signed-off-by: Régis Behmo <regis@behmo.com>
2023-10-07 17:33:12 +02:00
Albin Kerouanton
2c4551d86d
libnet: resolver: remove direct use of logrus
This causes logs written through `r.log(ctx)` to not end in OTEL traces.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-10-06 19:14:48 +02:00
Albin Kerouanton
4de8459265
libnet: add OTEL tracing to the embedded DNS
This change creates a few OTEL spans and plumb context through the DNS
resolver and DNS backends (ie. Sandbox and Network). This should help
better understand how much lock contention impacts performance, and
help debug issues related to DNS queries (we basically have no
visibility into what's happening here right now).

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-10-06 19:14:48 +02:00
Paweł Gronowski
b8ba263099
c8d/rmi: Handle explicit dangling name
This isn't something that user should do, but technically the dangling
images exist in the image store and user can pass its name (`moby-dangling@digest`).
Change it so rmi now recognizes that it's actually a dangling image and
doesn't handle it like a regular tagged image.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-10-06 10:20:37 +02:00
Paweł Gronowski
25a813e924
c8d/rmi: Don't emit Untagged for dangling images
A dangling image isn't really a tag.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-10-06 10:20:35 +02:00
Brian Goff
f6fa56194f
Merge pull request #46448 from xinfengliu/improve-stats-collector
Make one-shot stats faster
2023-10-05 08:10:29 -07:00
Paweł Gronowski
ee77ba5e49
Merge pull request #46593 from vvoland/integration-cli-TestDaemonNoSpaceLeftOnDeviceError-new-containerd
TestDaemonNoSpaceLeftOnDeviceError: Adjust to snapshotters
2023-10-05 11:52:22 +02:00
Paweł Gronowski
f99c4ec798
c8d/history: Fill ID and Tags for all entries
Find the parent images by image set by the legacy builder.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-10-04 16:01:17 +02:00
Paweł Gronowski
1f6bacc486
TestDaemonNoSpaceLeftOnDeviceError: Adjust to snapshotters
- Pass empty containerd socket which forces the daemon to create a new
  supervised containerd. Otherwise a global containerd daemon will be
  used and the pulled image data will be stored in its data directory,
  instead of the the newly specified `data-root` that has a limited
  storage capacity.

- Don't try to use `vfs` snapshotter, instead use `native` which is
  containerd's equivalent for `vfs`.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-10-04 15:59:30 +02:00
Paweł Gronowski
28d057cb0e
c8d/TestPsListContainersSize: Only check if size increased
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-10-04 14:29:51 +02:00
Paweł Gronowski
34400dd4a2
TestPsListContainersSize: Fix size parsing/formatting
Use go-units to parse/format.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-10-04 14:29:47 +02:00
Paweł Gronowski
e0e6ae741e
daemon/c8d: Use WithoutCancel instead of context.Background
Instead of passing a completely fresh context without any values, just
discard the cancellation.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-10-04 13:17:51 +02:00
Brian Goff
c47cdd5f39 Revert "daemon/logger/etwlogs: rewrite to use go-winio/pkg/etw"
This reverts commit a9fa147a92.
The commit is unfortunately broken as it is still using `providerHandle`
to write events but that handle is never actually set, so it is always
invalid. All logging fails.

Note: This is note a straight revert due to the change to
containerd/log.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2023-10-03 22:42:36 +00:00
Sebastiaan van Stijn
bc2c8279de
Merge pull request #46582 from vvoland/c8d-push-defer-noncancellable-context
daemon/c8d: Use non cancellable context in defers
2023-10-03 07:55:12 -07:00
Sebastiaan van Stijn
77a1411f9b
Merge pull request #46575 from thaJeztah/dockerfile_distribution
Dockerfile: update test-registry to v2.8.3
2023-10-03 07:51:11 -07:00
Paweł Gronowski
6f27bef9fc
daemon/c8d: Use non cancellable context in defers
Fixes leases not being released when operation was cancelled.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-10-03 15:27:40 +02:00
Paweł Gronowski
77f7c83114
c8d/push: Show progress only on blobs
To match the graphdriver's push behavior which only shows the progress
for layers.
Exclude indexes, manifests and image configs from the push progress.
Don't explicitly check for `IsLayerType` to also handle other
potentially big blobs (like buildkit attestations).

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-10-03 14:59:19 +02:00
Sebastiaan van Stijn
e8a7a9435f
Dockerfile: update test-registry to v2.8.3
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-02 11:33:14 -07:00
Sebastiaan van Stijn
591c98cdbe
Merge pull request #46573 from cpuguy83/double_trace_logs
Remove duplicated trace logs
2023-10-01 06:50:29 +02:00
Brian Goff
5b16dd6469 Remove duplicated trace logs
This was mistakenly added to bklog.
Since this is getting attached to the standard logger, and bklog is
using the standard logger, we only need this added once.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2023-09-30 22:59:47 +00:00
Akihiro Suda
ad877271f3
Limit OOMScoreAdj when running in UserNS ("Rootful-in-Rootless")
Fix issue 46563 "Rootful-in-Rootless dind doesn't work since systemd v250 (due to oom score adj)"

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2023-09-29 21:50:08 +09:00
Djordje Lukic
7d30dafd69
Merge pull request #46543 from rumpl/c8d-fix-manifest-platform 2023-09-29 13:58:57 +02:00
Djordje Lukic
4a276afeb5
Merge pull request #46519 from rumpl/c8d-skip-config-image 2023-09-29 13:57:48 +02:00
Xinfeng Liu
3d70af4ee4 adjust stats sampling order
Signed-off-by: Xinfeng Liu <XinfengLiu@icloud.com>
2023-09-29 15:33:48 +08:00
Sebastiaan van Stijn
e2c23ed57d
Merge pull request #46069 from thaJeztah/update_to_go1.21
update to go1.21.1, default to GOTOOLCHAIN=local
2023-09-28 20:24:21 +02:00
Sebastiaan van Stijn
1239c96413
Merge pull request #46562 from neersighted/go_mod_bump
vendor.mod: bump go line to 1.20
2023-09-28 20:18:42 +02:00
Sebastiaan van Stijn
c4f55b53b0
Merge pull request #46561 from vvoland/c8d-integrationcli-skipsome
c8d: Adjust some integration-cli tests
2023-09-28 18:46:50 +02:00
Bjorn Neergaard
16063c7456
vendor.mod: bump go line to 1.20
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-09-28 10:01:25 -06:00
Paweł Gronowski
a4822abe37
c8d: Skip TestPullFailsWithAlteredManifest
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-28 17:13:38 +02:00
Paweł Gronowski
f9f439763a
c8d/TestPullFailsWithAlteredManifest: Adjust error message
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-28 17:13:36 +02:00
Paweł Gronowski
4001241ccf
c8d/TestPullByDigestNoFallback: Adjust error message
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-28 17:13:33 +02:00
Marat Radchenko
9e3ed7b429 pkg/chrootarchive: fix Darwin build
Before this commit, `doPack`, `doUnpack` and `doUnpackLayer` were not implemented for Darwin, causing build failure.

This change allows all non-Linux Unixes to use FreeBSD reexec-based pack/unpack implementation

See also: moby/buildkit#4059
See also: 8b843732b3

Signed-off-by: Marat Radchenko <marat@slonopotamus.org>
2023-09-28 16:23:33 +03:00
Sebastiaan van Stijn
e465ebf2f3
update to go1.21.1, and fix download-URL
This required changes to the download-URL, as downloads are now provided
using the full version (including the `.0` patch version);

    curl -sI https://go.dev/dl/go1.21.windows-amd64.zip | grep 'location'
    location: https://dl.google.com/go/go1.21.windows-amd64.zip

    curl -sI https://dl.google.com/go/go1.21.windows-amd64.zip
    HTTP/2 404
    # ...

    curl -sI https://dl.google.com/go/go1.21.0.windows-amd64.zip
    HTTP/2 200
    # ...

Unfortunately this also means that the GO_VERSION can no longer be set to
versions lower than 1.21.0 (without additional changes), because older
versions do NOT provide the `.0` version, and Go 1.21.0 and up, no longer
provides URLs _without_ the `.0` version.

Co-authored-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-28 00:51:34 +02:00
Sebastiaan van Stijn
aa282973d4
Dockerfile: use GOTOOLCHAIN=local
Related discussion in https://github.com/docker-library/golang/issues/472

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-28 00:51:18 +02:00
Djordje Lukic
f66c000229 c8d: Fix getting the platform of a single-platform image
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-09-28 00:16:51 +02:00
Sebastiaan van Stijn
a1d966c492
Merge pull request #46555 from thaJeztah/distribution_test_fixes
fix faulty tests for distribution-client
2023-09-27 17:06:01 +02:00
Paweł Gronowski
7c0c525764
Merge pull request #46552 from vvoland/internal-compatcontext
internal: Add compatcontext.WithoutCancel
2023-09-27 16:50:18 +02:00
Sebastiaan van Stijn
2c0ad62b24
Merge pull request #46554 from thaJeztah/remove_intermediates
remove some intermediate vars, and small refactor for error-handling
2023-09-27 16:06:46 +02:00
Sebastiaan van Stijn
2c89640ab9
distribution: TestPullSchema2Config fix test response
The test was depending on the client constructing an error based on the
http-status code, and the client not reading the response body if the
response was not a JSON response.

This fix;

- adds the correct content-type headers in the response
- includes error-messages in the response
- adds additional tests to cover both the plain (non-JSON) and JSON
  error responses, as well as an empty response.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-27 14:03:59 +02:00
Sebastiaan van Stijn
a9fcb775e5
integration-cli: fix getTestTokenService not sending header
This utility was setting the content-type header after WriteHeader was
called, and the header was not sent because of that.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-27 13:57:48 +02:00
Sebastiaan van Stijn
3197160114
daemon: Daemon.SetNetworkBootstrapKeys: make error-handling idiomatic
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-27 12:08:28 +02:00
Sebastiaan van Stijn
f91c3cfda6
daemon: Daemon.initNetworkController: remove intermediate var
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-27 12:08:28 +02:00
Sebastiaan van Stijn
dcc75e1563
libnetwork: Controller: agentInit, agentDriverNotify rm intermediate vars
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-27 12:08:28 +02:00
Sebastiaan van Stijn
a384102fdf
libnetwork/datastore: Store.Map, Store.List: remove intermediate vars
Inline the closures, and rename a var to be more descriptive.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-27 12:07:31 +02:00
Sebastiaan van Stijn
bb5402e6fb
libnetwork: Controller: getKeys, getPrimaryKeyTag: slight refactor
- use named return variables to make the function more self-describing
- rename variable for readability
- slightly optimize slice initialization, and keep linters happy

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-27 12:01:54 +02:00
Sebastiaan van Stijn
603f49706e
libnetwork: Controller: getKeys, getPrimaryKeyTag: prevent panic
Prevent potential panics if we don't have the expected number of keys
for the subsystem.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-27 12:01:54 +02:00
Paweł Gronowski
f6e44bc0e8
internal: Add compatcontext.WithoutCancel
Copy the implementation of `context.WithoutCancel` introduced in Go 1.21
to be able to use it when building with older versions.
This will use the stdlib directly when building with Go 1.21+.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-27 11:46:30 +02:00