Commit graph

48972 commits

Author SHA1 Message Date
Bjorn Neergaard
1f539a6e85
Merge pull request #47569 from thaJeztah/make_fix_empty_check
Makefile: generate-files: fix check for empty TMP_OUT
2024-03-15 12:07:00 -06:00
Bjorn Neergaard
959c2ee6cf
Merge pull request #47558 from AkihiroSuda/fix-47248
plugin: fix mounting /etc/hosts when running in UserNS
2024-03-15 12:06:48 -06:00
Sebastiaan van Stijn
25c9e6e8df
Makefile: generate-files: fix check for empty TMP_OUT
commit c655b7dc78 added a check to make sure
the TMP_OUT variable was not set to an empty value, as such a situation would
perform an `rm -rf /**` during cleanup.

However, it was a bit too eager, because Makefile conditionals (`ifeq`) are
evaluated when parsing the Makefile, which happens _before_ the make target
is executed.

As a result `$@_TMP_OUT` was always empty when the `ifeq` was evaluated,
making it not possible to execute the `generate-files` target.

This patch changes the check to use a shell command to evaluate if the var
is set to an empty value.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-03-15 17:54:54 +01:00
Akihiro Suda
762ec4b60c
plugin: fix mounting /etc/hosts when running in UserNS
Fix `error mounting "/etc/hosts" to rootfs at "/etc/hosts": mount
/etc/hosts:/etc/hosts (via /proc/self/fd/6), flags: 0x5021: operation
not permitted`.

This error was introduced in 7d08d84b03
(`dockerd-rootless.sh: set rootlesskit --state-dir=DIR`) that changed
the filesystem of the state dir from /tmp to /run (in a typical setup).

Fix issue 47248

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2024-03-15 22:16:34 +09:00
Sebastiaan van Stijn
979f03f9f6
Merge pull request #47567 from thaJeztah/move_rootless_mountopts
daemon: move getUnprivilegedMountFlags to internal package
2024-03-15 14:13:23 +01:00
Sebastiaan van Stijn
7b414f5703
daemon: move getUnprivilegedMountFlags to internal package
This code is currently only used in the daemon, but is also needed in other
places. We should consider moving this code to github.com/moby/sys, so that
BuildKit can also use the same implementation instead of maintaining a fork;
moving it to internal allows us to reuse this code inside the repository, but
does not allow external consumers to depend on it (which we don't want as
it's not a permanent location).

As our code only uses this in linux files, I did not add a stub for other
platforms (but we may decide to do that in the moby/sys repository).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-03-15 12:55:09 +01:00
Sebastiaan van Stijn
ff05850e7e
Merge pull request #47563 from vvoland/buildkit-runc-override
builder-next: Add env-var to override runc used by buildkit
2024-03-14 20:17:01 +01:00
Sebastiaan van Stijn
cdf70c0a51
Merge pull request #47430 from vvoland/inspect-remove-container
api/image-inspect: Remove Container and ContainerConfig
2024-03-14 19:27:43 +01:00
Sebastiaan van Stijn
40c681355e
Merge pull request #47562 from thaJeztah/update_protobuf
vendor: google.golang.org/protobuf v1.33.0, github.com/golang/protobuf v1.5.4
2024-03-14 19:14:00 +01:00
Albin Kerouanton
790c3039d0 libnet: Don't forward to upstream resolvers on internal nw
Commit cbc2a71c2 makes `connect` syscall fail fast when a container is
only attached to an internal network. Thanks to that, if such a
container tries to resolve an "external" domain, the embedded resolver
returns an error immediately instead of waiting for a timeout.

This commit makes sure the embedded resolver doesn't even try to forward
to upstream servers.

Co-authored-by: Albin Kerouanton <albinker@gmail.com>
Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-03-14 17:46:48 +00:00
Paweł Gronowski
10bdc7136c
builder-next: Add env-var to override runc used by buildkit
Adds an experimental `DOCKER_BUILDKIT_RUNC_COMMAND` variable that allows
to specify different runc-compatible binary to be used by the buildkit's
runc executor.

This allows runtimes like sysbox be used for the containers spawned by
buildkit.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-14 17:35:26 +01:00
Paweł Gronowski
a8abb67c5e
Merge pull request #47561 from thaJeztah/bump_tools
Dockerfile: update buildx to v0.13.1,  compose v2.24.7
2024-03-14 13:46:24 +01:00
Sebastiaan van Stijn
1ca89d7eae
vendor: google.golang.org/protobuf v1.33.0, github.com/golang/protobuf v1.5.4
full diffs:

- https://github.com/protocolbuffers/protobuf-go/compare/v1.31.0...v1.33.0
- https://github.com/golang/protobuf/compare/v1.5.3...v1.5.4

From the Go security announcement list;

> Version v1.33.0 of the google.golang.org/protobuf module fixes a bug in
> the google.golang.org/protobuf/encoding/protojson package which could cause
> the Unmarshal function to enter an infinite loop when handling some invalid
> inputs.
>
> This condition could only occur when unmarshaling into a message which contains
> a google.protobuf.Any value, or when the UnmarshalOptions.UnmarshalUnknown
> option is set. Unmarshal now correctly returns an error when handling these
> inputs.
>
> This is CVE-2024-24786.

In a follow-up post;

> A small correction: This vulnerability applies when the UnmarshalOptions.DiscardUnknown
> option is set (as well as when unmarshaling into any message which contains a
> google.protobuf.Any). There is no UnmarshalUnknown option.
>
> In addition, version 1.33.0 of google.golang.org/protobuf inadvertently
> introduced an incompatibility with the older github.com/golang/protobuf
> module. (https://github.com/golang/protobuf/issues/1596) Users of the older
> module should update to github.com/golang/protobuf@v1.5.4.

govulncheck results in our code:

    govulncheck ./...
    Scanning your code and 1221 packages across 204 dependent modules for known vulnerabilities...

    === Symbol Results ===

    Vulnerability #1: GO-2024-2611
        Infinite loop in JSON unmarshaling in google.golang.org/protobuf
      More info: https://pkg.go.dev/vuln/GO-2024-2611
      Module: google.golang.org/protobuf
        Found in: google.golang.org/protobuf@v1.31.0
        Fixed in: google.golang.org/protobuf@v1.33.0
        Example traces found:
          #1: daemon/logger/gcplogs/gcplogging.go:154:18: gcplogs.New calls logging.Client.Ping, which eventually calls json.Decoder.Peek
          #2: daemon/logger/gcplogs/gcplogging.go:154:18: gcplogs.New calls logging.Client.Ping, which eventually calls json.Decoder.Read
          #3: daemon/logger/gcplogs/gcplogging.go:154:18: gcplogs.New calls logging.Client.Ping, which eventually calls protojson.Unmarshal

    Your code is affected by 1 vulnerability from 1 module.
    This scan found no other vulnerabilities in packages you import or modules you
    require.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-03-14 13:12:54 +01:00
Sebastiaan van Stijn
f40bdf5f63
Dockerfile: update compose to v2.24.7
full diff: https://github.com/docker/compose/compare/v2.24.5...v2.24.7

release notes:

- https://github.com/docker/compose/releases/tag/v2.24.6
- https://github.com/docker/compose/releases/tag/v2.24.7

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-03-14 12:40:26 +01:00
Sebastiaan van Stijn
3f73d23ea0
Dockerfile: update buildx to v0.13.1
release notes:

- https://github.com/docker/buildx/releases/tag/v0.13.1
- https://github.com/docker/buildx/releases/tag/v0.13.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-03-14 12:38:06 +01:00
Paweł Gronowski
77b05b97f4
Merge pull request #47556 from vvoland/deprecate-notls
Be more explicit about non-TLS TCP access deprecation
2024-03-14 12:07:42 +01:00
Lei Jitang
e3bc82f7d4
Merge pull request #47542 from eriksjolund/47407-clarify-git-clone
set-up-git.md: clarify URL in git clone command
2024-03-14 17:24:02 +08:00
Sebastiaan van Stijn
342923b01c
Merge pull request #47555 from rumpl/feat-c8d-prom
c8d: Prometheus metrics
2024-03-13 17:35:14 +01:00
Sebastiaan van Stijn
15122b3b1c
Merge pull request #47350 from vvoland/cache-refactor
c8d/cache: Use the same cache logic as graphdrivers
2024-03-13 17:19:36 +01:00
Djordje Lukic
388ecf65bc
c8d: Send push metrics to prom
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2024-03-13 15:03:42 +01:00
Djordje Lukic
bb3ab1edb7
c8d: Send pull metrics to prom
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2024-03-13 15:03:42 +01:00
Djordje Lukic
da245cab15
c8d: Send history metrics to prometheus
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2024-03-13 15:03:42 +01:00
Djordje Lukic
1cfd763214
c8d: Send image delete metrics to prometheus
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2024-03-13 15:03:42 +01:00
Djordje Lukic
0ce714a085
images: Export the image actions prometheus counter
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2024-03-13 15:03:36 +01:00
Paweł Gronowski
bcb4794eea
Be more explicit about non-TLS TCP access deprecation
Turn warnings into a deprecation notice and highlight that it will
prevent daemon startup in future releases.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-13 14:22:10 +01:00
Paweł Gronowski
0d5ef431a1
docker-py: Temporarily skip test_commit and test_commit_with_changes
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-13 13:03:48 +01:00
Paweł Gronowski
03cddc62f4
api/image-inspect: Remove Container and ContainerConfig
Don't include these fields starting from API v1.45.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-13 13:02:29 +01:00
Akihiro Suda
825635a5bf
Merge pull request #47552 from thaJeztah/vendor_containerd_1.7.14
vendor: github.com/containerd/containerd v1.7.14
2024-03-13 11:57:52 +09:00
Sebastiaan van Stijn
ec19fd6fed
vendor: github.com/containerd/containerd v1.7.14
- full diff: https://github.com/containerd/containerd/compare/v1.7.13...v1.7.14
- release notes: https://github.com/containerd/containerd/releases/tag/v1.7.14

Welcome to the v1.7.14 release of containerd!

The fourteenth patch release for containerd 1.7 contains various fixes and updates.

Highlights

- Update builds to use go 1.21.8
- Fix various timing issues with docker pusher
- Register imagePullThroughput and count with MiB
- Move high volume event logs to Trace level

Container Runtime Interface (CRI)

- Handle pod transition states gracefully while listing pod stats

Runtime

- Update runc-shim to process exec exits before init

Dependency Changes

- github.com/containerd/nri v0.4.0 -> v0.6.0
- github.com/containerd/ttrpc v1.2.2 -> v1.2.3
- google.golang.org/genproto/googleapis/rpc 782d3b101e98 -> cbb8c96f2d6d

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-03-12 12:46:19 +01:00
Sebastiaan van Stijn
d19f6d4b6d
vendor: github.com/containerd/ttrpc v1.2.3
full diff: https://github.com/containerd/ttrpc/compare/v1.2.2..v1.2.3

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-03-12 12:37:44 +01:00
Sebastiaan van Stijn
b8165a9cd1
Merge pull request #47494 from vvoland/devcontainer-golang
devcontainer: Add Golang extension
2024-03-11 17:50:13 +01:00
Erik Sjölund
a6a445d86b set-up-git.md: clarify URL in git clone command
Fixes https://github.com/moby/moby/issues/47407

Signed-off-by: Erik Sjölund <erik.sjolund@gmail.com>
2024-03-09 16:42:44 +01:00
Sebastiaan van Stijn
0fb845858d
Merge pull request #47505 from akerouanton/fix-TestBridgeICC-ipv6
inte/networking:  ping with -6 specified when needed
2024-03-08 18:33:46 +01:00
Paweł Gronowski
db2263749b
Merge pull request #47530 from vvoland/flaky-liverestore
volume: Don't decrement refcount below 0
2024-03-08 12:28:10 +01:00
Sebastiaan van Stijn
1abf17c779
Merge pull request #47512 from robmry/46329_internal_resolver_ipv6_upstream
Add IPv6 nameserver to the internal DNS's upstreams.
2024-03-07 21:21:12 +01:00
Paweł Gronowski
294fc9762e
volume: Don't decrement refcount below 0
With both rootless and live restore enabled, there's some race condition
which causes the container to be `Unmount`ed before the refcount is
restored.

This makes sure we don't underflow the refcount (uint64) when
decrementing it.

The root cause of this race condition still needs to be investigated and
fixed, but at least this unflakies the `TestLiveRestore`.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-07 20:42:20 +01:00
Paweł Gronowski
eef352b565
devcontainer: Use a separate devcontainer target
Use a separate `devcontainer` Dockerfile target, this allows to include
the `gopls` in the devcontainer so it doesn't have to be installed by
the Go vscode extension.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-07 20:39:56 +01:00
Paweł Gronowski
f4c696eef1
Merge pull request #47449 from vvoland/c8d-list-single
c8d/list: Add test and combine size
2024-03-07 18:49:19 +01:00
Albin Kerouanton
5a009cdd5b inte/networking: add isIPv6 flag
Make sure the `ping` command used by `TestBridgeICC` actually has
the `-6` flag when it runs IPv6 test cases. Without this flag,
IPv6 connectivity isn't tested properly.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-03-07 17:55:53 +01:00
Paweł Gronowski
2f1a32e3e5
c8d/list: Skip images with non matching platform
Currently this won't have any real effect because the platform matcher
matches all platform and is only used for sorting.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-07 16:27:12 +01:00
Paweł Gronowski
72f1f82f28
c8d/list: Remove outdated TODO
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-07 16:27:10 +01:00
Paweł Gronowski
52a80b40e2
c8d/list: Extract imageSummary function
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-07 16:27:09 +01:00
Paweł Gronowski
288a14e264
c8d/list: Simplify "best" image selection
Don't save all present images,  inline the sorting into the loop
instead.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-07 16:27:07 +01:00
Paweł Gronowski
b37ced2551
c8d/list: Count containers by their manifest
Move containers counting out of `singlePlatformImage` and count them
based on the `ImageManifest` property.

(also remove ChainIDs calculation as they're no longer used)

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-07 16:26:53 +01:00
Paweł Gronowski
a535a65c4b
c8d/list: Combine size
Multi-platform images are coalesced into one entry now.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-07 16:25:32 +01:00
Paweł Gronowski
582de4bc3c
c8d/list: Add TestImageList
Add unit test for `Images` implementation.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-07 16:25:31 +01:00
Paweł Gronowski
a6e7e67d3a
specialimage: Return optional ocispec.Index
To ease accessing image descriptors in tests.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-07 16:25:30 +01:00
Paweł Gronowski
1b108bdfeb
daemon/c8d: Cache SnapshotService
Avoid fetching `SnapshotService` from client every time. Fetch it once
and then store when creating the image service.

This also allows to pass custom snapshotter implementation for unit
testing.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-07 16:25:29 +01:00
Paweł Gronowski
74e2f23e1a
daemon/c8d: Use i.images and i.content
Use `image.Store` and `content.Store` stored in the ImageService struct
instead of fetching it every time from containerd client.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-07 16:25:27 +01:00
Paweł Gronowski
e8496b1ee4
imageService: Extract common code from MakeImageCache
Both containerd and graphdriver image service use the same code to
create the cache - they only supply their own `cacheAdaptor` struct.

Extract the shared code to `cache.New`.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-07 15:39:42 +01:00