Commit graph

48454 commits

Author SHA1 Message Date
Sebastiaan van Stijn
b12cf05c82
Merge pull request #47076 from AkihiroSuda/rootlesskit-v2
Update RootlessKit to v2.0.0 (support Pasta)
2024-01-16 12:12:25 +01:00
Paweł Gronowski
9f50e06993
image/save: Refactor saveLayer
`diffIDPaths` is not used and can be removed.

`savedConfig` stores if the config was already saved (ID of the image is
the ID of the config).
`savedLayers` stores if the layer (diff ID) was already saved.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-16 11:14:15 +01:00
Akihiro Suda
b5a5ecf4a3
dockerd-rootless.sh: add comments about pasta
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2024-01-16 10:56:30 +09:00
Akihiro Suda
f5d05b43d7
Dockerfile: update RootlessKit to v2.0.0
https://github.com/rootless-containers/rootlesskit/releases/tag/v2.0.0

=== Pasta ===
RootlessKit v2 adds the support for pasta (https://passt.top/passt/).
Pasta is similar to slirp4netns but its port forwarder achieves better
throughput than slirp4netns port driver.

It is still not faster than RootlessKit's `builtin` port driver, but unlike the
`builtin` port driver, pasta can retain source IP address information.

Network driver | Port driver    | Net throughput | Port throughput | Src IP | No SUID | Note
---------------|----------------|----------------|-----------------|--------|---------|--------------------------------------------
slirp4netns    | builtin        | Slow           | Fast          |      |       | Default in typical setup
vpnkit         | builtin        | Slow           | Fast          |      |       | Default when slirp4netns is not installed
slirp4netns    | slirp4netns    | Slow           | Slow            |      |       |
**pasta**      | **implicit**   | Slow           | Fast          |      |       | Experimental
lxc-user-nic   | builtin        | Fast         | Slow            |      |       | Experimental
(bypass4netns) | (bypass4netns) | Fast         | Fast          |      |       | (Not integrated to RootlessKit)

=== Detach-netns ===
Aside from pasta, RootlessKit v2 also brings the support for
"detach-netns" mode, which leaves the runtime in the host network namespace to
eliminate the slirp overhead for pull/push and to allow accessing the "real"
127.0.0.1.

See containerd/nerdctl PR 2723 for how detach-netns is being adopted in
nerdctl v2.

Integrating detach-netns into Docker/Moby will need an extra work and will be
deferred to Docker v26 (or later).

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2024-01-16 10:56:29 +09:00
Akihiro Suda
ba86626642
vendor.mod: github.com/rootless-containers/rootlesskit/v2 v2.0.0
https://github.com/rootless-containers/rootlesskit/releases/tag/v2.0.0

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2024-01-16 10:56:27 +09:00
Cory Snider
312450d079 integration: test container healthcheck is reset
Update the TestDaemonRestartKilContainers integration test to assert
that a container's healthcheck status is always reset to the Starting
state after a daemon restart, even when the container is live-restored.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-01-15 15:30:22 -05:00
Paweł Gronowski
c58acf06c2 image/save: Derive Descriptor from actual tar archive
Layer size is the sum of the individual files count, not the tar
archive. Use the total bytes read returned by `io.Copy` to populate the
`Size` field.

Also set the digest to the actual digest of the tar archive.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-15 16:35:54 +01:00
Sebastiaan van Stijn
3eba4216e0
Merge pull request #47080 from thaJeztah/bump_mountinfo
vendor: github.com/moby/sys/mountinfo v0.7.1
2024-01-15 13:11:50 +01:00
Sebastiaan van Stijn
6d90f1168b
Merge pull request #47079 from thaJeztah/bump_x_sys
vendor: golang.org/x/sys v0.16.0
2024-01-15 13:11:35 +01:00
Sebastiaan van Stijn
c4d2968183
vendor: github.com/moby/sys/mountinfo v0.7.1
full diff: https://github.com/moby/sys/compare/mountinfo/v0.6.2...mountinfo/v0.7.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-15 12:09:02 +01:00
Sebastiaan van Stijn
075e1141b8
vendor: golang.org/x/sys v0.16.0
full diff: https://github.com/golang/sys/compare/v0.15.0...v0.16.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-15 12:05:29 +01:00
Sebastiaan van Stijn
87041b207f
Merge pull request #46980 from thaJeztah/bump_compare
vendor: github.com/google/go-cmp v0.6.0, github.com/google/uuid v1.5.0
2024-01-15 11:55:41 +01:00
Sebastiaan van Stijn
770271d98f
Merge pull request #47075 from rapsealk/docs/doc-comments-on-volume-mounts-mountpoint-spec
doc: Correct typo in doc comments for volume/mounts/mounts/MountPoint
2024-01-15 10:44:47 +01:00
Sebastiaan van Stijn
a759acd958
vendor: github.com/google/uuid v1.5.0
full diff: https://github.com/google/uuid/compare/v1.3.1...v1.5.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-15 10:40:15 +01:00
Sebastiaan van Stijn
3bcb01a84f
vendor: github.com/google/go-cmp v0.6.0
full diff: https://github.com/google/go-cmp/compare/v0.5.9...v0.6.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-15 10:40:13 +01:00
Jeongseok Kang
36dd2f029f doc: Correct typo in doc comments for volume/mounts/mounts/MountPoint
Signed-off-by: Jeongseok Kang <piono623@naver.com>
2024-01-15 09:25:09 +09:00
Sebastiaan van Stijn
608238ead6
Merge pull request #46979 from thaJeztah/move_mergo
vendor: containerd v1.7.12, and switch to dario.cat/mergo v1.0.0
2024-01-12 21:14:59 +01:00
Sebastiaan van Stijn
3a15c6c9d7
Merge pull request #47070 from thaJeztah/bump_containerd_binary
update containerd binary to v1.7.12
2024-01-12 21:14:07 +01:00
Sebastiaan van Stijn
8f89f10b40
Merge pull request #47068 from vvoland/docker-save-fix-layers
image/save: Fix missing layers in manifest
2024-01-12 19:39:56 +01:00
Sebastiaan van Stijn
6e2e7641f9
Merge pull request #47067 from thaJeztah/bump_buildx_compose
Dockerfile: update buildx to v0.12.1
2024-01-12 18:37:10 +01:00
Sebastiaan van Stijn
92175dd78c
vendor: containerd v1.7.12, and switch to dario.cat/mergo v1.0.0
Switch github.com/imdario/mergo to dario.cat/mergo v1.0.0, because
the module was renamed, and reached v1.0.0

full diff: https://github.com/imdario/mergo/compare/v0.3.13...v1.0.0

vendor: github.com/containerd/containerd v1.7.12

- full diff: https://github.com/containerd/containerd/compare/v1.7.11...v1.7.12
- release notes: https://github.com/containerd/containerd/releases/tag/v1.7.12

Welcome to the v1.7.12 release of containerd!

The twelfth patch release for containerd 1.7 contains various fixes and updates.

Notable Updates

- Fix on dialer function for Windows
- Improve `/etc/group` handling when appending groups
- Update shim pidfile permissions to 0644
- Update runc binary to v1.1.11
- Allow import and export to reference missing content
- Remove runc import
- Update Go version to 1.20.13

Deprecation Warnings

- Emit deprecation warning for `containerd.io/restart.logpath` label usage

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-12 18:09:24 +01:00
Sebastiaan van Stijn
cd1709b0d4
update containerd binary to v1.7.12
- full diff: https://github.com/containerd/containerd/compare/v1.7.11...v1.7.12
- release notes: https://github.com/containerd/containerd/releases/tag/v1.7.12

Welcome to the v1.7.12 release of containerd!

The twelfth patch release for containerd 1.7 contains various fixes and updates.

Notable Updates

- Fix on dialer function for Windows
- Improve `/etc/group` handling when appending groups
- Update shim pidfile permissions to 0644
- Update runc binary to v1.1.11
- Allow import and export to reference missing content
- Remove runc import
- Update Go version to 1.20.13

Deprecation Warnings

- Emit deprecation warning for `containerd.io/restart.logpath` label usage

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-12 17:56:37 +01:00
Paweł Gronowski
2c45df2ab4 image/save: Fix missing layers in manifest
The new OCI-compatible archive export relies on the Descriptors returned
by the layer (`distribution.Describable` interface implementation).

The issue with that is that the `roLayer` and the `referencedCacheLayer`
types don't implement this interface. Implementing that interface for
them based on their `descriptor` doesn't work though, because that
descriptor is empty.

To workaround this issue, just create a new descriptor if the one
provided by the layer is empty.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-12 17:10:23 +01:00
Sebastiaan van Stijn
0790864064
Dockerfile: update buildx to v0.12.1
Update the version of buildx used in CI to the latest version.

- full diff: https://github.com/docker/buildx/compare/v0.12.0...v0.12.1
- release notes: https://github.com/docker/buildx/releases/tag/v0.12.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-12 16:13:34 +01:00
Sebastiaan van Stijn
ac6c791880
Merge pull request #47066 from thaJeztah/bump_compose
Dockerfile: update docker compose to v2.24.0
2024-01-12 16:12:58 +01:00
Sebastiaan van Stijn
a9d0128bd7
Dockerfile: update docker compose to v2.24.0
Update the version of compose used in CI to the latest version.

- full diff: https://github.com/docker/compose/compare/v2.20.1...v2.24.0
- release notes: https://github.com/docker/compose/releases/tag/v2.24.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-12 14:43:40 +01:00
Sebastiaan van Stijn
92cc0d6257
Merge pull request #47060 from vvoland/c8d-builder-commit-shareimagecreate
c8d/commit-builder: Extract common code
2024-01-11 14:37:20 +01:00
Paweł Gronowski
28f48ce1ac
c8d/commit-builder: Extract common code
Extract duplicated image creation code to a function.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-11 13:30:40 +01:00
Sebastiaan van Stijn
ef1a85ca5e
Merge pull request #47052 from corhere/libc8d/create-unstarted-tasks
libcontainerd: create unstarted tasks
2024-01-10 20:41:53 +01:00
Cory Snider
659d7b190f libcontainerd: create unstarted tasks
Split task creation and start into two separate method calls in the
libcontainerd API. Clients now have the opportunity to inspect the
freshly-created task and customize its runtime environment before
starting execution of the user-specified binary.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-01-10 13:50:26 -05:00
Sebastiaan van Stijn
7af5682b3c
Merge pull request #47053 from thaJeztah/update_golang_1.21.6
update to go1.21.6
2024-01-10 17:51:03 +01:00
Sebastiaan van Stijn
0fb6ba4ccc
Merge pull request #47048 from thaJeztah/fix_version_output
daemon/config: setPlatformDefaults: use debug for missing userland-proxy
2024-01-10 17:15:08 +01:00
Sebastiaan van Stijn
d7141cfd06
update to go1.21.6
go1.21.6 (released 2024-01-09) includes fixes to the compiler, the runtime, and
the crypto/tls, maps, and runtime/pprof packages. See the Go 1.21.6 milestone on
our issue tracker for details:

- https://github.com/golang/go/issues?q=milestone%3AGo1.21.6+label%3ACherryPickApproved
- full diff: https://github.com/golang/go/compare/go1.21.5...go1.21.6

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-10 09:46:37 +01:00
Cory Snider
0e62dbadcd daemon: reinit health monitor on live-restore
The container may have been running without health probes for an
indeterminate amount of time. The container may have become unhealthy in
the interim. We should probe it sooner than in steady-state, while also
giving it some leeway to recover from e.g. timed-out connections. This
is easy to achieve by probing the container like a freshly-started one.
The original author of health-checks came to the same conclusion; the
health monitor was reinitialized on live-restored containers before
v17.11.0, when health monitoring of live-restored containers was
accidentally broken. Revert to the original behavior.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-01-09 11:03:06 -05:00
Sebastiaan van Stijn
afc7e581e6
Merge pull request #47049 from vvoland/c8d-fix-labels-check-filter
c8d/getImageLabelByDigest: Fix misspelled `labels` check
2024-01-09 15:55:57 +01:00
Sebastiaan van Stijn
72303ef11a
Merge pull request #47050 from thaJeztah/rob_curator
Add Rob Murray (robmry) to curators
2024-01-09 15:40:54 +01:00
Sebastiaan van Stijn
4aa1bea04a
Merge pull request #47046 from thaJeztah/bump_rootlesskit
Dockerfile: update rootlesskit to v1.1.1, and use tags as reference
2024-01-09 13:28:37 +01:00
Sebastiaan van Stijn
71c9aec834
Add Rob Murray (robmry) to curators
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-09 13:20:44 +01:00
Sebastiaan van Stijn
e8e20c0897
daemon/config: setPlatformDefaults: use debug for missing userland-proxy
commit 4f9db655ed moved looking up the
userland-proxy binary to early in the startup process, and introduced
a log-message if the binary was missing.

However, a side-effect of this was this message would also be printed
when running "--version";

    dockerd --version
    time="2024-01-09T09:18:53.705271292Z" level=warning msg="failed to lookup default userland-proxy binary" error="exec: \"docker-proxy\": executable file not found in $PATH"
    Docker version v25.0.0-rc.1, build 9cebefa717

We should look if we can avoid this, but let's change the message to be
a debug message as a short-term workaround.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-09 13:18:04 +01:00
Paweł Gronowski
18a0e144c2
c8d/getImageLabelByDigest: Fix misspelled labels check
It should be `labels.*` not `label.*`.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-09 13:16:56 +01:00
Sebastiaan van Stijn
e27ffdab0f
Dockerfile: update rootlesskit to v1.1.1, and use tags as reference
Commit 0b1c1877c5 updated the version in
hack/dockerfile/install/rootlesskit.installer, but forgot to update the
version in Dockerfile.

Also updating both to use a tag, instead of commit. While it's good to pin by
an immutable reference, I think it's reasonably safe to use the tag, which is
easier to use, and what we do for other binaries, such as runc as well.

Full diff: https://github.com/rootless-containers/rootlesskit/compare/v1.1.0...v1.1.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-09 09:54:47 +01:00
Sebastiaan van Stijn
c41f94eda3
Merge pull request #47042 from thaJeztah/remove_tarsplit_deadcode
layer: ChecksumForGraphID: remove unused code for for migrating v1 layers
2024-01-08 22:40:14 +01:00
Sebastiaan van Stijn
22b08a10ff
Merge pull request #47032 from thaJeztah/otel_align_versions
vendor: otelhttptrace v0.45.0 to align with other OTEL packages
2024-01-08 17:07:16 +01:00
Sebastiaan van Stijn
97d91aff7b
Merge pull request #47027 from thaJeztah/bump_cgroups
vendor: github.com/containerd/cgroups/v3 v3.0.3
2024-01-08 13:09:33 +01:00
Sebastiaan van Stijn
13225400c8
Merge pull request #47039 from thaJeztah/bump_tarsplit
vendor: github.com/vbatts/tar-split v0.11.5
2024-01-08 13:09:07 +01:00
Sebastiaan van Stijn
adba0c2b63
Merge pull request #46202 from thaJeztah/daemon_deferrrrr
daemon: Daemon.connectToNetwork, Daemon.ContainerRename: improve err-handling in defers
2024-01-08 12:40:45 +01:00
Sebastiaan van Stijn
782fe1fe82
layer: ChecksumForGraphID: remove unused code for for migrating v1 layers
The only use is in `builder/builder-next/adapters/snapshot.EnsureLayer()`,
which always calls the function with an _empty_ `oldTarDataPath`;
7082aecd54/builder/builder-next/adapters/snapshot/layer.go (L81)

When called with an empty `oldTarDataPath`, this function was an alias for
`checksumForGraphIDNoTarsplit`, so let's make it that.

Note that this code was added in 500e77bad0, as
part of the migration from "v1" images to "v2" (content-addressable) images.

Given that the remaining code lives in a "migration" file, possibly more code
can be removed.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-08 12:28:41 +01:00
Sebastiaan van Stijn
1dd415df1a
vendor: github.com/vbatts/tar-split v0.11.5
full diff: https://github.com/vbatts/tar-split/compare/v0.11.3...v0.11.5

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-08 11:40:15 +01:00
Sebastiaan van Stijn
7082aecd54
Merge pull request #47030 from knight42/feat/check-wrapped-err
Make errdefs.Is<SomeError> helper functions work with wrapped errors
2024-01-08 09:40:32 +01:00
Sebastiaan van Stijn
e6f46e637c
Merge pull request #43697 from thaJeztah/builder_remove_removecontainer
Builder: some improvements on cleaning up containers
2024-01-08 09:37:58 +01:00