Commit graph

48469 commits

Author SHA1 Message Date
Paweł Gronowski
24da5233dd
integration: Fix Parallel before setupTest
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-17 13:06:05 +01:00
Bjorn Neergaard
d22c775e04
cdi: use separate feature-flag
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-17 11:31:37 +01:00
Paweł Gronowski
fa4deb02af
environment: Error when t.Parallel was called before Protect
Protecting the environment relies on the shared state (containers,
images, etc) which might already be mutated by other tests if the test
opted in into the Parallel execution before Protect was called.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-17 10:54:11 +01:00
Cory Snider
97d32bb7d7 daemon: stop checkpointing health probes to disk
The health status and probe log of containers are not mission-criticial
data which must survive a crash. It is not worth prematrely wearing out
consumer-grade flash storage by overwriting and fsync()ing the container
config on after every probe. Update only the live Container object and
the ViewDB replica on every container health probe instead. It will
eventually get checkpointed along with some other state (or config)
change. Running containers will not be checkpointed on daemon shutdown
when live-restore is enabled, but it does not matter: the health status
and probe log will be zeroed out when the daemon starts back up.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-01-16 14:09:40 -05:00
Sebastiaan van Stijn
2c47a6df0d
Merge pull request #46978 from vvoland/c8d-fix-export
c8d/exporter: Use WithSkipMissing
2024-01-16 20:04:17 +01:00
Brian Goff
353bccdf22
Merge pull request #47051 from corhere/reinit-health-on-live-restore 2024-01-16 10:08:20 -08:00
Sebastiaan van Stijn
9045bea756
Merge pull request #47082 from vvoland/c8d-integration-parallel-fix
integration/image: Move Parallel after setupTest
2024-01-16 18:41:30 +01:00
Sebastiaan van Stijn
8afd6daa3a
Merge pull request #47086 from AkihiroSuda/fix-47085
(*Daemon).fillRootlessVersion: fix nil panic
2024-01-16 18:34:30 +01:00
Akihiro Suda
a6d9462cb1
(*Daemon).fillRootlessVersion: fix nil panic
Fix issue 47085

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2024-01-17 00:50:53 +09:00
Akihiro Suda
5c4189e6b2
Merge pull request #47083 from AkihiroSuda/follow-up-47076
dockerd-rootless.sh: fix inconsistent description about "builtin" driver
2024-01-17 00:19:38 +09:00
Akihiro Suda
e954cdc188
dockerd-rootless.sh: fix inconsistent description about "builtin" driver
The "builtin" port driver was marked as "Slow" in the row for the lxc-user-nic
network driver, while it was marked as "Fast" in other rows.

It had to be consistently marked as "Fast" regardless to the network driver.
It is still not as fast as rootful.

Follow-up to PR 47076
Fixes: b5a5ecf4a3

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2024-01-16 23:11:40 +09:00
Paweł Gronowski
4ff8436ebd
Merge pull request #47081 from vvoland/c8d-integration-cli-skipcontainerconfig
c8d/integration-cli: Skip tests checking `ContainerConfig`
2024-01-16 14:50:02 +01:00
Paweł Gronowski
9de132dbf6
integration/image: Move Parallel after setupTest
setupTest should be called before Parallel as it modifies the test
environment which might produce:

```
fatal error: concurrent map writes

goroutine 143 [running]:
github.com/docker/docker/testutil/environment.(*Execution).ProtectContainer(...)
	/go/src/github.com/docker/docker/testutil/environment/protect.go:59
github.com/docker/docker/testutil/environment.ProtectContainers({0x12e8d98, 0xc00040e420}, {0x12f2878?, 0xc0004fc340}, 0xc0001fac00)
	/go/src/github.com/docker/docker/testutil/environment/protect.go:68 +0xb1
github.com/docker/docker/testutil/environment.ProtectAll({0x12e8d98, 0xc00040e210}, {0x12f2878, 0xc0004fc340}, 0xc0001fac00)
	/go/src/github.com/docker/docker/testutil/environment/protect.go:45 +0xf3
github.com/docker/docker/integration/image.setupTest(0xc0004fc340)
	/go/src/github.com/docker/docker/integration/image/main_test.go:46 +0x59
```

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-16 14:45:26 +01:00
Paweł Gronowski
f0efb90f21
c8d/integration-cli: Skip tests checking ContainerConfig
Field is not supported with containerd integration and deprecated in
API 1.44

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-16 14:03:58 +01:00
Paweł Gronowski
2a02403a94
c8d/exporter: Use WithSkipMissing
Save the unmodified manifest list to keep the image ID of the
multi-platform images when not all platforms are present.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-16 13:06:43 +01:00
Sebastiaan van Stijn
a42726f866
Merge pull request #47078 from vvoland/docker-save-fix-layers-2
image/save: Derive Descriptor from actual tar archive
2024-01-16 12:55:55 +01:00
Sebastiaan van Stijn
b12cf05c82
Merge pull request #47076 from AkihiroSuda/rootlesskit-v2
Update RootlessKit to v2.0.0 (support Pasta)
2024-01-16 12:12:25 +01:00
Paweł Gronowski
9f50e06993
image/save: Refactor saveLayer
`diffIDPaths` is not used and can be removed.

`savedConfig` stores if the config was already saved (ID of the image is
the ID of the config).
`savedLayers` stores if the layer (diff ID) was already saved.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-16 11:14:15 +01:00
Akihiro Suda
b5a5ecf4a3
dockerd-rootless.sh: add comments about pasta
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2024-01-16 10:56:30 +09:00
Akihiro Suda
f5d05b43d7
Dockerfile: update RootlessKit to v2.0.0
https://github.com/rootless-containers/rootlesskit/releases/tag/v2.0.0

=== Pasta ===
RootlessKit v2 adds the support for pasta (https://passt.top/passt/).
Pasta is similar to slirp4netns but its port forwarder achieves better
throughput than slirp4netns port driver.

It is still not faster than RootlessKit's `builtin` port driver, but unlike the
`builtin` port driver, pasta can retain source IP address information.

Network driver | Port driver    | Net throughput | Port throughput | Src IP | No SUID | Note
---------------|----------------|----------------|-----------------|--------|---------|--------------------------------------------
slirp4netns    | builtin        | Slow           | Fast          |      |       | Default in typical setup
vpnkit         | builtin        | Slow           | Fast          |      |       | Default when slirp4netns is not installed
slirp4netns    | slirp4netns    | Slow           | Slow            |      |       |
**pasta**      | **implicit**   | Slow           | Fast          |      |       | Experimental
lxc-user-nic   | builtin        | Fast         | Slow            |      |       | Experimental
(bypass4netns) | (bypass4netns) | Fast         | Fast          |      |       | (Not integrated to RootlessKit)

=== Detach-netns ===
Aside from pasta, RootlessKit v2 also brings the support for
"detach-netns" mode, which leaves the runtime in the host network namespace to
eliminate the slirp overhead for pull/push and to allow accessing the "real"
127.0.0.1.

See containerd/nerdctl PR 2723 for how detach-netns is being adopted in
nerdctl v2.

Integrating detach-netns into Docker/Moby will need an extra work and will be
deferred to Docker v26 (or later).

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2024-01-16 10:56:29 +09:00
Akihiro Suda
ba86626642
vendor.mod: github.com/rootless-containers/rootlesskit/v2 v2.0.0
https://github.com/rootless-containers/rootlesskit/releases/tag/v2.0.0

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2024-01-16 10:56:27 +09:00
Cory Snider
312450d079 integration: test container healthcheck is reset
Update the TestDaemonRestartKilContainers integration test to assert
that a container's healthcheck status is always reset to the Starting
state after a daemon restart, even when the container is live-restored.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-01-15 15:30:22 -05:00
Paweł Gronowski
c58acf06c2 image/save: Derive Descriptor from actual tar archive
Layer size is the sum of the individual files count, not the tar
archive. Use the total bytes read returned by `io.Copy` to populate the
`Size` field.

Also set the digest to the actual digest of the tar archive.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-15 16:35:54 +01:00
Sebastiaan van Stijn
3eba4216e0
Merge pull request #47080 from thaJeztah/bump_mountinfo
vendor: github.com/moby/sys/mountinfo v0.7.1
2024-01-15 13:11:50 +01:00
Sebastiaan van Stijn
6d90f1168b
Merge pull request #47079 from thaJeztah/bump_x_sys
vendor: golang.org/x/sys v0.16.0
2024-01-15 13:11:35 +01:00
Sebastiaan van Stijn
c4d2968183
vendor: github.com/moby/sys/mountinfo v0.7.1
full diff: https://github.com/moby/sys/compare/mountinfo/v0.6.2...mountinfo/v0.7.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-15 12:09:02 +01:00
Sebastiaan van Stijn
075e1141b8
vendor: golang.org/x/sys v0.16.0
full diff: https://github.com/golang/sys/compare/v0.15.0...v0.16.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-15 12:05:29 +01:00
Sebastiaan van Stijn
87041b207f
Merge pull request #46980 from thaJeztah/bump_compare
vendor: github.com/google/go-cmp v0.6.0, github.com/google/uuid v1.5.0
2024-01-15 11:55:41 +01:00
Sebastiaan van Stijn
770271d98f
Merge pull request #47075 from rapsealk/docs/doc-comments-on-volume-mounts-mountpoint-spec
doc: Correct typo in doc comments for volume/mounts/mounts/MountPoint
2024-01-15 10:44:47 +01:00
Sebastiaan van Stijn
a759acd958
vendor: github.com/google/uuid v1.5.0
full diff: https://github.com/google/uuid/compare/v1.3.1...v1.5.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-15 10:40:15 +01:00
Sebastiaan van Stijn
3bcb01a84f
vendor: github.com/google/go-cmp v0.6.0
full diff: https://github.com/google/go-cmp/compare/v0.5.9...v0.6.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-15 10:40:13 +01:00
Jeongseok Kang
36dd2f029f doc: Correct typo in doc comments for volume/mounts/mounts/MountPoint
Signed-off-by: Jeongseok Kang <piono623@naver.com>
2024-01-15 09:25:09 +09:00
Sebastiaan van Stijn
608238ead6
Merge pull request #46979 from thaJeztah/move_mergo
vendor: containerd v1.7.12, and switch to dario.cat/mergo v1.0.0
2024-01-12 21:14:59 +01:00
Sebastiaan van Stijn
3a15c6c9d7
Merge pull request #47070 from thaJeztah/bump_containerd_binary
update containerd binary to v1.7.12
2024-01-12 21:14:07 +01:00
Sebastiaan van Stijn
8f89f10b40
Merge pull request #47068 from vvoland/docker-save-fix-layers
image/save: Fix missing layers in manifest
2024-01-12 19:39:56 +01:00
Sebastiaan van Stijn
6e2e7641f9
Merge pull request #47067 from thaJeztah/bump_buildx_compose
Dockerfile: update buildx to v0.12.1
2024-01-12 18:37:10 +01:00
Sebastiaan van Stijn
92175dd78c
vendor: containerd v1.7.12, and switch to dario.cat/mergo v1.0.0
Switch github.com/imdario/mergo to dario.cat/mergo v1.0.0, because
the module was renamed, and reached v1.0.0

full diff: https://github.com/imdario/mergo/compare/v0.3.13...v1.0.0

vendor: github.com/containerd/containerd v1.7.12

- full diff: https://github.com/containerd/containerd/compare/v1.7.11...v1.7.12
- release notes: https://github.com/containerd/containerd/releases/tag/v1.7.12

Welcome to the v1.7.12 release of containerd!

The twelfth patch release for containerd 1.7 contains various fixes and updates.

Notable Updates

- Fix on dialer function for Windows
- Improve `/etc/group` handling when appending groups
- Update shim pidfile permissions to 0644
- Update runc binary to v1.1.11
- Allow import and export to reference missing content
- Remove runc import
- Update Go version to 1.20.13

Deprecation Warnings

- Emit deprecation warning for `containerd.io/restart.logpath` label usage

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-12 18:09:24 +01:00
Sebastiaan van Stijn
cd1709b0d4
update containerd binary to v1.7.12
- full diff: https://github.com/containerd/containerd/compare/v1.7.11...v1.7.12
- release notes: https://github.com/containerd/containerd/releases/tag/v1.7.12

Welcome to the v1.7.12 release of containerd!

The twelfth patch release for containerd 1.7 contains various fixes and updates.

Notable Updates

- Fix on dialer function for Windows
- Improve `/etc/group` handling when appending groups
- Update shim pidfile permissions to 0644
- Update runc binary to v1.1.11
- Allow import and export to reference missing content
- Remove runc import
- Update Go version to 1.20.13

Deprecation Warnings

- Emit deprecation warning for `containerd.io/restart.logpath` label usage

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-12 17:56:37 +01:00
Paweł Gronowski
2c45df2ab4 image/save: Fix missing layers in manifest
The new OCI-compatible archive export relies on the Descriptors returned
by the layer (`distribution.Describable` interface implementation).

The issue with that is that the `roLayer` and the `referencedCacheLayer`
types don't implement this interface. Implementing that interface for
them based on their `descriptor` doesn't work though, because that
descriptor is empty.

To workaround this issue, just create a new descriptor if the one
provided by the layer is empty.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-12 17:10:23 +01:00
Sebastiaan van Stijn
0790864064
Dockerfile: update buildx to v0.12.1
Update the version of buildx used in CI to the latest version.

- full diff: https://github.com/docker/buildx/compare/v0.12.0...v0.12.1
- release notes: https://github.com/docker/buildx/releases/tag/v0.12.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-12 16:13:34 +01:00
Sebastiaan van Stijn
ac6c791880
Merge pull request #47066 from thaJeztah/bump_compose
Dockerfile: update docker compose to v2.24.0
2024-01-12 16:12:58 +01:00
Sebastiaan van Stijn
a9d0128bd7
Dockerfile: update docker compose to v2.24.0
Update the version of compose used in CI to the latest version.

- full diff: https://github.com/docker/compose/compare/v2.20.1...v2.24.0
- release notes: https://github.com/docker/compose/releases/tag/v2.24.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-12 14:43:40 +01:00
Sebastiaan van Stijn
92cc0d6257
Merge pull request #47060 from vvoland/c8d-builder-commit-shareimagecreate
c8d/commit-builder: Extract common code
2024-01-11 14:37:20 +01:00
Paweł Gronowski
28f48ce1ac
c8d/commit-builder: Extract common code
Extract duplicated image creation code to a function.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-11 13:30:40 +01:00
Sebastiaan van Stijn
ef1a85ca5e
Merge pull request #47052 from corhere/libc8d/create-unstarted-tasks
libcontainerd: create unstarted tasks
2024-01-10 20:41:53 +01:00
Cory Snider
659d7b190f libcontainerd: create unstarted tasks
Split task creation and start into two separate method calls in the
libcontainerd API. Clients now have the opportunity to inspect the
freshly-created task and customize its runtime environment before
starting execution of the user-specified binary.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-01-10 13:50:26 -05:00
Sebastiaan van Stijn
7af5682b3c
Merge pull request #47053 from thaJeztah/update_golang_1.21.6
update to go1.21.6
2024-01-10 17:51:03 +01:00
Sebastiaan van Stijn
0fb6ba4ccc
Merge pull request #47048 from thaJeztah/fix_version_output
daemon/config: setPlatformDefaults: use debug for missing userland-proxy
2024-01-10 17:15:08 +01:00
Sebastiaan van Stijn
d7141cfd06
update to go1.21.6
go1.21.6 (released 2024-01-09) includes fixes to the compiler, the runtime, and
the crypto/tls, maps, and runtime/pprof packages. See the Go 1.21.6 milestone on
our issue tracker for details:

- https://github.com/golang/go/issues?q=milestone%3AGo1.21.6+label%3ACherryPickApproved
- full diff: https://github.com/golang/go/compare/go1.21.5...go1.21.6

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-10 09:46:37 +01:00
Cory Snider
0e62dbadcd daemon: reinit health monitor on live-restore
The container may have been running without health probes for an
indeterminate amount of time. The container may have become unhealthy in
the interim. We should probe it sooner than in steady-state, while also
giving it some leeway to recover from e.g. timed-out connections. This
is easy to achieve by probing the container like a freshly-started one.
The original author of health-checks came to the same conclusion; the
health monitor was reinitialized on live-restored containers before
v17.11.0, when health monitoring of live-restored containers was
accidentally broken. Revert to the original behavior.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-01-09 11:03:06 -05:00