Commit graph

48454 commits

Author SHA1 Message Date
Paweł Gronowski
7cbf01b994
Merge pull request #47105 from vvoland/c8d-mount-refcount-default
c8d/mount: Use ref-counted mounter by default
2024-01-18 17:26:22 +01:00
Paweł Gronowski
05695c2458
c8d/integation: Adjust error in TestPullLinuxImageFailsOnWindows
Message is different with containerd backend. The Linux test
`TestPullLinuxImageFailsOnLinux` was adjusted before, but we missed this
one.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-18 16:13:23 +01:00
Paweł Gronowski
ae6468b4b9
c8d/mount: Use ref-counted mounter by default
All commonly used filesystems should use ref-counted mounter, so make it
the default instead of having to whitelist them.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-18 15:38:51 +01:00
Sebastiaan van Stijn
c183fcf116
Merge pull request #47104 from rumpl/run-snapshotter-linux-tests
ci: Always run snapshotter tests on Linux
2024-01-18 15:24:53 +01:00
Sebastiaan van Stijn
982f85b281
Merge pull request #47101 from AkihiroSuda/refactor-47100
net=host: remove /var/run/docker/netns/default from OCI config
2024-01-18 14:55:53 +01:00
Djordje Lukic
bd481592ff
ci: Always run snapshotter tests on Linux
Now that we have a green CI on linux we can enable this for all PRs.

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2024-01-18 11:16:57 +01:00
Akihiro Suda
ed15f1d717
net=host: remove /var/run/docker/netns/default from OCI config
Prior to this commit, a container running with `--net=host` had
`{"type":"network","path":"/var/run/docker/netns/default"}` in
the ``.linux.namespaces` field of the OCI Runtime Config,
but this wasn't needed.

Close issue 47100

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2024-01-18 18:09:52 +09:00
Sebastiaan van Stijn
4f9c865edd
Merge pull request #46158 from elezar/refactor-rootless-tempdir
Add testutil.TempDir function
2024-01-17 17:38:21 +01:00
Sebastiaan van Stijn
436bf27e6c
Merge pull request #46539 from TBBle/containerd_image_store_pr46402
c8d: Just enough Windows support to run the test suite
2024-01-17 17:31:51 +01:00
Sebastiaan van Stijn
038729a7ab
Merge pull request #47091 from thaJeztah/unskip_some_tests
docker-py: unskip some tests
2024-01-17 17:08:05 +01:00
Sebastiaan van Stijn
8b0468e9ee
Merge pull request #47090 from thaJeztah/re_enable_check
Revert "temporary: Disable deprecate-integration-cli validation"
2024-01-17 17:05:39 +01:00
Sebastiaan van Stijn
568c064795
docker-py: unskip some tests
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-17 16:44:06 +01:00
Paul "TBBle" Hampson
e8f4bfb374
Root.Path for a process-isolated WCOW container must be the Volume GUID
The actual divergence is due to differences in the snapshotter and
graphfilter mount behaviour on Windows, but the snapshotter behaviour is
better, so we deal with it here rather than changing the snapshotter
behaviour.

We're relying on the internals of containerd's Windows mount
implementation here. Unless this code flow is replaced, future work is
to move getBackingDeviceForContainerdMount into containerd's mount
implementation.

Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-17 16:29:32 +01:00
Paul "TBBle" Hampson
ec041193f9
Windows doesn't have 'host' mode networking, so use 'auto'.
That means 'null', not that we can call builder-next on Windows. If and
when we do get builder-next going, this will need to be solved properly
in some way.

Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-17 16:29:30 +01:00
Paul "TBBle" Hampson
66325f7271
Implement GetLayerFolders for the containerd image store
The existing API ImageService.GetLayerFolders didn't have access to the
ID of the container, and once we have that, the snapshotter Mounts API
provides all the information we need here.

Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-17 16:29:28 +01:00
Paul "TBBle" Hampson
0dc07ccc3a
Unmount RWLayer during Commit
Needed for Diff on Windows. Don't remount it afterwards as the layer is
going to be released anyway.

Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-17 16:29:27 +01:00
Paul "TBBle" Hampson
a1f6b64e24
Don't call setupInit in a temp mount if setupInit is nil
This is consistent with layerStore's CreateRWLayer behaviour.

Potentially this can be refactored to avoid creating the -init layer,
but as noted in layerStore's initMount, this name may be special, and
should be cleared-out all-at-once.

Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-17 16:29:25 +01:00
Paul "TBBle" Hampson
efadb70ef8
The Windows snapshotter and graphdriver have different names
Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-17 16:29:24 +01:00
Paul "TBBle" Hampson
641050c93f
Typo fixes
* conatinerd => containerd
* ROLayer => RWLayer

Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-17 16:29:22 +01:00
Sebastiaan van Stijn
83de55b370
Merge pull request #47084 from AkihiroSuda/dockerd-rootless-setuptool-nsenter
dockerd-rootless-setuptool.sh: add `nsenter` subcommand (for debugging)
2024-01-17 16:16:23 +01:00
Sebastiaan van Stijn
508a2d979a
Revert "temporary: Disable deprecate-integration-cli validation"
This reverts commit bdc7d0c2db.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-17 15:59:52 +01:00
Sebastiaan van Stijn
1fd682930a
Merge pull request #46634 from rumpl/c8d-classic-builder-cache
c8d: make the cache in classic builder work
2024-01-17 15:51:16 +01:00
Evan Lezar
f7065ab207
Add testutil.TempDir function
This change adds a TempDir function that ensures the correct permissions for
the fake-root user in rootless mode.

Signed-off-by: Evan Lezar <elezar@nvidia.com>
2024-01-17 15:44:47 +01:00
Akihiro Suda
84d2229ffe
dockerd-rootless-setuptool.sh: add nsenter subcommand
Usage: `dockerd-rootless-setuptool.sh nsenter -- ip a`

Expected to be used for debugging.

Ported from nerdctl's `containerd-rootless-setuptool.sh`
https://github.com/containerd/nerdctl/blob/v1.7.2/extras/rootless/containerd-rootless-setuptool.sh#L142-L147

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2024-01-17 22:59:18 +09:00
Akihiro Suda
7d08d84b03
dockerd-rootless.sh: set rootlesskit --state-dir=DIR
Now the state dir is set to `${XDG_RUNTIME_DIR}/dockerd-rootless`.

This is similar to `${XDG_RUNTIME_DIR}/containerd-rootless` used in nerdctl:
https://github.com/containerd/nerdctl/blob/v1.7.2/extras/rootless/containerd-rootless.sh#L35

Prior to this commit, the state dir was unset and a random dir under `/tmp` was used.
(e.g., `/tmp/rootlesskit1869901982`)

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2024-01-17 22:59:18 +09:00
Akihiro Suda
ae9fbcb82c
testutil/daemon: shorten XDG_RUNTIME_DIR
XDG_RUNTIME_DIR will contain sockets so its path mustn't be too long.
Prior to this commit, it was set to very long path like
`/go/src/github.com/docker/docker/bundles/test-integration/TestDiskUsage/de4fb36576d7d/xdgrun`

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2024-01-17 22:59:18 +09:00
Paweł Gronowski
bdc7d0c2db
temporary: Disable deprecate-integration-cli validation
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-17 14:57:58 +01:00
Paweł Gronowski
8390bc5683
c8d/cache: Use ContainerConfig from content store
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-17 14:57:55 +01:00
Paweł Gronowski
f760cb4f97
c8d/builder: Store ContainerConfig
Serialize ContainerConfig to content store and store its digest in
label.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-17 14:57:54 +01:00
Paweł Gronowski
a5a15c7782
c8d/cache: Optimize FROM scratch case
Consider only images that were built `FROM scratch` as valid candidates
for the `FROM scratch` + INSTRUCTION build step.

The images are marked as `FROM scratch` based by the classic builder
with a special label. It must be a new label instead of empty parent
label, because empty label values are not persisted.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-17 14:57:53 +01:00
Djordje Lukic
71ebfc7c63
c8d: make the cache in classic builder work
In order for the cache in the classic builder to work we need to:
- use the came comparison function as the graph drivers implementation
- save the container config when commiting the image
- use all images to search a 'FROM "scratch"' image
- load all images if `cacheFrom` is empty

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-17 14:57:50 +01:00
Sebastiaan van Stijn
af4f6c124d
Merge pull request #47088 from vvoland/testing-setupTest-parallel
environment: Error when t.Parallel was called before Protect
2024-01-17 14:57:19 +01:00
Sebastiaan van Stijn
60ed73bb91
Merge pull request #47044 from corhere/lazy-health-status
Lazily checkpoint container health status to disk
2024-01-17 14:12:38 +01:00
Sebastiaan van Stijn
d3e08fe3cf
Merge pull request #47087 from neersighted/cdi_feature
cdi: use separate feature-flag
2024-01-17 14:11:41 +01:00
Paweł Gronowski
24da5233dd
integration: Fix Parallel before setupTest
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-17 13:06:05 +01:00
Bjorn Neergaard
d22c775e04
cdi: use separate feature-flag
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-17 11:31:37 +01:00
Paweł Gronowski
fa4deb02af
environment: Error when t.Parallel was called before Protect
Protecting the environment relies on the shared state (containers,
images, etc) which might already be mutated by other tests if the test
opted in into the Parallel execution before Protect was called.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-17 10:54:11 +01:00
Cory Snider
97d32bb7d7 daemon: stop checkpointing health probes to disk
The health status and probe log of containers are not mission-criticial
data which must survive a crash. It is not worth prematrely wearing out
consumer-grade flash storage by overwriting and fsync()ing the container
config on after every probe. Update only the live Container object and
the ViewDB replica on every container health probe instead. It will
eventually get checkpointed along with some other state (or config)
change. Running containers will not be checkpointed on daemon shutdown
when live-restore is enabled, but it does not matter: the health status
and probe log will be zeroed out when the daemon starts back up.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-01-16 14:09:40 -05:00
Sebastiaan van Stijn
2c47a6df0d
Merge pull request #46978 from vvoland/c8d-fix-export
c8d/exporter: Use WithSkipMissing
2024-01-16 20:04:17 +01:00
Brian Goff
353bccdf22
Merge pull request #47051 from corhere/reinit-health-on-live-restore 2024-01-16 10:08:20 -08:00
Sebastiaan van Stijn
9045bea756
Merge pull request #47082 from vvoland/c8d-integration-parallel-fix
integration/image: Move Parallel after setupTest
2024-01-16 18:41:30 +01:00
Sebastiaan van Stijn
8afd6daa3a
Merge pull request #47086 from AkihiroSuda/fix-47085
(*Daemon).fillRootlessVersion: fix nil panic
2024-01-16 18:34:30 +01:00
Akihiro Suda
a6d9462cb1
(*Daemon).fillRootlessVersion: fix nil panic
Fix issue 47085

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2024-01-17 00:50:53 +09:00
Akihiro Suda
5c4189e6b2
Merge pull request #47083 from AkihiroSuda/follow-up-47076
dockerd-rootless.sh: fix inconsistent description about "builtin" driver
2024-01-17 00:19:38 +09:00
Akihiro Suda
e954cdc188
dockerd-rootless.sh: fix inconsistent description about "builtin" driver
The "builtin" port driver was marked as "Slow" in the row for the lxc-user-nic
network driver, while it was marked as "Fast" in other rows.

It had to be consistently marked as "Fast" regardless to the network driver.
It is still not as fast as rootful.

Follow-up to PR 47076
Fixes: b5a5ecf4a3

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2024-01-16 23:11:40 +09:00
Paweł Gronowski
4ff8436ebd
Merge pull request #47081 from vvoland/c8d-integration-cli-skipcontainerconfig
c8d/integration-cli: Skip tests checking `ContainerConfig`
2024-01-16 14:50:02 +01:00
Paweł Gronowski
9de132dbf6
integration/image: Move Parallel after setupTest
setupTest should be called before Parallel as it modifies the test
environment which might produce:

```
fatal error: concurrent map writes

goroutine 143 [running]:
github.com/docker/docker/testutil/environment.(*Execution).ProtectContainer(...)
	/go/src/github.com/docker/docker/testutil/environment/protect.go:59
github.com/docker/docker/testutil/environment.ProtectContainers({0x12e8d98, 0xc00040e420}, {0x12f2878?, 0xc0004fc340}, 0xc0001fac00)
	/go/src/github.com/docker/docker/testutil/environment/protect.go:68 +0xb1
github.com/docker/docker/testutil/environment.ProtectAll({0x12e8d98, 0xc00040e210}, {0x12f2878, 0xc0004fc340}, 0xc0001fac00)
	/go/src/github.com/docker/docker/testutil/environment/protect.go:45 +0xf3
github.com/docker/docker/integration/image.setupTest(0xc0004fc340)
	/go/src/github.com/docker/docker/integration/image/main_test.go:46 +0x59
```

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-16 14:45:26 +01:00
Paweł Gronowski
f0efb90f21
c8d/integration-cli: Skip tests checking ContainerConfig
Field is not supported with containerd integration and deprecated in
API 1.44

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-16 14:03:58 +01:00
Paweł Gronowski
2a02403a94
c8d/exporter: Use WithSkipMissing
Save the unmodified manifest list to keep the image ID of the
multi-platform images when not all platforms are present.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-16 13:06:43 +01:00
Sebastiaan van Stijn
a42726f866
Merge pull request #47078 from vvoland/docker-save-fix-layers-2
image/save: Derive Descriptor from actual tar archive
2024-01-16 12:55:55 +01:00