Commit graph

48430 commits

Author SHA1 Message Date
Rob Murray
a8f7c5ee48 Detect IPv6 support in containers.
Some configuration in a container depends on whether it has support for
IPv6 (including default entries for '::1' etc in '/etc/hosts').

Before this change, the container's support for IPv6 was determined by
whether it was connected to any IPv6-enabled networks. But, that can
change over time, it isn't a property of the container itself.

So, instead, detect IPv6 support by looking for '::1' on the container's
loopback interface. It will not be present if the kernel does not have
IPv6 support, or the user has disabled it in new namespaces by other
means.

Once IPv6 support has been determined for the container, its '/etc/hosts'
is re-generated accordingly.

The daemon no longer disables IPv6 on all interfaces during initialisation.
It now disables IPv6 only for interfaces that have not been assigned an
IPv6 address. (But, even if IPv6 is disabled for the container using the
sysctl 'net.ipv6.conf.all.disable_ipv6=1', interfaces connected to IPv6
networks still get IPv6 addresses that appear in the internal DNS. There's
more to-do!)

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-01-19 20:24:07 +00:00
Cory Snider
0046b16d87 daemon: set libnetwork sandbox key w/o OCI hook
Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-01-19 20:23:12 +00:00
Sebastiaan van Stijn
31ccdbb7a8
Merge pull request #45687 from vvoland/volume-mount-subpath
volumes: Implement subpath mount
2024-01-19 18:41:12 +01:00
Paweł Gronowski
5bbcc41c20
volumes/subpath: Plumb context
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-19 17:32:21 +01:00
Paweł Gronowski
cb1af229f2
daemon/populateVolumes: Support volume subpath
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-19 17:32:20 +01:00
Paweł Gronowski
349a52b279
container: Change comment into debug log
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-19 17:32:18 +01:00
Paweł Gronowski
42afac91d7
internal/safepath: Add windows implementation
All components of the path are locked before the check, and
released once the path is already mounted.
This makes it impossible to replace the mounted directory until it's
actually mounted in the container.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-19 17:32:17 +01:00
Paweł Gronowski
5841ed4e5e
internal/safepath: Adapt k8s openat2 fallback
Adapts the function source code to the Moby codebase.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-19 17:32:16 +01:00
Paweł Gronowski
56bb143a4d
internal/safepath: Import k8s safeopen function
For use as a soft fallback if Openat2 is not available.
Source: 55fb1805a1/pkg/volume/util/subpath/subpath_linux.go

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-19 17:32:14 +01:00
Paweł Gronowski
3784316d46
internal/safepath: Handle EINTR in unix syscalls
Handle EINTR by retrying the syscall.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-19 17:32:13 +01:00
Paweł Gronowski
9a0cde66ba
internal/safepath: Add linux implementation
All subpath components are opened with openat, relative to the base
volume directory and checked against the volume escape.
The final file descriptor is mounted from the /proc/self/fd/<fd> to a
temporary mount point owned by the daemon and then passed to the
underlying container runtime.
Temporary mountpoint is removed after the container is started.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-19 17:32:12 +01:00
Paweł Gronowski
bfb810445c
volumes: Implement subpath mount
`VolumeOptions` now has a `Subpath` field which allows to specify a path
relative to the volume that should be mounted as a destination.

Symlinks are supported, but they cannot escape the base volume
directory.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-19 17:32:10 +01:00
Paweł Gronowski
f2e1105056
Introduce a helper that collects cleanup functions
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-19 17:27:17 +01:00
Paweł Gronowski
f07387466a
daemon/oci: Extract side effects from withMounts
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-19 17:27:16 +01:00
Paweł Gronowski
9c8752505f
volume/mounts: Rename errors in defer block
To make it easier to distinguish if an output variable is modified.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-19 17:27:15 +01:00
Sebastiaan van Stijn
17d80442cc
Merge pull request #47111 from robmry/fix_TestAddRemoveInterface
Fix libnetwork/osl test TestAddRemoveInterface
2024-01-19 16:15:49 +01:00
Sebastiaan van Stijn
d35e923a4c
Merge pull request #47116 from thaJeztah/minor_nits
assorted minor linting issues
2024-01-19 16:09:12 +01:00
Sebastiaan van Stijn
67de5c84d3
Merge pull request #47118 from vvoland/api-1.45
API: bump version to 1.45
2024-01-19 15:35:47 +01:00
Paweł Gronowski
5bcbedb7ee
API: bump version to 1.45
Docker 25.0 was released with API v1.44, so any change in the API should
now target v1.45.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-19 14:42:19 +01:00
Sebastiaan van Stijn
35789fce99
daemon.images: ImageService.getImage: use named fields in struct literals
Prevent things from breaking if additional fields are added to this struct.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-19 13:11:40 +01:00
Sebastiaan van Stijn
7c1914411f
daemon/images: ImageService.manifestMatchesPlatform: optimize logger
We constructed a "function level" logger, which was used once "as-is", but
also added additional Fields in a loop (for each resource), effectively
overwriting the previous one for each iteration. Adding additional
fields can result in some overhead, so let's construct a "logger" only for
inside the loop.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-19 13:08:30 +01:00
Sebastiaan van Stijn
5581efe7cd
rename "ociimage" var to be proper camelCase
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-19 12:50:13 +01:00
Sebastiaan van Stijn
66cf6e3a7a
rename "image" vars to prevent conflicts with imports
We have many "image" packages, so these vars easily conflict/shadow
imports. Let's rename them (and in some cases use a const) to
prevent that.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-19 12:49:53 +01:00
Tianon Gravi
4a40d10b60
Merge pull request #47109 from whalelines/git-url-regex
Fix isGitURL regular expression
2024-01-18 14:02:57 -08:00
Rob Murray
c72e458a72 Fix libnetwork/osl test TestAddRemoveInterface
For some time, when adding an interface with no IPv6 address (an
interface to a network that does not have IPv6 enabled), we've been
disabling IPv6 on that interface.

As part of a separate change, I'm removing that logic - there's nothing
wrong with having IPv6 enabled on an interface with no routable address.
The difference is that the kernel will assign a link-local address.

TestAddRemoveInterface does this...
- Assign an IPv6 link-local address to one end of a veth interface, and
  add it to a namespace.
- Add a bridge with no assigned IPv6 address to the namespace.
- Remove the veth interface from the namespace.
- Put the veth interface back into the namespace, still with an
  explicitly assigned IPv6 link local address.

When IPv6 is disabled on the bridge interface, the test passes.

But, when IPv6 is enabled, the bridge gets a kernel assigned link-local
address.

Then, when re-adding the veth interface, the test generates an error in
'osl/interface_linux.go:checkRouteConflict()'. The conflict is between
the explicitly assigned fe80::2 on the veth, and a route for fe80::/64
belonging to the bridge.

So, in preparation for not-disabling IPv6 on these interfaces, use a
unique-local address in the test instead of link-local.

I don't think that changes the intent of the test.

With the change to not-always disable IPv6, it is possible to repro the
problem with a real container, disconnect and re-connect a user-defined
network with '--subnet fe80::/64' while the container's connected to an
IPv4 network. So, strictly speaking, that will be a regression.

But, it's also possible to repro the problem in master, by disconnecting
and re-connecting the fe80::/64 network while another IPv6 network is
connected. So, I don't think it's a problem we need to address, perhaps
other than by prohibiting '--subnet fe80::/64'.

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-01-18 21:01:41 +00:00
David Dooling
768146b1b0
Fix isGitURL regular expression
Escape period (.) so regular expression does not match any character before "git".

Signed-off-by: David Dooling <david.dooling@docker.com>
2024-01-18 14:14:08 -06:00
Paweł Gronowski
615dfdf672
Merge pull request #47106 from vvoland/c8d-windows-pull-error-msg
c8d/integration: Adjust error in TestPullLinuxImageFailsOnWindows
2024-01-18 17:28:52 +01:00
Paweł Gronowski
7cbf01b994
Merge pull request #47105 from vvoland/c8d-mount-refcount-default
c8d/mount: Use ref-counted mounter by default
2024-01-18 17:26:22 +01:00
Paweł Gronowski
05695c2458
c8d/integation: Adjust error in TestPullLinuxImageFailsOnWindows
Message is different with containerd backend. The Linux test
`TestPullLinuxImageFailsOnLinux` was adjusted before, but we missed this
one.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-18 16:13:23 +01:00
Paweł Gronowski
ae6468b4b9
c8d/mount: Use ref-counted mounter by default
All commonly used filesystems should use ref-counted mounter, so make it
the default instead of having to whitelist them.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-18 15:38:51 +01:00
Sebastiaan van Stijn
c183fcf116
Merge pull request #47104 from rumpl/run-snapshotter-linux-tests
ci: Always run snapshotter tests on Linux
2024-01-18 15:24:53 +01:00
Sebastiaan van Stijn
982f85b281
Merge pull request #47101 from AkihiroSuda/refactor-47100
net=host: remove /var/run/docker/netns/default from OCI config
2024-01-18 14:55:53 +01:00
Djordje Lukic
bd481592ff
ci: Always run snapshotter tests on Linux
Now that we have a green CI on linux we can enable this for all PRs.

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2024-01-18 11:16:57 +01:00
Akihiro Suda
ed15f1d717
net=host: remove /var/run/docker/netns/default from OCI config
Prior to this commit, a container running with `--net=host` had
`{"type":"network","path":"/var/run/docker/netns/default"}` in
the ``.linux.namespaces` field of the OCI Runtime Config,
but this wasn't needed.

Close issue 47100

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2024-01-18 18:09:52 +09:00
Sebastiaan van Stijn
4f9c865edd
Merge pull request #46158 from elezar/refactor-rootless-tempdir
Add testutil.TempDir function
2024-01-17 17:38:21 +01:00
Sebastiaan van Stijn
436bf27e6c
Merge pull request #46539 from TBBle/containerd_image_store_pr46402
c8d: Just enough Windows support to run the test suite
2024-01-17 17:31:51 +01:00
Sebastiaan van Stijn
038729a7ab
Merge pull request #47091 from thaJeztah/unskip_some_tests
docker-py: unskip some tests
2024-01-17 17:08:05 +01:00
Sebastiaan van Stijn
8b0468e9ee
Merge pull request #47090 from thaJeztah/re_enable_check
Revert "temporary: Disable deprecate-integration-cli validation"
2024-01-17 17:05:39 +01:00
Sebastiaan van Stijn
568c064795
docker-py: unskip some tests
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-17 16:44:06 +01:00
Paul "TBBle" Hampson
e8f4bfb374
Root.Path for a process-isolated WCOW container must be the Volume GUID
The actual divergence is due to differences in the snapshotter and
graphfilter mount behaviour on Windows, but the snapshotter behaviour is
better, so we deal with it here rather than changing the snapshotter
behaviour.

We're relying on the internals of containerd's Windows mount
implementation here. Unless this code flow is replaced, future work is
to move getBackingDeviceForContainerdMount into containerd's mount
implementation.

Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-17 16:29:32 +01:00
Paul "TBBle" Hampson
ec041193f9
Windows doesn't have 'host' mode networking, so use 'auto'.
That means 'null', not that we can call builder-next on Windows. If and
when we do get builder-next going, this will need to be solved properly
in some way.

Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-17 16:29:30 +01:00
Paul "TBBle" Hampson
66325f7271
Implement GetLayerFolders for the containerd image store
The existing API ImageService.GetLayerFolders didn't have access to the
ID of the container, and once we have that, the snapshotter Mounts API
provides all the information we need here.

Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-17 16:29:28 +01:00
Paul "TBBle" Hampson
0dc07ccc3a
Unmount RWLayer during Commit
Needed for Diff on Windows. Don't remount it afterwards as the layer is
going to be released anyway.

Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-17 16:29:27 +01:00
Paul "TBBle" Hampson
a1f6b64e24
Don't call setupInit in a temp mount if setupInit is nil
This is consistent with layerStore's CreateRWLayer behaviour.

Potentially this can be refactored to avoid creating the -init layer,
but as noted in layerStore's initMount, this name may be special, and
should be cleared-out all-at-once.

Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-17 16:29:25 +01:00
Paul "TBBle" Hampson
efadb70ef8
The Windows snapshotter and graphdriver have different names
Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-17 16:29:24 +01:00
Paul "TBBle" Hampson
641050c93f
Typo fixes
* conatinerd => containerd
* ROLayer => RWLayer

Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-17 16:29:22 +01:00
Sebastiaan van Stijn
83de55b370
Merge pull request #47084 from AkihiroSuda/dockerd-rootless-setuptool-nsenter
dockerd-rootless-setuptool.sh: add `nsenter` subcommand (for debugging)
2024-01-17 16:16:23 +01:00
Sebastiaan van Stijn
508a2d979a
Revert "temporary: Disable deprecate-integration-cli validation"
This reverts commit bdc7d0c2db.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-17 15:59:52 +01:00
Sebastiaan van Stijn
1fd682930a
Merge pull request #46634 from rumpl/c8d-classic-builder-cache
c8d: make the cache in classic builder work
2024-01-17 15:51:16 +01:00
Evan Lezar
f7065ab207
Add testutil.TempDir function
This change adds a TempDir function that ensures the correct permissions for
the fake-root user in rootless mode.

Signed-off-by: Evan Lezar <elezar@nvidia.com>
2024-01-17 15:44:47 +01:00