Commit graph

39224 commits

Author SHA1 Message Date
Sebastiaan van Stijn
94feac18d2
Update rootlesskit to v0.13.1 to fix handling of IPv6 addresses
v0.13.1

- Refactor `ParsePortSpec` to handle IPv6 addresses, and improve validation

v0.13.0

- `rootlesskit --pidns`: fix propagating exit status
- Support cgroup2 evacuation, e.g., `systemd-run -p Delegate=yes --user -t rootlesskit --cgroupns --pidns --evacuate-cgroup2=evac --net=slirp4netns bash`

v0.12.0

- Port forwarding API now supports setting `ChildIP`
- The `vendor` directory is no longer included in this repo. Run `go mod vendor` if you need

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit e32ae1973a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-02-17 21:11:17 +01:00
Brian Goff
fae366b323
Merge pull request #41970 from thaJeztah/20.10_backport_testing_fixes 2021-02-17 09:37:19 -08:00
Brian Goff
dfce527001
Merge pull request #42030 from thaJeztah/20.10_backport_cgroup2ci_jenkins 2021-02-16 09:37:37 -08:00
Sebastiaan van Stijn
fc07fecfb5
TestBuildUserNamespaceValidateCapabilitiesAreV2: verify build completed
Check if the `docker build` completed successfully before continuing.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit fa480403c7)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-02-16 14:20:02 +01:00
Sebastiaan van Stijn
f7893961de
TestBuildUserNamespaceValidateCapabilitiesAreV2: use correct image name
This currently doesn't make a difference, because load.FrozenImagesLinux()
currently loads all frozen images, not just the specified one, but in case
that is fixed/implemented at some point.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 26965fbfa0)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-02-16 14:20:00 +01:00
Akihiro Suda
d31b2141ae
Jenkinsfile: add cgroup2
Thanks to Stefan Scherer for setting up the Jenkins nodes.

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit c23b99f4db)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-02-16 09:22:30 +01:00
Akihiro Suda
5de9bc7e01
TestInspectOomKilledTrue: skip on cgroup v2
The test fails intermittently on cgroup v2.

```
=== FAIL: amd64.integration.container TestInspectOomKilledTrue (0.53s)
    kill_test.go:171: assertion failed: true (true bool) != false (inspect.State.OOMKilled bool)
```

Tracked in issue 41929

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit c316dd7cc5)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-02-16 09:22:27 +01:00
Tibor Vass
11443bc681
Merge pull request #41957 from AkihiroSuda/cherrypick-41892-2010
[20.10 backport] pkg/archive: allow mknodding FIFO inside userns
2021-02-11 11:56:54 -08:00
Sebastiaan van Stijn
49e706e14c
Dockerfile.buildx: update buildx to v0.5.1
full diff: https://github.com/docker/buildx/compare/v0.3.1...v0.5.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 30b20a6bdd)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-02-03 13:54:27 +01:00
Sebastiaan van Stijn
0211909bde
testing: update docker-py 4.4.1
run docker-py integration tests of the latest release;

full diff: https://github.com/docker/docker-py/compare/4.3.0...4.4.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 14fb165085)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-02-03 13:54:22 +01:00
Sebastiaan van Stijn
faf6442f80
integration: fix TestBuildUserNamespaceValidateCapabilitiesAreV2 not using frozen image
Commit f2f5106c92 added this test to verify loading
of images that were built with user-namespaces enabled.

However, because this test spins up a new daemon, not the daemon that's set up by
the test-suite's `TestMain()` (which loads the frozen images).

As a result, the `debian:bullseye` image was pulled from Docker Hub when running
the test;

    Calling POST /v1.41/images/load?quiet=1
    Applying tar in /go/src/github.com/docker/docker/bundles/test-integration/TestBuildUserNamespaceValidateCapabilitiesAreV2/d4d366b15997b/root/165536.165536/overlay2/3f7f9375197667acaf7bc810b34689c21f8fed9c52c6765c032497092ca023d6/diff" storage-driver=overlay
    Applied tar sha256:845f0e5159140e9dbcad00c0326c2a506fbe375aa1c229c43f082867d283149c to 3f7f9375197667acaf7bc810b34689c21f8fed9c52c6765c032497092ca023d6, size: 5922359
    Calling POST /v1.41/build?buildargs=null&cachefrom=null&cgroupparent=&cpuperiod=0&cpuquota=0&cpusetcpus=&cpusetmems=&cpushares=0&dockerfile=&labels=null&memory=0&memswap=0&networkmode=&rm=0&shmsize=0&t=capabilities%3A1.0&target=&ulimits=null&version=
    Trying to pull debian from https://registry-1.docker.io v2
    Fetching manifest from remote" digest="sha256:f169dbadc9021fc0b08e371d50a772809286a167f62a8b6ae86e4745878d283d" error="<nil>" remote="docker.io/library/debian:bullseye
    Pulling ref from V2 registry: debian:bullseye
    ...

This patch updates `TestBuildUserNamespaceValidateCapabilitiesAreV2` to load the
frozen image. `StartWithBusybox` is also changed to `Start`, because the test
is not using the busybox image, so there's no need to load it.

In a followup, we should probably add some utilities to make this easier to set up
(and to allow passing the list frozen images that we want to load, without having
to "hard-code" the image name to load).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 46dfc31342)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-02-03 13:54:16 +01:00
Brian Goff
f0e526f43e
Make test work with rootless mode
Using `d.Kill()` with rootless mode causes the restarted daemon to not
be able to start containerd (it times out).

Originally this was SIGKILLing the daemon because we were hoping to not
have to manipulate on disk state, but since we need to anyway we can
shut it down normally.

I also tested this to ensure the test fails correctly without the fix
that the test was added to check for.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit e6591a9c7a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-02-03 13:54:09 +01:00
Brian Goff
11ecfe8a81
Merge pull request #41959 from AkihiroSuda/cherrypick-41917-2010
[20.10 backport] TestCgroupNamespacesRunOlderClient: support cgroup v2
2021-02-02 10:54:01 -08:00
Brian Goff
49df387b71
Merge pull request #41958 from AkihiroSuda/cherrypick-41894-2010 2021-02-02 10:52:42 -08:00
Brian Goff
54f561aeb9
Merge pull request #41956 from AkihiroSuda/cherrypick-41947-2010
[20.10 backport] rootless: prevent the service hanging when stopping (set systemd KillMode to mixed)
2021-02-02 10:51:08 -08:00
Akihiro Suda
519a55f491
TestCgroupNamespacesRunOlderClient: support cgroup v2
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-02-02 14:34:08 +09:00
Akihiro Suda
b6a6a35684
docker info: adjust warning strings for cgroup v2
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit 00225e220f)
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-02-02 14:32:13 +09:00
Akihiro Suda
25bd941ae4
docker info: silence unhandleable warnings
The following warnings in `docker info` are now discarded,
because there is no action user can actually take.

On cgroup v1:
- "WARNING: No blkio weight support"
- "WARNING: No blkio weight_device support"

On cgroup v2:
- "WARNING: No kernel memory TCP limit support"
- "WARNING: No oom kill disable support"

`docker run` still prints warnings when the missing feature is being attempted to use.

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit 8086443a44)
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-02-02 14:32:00 +09:00
Akihiro Suda
a287e76e15
pkg/archive: allow mknodding FIFO inside userns
Fix #41803

Also attempt to mknod devices.
Mknodding devices are likely to fail, but still worth trying when
running with a seccomp user notification.

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit d5d5cccb7e)
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-02-02 14:29:49 +09:00
Akihiro Suda
58283298d7
rootless: set systemd KillMode to mixed
Now `systemctl --user stop docker` completes just with in 1 or 2 seconds.

Fix issue 41944 ("Docker rootless does not exit properly if containers are running")

See systemd.kill(5) https://www.freedesktop.org/software/systemd/man/systemd.kill.html

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit 05566adf71)
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-02-02 14:26:47 +09:00
Tibor Vass
46229ca1d8 Use golang.org/x/sys/execabs
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 7ca0cb7ffa)
Signed-off-by: Tibor Vass <tibor@docker.com>
2021-01-28 21:33:12 +00:00
Brian Goff
a7d4af84bd pull: Validate layer digest format
Otherwise a malformed or empty digest may cause a panic.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2021-01-28 21:33:12 +00:00
Brian Goff
611eb6ffb3 buildkit: Apply apparmor profile
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2021-01-28 21:33:12 +00:00
Tibor Vass
4afe620fac vendor buildkit 68bb095353c65bc3993fd534c26cf77fe05e61b1
Signed-off-by: Tibor Vass <tibor@docker.com>
2021-01-28 20:20:56 +00:00
Brian Goff
e908cc3901 Use real root with 0701 perms
Various dirs in /var/lib/docker contain data that needs to be mounted
into a container. For this reason, these dirs are set to be owned by the
remapped root user, otherwise there can be permissions issues.
However, this uneccessarily exposes these dirs to an unprivileged user
on the host.

Instead, set the ownership of these dirs to the real root (or rather the
UID/GID of dockerd) with 0701 permissions, which allows the remapped
root to enter the directories but not read/write to them.
The remapped root needs to enter these dirs so the container's rootfs
can be configured... e.g. to mount /etc/resolve.conf.

This prevents an unprivileged user from having read/write access to
these dirs on the host.
The flip side of this is now any user can enter these directories.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2021-01-26 17:23:32 +00:00
Brian Goff
bfedd27259 Do not set DOCKER_TMP to be owned by remapped root
The remapped root does not need access to this dir.
Having this owned by the remapped root opens the host up to an
uprivileged user on the host being able to escalate privileges.

While it would not be normal for the remapped UID to be used outside of
the container context, it could happen.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2021-01-26 17:23:32 +00:00
Brian Goff
edb62a3ace Ensure MkdirAllAndChown also sets perms
Generally if we ever need to change perms of a dir, between versions,
this ensures the permissions actually change when we think it should
change without having to handle special cases if it already existed.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2021-01-26 17:23:32 +00:00
Akihiro Suda
8891c58a43
Merge pull request #41786 from thaJeztah/test_selinux_tip
vendor: opencontainers/selinux v1.8.0, and remove selinux build-tag and stubs
2020-12-26 00:07:49 +09:00
Tibor Vass
ffc4dc9aec
Merge pull request #41817 from simonferquel/desktop-startup-hang
Fix a potential hang when starting after a non-clean shutdown
2020-12-23 23:22:00 -08:00
Sebastiaan van Stijn
1c0af18c6c
vendor: opencontainers/selinux v1.8.0, and remove selinux build-tag and stubs
full diff: https://github.com/opencontainers/selinux/compare/v1.7.0...v1.8.0

Remove "selinux" build tag

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-12-24 00:47:16 +01:00
Brian Goff
4a175fd050 Cleanup container shutdown check and add test
Adds a test case for the case where dockerd gets stuck on startup due to
hanging `daemon.shutdownContainer`

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2020-12-23 16:59:03 +00:00
Tibor Vass
249d0ab744
Merge pull request #41805 from arkodg/vendor-libnetwork-ipv6-port-forwarding
Vendor in Libnetwork changes
2020-12-23 08:49:46 -08:00
Akihiro Suda
3ec41db836
Merge pull request #41801 from thaJeztah/selinux_1.7
vendor: opencontainers/selinux v1.7.0
2020-12-23 03:50:26 +09:00
Akihiro Suda
d9a9aeea22
Merge pull request #41832 from thaJeztah/sd_notify_stopping
cmd/dockerd: sd_notify STOPPING=1 when shutting down
2020-12-23 03:49:35 +09:00
Sebastiaan van Stijn
ab06bb3ec0
vendor: opencontainers/selinux v1.7.0
full diff: https://github.com/opencontainers/selinux/compare/v1.6.0...v1.7.0

- Implement get_default_context_with_level() from libselinux
- Wrap some syscalls (lgetattr, lsetattr, fstatfs, statfs) to retry on EINTR.
- Improve code quality by turning fixing many problems found by linters
- Use bufio.Scanner for parsing labels and policy confilabelg
- Cache the value for SELinux policy directory
- test on ppc64le and go 1.15

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-12-22 13:57:09 +01:00
Sebastiaan van Stijn
4d6bc59f81
Merge pull request #41740 from EricMountain/dishonoured-capabilities-test
Dishonoured capabilities test
2020-12-22 13:23:36 +01:00
Sebastiaan van Stijn
f3d0f7054d
cmd/dockerd: sd_notify STOPPING=1 when shutting down
Signal systemd when we start shutting down to complement the "READY" notify
that was originally implemented in 97088ebef7

From [sd_notify(3)](https://www.freedesktop.org/software/systemd/man/sd_notify.html#STOPPING=1)

> STOPPING=1
> Tells the service manager that the service is beginning its shutdown. This is useful
> to allow the service manager to track the service's internal state, and present it to
> the user.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-12-22 10:51:17 +01:00
Akihiro Suda
aa1ada6b2a
Merge pull request #41830 from ob/master
Fix an off-by-one bug
2020-12-22 13:43:56 +09:00
Oscar Bonilla
c923f6ac3b Fix off-by-one bug
This is a fix for https://github.com/docker/for-linux/issues/1012.

The code was not considering that C strings are NULL-terminated so
we need to leave one extra byte.

Without this fix, the testcase in https://github.com/docker/for-linux/issues/1012
fails with

```
Step 61/1001 : RUN echo 60 > 60
 ---> Running in dde85ac3b1e3
Removing intermediate container dde85ac3b1e3
 ---> 80a12a18a241
Step 62/1001 : RUN echo 61 > 61
error creating overlay mount to /23456789112345678921234/overlay2/d368abcc97d6c6ebcf23fa71225e2011d095295d5d8c9b31d6810bea748bdf07-init/merged: no such file or directory
```

with the output of `dmesg -T` as:

```
[Sat Dec 19 02:35:40 2020] overlayfs: failed to resolve '/23456789112345678921234/overlay2/89e435a1b24583c463abb73e8abfad8bf8a88312ef8253455390c5fa0a765517-init/wor': -2
```

with this fix, you get the expected:

```
Step 126/1001 : RUN echo 125 > 125
 ---> Running in 2f2e56da89e0
max depth exceeded
```

Signed-off-by: Oscar Bonilla <6f6231@gmail.com>
2020-12-20 16:23:25 -08:00
Eric Mountain
1c5806cf57
Move use of debian:buster frozen image to debian:bullseye
Signed-off-by: Eric Mountain <eric.mountain@datadoghq.com>
2020-12-19 17:59:36 +01:00
Simon Ferquel
af0665861b Fix a potential hang when starting after a non-clean shutdown
Previous startup sequence used to call "containerStop" on containers that were persisted with a running state but are not alive when restarting (can happen on non-clean shutdown).
This call was made before fixing-up the RunningState of the container, and tricked the daemon to trying to kill a non-existing process and ultimately hang.

The fix is very simple - just add a condition on calling containerStop.

Signed-off-by: Simon Ferquel <simon.ferquel@docker.com>
2020-12-18 10:20:56 +01:00
Eric Mountain
7ba1af38b8
TESTING.md: note that integration-cli is deprecated
Signed-off-by: Eric Mountain <eric.mountain@datadoghq.com>
2020-12-18 07:51:46 +01:00
Eric Mountain
a0fc7a4b46
TESTING.md: Fix path the test environment conditions
Signed-off-by: Eric Mountain <eric.mountain@datadoghq.com>
2020-12-18 07:51:45 +01:00
Eric Mountain
f2f5106c92
Add a test for #41723
Capabilities are serialised in VFS_CAP_REVISION_3 when an image is
built in a user-namespaced daemon, instead of VFS_CAP_REVISION_2.

This adds a test for this, though it's currently wired to fail if
the capabilities are serialised in VFS_CAP_REVISION_2 instead in this
situation, since this is unexpected.

Signed-off-by: Eric Mountain <eric.mountain@datadoghq.com>
2020-12-18 07:51:44 +01:00
Eric Mountain
b314e901d8
Add IsUserNamespaceInKernel() test environment condition
Signed-off-by: Eric Mountain <eric.mountain@datadoghq.com>
2020-12-18 07:51:43 +01:00
Eric Mountain
58a4c03fed
Add debian:bullseye to frozen images
Signed-off-by: Eric Mountain <eric.mountain@datadoghq.com>
2020-12-18 07:51:40 +01:00
Brian Goff
b5f863c67e
Merge pull request #41811 from AkihiroSuda/fuseoverlayfs-wrong-comment
fuse-overlayfs: fix godoc
2020-12-17 10:15:02 -08:00
Akihiro Suda
727d597452
Merge pull request #41806 from dperny/fix-jobs-filter-spelling
Fix service job mode filter
2020-12-16 20:05:41 +09:00
Akihiro Suda
188a691db7
fuse-overlayfs: fix godoc
"fuse-overlayfs" storage driver had wrong godoc comments
that were copied from "overlay2".

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-12-16 19:21:03 +09:00
Akihiro Suda
109be6b2bd
Merge pull request #41800 from thaJeztah/daemon_improve_logging
daemon: improve log messages during startup / shutdown
2020-12-16 09:58:32 +09:00