Commit graph

44618 commits

Author SHA1 Message Date
Cory Snider
1e0f2186a9 Fix containerd task deletion after failed start
Deleting a containerd task whose status is Created fails with a
"precondition failed" error. This is because (aside from Windows)
a process is spawned when the task is created, and deleting the task
while the process is running would leak the process if it was allowed.
libcontainerd mistakenly tries to clean up from a failed start by
deleting the created task, which will always fail with the
aforementioned error. Change it to pass the `WithProcessKill` delete
option so the cleanup has a chance to succeed.

Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit 1bef9e3fbf)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-11-02 16:59:22 -04:00
Cory Snider
4404c36460
Merge pull request #44376 from corhere/backport-22.06/gh-44363
[22.06 backport] Fix the max-concurrent-downloads and max-concurrent-uploads configs documentation
2022-10-31 13:00:46 -04:00
Cory Snider
75634f9a1e daemon: fix docs for config-default constants
Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit ad4073edc1)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-10-31 11:51:27 -04:00
Luis Henrique Mulinari
ad11d3f232 Fix the max-concurrent-downloads and max-concurrent-uploads configs documentation
This fix tries to address issues raised in #44346.
The max-concurrent-downloads and max-concurrent-uploads limits are applied for the whole engine and not for each pull/push command.

Signed-off-by: Luis Henrique Mulinari <luis.mulinari@gmail.com>
(cherry picked from commit 6c0aa5b00a)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-10-31 11:51:27 -04:00
Sebastiaan van Stijn
cbaf1808cb
Merge pull request #44360 from neersighted/backport_44224
[22.06 backport] Fix force-remove for cluster volumes
2022-10-26 10:36:39 -04:00
Drew Erny
03015fe6de
fix force remove for cluster volumes
Signed-off-by: Drew Erny <derny@mirantis.com>
(cherry picked from commit 3246db3755)
Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
2022-10-25 15:18:34 -06:00
Sebastiaan van Stijn
fa3804f8ba
Merge pull request #44357 from thaJeztah/22.06_backport_busybox_w32_img
[22.06 backport] integration: download busybox-w32 from GitHub Release
2022-10-25 07:44:47 -04:00
Sebastiaan van Stijn
4c1a3f096c
Merge pull request #44355 from thaJeztah/22.06_vendor_containerd_1.6.9
[22.06 backport] vendor: github.com/containerd/containerd v1.6.9
2022-10-24 17:58:09 -04:00
CrazyMax
09a2f7a667
integration: download busybox-w32 from GitHub Release
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 4f1d1422de)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-24 17:04:10 -04:00
Brian Goff
02e02e512f
Merge pull request #44352 from thaJeztah/22.06_update_containerd_binary
[22.06 backport] update containerd binary to v1.6.9
2022-10-24 11:57:06 -07:00
Sebastiaan van Stijn
24de1f7adc
vendor: github.com/containerd/containerd v1.6.9
release notes: https://github.com/containerd/containerd/releases/tag/v1.6.9

full diff: https://github.com/containerd/containerd/compare/v1.6.8...v1.6.9

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 04dc007c76)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-24 14:24:27 -04:00
Sebastiaan van Stijn
c4685540e4
update containerd binary to v1.6.9
release notes: https://github.com/containerd/containerd/releases/tag/v1.6.9

full diff: containerd/containerd@v1.6.8...v1.6.9

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit ac79a02ace)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-24 13:55:25 -04:00
Sebastiaan van Stijn
5aac513617
Merge pull request #44345 from thaJeztah/22.06_backport_go1.18_compat
[22.06 backport] builder/remotecontext/git: allow building on go1.18
2022-10-21 19:39:10 +02:00
Sebastiaan van Stijn
80dc5186ec
builder/remotecontext/git: allow building on go1.18
cmd.Environ() is new in go1.19, and not needed for this specific case.
Without this, trying to use this package in code that uses go1.18 will fail;

    builder/remotecontext/git/gitutils.go:216:23: cmd.Environ undefined (type *exec.Cmd has no field or method Environ)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 4fdc1bb1fb)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-21 17:44:20 +02:00
Sebastiaan van Stijn
f9cb47a052
Merge pull request #44341 from thaJeztah/22.06_backport_buildkit_skip_unit
[22.06 backport] gha: buildkit: remove "skip-integration-tests" from matrix
2022-10-21 14:21:14 +02:00
Sebastiaan van Stijn
5202b5c781
Merge pull request #44328 from thaJeztah/22.06_backport_ghsa-ambiguous-pull-by-digest
[22.06 backport] Validate digest in repo for pull by digest
2022-10-21 14:20:22 +02:00
Sebastiaan van Stijn
28c34259c7
Merge pull request #44297 from thaJeztah/22.06_backport_windows_bits
[22.06 backport] windows cleanups
2022-10-21 02:44:02 +02:00
Sebastiaan van Stijn
67ea873f61
Merge pull request #44325 from corhere/backport-22.06/fix-git-file-leak
[22.06 backport] builder: Isolate Git from local system
2022-10-21 02:11:56 +02:00
Sebastiaan van Stijn
f72c96c5c4
gha: buildkit: make checks more readable
GitHub uses these parameters to construct a name; removing the ./ prefix
to make them more readable (and add them back where it's used)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 0760c6f4e1)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-21 02:07:29 +02:00
Sebastiaan van Stijn
1bbb6f2454
gha: buildkit: remove "skip-integration-tests" from matrix
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit cfa2f9a2f2)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-21 02:07:26 +02:00
Sebastiaan van Stijn
c0be73f88d
skip TestImagePullStoredfDigestForOtherRepo() on Windows and rootless
- On Windows, we don't build and run a local  test registry (we're not running
  docker-in-docker), so we need to skip this test.
- On rootless, networking doesn't support this (currently)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 4f43cb660a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-21 01:50:09 +02:00
Brian Goff
727c4fdee3
Validate digest in repo for pull by digest
This is accomplished by storing the distribution source in the content
labels. If the distribution source is not found then we check to the
registry to see if the digest exists in the repo, if it does exist then
the puller will use it.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 27530efedb)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-21 01:50:09 +02:00
Sebastiaan van Stijn
b4c4be1f22
Revert "testutil/registry: remove unused WithStdout(), WithStErr() opts"
This reverts commit 1f21c4dd05.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 92eca900b0)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-21 01:50:09 +02:00
Sebastiaan van Stijn
7106874e39
Merge pull request #44338 from thaJeztah/22.06_backport_buildkit_testskips
[22.06 backport] gha: update buildkit to v0.10.5-6-ge27c8e24 to skip some tests
2022-10-21 01:48:00 +02:00
Sebastiaan van Stijn
4bef6f5510
gha: update buildkit to v0.10.5-6-ge27c8e24 to skip some tests
full diff: https://github.com/moby/buildkit/compare/v0.10.5...v0.10.5-6-ge27c8e24

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 201fdf67ac)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-20 23:50:16 +02:00
Cory Snider
f056df579a builder: add missing doc comment
Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-10-20 16:46:23 -04:00
Cory Snider
c062238ea4 builder: fix running git commands on Windows
Setting cmd.Env overrides the default of passing through the parent
process' environment, which works out fine most of the time, except when
it doesn't. For whatever reason, leaving out all the environment causes
git-for-windows sh.exe subprocesses to enter an infinite loop of
access violations during Cygwin initialization in certain environments
(specifically, our very own dev container image).

Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-10-20 16:46:23 -04:00
Cory Snider
20ff8a2380 builder: make git config isolation opt-in
While it is undesirable for the system or user git config to be used
when the daemon clones a Git repo, it could break workflows if it was
unconditionally applied to docker/cli as well.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-10-20 16:46:23 -04:00
Cory Snider
ca99cab891 builder: isolate git from local system
Prevent git commands we run from reading the user or system
configuration, or cloning submodules from the local filesystem.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-10-20 16:46:23 -04:00
Cory Snider
5829b244ec builder: explicitly set CWD for all git commands
Keep It Simple! Set the working directory for git commands by...setting
the git process's working directory. Git commands can be run in the
parent process's working directory by passing the empty string.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-10-20 16:46:23 -04:00
Cory Snider
3bc8fccc1b builder: modernize TestCheckoutGit
Make the test more debuggable by logging all git command output and
running each table-driven test case as a subtest.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-10-20 16:46:23 -04:00
Sebastiaan van Stijn
4a96094bf5
Merge pull request #44321 from thaJeztah/22.06_backport_bump_buildkit
[22.06 backport] vendor: github.com/moby/buildkit v0.10.5
2022-10-19 10:08:32 +02:00
Sebastiaan van Stijn
00b44caa69
vendor: github.com/moby/buildkit v0.10.5
https://github.com/moby/buildkit/releases/tag/v0.10.5

full diff: https://github.com/moby/buildkit/compare/v0.10.4...v0.10.5

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 0fc17c42af)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-18 22:39:56 +02:00
Sebastiaan van Stijn
1fcb1dd728
Merge pull request #44314 from tianon/22.06-distributable
[22.06 backport] registry: allow "allow-nondistributable-artifacts" for Docker Hub
2022-10-18 14:39:53 +02:00
Sebastiaan van Stijn
aaa8f96cc9 registry: allow "allow-nondistributable-artifacts" for Docker Hub
Previously, Docker Hub was excluded when configuring "allow-nondistributable-artifacts".
With the updated policy announced by Microsoft, we can remove this restriction;
https://techcommunity.microsoft.com/t5/containers/announcing-windows-container-base-image-redistribution-rights/ba-p/3645201

There are plans to deprecated support for foreign layers altogether in the OCI,
and we should consider to make this option the default, but as that requires
deprecating the option (and possibly keeping an "opt-out" option), we can look
at that separately.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 30e5333ce3)
2022-10-17 15:04:59 -07:00
Sebastiaan van Stijn
671bf589e2
Change restart delay for Windows service to 15s
Previously we waited for 60 seconds after the service faults to restart
it. However, there isn't much benefit to waiting this long. We expect
15 seconds to be a more reasonable delay.

Co-Authored-by: Kevin Parsons <kevpar@microsoft.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 624daf8d9e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-13 23:08:42 +02:00
Sebastiaan van Stijn
e1b240d6bd
cmd/dockerd: use golang.org/x/sys Service.SetRecoveryActions()
This is the equivalent of the local implementation.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 3c585e6567)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-13 23:08:42 +02:00
Sebastiaan van Stijn
18a54ed59c
cmd/dockerd: use golang.org/x/sys/windows.SetStdHandle()
golang.org/x/sys/windows now implements this, so we can use that
instead of a local implementation.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 6176ab5901)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-13 23:05:49 +02:00
Sebastiaan van Stijn
0c66bc948a
cmd/dockerd: replace deprecated windows.IsAnInteractiveSession()
The `IsAnInteractiveSession` was deprecated, and `IsWindowsService` is marked
as the recommended replacement.

For details, see 280f808b4a

> CL 244958 includes isWindowsService function that determines if a
> process is running as a service. The code of the function is based on
> public .Net implementation.
>
> IsAnInteractiveSession function implements similar functionality, but
> is based on an old Stackoverflow post., which is not as authoritative
> as code written by Microsoft for their official product.
>
> This change copies CL 244958 isWindowsService function into svc package
> and makes it public. The intention is that future users will prefer
> IsWindowsService to IsAnInteractiveSession.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit ffcddc908e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-13 23:05:48 +02:00
Sebastiaan van Stijn
a12d359c1a
daemon/graphdriver/windows: Remove() don't use defer() in a loop
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 75bdbf02a6)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-13 23:05:48 +02:00
Sebastiaan van Stijn
2d12e69c9f
daemon/graphdriver/windows: use go-winio.GetFileSystemType()
go-winio now defines this function, so we can consume that.

Note that there's a difference between the old implementation and the original
one (added in 1cb9e9b44e). The old implementation
had special handling for win32 error codes, which was removed in the go-winio
implementation in 0966e1ad56

As `go-winio.GetFileSystemType()` calls `filepath.VolumeName(path)` internally,
this patch also removes the `string(home[0])`, which is redundant, and could
potentially panic if an empty string would be passed.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 90431d1857)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-13 23:05:45 +02:00
Sebastiaan van Stijn
33ab36d6b3
Merge pull request #44279 from thaJeztah/22.06_backport_overlay_remove_kernel_check_override
[22.06 backport] daemon/graphdriver/overlay2: remove deprecated overrideKernelCheck
2022-10-13 22:35:45 +02:00
Sebastiaan van Stijn
fa10084a82
daemon/graphdriver/overlay2: remove deprecated overrideKernelCheck
Commit 955c1f881a (Docker v17.12.0) replaced
detection of support for multiple lowerdirs (as required by overlay2) to not
depend on the kernel version. The `overlay2.override_kernel_check` was still
used to print a warning that older kernel versions may not have full support.

After this, commit e226aea280 (Docker v20.10.0,
backported to v19.03.7) removed uses of the `overlay2.override_kernel_check`
option altogether, but we were still parsing it.

This patch changes the `parseOptions()` function to not parse the option,
printing a deprecation warning instead. We should change this to be an error,
but the  `overlay2.override_kernel_check` option was not deprecated in the
documentation, so keeping it around for one more release.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit e35700eb50)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-10 15:00:30 +02:00
Sebastiaan van Stijn
43ce8f7d24
integration/plugin: remove deprecated overlay2.override_kernel_check
It's no longer used since e226aea280

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit b43a7ac530)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-10 15:00:29 +02:00
Sebastiaan van Stijn
87d9d96ab0
Merge pull request #44262 from thaJeztah/22.06_backport_bump_selinux
[22.06 backport] vendor: github.com/opencontainers/selinux v1.10.2
2022-10-06 20:54:38 +02:00
Sebastiaan van Stijn
a5ecbf4d22
vendor: github.com/opencontainers/selinux v1.10.2
full diff: https://github.com/opencontainers/selinux/compare/v1.10.1...v1.10.2

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit a5e1baf3ab)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-06 16:48:58 +02:00
Sebastiaan van Stijn
99aa9bb766
Merge pull request #44260 from thaJeztah/22.06_backport_deprecate_pkg_fsutil
[22.06 backport] pkg/fsutils: deprecate in favor of containerd/continuity/fs
2022-10-06 02:50:07 +02:00
Brian Goff
6442025060
Merge pull request #44259 from thaJeztah/22.06_backport_volume_unnamed_label
[22.06 backport] Volume prune: only prune anonymous volumes by default
2022-10-05 17:01:24 -07:00
Sebastiaan van Stijn
ac6624773e
pkg/fsutils: deprecate in favor of containerd/continuity/fs
The pkg/fsutils package was forked in containerd, and later moved to
containerd/continuity/fs. As we're moving more bits to containerd, let's also
use the same implementation to reduce code-duplication and to prevent them from
diverging.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 5b6b42162b)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-05 23:52:55 +02:00
Brian Goff
4669418731
Volume prune: only prune anonymous volumes by default
This adds a new filter argument to the volume prune endpoint "all".
When this is not set, or it is a false-y value, then only anonymous
volumes are considered for pruning.

When `all` is set to a truth-y value, you get the old behavior.

This is an API change, but I think one that is what most people would
want.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 618f26ccbc)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-05 23:50:01 +02:00