Commit graph

44532 commits

Author SHA1 Message Date
Bjorn Neergaard
7f45eb041c
ci(actions): migrate to file-based commands
Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
(cherry picked from commit 0557569947)
Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
2022-11-10 16:48:33 -07:00
Tianon Gravi
b76a60dee6
Merge pull request #44414 from thaJeztah/22.06_backport_rm_deprecated_arm_fallback
[22.06 backport] Remove long-deprecated "arm" fallback
2022-11-10 12:21:14 -08:00
Sebastiaan van Stijn
4acfbaba1e
Merge pull request #44430 from thaJeztah/22.06_swap_digestset
[22.06 backport] replace distribution/digestset with opencontainers/go-digest/digestset
2022-11-10 21:09:01 +01:00
Sebastiaan van Stijn
e749a31322
Merge pull request #44416 from thaJeztah/22.06_backport_enable_deprecated_check
[22.06 backport] Revert "validation: temporarily allows changes in integration-cli"
2022-11-10 18:02:41 +01:00
Sebastiaan van Stijn
7370bbc034
replace distribution/digestset with opencontainers/go-digest/digestset
opencontainers/go-digest is a 1:1 copy of the one in distribution. It's no
longer used in distribution itself, so may be removed there at some point.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 6174d00c03)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-09 10:22:38 +01:00
Samuel Karp
38152f4d5b
Merge pull request #44411 from thaJeztah/22.06_backport_bump_go1.19.3
fixes https://github.com/golang/go/issues/56309
2022-11-08 19:12:03 -08:00
Sebastiaan van Stijn
21feb1808d
Revert "validation: temporarily allows changes in integration-cli"
This reverts commit 7ed823ead9.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 9b71a46899)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-05 18:35:47 +01:00
Tianon Gravi
7175841ebd
Remove long-deprecated "arm" fallback
This fallback is used when we filter the manifest list by the user-provided platform and find no matches such that we match the previous Docker behavior (before it supported variant matching).  This has been deprecated long enough that I think it's time we finally stop supporting this weird fallback, especially since it makes for buggy behavior like `docker pull --platform linux/arm/v5 alpine:3.16` leading to a `linux/arm/v6` image being pulled (I specified a variant, every manifest list entry specifies a variant, so clearly the only behavior I as a user could reasonably expect is an error that `linux/arm/v5` is not supported, but instead I get an explicitly incompatible image despite doing everything I as a user can to prevent that situation).

Signed-off-by: Tianon Gravi <admwiggin@gmail.com>
(cherry picked from commit 5bc17c3e54)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-05 18:23:43 +01:00
Cory Snider
f3e180b704
Update to Go 1.19.3 to address CVE-2022-41716
On Windows, syscall.StartProcess and os/exec.Cmd did not properly
    check for invalid environment variable values. A malicious
    environment variable value could exploit this behavior to set a
    value for a different environment variable. For example, the
    environment variable string "A=B\x00C=D" set the variables "A=B" and
    "C=D".

    Thanks to RyotaK (https://twitter.com/ryotkak) for reporting this
    issue.

    This is CVE-2022-41716 and Go issue https://go.dev/issue/56284.

This Go release also fixes https://github.com/golang/go/issues/56309, a
runtime bug which can cause random memory corruption when a goroutine
exits with runtime.LockOSThread() set. This fix is necessary to unblock
work to replace certain uses of pkg/reexec with unshared OS threads.

Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit f9d4589976)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-05 17:52:37 +01:00
Sebastiaan van Stijn
afdc9a804a
Merge pull request #44404 from neersighted/swarmkit_revendor_22.06
[22.06 backport] vendor: github.com/moby/swarmkit/v2 v2.0.0-20221102165002-6341884e5fc9
2022-11-03 22:27:56 +01:00
Sebastiaan van Stijn
e24277883f
Merge pull request #44405 from vvoland/oci-artifacts-error-2206
[22.06 backport] distribution: Error when pulling OCI artifacts
2022-11-03 22:27:37 +01:00
Paweł Gronowski
07e84005ac
distribution: Error when pulling OCI artifacts
Currently an attempt to pull a reference which resolves to an OCI
artifact (Helm chart for example), results in a bit unrelated error
message `invalid rootfs in image configuration`.

This provides a more meaningful error in case a user attempts to
download a media type which isn't image related.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2022-11-03 19:41:51 +01:00
Bjorn Neergaard
39d3d3db56
vendor: github.com/moby/swarmkit/v2 v2.0.0-20221102165002-6341884e5fc9
full diff: 48dd89375d...6341884e5f

Pulls in a set of fixes to SwarmKit's nascent Cluster Volumes support
discovered during subsequent development and testing.

Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
(cherry picked from commit 57c2545cd5)
Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
2022-11-03 12:30:53 -06:00
Cory Snider
4b79d9078a
Merge pull request #44400 from corhere/backport-22.06/fix-task-delete-on-failed-start
[22.06 backport] Fix containerd task deletion after failed start
2022-11-02 18:15:19 -04:00
Cory Snider
1e0f2186a9 Fix containerd task deletion after failed start
Deleting a containerd task whose status is Created fails with a
"precondition failed" error. This is because (aside from Windows)
a process is spawned when the task is created, and deleting the task
while the process is running would leak the process if it was allowed.
libcontainerd mistakenly tries to clean up from a failed start by
deleting the created task, which will always fail with the
aforementioned error. Change it to pass the `WithProcessKill` delete
option so the cleanup has a chance to succeed.

Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit 1bef9e3fbf)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-11-02 16:59:22 -04:00
Cory Snider
4404c36460
Merge pull request #44376 from corhere/backport-22.06/gh-44363
[22.06 backport] Fix the max-concurrent-downloads and max-concurrent-uploads configs documentation
2022-10-31 13:00:46 -04:00
Cory Snider
75634f9a1e daemon: fix docs for config-default constants
Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit ad4073edc1)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-10-31 11:51:27 -04:00
Luis Henrique Mulinari
ad11d3f232 Fix the max-concurrent-downloads and max-concurrent-uploads configs documentation
This fix tries to address issues raised in #44346.
The max-concurrent-downloads and max-concurrent-uploads limits are applied for the whole engine and not for each pull/push command.

Signed-off-by: Luis Henrique Mulinari <luis.mulinari@gmail.com>
(cherry picked from commit 6c0aa5b00a)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-10-31 11:51:27 -04:00
Sebastiaan van Stijn
cbaf1808cb
Merge pull request #44360 from neersighted/backport_44224
[22.06 backport] Fix force-remove for cluster volumes
2022-10-26 10:36:39 -04:00
Drew Erny
03015fe6de
fix force remove for cluster volumes
Signed-off-by: Drew Erny <derny@mirantis.com>
(cherry picked from commit 3246db3755)
Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
2022-10-25 15:18:34 -06:00
Sebastiaan van Stijn
fa3804f8ba
Merge pull request #44357 from thaJeztah/22.06_backport_busybox_w32_img
[22.06 backport] integration: download busybox-w32 from GitHub Release
2022-10-25 07:44:47 -04:00
Sebastiaan van Stijn
4c1a3f096c
Merge pull request #44355 from thaJeztah/22.06_vendor_containerd_1.6.9
[22.06 backport] vendor: github.com/containerd/containerd v1.6.9
2022-10-24 17:58:09 -04:00
CrazyMax
09a2f7a667
integration: download busybox-w32 from GitHub Release
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 4f1d1422de)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-24 17:04:10 -04:00
Brian Goff
02e02e512f
Merge pull request #44352 from thaJeztah/22.06_update_containerd_binary
[22.06 backport] update containerd binary to v1.6.9
2022-10-24 11:57:06 -07:00
Sebastiaan van Stijn
24de1f7adc
vendor: github.com/containerd/containerd v1.6.9
release notes: https://github.com/containerd/containerd/releases/tag/v1.6.9

full diff: https://github.com/containerd/containerd/compare/v1.6.8...v1.6.9

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 04dc007c76)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-24 14:24:27 -04:00
Sebastiaan van Stijn
c4685540e4
update containerd binary to v1.6.9
release notes: https://github.com/containerd/containerd/releases/tag/v1.6.9

full diff: containerd/containerd@v1.6.8...v1.6.9

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit ac79a02ace)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-24 13:55:25 -04:00
Sebastiaan van Stijn
5aac513617
Merge pull request #44345 from thaJeztah/22.06_backport_go1.18_compat
[22.06 backport] builder/remotecontext/git: allow building on go1.18
2022-10-21 19:39:10 +02:00
Sebastiaan van Stijn
80dc5186ec
builder/remotecontext/git: allow building on go1.18
cmd.Environ() is new in go1.19, and not needed for this specific case.
Without this, trying to use this package in code that uses go1.18 will fail;

    builder/remotecontext/git/gitutils.go:216:23: cmd.Environ undefined (type *exec.Cmd has no field or method Environ)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 4fdc1bb1fb)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-21 17:44:20 +02:00
Sebastiaan van Stijn
f9cb47a052
Merge pull request #44341 from thaJeztah/22.06_backport_buildkit_skip_unit
[22.06 backport] gha: buildkit: remove "skip-integration-tests" from matrix
2022-10-21 14:21:14 +02:00
Sebastiaan van Stijn
5202b5c781
Merge pull request #44328 from thaJeztah/22.06_backport_ghsa-ambiguous-pull-by-digest
[22.06 backport] Validate digest in repo for pull by digest
2022-10-21 14:20:22 +02:00
Sebastiaan van Stijn
28c34259c7
Merge pull request #44297 from thaJeztah/22.06_backport_windows_bits
[22.06 backport] windows cleanups
2022-10-21 02:44:02 +02:00
Sebastiaan van Stijn
67ea873f61
Merge pull request #44325 from corhere/backport-22.06/fix-git-file-leak
[22.06 backport] builder: Isolate Git from local system
2022-10-21 02:11:56 +02:00
Sebastiaan van Stijn
f72c96c5c4
gha: buildkit: make checks more readable
GitHub uses these parameters to construct a name; removing the ./ prefix
to make them more readable (and add them back where it's used)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 0760c6f4e1)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-21 02:07:29 +02:00
Sebastiaan van Stijn
1bbb6f2454
gha: buildkit: remove "skip-integration-tests" from matrix
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit cfa2f9a2f2)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-21 02:07:26 +02:00
Sebastiaan van Stijn
c0be73f88d
skip TestImagePullStoredfDigestForOtherRepo() on Windows and rootless
- On Windows, we don't build and run a local  test registry (we're not running
  docker-in-docker), so we need to skip this test.
- On rootless, networking doesn't support this (currently)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 4f43cb660a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-21 01:50:09 +02:00
Brian Goff
727c4fdee3
Validate digest in repo for pull by digest
This is accomplished by storing the distribution source in the content
labels. If the distribution source is not found then we check to the
registry to see if the digest exists in the repo, if it does exist then
the puller will use it.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 27530efedb)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-21 01:50:09 +02:00
Sebastiaan van Stijn
b4c4be1f22
Revert "testutil/registry: remove unused WithStdout(), WithStErr() opts"
This reverts commit 1f21c4dd05.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 92eca900b0)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-21 01:50:09 +02:00
Sebastiaan van Stijn
7106874e39
Merge pull request #44338 from thaJeztah/22.06_backport_buildkit_testskips
[22.06 backport] gha: update buildkit to v0.10.5-6-ge27c8e24 to skip some tests
2022-10-21 01:48:00 +02:00
Sebastiaan van Stijn
4bef6f5510
gha: update buildkit to v0.10.5-6-ge27c8e24 to skip some tests
full diff: https://github.com/moby/buildkit/compare/v0.10.5...v0.10.5-6-ge27c8e24

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 201fdf67ac)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-20 23:50:16 +02:00
Cory Snider
f056df579a builder: add missing doc comment
Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-10-20 16:46:23 -04:00
Cory Snider
c062238ea4 builder: fix running git commands on Windows
Setting cmd.Env overrides the default of passing through the parent
process' environment, which works out fine most of the time, except when
it doesn't. For whatever reason, leaving out all the environment causes
git-for-windows sh.exe subprocesses to enter an infinite loop of
access violations during Cygwin initialization in certain environments
(specifically, our very own dev container image).

Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-10-20 16:46:23 -04:00
Cory Snider
20ff8a2380 builder: make git config isolation opt-in
While it is undesirable for the system or user git config to be used
when the daemon clones a Git repo, it could break workflows if it was
unconditionally applied to docker/cli as well.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-10-20 16:46:23 -04:00
Cory Snider
ca99cab891 builder: isolate git from local system
Prevent git commands we run from reading the user or system
configuration, or cloning submodules from the local filesystem.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-10-20 16:46:23 -04:00
Cory Snider
5829b244ec builder: explicitly set CWD for all git commands
Keep It Simple! Set the working directory for git commands by...setting
the git process's working directory. Git commands can be run in the
parent process's working directory by passing the empty string.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-10-20 16:46:23 -04:00
Cory Snider
3bc8fccc1b builder: modernize TestCheckoutGit
Make the test more debuggable by logging all git command output and
running each table-driven test case as a subtest.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-10-20 16:46:23 -04:00
Sebastiaan van Stijn
4a96094bf5
Merge pull request #44321 from thaJeztah/22.06_backport_bump_buildkit
[22.06 backport] vendor: github.com/moby/buildkit v0.10.5
2022-10-19 10:08:32 +02:00
Sebastiaan van Stijn
00b44caa69
vendor: github.com/moby/buildkit v0.10.5
https://github.com/moby/buildkit/releases/tag/v0.10.5

full diff: https://github.com/moby/buildkit/compare/v0.10.4...v0.10.5

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 0fc17c42af)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-18 22:39:56 +02:00
Sebastiaan van Stijn
1fcb1dd728
Merge pull request #44314 from tianon/22.06-distributable
[22.06 backport] registry: allow "allow-nondistributable-artifacts" for Docker Hub
2022-10-18 14:39:53 +02:00
Sebastiaan van Stijn
aaa8f96cc9 registry: allow "allow-nondistributable-artifacts" for Docker Hub
Previously, Docker Hub was excluded when configuring "allow-nondistributable-artifacts".
With the updated policy announced by Microsoft, we can remove this restriction;
https://techcommunity.microsoft.com/t5/containers/announcing-windows-container-base-image-redistribution-rights/ba-p/3645201

There are plans to deprecated support for foreign layers altogether in the OCI,
and we should consider to make this option the default, but as that requires
deprecating the option (and possibly keeping an "opt-out" option), we can look
at that separately.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 30e5333ce3)
2022-10-17 15:04:59 -07:00
Sebastiaan van Stijn
671bf589e2
Change restart delay for Windows service to 15s
Previously we waited for 60 seconds after the service faults to restart
it. However, there isn't much benefit to waiting this long. We expect
15 seconds to be a more reasonable delay.

Co-Authored-by: Kevin Parsons <kevpar@microsoft.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 624daf8d9e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-13 23:08:42 +02:00