Commit graph

39883 commits

Author SHA1 Message Date
Sebastiaan van Stijn
e3fef9aea9
Merge pull request #44340 from thaJeztah/20.10_backport_skip_tests
[20.10 backport] skip TestImagePullStoredfDigestForOtherRepo() on Windows and rootless
2022-10-21 11:30:04 +02:00
Sebastiaan van Stijn
9c84417c1b
skip TestImagePullStoredfDigestForOtherRepo() on Windows and rootless
- On Windows, we don't build and run a local  test registry (we're not running
  docker-in-docker), so we need to skip this test.
- On rootless, networking doesn't support this (currently)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 4f43cb660a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-21 01:52:32 +02:00
Sebastiaan van Stijn
93c2c3bc60
Merge pull request #44332 from corhere/backport-20.10/fix-git-windows-environ
[20.10] builder: fix running git commands on Windows
2022-10-20 21:16:42 +02:00
Cory Snider
5b5b5c6f13 builder: add missing doc comment
Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-10-20 14:03:36 -04:00
Cory Snider
05e25f7892 builder: fix running git commands on Windows
Setting cmd.Env overrides the default of passing through the parent
process' environment, which works out fine most of the time, except when
it doesn't. For whatever reason, leaving out all the environment causes
git-for-windows sh.exe subprocesses to enter an infinite loop of
access violations during Cygwin initialization in certain environments
(specifically, our very own dev container image).

Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-10-20 13:41:42 -04:00
Sebastiaan van Stijn
2b7f97a974
Merge pull request #44313 from tianon/20.10-distributable
[20.10 backport] registry: allow "allow-nondistributable-artifacts" for Docker Hub
2022-10-19 23:27:58 +02:00
Sebastiaan van Stijn
03df974ae9
Merge pull request #6 from moby/ghsa-ambiguous-pull-by-digest
[20.10] Validate digest in repo for pull by digest
2022-10-18 19:26:23 +02:00
Sebastiaan van Stijn
3adff51f85
Merge pull request #9 from moby/update_buildkit
[20.10] vendor moby/buildkit v0.8.3-31-gc0149372
2022-10-18 19:17:40 +02:00
Sebastiaan van Stijn
d7c33ad827
Merge pull request #8 from moby/20.10_fix_git_file_leak
[20.10 backport] builder: Isolate Git from local system
2022-10-18 19:08:22 +02:00
Sebastiaan van Stijn
2f3bf18014
[20.10] vendor moby/buildkit v0.8.3-31-gc0149372
full diff: 3a1eeca59a...c014937225

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-18 10:12:37 +02:00
Sebastiaan van Stijn
6699afa549 registry: allow "allow-nondistributable-artifacts" for Docker Hub
Previously, Docker Hub was excluded when configuring "allow-nondistributable-artifacts".
With the updated policy announced by Microsoft, we can remove this restriction;
https://techcommunity.microsoft.com/t5/containers/announcing-windows-container-base-image-redistribution-rights/ba-p/3645201

There are plans to deprecated support for foreign layers altogether in the OCI,
and we should consider to make this option the default, but as that requires
deprecating the option (and possibly keeping an "opt-out" option), we can look
at that separately.

(cherry picked from commit 30e5333ce3)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Tianon Gravi <admwiggin@gmail.com>
2022-10-17 15:12:13 -07:00
Brian Goff
4b9902bad4
Validate digest in repo for pull by digest
This is accomplished by storing the distribution source in the content
labels. If the distribution source is not found then we check to the
registry to see if the digest exists in the repo, if it does exist then
the puller will use it.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-17 17:47:46 +02:00
Cory Snider
c0d1188c14 builder: make git config isolation opt-in
While it is undesirable for the system or user git config to be used
when the daemon clones a Git repo, it could break workflows if it was
unconditionally applied to docker/cli as well.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-10-13 17:34:23 -04:00
Cory Snider
9f5f3abcee builder: isolate git from local system
Prevent git commands we run from reading the user or system
configuration, or cloning submodules from the local filesystem.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-10-13 17:34:20 -04:00
Cory Snider
10db4c2db7 builder: explicitly set CWD for all git commands
Keep It Simple! Set the working directory for git commands by...setting
the git process's working directory. Git commands can be run in the
parent process's working directory by passing the empty string.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-10-13 17:34:20 -04:00
Cory Snider
8816c3c2aa builder: modernize TestCheckoutGit
Make the test more debuggable by logging all git command output and
running each table-driven test case as a subtest.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-10-13 17:34:20 -04:00
Sebastiaan van Stijn
c964641a0d
Merge pull request #44122 from thaJeztah/20.10_bump_buildkit
[20.10] vendor: github.com/moby/buildkit 3a1eeca59a9263613d996ead67d53a4b7d45723d (v0.8 branch)
2022-10-06 20:53:28 +02:00
Brian Goff
3bb99735b6
Merge pull request #44127 from thaJeztah/20.10_backport_image_spec_no_literal
[20.10] Update uses of Image platform fields in OCI image-spec
2022-10-06 11:18:55 -07:00
Sebastiaan van Stijn
435c40ce15
Merge pull request #44247 from thaJeztah/20.10_bump_go_1.18.7
[20.10] Update to go 1.18.7 to address CVE-2022-2879, CVE-2022-2880, CVE-2022-41715
2022-10-06 20:13:47 +02:00
Sebastiaan van Stijn
11bdbf40b9
[20.10] Update to go 1.18.7 to address CVE-2022-2879, CVE-2022-2880, CVE-2022-41715
From the mailing list:

We have just released Go versions 1.19.2 and 1.18.7, minor point releases.

These minor releases include 3 security fixes following the security policy:

- archive/tar: unbounded memory consumption when reading headers

  Reader.Read did not set a limit on the maximum size of file headers.
  A maliciously crafted archive could cause Read to allocate unbounded
  amounts of memory, potentially causing resource exhaustion or panics.
  Reader.Read now limits the maximum size of header blocks to 1 MiB.

  Thanks to Adam Korczynski (ADA Logics) and OSS-Fuzz for reporting this issue.

  This is CVE-2022-2879 and Go issue https://go.dev/issue/54853.

- net/http/httputil: ReverseProxy should not forward unparseable query parameters

  Requests forwarded by ReverseProxy included the raw query parameters from the
  inbound request, including unparseable parameters rejected by net/http. This
  could permit query parameter smuggling when a Go proxy forwards a parameter
  with an unparseable value.

  ReverseProxy will now sanitize the query parameters in the forwarded query
  when the outbound request's Form field is set after the ReverseProxy.Director
  function returns, indicating that the proxy has parsed the query parameters.
  Proxies which do not parse query parameters continue to forward the original
  query parameters unchanged.

  Thanks to Gal Goldstein (Security Researcher, Oxeye) and
  Daniel Abeles (Head of Research, Oxeye) for reporting this issue.

  This is CVE-2022-2880 and Go issue https://go.dev/issue/54663.

- regexp/syntax: limit memory used by parsing regexps

  The parsed regexp representation is linear in the size of the input,
  but in some cases the constant factor can be as high as 40,000,
  making relatively small regexps consume much larger amounts of memory.

  Each regexp being parsed is now limited to a 256 MB memory footprint.
  Regular expressions whose representation would use more space than that
  are now rejected. Normal use of regular expressions is unaffected.

  Thanks to Adam Korczynski (ADA Logics) and OSS-Fuzz for reporting this issue.

  This is CVE-2022-41715 and Go issue https://go.dev/issue/55949.

View the release notes for more information: https://go.dev/doc/devel/release#go1.18.7

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-04 22:20:50 +02:00
Sebastiaan van Stijn
35eaf7ecc4
Merge pull request #44238 from cpuguy83/20.10_fix_restore_volumerefs
[20.10] Fix live-restore w/ restart policies + volume refs
2022-10-03 23:32:53 +02:00
Brian Goff
66ddb7f91c Fix live-restore w/ restart policies + volume refs
Before this change restarting the daemon in live-restore with running
containers + a restart policy meant that volume refs were not restored.
This specifically happens when the container is still running *and*
there is a restart policy that would make sure the container was running
again on restart.

The bug allows volumes to be removed even though containers are
referencing them. 😱

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit 4c0e0979b4)
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2022-10-03 18:41:34 +00:00
Sebastiaan van Stijn
f219cb5ded
Merge pull request #44218 from thaJeztah/20.10_backport_more_robust_rootless
[20.10 backport] docker-rootless-setuptools.sh fixes
2022-09-29 22:09:04 +02:00
Sebastiaan van Stijn
c003392582
contrib: make dockerd-rootless-setuptool.sh more robust
The `docker` CLI currently doesn't handle situations where the current context
(as defined in `~/.docker/config.json`) is invalid or doesn't exist. As loading
(and checking) the context happens during initialization of the CLI, this
prevents `docker context` commands from being used, which makes it complicated
to fix the situation. For example, running `docker context use <correct context>`
would fail, which makes it not possible to update the `~/.docker/config.json`,
unless doing so manually.

For example, given the following `~/.docker/config.json`:

```json
{
        "currentContext": "nosuchcontext"
}
```

All of the commands below fail:

```bash
docker context inspect rootless
Current context "nosuchcontext" is not found on the file system, please check your config file at /Users/thajeztah/.docker/config.json

docker context rm --force rootless
Current context "nosuchcontext" is not found on the file system, please check your config file at /Users/thajeztah/.docker/config.json

docker context use default
Current context "nosuchcontext" is not found on the file system, please check your config file at /Users/thajeztah/.docker/config.json
```

While these things should be fixed, this patch updates the script to switch
the context using the `--context` flag; this flag is taken into account when
initializing the CLI, so that having an invalid context configured won't
block `docker context` commands from being executed. Given that all `context`
commands are local operations, "any" context can be used (it doesn't need to
make a connection with the daemon).

With this patch, those commands can now be run (and won't fail for the wrong
reason);

```bash
 docker --context=default context inspect -f "{{.Name}}" rootless
rootless

docker --context=default context inspect -f "{{.Name}}" rootless-doesnt-exist
context "rootless-doesnt-exist" does not exist
```

One other issue may also cause things to fail during uninstall; trying to remove
a context that doesn't exist will fail (even with the `-f` / `--force` option
set);

```bash
docker --context=default context rm blablabla
Error: context "blablabla": not found
```

While this is "ok" in most circumstances, it also means that (potentially) the
current context is not reset to "default", so this patch adds an explicit
`docker context use`, as well as unsetting the `DOCKER_HOST` and `DOCKER_CONTEXT`
environment variables.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit e2114731e7)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-29 10:05:56 +02:00
Mathieu PATUREL
53313be0f3
docker-rootless-setuptools.sh: use context after install
Signed-off-by: Mathieu PATUREL <mathieu.paturel@gmail.com>
(cherry picked from commit 7c17ad8735)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-29 10:05:51 +02:00
Sebastiaan van Stijn
4163c55082
Merge pull request #44202 from crazy-max/20.10_backport_api-fix-logo
[20.10 backport] swagger: update links to logo
2022-09-27 17:54:20 +02:00
CrazyMax
9c486bd267
swagger: update links to logo
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 7f3602f1c9)
2022-09-27 11:57:16 +02:00
Sebastiaan van Stijn
956161115f
Merge pull request #44167 from thaJeztah/20.10_backport_update_containerd_console
[20.10 backport] vendor: github.com/containerd/console v1.0.2
2022-09-22 20:16:05 +02:00
Sebastiaan van Stijn
668ab11a67
Merge pull request #44145 from neersighted/bump_memberlist_20.10
[20.10] vendor: github.com/hashicorp/memberlist v0.4.0
2022-09-21 17:51:18 +02:00
Sebastiaan van Stijn
fa17fab895
vendor: github.com/containerd/console v1.0.2
Contains fix for s390x and support for FreeBSD

full diff: https://github.com/containerd/console/compare/v1.0.1...v1.0.2

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 948e201c1c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-21 16:41:46 +02:00
Bjorn Neergaard
481bee51b5 vendor: github.com/armon/go-metrics v0.4.1
from github.com/hashicorp/memberlist v0.4.0 go.mod

Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
2022-09-21 08:12:24 -06:00
Bjorn Neergaard
39ba2873e8 vendor: github.com/google/btree v1.1.2
from github.com/hashicorp/memberlist v0.4.0 go.mod

Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
2022-09-21 08:12:24 -06:00
Bjorn Neergaard
c2755f40cd vendor: github.com/hasicorp/memberlist v0.4.0
Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
2022-09-21 08:12:24 -06:00
Sebastiaan van Stijn
cc3aa33f54
Merge pull request #44123 from linux-on-ibm-z/20.10-CI
[20.10] Dockerfile: switch to bullseye, add missing libseccomp-dev, remove build pack
2022-09-21 14:41:13 +02:00
Prabhav Thali
5ba3208ec7 Dockerfile: Update Dockerfile syntax, switch to bullseye, add missing libseccomp-dev, remove build pack
Signed-off-by: Prabhav Thali <Prabhav.Thali1@ibm.com>
2022-09-20 16:18:21 +05:30
Sebastiaan van Stijn
6d6a236286
[20.10] Update uses of Image platform fields in OCI image-spec
The OCI image spec is considering to change the Image struct and embedding the
Platform type (see opencontainers/image-spec#959) in the go implementation.
Moby currently uses some struct-literals to propagate the platform fields,
which will break once those changes in the OCI spec are merged.

Ideally (once that change arrives) we would update the code to set the Platform
information as a whole, instead of assigning related fields individually, but
in some cases in the code, image platform information is only partially set
(for example, OSVersion and OSFeatures are not preserved in all cases). This
may be on purpose, so needs to be reviewed.

This patch keeps the current behavior (assigning only specific fields), but
removes the use of struct-literals to make the code compatible with the
upcoming changes in the image-spec module.

(similar to commit 3cb933db9d)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-10 12:25:17 +02:00
Sebastiaan van Stijn
2570784169
[20.10] vendor: github.com/moby/buildkit 3a1eeca59a9263613d996ead67d53a4b7d45723d (v0.8 branch)
- update to go1.18
- metadata: hold lock on storageitem update
- cache: avoid concurrent maps write on prune
- update containerd to latest of docker-20.10 branch

full diff: bc07b2b81b...3a1eeca59a

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-09 13:11:08 +02:00
Sebastiaan van Stijn
e42327a6d3
Merge pull request #44120 from thaJeztah/20.10_backport_update_tests
[20.10 backport] Update some tests for supplementary group permissions
2022-09-09 00:50:10 +02:00
Sebastiaan van Stijn
fcd4df906b
Update some tests for supplementary group permissions
Update tests checking for groups to adjust for new policy updated in
de7af816e7, which caused those tests
to fail:

    === FAIL: amd64.integration-cli TestDockerSwarmSuite/TestSwarmServiceWithGroup (1.94s)
    docker_cli_swarm_test.go:311: assertion failed: uid=0(root) gid=0(root) groups=0(root),10(wheel),29(audio),50(staff),777 (string) != uid=0(root) gid=0(root) groups=10(wheel),29(audio),50(staff),777
(string)
    --- FAIL: TestDockerSwarmSuite/TestSwarmServiceWithGroup (1.94s)

    === FAIL: amd64.integration-cli TestDockerCLIRunSuite/TestRunGroupAdd (0.41s)
    docker_cli_run_test.go:1091: expected output uid=0(root) gid=0(root) groups=10(wheel),29(audio),50(staff),777 received uid=0(root) gid=0(root) groups=0(root),10(wheel),29(audio),50(staff),777
    --- FAIL: TestDockerCLIRunSuite/TestRunGroupAdd (0.41s)

    === FAIL: amd64.integration-cli TestDockerCLIRunSuite/TestRunUserByIDZero (0.41s)
    docker_cli_run_test.go:790: expected daemon user got uid=0(root) gid=0(root) groups=0(root),10(wheel)
    --- FAIL: TestDockerCLIRunSuite/TestRunUserByIDZero (0.41s)

    === FAIL: amd64.integration-cli TestDockerCLIRunSuite (195.70s)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit c7e77dba7f)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-08 23:29:34 +02:00
Sebastiaan van Stijn
bb0197c829
Merge pull request from GHSA-rc4r-wh2q-q6c4
[20.10 backport] AdditionalGids must include effective group ID
2022-09-08 21:29:35 +02:00
Brian Goff
d348775bd5
Merge pull request #44109 from rumpl/20.10-fix-local-context
[20.10 backport] fix local context
2022-09-08 11:11:32 -07:00
Djordje Lukic
6a0186b357 Wrap local calls to the content and lease service
The wrapper sets the default namespace in the context if none is
provided, this is needed because we are calling these services directly
and not trough GRPC that has an interceptor to set the default namespace
to all calls.

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
(cherry picked from commit 878906630b)
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2022-09-08 16:56:51 +02:00
Sebastiaan van Stijn
b73e9c2c8a
Merge pull request #44098 from thaJeztah/20.10_backport_swagger_updates
[20.10 backport] assorted swagger updates in documentation
2022-09-07 20:47:06 +02:00
Samuel Karp
4855c28780
Merge pull request #44101 from thaJeztah/20.10_backport_bump_golang_1.18.6 2022-09-06 20:45:51 -07:00
Sebastiaan van Stijn
3d4616f943
Update to go 1.18.6 to address CVE-2022-27664, CVE-2022-32190
From the mailing list:

We have just released Go versions 1.19.1 and 1.18.6, minor point releases.
These minor releases include 2 security fixes following the security policy:

- net/http: handle server errors after sending GOAWAY
  A closing HTTP/2 server connection could hang forever waiting for a clean
  shutdown that was preempted by a subsequent fatal error. This failure mode
  could be exploited to cause a denial of service.

  Thanks to Bahruz Jabiyev, Tommaso Innocenti, Anthony Gavazzi, Steven Sprecher,
  and Kaan Onarlioglu for reporting this.

  This is CVE-2022-27664 and Go issue https://go.dev/issue/54658.

- net/url: JoinPath does not strip relative path components in all circumstances
  JoinPath and URL.JoinPath would not remove `../` path components appended to a
  relative path. For example, `JoinPath("https://go.dev", "../go")` returned the
  URL `https://go.dev/../go`, despite the JoinPath documentation stating that
  `../` path elements are cleaned from the result.

  Thanks to q0jt for reporting this issue.

  This is CVE-2022-32190 and Go issue https://go.dev/issue/54385.

Release notes:

go1.18.6 (released 2022-09-06) includes security fixes to the net/http package,
as well as bug fixes to the compiler, the go command, the pprof command, the
runtime, and the crypto/tls, encoding/xml, and net packages. See the Go 1.18.6
milestone on the issue tracker for details;

https://github.com/golang/go/issues?q=milestone%3AGo1.18.6+label%3ACherryPickApproved

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit cba36a064d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-06 22:25:44 +02:00
Sebastiaan van Stijn
d56101e0d2
Merge pull request #44093 from thaJeztah/20.10_backport_remove_tereshkova
[20.10] cleanup namesgenerator
2022-09-06 21:57:49 +02:00
Niel Drummond
23c7d84b84
docs: api: adjust ContainerWaitResponse error as optional
Signed-off-by: Niel Drummond <niel@drummond.lu>
(cherry picked from commit 2359da802b)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-06 17:07:53 +02:00
Sebastiaan van Stijn
3e9e79d134
docs: api: document ImageSummary fields (api v1.39-v1.41)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit b9de761c26)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-06 17:07:22 +02:00
Sebastiaan van Stijn
fdd438ae03
api: docs: improve documentation of ContainerConfig type (API v1.30-v1.41)
ContainerConfig is used in multiple locations (for example, both for
Image.Config and Image.ContainerConfig). Unfortunately, swagger does
not allow documenting individual uses if a type is used; for this type,
the content is _optional_ when used as Image.ContainerConfig (which is
set by the classic builder, which does a "commit" of a container, but
not used when building an image with BuildKit).

This patch attempts to address this confusion by documenting that
"it may be empty (or fields not propagated) if it's used for the
Image.ContainerConfig field".

Perhaps alternatives are possible (aliasing the type?) but we can
look at those in a follow-up.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 07dba5d9fe)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-06 17:06:55 +02:00
Sebastiaan van Stijn
97014a8db5
namesgenerator: remove Valentina Tereshkova
While the name generator has been frozen for new additions in 624b3cfbe8,
this person has become controversial. Our intent is for this list to be inclusive
and non-controversial.

This patch removes the name from the list.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 0f052eb4f5)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-06 13:57:50 +02:00