Commit graph

48913 commits

Author SHA1 Message Date
Sebastiaan van Stijn
0c9ff4ca23
Merge pull request #47690 from vvoland/v26.0-47682
[26.0 backport] ci/validate-pr: Use `::error::` command to print errors
2024-04-08 19:07:59 +02:00
Paweł Gronowski
dea47c0810
ci/validate-pr: Use ::error:: command to print errors
This will make Github render the log line as an error.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit fb92caf2aa)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-04-08 14:21:21 +02:00
Sebastiaan van Stijn
f3842ab533
Merge pull request #47671 from vvoland/v26.0-47670
[26.0 backport] update to go1.21.9
2024-04-04 14:30:13 +02:00
Paweł Gronowski
e0815819de
update to go1.21.9
go1.21.9 (released 2024-04-03) includes a security fix to the net/http
package, as well as bug fixes to the linker, and the go/types and
net/http packages. See the [Go 1.21.9 milestone](https://github.com/golang/go/issues?q=milestone%3AGo1.21.9+label%3ACherryPickApproved)
for more details.

These minor releases include 1 security fixes following the security policy:

- http2: close connections when receiving too many headers

Maintaining HPACK state requires that we parse and process all HEADERS
and CONTINUATION frames on a connection. When a request's headers exceed
MaxHeaderBytes, we don't allocate memory to store the excess headers but
we do parse them. This permits an attacker to cause an HTTP/2 endpoint
to read arbitrary amounts of header data, all associated with a request
which is going to be rejected. These headers can include Huffman-encoded
data which is significantly more expensive for the receiver to decode
than for an attacker to send.

Set a limit on the amount of excess header frames we will process before
closing a connection.

Thanks to Bartek Nowotarski (https://nowotarski.info/) for reporting this issue.

This is CVE-2023-45288 and Go issue https://go.dev/issue/65051.

View the release notes for more information:
https://go.dev/doc/devel/release#go1.22.2

- https://github.com/golang/go/issues?q=milestone%3AGo1.21.9+label%3ACherryPickApproved
- full diff: https://github.com/golang/go/compare/go1.21.8...go1.21.9

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 329d403e20)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-04-04 10:15:00 +02:00
Paweł Gronowski
07c797281e
Merge pull request #47637 from vvoland/v26.0-47610
[26.0 backport] Dockerfile: update docker CLI to v26.0.0
2024-04-02 10:39:01 +02:00
Albin Kerouanton
f2550b3c09
Merge pull request #47646 from vvoland/v26.0-47621
[26.0 backport] Restore the SetKey prestart hook.
2024-03-28 10:15:52 +00:00
Rob Murray
fc14d8f932
Restore the SetKey prestart hook.
Partially reverts 0046b16 "daemon: set libnetwork sandbox key w/o OCI hook"

Running SetKey to store the OCI Sandbox key after task creation, rather
than from the OCI prestart hook, meant it happened after sysctl settings
were applied by the runtime - which was the intention, we wanted to
complete Sandbox configuration after IPv6 had been disabled by a sysctl
if that was going to happen.

But, it meant '--sysctl' options for a specfic network interface caused
container task creation to fail, because the interface is only moved into
the network namespace during SetKey.

This change restores the SetKey prestart hook, and regenerates config
files that depend on the container's support for IPv6 after the task has
been created. It also adds a regression test that makes sure it's possible
to set an interface-specfic sysctl.

Signed-off-by: Rob Murray <rob.murray@docker.com>
(cherry picked from commit fde80fe2e7)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-28 10:05:08 +01:00
Paweł Gronowski
c035ef2283
Merge pull request #47638 from vvoland/v26.0-47636
[26.0 backport] ci: update workflow artifacts retention
2024-03-27 16:58:28 +01:00
CrazyMax
703f14793e
ci: update workflow artifacts retention
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
(cherry picked from commit aff003139c)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-27 13:21:09 +01:00
Paweł Gronowski
30e94dadbc
Dockerfile: update docker CLI to v26.0.0
Update the CLI that's used in the dev-container

- full diff: https://github.com/docker/cli/compare/v26.0.0-rc2...v26.0.0

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit ea72f9f72c)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-27 13:17:05 +01:00
Bjorn Neergaard
8b79278316
Merge pull request #47599 from neersighted/short_id_aliases_removal
api: document changed behavior of the `Aliases` field in v1.45
2024-03-20 08:33:39 -06:00
Bjorn Neergaard
22726fb63b
api: document changed behavior of the Aliases field in v1.45
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2024-03-20 08:23:48 -06:00
Bjorn Neergaard
963e1f3eed
Merge pull request #47597 from vvoland/c8d-list-fix-shared-size
c8d/list: Fix shared size calculation
2024-03-20 07:26:09 -06:00
Paweł Gronowski
3312b82515
c8d/list: Add a test case for images sharing a top layer
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-20 13:17:56 +01:00
Paweł Gronowski
ad8a5a5732
c8d/list: Fix diffIDs being outputted instead of chainIDs
The `identity.ChainIDs` call was accidentally removed in
b37ced2551.

This broke the shared size calculation for images with more than one
layer that were sharing the same compressed layer.

This was could be reproduced with:
```
$ docker pull docker.io/docker/desktop-kubernetes-coredns:v1.11.1
$ docker pull docker.io/docker/desktop-kubernetes-etcd:3.5.10-0
$ docker system df
```

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-20 11:17:50 +01:00
Paweł Gronowski
0c2d83b5fb
c8d/list: Handle unpacked layers when calculating shared size
After a535a65c4b the size reported by the
image list was changed to include all platforms of that image.

This made the "shared size" calculation consider all diff ids of all the
platforms available in the image which caused "snapshot not found"
errors when multiple images were sharing the same layer which wasn't
unpacked.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-20 11:17:28 +01:00
Paweł Gronowski
330d777c53
Merge pull request #47591 from vvoland/api-1.45
docs/api: add documentation for API v1.45
2024-03-19 14:27:45 +01:00
Paweł Gronowski
3d2a56e7cf
docs/api: add documentation for API v1.45
Copy the swagger / OpenAPI file to the documentation. This is the API
version used by the upcoming v26.0.0 release.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-19 13:37:05 +01:00
Paweł Gronowski
4531a371f2
Merge pull request #47580 from vvoland/c8d-list-slow
c8d/list: Generate image summary concurrently
2024-03-19 13:32:52 +01:00
Paweł Gronowski
731a64069f
c8d/list: Generate image summary concurrently
Run `imageSummary` concurrently to avoid being IO blocked on the
containerd gRPC.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-19 09:38:58 +01:00
Paweł Gronowski
dade279565
c8d/list: Add Images benchmark
Benchmark the `Images` implementation (image list) against an image
store with 10, 100 and 1000 random images. Currently the images are
single-platform only.

The images are generated randomly, but a fixed seed is used so the
actual testing data will be the same across different executions.

Because the content store is not a real containerd image store but a
local implementation, a small delay (500us) is added to each content
store method call. This is to simulate a real-world usage where each
containerd client call requires a gRPC call.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-19 09:38:56 +01:00
Sebastiaan van Stijn
23e1af45c6
Merge pull request #47582 from vvoland/vendor-buildkit-0.13.1
vendor: github.com/moby/buildkit v0.13.1
2024-03-18 21:53:15 +01:00
Paweł Gronowski
e7c60a30e6
vendor: github.com/moby/buildkit v0.13.1
full diff: https://github.com/moby/buildkit/compare/v0.13.0...v0.13.1

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-18 20:16:09 +01:00
Bjorn Neergaard
641e341eed
Merge pull request #47538 from robmry/libnet-resolver-nxdomain
libnet: Don't forward to upstream resolvers on internal nw
2024-03-18 11:22:59 -06:00
Sebastiaan van Stijn
dd146571ea
Merge pull request #47568 from vvoland/c8d-list-fix
c8d/list: Fix premature `Images` return
2024-03-18 15:28:09 +01:00
Paweł Gronowski
fe70ee9477
Merge pull request #47577 from vvoland/c8d-list-labels-filter
c8d/list: Don't setup label filter if it's not specified
2024-03-18 15:13:40 +01:00
Sebastiaan van Stijn
307962dbd5
Merge pull request #47578 from thaJeztah/fix_resolvconf_go_version
resolvconf: add //go:build directives to prevent downgrading to go1.16 language
2024-03-18 14:00:03 +01:00
Sebastiaan van Stijn
7e56442cee
Merge pull request #47574 from thaJeztah/bump_tools
Dockerfile: update docker CLI to v26.0.0-rc2, docker compose v2.25.0
2024-03-18 13:59:42 +01:00
Sebastiaan van Stijn
ebf300c165
Merge pull request #47579 from vvoland/flaky-testdiskusage
integration: Remove Parallel from TestDiskUsage
2024-03-18 13:59:28 +01:00
Paweł Gronowski
2e4ebf032a
c8d/list: Pass ctx to setupLabelFilter
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-18 13:50:45 +01:00
Paweł Gronowski
153de36b3f
c8d/list: Add empty index test case
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-18 12:50:10 +01:00
Sebastiaan van Stijn
4ff655f4b8
resolvconf: add //go:build directives to prevent downgrading to go1.16 language
Commit 8921897e3b introduced the uses of `clear()`,
which requires go1.21, but Go is downgrading this file to go1.16 when used in
other projects (due to us not yet being a go module);

    0.175 + xx-go build '-gcflags=' -ldflags '-X github.com/moby/buildkit/version.Version=b53a13e -X github.com/moby/buildkit/version.Revision=b53a13e4f5c8d7e82716615e0f23656893df89af -X github.com/moby/buildkit/version.Package=github.com/moby/buildkit -extldflags '"'"'-static'"'" -tags 'osusergo netgo static_build seccomp ' -o /usr/bin/buildkitd ./cmd/buildkitd
    181.8 # github.com/docker/docker/libnetwork/internal/resolvconf
    181.8 vendor/github.com/docker/docker/libnetwork/internal/resolvconf/resolvconf.go:509:2: clear requires go1.21 or later (-lang was set to go1.16; check go.mod)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-03-18 12:28:21 +01:00
Paweł Gronowski
1c03312378
integration: Remove Parallel from TestDiskUsage
Check if removing the Parallel execution from that test fixes its
flakiness.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-18 12:11:34 +01:00
Paweł Gronowski
f512dba037
c8d/list: Fix premature Images return
52a80b40e2 extracted the `imageSummary`
function but introduced a bug causing the whole caller function to
return if the image should be skipped.

`imageSummary` returns a nil error and nil image when the image doesn't
have any platform or all its platforms are not available locally.
In this case that particular image should be skipped, instead of failing
the whole image list operation.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-18 10:43:12 +01:00
Paweł Gronowski
89dc2860ba
c8d/list: Handle missing configs in label filter
Don't error out the filter if an image config is missing.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-18 10:29:16 +01:00
Paweł Gronowski
6f3892dc99
c8d/list: Don't setup label filter if it's not specified
Don't run filter function which would only run through the images
reading theirs config without checking any label anyway.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-18 10:23:31 +01:00
Paweł Gronowski
a9bca45e92
Merge pull request #47575 from thaJeztah/bump_shfmt
Dockerfile: update mvdan/shfmt to v3.8.0
2024-03-18 09:26:35 +01:00
Sebastiaan van Stijn
fe8fb9b9a1
Dockerfile: update mvdan/shfmt to v3.8.0
- full diff: https://github.com/mvdan/sh/compare/v3.7.0...v3.8.0
- 3.7.0 release notes: https://github.com/mvdan/sh/releases/tag/v3.7.0
- 3.8.0 release notes: https://github.com/mvdan/sh/releases/tag/v3.8.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-03-17 13:36:43 +01:00
Sebastiaan van Stijn
70e46f2c7c
Merge pull request #47559 from AkihiroSuda/fix-47436
rootless: fix `open /etc/docker/plugins: permission denied`
2024-03-16 15:54:09 +01:00
Sebastiaan van Stijn
23339a6147
Merge pull request #47570 from thaJeztah/bump_xx_1.4
Dockerfile: update xx to v1.4.0
2024-03-16 15:53:49 +01:00
Sebastiaan van Stijn
4bd30829d1
Dockerfile: update docker compose to v2.25.0
Update the version of compose that's used in the dev-container.

- full diff: https://github.com/docker/compose/compare/v2.24.7...v2.25.0
- release notes: https://github.com/docker/compose/releases/tag/v2.25.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-03-16 14:22:42 +01:00
Sebastiaan van Stijn
971562b005
Dockerfile: update docker CLI to v26.0.0-rc2
Update the CLI that's used in the dev-container to the latest rc

- full diff: https://github.com/docker/cli/compare/v26.0.0-rc1...v26.0.0-rc2

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-03-16 14:22:30 +01:00
Akihiro Suda
d742659877
rootless: fix open /etc/docker/plugins: permission denied
Fix issue 47436

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2024-03-16 22:03:34 +09:00
Sebastiaan van Stijn
4f46c44725
Dockerfile: update xx to v1.4.0
full diff: https://github.com/tonistiigi/xx/compare/v1.2.1...v1.4.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-03-15 19:59:48 +01:00
Bjorn Neergaard
1f539a6e85
Merge pull request #47569 from thaJeztah/make_fix_empty_check
Makefile: generate-files: fix check for empty TMP_OUT
2024-03-15 12:07:00 -06:00
Bjorn Neergaard
959c2ee6cf
Merge pull request #47558 from AkihiroSuda/fix-47248
plugin: fix mounting /etc/hosts when running in UserNS
2024-03-15 12:06:48 -06:00
Sebastiaan van Stijn
25c9e6e8df
Makefile: generate-files: fix check for empty TMP_OUT
commit c655b7dc78 added a check to make sure
the TMP_OUT variable was not set to an empty value, as such a situation would
perform an `rm -rf /**` during cleanup.

However, it was a bit too eager, because Makefile conditionals (`ifeq`) are
evaluated when parsing the Makefile, which happens _before_ the make target
is executed.

As a result `$@_TMP_OUT` was always empty when the `ifeq` was evaluated,
making it not possible to execute the `generate-files` target.

This patch changes the check to use a shell command to evaluate if the var
is set to an empty value.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-03-15 17:54:54 +01:00
Akihiro Suda
762ec4b60c
plugin: fix mounting /etc/hosts when running in UserNS
Fix `error mounting "/etc/hosts" to rootfs at "/etc/hosts": mount
/etc/hosts:/etc/hosts (via /proc/self/fd/6), flags: 0x5021: operation
not permitted`.

This error was introduced in 7d08d84b03
(`dockerd-rootless.sh: set rootlesskit --state-dir=DIR`) that changed
the filesystem of the state dir from /tmp to /run (in a typical setup).

Fix issue 47248

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2024-03-15 22:16:34 +09:00
Sebastiaan van Stijn
979f03f9f6
Merge pull request #47567 from thaJeztah/move_rootless_mountopts
daemon: move getUnprivilegedMountFlags to internal package
2024-03-15 14:13:23 +01:00
Sebastiaan van Stijn
7b414f5703
daemon: move getUnprivilegedMountFlags to internal package
This code is currently only used in the daemon, but is also needed in other
places. We should consider moving this code to github.com/moby/sys, so that
BuildKit can also use the same implementation instead of maintaining a fork;
moving it to internal allows us to reuse this code inside the repository, but
does not allow external consumers to depend on it (which we don't want as
it's not a permanent location).

As our code only uses this in linux files, I did not add a stub for other
platforms (but we may decide to do that in the moby/sys repository).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-03-15 12:55:09 +01:00