Commit graph

48927 commits

Author SHA1 Message Date
Rob Murray
8cdcc4f3c7 Move dummy DNS server to integration/internal/network
Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-04-10 11:48:58 +01:00
Paweł Gronowski
ed752f6544
Merge pull request #47701 from vvoland/v26.0-47691
[26.0 backport] vendor: github.com/containerd/containerd v1.7.15
2024-04-09 14:15:36 +02:00
Paweł Gronowski
9db1b6f28c
Merge pull request #47702 from vvoland/v26.0-47647
[26.0 backport] github/ci: Check if backport is opened against the expected branch
2024-04-09 13:50:28 +02:00
Sebastiaan van Stijn
6261281247
Merge pull request #47700 from vvoland/v26.0-47673
[26.0 backport] vendor: golang.org/x/net v0.23.0
2024-04-09 13:33:11 +02:00
Sebastiaan van Stijn
90355e5166
Merge pull request #47696 from vvoland/v26.0-47658
[26.0 backport] Fix cases where we are wrapping a nil error
2024-04-09 13:29:51 +02:00
Paweł Gronowski
72615b19db
github/ci: Check if backport is opened against the expected branch
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 61269e718f)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-04-09 12:13:00 +02:00
Paweł Gronowski
23e7919e0a
Merge pull request #47694 from cpuguy83/26_oci_tar_no_platform
[26.0] save: Remove platform from config descriptor
2024-04-09 12:11:11 +02:00
Paweł Gronowski
c943936458
vendor: github.com/containerd/containerd v1.7.15
full diff: https://github.com/containerd/containerd/compare/v1.7.14...v1.7.15

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 5ae5969739)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-04-09 12:10:09 +02:00
Sebastiaan van Stijn
8b7940f037
vendor: golang.org/x/net v0.23.0
full diff: https://github.com/golang/net/compare/v0.22.0...v0.23.0

Includes a fix for CVE-2023-45288, which is also addressed in go1.22.2
and go1.21.9;

> http2: close connections when receiving too many headers
>
> Maintaining HPACK state requires that we parse and process
> all HEADERS and CONTINUATION frames on a connection.
> When a request's headers exceed MaxHeaderBytes, we don't
> allocate memory to store the excess headers but we do
> parse them. This permits an attacker to cause an HTTP/2
> endpoint to read arbitrary amounts of data, all associated
> with a request which is going to be rejected.
>
> Set a limit on the amount of excess header frames we
> will process before closing a connection.
>
> Thanks to Bartek Nowotarski for reporting this issue.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit d66589496e)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-04-09 11:25:28 +02:00
Sebastiaan van Stijn
55207ea637
vendor: golang.org/x/net v0.22.0, golang.org/x/crypto v0.21.0
full diffs changes relevant to vendored code:

- https://github.com/golang/net/compare/v0.18.0...v0.22.0
    - websocket: add support for dialing with context
    - http2: remove suspicious uint32->v conversion in frame code
    - http2: send an error of FLOW_CONTROL_ERROR when exceed the maximum octets
- https://github.com/golang/crypto/compare/v0.17.0...v0.21.0
    - internal/poly1305: drop Go 1.12 compatibility
    - internal/poly1305: improve sum_ppc64le.s
    - ocsp: don't use iota for externally defined constants

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit e1ca74361b)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-04-09 11:25:24 +02:00
Brian Goff
a7fa5e1deb
Fix cases where we are wrapping a nil error
This was using `errors.Wrap` when there was no error to wrap, meanwhile
we are supposed to be creating a new error.

Found this while investigating some log corruption issues and
unexpectedly getting a nil reader and a nil error from `getTailReader`.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit 0a48d26fbc)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-04-09 10:19:58 +02:00
Sebastiaan van Stijn
8730cccee2
Merge pull request #47692 from vvoland/v26.0-47689
[26.0 backport] update containerd binary to v1.7.15
2024-04-09 09:42:11 +02:00
Brian Goff
61d547bf00 save: Remove platform from config descriptor
This was brought up by bmitch that its not expected to have a platform
object in the config descriptor.
Also checked with tianon who agreed, its not _wrong_ but is unexpected
and doesn't neccessarily make sense to have it there.

Also, while technically incorrect, ECR is throwing an error when it sees
this.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit 9160b9fda6)
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2024-04-08 17:11:50 +00:00
Sebastiaan van Stijn
0c9ff4ca23
Merge pull request #47690 from vvoland/v26.0-47682
[26.0 backport] ci/validate-pr: Use `::error::` command to print errors
2024-04-08 19:07:59 +02:00
Paweł Gronowski
46ca4a74b4
update containerd binary to v1.7.15
Update the containerd binary that's used in CI

- full diff: https://github.com/containerd/containerd/compare/v1.7.13...v1.7.15
- release notes: https://github.com/containerd/containerd/releases/tag/v1.7.15

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 3485cfbb1e)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-04-08 18:09:01 +02:00
Paweł Gronowski
dea47c0810
ci/validate-pr: Use ::error:: command to print errors
This will make Github render the log line as an error.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit fb92caf2aa)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-04-08 14:21:21 +02:00
Sebastiaan van Stijn
f3842ab533
Merge pull request #47671 from vvoland/v26.0-47670
[26.0 backport] update to go1.21.9
2024-04-04 14:30:13 +02:00
Paweł Gronowski
e0815819de
update to go1.21.9
go1.21.9 (released 2024-04-03) includes a security fix to the net/http
package, as well as bug fixes to the linker, and the go/types and
net/http packages. See the [Go 1.21.9 milestone](https://github.com/golang/go/issues?q=milestone%3AGo1.21.9+label%3ACherryPickApproved)
for more details.

These minor releases include 1 security fixes following the security policy:

- http2: close connections when receiving too many headers

Maintaining HPACK state requires that we parse and process all HEADERS
and CONTINUATION frames on a connection. When a request's headers exceed
MaxHeaderBytes, we don't allocate memory to store the excess headers but
we do parse them. This permits an attacker to cause an HTTP/2 endpoint
to read arbitrary amounts of header data, all associated with a request
which is going to be rejected. These headers can include Huffman-encoded
data which is significantly more expensive for the receiver to decode
than for an attacker to send.

Set a limit on the amount of excess header frames we will process before
closing a connection.

Thanks to Bartek Nowotarski (https://nowotarski.info/) for reporting this issue.

This is CVE-2023-45288 and Go issue https://go.dev/issue/65051.

View the release notes for more information:
https://go.dev/doc/devel/release#go1.22.2

- https://github.com/golang/go/issues?q=milestone%3AGo1.21.9+label%3ACherryPickApproved
- full diff: https://github.com/golang/go/compare/go1.21.8...go1.21.9

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 329d403e20)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-04-04 10:15:00 +02:00
Paweł Gronowski
07c797281e
Merge pull request #47637 from vvoland/v26.0-47610
[26.0 backport] Dockerfile: update docker CLI to v26.0.0
2024-04-02 10:39:01 +02:00
Albin Kerouanton
f2550b3c09
Merge pull request #47646 from vvoland/v26.0-47621
[26.0 backport] Restore the SetKey prestart hook.
2024-03-28 10:15:52 +00:00
Rob Murray
fc14d8f932
Restore the SetKey prestart hook.
Partially reverts 0046b16 "daemon: set libnetwork sandbox key w/o OCI hook"

Running SetKey to store the OCI Sandbox key after task creation, rather
than from the OCI prestart hook, meant it happened after sysctl settings
were applied by the runtime - which was the intention, we wanted to
complete Sandbox configuration after IPv6 had been disabled by a sysctl
if that was going to happen.

But, it meant '--sysctl' options for a specfic network interface caused
container task creation to fail, because the interface is only moved into
the network namespace during SetKey.

This change restores the SetKey prestart hook, and regenerates config
files that depend on the container's support for IPv6 after the task has
been created. It also adds a regression test that makes sure it's possible
to set an interface-specfic sysctl.

Signed-off-by: Rob Murray <rob.murray@docker.com>
(cherry picked from commit fde80fe2e7)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-28 10:05:08 +01:00
Paweł Gronowski
c035ef2283
Merge pull request #47638 from vvoland/v26.0-47636
[26.0 backport] ci: update workflow artifacts retention
2024-03-27 16:58:28 +01:00
CrazyMax
703f14793e
ci: update workflow artifacts retention
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
(cherry picked from commit aff003139c)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-27 13:21:09 +01:00
Paweł Gronowski
30e94dadbc
Dockerfile: update docker CLI to v26.0.0
Update the CLI that's used in the dev-container

- full diff: https://github.com/docker/cli/compare/v26.0.0-rc2...v26.0.0

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit ea72f9f72c)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-27 13:17:05 +01:00
Bjorn Neergaard
8b79278316
Merge pull request #47599 from neersighted/short_id_aliases_removal
api: document changed behavior of the `Aliases` field in v1.45
2024-03-20 08:33:39 -06:00
Bjorn Neergaard
22726fb63b
api: document changed behavior of the Aliases field in v1.45
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2024-03-20 08:23:48 -06:00
Bjorn Neergaard
963e1f3eed
Merge pull request #47597 from vvoland/c8d-list-fix-shared-size
c8d/list: Fix shared size calculation
2024-03-20 07:26:09 -06:00
Paweł Gronowski
3312b82515
c8d/list: Add a test case for images sharing a top layer
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-20 13:17:56 +01:00
Paweł Gronowski
ad8a5a5732
c8d/list: Fix diffIDs being outputted instead of chainIDs
The `identity.ChainIDs` call was accidentally removed in
b37ced2551.

This broke the shared size calculation for images with more than one
layer that were sharing the same compressed layer.

This was could be reproduced with:
```
$ docker pull docker.io/docker/desktop-kubernetes-coredns:v1.11.1
$ docker pull docker.io/docker/desktop-kubernetes-etcd:3.5.10-0
$ docker system df
```

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-20 11:17:50 +01:00
Paweł Gronowski
0c2d83b5fb
c8d/list: Handle unpacked layers when calculating shared size
After a535a65c4b the size reported by the
image list was changed to include all platforms of that image.

This made the "shared size" calculation consider all diff ids of all the
platforms available in the image which caused "snapshot not found"
errors when multiple images were sharing the same layer which wasn't
unpacked.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-20 11:17:28 +01:00
Paweł Gronowski
330d777c53
Merge pull request #47591 from vvoland/api-1.45
docs/api: add documentation for API v1.45
2024-03-19 14:27:45 +01:00
Paweł Gronowski
3d2a56e7cf
docs/api: add documentation for API v1.45
Copy the swagger / OpenAPI file to the documentation. This is the API
version used by the upcoming v26.0.0 release.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-19 13:37:05 +01:00
Paweł Gronowski
4531a371f2
Merge pull request #47580 from vvoland/c8d-list-slow
c8d/list: Generate image summary concurrently
2024-03-19 13:32:52 +01:00
Paweł Gronowski
731a64069f
c8d/list: Generate image summary concurrently
Run `imageSummary` concurrently to avoid being IO blocked on the
containerd gRPC.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-19 09:38:58 +01:00
Paweł Gronowski
dade279565
c8d/list: Add Images benchmark
Benchmark the `Images` implementation (image list) against an image
store with 10, 100 and 1000 random images. Currently the images are
single-platform only.

The images are generated randomly, but a fixed seed is used so the
actual testing data will be the same across different executions.

Because the content store is not a real containerd image store but a
local implementation, a small delay (500us) is added to each content
store method call. This is to simulate a real-world usage where each
containerd client call requires a gRPC call.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-19 09:38:56 +01:00
Sebastiaan van Stijn
23e1af45c6
Merge pull request #47582 from vvoland/vendor-buildkit-0.13.1
vendor: github.com/moby/buildkit v0.13.1
2024-03-18 21:53:15 +01:00
Paweł Gronowski
e7c60a30e6
vendor: github.com/moby/buildkit v0.13.1
full diff: https://github.com/moby/buildkit/compare/v0.13.0...v0.13.1

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-18 20:16:09 +01:00
Bjorn Neergaard
641e341eed
Merge pull request #47538 from robmry/libnet-resolver-nxdomain
libnet: Don't forward to upstream resolvers on internal nw
2024-03-18 11:22:59 -06:00
Sebastiaan van Stijn
dd146571ea
Merge pull request #47568 from vvoland/c8d-list-fix
c8d/list: Fix premature `Images` return
2024-03-18 15:28:09 +01:00
Paweł Gronowski
fe70ee9477
Merge pull request #47577 from vvoland/c8d-list-labels-filter
c8d/list: Don't setup label filter if it's not specified
2024-03-18 15:13:40 +01:00
Sebastiaan van Stijn
307962dbd5
Merge pull request #47578 from thaJeztah/fix_resolvconf_go_version
resolvconf: add //go:build directives to prevent downgrading to go1.16 language
2024-03-18 14:00:03 +01:00
Sebastiaan van Stijn
7e56442cee
Merge pull request #47574 from thaJeztah/bump_tools
Dockerfile: update docker CLI to v26.0.0-rc2, docker compose v2.25.0
2024-03-18 13:59:42 +01:00
Sebastiaan van Stijn
ebf300c165
Merge pull request #47579 from vvoland/flaky-testdiskusage
integration: Remove Parallel from TestDiskUsage
2024-03-18 13:59:28 +01:00
Paweł Gronowski
2e4ebf032a
c8d/list: Pass ctx to setupLabelFilter
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-18 13:50:45 +01:00
Paweł Gronowski
153de36b3f
c8d/list: Add empty index test case
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-18 12:50:10 +01:00
Sebastiaan van Stijn
4ff655f4b8
resolvconf: add //go:build directives to prevent downgrading to go1.16 language
Commit 8921897e3b introduced the uses of `clear()`,
which requires go1.21, but Go is downgrading this file to go1.16 when used in
other projects (due to us not yet being a go module);

    0.175 + xx-go build '-gcflags=' -ldflags '-X github.com/moby/buildkit/version.Version=b53a13e -X github.com/moby/buildkit/version.Revision=b53a13e4f5c8d7e82716615e0f23656893df89af -X github.com/moby/buildkit/version.Package=github.com/moby/buildkit -extldflags '"'"'-static'"'" -tags 'osusergo netgo static_build seccomp ' -o /usr/bin/buildkitd ./cmd/buildkitd
    181.8 # github.com/docker/docker/libnetwork/internal/resolvconf
    181.8 vendor/github.com/docker/docker/libnetwork/internal/resolvconf/resolvconf.go:509:2: clear requires go1.21 or later (-lang was set to go1.16; check go.mod)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-03-18 12:28:21 +01:00
Paweł Gronowski
1c03312378
integration: Remove Parallel from TestDiskUsage
Check if removing the Parallel execution from that test fixes its
flakiness.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-18 12:11:34 +01:00
Paweł Gronowski
f512dba037
c8d/list: Fix premature Images return
52a80b40e2 extracted the `imageSummary`
function but introduced a bug causing the whole caller function to
return if the image should be skipped.

`imageSummary` returns a nil error and nil image when the image doesn't
have any platform or all its platforms are not available locally.
In this case that particular image should be skipped, instead of failing
the whole image list operation.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-18 10:43:12 +01:00
Paweł Gronowski
89dc2860ba
c8d/list: Handle missing configs in label filter
Don't error out the filter if an image config is missing.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-18 10:29:16 +01:00
Paweł Gronowski
6f3892dc99
c8d/list: Don't setup label filter if it's not specified
Don't run filter function which would only run through the images
reading theirs config without checking any label anyway.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-18 10:23:31 +01:00