Commit graph

48329 commits

Author SHA1 Message Date
Sebastiaan van Stijn
cd1709b0d4
update containerd binary to v1.7.12
- full diff: https://github.com/containerd/containerd/compare/v1.7.11...v1.7.12
- release notes: https://github.com/containerd/containerd/releases/tag/v1.7.12

Welcome to the v1.7.12 release of containerd!

The twelfth patch release for containerd 1.7 contains various fixes and updates.

Notable Updates

- Fix on dialer function for Windows
- Improve `/etc/group` handling when appending groups
- Update shim pidfile permissions to 0644
- Update runc binary to v1.1.11
- Allow import and export to reference missing content
- Remove runc import
- Update Go version to 1.20.13

Deprecation Warnings

- Emit deprecation warning for `containerd.io/restart.logpath` label usage

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-12 17:56:37 +01:00
Sebastiaan van Stijn
ac6c791880
Merge pull request #47066 from thaJeztah/bump_compose
Dockerfile: update docker compose to v2.24.0
2024-01-12 16:12:58 +01:00
Sebastiaan van Stijn
a9d0128bd7
Dockerfile: update docker compose to v2.24.0
Update the version of compose used in CI to the latest version.

- full diff: https://github.com/docker/compose/compare/v2.20.1...v2.24.0
- release notes: https://github.com/docker/compose/releases/tag/v2.24.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-12 14:43:40 +01:00
Sebastiaan van Stijn
92cc0d6257
Merge pull request #47060 from vvoland/c8d-builder-commit-shareimagecreate
c8d/commit-builder: Extract common code
2024-01-11 14:37:20 +01:00
Paweł Gronowski
28f48ce1ac
c8d/commit-builder: Extract common code
Extract duplicated image creation code to a function.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-11 13:30:40 +01:00
Sebastiaan van Stijn
ef1a85ca5e
Merge pull request #47052 from corhere/libc8d/create-unstarted-tasks
libcontainerd: create unstarted tasks
2024-01-10 20:41:53 +01:00
Cory Snider
659d7b190f libcontainerd: create unstarted tasks
Split task creation and start into two separate method calls in the
libcontainerd API. Clients now have the opportunity to inspect the
freshly-created task and customize its runtime environment before
starting execution of the user-specified binary.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-01-10 13:50:26 -05:00
Sebastiaan van Stijn
7af5682b3c
Merge pull request #47053 from thaJeztah/update_golang_1.21.6
update to go1.21.6
2024-01-10 17:51:03 +01:00
Sebastiaan van Stijn
0fb6ba4ccc
Merge pull request #47048 from thaJeztah/fix_version_output
daemon/config: setPlatformDefaults: use debug for missing userland-proxy
2024-01-10 17:15:08 +01:00
Sebastiaan van Stijn
d7141cfd06
update to go1.21.6
go1.21.6 (released 2024-01-09) includes fixes to the compiler, the runtime, and
the crypto/tls, maps, and runtime/pprof packages. See the Go 1.21.6 milestone on
our issue tracker for details:

- https://github.com/golang/go/issues?q=milestone%3AGo1.21.6+label%3ACherryPickApproved
- full diff: https://github.com/golang/go/compare/go1.21.5...go1.21.6

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-10 09:46:37 +01:00
Sebastiaan van Stijn
afc7e581e6
Merge pull request #47049 from vvoland/c8d-fix-labels-check-filter
c8d/getImageLabelByDigest: Fix misspelled `labels` check
2024-01-09 15:55:57 +01:00
Sebastiaan van Stijn
72303ef11a
Merge pull request #47050 from thaJeztah/rob_curator
Add Rob Murray (robmry) to curators
2024-01-09 15:40:54 +01:00
Sebastiaan van Stijn
4aa1bea04a
Merge pull request #47046 from thaJeztah/bump_rootlesskit
Dockerfile: update rootlesskit to v1.1.1, and use tags as reference
2024-01-09 13:28:37 +01:00
Sebastiaan van Stijn
71c9aec834
Add Rob Murray (robmry) to curators
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-09 13:20:44 +01:00
Sebastiaan van Stijn
e8e20c0897
daemon/config: setPlatformDefaults: use debug for missing userland-proxy
commit 4f9db655ed moved looking up the
userland-proxy binary to early in the startup process, and introduced
a log-message if the binary was missing.

However, a side-effect of this was this message would also be printed
when running "--version";

    dockerd --version
    time="2024-01-09T09:18:53.705271292Z" level=warning msg="failed to lookup default userland-proxy binary" error="exec: \"docker-proxy\": executable file not found in $PATH"
    Docker version v25.0.0-rc.1, build 9cebefa717

We should look if we can avoid this, but let's change the message to be
a debug message as a short-term workaround.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-09 13:18:04 +01:00
Paweł Gronowski
18a0e144c2
c8d/getImageLabelByDigest: Fix misspelled labels check
It should be `labels.*` not `label.*`.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-09 13:16:56 +01:00
Sebastiaan van Stijn
e27ffdab0f
Dockerfile: update rootlesskit to v1.1.1, and use tags as reference
Commit 0b1c1877c5 updated the version in
hack/dockerfile/install/rootlesskit.installer, but forgot to update the
version in Dockerfile.

Also updating both to use a tag, instead of commit. While it's good to pin by
an immutable reference, I think it's reasonably safe to use the tag, which is
easier to use, and what we do for other binaries, such as runc as well.

Full diff: https://github.com/rootless-containers/rootlesskit/compare/v1.1.0...v1.1.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-09 09:54:47 +01:00
Sebastiaan van Stijn
c41f94eda3
Merge pull request #47042 from thaJeztah/remove_tarsplit_deadcode
layer: ChecksumForGraphID: remove unused code for for migrating v1 layers
2024-01-08 22:40:14 +01:00
Sebastiaan van Stijn
22b08a10ff
Merge pull request #47032 from thaJeztah/otel_align_versions
vendor: otelhttptrace v0.45.0 to align with other OTEL packages
2024-01-08 17:07:16 +01:00
Sebastiaan van Stijn
97d91aff7b
Merge pull request #47027 from thaJeztah/bump_cgroups
vendor: github.com/containerd/cgroups/v3 v3.0.3
2024-01-08 13:09:33 +01:00
Sebastiaan van Stijn
13225400c8
Merge pull request #47039 from thaJeztah/bump_tarsplit
vendor: github.com/vbatts/tar-split v0.11.5
2024-01-08 13:09:07 +01:00
Sebastiaan van Stijn
adba0c2b63
Merge pull request #46202 from thaJeztah/daemon_deferrrrr
daemon: Daemon.connectToNetwork, Daemon.ContainerRename: improve err-handling in defers
2024-01-08 12:40:45 +01:00
Sebastiaan van Stijn
782fe1fe82
layer: ChecksumForGraphID: remove unused code for for migrating v1 layers
The only use is in `builder/builder-next/adapters/snapshot.EnsureLayer()`,
which always calls the function with an _empty_ `oldTarDataPath`;
7082aecd54/builder/builder-next/adapters/snapshot/layer.go (L81)

When called with an empty `oldTarDataPath`, this function was an alias for
`checksumForGraphIDNoTarsplit`, so let's make it that.

Note that this code was added in 500e77bad0, as
part of the migration from "v1" images to "v2" (content-addressable) images.

Given that the remaining code lives in a "migration" file, possibly more code
can be removed.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-08 12:28:41 +01:00
Sebastiaan van Stijn
1dd415df1a
vendor: github.com/vbatts/tar-split v0.11.5
full diff: https://github.com/vbatts/tar-split/compare/v0.11.3...v0.11.5

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-08 11:40:15 +01:00
Sebastiaan van Stijn
7082aecd54
Merge pull request #47030 from knight42/feat/check-wrapped-err
Make errdefs.Is<SomeError> helper functions work with wrapped errors
2024-01-08 09:40:32 +01:00
Sebastiaan van Stijn
e6f46e637c
Merge pull request #43697 from thaJeztah/builder_remove_removecontainer
Builder: some improvements on cleaning up containers
2024-01-08 09:37:58 +01:00
Sebastiaan van Stijn
b7042cb767
Merge pull request #47031 from thaJeztah/go_connection_0.5.0
vendor: github.com/docker/go-connections v0.5.0
2024-01-06 11:12:34 +01:00
Sebastiaan van Stijn
372cd16b88
vendor: otelhttptrace v0.45.0 to align with other OTEL packages
This package / module is used by BuildKit's detect package; the buildkit
version we use (v0.12.x) [is using OTEL v0.40.0][1], whereas Moby and Containerd
moved to v0.45.0.

Go module [minimum-version-selection (MVS)][1] will pick the minimum required
version, and because there's no "reverse" dependency from the "main" OTEL modules
to this module, it selects the lowest version (specified in the BuildKit module).

Usually this would be a _good_ thing, but we know that OTEL likes all versions
to be aligned, so let's make it so.

full diff: https://github.com/open-telemetry/opentelemetry-go-contrib/compare/instrumentation/net/http/httptrace/otelhttptrace/v0.40.0...instrumentation/net/http/httptrace/otelhttptrace/v0.45.0

[1]: https://github.com/moby/buildkit/blob/v0.12.4-2-g3b6880d2a/go.mod#L73-L83
[2]: https://go.dev/ref/mod#glos-minimal-version-selection

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-06 11:04:13 +01:00
Sebastiaan van Stijn
22597efa06
vendor: github.com/docker/go-connections v0.5.0
no diff, as the tag is the same commit as we used already;
https://github.com/docker/go-connections/compare/fa09c952e3ea...v0.5.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-05 18:20:12 +01:00
Sebastiaan van Stijn
1f9098e7d0
builder: remove redundant ExecBackend.ContainerKill()
The `ExecBackend.ContainerKill()` function was called before removing a build-
container.

This function is backed by `daemon.ContainerKill()` which, if no signal is passed,
performed a `daemon.Kill()`, using `SIGKILL` as signal. However, the
`ExecBackend.ContainerRm()` (backed by `daemonContainerRm()`), which is called
after this, is executed with the `ForceRemove` option set, which calls
`daemon.cleanupContainer()` with `ForceRemove` set, which also results in
`daemon.Kill()` being called:
1a0c15abbb/daemon/delete.go (L84-L95)

This makes the `ExecBackend.ContainerKill()` redundant, so removing this from
the interface.

While looking at this code, one (possible) race-condition was found in
`daemon.cleanupContainer()`, where `daemon.Kill()` could return a `errdefs.Conflict`
if the container was already stopped. An extra check was added for this case to
prevent `daemon.cleanupContainer()` from terminating early.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-05 17:36:41 +01:00
Sebastiaan van Stijn
0aa96c5512
builder/dockerfile: containerManager.RemoveAll() prevent partial cleanup
Prevent cleanup from terminating early when failing to remove a container;

- continue trying to remove remaining containers
- ignore errors due to containers that were not found

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-05 17:34:34 +01:00
Sebastiaan van Stijn
03a8188a9a
builder/dockerfile: remove containerManager.removeContainer()
This was just a very thin wrapper for backend.ContainerRm(), and the
error it returned was not handled, so moving this code inline.

Moving it inline also allows differentiating the error message to
distinguish the "removing all intermediate containers" from "removing container"
(when cancelling a build).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-05 17:32:54 +01:00
Sebastiaan van Stijn
6b1baf8dd2
Merge pull request #47028 from vvoland/ci-mobybin-strip-tag-v
hack/make: Strip "v" tag prefix name from VERSION
2024-01-05 16:18:08 +01:00
Sebastiaan van Stijn
a38b5d7430
daemon: Daemon.ContainerRename: move vars closer to where they're used
Also break-up some "if" statements that were hiding that they were updating
existing variables.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-05 16:00:47 +01:00
Sebastiaan van Stijn
00a863b25e
daemon: Daemon.connectToNetwork: rename named return to prevent shadowing
The output var was used in a `defer`, but named `err` and shadowed in various
places. Rename the var to a more explicit name to make clear where it's used
and to prevent it being accidentally shadowed.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-05 15:56:17 +01:00
Jian Zeng
0f337e9c14
feat: make errdefs.IsXXX helper functions work with wrapped errors
Signed-off-by: Jian Zeng <anonymousknight96@gmail.com>
2024-01-05 22:54:13 +08:00
Paweł Gronowski
dd41f86339
hack/make: Refactor VERSION normalization
Turn the if-else tree into a `case`.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-05 13:21:13 +01:00
Sebastiaan van Stijn
2af17cd3f5
vendor: github.com/containerd/cgroups/v3 v3.0.3
release notes:

- NewSystemd handles UnitExists when starting units
- makefile fixes
- cgroups2: export memory max usage and swap max usage
- build(deps): bump github.com/cilium/ebpf from v0.9.1 to v0.11.0
- support psi
- feat: add Threads for cgroupv2
- Linux.Swap is defined as memory+swap combined, while in cgroup v2 swap is a separate value
- fix(): support re-enabling oom killer refs #307 by @kestrelcjx

full diff: https://github.com/containerd/cgroups/compare/v3.0.2...v3.0.3

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-05 13:18:14 +01:00
Paweł Gronowski
9a5393d500
hack/make: Strip "v" tag prefix name from VERSION
To make the version format in the `moby-bin` consistent with the
version we use in the release pipeline.

```diff
Server: Docker Engine - Community
 Engine:
-  Version:          v25.0.0
+  Version:          25.0.0
...
```

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-05 13:17:47 +01:00
Sebastiaan van Stijn
12128189eb
vendor: github.com/cilium/ebpf v0.11.0
full diff: https://github.com/cilium/ebpf/compare/v0.9.1...v0.11.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-05 13:14:49 +01:00
Sebastiaan van Stijn
c3c1ee0150
Merge pull request #47024 from thaJeztah/fix_daemon_side_digest_resolve
daemon/cluster: Cluster.imageWithDigestString: include mirrors to resolve digest
2024-01-05 12:13:35 +01:00
Sebastiaan van Stijn
07d2ad30e5
daemon/cluster: Cluster.imageWithDigestString: include mirrors to resolve digest
If the daemon is configured to use a mirror for the default (Docker Hub)
registry, the endpoint did not fall back to querying the upstream if the mirror
did not contain the given reference.

For pull-through registry-mirrors, this was not a problem, as in that case the
registry would forward the request, but for other mirrors, no fallback would
happen. This was inconsistent with how "pulling" images handled this situation;
when pulling images, both the mirror and upstream would be tried.

This patch brings the daemon-side lookup of image-manifests on-par with the
client-side lookup (the GET /distribution endpoint) as used in API 1.30 and
higher.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-04 16:53:45 +01:00
Sebastiaan van Stijn
9cebefa717
Merge pull request #47023 from thaJeztah/fix_distribution_mirrors
api: fix "GET /distribution" endpoint ignoring mirrors
2024-01-04 16:38:22 +01:00
Sebastiaan van Stijn
8aacbb3ba9
api: fix "GET /distribution" endpoint ignoring mirrors
If the daemon is configured to use a mirror for the default (Docker Hub)
registry, the endpoint did not fall back to querying the upstream if the mirror
did not contain the given reference.

If the daemon is configured to use a mirror for the default (Docker Hub)
registry, did not fall back to querying the upstream if the mirror did not
contain the given reference.

For pull-through registry-mirrors, this was not a problem, as in that case the
registry would forward the request, but for other mirrors, no fallback would
happen. This was inconsistent with how "pulling" images handled this situation;
when pulling images, both the mirror and upstream would be tried.

This problem was caused by the logic used in GetRepository, which had an
optimization to only return the first registry it was successfully able to
configure (and connect to), with the assumption that the mirror either contained
all images used, or to be configured as a pull-through mirror.

This patch:

- Introduces a GetRepositories method, which returns all candidates (both
  mirror(s) and upstream).
- Updates the endpoint to try all

Before this patch:

    # the daemon is configured to use a mirror for Docker Hub
    cat /etc/docker/daemon.json
    { "registry-mirrors": ["http://localhost:5000"]}

    # start the mirror (empty registry, not configured as pull-through mirror)
    docker run -d --name registry -p 127.0.0.1:5000:5000 registry:2

    # querying the endpoint fails, because the image-manifest is not found in the mirror:
    curl -s --unix-socket /var/run/docker.sock http://localhost/v1.43/distribution/docker.io/library/hello-world:latest/json
    {
      "message": "manifest unknown: manifest unknown"
    }

With this patch applied:

    # the daemon is configured to use a mirror for Docker Hub
    cat /etc/docker/daemon.json
    { "registry-mirrors": ["http://localhost:5000"]}

    # start the mirror (empty registry, not configured as pull-through mirror)
    docker run -d --name registry -p 127.0.0.1:5000:5000 registry:2

    # querying the endpoint succeeds (manifest is fetched from the upstream Docker Hub registry):
    curl -s --unix-socket /var/run/docker.sock http://localhost/v1.43/distribution/docker.io/library/hello-world:latest/json | jq .
    {
      "Descriptor": {
        "mediaType": "application/vnd.oci.image.index.v1+json",
        "digest": "sha256:1b9844d846ce3a6a6af7013e999a373112c3c0450aca49e155ae444526a2c45e",
        "size": 3849
      },
      "Platforms": [
        {
          "architecture": "amd64",
          "os": "linux"
        }
      ]
    }

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-04 15:46:32 +01:00
Sebastiaan van Stijn
e9042385ac
Merge pull request #47022 from akerouanton/libnet-bridge-dead-check-on-stale-default-nw
libnet/d/bridge: dead code: no conflict on stale default nw
2024-01-04 14:39:14 +01:00
Albin Kerouanton
b9e27acabc
libnet/d/bridge: dead code: no conflict on stale default nw
A check was added to the bridge driver to detect when it was called to
create the default bridge nw whereas a stale default bridge already
existed. In such case, the bridge driver was deleting the stale network
before re-creating it. This check was introduced in docker/libnetwork@6b158eac6a
to fix an issue related to newly introduced live-restore.

However, since commit docker/docker@ecffb6d58c,
the daemon doesn't even try to create default networks if there're
active sandboxes (ie. due to live-restore).

Thus, now it's impossible for the default bridge network to be stale and
to exists when the driver's CreateNetwork() method is called. As such,
the check introduced in the first commit mentioned above is dead code
and can be safely removed.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-01-04 11:50:04 +01:00
Albin Kerouanton
0a26cdf344
libnet/d/bridge: remove dead ActiveEndpointsError
This error is unused since docker/libnetwork@6b158eac6.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-01-04 11:12:53 +01:00
Brian Goff
12c7411b6b
Merge pull request #46750 from thaJeztah/daemon_start_log 2024-01-03 17:19:59 -08:00
Brian Goff
e2b78f074b
Merge pull request #47020 from thaJeztah/fix_go_compilerbug
daemon: work around go1.21 compiler bug
2024-01-03 16:10:00 -08:00
Sebastiaan van Stijn
cc6d826fdd
Merge pull request #46971 from thaJeztah/api_docs
docs/api: add documentation for API v1.44
2024-01-04 00:39:05 +01:00