Commit graph

48319 commits

Author SHA1 Message Date
Cory Snider
659d7b190f libcontainerd: create unstarted tasks
Split task creation and start into two separate method calls in the
libcontainerd API. Clients now have the opportunity to inspect the
freshly-created task and customize its runtime environment before
starting execution of the user-specified binary.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-01-10 13:50:26 -05:00
Sebastiaan van Stijn
afc7e581e6
Merge pull request #47049 from vvoland/c8d-fix-labels-check-filter
c8d/getImageLabelByDigest: Fix misspelled `labels` check
2024-01-09 15:55:57 +01:00
Sebastiaan van Stijn
72303ef11a
Merge pull request #47050 from thaJeztah/rob_curator
Add Rob Murray (robmry) to curators
2024-01-09 15:40:54 +01:00
Sebastiaan van Stijn
4aa1bea04a
Merge pull request #47046 from thaJeztah/bump_rootlesskit
Dockerfile: update rootlesskit to v1.1.1, and use tags as reference
2024-01-09 13:28:37 +01:00
Sebastiaan van Stijn
71c9aec834
Add Rob Murray (robmry) to curators
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-09 13:20:44 +01:00
Paweł Gronowski
18a0e144c2
c8d/getImageLabelByDigest: Fix misspelled labels check
It should be `labels.*` not `label.*`.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-09 13:16:56 +01:00
Sebastiaan van Stijn
e27ffdab0f
Dockerfile: update rootlesskit to v1.1.1, and use tags as reference
Commit 0b1c1877c5 updated the version in
hack/dockerfile/install/rootlesskit.installer, but forgot to update the
version in Dockerfile.

Also updating both to use a tag, instead of commit. While it's good to pin by
an immutable reference, I think it's reasonably safe to use the tag, which is
easier to use, and what we do for other binaries, such as runc as well.

Full diff: https://github.com/rootless-containers/rootlesskit/compare/v1.1.0...v1.1.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-09 09:54:47 +01:00
Sebastiaan van Stijn
c41f94eda3
Merge pull request #47042 from thaJeztah/remove_tarsplit_deadcode
layer: ChecksumForGraphID: remove unused code for for migrating v1 layers
2024-01-08 22:40:14 +01:00
Sebastiaan van Stijn
22b08a10ff
Merge pull request #47032 from thaJeztah/otel_align_versions
vendor: otelhttptrace v0.45.0 to align with other OTEL packages
2024-01-08 17:07:16 +01:00
Sebastiaan van Stijn
97d91aff7b
Merge pull request #47027 from thaJeztah/bump_cgroups
vendor: github.com/containerd/cgroups/v3 v3.0.3
2024-01-08 13:09:33 +01:00
Sebastiaan van Stijn
13225400c8
Merge pull request #47039 from thaJeztah/bump_tarsplit
vendor: github.com/vbatts/tar-split v0.11.5
2024-01-08 13:09:07 +01:00
Sebastiaan van Stijn
adba0c2b63
Merge pull request #46202 from thaJeztah/daemon_deferrrrr
daemon: Daemon.connectToNetwork, Daemon.ContainerRename: improve err-handling in defers
2024-01-08 12:40:45 +01:00
Sebastiaan van Stijn
782fe1fe82
layer: ChecksumForGraphID: remove unused code for for migrating v1 layers
The only use is in `builder/builder-next/adapters/snapshot.EnsureLayer()`,
which always calls the function with an _empty_ `oldTarDataPath`;
7082aecd54/builder/builder-next/adapters/snapshot/layer.go (L81)

When called with an empty `oldTarDataPath`, this function was an alias for
`checksumForGraphIDNoTarsplit`, so let's make it that.

Note that this code was added in 500e77bad0, as
part of the migration from "v1" images to "v2" (content-addressable) images.

Given that the remaining code lives in a "migration" file, possibly more code
can be removed.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-08 12:28:41 +01:00
Sebastiaan van Stijn
1dd415df1a
vendor: github.com/vbatts/tar-split v0.11.5
full diff: https://github.com/vbatts/tar-split/compare/v0.11.3...v0.11.5

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-08 11:40:15 +01:00
Sebastiaan van Stijn
7082aecd54
Merge pull request #47030 from knight42/feat/check-wrapped-err
Make errdefs.Is<SomeError> helper functions work with wrapped errors
2024-01-08 09:40:32 +01:00
Sebastiaan van Stijn
e6f46e637c
Merge pull request #43697 from thaJeztah/builder_remove_removecontainer
Builder: some improvements on cleaning up containers
2024-01-08 09:37:58 +01:00
Sebastiaan van Stijn
b7042cb767
Merge pull request #47031 from thaJeztah/go_connection_0.5.0
vendor: github.com/docker/go-connections v0.5.0
2024-01-06 11:12:34 +01:00
Sebastiaan van Stijn
372cd16b88
vendor: otelhttptrace v0.45.0 to align with other OTEL packages
This package / module is used by BuildKit's detect package; the buildkit
version we use (v0.12.x) [is using OTEL v0.40.0][1], whereas Moby and Containerd
moved to v0.45.0.

Go module [minimum-version-selection (MVS)][1] will pick the minimum required
version, and because there's no "reverse" dependency from the "main" OTEL modules
to this module, it selects the lowest version (specified in the BuildKit module).

Usually this would be a _good_ thing, but we know that OTEL likes all versions
to be aligned, so let's make it so.

full diff: https://github.com/open-telemetry/opentelemetry-go-contrib/compare/instrumentation/net/http/httptrace/otelhttptrace/v0.40.0...instrumentation/net/http/httptrace/otelhttptrace/v0.45.0

[1]: https://github.com/moby/buildkit/blob/v0.12.4-2-g3b6880d2a/go.mod#L73-L83
[2]: https://go.dev/ref/mod#glos-minimal-version-selection

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-06 11:04:13 +01:00
Sebastiaan van Stijn
22597efa06
vendor: github.com/docker/go-connections v0.5.0
no diff, as the tag is the same commit as we used already;
https://github.com/docker/go-connections/compare/fa09c952e3ea...v0.5.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-05 18:20:12 +01:00
Sebastiaan van Stijn
1f9098e7d0
builder: remove redundant ExecBackend.ContainerKill()
The `ExecBackend.ContainerKill()` function was called before removing a build-
container.

This function is backed by `daemon.ContainerKill()` which, if no signal is passed,
performed a `daemon.Kill()`, using `SIGKILL` as signal. However, the
`ExecBackend.ContainerRm()` (backed by `daemonContainerRm()`), which is called
after this, is executed with the `ForceRemove` option set, which calls
`daemon.cleanupContainer()` with `ForceRemove` set, which also results in
`daemon.Kill()` being called:
1a0c15abbb/daemon/delete.go (L84-L95)

This makes the `ExecBackend.ContainerKill()` redundant, so removing this from
the interface.

While looking at this code, one (possible) race-condition was found in
`daemon.cleanupContainer()`, where `daemon.Kill()` could return a `errdefs.Conflict`
if the container was already stopped. An extra check was added for this case to
prevent `daemon.cleanupContainer()` from terminating early.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-05 17:36:41 +01:00
Sebastiaan van Stijn
0aa96c5512
builder/dockerfile: containerManager.RemoveAll() prevent partial cleanup
Prevent cleanup from terminating early when failing to remove a container;

- continue trying to remove remaining containers
- ignore errors due to containers that were not found

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-05 17:34:34 +01:00
Sebastiaan van Stijn
03a8188a9a
builder/dockerfile: remove containerManager.removeContainer()
This was just a very thin wrapper for backend.ContainerRm(), and the
error it returned was not handled, so moving this code inline.

Moving it inline also allows differentiating the error message to
distinguish the "removing all intermediate containers" from "removing container"
(when cancelling a build).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-05 17:32:54 +01:00
Sebastiaan van Stijn
6b1baf8dd2
Merge pull request #47028 from vvoland/ci-mobybin-strip-tag-v
hack/make: Strip "v" tag prefix name from VERSION
2024-01-05 16:18:08 +01:00
Sebastiaan van Stijn
a38b5d7430
daemon: Daemon.ContainerRename: move vars closer to where they're used
Also break-up some "if" statements that were hiding that they were updating
existing variables.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-05 16:00:47 +01:00
Sebastiaan van Stijn
00a863b25e
daemon: Daemon.connectToNetwork: rename named return to prevent shadowing
The output var was used in a `defer`, but named `err` and shadowed in various
places. Rename the var to a more explicit name to make clear where it's used
and to prevent it being accidentally shadowed.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-05 15:56:17 +01:00
Jian Zeng
0f337e9c14
feat: make errdefs.IsXXX helper functions work with wrapped errors
Signed-off-by: Jian Zeng <anonymousknight96@gmail.com>
2024-01-05 22:54:13 +08:00
Paweł Gronowski
dd41f86339
hack/make: Refactor VERSION normalization
Turn the if-else tree into a `case`.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-05 13:21:13 +01:00
Sebastiaan van Stijn
2af17cd3f5
vendor: github.com/containerd/cgroups/v3 v3.0.3
release notes:

- NewSystemd handles UnitExists when starting units
- makefile fixes
- cgroups2: export memory max usage and swap max usage
- build(deps): bump github.com/cilium/ebpf from v0.9.1 to v0.11.0
- support psi
- feat: add Threads for cgroupv2
- Linux.Swap is defined as memory+swap combined, while in cgroup v2 swap is a separate value
- fix(): support re-enabling oom killer refs #307 by @kestrelcjx

full diff: https://github.com/containerd/cgroups/compare/v3.0.2...v3.0.3

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-05 13:18:14 +01:00
Paweł Gronowski
9a5393d500
hack/make: Strip "v" tag prefix name from VERSION
To make the version format in the `moby-bin` consistent with the
version we use in the release pipeline.

```diff
Server: Docker Engine - Community
 Engine:
-  Version:          v25.0.0
+  Version:          25.0.0
...
```

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-05 13:17:47 +01:00
Sebastiaan van Stijn
12128189eb
vendor: github.com/cilium/ebpf v0.11.0
full diff: https://github.com/cilium/ebpf/compare/v0.9.1...v0.11.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-05 13:14:49 +01:00
Sebastiaan van Stijn
c3c1ee0150
Merge pull request #47024 from thaJeztah/fix_daemon_side_digest_resolve
daemon/cluster: Cluster.imageWithDigestString: include mirrors to resolve digest
2024-01-05 12:13:35 +01:00
Sebastiaan van Stijn
07d2ad30e5
daemon/cluster: Cluster.imageWithDigestString: include mirrors to resolve digest
If the daemon is configured to use a mirror for the default (Docker Hub)
registry, the endpoint did not fall back to querying the upstream if the mirror
did not contain the given reference.

For pull-through registry-mirrors, this was not a problem, as in that case the
registry would forward the request, but for other mirrors, no fallback would
happen. This was inconsistent with how "pulling" images handled this situation;
when pulling images, both the mirror and upstream would be tried.

This patch brings the daemon-side lookup of image-manifests on-par with the
client-side lookup (the GET /distribution endpoint) as used in API 1.30 and
higher.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-04 16:53:45 +01:00
Sebastiaan van Stijn
9cebefa717
Merge pull request #47023 from thaJeztah/fix_distribution_mirrors
api: fix "GET /distribution" endpoint ignoring mirrors
2024-01-04 16:38:22 +01:00
Sebastiaan van Stijn
8aacbb3ba9
api: fix "GET /distribution" endpoint ignoring mirrors
If the daemon is configured to use a mirror for the default (Docker Hub)
registry, the endpoint did not fall back to querying the upstream if the mirror
did not contain the given reference.

If the daemon is configured to use a mirror for the default (Docker Hub)
registry, did not fall back to querying the upstream if the mirror did not
contain the given reference.

For pull-through registry-mirrors, this was not a problem, as in that case the
registry would forward the request, but for other mirrors, no fallback would
happen. This was inconsistent with how "pulling" images handled this situation;
when pulling images, both the mirror and upstream would be tried.

This problem was caused by the logic used in GetRepository, which had an
optimization to only return the first registry it was successfully able to
configure (and connect to), with the assumption that the mirror either contained
all images used, or to be configured as a pull-through mirror.

This patch:

- Introduces a GetRepositories method, which returns all candidates (both
  mirror(s) and upstream).
- Updates the endpoint to try all

Before this patch:

    # the daemon is configured to use a mirror for Docker Hub
    cat /etc/docker/daemon.json
    { "registry-mirrors": ["http://localhost:5000"]}

    # start the mirror (empty registry, not configured as pull-through mirror)
    docker run -d --name registry -p 127.0.0.1:5000:5000 registry:2

    # querying the endpoint fails, because the image-manifest is not found in the mirror:
    curl -s --unix-socket /var/run/docker.sock http://localhost/v1.43/distribution/docker.io/library/hello-world:latest/json
    {
      "message": "manifest unknown: manifest unknown"
    }

With this patch applied:

    # the daemon is configured to use a mirror for Docker Hub
    cat /etc/docker/daemon.json
    { "registry-mirrors": ["http://localhost:5000"]}

    # start the mirror (empty registry, not configured as pull-through mirror)
    docker run -d --name registry -p 127.0.0.1:5000:5000 registry:2

    # querying the endpoint succeeds (manifest is fetched from the upstream Docker Hub registry):
    curl -s --unix-socket /var/run/docker.sock http://localhost/v1.43/distribution/docker.io/library/hello-world:latest/json | jq .
    {
      "Descriptor": {
        "mediaType": "application/vnd.oci.image.index.v1+json",
        "digest": "sha256:1b9844d846ce3a6a6af7013e999a373112c3c0450aca49e155ae444526a2c45e",
        "size": 3849
      },
      "Platforms": [
        {
          "architecture": "amd64",
          "os": "linux"
        }
      ]
    }

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-04 15:46:32 +01:00
Sebastiaan van Stijn
e9042385ac
Merge pull request #47022 from akerouanton/libnet-bridge-dead-check-on-stale-default-nw
libnet/d/bridge: dead code: no conflict on stale default nw
2024-01-04 14:39:14 +01:00
Albin Kerouanton
b9e27acabc
libnet/d/bridge: dead code: no conflict on stale default nw
A check was added to the bridge driver to detect when it was called to
create the default bridge nw whereas a stale default bridge already
existed. In such case, the bridge driver was deleting the stale network
before re-creating it. This check was introduced in docker/libnetwork@6b158eac6a
to fix an issue related to newly introduced live-restore.

However, since commit docker/docker@ecffb6d58c,
the daemon doesn't even try to create default networks if there're
active sandboxes (ie. due to live-restore).

Thus, now it's impossible for the default bridge network to be stale and
to exists when the driver's CreateNetwork() method is called. As such,
the check introduced in the first commit mentioned above is dead code
and can be safely removed.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-01-04 11:50:04 +01:00
Albin Kerouanton
0a26cdf344
libnet/d/bridge: remove dead ActiveEndpointsError
This error is unused since docker/libnetwork@6b158eac6.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-01-04 11:12:53 +01:00
Brian Goff
12c7411b6b
Merge pull request #46750 from thaJeztah/daemon_start_log 2024-01-03 17:19:59 -08:00
Brian Goff
e2b78f074b
Merge pull request #47020 from thaJeztah/fix_go_compilerbug
daemon: work around go1.21 compiler bug
2024-01-03 16:10:00 -08:00
Sebastiaan van Stijn
cc6d826fdd
Merge pull request #46971 from thaJeztah/api_docs
docs/api: add documentation for API v1.44
2024-01-04 00:39:05 +01:00
Cory Snider
6d2c4f87af
daemon: work around go1.21 compiler bug
The Go 1.21.5 compiler has a bug: per-file language version override
directives do not take effect when instantiating generic functions which
have certain nontrivial type constraints. Consequently, a module-mode
project with Moby as a dependency may fail to compile when the compiler
incorrectly applies go1.16 semantics to the generic function call.

As the offending function is trivial and is only used in one place, work
around the issue by converting it to a concretely-typed function.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-01-04 00:02:39 +01:00
Sebastiaan van Stijn
69c23cc0c9
docs/api: add documentation for API v1.44
Copy the swagger / OpenAPI file to the documentation. This is the API
version used by the upcoming v25.0.0 release.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-03 23:45:54 +01:00
Sebastiaan van Stijn
e7001c1455
Merge pull request #47015 from thaJeztah/api_fields_deprecation
api: Deprecate outdated fields in NetworkSettingsBase
2024-01-03 19:41:59 +01:00
Sebastiaan van Stijn
7df554acb0
Merge pull request #46974 from akerouanton/fud-debug-log
libnet: Improve the debug log written when the extKeyListener is stopped
2024-01-03 19:28:58 +01:00
Sebastiaan van Stijn
08632253d9
Merge pull request #46972 from dmcgowan/fix-userns-capabilities
c8d: Fix image commit with userns mapping (carry)
2024-01-03 19:18:12 +01:00
Albin Kerouanton
ce574949f7
api: Deprecate outdated fields in NetworkSettingsBase
The following fields are never written and are now marked as deprecated:

- `HairpinMode`
- `LinkLocalIPv6Address`
- `LinkLocalIPv6PrefixLen`
- `SecondaryIPAddress`
- `SecondaryIPv6Addresses`

Co-authored-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-01-03 18:51:17 +01:00
Sebastiaan van Stijn
57a37ed583
Merge pull request #47016 from thaJeztah/bump_golangci_lint
update golangci-lint to v1.55.2
2024-01-03 18:45:37 +01:00
Sebastiaan van Stijn
1f6c42c678
Merge pull request #47006 from thaJeztah/deprecate_ResolveScopedPath
pkg/containerfs: unify CleanScopedPath implementation, and deprecate ResolveScopedPath
2024-01-03 15:38:52 +01:00
Sebastiaan van Stijn
f8159163c1
Merge pull request #47014 from thaJeztah/daemon_deadcode
daemon: remove dead code relying on SecondaryIPAddress
2024-01-03 14:20:46 +01:00
Sebastiaan van Stijn
d5a3fccb06
update golangci-lint to v1.55.2
- full diff: https://github.com/golangci/golangci-lint/compare/v1.54.2...v1.55.2

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-03 13:41:37 +01:00