Commit graph

48965 commits

Author SHA1 Message Date
Cory Snider
7b0ab1011c Vendor dependency cycle-free swarmkit
Moby imports Swarmkit; Swarmkit no longer imports Moby. In order to
accomplish this feat, Swarmkit has introduced a new plugin.Getter
interface so it could stop importing our pkg/plugingetter package. This
new interface is not entirely compatible with our
plugingetter.PluginGetter interface, necessitating a thin adapter.

Swarmkit had to jettison the CNM network allocator to stop having to
import libnetwork as the cnmallocator package is deeply tied to
libnetwork. Move the CNM network allocator into libnetwork, where it
belongs. The package had a short an uninteresting Git history in the
Swarmkit repository so no effort was made to retain history.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-02-28 09:46:45 -05:00
Sebastiaan van Stijn
3ca1d751e5
Merge pull request #47461 from vvoland/vendor-buildkit-0.13.0-rc3
vendor: github.com/moby/buildkit v0.13.0-rc3
2024-02-28 14:12:43 +01:00
Sebastiaan van Stijn
589dc5e647
Merge pull request #47456 from huang-jl/fix_restore_digest
libcontainerd: change the digest used when restoring
2024-02-28 14:05:40 +01:00
Sebastiaan van Stijn
62b33a2604
disable pulling legacy image formats by default
This patch disables pulling legacy (schema1 and schema 2, version 1) images by
default.

A `DOCKER_ENABLE_DEPRECATED_PULL_SCHEMA_1_IMAGE` environment-variable is
introduced to allow re-enabling this feature, aligning with the environment
variable used in containerd 2.0 (`CONTAINERD_ENABLE_DEPRECATED_PULL_SCHEMA_1_IMAGE`).

With this patch, attempts to pull a legacy image produces an error:

With graphdrivers:

    docker pull docker:1.0
    1.0: Pulling from library/docker
    [DEPRECATION NOTICE] Docker Image Format v1, and Docker Image manifest version 2, schema 1 support will be removed in an upcoming release. Suggest the author of docker.io/library/docker:1.0 to upgrade the image to the OCI Format, or Docker Image manifest v2, schema 2. More information at https://docs.docker.com/go/deprecated-image-specs/

With the containerd image store enabled, output is slightly different
as it returns the error before printing the `1.0: pulling ...`:

    docker pull docker:1.0
    Error response from daemon: [DEPRECATION NOTICE] Docker Image Format v1 and Docker Image manifest version 2, schema 1 support is disabled by default and will be removed in an upcoming release. Suggest the author of docker.io/library/docker:1.0 to upgrade the image to the OCI Format or Docker Image manifest v2, schema 2. More information at https://docs.docker.com/go/deprecated-image-specs/

Using the "distribution" endpoint to resolve the digest for an image also
produces an error:

    curl -v --unix-socket /var/run/docker.sock http://foo/distribution/docker.io/library/docker:1.0/json
    *   Trying /var/run/docker.sock:0...
    * Connected to foo (/var/run/docker.sock) port 80 (#0)
    > GET /distribution/docker.io/library/docker:1.0/json HTTP/1.1
    > Host: foo
    > User-Agent: curl/7.88.1
    > Accept: */*
    >
    < HTTP/1.1 400 Bad Request
    < Api-Version: 1.45
    < Content-Type: application/json
    < Docker-Experimental: false
    < Ostype: linux
    < Server: Docker/dev (linux)
    < Date: Tue, 27 Feb 2024 16:09:42 GMT
    < Content-Length: 354
    <
    {"message":"[DEPRECATION NOTICE] Docker Image Format v1, and Docker Image manifest version 2, schema 1 support will be removed in an upcoming release. Suggest the author of docker.io/library/docker:1.0 to upgrade the image to the OCI Format, or Docker Image manifest v2, schema 2. More information at https://docs.docker.com/go/deprecated-image-specs/"}
    * Connection #0 to host foo left intact

Starting the daemon with the `DOCKER_ENABLE_DEPRECATED_PULL_SCHEMA_1_IMAGE`
env-var set to a non-empty value allows pulling the image;

    docker pull docker:1.0
    [DEPRECATION NOTICE] Docker Image Format v1 and Docker Image manifest version 2, schema 1 support is disabled by default and will be removed in an upcoming release. Suggest the author of docker.io/library/docker:1.0 to upgrade the image to the OCI Format or Docker Image manifest v2, schema 2. More information at https://docs.docker.com/go/deprecated-image-specs/
    b0a0e6710d13: Already exists
    d193ad713811: Already exists
    ba7268c3149b: Already exists
    c862d82a67a2: Already exists
    Digest: sha256:5e7081837926c7a40e58881bbebc52044a95a62a2ea52fb240db3fc539212fe5
    Status: Image is up to date for docker:1.0
    docker.io/library/docker:1.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-28 13:57:43 +01:00
Paweł Gronowski
5fe96e234d
integration: Reset OTEL_EXPORTER_OTLP_ENDPOINT for sub-daemons
When creating a new daemon in the `TestDaemonProxy`, reset the
`OTEL_EXPORTER_OTLP_ENDPOINT` to an empty value to disable OTEL
collection to avoid it hitting the proxy.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-28 10:48:07 +01:00
Paweł Gronowski
84eecc4a30
Revert "integration/TestDaemonProxy: Remove OTEL span"
This reverts commit 56aeb548b2.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-28 10:48:03 +01:00
Paweł Gronowski
261dccc98a
builder-next: Add Info to emptyProvider
To satisfy the `content.InfoReaderProvider` interface.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-28 10:20:55 +01:00
Paweł Gronowski
2c9c5e1c03
vendor: github.com/moby/buildkit v0.13.0-rc3
full diff: https://github.com/moby/buildkit/compare/v0.13.0-rc2...v0.13.0-rc3

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-28 10:05:02 +01:00
serhii.n
b649e272bb Allow to enable host loopback and use 10.0.2.2 to connect to the host (OPTIONALLY)
This should allow to enable host loopback by setting
DOCKERD_ROOTLESS_ROOTLESSKIT_DISABLE_HOST_LOOPBACK to false,
defaults true.

Signed-off-by: serhii.n <serhii.n@thescimus.com>
2024-02-28 00:52:35 +02:00
Paweł Gronowski
e4de4dea5c
ci: Make find for test reports more specific
Don't use all `*.json` files blindly, take only these that are likely to
be reports from go test.
Also, use `find ... -exec` instead of piping results to `xargs`.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-27 23:38:03 +01:00
Sebastiaan van Stijn
b37f8c8070
Merge pull request #47460 from thaJeztah/bump_bolt
vendor: go.etcd.io/bbolt v1.3.9
2024-02-27 20:01:52 +01:00
Sebastiaan van Stijn
9be820d8ca
vendor: go.etcd.io/bbolt v1.3.9
full diff: https://github.com/etcd-io/bbolt/compare/v1.3.7...v1.3.9

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-27 18:24:01 +01:00
Sebastiaan van Stijn
f6fa6ff9ed
Merge pull request #47391 from vvoland/rro-backwards-compatible
api/pre-1.44: Default `ReadOnlyNonRecursive` to true
2024-02-27 18:04:46 +01:00
Sebastiaan van Stijn
220835106b
Merge pull request #47364 from vvoland/buildkit-v13
vendor: github.com/moby/buildkit v0.13.0-rc2
2024-02-27 16:38:04 +01:00
Paweł Gronowski
2c25ca9dba
Merge pull request #47455 from vvoland/c8d-skip-last-windows-tests
c8d/windows: Temporarily skip two failing tests
2024-02-27 14:01:31 +01:00
Paweł Gronowski
94f9f39b24
Merge pull request #47454 from vvoland/c8d-pull-pullingfslayer-truncated
c8d/pull: Output truncated id for `Pulling fs layer`
2024-02-27 13:28:38 +01:00
huang-jl
da643c0b8a libcontainerd: change the digest used when restoring
For current implementation of Checkpoint Restore (C/R) in docker, it
will write the checkpoint to content store. However, when restoring
libcontainerd uses .Digest().Encoded(), which will remove the info
of alg, leading to error.

Signed-off-by: huang-jl <1046678590@qq.com>
2024-02-27 20:17:31 +08:00
Rob Murray
9083c2f10d Test DNS on Windows 'nat' networks
Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-02-27 11:40:11 +00:00
Paweł Gronowski
44167988c3
c8d/windows: Temporarily skip two failing tests
They're failing the CI and we have a tracking ticket: #47107

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-27 12:04:21 +01:00
Paweł Gronowski
2d31532a00
otel: Default metrics protocol to http/protobuf
Buildkit added support for exporting metrics in:
7de2e4fb32

Explicitly set the protocol for exporting metrics like we do for the
traces. We need that because Buildkit defaults to grpc.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-27 11:27:12 +01:00
CrazyMax
60358bfcab
ci(buildkit): dedicated step to build test image
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-27 11:27:11 +01:00
Paweł Gronowski
f5722da5e0
mobyexporter: Store temporary config descriptor
Temporarily store the produced config descriptor for the buildkit
history to work.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-27 11:27:09 +01:00
Paweł Gronowski
951e42cd60
builder-next: Replace ResolveImageConfig with ResolveSourceMetadata
30c069cb03
removed the `ResolveImageConfig` method in favor of more generic
`ResolveSourceMetadata` that can also support other things than images.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-27 11:26:37 +01:00
Paweł Gronowski
e01a1c5d09
builder/mobyexporter: Set image.name response key
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-27 11:26:35 +01:00
Paweł Gronowski
fa467caf4d
builder-next/mobyexporter: Use OptKeyName const
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-27 11:26:33 +01:00
Paweł Gronowski
59ad1690f7
builder-next: Adjust to source changes
Adjust to cache sources changes from:
6b27487fec

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-27 11:26:17 +01:00
Paweł Gronowski
b04a2dad6b
builder/controller: Adjust NewWorkerOpt call
8bfd280ab7
added a new argument that allows to specify different runtime.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-27 11:26:15 +01:00
Paweł Gronowski
bc6d88c09a
cmd/dockerd: Fix overriding OTEL resource
e358792815
changed that field to a function and added an `OverrideResource`
function that allows to override it.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-27 11:26:14 +01:00
Paweł Gronowski
a79bb1e832
builder-next/exporter: Sync with new signature
1c1777b7c0
added an explicit id argument to the Resolve method.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-27 11:26:12 +01:00
Paweł Gronowski
e68f71259a
integration/build: Use fsutil.NewFS
StaticDirSource definition changed and can no longer be initialized from
the composite literal.

a80b48544c

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-27 11:26:10 +01:00
Paweł Gronowski
dd6992617e
integration/build: Use new buildkit progressui
Introduced in: 37131781d7

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-27 11:26:09 +01:00
Paweł Gronowski
31545c3b67
vendor: github.com/moby/buildkit v0.13.0-rc2
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-27 11:26:07 +01:00
CrazyMax
f90b03ee5d
go.mod: bump to go 1.21 and use local toolchain when vendoring
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-27 11:25:20 +01:00
Paweł Gronowski
16aa7dd67f
c8d/pull: Output truncated id for Pulling fs layer
All other progress updates are emitted with truncated id.

```diff
$ docker pull --platform linux/amd64 alpine
Using default tag: latest
latest: Pulling from library/alpine
-sha256:4abcf20661432fb2d719aaf90656f55c287f8ca915dc1c92ec14ff61e67fbaf8: Pulling fs layer
+4abcf2066143: Download complete
Digest: sha256:c5b1261d6d3e43071626931fc004f70149baeba2c8ec672bd4f27761f8e1ad6b
Status: Image is up to date for alpine:latest
docker.io/library/alpine:latest
```

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-27 11:09:16 +01:00
Sebastiaan van Stijn
8cdb5a9070
Merge pull request #47450 from neersighted/image_created_omitempty
api: omit missing Created field from ImageInspect response
2024-02-26 20:06:52 +01:00
Sebastiaan van Stijn
ffd294ebcc
Merge pull request #45967 from tianon/c8d-image-list
c8d: Adjust "image list" to return only a single item for each image store entry
2024-02-26 20:05:29 +01:00
Bjorn Neergaard
881260148f
api: omit missing Created field from ImageInspect response
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2024-02-26 10:26:15 -07:00
Paweł Gronowski
432390320e
api/pre-1.44: Default ReadOnlyNonRecursive to true
Don't change the behavior for older clients and keep the same behavior.
Otherwise client can't opt-out (because `ReadOnlyNonRecursive` is
unsupported before 1.44).

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-26 11:37:30 +01:00
Sebastiaan van Stijn
c70d7905fb
Merge pull request #47432 from vvoland/c8d-pull-fslayer
c8d/pull: Progress fixes
2024-02-26 10:38:00 +01:00
Sebastiaan van Stijn
0eecd59153
Merge pull request #47245 from thaJeztah/bump_otel
vendor: OTEL v0.46.1 / v1.21.0
2024-02-23 17:47:27 +01:00
Sebastiaan van Stijn
6aea26b431
client: fix connection-errors being shadowed by API version mismatch errors
Commit e6907243af applied a fix for situations
where the client was configured with API-version negotiation, but did not yet
negotiate a version.

However, the checkVersion() function that was implemented copied the semantics
of cli.NegotiateAPIVersion, which ignored connection failures with the
assumption that connection errors would still surface further down.

However, when using the result of a failed negotiation for NewVersionError,
an API version mismatch error would be produced, masking the actual connection
error.

This patch changes the signature of checkVersion to return unexpected errors,
including failures to connect to the API.

Before this patch:

    docker -H unix:///no/such/socket.sock secret ls
    "secret list" requires API version 1.25, but the Docker daemon API version is 1.24

With this patch applied:

    docker -H unix:///no/such/socket.sock secret ls
    Cannot connect to the Docker daemon at unix:///no/such/socket.sock. Is the docker daemon running?

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-23 15:17:10 +01:00
Sebastiaan van Stijn
913478b428
client: doRequest: make sure we return a connection-error
This function has various errors that are returned when failing to make a
connection (due to permission issues, TLS mis-configuration, or failing to
resolve the TCP address).

The errConnectionFailed error is currently used as a special case when
processing Ping responses. The current code did not consistently treat
connection errors, and because of that could either absorb the error,
or process the empty response.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-23 15:13:22 +01:00
Sebastiaan van Stijn
901b90593d
client: NegotiateAPIVersion: do not ignore (connection) errors from Ping
NegotiateAPIVersion was ignoring errors returned by Ping. The intent here
was to handle API responses from a daemon that may be in an unhealthy state,
however this case is already handled by Ping itself.

Ping only returns an error when either failing to connect to the API (daemon
not running or permissions errors), or when failing to parse the API response.

Neither of those should be ignored in this code, or considered a successful
"ping", so update the code to return

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-23 14:31:55 +01:00
Sebastiaan van Stijn
349abc64ed
client: fix TestPingWithError
This test was added in 27ef09a46f, which changed
the Ping handling to ignore internal server errors. That case is tested in
TestPingFail, which verifies that we accept the Ping response if a 500
status code was received.

The TestPingWithError test was added to verify behavior if a protocol
(connection) error occurred; however the mock-client returned both a
response, and an error; the error returned would only happen if a connection
error occurred, which means that the server would not provide a reply.

Running the test also shows that returning a response is unexpected, and
ignored:

    === RUN   TestPingWithError
    2024/02/23 14:16:49 RoundTripper returned a response & error; ignoring response
    2024/02/23 14:16:49 RoundTripper returned a response & error; ignoring response
    --- PASS: TestPingWithError (0.00s)
    PASS

This patch updates the test to remove the response.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-23 14:25:52 +01:00
Sebastiaan van Stijn
24fe934a7b
Merge pull request #47423 from vvoland/ci-check-changelog
ci: Require changelog description
2024-02-23 12:24:13 +01:00
Paweł Gronowski
05b883bdc8
mounts/validate: Don't check source exists with CreateMountpoint
Don't error out when mount source doesn't exist and mounts has
`CreateMountpoint` option enabled.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-23 11:20:55 +01:00
Sebastiaan van Stijn
c516804d6f
vendor: OTEL v0.46.1 / v1.21.0
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-23 10:11:07 +01:00
Sebastiaan van Stijn
445d500aeb
Merge pull request #47410 from vvoland/test-daemonproxy-no-otel
integration/TestDaemonProxy: Remove OTEL span
2024-02-22 22:19:53 +01:00
Sebastiaan van Stijn
1ffaf469ba
Merge pull request #47175 from corhere/best-effort-xattrs-classic-builder
builder/dockerfile: ADD with best-effort xattrs
2024-02-22 20:14:22 +01:00
Albin Kerouanton
842d1b3c12
Merge pull request #47433 from akerouanton/libnet-ds-extra-space-in-err
libnet/ds: remove extra space in error msg
2024-02-22 19:38:26 +01:00