Commit graph

48749 commits

Author SHA1 Message Date
Sebastiaan van Stijn
62b33a2604
disable pulling legacy image formats by default
This patch disables pulling legacy (schema1 and schema 2, version 1) images by
default.

A `DOCKER_ENABLE_DEPRECATED_PULL_SCHEMA_1_IMAGE` environment-variable is
introduced to allow re-enabling this feature, aligning with the environment
variable used in containerd 2.0 (`CONTAINERD_ENABLE_DEPRECATED_PULL_SCHEMA_1_IMAGE`).

With this patch, attempts to pull a legacy image produces an error:

With graphdrivers:

    docker pull docker:1.0
    1.0: Pulling from library/docker
    [DEPRECATION NOTICE] Docker Image Format v1, and Docker Image manifest version 2, schema 1 support will be removed in an upcoming release. Suggest the author of docker.io/library/docker:1.0 to upgrade the image to the OCI Format, or Docker Image manifest v2, schema 2. More information at https://docs.docker.com/go/deprecated-image-specs/

With the containerd image store enabled, output is slightly different
as it returns the error before printing the `1.0: pulling ...`:

    docker pull docker:1.0
    Error response from daemon: [DEPRECATION NOTICE] Docker Image Format v1 and Docker Image manifest version 2, schema 1 support is disabled by default and will be removed in an upcoming release. Suggest the author of docker.io/library/docker:1.0 to upgrade the image to the OCI Format or Docker Image manifest v2, schema 2. More information at https://docs.docker.com/go/deprecated-image-specs/

Using the "distribution" endpoint to resolve the digest for an image also
produces an error:

    curl -v --unix-socket /var/run/docker.sock http://foo/distribution/docker.io/library/docker:1.0/json
    *   Trying /var/run/docker.sock:0...
    * Connected to foo (/var/run/docker.sock) port 80 (#0)
    > GET /distribution/docker.io/library/docker:1.0/json HTTP/1.1
    > Host: foo
    > User-Agent: curl/7.88.1
    > Accept: */*
    >
    < HTTP/1.1 400 Bad Request
    < Api-Version: 1.45
    < Content-Type: application/json
    < Docker-Experimental: false
    < Ostype: linux
    < Server: Docker/dev (linux)
    < Date: Tue, 27 Feb 2024 16:09:42 GMT
    < Content-Length: 354
    <
    {"message":"[DEPRECATION NOTICE] Docker Image Format v1, and Docker Image manifest version 2, schema 1 support will be removed in an upcoming release. Suggest the author of docker.io/library/docker:1.0 to upgrade the image to the OCI Format, or Docker Image manifest v2, schema 2. More information at https://docs.docker.com/go/deprecated-image-specs/"}
    * Connection #0 to host foo left intact

Starting the daemon with the `DOCKER_ENABLE_DEPRECATED_PULL_SCHEMA_1_IMAGE`
env-var set to a non-empty value allows pulling the image;

    docker pull docker:1.0
    [DEPRECATION NOTICE] Docker Image Format v1 and Docker Image manifest version 2, schema 1 support is disabled by default and will be removed in an upcoming release. Suggest the author of docker.io/library/docker:1.0 to upgrade the image to the OCI Format or Docker Image manifest v2, schema 2. More information at https://docs.docker.com/go/deprecated-image-specs/
    b0a0e6710d13: Already exists
    d193ad713811: Already exists
    ba7268c3149b: Already exists
    c862d82a67a2: Already exists
    Digest: sha256:5e7081837926c7a40e58881bbebc52044a95a62a2ea52fb240db3fc539212fe5
    Status: Image is up to date for docker:1.0
    docker.io/library/docker:1.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-28 13:57:43 +01:00
Sebastiaan van Stijn
b37f8c8070
Merge pull request #47460 from thaJeztah/bump_bolt
vendor: go.etcd.io/bbolt v1.3.9
2024-02-27 20:01:52 +01:00
Sebastiaan van Stijn
9be820d8ca
vendor: go.etcd.io/bbolt v1.3.9
full diff: https://github.com/etcd-io/bbolt/compare/v1.3.7...v1.3.9

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-27 18:24:01 +01:00
Sebastiaan van Stijn
f6fa6ff9ed
Merge pull request #47391 from vvoland/rro-backwards-compatible
api/pre-1.44: Default `ReadOnlyNonRecursive` to true
2024-02-27 18:04:46 +01:00
Sebastiaan van Stijn
220835106b
Merge pull request #47364 from vvoland/buildkit-v13
vendor: github.com/moby/buildkit v0.13.0-rc2
2024-02-27 16:38:04 +01:00
Paweł Gronowski
2c25ca9dba
Merge pull request #47455 from vvoland/c8d-skip-last-windows-tests
c8d/windows: Temporarily skip two failing tests
2024-02-27 14:01:31 +01:00
Paweł Gronowski
94f9f39b24
Merge pull request #47454 from vvoland/c8d-pull-pullingfslayer-truncated
c8d/pull: Output truncated id for `Pulling fs layer`
2024-02-27 13:28:38 +01:00
Paweł Gronowski
44167988c3
c8d/windows: Temporarily skip two failing tests
They're failing the CI and we have a tracking ticket: #47107

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-27 12:04:21 +01:00
Paweł Gronowski
2d31532a00
otel: Default metrics protocol to http/protobuf
Buildkit added support for exporting metrics in:
7de2e4fb32

Explicitly set the protocol for exporting metrics like we do for the
traces. We need that because Buildkit defaults to grpc.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-27 11:27:12 +01:00
CrazyMax
60358bfcab
ci(buildkit): dedicated step to build test image
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-27 11:27:11 +01:00
Paweł Gronowski
f5722da5e0
mobyexporter: Store temporary config descriptor
Temporarily store the produced config descriptor for the buildkit
history to work.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-27 11:27:09 +01:00
Paweł Gronowski
951e42cd60
builder-next: Replace ResolveImageConfig with ResolveSourceMetadata
30c069cb03
removed the `ResolveImageConfig` method in favor of more generic
`ResolveSourceMetadata` that can also support other things than images.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-27 11:26:37 +01:00
Paweł Gronowski
e01a1c5d09
builder/mobyexporter: Set image.name response key
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-27 11:26:35 +01:00
Paweł Gronowski
fa467caf4d
builder-next/mobyexporter: Use OptKeyName const
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-27 11:26:33 +01:00
Paweł Gronowski
59ad1690f7
builder-next: Adjust to source changes
Adjust to cache sources changes from:
6b27487fec

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-27 11:26:17 +01:00
Paweł Gronowski
b04a2dad6b
builder/controller: Adjust NewWorkerOpt call
8bfd280ab7
added a new argument that allows to specify different runtime.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-27 11:26:15 +01:00
Paweł Gronowski
bc6d88c09a
cmd/dockerd: Fix overriding OTEL resource
e358792815
changed that field to a function and added an `OverrideResource`
function that allows to override it.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-27 11:26:14 +01:00
Paweł Gronowski
a79bb1e832
builder-next/exporter: Sync with new signature
1c1777b7c0
added an explicit id argument to the Resolve method.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-27 11:26:12 +01:00
Paweł Gronowski
e68f71259a
integration/build: Use fsutil.NewFS
StaticDirSource definition changed and can no longer be initialized from
the composite literal.

a80b48544c

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-27 11:26:10 +01:00
Paweł Gronowski
dd6992617e
integration/build: Use new buildkit progressui
Introduced in: 37131781d7

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-27 11:26:09 +01:00
Paweł Gronowski
31545c3b67
vendor: github.com/moby/buildkit v0.13.0-rc2
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-27 11:26:07 +01:00
CrazyMax
f90b03ee5d
go.mod: bump to go 1.21 and use local toolchain when vendoring
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-27 11:25:20 +01:00
Paweł Gronowski
16aa7dd67f
c8d/pull: Output truncated id for Pulling fs layer
All other progress updates are emitted with truncated id.

```diff
$ docker pull --platform linux/amd64 alpine
Using default tag: latest
latest: Pulling from library/alpine
-sha256:4abcf20661432fb2d719aaf90656f55c287f8ca915dc1c92ec14ff61e67fbaf8: Pulling fs layer
+4abcf2066143: Download complete
Digest: sha256:c5b1261d6d3e43071626931fc004f70149baeba2c8ec672bd4f27761f8e1ad6b
Status: Image is up to date for alpine:latest
docker.io/library/alpine:latest
```

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-27 11:09:16 +01:00
Sebastiaan van Stijn
8cdb5a9070
Merge pull request #47450 from neersighted/image_created_omitempty
api: omit missing Created field from ImageInspect response
2024-02-26 20:06:52 +01:00
Sebastiaan van Stijn
ffd294ebcc
Merge pull request #45967 from tianon/c8d-image-list
c8d: Adjust "image list" to return only a single item for each image store entry
2024-02-26 20:05:29 +01:00
Bjorn Neergaard
881260148f
api: omit missing Created field from ImageInspect response
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2024-02-26 10:26:15 -07:00
Paweł Gronowski
432390320e
api/pre-1.44: Default ReadOnlyNonRecursive to true
Don't change the behavior for older clients and keep the same behavior.
Otherwise client can't opt-out (because `ReadOnlyNonRecursive` is
unsupported before 1.44).

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-26 11:37:30 +01:00
Sebastiaan van Stijn
c70d7905fb
Merge pull request #47432 from vvoland/c8d-pull-fslayer
c8d/pull: Progress fixes
2024-02-26 10:38:00 +01:00
Sebastiaan van Stijn
0eecd59153
Merge pull request #47245 from thaJeztah/bump_otel
vendor: OTEL v0.46.1 / v1.21.0
2024-02-23 17:47:27 +01:00
Sebastiaan van Stijn
24fe934a7b
Merge pull request #47423 from vvoland/ci-check-changelog
ci: Require changelog description
2024-02-23 12:24:13 +01:00
Paweł Gronowski
05b883bdc8
mounts/validate: Don't check source exists with CreateMountpoint
Don't error out when mount source doesn't exist and mounts has
`CreateMountpoint` option enabled.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-23 11:20:55 +01:00
Sebastiaan van Stijn
c516804d6f
vendor: OTEL v0.46.1 / v1.21.0
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-23 10:11:07 +01:00
Sebastiaan van Stijn
445d500aeb
Merge pull request #47410 from vvoland/test-daemonproxy-no-otel
integration/TestDaemonProxy: Remove OTEL span
2024-02-22 22:19:53 +01:00
Sebastiaan van Stijn
1ffaf469ba
Merge pull request #47175 from corhere/best-effort-xattrs-classic-builder
builder/dockerfile: ADD with best-effort xattrs
2024-02-22 20:14:22 +01:00
Albin Kerouanton
842d1b3c12
Merge pull request #47433 from akerouanton/libnet-ds-extra-space-in-err
libnet/ds: remove extra space in error msg
2024-02-22 19:38:26 +01:00
Albin Kerouanton
83c02f7a11 libnet/ds: remove extra space in error msg
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-02-22 18:49:28 +01:00
Paweł Gronowski
14df52b709
c8d/pull: Don't emit Downloading with 0 progress
To align with the graphdrivers behavior and don't send unnecessary
progress messages.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-22 18:03:16 +01:00
Paweł Gronowski
ff5f780f2b
c8d/pull: Emit Pulling fs layer
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-22 18:03:15 +01:00
Paweł Gronowski
5689dabfb3
pkg/streamformatter: Make progressOutput concurrency safe
Sync access to the underlying `io.Writer` with a mutex.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-22 18:03:13 +01:00
Sebastiaan van Stijn
7d081179e9
Merge pull request #47422 from akerouanton/libnet-ds-DeleteIdempotent
libnet: Replace DeleteAtomic in retry loops with Delete
2024-02-22 17:24:05 +01:00
Paweł Gronowski
3865c63d45
Merge pull request #47426 from vvoland/vendor-continuity
vendor: github.com/containerd/continuity v0.4.3
2024-02-22 14:28:41 +01:00
Paweł Gronowski
1d473549e8
ci: Require changelog description
Any PR that is labeled with any `impact/*` label should have a
description for the changelog and an `area/*` label.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-22 13:40:23 +01:00
Paweł Gronowski
b2aaf5c2b0
vendor: github.com/containerd/continuity v0.4.3
full diff: https://github.com/containerd/continuity/compare/v0.4.3...v0.4.2

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-22 10:24:52 +01:00
Albin Kerouanton
cbd45e83cf libnet: Replace DeleteAtomic in retry loops with DeleteIdempotent
A common pattern in libnetwork is to delete an object using
`DeleteAtomic`, ie. to check the optimistic lock, but put in a retry
loop to refresh the data and the version index used by the optimistic
lock.

This commit introduces a new `Delete` method to delete without
checking the optimistic lock. It focuses only on the few places where
it's obvious the calling code doesn't rely on the side-effects of the
retry loop (ie. refreshing the object to be deleted).

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-02-22 08:22:09 +01:00
Sebastiaan van Stijn
cba87125b2
Merge pull request #47405 from vvoland/validate-vendor-nopager
validate/vendor: Disable pager for git diff
2024-02-21 17:16:11 +01:00
CrazyMax
2a41ce93fe
Merge pull request #47409 from crazy-max/ci-codecov-token
ci: set codecov token
2024-02-21 16:32:15 +01:00
Sebastiaan van Stijn
c42ae61e62
Merge pull request #47417 from thaJeztah/resolver_improve_logs
libnetwork: resolve: use structured logs for DNS error
2024-02-21 10:41:06 +01:00
Sebastiaan van Stijn
d9e082ff54
libnetwork: resolve: use structured logs for DNS error
I noticed that this log didn't use structured logs;

    [resolver] failed to query DNS server: 10.115.11.146:53, query: ;google.com.\tIN\t A" error="read udp 172.19.0.2:46361->10.115.11.146:53: i/o timeout
    [resolver] failed to query DNS server: 10.44.139.225:53, query: ;google.com.\tIN\t A" error="read udp 172.19.0.2:53991->10.44.139.225:53: i/o timeout

But other logs did;

    DEBU[2024-02-20T15:48:51.026704088Z] [resolver] forwarding query                   client-addr="udp:172.19.0.2:39661" dns-server="udp:192.168.65.7:53" question=";google.com.\tIN\t A"
    DEBU[2024-02-20T15:48:51.028331088Z] [resolver] forwarding query                   client-addr="udp:172.19.0.2:35163" dns-server="udp:192.168.65.7:53" question=";google.com.\tIN\t AAAA"
    DEBU[2024-02-20T15:48:51.057329755Z] [resolver] received AAAA record "2a00:1450:400e:801::200e" for "google.com." from udp:192.168.65.7
    DEBU[2024-02-20T15:48:51.057666880Z] [resolver] received A record "142.251.36.14" for "google.com." from udp:192.168.65.7

As we're already constructing a logger with these fields, we may as well use it.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-20 17:01:06 +01:00
Paweł Gronowski
8761bffcaf
Makefile: Pass PAGER/GIT_PAGER variable
Allow to override the PAGER/GIT_PAGER variables inside the container.
Use `cat` as pager when running in Github Actions (to avoid things like
`git diff` stalling the CI).

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-20 11:52:58 +01:00
Paweł Gronowski
56aeb548b2
integration/TestDaemonProxy: Remove OTEL span
Don't use OTEL tracing in this test because we're testing the HTTP proxy
behavior here and we don't want OTEL to interfere.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-20 10:21:53 +01:00