Commit graph

7491 commits

Author SHA1 Message Date
Brian Goff
d0df6c983c
Merge pull request #45359 from thaJeztah/remove_deprecated_overlay1
remove deprecated legacy "overlay" storage-driver
2023-04-20 11:08:52 -07:00
Sebastiaan van Stijn
5745ba6a8e
Merge pull request #45354 from thaJeztah/c8d_test_fixes
containerd integration: some test-fixes
2023-04-19 18:58:05 +02:00
Sebastiaan van Stijn
f72548956f
remove deprecated legacy "overlay" storage-driver
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-19 17:06:45 +02:00
Tianon Gravi
bbb94fd499
Merge pull request #45342 from thaJeztah/remove_deprecated_aufs_storage_driver
daemon: remove deprecated AuFS storage driver
2023-04-18 16:05:54 -07:00
Sebastiaan van Stijn
4ecb587eff
daemon: skip TestGetInspectData when using containerd snapshotters
This test does not apply when running with snapshotters enabled;

    go test -v -run TestGetInspectData .
    === RUN   TestGetInspectData
        inspect_test.go:27: RWLayer of container inspect-me is unexpectedly nil
    --- FAIL: TestGetInspectData (0.00s)
    FAIL
    FAIL	github.com/docker/docker/daemon	0.049s
    FAIL

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-18 15:02:41 +02:00
Sebastiaan van Stijn
a82c434447
daemon: setupFakeDaemon(): add fakeImageService
To prevent a panic happening when running tests:

    === FAIL: daemon TestTmpfsDevShmNoDupMount (0.01s)
    panic: runtime error: invalid memory address or nil pointer dereference [recovered]
        panic: runtime error: invalid memory address or nil pointer dereference
    [signal SIGSEGV: segmentation violation code=0x1 addr=0x120 pc=0x261a373]

    goroutine 134 [running]:
    testing.tRunner.func1.2({0x28baf20, 0x3ea8000})
        /usr/local/go/src/testing/testing.go:1526 +0x24e
    testing.tRunner.func1()
        /usr/local/go/src/testing/testing.go:1529 +0x39f
    panic({0x28baf20, 0x3ea8000})
        /usr/local/go/src/runtime/panic.go:884 +0x213
    github.com/docker/docker/daemon.(*Daemon).createSpec(0xc0006e0000, {0x2ea5588, 0xc00012a008}, 0xc0003b5900)
        /go/src/github.com/docker/docker/daemon/oci_linux.go:1060 +0xf33
    github.com/docker/docker/daemon.TestTmpfsDevShmNoDupMount(0xc000b781a0?)
        /go/src/github.com/docker/docker/daemon/oci_linux_test.go:77 +0x20a
    testing.tRunner(0xc000b78340, 0x2c74210)
        /usr/local/go/src/testing/testing.go:1576 +0x10b
    created by testing.(*T).Run
        /usr/local/go/src/testing/testing.go:1629 +0x3ea

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-18 15:02:41 +02:00
Sebastiaan van Stijn
1261fe69a3
API: deprecate VirtualSize field for /images/json and /images/{id}/json
In versions of Docker before v1.10, this field was calculated from
the image itself and all of its parent images. Images are now stored
self-contained, and no longer use a parent-chain, making this field
an equivalent of the Size field.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-18 14:46:11 +02:00
Sebastiaan van Stijn
20a1d23b39
Merge pull request #45320 from akerouanton/info-no-new-privileges
Add no-new-privileges to SecurityOptions returned by /info
2023-04-18 14:37:15 +02:00
Albin Kerouanton
eb7738221c
Add no-new-privileges to SecurityOptions returned by /info
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-04-18 09:34:08 +02:00
Sebastiaan van Stijn
e7980275c0
c8d: align "Size" and "VirtualSize" for images
In versions of Docker before v1.10, this field was calculated from
the image itself and all of its parent images. Images are now stored
self-contained, and no longer use a parent-chain, making this field
an equivalent of the Size field.

For the containerd integration, the Size should be the sum of the
image's compressed / packaged and unpacked (snapshots) layers.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-17 22:39:00 +02:00
Sebastiaan van Stijn
3903f16cd6
daemon: remove deprecated AuFS storage driver
There's still some locations refering to AuFS;

- pkg/archive: I suspect most of that code is because the whiteout-files
  are modelled after aufs (but possibly some code is only relevant to
  images created with AuFS as storage driver; to be looked into).
- contrib/apparmor/template: likely some rules can be removed
- contrib/dockerize-disk.sh: very old contribution, and unlikely used
  by anyone, but perhaps could be updated if we want to (or just removed).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-15 01:27:16 +02:00
Sebastiaan van Stijn
5bf405b2af
Merge pull request #45227 from PettitWesley/fix-awslogs-non-blocking
awslogs: fix non-blocking log drop bug
2023-04-15 00:56:57 +02:00
Djordje Lukic
bc37e3f830
Fix returning errors from snapshotter.Prepare
In the case of an error when calling snapshotter.Prepare we would return
nil. This change fixes that and returns the error from Prepare all the
time.

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-04-14 10:10:52 +02:00
Sebastiaan van Stijn
d69f9fe266
Merge pull request #45270 from vvoland/c8d-prune-upstream
c8d: Implement prune
2023-04-14 02:29:38 +02:00
Sebastiaan van Stijn
61656464d8
Merge pull request #45315 from thaJeztah/deprecate_oom_score_adjust
daemon: deprecate --oom-score-adjust for the daemon
2023-04-14 00:06:58 +02:00
Paweł Gronowski
f846349824
c8d/list: Implement until filter
It's not originally supported by image list, but we need it for `prune`
needs it, so `list` gets it for free.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-04-13 20:30:29 +02:00
Paweł Gronowski
d89590eab9
c8d: Implement prune
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-04-13 20:30:27 +02:00
Sebastiaan van Stijn
9bc78bdc5b
Merge pull request #45289 from vvoland/c8d-label-filter
c8d/list: Fix `label` (and add `label!`) filter
2023-04-13 18:01:23 +02:00
Sebastiaan van Stijn
93e5c99fff
Merge pull request #45324 from vvoland/image-children-error
images/Children: Return error
2023-04-13 18:00:47 +02:00
Sebastiaan van Stijn
9b83875c13
Merge pull request #45300 from thaJeztah/errdefs_alias
use consistent alias for containerd's errdefs package, and validate in CI
2023-04-13 16:20:49 +02:00
Paweł Gronowski
9cb5da400c
images/Children: Return error
Change return value in function signature and return fatal errors so
they can actually be reported to the caller instead of just being logged
to daemon log.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-04-13 16:20:02 +02:00
Sebastiaan van Stijn
45483a1d0d
Merge pull request #45299 from thaJeztah/cleanup_stats
daemon: move code related to stats together
2023-04-13 16:16:47 +02:00
Paweł Gronowski
edf802956e
c8d/list: Implement label and label! filter
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-04-13 15:39:52 +02:00
Sebastiaan van Stijn
5a922dc162
daemon: deprecate --oom-score-adjust for the daemon
The `oom-score-adjust` option was added in a894aec8d8,
to prevent the daemon from being OOM-killed before other processes. This
option was mostly added as a "convenience", as running the daemon as a
systemd unit was not yet common.

Having the daemon set its own limits is not best-practice, and something
better handled by the process-manager starting the daemon.

Commit cf7a5be0f2 fixed this option to allow
disabling it, and 2b8e68ef06 removed the default
score adjust.

This patch deprecates the option altogether, recommending users to set these
limits through the process manager used, such as the "OOMScoreAdjust" option
in systemd units.

With this patch:

    dockerd --oom-score-adjust=-500 --validate
    Flag --oom-score-adjust has been deprecated, and will be removed in the next release.
    configuration OK

    echo '{"oom-score-adjust":-500}' > /etc/docker/daemon.json
    dockerd
    INFO[2023-04-12T21:34:51.133389627Z] Starting up
    INFO[2023-04-12T21:34:51.135607544Z] containerd not running, starting managed containerd
    WARN[2023-04-12T21:34:51.135629086Z] DEPRECATED: The "oom-score-adjust" config parameter and the dockerd "--oom-score-adjust" option will be removed in the next release.

    docker info
    Client:
      Context:    default
      Debug Mode: false
    ...
    DEPRECATED: The "oom-score-adjust" config parameter and the dockerd "--oom-score-adjust" option will be removed in the next release

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-13 00:02:39 +02:00
Djordje Lukic
f696a1b3b3
Fix error checking when pulling from an insecure registry
The call to an unsecure registry doesn't return an error saying that the
"server gave an HTTP response to an HTTPS client" but a
tls.RecordHeaderError saying that the "first record does not look like a
TLS handshake", this changeset looks for the right error for that case.

This fixes the http fallback when using an insecure registry

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-04-12 21:56:52 +02:00
Sebastiaan van Stijn
860db986fc
Merge pull request #45267 from rumpl/c8d-fix-exec-user
c8d: Set the process user on exec
2023-04-11 16:03:40 +02:00
Sebastiaan van Stijn
74a0fdf961
Merge pull request #45265 from vvoland/c8d-children-upstream
c8d: Implement Children by comparing diff ids
2023-04-11 15:45:27 +02:00
Sebastiaan van Stijn
a5d46a15f5
split GetRepository from ImageService
The GetRepository method interacts directly with the registry, and does
not depend on the snapshotter, but is used for two purposes;

For the GET /distribution/{name:.*}/json route;
dd3b71d17c/api/server/router/distribution/backend.go (L11-L15)

And to satisfy the "executor.ImageBackend" interface as used by Swarm;
58c027ac8b/daemon/cluster/executor/backend.go (L77)

This patch removes the method from the ImageService interface, and instead
implements it through an composite struct that satisfies both interfaces,
and an ImageBackend() method is added to the daemon.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

remove GetRepository from ImageService

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-09 12:07:57 +02:00
Sebastiaan van Stijn
81e62af94a
use consistent alias for containerd's errdefs package
The signatures of functions in containerd's errdefs packages are very
similar to those in our own, and it's easy to accidentally use the wrong
package.

This patch uses a consistent alias for all occurrences of this import.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-08 19:30:33 +02:00
Sebastiaan van Stijn
f691b13450
daemon: move code related to stats together
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-08 19:00:01 +02:00
Djordje Lukic
41a230758c
c8d: Set the process user on exec
This change makes is possible to run `docker exec -u <UID> ...` when the
containerd integration is activated.

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-04-07 10:53:25 +02:00
Sebastiaan van Stijn
79bf167c0d
Merge pull request #45269 from vvoland/c8d-reference-filter
c8d/list: Implement `reference`
2023-04-06 23:30:55 +02:00
Paweł Gronowski
e0f36f9d8b
ImageService: Pass ctx to Children
This only makes the containerd ImageService implementation respect
context cancellation though.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-04-06 14:27:38 +02:00
Paweł Gronowski
bea751beb7
c8d: Implement Children by comparing diff ids
Implement Children method for containerd image store which makes the
`ancestor` filter work for `docker ps`. Checking if image is a children
of other image is implemented by comparing their rootfs diffids because
containerd image store doesn't have a concept of image parentship like
the graphdriver store. The child is expected to have more layers than
the parent and should start with all parent layers.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-04-06 14:27:37 +02:00
Paweł Gronowski
ba47cdcc0b
c8d/list: Implement reference filter
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-04-06 13:44:52 +02:00
Sebastiaan van Stijn
e8a79114b8
Merge pull request #45274 from thaJeztah/container_as_source_of_truth
c8d: use the container's configured snapshotter where possible
2023-04-05 17:03:18 +02:00
Sebastiaan van Stijn
465cbccdaf
c8d: use the container's configured snapshotter where possible
While we currently do not provide an option to specify the snapshotter to use
for individual containers (we may want to add this option in future), currently
it already is possible to configure the snapshotter in the daemon configuration,
which could (likely) cause issues when changing and restarting the daemon.

This patch updates some code-paths that have the container available to use
the snapshotter that's configured for the container (instead of the default
snapshotter configured).

There are still code-paths to be looked into, and a tracking ticket as well as
some TODO's were added for those.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-05 14:09:21 +02:00
Sebastiaan van Stijn
c2c83e16a2
Merge pull request #45266 from rumpl/c8d-tag-delete-events
c8d: Send an event when an image is tagged or deleted
2023-04-05 09:10:43 +02:00
Wesley Pettit
c8f8d11ac4
awslogs: fix non-blocking log drop bug
Previously, the AWSLogs driver attempted to implement
non-blocking itself. Non-blocking is supposed to
implemented solely by the Docker RingBuffer that
wraps the log driver.

Please see issue and explanation here:
https://github.com/moby/moby/issues/45217

Signed-off-by: Wesley Pettit <wppttt@amazon.com>
2023-04-04 17:03:06 -07:00
Djordje Lukic
1b178dffa3
Send an event when an image is deleted
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-04-04 15:54:11 +02:00
Djordje Lukic
36d28b73fc
Send an event when an image is tagged
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-04-04 11:32:02 +02:00
Cory Snider
750218c5e5 d/config: fix default-network-opts in daemon.json
Prevent the daemon from erroring out if daemon.json contains default
network options for network drivers aside from bridge. Configuring
defaults for the bridge driver previously worked by coincidence because
the unrelated CLI flag '--bridge' exists.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-04-03 15:59:53 -04:00
Sebastiaan van Stijn
816cd9802f
Merge pull request #45259 from rumpl/c8d-load-save-events
c8d: Send event when an image is imported
2023-04-03 20:05:45 +02:00
Sebastiaan van Stijn
781740c19d
Merge pull request #45124 from vvoland/c8d-list-ignore-attestations
c8d/list: Ignore attestation manifests
2023-04-03 19:56:29 +02:00
Paweł Gronowski
92e38b6e27
c8d/list: Ignore attestation manifests
Attestation manifests have an OCI image media type, which makes them
being listed like they were a separate platform supported by the image.

Don't use `images.Platforms` and walk the manifest list ourselves
looking for all manifests that are an actual image manifest.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-04-03 17:50:56 +02:00
Djordje Lukic
773a9d85a1
c8d: Send event when an image is exported
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-04-03 15:30:04 +02:00
Djordje Lukic
00545ef4d6
c8d: Send event when an image is imported
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-04-03 15:21:18 +02:00
Sebastiaan van Stijn
348f83670a
Merge pull request #45249 from vvoland/c8d-push-upstream-2
c8d/push: Follow up fixes
2023-03-31 22:37:59 +02:00
Sebastiaan van Stijn
126e212fb0
Merge pull request #45243 from thaJeztah/c8d_push_nits
c8d: push: addressing some nits
2023-03-31 19:28:43 +02:00
Paweł Gronowski
027f70562e
c8d/push: Rename jobs to avoid shadowing type
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-03-31 16:30:14 +02:00