Commit graph

140 commits

Author SHA1 Message Date
Djordje Lukic
8bbfa32741
c8d: The authorizer needs to be set even if AuthConfig is empty
Without the authorizer pulling will fail if the user is not logged-in

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-05-12 09:47:32 +02:00
Sebastiaan van Stijn
807e415260
Merge pull request #45526 from laurazard/backport-classic-builder
[24.0 backport] c8d: implement classic builder
2023-05-11 21:12:01 +02:00
Laura Brehm
8587a1c617
c8d/builder: implement cache
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
(cherry picked from commit bd6868557d)
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
2023-05-11 19:13:51 +01:00
Laura Brehm
9717369913
c8d: implement classic builder
Co-authored-by: Djordje Lukic <djordje.lukic@docker.com>
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
(cherry picked from commit e46674b6a7)
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
2023-05-11 19:13:13 +01:00
Paweł Gronowski
233c49438b
c8d: Don't create authorizer for empty AuthConfig
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 3309e45ca1)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-11 17:44:06 +02:00
Paweł Gronowski
2b7424512a
c8d/authorizer: Default to docker.io
When the `ServerAddress` in the `AuthConfig` provided by the client is
empty, default to the default registry (registry-1.docker.io).

This makes the behaviour the same as with the containerd image store
integration disabled.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 2ad499f93e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-11 17:43:46 +02:00
Sebastiaan van Stijn
c01f02cfcb
Merge pull request #45515 from laurazard/cherry-pick-image-delete
[24.0 backport] c8d: implement missing image delete logic
2023-05-10 22:16:18 +02:00
Laura Brehm
1235338836
c8d: implement missing image delete logic
Ports over all the previous image delete logic, such as:
- Introduce `prune` and `force` flags
- Introduce the concept of hard and soft image delete conflics, which represent:
  - image referenced in multiple tags (soft conflict)
  - image being used by a stopped container (soft conflict)
  - image being used by a running container (hard conflict)
- Implement delete logic such as:
  - if deleting by reference, and there are other references to the same image, just
    delete the passed reference
  - if deleting by reference, and there is only 1 reference and the image is being used
    by a running container, throw an error if !force, or delete the reference and create
    a dangling reference otherwise
  - if deleting by imageID, and force is true, remove all tags (otherwise soft conflict)
  - if imageID, check if stopped container is using the image (soft conflict), and
    delete anyway if force
  - if imageID was passed in, check if running container is using the image (hard conflict)
  - if `prune` is true, and the image being deleted has dangling parents, remove them

This commit also implements logic to get image parents in c8d by comparing shared layers.

Signed-off-by: Laura Brehm <laurabrehm@hey.com>
(cherry picked from commit cad97135b3)
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
2023-05-10 17:52:42 +01:00
Laura Brehm
763d2b7996
c8d: fix image history for dangling images
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
(cherry picked from commit 4603b6d6b6)
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
2023-05-10 15:03:26 +01:00
Laura Brehm
86770904be
c8d: fix missing image history
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
(cherry picked from commit e8be792130)
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
2023-05-09 18:10:06 +01:00
Sebastiaan van Stijn
2adec6c8c0
Merge pull request #45438 from thaJeztah/c8d_fix_image_commit
c8d: commit: generateCommitImageConfig: don't merge image config
2023-05-04 11:28:43 +02:00
Sebastiaan van Stijn
9613da63dc
c8d: commit: generateCommitImageConfig: don't merge image config
daemon.CreateImageFromContainer() already constructs a new config by taking
the image config, applying custom options (`docker commit --change ..`) (if
any), and merging those with the containers' configuration, so there is
no need to merge options again.

e22758bfb2/daemon/commit.go (L152-L158)

This patch removes the merge logic from generateCommitImageConfig, and
removes the unused arguments and error-return.

Co-authored-by: Djordje Lukic <djordje.lukic@docker.com>
Co-authored-by: Laura Brehm <laurabrehm@hey.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-29 02:53:54 +02:00
Sebastiaan van Stijn
e510abbc8c
daemon/containerd: containerConfigToOciImageConfig: add ArgsEscaped
The OCI image-spec now also provides ArgsEscaped for backward compatibility
with the option used by Docker.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-29 00:20:07 +02:00
Brian Goff
dffad6b0b7
Merge pull request #45426 from vvoland/c8d-import-dangling
c8d/import: Don't gc unnamed images
2023-04-28 11:18:55 -07:00
Sebastiaan van Stijn
3cad7f99e0
Merge pull request #45431 from laurazard/remove-dangling-pull
c8d: delete dangling image on pull
2023-04-28 19:52:55 +02:00
Laura Brehm
44a6587608
c8d: delete dangling image on pull
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
2023-04-28 17:41:56 +01:00
Paweł Gronowski
1845a915b0
c8d/import: Don't gc unnamed images
Create dangling images for imported images which don't have a name
annotation attached. Previously the content got loaded, but no image
referencing it was created which caused it to be garbage collected
immediately.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-04-28 09:30:04 +02:00
Sebastiaan van Stijn
79dd264517
Merge pull request #45339 from vvoland/c8d-prune-upstream-gc
c8d/prune: Remove gc.ref labels from configs of deleted images
2023-04-26 22:54:10 +02:00
Sebastiaan van Stijn
8ff481b436
Merge pull request #45393 from vvoland/c8d-fix-panic-imagemanifest
containerd integration: Handle nil container.ImageManifest
2023-04-26 17:02:06 +02:00
Paweł Gronowski
520aa08d42
c8d/prune: Logging fixes
- Use logrus.Fields instead of multiple WithField
- Split one giant debug log into one log per image

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-04-26 16:40:37 +02:00
Paweł Gronowski
52af6d957e
c8d/prune: Remove gc.ref labels from configs of deleted images
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-04-26 16:40:34 +02:00
Paweł Gronowski
9fe5ac6614
c8d: Handle nil container.ImageManifest
Don't panic when processing containers created under fork containerd
integration (this field was added in the upstream and didn't exist in
fork).

Co-authored-by: Djordje Lukic <djordje.lukic@docker.com>
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-04-26 14:54:01 +02:00
Sebastiaan van Stijn
2379b49177
Merge pull request #45362 from laurazard/c8d-delete-short-images
c8d: add support for removing images by shortID
2023-04-26 11:34:34 +02:00
Sebastiaan van Stijn
801202349a
Merge pull request #45387 from cuishuang/master
fix some comments
2023-04-25 16:31:22 +02:00
Sebastiaan van Stijn
a343ed13e5
Merge pull request #45390 from vvoland/c8d-imagelist-fix-rootlen
c8d/list: Initialize capacity instead of length
2023-04-25 12:26:41 +02:00
Paweł Gronowski
391f9635cc
c8d/list: Initialize capacity instead of length
The slice which stores chain ids used for computing shared size was
mistakenly initialized with the length set instead of the capacity.
This caused a panic when iterating over it later and dereferncing nil
pointer from empty items.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-04-25 10:38:23 +02:00
cui fliter
f66684fdeb fix some comments
Signed-off-by: cui fliter <imcusg@gmail.com>
2023-04-25 13:39:28 +08:00
Laura Brehm
8df3db4b97
c8d: add support for removing images by shortID
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
2023-04-21 10:30:53 +01:00
Sebastiaan van Stijn
1261fe69a3
API: deprecate VirtualSize field for /images/json and /images/{id}/json
In versions of Docker before v1.10, this field was calculated from
the image itself and all of its parent images. Images are now stored
self-contained, and no longer use a parent-chain, making this field
an equivalent of the Size field.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-18 14:46:11 +02:00
Sebastiaan van Stijn
e7980275c0
c8d: align "Size" and "VirtualSize" for images
In versions of Docker before v1.10, this field was calculated from
the image itself and all of its parent images. Images are now stored
self-contained, and no longer use a parent-chain, making this field
an equivalent of the Size field.

For the containerd integration, the Size should be the sum of the
image's compressed / packaged and unpacked (snapshots) layers.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-17 22:39:00 +02:00
Djordje Lukic
bc37e3f830
Fix returning errors from snapshotter.Prepare
In the case of an error when calling snapshotter.Prepare we would return
nil. This change fixes that and returns the error from Prepare all the
time.

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-04-14 10:10:52 +02:00
Paweł Gronowski
f846349824
c8d/list: Implement until filter
It's not originally supported by image list, but we need it for `prune`
needs it, so `list` gets it for free.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-04-13 20:30:29 +02:00
Paweł Gronowski
d89590eab9
c8d: Implement prune
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-04-13 20:30:27 +02:00
Sebastiaan van Stijn
9bc78bdc5b
Merge pull request #45289 from vvoland/c8d-label-filter
c8d/list: Fix `label` (and add `label!`) filter
2023-04-13 18:01:23 +02:00
Paweł Gronowski
9cb5da400c
images/Children: Return error
Change return value in function signature and return fatal errors so
they can actually be reported to the caller instead of just being logged
to daemon log.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-04-13 16:20:02 +02:00
Paweł Gronowski
edf802956e
c8d/list: Implement label and label! filter
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-04-13 15:39:52 +02:00
Djordje Lukic
f696a1b3b3
Fix error checking when pulling from an insecure registry
The call to an unsecure registry doesn't return an error saying that the
"server gave an HTTP response to an HTTPS client" but a
tls.RecordHeaderError saying that the "first record does not look like a
TLS handshake", this changeset looks for the right error for that case.

This fixes the http fallback when using an insecure registry

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-04-12 21:56:52 +02:00
Sebastiaan van Stijn
74a0fdf961
Merge pull request #45265 from vvoland/c8d-children-upstream
c8d: Implement Children by comparing diff ids
2023-04-11 15:45:27 +02:00
Sebastiaan van Stijn
a5d46a15f5
split GetRepository from ImageService
The GetRepository method interacts directly with the registry, and does
not depend on the snapshotter, but is used for two purposes;

For the GET /distribution/{name:.*}/json route;
dd3b71d17c/api/server/router/distribution/backend.go (L11-L15)

And to satisfy the "executor.ImageBackend" interface as used by Swarm;
58c027ac8b/daemon/cluster/executor/backend.go (L77)

This patch removes the method from the ImageService interface, and instead
implements it through an composite struct that satisfies both interfaces,
and an ImageBackend() method is added to the daemon.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

remove GetRepository from ImageService

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-09 12:07:57 +02:00
Sebastiaan van Stijn
79bf167c0d
Merge pull request #45269 from vvoland/c8d-reference-filter
c8d/list: Implement `reference`
2023-04-06 23:30:55 +02:00
Paweł Gronowski
e0f36f9d8b
ImageService: Pass ctx to Children
This only makes the containerd ImageService implementation respect
context cancellation though.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-04-06 14:27:38 +02:00
Paweł Gronowski
bea751beb7
c8d: Implement Children by comparing diff ids
Implement Children method for containerd image store which makes the
`ancestor` filter work for `docker ps`. Checking if image is a children
of other image is implemented by comparing their rootfs diffids because
containerd image store doesn't have a concept of image parentship like
the graphdriver store. The child is expected to have more layers than
the parent and should start with all parent layers.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-04-06 14:27:37 +02:00
Paweł Gronowski
ba47cdcc0b
c8d/list: Implement reference filter
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-04-06 13:44:52 +02:00
Sebastiaan van Stijn
e8a79114b8
Merge pull request #45274 from thaJeztah/container_as_source_of_truth
c8d: use the container's configured snapshotter where possible
2023-04-05 17:03:18 +02:00
Sebastiaan van Stijn
465cbccdaf
c8d: use the container's configured snapshotter where possible
While we currently do not provide an option to specify the snapshotter to use
for individual containers (we may want to add this option in future), currently
it already is possible to configure the snapshotter in the daemon configuration,
which could (likely) cause issues when changing and restarting the daemon.

This patch updates some code-paths that have the container available to use
the snapshotter that's configured for the container (instead of the default
snapshotter configured).

There are still code-paths to be looked into, and a tracking ticket as well as
some TODO's were added for those.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-05 14:09:21 +02:00
Djordje Lukic
1b178dffa3
Send an event when an image is deleted
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-04-04 15:54:11 +02:00
Djordje Lukic
36d28b73fc
Send an event when an image is tagged
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-04-04 11:32:02 +02:00
Sebastiaan van Stijn
816cd9802f
Merge pull request #45259 from rumpl/c8d-load-save-events
c8d: Send event when an image is imported
2023-04-03 20:05:45 +02:00
Sebastiaan van Stijn
781740c19d
Merge pull request #45124 from vvoland/c8d-list-ignore-attestations
c8d/list: Ignore attestation manifests
2023-04-03 19:56:29 +02:00
Paweł Gronowski
92e38b6e27
c8d/list: Ignore attestation manifests
Attestation manifests have an OCI image media type, which makes them
being listed like they were a separate platform supported by the image.

Don't use `images.Platforms` and walk the manifest list ourselves
looking for all manifests that are an actual image manifest.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-04-03 17:50:56 +02:00