Commit graph

305 commits

Author SHA1 Message Date
Paweł Gronowski
e8496b1ee4
imageService: Extract common code from MakeImageCache
Both containerd and graphdriver image service use the same code to
create the cache - they only supply their own `cacheAdaptor` struct.

Extract the shared code to `cache.New`.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-07 15:39:42 +01:00
Paweł Gronowski
d66177591e
c8d/cache: Use the same cache logic as graphdrivers
Implement the cache adaptor for containerd image store and use the same
cache logic.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-07 14:58:06 +01:00
Paweł Gronowski
bf30fee58a
image/cache: Refactor backend specific code
Move image store backend specific code out of the cache code and move it
to a separate interface to allow using the same cache code with
containerd image store.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-07 14:58:04 +01:00
Sebastiaan van Stijn
03a17a2887
migrate image spec to github.com/moby/docker-image-spec
The specification was migrated to a separate module:
https://github.com/moby/docker-image-spec

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-09 19:12:18 +01:00
Sebastiaan van Stijn
b71c2792d2
image/spec: remove link to docs.docker.com "registry" specification
This spec is not directly relevant for the image spec, and the Docker
documentation no longer includes the actual specification.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-08 17:39:18 +01:00
Paweł Gronowski
2c01d53d96
image/cache: Use Platform from ocispec
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-06 14:26:51 +01:00
Sebastiaan van Stijn
203ffb1c09
Merge pull request #47330 from vvoland/cache-fix-older-windows
image/cache: Ignore Build and Revision on Windows
2024-02-06 13:02:00 +01:00
Sebastiaan van Stijn
2156635843
Merge pull request #47232 from vvoland/fix-save-manifests
image/save: Fix untagged images not present in index.json
2024-02-05 19:06:54 +01:00
Paweł Gronowski
91ea04089b
image/cache: Ignore Build and Revision on Windows
The compatibility depends on whether `hyperv` or `process` container
isolation is used.
This fixes cache not being used when building images based on older
Windows versions on a newer Windows host.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-05 17:41:21 +01:00
Paweł Gronowski
d131f00fff
image/save: Fix untagged images not present in index.json
Saving an image via digested reference, ID or truncated ID doesn't store
the image reference in the archive. This also causes the save code to
not add the image's manifest to the index.json.
This commit explicitly adds the untagged manifests to the index.json if
no tagged manifests were added.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-29 16:26:37 +01:00
Paweł Gronowski
877ebbe038
image/cache: Check image platform
Make sure the cache candidate platform matches the requested.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-25 16:04:05 +01:00
Paweł Gronowski
96ac22768a
image/cache: Restrict cache candidates to locally built images
Restrict cache candidates only to images that were built locally.
This doesn't affect builds using `--cache-from`.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-25 16:04:03 +01:00
Paweł Gronowski
c6156dc51b
daemon/imageStore: Mark images built locally
Store additional image property which makes it possible to distinguish
if image was built locally.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-25 16:04:00 +01:00
Paweł Gronowski
537348763f
image/cache: Compare all config fields
Add checks for some image config fields that were missing.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-25 16:03:58 +01:00
Sebastiaan van Stijn
f7e2357745
image: remove deprecated IDFromDigest
This function was deprecated in 456ea1bb1d
(Docker v24.0), and is no longer used.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-24 01:54:59 +01:00
Paweł Gronowski
17fd6562bf
image/save: Fix layers order in OCI manifest
Order the layers in OCI manifest by their actual apply order. This is
required by the OCI image spec.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-22 13:48:12 +01:00
Paweł Gronowski
4979605212
image/save: Change layers type to DiffID
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-22 13:47:57 +01:00
Paweł Gronowski
9f50e06993
image/save: Refactor saveLayer
`diffIDPaths` is not used and can be removed.

`savedConfig` stores if the config was already saved (ID of the image is
the ID of the config).
`savedLayers` stores if the layer (diff ID) was already saved.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-16 11:14:15 +01:00
Paweł Gronowski
c58acf06c2 image/save: Derive Descriptor from actual tar archive
Layer size is the sum of the individual files count, not the tar
archive. Use the total bytes read returned by `io.Copy` to populate the
`Size` field.

Also set the digest to the actual digest of the tar archive.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-15 16:35:54 +01:00
Paweł Gronowski
2c45df2ab4 image/save: Fix missing layers in manifest
The new OCI-compatible archive export relies on the Descriptors returned
by the layer (`distribution.Describable` interface implementation).

The issue with that is that the `roLayer` and the `referencedCacheLayer`
types don't implement this interface. Implementing that interface for
them based on their `descriptor` doesn't work though, because that
descriptor is empty.

To workaround this issue, just create a new descriptor if the one
provided by the layer is empty.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-12 17:10:23 +01:00
Ben Lovy
341e7b35a6 Updated manifest v2 URLs in image spec README
Signed-off-by: Ben Lovy <ben@deciduously.com>
2023-12-28 16:13:26 -05:00
Sebastiaan van Stijn
f05dce8d3a
replace some strings with OCI consts
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-17 16:34:52 +02:00
Sebastiaan van Stijn
cff4f20c44
migrate to github.com/containerd/log v0.1.0
The github.com/containerd/containerd/log package was moved to a separate
module, which will also be used by upcoming (patch) releases of containerd.

This patch moves our own uses of the package to use the new module.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-11 17:52:23 +02:00
Sebastiaan van Stijn
a3c97beee0
image: implement CheckOS, deprecate pkg/system IsOSSupported
Implement a function that returns an error to replace existing uses of
the IsOSSupported utility, where callers had to produce the error after
checking.

The IsOSSupported function was used in combination with images, so implementing
a utility in "image" to prevent having to import pkg/system (which contains many
unrelated functions)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-07 22:14:44 +02:00
Sebastiaan van Stijn
1148a24e64
migrate to new github.com/distribution/reference module
The "reference" package was moved to a separate module, which was extracted
from b9b19409cf

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-05 12:09:26 +02:00
Sebastiaan van Stijn
9c4e82435e
Merge pull request #46351 from thaJeztah/api_events_actions_enum
api/types/events: define "Action" type and consts
2023-09-05 11:11:42 +02:00
Paweł Gronowski
366a5f1d74
image/spec: Add Go structs
Add Go structs describing the image spec which extend the OCI types.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-08-31 12:29:45 +02:00
Sebastiaan van Stijn
e9b3a16cdc
Merge pull request #46381 from vvoland/image-spec-healthcheck-startinterval
image/spec: Add Healthcheck.StartInterval (v1.3)
2023-08-31 12:23:32 +02:00
Paweł Gronowski
536826d66f
image/spec: Add Healthcheck.StartInterval (v1.3)
This field was added in 2216d3ca8d.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-08-31 11:22:54 +02:00
Paweł Gronowski
6dd78a9ac2
image/spec: Fix typo in "superseded"
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-08-31 10:35:14 +02:00
Sebastiaan van Stijn
d864d1eb97
image/spec: add back versioned files
Add back files at the old locations, as there may be external links
referencing the specification.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-30 16:17:17 +02:00
Sebastiaan van Stijn
7aa0bc5790
merge image-spec v1.2 into specs.md
This merges the v1.2 specs to provide a single history of the
specification.

To view the combined history:

    git log --follow image/spec/spec.md

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-30 16:11:14 +02:00
Sebastiaan van Stijn
823d0b3765
merge image-spec v1.1 into specs.md
This merges the v1.1 specs to provide a single history of the
specification.

To view the combined history:

    git log --follow image/spec/spec.md
2023-08-30 16:09:30 +02:00
Sebastiaan van Stijn
b8de6c01b5
image/spec: rename v1.2.md to spec.md
In preparation of merging the versioned-specs into a single file,
with history.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-30 16:06:36 +02:00
Sebastiaan van Stijn
c3f2bd1009
image/spec: rename v1.1.md to spec.md
In preparation of merging the versioned-specs into a single file,
with history.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-30 16:06:07 +02:00
Sebastiaan van Stijn
7ff38506ab
image/spec: rename v1.0.md to spec.md
In preparation of merging the versioned-specs into a single file,
with history.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-30 16:05:34 +02:00
Sebastiaan van Stijn
32d5e6312d
image/spec: use permalink to versioned documents
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-30 16:04:08 +02:00
Sebastiaan van Stijn
7b7019a4da
image/spec: add missing "Shell" field (v1.2)
This field was added in b18ae8c9cc, which
was part of v1.12.0-rc1 and later, which used image spec v1.2.0.

This patch amends the v1.2 spec to include the missing field.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-30 14:33:11 +02:00
Sebastiaan van Stijn
5380f3f0c4
image/spec: add missing "ArgsEscaped" field (v1.1, v1.2)
This field was added in 9db5db1b94, which
was part of v1.10.0-rc1 and later, which used image spec v1.1.0.

It's worth noting that documentation for the v1.1.0 image spec was not
yet available until commit 4fa0eccd10,
which was included in v1.12.0-rc1 and up. The `ArgsEscaped` field was
also adopted by the OCI image spec since [v1.1.0-rc3][1], but considered
deprecated, and not recommended to be used.

This patch amends the v1.1 and v1.2 specifications to describe the field.

[1]: 59780aa569

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-30 13:25:00 +02:00
Sebastiaan van Stijn
f33370219b
image/spec: add missing "OnBuild" field
This field was added in commit 9f994c9646,
which was merged before the image-spec v1.0.0 was released (which happened
in commit 79910625f0).

This patch backfills the specifications to describe the property.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-30 13:04:26 +02:00
Sebastiaan van Stijn
e19edd3f25
image/spec: minor fixes and touch-ups in markdown files
- remove some trailing commas, which made the JSON invalid (some of these
  were fixed in the 1.2 spec, but not in older versions).
- synchronise some formatting / phrasing between versions, to make them
  easier to compare.
- remove non-breaking spaces (`NBSP`) in example outputs, and replace
  them with regular spaces.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-30 11:18:20 +02:00
Sebastiaan van Stijn
0f871f8cb7
api/types/events: define "Action" type and consts
Define consts for the Actions we use for events, instead of "ad-hoc" strings.
Having these consts makes it easier to find where specific events are triggered,
makes the events less error-prone, and allows documenting each Action (if needed).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-29 00:38:08 +02:00
Sebastiaan van Stijn
87cedbe443
image: format code with gofumpt
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-24 17:57:22 +02:00
Paweł Gronowski
01214bafd2
c8d/builder: Don't drop fields from created image
Previous image created a new partially filled image.
This caused child images to lose their parent's layers.

Instead of creating a new object and trying to replace its fields, just
clone the original passed image and change its ID to the manifest
digest.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-08-24 09:44:12 +02:00
Sebastiaan van Stijn
5e2a1195d7
swap logrus types for their containerd/logs aliases
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-01 13:02:55 +02:00
Brian Goff
74da6a6363 Switch all logging to use containerd log pkg
This unifies our logging and allows us to propagate logging and trace
contexts together.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2023-06-24 00:23:44 +00:00
Tianon Gravi
2a6ff3c24f Use OCI "History" type instead of inventing our own copy
The most notable change here is that the OCI's type uses a pointer for `Created`, which we probably should've been too, so most of these changes are accounting for that (and embedding our `Equal` implementation in the one single place it was used).

Signed-off-by: Tianon Gravi <admwiggin@gmail.com>
2023-06-12 13:47:17 -07:00
Brian Goff
1e55ace875 Add oci index and layout files to exported tars
This makes the output of `docker save` fully OCI compliant.

When using the containerd image store, this code is not used. That
exporter will just use containerd's export method and should give us the
output we want for multi-arch images.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2023-05-28 20:35:14 +00:00
Brian Goff
ddd67b2535 Make image save more like the OCI layout for blobs
This moves the blobs around so they follow the OCI spec.
Note that because docker reads paths from the manifest.json inside the
tar this is not a breaking change.

This does, however, remove the old layer "VERSION" file which had a big
"why is this even here" in the code comments. I suspect it does not
matter at all even for really old versions of Docker. In any case it is
a useless file for any even relatively modern version of Docker.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2023-05-22 22:29:29 +00:00
arcosx
521c2b25c2 docs: fix JSON format error
Signed-off-by: arcosx <arcosx@outlook.com>
2023-02-03 08:16:48 +00:00