Commit graph

249 commits

Author SHA1 Message Date
Nicolas De Loof
8fb71ce208
manage image inspect data in backend
This allows differentiating how the detailed data is collected between
the containerd-integration code and the existing implementation.

Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-09 19:02:12 +01:00
Paweł Gronowski
e214503789
image/store: Use errdefs for errors
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2022-12-08 17:17:07 +01:00
Sebastiaan van Stijn
7bbf2fb652
Merge pull request #44428 from thaJeztah/fix_usage_of_deprecated_funcs
Remove uses of deprecated go-digest.NewDigestFromHex, go-digest.Digest.Hex
2022-11-10 17:52:43 +01:00
Sebastiaan van Stijn
0f7c9cd27e
Remove uses of deprecated go-digest.NewDigestFromHex, go-digest.Digest.Hex
Both of these were deprecated in 55f675811a,
but the format of the GoDoc comments didn't follow the correct format, which
caused them not being picked up by tools as "deprecated".

This patch updates uses in the codebase to use the alternatives.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-08 16:42:13 +01:00
Sebastiaan van Stijn
6174d00c03
replace distribution/digestset with opencontainers/go-digest/digestset
opencontainers/go-digest is a 1:1 copy of the one in distribution. It's no
longer used in distribution itself, so may be removed there at some point.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-08 14:15:13 +01:00
Sebastiaan van Stijn
8a2e1245d4
runconfig, oci, image, layer, distribution: fix empty-lines (revive)
runconfig/config_test.go:23:46: empty-lines: extra empty line at the start of a block (revive)
    runconfig/config_test.go:75:55: empty-lines: extra empty line at the start of a block (revive)

    oci/devices_linux.go:57:34: empty-lines: extra empty line at the start of a block (revive)
    oci/devices_linux.go:60:69: empty-lines: extra empty line at the start of a block (revive)

    image/fs_test.go:53:38: empty-lines: extra empty line at the end of a block (revive)
    image/tarexport/save.go:88:29: empty-lines: extra empty line at the end of a block (revive)

    layer/layer_unix_test.go:21:34: empty-lines: extra empty line at the end of a block (revive)

    distribution/xfer/download.go:302:9: empty-lines: extra empty line at the end of a block (revive)
    distribution/manifest_test.go:154:99: empty-lines: extra empty line at the end of a block (revive)
    distribution/manifest_test.go:329:52: empty-lines: extra empty line at the end of a block (revive)
    distribution/manifest_test.go:354:59: empty-lines: extra empty line at the end of a block (revive)

    registry/config_test.go:323:42: empty-lines: extra empty line at the end of a block (revive)
    registry/config_test.go:350:33: empty-lines: extra empty line at the end of a block (revive)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-28 01:58:52 +02:00
Sebastiaan van Stijn
f6b695d2fb
use consistent alias for gotest.tools/v3/assert/cmp
Make sure we use the same alias everywhere for easier finding,
and to prevent accidentally introducing duplicate imports with
different aliases for the same package.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-08 16:45:16 +02:00
Sebastiaan van Stijn
509f19f611
replace pkg/system Sequential funcs with moby/sys/sequential
Migrating these functions to allow them being shared between moby, docker/cli,
and containerd, and to allow using them without importing all of sys / system,
which (in containerd) also depends on hcsshim and more.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-30 09:33:49 +02:00
Sebastiaan van Stijn
78095e4d12
Remove unused image/v1 code
This image format is only used for docker save / docker load.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-05-10 23:45:03 +02:00
Sebastiaan van Stijn
3c6c9fa180
Merge pull request #43300 from thaJeztah/api_update_imageinspect_docs
API: improve swagger and go-doc for ImageInspect structs
2022-03-05 17:54:37 +01:00
Sebastiaan van Stijn
a0230f3d9a
remove unneeded "digest" alias for "go-digest"
I think this was there for historic reasons (may have been goimports expected
this, and we used to have a linter that wanted it), but it's not needed, so
let's remove it (to make my IDE less complaining about unneeded aliases).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-04 14:49:42 +01:00
Sebastiaan van Stijn
06df530b6d
image: improve godoc for V1Image
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-03 22:57:03 +01:00
Sebastiaan van Stijn
1ef0a5bb91
image: remove leftover LCOW platform checks
This removes some of the checks that were added in 0cba7740d4,
but should no longer be needed.

- `image/store.Delete()`: no need to validate image platform to delete it.
- `image/tarexporter/takeLayerReference()`: use `image.OperatingSystem()` and
  fail early to prevent constructing the `ChainID()`.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-25 12:18:44 +01:00
haining.cao
715e78631d fix image spec JSON example
Signed-off-by: haining.cao <haining.cao@daocloud.io>
2021-11-22 23:37:23 +08:00
Eng Zer Jun
c55a4ac779
refactor: move from io/ioutil to io and os package
The io/ioutil package has been deprecated in Go 1.16. This commit
replaces the existing io/ioutil functions with their new definitions in
io and os packages.

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2021-08-27 14:56:57 +08:00
Sebastiaan van Stijn
13cb04e57c
remove various LCOW bits (container, image, pkg/containerfs)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-27 13:36:21 +02:00
Sebastiaan van Stijn
dc7cbb9b33
remove layerstore indexing by OS (used for LCOW)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-06-10 17:49:11 +02:00
Sebastiaan van Stijn
e047d984dc
Remove LCOW code (step 1)
The LCOW implementation in dockerd has been deprecated in favor of re-implementation
in containerd (in progress). Microsoft started removing the LCOW V1 code from the
build dependencies we use in Microsoft/opengcs (soon to be part of Microsoft/hcshhim),
which means that we need to start removing this code.

This first step removes the lcow graphdriver, the LCOW initialization code, and
some LCOW-related utilities.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-06-03 21:16:21 +02:00
Sebastiaan van Stijn
4a054ec00f
Merge pull request #41884 from rcowsill/41829-load-valid-parent
Fix spurious error from "docker load"
2021-02-22 22:00:29 +01:00
Sebastiaan van Stijn
1c39b1c44c
Merge pull request #41842 from jchorl/master
Reject null manifests during tar import
2021-02-09 12:06:27 +01:00
Josh Chorlton
654f854fae reject null manifests
Signed-off-by: Josh Chorlton <jchorlton@gmail.com>
2021-02-02 09:24:53 -08:00
Rob Cowsill
42dba8c3f9
Fix spurious error from "docker load"
"docker load" validates parent links by comparing image histories, and the
History struct has a time.Time member "Created". Time.UnmarshalJSON can read
RFC3339 timestamps with offset "+00:00", but t.MarshalJSON writes them with
offset "Z". Equivalent times in these two formats are not equal when compared
with the == operator.

This causes checkValidParent to incorrectly return false when the parent image
history contains times using offset "+00:00". In that case the history copied
to the child image will have been converted into "Z" form when marshaled out.

This patch adds an "Equal" method to History, which compares "Created" times
with t.Equal. This is used instead of reflect.DeepEqual in checkValidParent.

Signed-off-by: Rob Cowsill <42620235+rcowsill@users.noreply.github.com>
2021-01-21 22:04:23 +00:00
Jim Lin
c9ec21e17a replace json.Unmarshal with NewFromJSON in Create
Signed-off-by: Jim Lin <b04705003@ntu.edu.tw>
2021-01-11 23:01:44 +08:00
Sebastiaan van Stijn
dc3c382b34
replace pkg/symlink with github.com/moby/sys/symlink
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-11-03 11:17:12 +01:00
Sebastiaan van Stijn
9f0b3f5609
bump gotest.tools v3.0.1 for compatibility with Go 1.14
full diff: https://github.com/gotestyourself/gotest.tools/compare/v2.3.0...v3.0.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-02-11 00:06:42 +01:00
Sebastiaan van Stijn
cbc9de0abe
image: normalize comment formatting
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-11-27 15:45:08 +01:00
Chris Price
c21a3cf432 Add variant to image.Image and legacy builder
This commit adds the image variant to the image.(Image) type and
updates related functionality. Images built from another will
inherit the OS, architecture and variant.

Note that if a base image does not specify an architecture, the
local machine's architecture is used for inherited images. On the
other hand, the variant is set equal to the parent image's variant,
even when the parent image's variant is unset.

The legacy builder is also updated to allow the user to specify
a '--platform' argument on the command line when creating an image
FROM scratch. A complete platform specification, including variant,
is supported. The built image will include the variant, as will any
derived images.

Signed-off-by: Chris Price <chris.price@docker.com>
2019-09-24 22:18:16 +00:00
Sebastiaan van Stijn
d78b667af6
govet: suppress warning about Parent field that repeated json tag "parent"
```
image/image.go:65:2: structtag: struct field Parent repeats json tag "parent" also at image.go:39 (govet)
	Parent     ID        `json:"parent,omitempty"`
```

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-18 12:57:40 +02:00
Sebastiaan van Stijn
2c31edbbb6
unconvert: remove unnescessary conversions
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-18 12:57:33 +02:00
Sebastiaan van Stijn
07ff4f1de8
goimports: fix imports
Format the source according to latest goimports.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-18 12:56:54 +02:00
Sebastiaan van Stijn
8ad6dcf2a5
Merge pull request #38888 from quasilyte/fix_copying
image: do actual RootFS.DiffIDs copying in Clone()
2019-03-21 01:48:27 +01:00
Iskander (Alex) Sharipov
3429e99930 image: do actual RootFS.DiffIDs copying in Clone()
append(newRoot.DiffIDs) without element does nothing,
so it's probably not what was intended. Changed code
to perform a slice copying instead.

Fixes #38834.

Signed-off-by: Iskander Sharipov <quasilyte@gmail.com>
2019-03-18 01:05:40 +03:00
John Howard
20833b06a0 Windows: (WCOW) Generate OCI spec that remote runtime can escape
Signed-off-by: John Howard <jhoward@microsoft.com>

Also fixes https://github.com/moby/moby/issues/22874

This commit is a pre-requisite to moving moby/moby on Windows to using
Containerd for its runtime.

The reason for this is that the interface between moby and containerd
for the runtime is an OCI spec which must be unambigious.

It is the responsibility of the runtime (runhcs in the case of
containerd on Windows) to ensure that arguments are escaped prior
to calling into HCS and onwards to the Win32 CreateProcess call.

Previously, the builder was always escaping arguments which has
led to several bugs in moby. Because the local runtime in
libcontainerd had context of whether or not arguments were escaped,
it was possible to hack around in daemon/oci_windows.go with
knowledge of the context of the call (from builder or not).

With a remote runtime, this is not possible as there's rightly
no context of the caller passed across in the OCI spec. Put another
way, as I put above, the OCI spec must be unambigious.

The other previous limitation (which leads to various subtle bugs)
is that moby is coded entirely from a Linux-centric point of view.

Unfortunately, Windows != Linux. Windows CreateProcess uses a
command line, not an array of arguments. And it has very specific
rules about how to escape a command line. Some interesting reading
links about this are:

https://blogs.msdn.microsoft.com/twistylittlepassagesallalike/2011/04/23/everyone-quotes-command-line-arguments-the-wrong-way/
https://stackoverflow.com/questions/31838469/how-do-i-convert-argv-to-lpcommandline-parameter-of-createprocess
https://docs.microsoft.com/en-us/cpp/cpp/parsing-cpp-command-line-arguments?view=vs-2017

For this reason, the OCI spec has recently been updated to cater
for more natural syntax by including a CommandLine option in
Process.

What does this commit do?

Primary objective is to ensure that the built OCI spec is unambigious.

It changes the builder so that `ArgsEscaped` as commited in a
layer is only controlled by the use of CMD or ENTRYPOINT.

Subsequently, when calling in to create a container from the builder,
if follows a different path to both `docker run` and `docker create`
using the added `ContainerCreateIgnoreImagesArgsEscaped`. This allows
a RUN from the builder to control how to escape in the OCI spec.

It changes the builder so that when shell form is used for RUN,
CMD or ENTRYPOINT, it builds (for WCOW) a more natural command line
using the original as put by the user in the dockerfile, not
the parsed version as a set of args which loses fidelity.
This command line is put into args[0] and `ArgsEscaped` is set
to true for CMD or ENTRYPOINT. A RUN statement does not commit
`ArgsEscaped` to the commited layer regardless or whether shell
or exec form were used.
2019-03-12 18:41:55 -07:00
bevisy
65f669331a delete unnecessary blank lines
Signed-off-by: Bevisy Zhang <binbin36520@gmail.com>
2018-12-11 09:07:37 +08:00
Tonis Tiigi
f099771665 system: add back lcow validation function
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-06-27 15:24:26 -07:00
Dennis Chen
7f334d3acf Initial support for OCI multi-platform image
Add the OCI spec compatible image support in client side.

Signed-off-by: Dennis Chen <dennis.chen@arm.com>
2018-06-26 11:39:29 -07:00
John Howard
35193c0e7d LCOW: Auto-select OS
Signed-off-by: John Howard <jhoward@microsoft.com>

Addresses https://github.com/moby/moby/pull/35089#issuecomment-367802698.
This change enables the daemon to automatically select an image under LCOW
that can be used if the API doesn't specify an explicit platform.

For example:

FROM supertest2014/nyan
ADD Dockerfile /

And docker build . will download the linux image (not a multi-manifest image)

And similarly docker pull ubuntu will match linux/amd64
2018-06-26 11:37:43 -07:00
Vincent Demeester
3845728524
Update tests to use gotest.tools 👼
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2018-06-13 09:04:30 +02:00
Sebastiaan van Stijn
f23c00d870
Various code-cleanup
remove unnescessary import aliases, brackets, and so on.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-05-23 17:50:54 +02:00
Sebastiaan van Stijn
55bebbaecf
Replace deprecated testutil.ErrorContains()
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-05-21 00:13:04 +02:00
Alex Goodman
4fb59c20a4
image spec formatting fix
Embedded new lines aren't interpreted correctly in markdown renderers (they are treated as preformatted text instead). I removed the embedded newlines in the docker image spec.

Signed-off-by: Alex Goodman <wagoodman@gmail.com>
2018-05-13 13:30:53 -04:00
Sebastiaan van Stijn
fef1a5ea80
Merge pull request #36816 from arm64b/using-parent-img-arch4dockerbuilder
Dockerbuilder: use the arch info from base image
2018-04-26 20:06:40 +02:00
Anusha Ragunathan
65c575f592
Merge pull request #36599 from Microsoft/jjh/dockersave
LCOW: Implement `docker save`
2018-04-24 11:14:32 -07:00
Dennis Chen
92b17b10ba Dockerbuilder: use the arch info from base image
Currently we hardcode the architecture to the `runtime.GOARCH` when
building a docker image, this will result in a confusing info if the
arch in the base image is different from the one on the host.

This PR takes use of the arch data from the base image during the build
process, thus we can get consistent arch info between the base image
and the finally built image.

Signed-off-by: Dennis Chen <dennis.chen@arm.com>
2018-04-09 10:40:21 +00:00
Sebastiaan van Stijn
2e3056128c
Merge pull request #35149 from AkihiroSuda/image-v1
image/spec: add historical information about v1 spec
2018-03-23 16:47:16 +01:00
Daniel Nephin
c9e52bd0da Post migration assertion fixes
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2018-03-16 11:03:46 -04:00
Daniel Nephin
6be0f70983 Automated migration using
gty-migrate-from-testify --ignore-build-tags

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2018-03-16 11:03:43 -04:00
John Howard
0a49de4eb5 LCOW: Write saved manifest.json in Unix paths
Signed-off-by: John Howard <jhoward@microsoft.com>
2018-03-14 11:56:00 -07:00
John Howard
d5c781323d LCOW: Make sure OS is copied for docker save
Signed-off-by: John Howard <jhoward@microsoft.com>
2018-03-14 11:46:46 -07:00
John Howard
d8dbba14fd LCOW: Check OS in takeLayerReference
Signed-off-by: John Howard <jhoward@microsoft.com>
2018-03-14 11:46:35 -07:00