Commit graph

73 commits

Author SHA1 Message Date
Sebastiaan van Stijn
f10c50958c
replace pkg/system Sequential funcs with moby/sys/sequential
Migrating these functions to allow them being shared between moby, docker/cli,
and containerd, and to allow using them without importing all of sys / system,
which (in containerd) also depends on hcsshim and more.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 509f19f611)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-30 09:34:33 +02:00
Sebastiaan van Stijn
a0230f3d9a
remove unneeded "digest" alias for "go-digest"
I think this was there for historic reasons (may have been goimports expected
this, and we used to have a linter that wanted it), but it's not needed, so
let's remove it (to make my IDE less complaining about unneeded aliases).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-04 14:49:42 +01:00
Sebastiaan van Stijn
1ef0a5bb91
image: remove leftover LCOW platform checks
This removes some of the checks that were added in 0cba7740d4,
but should no longer be needed.

- `image/store.Delete()`: no need to validate image platform to delete it.
- `image/tarexporter/takeLayerReference()`: use `image.OperatingSystem()` and
  fail early to prevent constructing the `ChainID()`.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-25 12:18:44 +01:00
Eng Zer Jun
c55a4ac779
refactor: move from io/ioutil to io and os package
The io/ioutil package has been deprecated in Go 1.16. This commit
replaces the existing io/ioutil functions with their new definitions in
io and os packages.

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2021-08-27 14:56:57 +08:00
Sebastiaan van Stijn
13cb04e57c
remove various LCOW bits (container, image, pkg/containerfs)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-27 13:36:21 +02:00
Sebastiaan van Stijn
dc7cbb9b33
remove layerstore indexing by OS (used for LCOW)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-06-10 17:49:11 +02:00
Sebastiaan van Stijn
e047d984dc
Remove LCOW code (step 1)
The LCOW implementation in dockerd has been deprecated in favor of re-implementation
in containerd (in progress). Microsoft started removing the LCOW V1 code from the
build dependencies we use in Microsoft/opengcs (soon to be part of Microsoft/hcshhim),
which means that we need to start removing this code.

This first step removes the lcow graphdriver, the LCOW initialization code, and
some LCOW-related utilities.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-06-03 21:16:21 +02:00
Sebastiaan van Stijn
4a054ec00f
Merge pull request #41884 from rcowsill/41829-load-valid-parent
Fix spurious error from "docker load"
2021-02-22 22:00:29 +01:00
Josh Chorlton
654f854fae reject null manifests
Signed-off-by: Josh Chorlton <jchorlton@gmail.com>
2021-02-02 09:24:53 -08:00
Rob Cowsill
42dba8c3f9
Fix spurious error from "docker load"
"docker load" validates parent links by comparing image histories, and the
History struct has a time.Time member "Created". Time.UnmarshalJSON can read
RFC3339 timestamps with offset "+00:00", but t.MarshalJSON writes them with
offset "Z". Equivalent times in these two formats are not equal when compared
with the == operator.

This causes checkValidParent to incorrectly return false when the parent image
history contains times using offset "+00:00". In that case the history copied
to the child image will have been converted into "Z" form when marshaled out.

This patch adds an "Equal" method to History, which compares "Created" times
with t.Equal. This is used instead of reflect.DeepEqual in checkValidParent.

Signed-off-by: Rob Cowsill <42620235+rcowsill@users.noreply.github.com>
2021-01-21 22:04:23 +00:00
Sebastiaan van Stijn
dc3c382b34
replace pkg/symlink with github.com/moby/sys/symlink
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-11-03 11:17:12 +01:00
Sebastiaan van Stijn
cbc9de0abe
image: normalize comment formatting
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-11-27 15:45:08 +01:00
Sebastiaan van Stijn
07ff4f1de8
goimports: fix imports
Format the source according to latest goimports.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-18 12:56:54 +02:00
Tonis Tiigi
f099771665 system: add back lcow validation function
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-06-27 15:24:26 -07:00
Dennis Chen
7f334d3acf Initial support for OCI multi-platform image
Add the OCI spec compatible image support in client side.

Signed-off-by: Dennis Chen <dennis.chen@arm.com>
2018-06-26 11:39:29 -07:00
Sebastiaan van Stijn
f23c00d870
Various code-cleanup
remove unnescessary import aliases, brackets, and so on.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-05-23 17:50:54 +02:00
John Howard
0a49de4eb5 LCOW: Write saved manifest.json in Unix paths
Signed-off-by: John Howard <jhoward@microsoft.com>
2018-03-14 11:56:00 -07:00
John Howard
d5c781323d LCOW: Make sure OS is copied for docker save
Signed-off-by: John Howard <jhoward@microsoft.com>
2018-03-14 11:46:46 -07:00
John Howard
d8dbba14fd LCOW: Check OS in takeLayerReference
Signed-off-by: John Howard <jhoward@microsoft.com>
2018-03-14 11:46:35 -07:00
Daniel Nephin
4f0d95fa6e Add canonical import comment
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2018-02-05 16:51:57 -05:00
John Howard
afd305c4b5 LCOW: Refactor to multiple layer-stores based on feedback
Signed-off-by: John Howard <jhoward@microsoft.com>
2018-01-18 08:31:05 -08:00
John Howard
ce8e529e18 LCOW: Re-coalesce stores
Signed-off-by: John Howard <jhoward@microsoft.com>

The re-coalesces the daemon stores which were split as part of the
original LCOW implementation.

This is part of the work discussed in https://github.com/moby/moby/issues/34617,
in particular see the document linked to in that issue.
2018-01-18 08:29:19 -08:00
John Howard
0380fbff37 LCOW: API: Add platform to /images/create and /build
Signed-off-by: John Howard <jhoward@microsoft.com>

This PR has the API changes described in https://github.com/moby/moby/issues/34617.
Specifically, it adds an HTTP header "X-Requested-Platform" which is a JSON-encoded
OCI Image-spec `Platform` structure.

In addition, it renames (almost all) uses of a string variable platform (and associated)
methods/functions to os. This makes it much clearer to disambiguate with the swarm
"platform" which is really os/arch. This is a stepping stone to getting the daemon towards
fully multi-platform/arch-aware, and makes it clear when "operating system" is being
referred to rather than "platform" which is misleadingly used - sometimes in the swarm
meaning, but more often as just the operating system.
2017-10-06 11:44:18 -07:00
Daniel Nephin
f7f101d57e Add gosimple linter
Update gometalinter

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-09-12 12:09:59 -04:00
Daniel Nephin
709bf8b7bc Add interfacer linter
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-08-24 15:08:26 -04:00
Derek McGowan
1009e6a40b
Update logrus to v1.0.1
Fixes case sensitivity issue

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2017-07-31 13:16:46 -07:00
John Stephens
b9255e4a53
Stop trying to load images on an incompatible OS
Signed-off-by: John Stephens <johnstep@docker.com>
2017-06-26 15:11:24 -07:00
John Howard
42c5c1a9ec LCOW: Pass platform through into layer store
Signed-off-by: John Howard <jhoward@microsoft.com>
2017-06-20 09:21:37 -07:00
Brian Goff
c8141a1fb1 Merge pull request #32293 from moypray/save_delete
Fix delete a image while saving it, delete successfully but failed to save it
2017-06-05 09:55:56 -04:00
Wentao Zhang
4a014e6b0d Fix delete a image while saving it, delete successfully but failed to save it
Issue Description:
 * 1. Saving more than one images, `docker save -o a.tar aaa bbb`
 * 2. Delete the last image which in saving progress.  `docker rmi bbb`

Espected:
 Saving images operation shouldn't be disturbed. But the real result is that failed to
save image and get an error as below:
  `Error response from daemon: open
/var/lib/docker/image/devicemapper/imagedb/content/sha256/7c24e4d533a76e801662ad1b7e6e06bc1204f80110b5623e96ba2877c51479a1:
no such file or directory`

Analysis:
  1. While saving more than one images, it will get all the image info from reference/imagestore,
     and then using the `cached data` to save the images to a tar file.
  2. But this process doesn't have a resource lock, if a deletion operation comes, the image will be deleted,
     so saving operation will fail.

Solution:
 When begin to save an image, `Get` all the layers first. then the
deletion operation won't delete the layers.

Signed-off-by: Wentao Zhang <zhangwentao234@huawei.com>
2017-06-01 18:13:20 +08:00
Daniel Nephin
c87d67b0ad Refacator pkg/streamformatter
StreamFormatter suffered was two distinct structs mixed into a single struct
without any overlap.

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-05-02 17:38:12 -04:00
Derek McGowan
3a1279393f
Use distribution reference
Remove forked reference package. Use normalized named values
everywhere and familiar functions to convert back to familiar
strings for UX and storage compatibility.

Enforce that the source repository in the distribution metadata
is always a normalized string, ignore invalid values which are not.
Update distribution tests to use normalized values.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2017-02-07 11:08:37 -08:00
Lei Jitang
ee5228e799 Follow #28926, fix docker save can't share layers in the tar archive.
After https://github.com/docker/docker/pull/28926, if saving multiple images
which have common layers at same time, the common layers can't share in the tar archive
because the hash ID changes because of the Create time. The Create time is used for
pre v1.9 which treat each layer as a image and make no sense for after v1.10.
To make the hash ID consistent and keep the image save from >1.10 working properly
on pre v1.9, using a constant Create time `time.Unix(0,0)`.

Signed-off-by: Lei Jitang <leijitang@huawei.com>
2017-01-17 05:40:59 -05:00
Antonio Murdaca
675f4140e7
image: tarexport: do not quote integers in format string
Basically, prevents outputs like:
```
invalid manifest, layers length mismatch: expected '\x02', got '\x01'
```

Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2017-01-14 20:03:11 +01:00
Stephen J Day
7a85579917
*: use opencontainers/go-digest package
The `digest` data type, used throughout docker for image verification
and identity, has been broken out into `opencontainers/go-digest`. This
PR updates the dependencies and moves uses over to the new type.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-01-06 18:48:41 -08:00
Vincent Demeester
ba1c20f8d6 Merge pull request #29345 from yuexiao-wang/fix-err-info
Fix incorrect words and formats in image
2016-12-21 21:50:11 +01:00
yuexiao-wang
b2ec509a3f Fix incorrect info and format of error in image
Signed-off-by: yuexiao-wang <wang.yuexiao@zte.com.cn>
2016-12-20 18:20:15 +08:00
unclejack
2c187a24e0 return directly without ifs in remaining packages
Signed-off-by: Cristian Staretu <cristian.staretu@gmail.com>
2016-12-14 23:28:27 +02:00
Tõnis Tiigi
dd20fa60b4 Merge pull request #28926 from coolljt0725/fix_save
Fix docker save with empty timestamp of layer created time
2016-11-30 11:23:19 -08:00
Stephen J Day
8616b0690a
tarexport: check symlink error when saving layer
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2016-11-29 13:08:24 -08:00
Lei Jitang
e0b2c0af4d Fix docker save with empty timestamp of layer created time
Signed-off-by: Lei Jitang <leijitang@huawei.com>
2016-11-29 03:58:23 -05:00
John Howard
7c29f5beed Windows: Use sequential file access
Signed-off-by: John Howard <jhoward@microsoft.com>
2016-11-14 17:41:52 -08:00
Tonis Tiigi
805223982c Generalize content addressable and reference storage
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2016-09-15 18:08:19 -07:00
Wenkai Yin
8f37abd056 fixes #25654
Signed-off-by: Wenkai Yin <yinw@vmware.com>
2016-08-31 15:37:52 +08:00
Vincent Demeester
353af0b3ec Merge pull request #25557 from yongtang/25529-image-load-api
Return `application/json` for image/load API (quiet=1)
2016-08-12 11:07:43 +02:00
Yong Tang
2f27632cde Return application/json for image/load API (quite=1)
This fix tries to address the issue raised in 25529 wehre the
image/load API returns `application/json` for quiet=0 and
`text/plain` for quite=1.

This fix makes the change so that `application/json` is returned
for both quiet=0 and quite=1.

This fix has been tested manually.

This fix fixes 25529.

Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
2016-08-09 17:55:42 -07:00
allencloud
0ead624473 add defer file.Close to avoid potential fd leak
Signed-off-by: allencloud <allen.sun@daocloud.io>
2016-08-10 08:36:09 +08:00
Derek McGowan
5487881e12 Fix calculation of relative path for symlink on docker save
Relative paths are now calculated from a base path rather than from the file path, which gets treated like a directory.
Symlinks will now properly point to the file as "../<layer dir>/layer.tar" rather the incorrect "../../<layer dir>/layer.tar".

Fixes #24951

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2016-07-25 13:28:10 -07:00
Alexander Morozov
9809446060 Merge pull request #24003 from tonistiigi/fix-duplicate-layers
Fix duplicate layers in exported tar
2016-06-30 17:07:58 -07:00
Tonis Tiigi
41bf7de73b Fix duplicate layers in exported tar
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2016-06-29 18:06:42 -07:00