Commit graph

79 commits

Author SHA1 Message Date
Sebastiaan van Stijn
4952dc2844
distribution: remove unused RequireSchema2
It's never set, so we can remove it.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 85fddc0081)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-03 18:09:17 +01:00
Sebastiaan van Stijn
f2463e1e86
distribution: use ad-hoc trustkey for tests
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit e854b2a459)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-03 18:09:16 +01:00
Sebastiaan van Stijn
2b0da89366
distribution: move newPusher() and newPuller() together with definition
Also moving writeStatus() to the puller, which is where it's used, and makes
it slightly easier to consume.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-21 23:13:39 +02:00
Sebastiaan van Stijn
566c8db66d
distribution: add GetRepository(), un-export NewV2Repository, ValidateRepoName
These were only exported to facilitate ImageService.GetRepository() (used for
the `GET /distribution/{name:.*}/json` endpoint.

Moving the core functionality of that to the distribution package makes it
more consistent with (e.g.) "pull" operations, and allows us to keep more things
internal.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-21 23:12:02 +02:00
Sebastiaan van Stijn
572c7e0184
distribution: remove RootFSFromConfig(), PlatformFromConfig() from ImageConfigStore
These functions did not require the ImageConfigStore, so could just be local
utilities.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-21 23:12:00 +02:00
Sebastiaan van Stijn
1e75ab0ab9
distribution: remove Pusher interface, NewPusher(), and redundant V1 checks
It's only used internally, so we can refer to the implementation itself. Given
that RegistryService.LookupPushEndpoints now only returns V2 endpoints, we
no longer need to check if an endpoint is possibly V1.

Also rename some types that had "v2" in their name, now that we only support v2.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-21 23:11:54 +02:00
Sebastiaan van Stijn
a0230f3d9a
remove unneeded "digest" alias for "go-digest"
I think this was there for historic reasons (may have been goimports expected
this, and we used to have a linter that wanted it), but it's not needed, so
let's remove it (to make my IDE less complaining about unneeded aliases).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-04 14:49:42 +01:00
Sebastiaan van Stijn
f9a1846ca2
distribution: PushLayer.Size(): remove unused error return
None of the implementations returned an error for this function, so removing it.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-24 18:45:51 +01:00
Sebastiaan van Stijn
1cdad98a7a
distribution: do not retry unsupported layer media types
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-10-27 12:33:30 +02:00
Sebastiaan van Stijn
2773f81aa5
Merge pull request #42445 from thaJeztah/bump_golang_ci
[testing] ~update~ fix linting issues found by golangci-lint v1.40.1
2021-06-16 22:15:01 +02:00
Sebastiaan van Stijn
dc7cbb9b33
remove layerstore indexing by OS (used for LCOW)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-06-10 17:49:11 +02:00
Sebastiaan van Stijn
d13997b4ba
gosec: G601: Implicit memory aliasing in for loop
plugin/v2/plugin.go:141:50: G601: Implicit memory aliasing in for loop. (gosec)
                    updateSettingsEnv(&p.PluginObj.Settings.Env, &s)
                                                                 ^
    libcontainerd/remote/client.go:572:13: G601: Implicit memory aliasing in for loop. (gosec)
                cpDesc = &m
                         ^
    distribution/push_v2.go:400:34: G601: Implicit memory aliasing in for loop. (gosec)
                (metadata.CheckV2MetadataHMAC(&mountCandidate, pd.hmacKey) ||
                                              ^
    builder/dockerfile/builder.go:261:84: G601: Implicit memory aliasing in for loop. (gosec)
            currentCommandIndex = printCommand(b.Stdout, currentCommandIndex, totalCommands, &meta)
                                                                                             ^
    builder/dockerfile/builder.go:278:46: G601: Implicit memory aliasing in for loop. (gosec)
            if err := initializeStage(dispatchRequest, &stage); err != nil {
                                                       ^
    daemon/container.go:283:40: G601: Implicit memory aliasing in for loop. (gosec)
            if err := parser.ValidateMountConfig(&cfg); err != nil {
                                                 ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-06-10 13:03:29 +02:00
Tibor Vass
7a50fe8a52
Remove more of registry v1 code.
Signed-off-by: Tibor Vass <tibor@docker.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-02-23 09:49:46 +01:00
Brian Goff
9ca3bb632e Store image manifests in containerd content store
This allows us to cache manifests and avoid extra round trips to the
registry for content we already know about.

dockerd currently does not support containerd on Windows, so this does
not store manifests on Windows, yet.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2020-11-05 20:02:18 +00:00
Sebastiaan van Stijn
6302dbbf46
Disable manifest v2 schema 1 push
For CI, a temporary `DOCKER_ALLOW_SCHEMA1_PUSH_DONOTUSE` environment
variable was added while we work out a solution for testing schema 1
pulls (which currently require pushing them to a local registry first
for testing).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-07-31 20:13:58 +02:00
zhipengzuo
e07a88c95f fix code note when push all tags
Signed-off-by: zhipengzuo <zuozhipeng@baidu.com>
2020-06-16 21:05:10 +08:00
Brian Goff
6859bc7eee Move schema1 deprecation notice
Currently we show this deprecation notice for any error returned by a
registry.
Registries can return an error for any number of reasons.
Instead let's show the deprecation notice only if the fallback was
successful.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2020-03-12 11:17:20 -07:00
Sebastiaan van Stijn
07ff4f1de8
goimports: fix imports
Format the source according to latest goimports.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-18 12:56:54 +02:00
Tibor Vass
647dfe99a5 distribution: modify warning logic when pulling v2 schema1 manifests
The warning on pull was incorrectly asking to contact registry admins.
It is kept on push however.

Pulling manifest lists with v2 schema1 manifests will not be supported thus
there is a warning for those, but wording changed to suggest repository author
to upgrade.

Finally, a milder warning on regular pull is kept ONLY for DockerHub users
in order to incite moving away from schema1.

Signed-off-by: Tibor Vass <tibor@docker.com>
2019-09-12 18:52:49 +00:00
Tibor Vass
d35f8f4329 Add deprecation message for schema1
This will add a warning log in the daemon, and will send the message
to be displayed by the CLI.

Signed-off-by: Tibor Vass <tibor@docker.com>
2019-06-18 01:40:25 +00:00
Tibor Vass
f695e98cb7 Revert "Remove the rest of v1 manifest support"
This reverts commit 98fc09128b in order to
keep registry v2 schema1 handling and libtrust-key-based engine ID.

Because registry v2 schema1 was not officially deprecated and
registries are still relying on it, this patch puts its logic back.

However, registry v1 relics are not added back since v1 logic has been
removed a while ago.

This also fixes an engine upgrade issue in a swarm cluster. It was relying
on the Engine ID to be the same upon upgrade, but the mentioned commit
modified the logic to use UUID and from a different file.

Since the libtrust key is always needed to support v2 schema1 pushes,
that the old engine ID is based on the libtrust key, and that the engine ID
needs to be conserved across upgrades, adding a UUID-based engine ID logic
seems to add more complexity than it solves the problems.

Hence reverting the engine ID changes as well.

Signed-off-by: Tibor Vass <tibor@docker.com>
2019-06-18 00:36:01 +00:00
Aleksa Sarai
d283c7fa2b
*: remove interfacer linter from CI
It has been declared deprecated by the author, and has a knack for
false-positives (as well as giving bad advice when it comes to APIs --
which is quite clear when looking at "nolint: interfacer" comments).

Signed-off-by: Aleksa Sarai <asarai@suse.de>
2019-03-13 11:48:39 +11:00
Justin Cormack
98fc09128b Remove the rest of v1 manifest support
As people are using the UUID in `docker info` that was based on the v1 manifest signing key, replace
with a UUID instead.

Remove deprecated `--disable-legacy-registry` option that was scheduled to be removed in 18.03.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2019-03-02 10:46:37 -08:00
Sebastiaan van Stijn
f23c00d870
Various code-cleanup
remove unnescessary import aliases, brackets, and so on.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-05-23 17:50:54 +02:00
Kir Kolyshkin
7d62e40f7e Switch from x/net/context -> context
Since Go 1.7, context is a standard package. Since Go 1.9, everything
that is provided by "x/net/context" is a couple of type aliases to
types in "context".

Many vendored packages still use x/net/context, so vendor entry remains
for now.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2018-04-23 13:52:44 -07:00
慕陶
8b387b165a fix(distribution): digest cache should not be moved if it was an auth
error

local digest cache will be removed when error occured on push image
but it should not be removed if it is an auth error while on auth was
provided

https://github.com/moby/moby/issues/36309
Signed-off-by: 慕陶 <jihui.xjh@alibaba-inc.com>
2018-03-17 17:29:01 +08:00
John Howard
83908836d3 Windows: Block pulling uplevel images
Signed-off-by: John Howard <jhoward@microsoft.com>
2018-02-26 12:33:54 -08:00
Daniel Nephin
4f0d95fa6e Add canonical import comment
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2018-02-05 16:51:57 -05:00
John Howard
afd305c4b5 LCOW: Refactor to multiple layer-stores based on feedback
Signed-off-by: John Howard <jhoward@microsoft.com>
2018-01-18 08:31:05 -08:00
Jake Sanders
305801f58f Disambiguate mirror -> other endpoint fallbacks from V2 -> V1
Signed-off-by: Jake Sanders <jsand@google.com>
2017-11-30 20:01:03 -08:00
John Howard
0380fbff37 LCOW: API: Add platform to /images/create and /build
Signed-off-by: John Howard <jhoward@microsoft.com>

This PR has the API changes described in https://github.com/moby/moby/issues/34617.
Specifically, it adds an HTTP header "X-Requested-Platform" which is a JSON-encoded
OCI Image-spec `Platform` structure.

In addition, it renames (almost all) uses of a string variable platform (and associated)
methods/functions to os. This makes it much clearer to disambiguate with the swarm
"platform" which is really os/arch. This is a stepping stone to getting the daemon towards
fully multi-platform/arch-aware, and makes it clear when "operating system" is being
referred to rather than "platform" which is misleadingly used - sometimes in the swarm
meaning, but more often as just the operating system.
2017-10-06 11:44:18 -07:00
Daniel Nephin
f7f101d57e Add gosimple linter
Update gometalinter

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-09-12 12:09:59 -04:00
Daniel Nephin
709bf8b7bc Add interfacer linter
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-08-24 15:08:26 -04:00
Derek McGowan
1009e6a40b
Update logrus to v1.0.1
Fixes case sensitivity issue

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2017-07-31 13:16:46 -07:00
lixiaobing10051267
80b2c326de optimize for loop with rootfs.DiffIDs
Signed-off-by: lixiaobing10051267 <li.xiaobing1@zte.com.cn>
2017-06-27 20:31:27 +08:00
John Howard
42c5c1a9ec LCOW: Pass platform through into layer store
Signed-off-by: John Howard <jhoward@microsoft.com>
2017-06-20 09:21:37 -07:00
Noah Treuhaft
67fdf574d5 Add daemon option to push foreign layers
The --allow-nondistributable-artifacts daemon option specifies
registries to which foreign layers should be pushed.  (By default,
foreign layers are not pushed to registries.)

Additionally, to make this option effective, foreign layers are now
pulled from the registry if possible, falling back to the URLs in the
image manifest otherwise.

This option is useful when pushing images containing foreign layers to a
registry on an air-gapped network so hosts on that network can pull the
images without connecting to another server.

Signed-off-by: Noah Treuhaft <noah.treuhaft@docker.com>
2017-05-16 14:36:36 -07:00
yupengzte
595901bd7e check err
Signed-off-by: yupengzte <yu.peng36@zte.com.cn>
2017-05-09 19:24:21 +08:00
Jon Johnson
cdc46abde5 Always HEAD blobs at least once during pushes
Signed-off-by: Jon Johnson <jonjohnson@google.com>
2017-03-08 15:36:12 -08:00
Aaron.L.Xu
e0577d5fe8 fix some typos from module contrib to man
Signed-off-by: Aaron.L.Xu <likexu@harmonycloud.cn>
2017-02-18 10:08:55 +08:00
Derek McGowan
3a1279393f
Use distribution reference
Remove forked reference package. Use normalized named values
everywhere and familiar functions to convert back to familiar
strings for UX and storage compatibility.

Enforce that the source repository in the distribution metadata
is always a normalized string, ignore invalid values which are not.
Update distribution tests to use normalized values.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2017-02-07 11:08:37 -08:00
Stephen J Day
7a85579917
*: use opencontainers/go-digest package
The `digest` data type, used throughout docker for image verification
and identity, has been broken out into `opencontainers/go-digest`. This
PR updates the dependencies and moves uses over to the new type.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-01-06 18:48:41 -08:00
Tonis Tiigi
13222160e8 Define PushResult in api types
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2016-12-26 18:53:22 -08:00
Derek McGowan
3c7676a057
Abstract distribution interfaces from image specific types
Move configurations into a single file.
Abstract download manager in pull config.
Add supports for schema2 only and schema2 type checking.
Add interface for providing push layers.
Abstract image store to generically handle configurations.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2016-12-19 10:55:00 -08:00
Aaron Lehmann
e0702e9f37 distribution: Fix panic on push
When building a manifest during a push operation, all layers must have
an associated descriptor. If a layer is missing a descriptor, that leads
to a panic.

A break inside a switch in layerAlreadyExists meant to break from the
loop surrounding the switch, but instead breaks from the switch. This
causes the loop to continue, and can overwrite the descriptor with an
empty one, leading to the panic.

Also, fix layerAlreadyExists not to abort the push when a speculative
stat on a candidate layer digest fails with an error. This could happen
in situations like a potential cross-repository mount where the user
does not have permission to access the source repository.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-11-29 18:13:39 -08:00
Derek McGowan
c85eb00841
Update for distribution vendor
Handle updates to reference package.
Updates for refactoring of challenge manager.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2016-11-10 17:34:12 -08:00
Sebastiaan van Stijn
762556c705 Merge pull request #26564 from miminar/prevent-blob-uploads
Avoid unnecessary blob uploads
2016-10-01 12:53:10 +02:00
Michal Minář
81f7b1f1e5
Different number of retries for layers of different sizes
Classify blobs into three categories based on size.
Use a very limited number of mount attempts and no existence check for
small blobs. Use more attempts for bigger blobs.

Also remember blob associations during layer existence check.

Blob digests are now checked in the target repository from newest to
latest. If the blob exists and the metadata entry does not, it will be
created. If the blob is not found, the metadata entry will be removed.

Signed-off-by: Michal Minář <miminar@redhat.com>
2016-09-29 14:16:01 +02:00
Michal Minář
d3bd14a4fb
Allow to mock V2MetadataService in unit tests
Signed-off-by: Michal Minář <miminar@redhat.com>
2016-09-29 14:16:00 +02:00
Michal Minář
c6dd51c32c
Try to cross-repo mount until success
Don't fallback back immediately to blob upload if the cross-repo mount
fails and layer upload is initiated by registry.

Instead cancel the upload and re-try cross-repo mount from different
source repository before doing full re-upload.

Signed-off-by: Michal Minář <miminar@redhat.com>
2016-09-29 14:16:00 +02:00