As noted by #30083, the new strict checking of mediatypes misses some
cases where earlier bugs caused nonstandard mediatypes to be stored in
manifests. Two of the known cases are text/html and application/json,
which were returned by certain registries and stored by earlier versions
of Docker. Add special cases for text/html and application/json.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
(cherry picked from commit a215e15cb1)
Signed-off-by: Victor Vieux <vieux@docker.com>
When storeLayer.Parent returns the parent layer, it needs to use the same logic as Get where it wraps in a describablyStoreLayer if the layer is describable. Otherwise, on Windows, this can result in pushing the foreign layers, which is not supposed to be allowed.
This fixes https://github.com/docker/docker/issues/30080.
Signed-off-by: Stefan J. Wernli <swernli@microsoft.com>
(cherry picked from commit d14b7212ad)
Signed-off-by: Victor Vieux <vieux@docker.com>
Move plugins to shared distribution stack with images.
Create immutable plugin config that matches schema2 requirements.
Ensure data being pushed is same as pulled/created.
Store distribution artifacts in a blobstore.
Run init layer setup for every plugin start.
Fix breakouts from unsafe file accesses.
Add support for `docker plugin install --alias`
Uses normalized references for default names to avoid collisions when using default hosts/tags.
Some refactoring of the plugin manager to support the change, like removing the singleton manager and adding manager config struct.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
(cherry picked from commit 3d86b0c79b)
Move configurations into a single file.
Abstract download manager in pull config.
Add supports for schema2 only and schema2 type checking.
Add interface for providing push layers.
Abstract image store to generically handle configurations.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
(cherry picked from commit 3c7676a057)
When building a manifest during a push operation, all layers must have
an associated descriptor. If a layer is missing a descriptor, that leads
to a panic.
A break inside a switch in layerAlreadyExists meant to break from the
loop surrounding the switch, but instead breaks from the switch. This
causes the loop to continue, and can overwrite the descriptor with an
empty one, leading to the panic.
Also, fix layerAlreadyExists not to abort the push when a speculative
stat on a candidate layer digest fails with an error. This could happen
in situations like a potential cross-repository mount where the user
does not have permission to access the source repository.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
(cherry picked from commit e0702e9f37)
Signed-off-by: Victor Vieux <victorvieux@gmail.com>
Translate pull errors to provide a more consistent and user friendly
error message.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Remove the following comment in pullV2Tag:
// NOTE: not using TagService.Get, since it uses HEAD requests
// against the manifests endpoint, which are not supported by
// all registry versions.
This is actually not an issue, because TagService.Get does a fallback to
GET if HEAD fails. It has done this ever since TagService was added to
the distribution API, so this comment was probably based on an early
version of TagService before it was merged, or was always a
misunderstanding.
However, we continue to use ManifestService.Get instead because it
saves a round trip. The manifest can be retrieved directly instead of
resolving the digest first.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Allow built images to be squash to scratch.
Squashing does not destroy any images or layers, and preserves the
build cache.
Introduce a new CLI argument --squash to docker build
Introduce a new param to the build API endpoint `squash`
Once the build is complete, docker creates a new image loading the diffs
from each layer into a single new layer and references all the parent's
layers.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Classify blobs into three categories based on size.
Use a very limited number of mount attempts and no existence check for
small blobs. Use more attempts for bigger blobs.
Also remember blob associations during layer existence check.
Blob digests are now checked in the target repository from newest to
latest. If the blob exists and the metadata entry does not, it will be
created. If the blob is not found, the metadata entry will be removed.
Signed-off-by: Michal Minář <miminar@redhat.com>
Don't fallback back immediately to blob upload if the cross-repo mount
fails and layer upload is initiated by registry.
Instead cancel the upload and re-try cross-repo mount from different
source repository before doing full re-upload.
Signed-off-by: Michal Minář <miminar@redhat.com>
to avoid unnecessary blob re-uploads.
Cross repository mount from particular repo will most probably fail if
the user pushing to the registry is not the same as the one who pulled
or pushed to the source repo.
This PR attempts first to cross-repo mount from the source repositories
associated with the pusher's auth config. Then it falls back to other
repositories sorted from the most similar to the target repo to the
least.
It also prevents metadata deletion in cases where cross-repo mount fails
and the auth config hashes differ.
Signed-off-by: Michal Minář <miminar@redhat.com>
Windows base layers are no longer the special "layers+base" type, so we can remove all the special handling for that.
Signed-off-by: Stefan J. Wernli <swernli@microsoft.com>
Ensure that cancellation of a pull propagates rather than continuing to
container creation. This ensures that the `Prepare` method is properly
re-entrant.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
This fixes the hard coded restriction for non-linux platforms to v2 registries. Previously, the check was above the flag parsing, which would overwrite the hard coded value and prevent correct operation. This change also removes the related daemon flag from Windows to avoid confusion, as it has no meaning when the value is going to always be hard coded to true.
Signed-off-by: Stefan J. Wernli <swernli@microsoft.com>
Updates the v1 search endpoint to also support v2 auth when an identity token is given.
Only search v1 endpoint is supported since there is not v2 search currently defined to replace it.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
If the remote registry responds with a NAME_UNKNOWN error, treat this as
a fatal error and don't retry the push.
Tested against an ECR registry.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Always attempt to add digest even when tag already exists.
Ensure digest does not currently exist.
When image id is mismatched, output an error log.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Layer uploads are deduplicated by a "key" made up of the layer DiffID
and the repository name. The repository name being used to form this key
was a remote version of the name that didn't include the name of the
registry. Consequently, pushes of the same layer in a repository with
the same remote name to different registries would wrongly be
deduplicated.
Correct the key by using the full name of the repository, which includes
the registry hostname as well as the image's name.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
This patch introduces a new experimental engine-level plugin management
with a new API and command line. Plugins can be distributed via a Docker
registry, and their lifecycle is managed by the engine.
This makes plugins a first-class construct.
For more background, have a look at issue #20363.
Documentation is in a separate commit. If you want to understand how the
new plugin system works, you can start by reading the documentation.
Note: backwards compatibility with existing plugins is maintained,
albeit they won't benefit from the advantages of the new system.
Signed-off-by: Tibor Vass <tibor@docker.com>
Signed-off-by: Anusha Ragunathan <anusha@docker.com>
This fix tries to fix logrus formatting by removing `f` from
`logrus.[Error|Warn|Debug|Fatal|Panic|Info]f` when formatting string
is not present.
This fix fixes#23459.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
… and refactor a little bit some daemon on the way.
- Move `SearchRegistryForImages` to a new file (`daemon/search.go`) as
`daemon.go` is getting pretty big.
- `registry.Service` is now an interface (allowing us to decouple it a
little bit and thus unit test easily).
- Add some unit test for `SearchRegistryForImages`.
- Use UniqueExactMatch for search filters
- And use empty restore id for now in client.ContainerStart.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
