The call to io.Copy was not being checked for errors at a critical point in the
pull logic. This patch will log the error value if not nil.
Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
pushV2Image() calls TarLayer() which returns an archive. One needs to
Close() the archive once done otherwise it will leave mounted devices
if devicemapper graph driver is being used.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
manifest code calls TarLayer() and gets archive. This archive needs to
be closed once caller is done using it to release the resrouces held
by archive. For the devicemapper graphdriver, archive keeps a device
mounted (device which is backing the layer). If archive.Close() is not
called, that device remains mounted and later deletion of device fails
leading to various other issues.
Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
The trust graph currently pulls the keys needed to verify official images. For non-official images this graph check is useless. To avoid hitting the key statement url when pulling from private v2 registries, restrict the check to official image pulls.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Pushing by image ID is not allowed in the Docker CLI and not supported by the registry. An unnamed image also cannot be pushed to a private registry, since no endpoint is specified and it will default to the hub. The hub also does not support this use case, therefore removing the code path is the best solution.
The ability to push a layer without a name is unsupported by the v2 registry.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Only show the verification message if all the tarsum checks pass and the image manifest is verified.
No longer return an error when a tarsum verification fails, just reset the verification flag.
Tarsum verification is less meaningful without a verified manifest and therefore it should not cause an error.
Updated the verified image test to pull an image which expected to have a verified manifest and contents.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
It didn't work on v2 anyways. And an image with a lot of aliases was slow to
fetch.
Docker-DCO-1.1-Signed-off-by: Jessie Frazelle <princess@docker.com> (github: jfrazelle)
On pull treat an unavailable v2 registry as a non-error fallback.
On push only show v2 error message in debug mode.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
No longer push to the official v2 registry when it is available. This allows pulling images from the v2 registry without defaulting push. Only pull official images from the v2 official registry.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
When requesting a token, the basic auth header is always being set even
if there is no username value. This patch corrects this and does not set
the basic auth header if the username is empty.
Also fixes an issue where pulling all tags from a v2 registry succeeds
when the image does not actually exist on the registry.
Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
The checksum is now being stored in a separate file beside the image
JSON file.
Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
Before the V2 registry changes, images with no config could be pushed.
This change fixes a regression that made those images not able to be
pushed to a registry.
Signed-off-by: Euan Kemp <euank@euank.com>
When progress reader closes it overwrites the progress line with the full progress bar, replaces the completed message.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
The v2 session code will no longer update the indexEndpoint value, therefore it is not necessary to save and restore the value for use with v1.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
While the v2 pull operation is writing the body of the layer blob to disk
it now computes the tarsum checksum of the archive before extracting it to
the backend storage driver. If the checksum does not match that from the
image manifest an error is raised.
Also adds more debug logging to the pull operation and fixes existing test
cases which were failing. Adds a reverse lookup constructor to the tarsum
package so that you can get a tarsum object using a checksum label.
Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
This refactors the starting work by the prior commits to make this safe
for access. A maximum of 5 worker go routines are started to lookup
images on the endpoint. Another go routine consumes the images that are
required to be pushed into a map for quick lookups. The map is required
because the pushing of the image json and layer have to be done in the
correct order or the registry will explode in fire.
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
Passing RepositoryInfo to ResolveAuthConfig, pullRepository, and pushRepository
Moving --registry-mirror configuration to registry config
Created resolve_repository job
Repo names with 'index.docker.io' or 'docker.io' are now synonymous with omitting an index name.
Adding test for RepositoryInfo
Adding tests for opts.StringSetOpts and registry.ValidateMirror
Fixing search term use of repoInfo
Adding integration tests for registry mirror configuration
Normalizing LookupImage image name to match LocalName parsing rules
Normalizing repository LocalName to avoid multiple references to an official image
Removing errorOut use in tests
Removing TODO comment
gofmt changes
golint comments cleanup. renaming RegistryOptions => registry.Options, and RegistryServiceConfig => registry.ServiceConfig
Splitting out builtins.Registry and registry.NewService calls
Stray whitespace cleanup
Moving integration tests for Mirrors and InsecureRegistries into TestNewIndexInfo unit test
Factoring out ValidateRepositoryName from NewRepositoryInfo
Removing unused IndexServerURL
Allowing json marshaling of ServiceConfig. Exposing ServiceConfig in /info
Switching to CamelCase for json marshaling
PR cleanup; removing 'Is' prefix from boolean members. Removing unneeded json tags.
Removing non-cleanup related fix for 'localhost:[port]' in splitReposName
Merge fixes for gh9735
Fixing integration test
Reapplying #9754
Adding comment on config.IndexConfigs use from isSecureIndex
Remove unused error return value from isSecureIndex
Signed-off-by: Don Kjer <don.kjer@gmail.com>
Adding back comment in isSecureIndex
Signed-off-by: Don Kjer <don.kjer@gmail.com>
If .dockerignore mentions either then the client will send them to the
daemon but the daemon will erase them after the Dockerfile has been parsed
to simulate them never being sent in the first place.
an events test kept failing for me so I tried to fix that too
Closes#8330
Signed-off-by: Doug Davis <dug@us.ibm.com>
There has been a lot of discussion (issues 4242 and 5262) about making
`FROM scratch` either a special case or making `FROM` optional, implying
starting from an empty file system.
This patch makes the build command `FROM scratch` special cased from now on
and if used does not pull/set the the initial layer of the build to the ancient
image ID (511136ea..) but instead marks the build as having no base image. The
next command in the dockerfile will create an image with a parent image ID of "".
This means every image ever can now use one fewer layer!
This also makes the image name `scratch` a reserved name by the TagStore. You
will not be able to tag an image with this name from now on. If any users
currently have an image tagged as `scratch`, they will still be able to use that
image, but will not be able to tag a new image with that name.
Goodbye '511136ea3c5a64f264b78b5433614aec563103b4d4702f3ba7d4d2698e22c158',
it was nice knowing you.
Fixes#4242
Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
This commit is patch for following comment
// TODO: This method should return the errors instead of masking them and returning false
Signed-off-by: Daehyeok Mun <daehyeok@gmail.com>
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
When we use the engine/env object we can run into a situation where
a string is passed in as the value but later on when we json serialize
the name/value pairs, because the string is made up of just numbers
it appears as an integer and not a string - meaning no quotes. This
can cause parsing issues for clients.
I tried to find all spots where we call env.Set() and the type of the
name being set might end up having a value that could look like an int
(like author). In those cases I switched it to use env.SetJson() instead
because that will wrap it in quotes.
One interesting thing to note about the testcase that I modified is that
the escaped quotes should have been there all along and we were incorrectly
letting it thru. If you look at the metadata stored for that resource you
can see the quotes were escaped and we lost them during the serialization
steps because of the env.Set() stuff. The use of env is probably not the
best way to do all of this.
Closes: #9602
Signed-off-by: Doug Davis <dug@us.ibm.com>
I was confused earlier when I did:
```
docker push localhost.localdomain:1234/foo
```
Because docker told me:
```
No such id: localhost.localdomain:1234/foo
```
I actually had buried in my mind the solution to this, but the error
message
confused me because I had recently had some fun trying to get the
registry
working and therefore thought it was telling me that I didn't have an
account
on the registry.
This pull request makes it unambiguous that the error is that the
specified
image is unknown.
/cc @cpuguy83
Docker-DCO-1.1-Signed-off-by: Peter Waller <p@pwaller.net> (github: pwaller)
Therer is a bug in the 'skip' decision when exporting a repository
(`docker save repo`)
Only the layers of the first image are included in the archive (the
layers of the next images are missing)
Signed-off-by: Anthony Baire <Anthony.Baire@irisa.fr>
Now, newly created/imported layers will have the checksum of
the layer diff computed and stored in the image json file.
For now, it is not an error if the computed checksum does not
match an existing checksum, only a warning message is logged. The
eventual goal is to use the checksums in the image JSON to verify
the integrity of the layer contents when doing `docker load` or
`docker pull`, and error out if it does not match.
Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
The argument specified the json data to save to disk when registering
a new image into the image graph. If it is nil, then the given image
is serialized to json and that is written by default. This default
behavior is sufficient if the given image was originally deserialzed
from this jsonData to begin with which has always been the case.
Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
Since V2 registry does not yet implement mirrors, when mirrors are given automatically fallback to V1 without checking V2 first.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Fix issue with restoring the tag store and setting static configuration
from the daemon. i.e. the field on the TagStore struct must be made
internal or the json.Unmarshal in restore will overwrite the insecure
registries to be an empty struct.
Signed-off-by: Michael Crosby <michael@docker.com>
Conflicts:
graph/pull.go
graph/push.go
graph/tags.go
As this feature requires more testing it is much safter to get the
underlying changes into the codebase first then enable the feature in
another release after proper testing and verification can be done.
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
Add support for pulling signed images from a version 2 registry.
Only official images within the library namespace will be pull from the
new registry and check the build signature.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Using repo tag in the status message for better usability, as per review comments
Added documentation and Changed code to print Status after downloads are complete
Addresses #2404
Signed-off-by: Srini Brahmaroutu <srbrahma@us.ibm.com>
Now that the archive package does not depend on any docker-specific
packages, only those in pkg and vendor, it can be safely moved into pkg.
Signed-off-by: Rafe Colton <rafael.colton@gmail.com>
When user passes an invalid tag to `docker build`
(i.e. `docker build -t abcd:A0123456789B0123456789C0123456789 .`), check the
tag first and terminate-early so user can specify the tag again
Docker-DCO-1.1-Signed-off-by: Daniel, Dao Quang Minh <dqminh89@gmail.com> (github: dqminh)
Currently inspect output just shows Size info, and that usally be
very small and even 0 which is confusing.
Fixes#8016
Signed-off-by: Qiang Huang <h.huangqiang@huawei.com>
Some graphdrivers are Differs and type assertions are made
in various places throughout the project. Differ offers some
convenience in generating/applying diffs of filesystem layers
but for most graphdrivers another code path is taken.
This patch brings all of the logic related to filesystem
diffs in one place, and simplifies the implementation of some
common types like Image, Daemon, and Container.
Signed-off-by: Josh Hawn <josh.hawn@docker.com>
Adds support for a --registry-mirror=scheme://<host>[:port]
daemon flag. The flag may be present multiple times. If
provided, mirrors are prepended to the list of endpoints used
for image pull. Note that only mirrors of the public
index.docker.io registry are supported, and image/tag resolution
is still performed via the official index.
Docker-DCO-1.1-Signed-off-by: Tim Smith <timbot@google.com> (github: timbot)
Now from a single invocation of `docker save`, you can specify multiple
images to include in the output tar, or even just multiple tags of a
particular image/repo.
```
> docker save -o bundle.tar busybox ubuntu:lucid ubuntu:saucy fedora:latest
> tar tf ./bundle.tar | wc -l
42
> tar xOf ./bundle.tar repositories
{"busybox":{"latest":"2d8e5b282c81244037eb15b2068e1c46319c1a42b80493acb128da24b2090739"},"fedora":{"latest":"58394af373423902a1b97f209a31e3777932d9321ef10e64feaaa7b4df609cf9"},"ubuntu":{"lucid":"9cc9ea5ea540116b89e41898dd30858107c1175260fb7ff50322b34704092232","saucy":"9f676bd305a43a931a8d98b13e5840ffbebcd908370765373315926024c7c35e"}}
```
Further, this fixes the bug where the `repositories` file is not created
when saving a specific tag of an image (e.g. ubuntu:latest)
document multi-image save and updated API docs
Docker-DCO-1.1-Signed-off-by: Vincent Batts <vbatts@redhat.com> (github: vbatts)
Graph.Get function already returns nil when the image doesn't exist so
following comment is obsolete.
FIXME: return nil when the image doesn't exist, instead of an error
Signed-off-by: Daehyeok.Mun <daehyeok@gmail.com>
renaming this struct to more clearly be session, as that is what it
handles.
Splitting out files for easier readability.
Signed-off-by: Vincent Batts <vbatts@redhat.com>
This is part of the ongoing effort to remove the deprecated server/
package, and generally cleanup and simplify the codebase.
Signed-off-by: Solomon Hykes <solomon@docker.com>
- Directories should have u+x (0600 -> 0700)
- Don't chown files to root because we don't have to
Docker-DCO-1.1-Signed-off-by: Kato Kazuyoshi <kato.kazuyoshi@gmail.com> (github: kzys)
functions to pkg/parsers/kernel, and parsing filters to
pkg/parsers/filter. Adjust imports and package references.
Docker-DCO-1.1-Signed-off-by: Erik Hollensbe <github@hollensbe.org> (github: erikh)
This code was introduced with commented out sections. Just delete them.
Original commit: 18fc707fdf
Docker-DCO-1.1-Signed-off-by: Brandon Philips <brandon.philips@coreos.com> (github: philips)
All modern distros set up /run to be a tmpfs, see for instance:
https://wiki.debian.org/ReleaseGoals/RunDirectory
Its a very useful place to store pid-files, sockets and other things
that only live at runtime and that should not be stored in the image.
This is also useful when running systemd inside a container, as it
will try to mount /run if not already mounted, which will fail for
non-privileged container.
Docker-DCO-1.1-Signed-off-by: Alexander Larsson <alexl@redhat.com> (github: alexlarsson)
This is a first step towards moving all code related to local
manipulation of images into a cleanly separated subsystem,
accessible via a stable set of commands in the engine API.
`graph.TagStore` now implements `engine.Installer`. For now, it
is installed by `Server.InitServer`, along with all other Server
commands. However this will change in future patches.
`graph.TagStore.Install` registers the following commands:
* `image_set` creates a new image and stores it locally.
* `image_get` returns information about an image stored locally.
* `image_tag` assigns a new name and tag to an existing image.
These commands are a pre-requisite for moving 'push' and 'pull'
out of `Server`.
Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
This has every container using the docker daemon's pid for the processes
label so it does not work correctly.
Docker-DCO-1.1-Signed-off-by: Michael Crosby <michael@crosbymichael.com> (github: crosbymichael)
Fixes#5166
Current graph.restore is essentially O(n^2 log n) due to how
suffixarray creation works.
Rather than create/append/create new this supports creation from a seed
array of ids.
Functional testing shows this eliminates the hang on Creating image
graph reported on list.
Docker-DCO-1.1-Signed-off-by: Paul Nasrat <pnasrat@gmail.com> (github: pnasrat)
symlink /proc/mounts /var/lib/docker/btrfs/subvolumes/1763d6602b8b871f0a79754f1cb0a31b3928bb95de5232b1b8c15c60fa1017f6-init/etc/mtab: no such file or directory
Docker-DCO-1.1-Signed-off-by: Andrew Page <admwiggin@gmail.com> (github: tianon)
docker will run the process(es) within the container with an SELinux label and will label
all of the content within the container with mount label. Any temporary file systems
created within the container need to be mounted with the same mount label.
The user can override the process label by specifying
-Z With a string of space separated options.
-Z "user=unconfined_u role=unconfined_r type=unconfined_t level=s0"
Would cause the process label to run with unconfined_u:unconfined_r:unconfined_t:s0"
By default the processes will run execute within the container as svirt_lxc_net_t.
All of the content in the container as svirt_sandbox_file_t.
The process mcs level is based of the PID of the docker process that is creating the container.
If you run the container in --priv mode, the labeling will be disabled.
Docker-DCO-1.1-Signed-off-by: Dan Walsh <dwalsh@redhat.com> (github: rhatdan)
