Commit graph

311 commits

Author SHA1 Message Date
Antonio Murdaca
c9207bc0aa Format times in inspect command with a template as RFC3339Nano
In 1.6.2 we were decoding inspect API response into interface{}.
time.Time fields were JSON encoded as RFC3339Nano in the response
and when decoded into interface{} they were just strings so the inspect
template treated them as just strings.
From 1.7 we are decoding into types.ContainerJSON and when the template
gets executed it now gets a time.Time and it's formatted as
2015-07-22 05:02:38.091530369 +0000 UTC.
This patch brings back the old behavior by typing time.Time fields
as string so they gets formatted as they were encoded in JSON -- RCF3339Nano

Signed-off-by: Antonio Murdaca <runcom@linux.com>
2015-07-26 15:25:08 +02:00
Derek McGowan
ed13c3abfb Use notary library for trusted image fetch and signing
Add a trusted flag to force the cli to resolve a tag into a digest via the notary trust library and pull by digest.
On push the flag the trust flag will indicate the digest and size of a manifest should be signed and push to a notary server.
If a tag is given, the cli will resolve the tag into a digest and pull by digest.
After pulling, if a tag is given the cli makes a request to tag the image.

Use certificate directory for notary requests

Read certificates using same logic used by daemon for registry requests.

Catch JSON syntax errors from Notary client

When an uncaught error occurs in Notary it may show up in Docker as a JSON syntax error, causing a confusing error message to the user.
Provide a generic error when a JSON syntax error occurs.

Catch expiration errors and wrap in additional context.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2015-07-24 14:08:20 -07:00
Aaron Lehmann
4fcb9ac40c Improve documentation and golint compliance of registry package
* Add godoc documentation where it was missing

* Change identifier names that don't match Go style, such as INDEX_NAME

* Rename RegistryInfo to PingResult, which more accurately describes
  what this structure is for. It also has the benefit of making the name
  not stutter if used outside the package.

Updates #14756

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-07-24 11:55:07 -07:00
Doug Davis
b874ef8f43 Do Docker edits so we can use the new distribution code
Signed-off-by: Doug Davis <dug@us.ibm.com>
2015-07-23 10:24:46 -07:00
Aaron Lehmann
810d3b2642 Avoid redundant HEAD requests for identical layers on push
pushV2Tag already deduplicates layers, but the scope of this
deduplication is only for a particular tag. If we are pushing all tags
in a repository, we may check layers several times. Fix this by moving
the layersSeen map from the pushV2Tag function to the v2Pusher struct.

In addition to avoiding some useless round-trips, this makes the "docker
push" output less confusing. It formerly could contain many repeated
lines like:

    124e2127157f: Image already exists
    124e2127157f: Image already exists
    ...

Add test coverage based on the "docker push" output: a hash should not
appear multiple times when pushing multiple tags.

Fixes #14873

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-07-22 17:54:20 -07:00
Vincent Batts
22347fdb63 graph: isolate the (dis)assembly logic
with the current duplication of code in the grap.go split-up, this puts
all assembly/disassembly logic into isolated functions

Signed-off-by: Vincent Batts <vbatts@redhat.com>
2015-07-22 11:36:15 -04:00
Vincent Batts
5a00326d29 graph: use tar archive entries for TarLayer
if there is a tar-data.json.gz present for an image layer, then use it
to create the tar archive, instead of the traditional graphdriver Diff.

Signed-off-by: Vincent Batts <vbatts@redhat.com>

Conflicts:
	graph/graph.go
2015-07-22 11:36:15 -04:00
Vincent Batts
ba1f76cbfa graph: variablize file names
and add a comment.. :-)

Signed-off-by: Vincent Batts <vbatts@redhat.com>

Conflicts:
	graph/graph.go
2015-07-22 11:36:15 -04:00
Vincent Batts
5d9f06599c graph: preserve tar archive entries
Preserve the entries from the tar archive for layers added to the graph.

With these entries and relative filesystem path, the tar archives can be
reassembled later.

Signed-off-by: Vincent Batts <vbatts@redhat.com>
2015-07-22 11:36:15 -04:00
John Howard
9001ea26e7 Fixing Image struct to no longer use Graph.
Signed-off-by:  John Howard <jhoward@microsoft.com>
2015-07-20 13:59:53 -07:00
David Calavera
7f353a11e4 Merge pull request #13681 from tiborvass/carry-11784
Carry 11784: rmi dangling is unsafe when pulling
2015-07-17 16:17:18 -07:00
Ma Shimiao
1b67c38f6f fix 8926: rmi dangling is unsafe when pulling
Signed-off-by: Ma Shimiao <mashimiao.fnst@cn.fujitsu.com>
Signed-off-by: Tibor Vass <tibor@docker.com>
2015-07-17 11:39:57 -04:00
Vincent Batts
a40e337882 graph: clarify the need for named error
Signed-off-by: Vincent Batts <vbatts@redhat.com>
2015-07-17 10:01:52 -04:00
Stephen Day
212525f951 Merge pull request #14664 from calavera/fix_load_tag_with_digest
Check if a tag name to load is a valid digest.
2015-07-16 12:46:05 -07:00
David Calavera
1ec25653d8 Check if a tag name to load is a valid digest.
Signed-off-by: David Calavera <david.calavera@gmail.com>
2015-07-16 10:53:56 -07:00
Derek McGowan
19515a7ad8 Update graph to use vendored distribution client for the v2 codepath
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Signed-off-by: Tibor Vass <tibor@docker.com>
2015-07-16 13:13:47 -04:00
Tibor Vass
276c640be4 remove pkg/transport and use the one from distribution
Signed-off-by: Tibor Vass <tibor@docker.com>
2015-07-16 13:13:46 -04:00
David Calavera
cd642973fa Merge pull request #14661 from LK4D4/vet_warns
Fix some formatting calls
2015-07-15 16:41:18 -07:00
Alexander Morozov
a5142f6ac3 Fix some formatting calls
Signed-off-by: Alexander Morozov <lk4d4@docker.com>
2015-07-15 12:25:50 -07:00
Derek McGowan
7f48cd7dce Set canonical name correctly
Currently canonical name gets set to the local name and displayed in the errors.
Canonical name should be the unique and canonical name for an image.
Use docker.io as the canonical domain for images on the public registry.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2015-07-14 17:45:49 -07:00
John Howard
52f4d09ffb Windows: Graph driver implementation
Signed-off-by: John Howard <jhoward@microsoft.com>
2015-07-10 14:33:11 -07:00
Roman Strashkin
cc955ae73c added ability to iterate over all indexes and use index.Iterate() instead of ReadDir() to walk over the graph
Signed-off-by: Roman Strashkin <roman.strashkin@gmail.com>
2015-07-07 22:13:28 +03:00
Derek McGowan
35081ea4b6 Fix duplicate layers in manifest
Currently the layer array is initialized with the first layer then the first layer is appened to the layer list. Adding the first layer twice causes the layer to appear twice in the manifest, making a duplicate push and pull attempt occur.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2015-07-06 17:31:06 -07:00
Vincent Demeester
10e114fb95 Replace latest log by logrus
Related to #11618 and #11614

Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2015-07-02 16:11:52 +02:00
Stephen J Day
7eac23cf8d Attempt to protect on disk image store with mutex
During `(*Graph).Register, there was no protection on adding new layers
concurrently. In some cases, this resulted in corruption of a layer by creating
the directory but not the underlying data. This manifested in several different
IO errors reported in the client.  This attempts to fix this by adding a mutex
by Image ID to protect the Register operation.

We do not completely understand the root cause of this corruption other than
the result is somehow tied to this particular function.  This fix has been
confirmed to address the issue through testing.

Unfortunately, this fix does not address existing corruption. The user will
have to remove and re-pull the corrupt layer to stop the error from happening
in the future. This change only ensures that the layer will not become corrupt.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-06-25 20:16:37 -07:00
Antonio Murdaca
927d13bc3c Remove dead code
Signed-off-by: Antonio Murdaca <runcom@linux.com>
2015-06-20 19:14:15 +02:00
Derek McGowan
d86345b9f7 Merge pull request #13575 from mattmoor/consistent-push-fallback
Make v2 push have v1-fallback behavior consistent with pull.
2015-06-17 13:41:58 -07:00
Phil Estes
c107e9d790 Merge pull request #13870 from lindenlab/pull-single-tag
Only request a single repository tag when pulling a specific image:tag
2015-06-17 15:29:39 -04:00
Phil Estes
a27d8f9aa4 Merge pull request #13975 from stevvooe/move-setup-init-layer
Move graph.SetupInitLayer to daemon package where it is used
2015-06-16 22:37:51 -04:00
Stephen J Day
b7f887a9a2 Move graph.SetupInitLayer to daemon package where it is used
An inspection of the graph package showed this function to be way out of place.
It is only depended upon by the daemon code. The function prepares a top-level
readonly layer used to provide a consistent runtime environment for docker
images.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-06-16 16:50:56 -07:00
David Calavera
ba9db62e68 Merge pull request #13685 from yuchangchun1/tag_regx
add a more accurate error description for invalid tag name
2015-06-16 16:33:17 -07:00
Vincent Batts
e69df2589c Merge pull request #13198 from rhvgoyal/extend-docker-inspect
docker-inspect: Extend docker inspect to export image metadata related to graph driver
2015-06-16 15:03:14 -05:00
Don Kjer
b349a74c71 Only pulling single repository tag on pull for a specific tag. extending TestGetRemoteTags unit test
Splitting out GetRemoteTag from GetRemoteTags.  Adding registry.ErrRepoNotFound error

Signed-off-by: Don Kjer <don.kjer@gmail.com>
2015-06-16 07:10:09 +00:00
Derek McGowan
a98ea87e46 Store layer digests on pull
Currently digests are not stored on pull, causing a simple re-tag or re-push to send up all layers. Storing the digests on pull will allow subsequent pushes to the same repository to not push up content.
This does not address pushing content to a new repository. When content is pushed to a new repository, the digest will be recalculated. Since only one digest is currently stored, it may cause a new content push to the original repository.

Fixes #13883

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2015-06-15 11:08:21 -07:00
Vivek Goyal
407a626be6 docker-inspect: Extend docker inspect to export image/container metadata related to graph driver
Export image/container metadata stored in graph driver. Right now 3 fields
DeviceId, DeviceSize and DeviceName are being exported from devicemapper.
Other graph drivers can export fields as they see fit.

This data can be used to mount the thin device outside of docker and tools
can look into image/container and do some kind of inspection.

Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
2015-06-15 14:05:10 -04:00
Arnaud Porterie
00b8fec75f Merge pull request #13773 from dmcgowan/refactor-1-image-graph-separation
refactor: separate graph from image
2015-06-11 17:44:37 -07:00
John Howard
141cd2a1f2 Show actual number of elements restored
Signed-off-by: John Howard <jhoward@microsoft.com>
2015-06-10 15:18:51 -07:00
Derek McGowan
c0b4421819 Update graph to use digest type
Update get and set functions to use digests.
Update push code to use the digest type instead of string

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2015-06-05 18:07:41 -07:00
Derek McGowan
bb50a4159b Update graph walkhistory to pass by value
Remove unused graph history function

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2015-06-05 18:06:09 -07:00
Derek McGowan
2b58b677a5 Separate graph from image
Move graph related functions in image to graph package.
Consolidating graph functionality is the first step in refactoring graph into an image store model.
Subsequent refactors will involve breaking up graph into multiple types with a strongly defined interface.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2015-06-05 18:06:09 -07:00
Matt Moore
bd2575cc4f Make the v2 logic fallback on v1 when v2 requests cannot be authorized.
Signed-off-by: Matt Moore <mattmoor@google.com>
2015-06-02 21:22:59 -07:00
yuchangchun
e0475d331b add a more accurate error description for invalid tag name
Signed-off-by: yuchangchun <yuchangchun1@huawei.com>
2015-06-03 09:38:39 +08:00
Arnaud Porterie
274baf70bf Merge pull request #13576 from stevvooe/verify-digests
Properly verify manifests and layer digests on pull
2015-06-02 11:16:23 -07:00
Richard
6e4ff1bb13 If no endpoint could be established with the given mirror configuration,
fallback to pulling from the hub as per v1 behavior.

Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2015-06-01 15:18:56 -07:00
Stephen J Day
84413be3c9 Break down loadManifest function into constituent parts
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-06-01 13:02:50 -07:00
Stephen J Day
74528be903 Add tests for loadManifest digest verification
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-05-29 16:30:21 -07:00
Stephen J Day
1e653ab645 Attempt to retain tagging behavior
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-05-29 15:20:37 -07:00
Stephen J Day
06612cc0fe Properly verify manifests and layer digests on pull
To ensure manifest integrity when pulling by digest, this changeset ensures
that not only the remote digest provided by the registry is verified but also
that the digest provided on the command line is checked, as well. If this check
fails, the pull is cancelled as with an error. Inspection also should that
while layers were being verified against their digests, the error was being
treated as tech preview image signing verification error. This, in fact, is not
a tech preview and opens up the docker daemon to man in the middle attacks that
can be avoided with the v2 registry protocol.

As a matter of cleanliness, the digest package from the distribution project
has been updated to latest version. There were some recent improvements in the
digest package.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-05-29 15:20:28 -07:00
Tibor Vass
2daede5a9c Merge pull request #13374 from RichardScothern/v2-mirror
V2 mirror support
2015-05-27 21:15:26 -04:00
Richard
e817e08481 Review feedback:
- Match verbiage with other output
    - Remove dead code and clearer flow

Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2015-05-27 17:29:24 -07:00