Commit graph

209 commits

Author SHA1 Message Date
Daniel Nephin
c87d67b0ad Refacator pkg/streamformatter
StreamFormatter suffered was two distinct structs mixed into a single struct
without any overlap.

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-05-02 17:38:12 -04:00
Vincent Demeester
4261687d78 Merge pull request #31720 from jonjohnsonjr/always-head
Always HEAD blobs at least once during pushes
2017-03-16 14:39:02 +01:00
Tibor Vass
e10f9db26d Merge pull request #30047 from duglin/distError
Add the mediaType to the error
2017-03-10 16:44:49 -08:00
Jon Johnson
cdc46abde5 Always HEAD blobs at least once during pushes
Signed-off-by: Jon Johnson <jonjohnson@google.com>
2017-03-08 15:36:12 -08:00
Sebastiaan van Stijn
57fd478169 Merge pull request #31083 from xulike666/fight-for-readability
fix some typos from module contrib to man
2017-03-03 15:13:01 +01:00
Akihiro Suda
91f11437b6 Merge pull request #30888 from estesp/moar-manifestlist-debug-output
Add debug output to manifest list parsing
2017-03-03 13:09:38 +09:00
Doug Davis
c127d9614f Add the mediaType to the error
Without this fix the error the client might see is:
	target is unknown
which wasn't helpful to me when I saw this today. With this fix I
now see:
	MediaType is unknown: 'text/html'
which helped me track down the issue to the registry I was talking to.

Signed-off-by: Doug Davis <dug@us.ibm.com>
2017-02-28 11:36:32 -08:00
Phil Estes
9a8cb9313c
Add debug output to manifest list parsing
Per request for more debug info on how the engine deals with
multi-platform "manifest list" images, this adds information about the
manifest list entries and whether it found an os/arch match, and the
digest of the match.

Docker-DCO-1.1-Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com>
2017-02-28 14:11:13 -05:00
Sebastiaan van Stijn
8606144159
log fallback errors as "info"
Fallback errors are not an error, but an
informational message.

This changes those errors to be logged
as "Info" instead of "Error".

After this patch, debug logs look like this;

    DEBU[0050] Calling GET /_ping
    DEBU[0050] Calling POST /v1.27/images/create?fromImage=localhost%3A5000%2Ffoo&tag=latest
    DEBU[0050] Trying to pull localhost:5000/foo from https://localhost:5000 v2
    WARN[0050] Error getting v2 registry: Get https://localhost:5000/v2/: http: server gave HTTP response to HTTPS client
    INFO[0050] Attempting next endpoint for pull after error: Get https://localhost:5000/v2/: http: server gave HTTP response to HTTPS client
    DEBU[0050] Trying to pull localhost:5000/foo from http://localhost:5000 v2
    INFO[0050] Attempting next endpoint for pull after error: manifest unknown: manifest unknown
    DEBU[0050] Trying to pull localhost:5000/foo from https://localhost:5000 v1
    DEBU[0050] attempting v1 ping for registry endpoint https://localhost:5000/v1/
    DEBU[0050] Fallback from error: Get https://localhost:5000/v1/_ping: http: server gave HTTP response to HTTPS client
    INFO[0050] Attempting next endpoint for pull after error: Get https://localhost:5000/v1/_ping: http: server gave HTTP response to HTTPS client
    DEBU[0050] Trying to pull localhost:5000/foo from http://localhost:5000 v1
    DEBU[0050] [registry] Calling GET http://localhost:5000/v1/repositories/foo/images
    ERRO[0050] Not continuing with pull after error: Error: image foo:latest not found

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2017-02-24 14:27:35 +01:00
Aaron.L.Xu
e0577d5fe8 fix some typos from module contrib to man
Signed-off-by: Aaron.L.Xu <likexu@harmonycloud.cn>
2017-02-18 10:08:55 +08:00
Vincent Demeester
c31f73abfd Merge pull request #29001 from darrenstahlmsft/WindowsOnLinux
Block pulling Windows images on non-Windows daemons
2017-02-16 19:57:54 +01:00
Derek McGowan
3a1279393f
Use distribution reference
Remove forked reference package. Use normalized named values
everywhere and familiar functions to convert back to familiar
strings for UX and storage compatibility.

Enforce that the source repository in the distribution metadata
is always a normalized string, ignore invalid values which are not.
Update distribution tests to use normalized values.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2017-02-07 11:08:37 -08:00
Darren Stahl
d55304065b Block Windows images on Linux
Signed-off-by: Darren Stahl <darst@microsoft.com>
2017-02-02 11:07:30 -08:00
Matt Moore
d5482089bf Add tracking to elided layer pulls.
Signed-off-by: Matt Moore <mattmoor@google.com>
2017-01-31 16:36:17 -08:00
Aaron Lehmann
e9c0de0de6 Merge pull request #30043 from dmcgowan/distribution-reference-update-1
Distribution reference update
2017-01-24 20:38:20 -08:00
Sebastiaan van Stijn
f633764913 Merge pull request #30262 from aaronlehmann/text-html
distribution: Add text/html and application/json as image mediatypes
2017-01-24 01:13:35 +01:00
Derek McGowan
0421f5173d
Remove use of forked reference package for cli
Use resolving to repo info as the split point between the
legitimate reference package and forked reference package.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2017-01-19 16:04:50 -08:00
Aaron Lehmann
a215e15cb1 distribution: Add text/html and application/json as image mediatypes
As noted by #30083, the new strict checking of mediatypes misses some
cases where earlier bugs caused nonstandard mediatypes to be stored in
manifests. Two of the known cases are text/html and application/json,
which were returned by certain registries and stored by earlier versions
of Docker. Add special cases for text/html and application/json.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2017-01-19 11:16:28 -08:00
Sebastiaan van Stijn
de0328560b Merge pull request #30165 from xulike666/fix-typo-6/36
[combined] fix typo
2017-01-19 17:21:22 +01:00
Aaron.L.Xu
40af569164 fix typo
fix typo I found AMAP in integration-cli/*

fix typo mentioned by Allencloud

Signed-off-by: Aaron.L.Xu <likexu@harmonycloud.cn>
2017-01-19 15:52:28 +08:00
Daehyeok Mun
6306019d0b Refactoring ineffectual assignments
This patch fixed below 4 types of code line
1. Remove unnecessary variable assignment
2. Use variables declaration instead of explicit initial zero value
3. Change variable name to underbar when variable not used
4. Add erro check and return for ignored error

Signed-off-by: Daehyeok Mun <daehyeok@gmail.com>
2017-01-13 00:27:01 -07:00
Stefan J. Wernli
d14b7212ad storeLayer.Parent should return describableStoreLayers
When storeLayer.Parent returns the parent layer, it needs to use the same logic as Get where it wraps in a describablyStoreLayer if the layer is describable. Otherwise, on Windows, this can result in pushing the foreign layers, which is not supposed to be allowed.
This fixes https://github.com/docker/docker/issues/30080.

Signed-off-by: Stefan J. Wernli <swernli@microsoft.com>
2017-01-11 19:01:13 -08:00
Stephen J Day
7a85579917
*: use opencontainers/go-digest package
The `digest` data type, used throughout docker for image verification
and identity, has been broken out into `opencontainers/go-digest`. This
PR updates the dependencies and moves uses over to the new type.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-01-06 18:48:41 -08:00
Sebastiaan van Stijn
d6be0e9802 Merge pull request #28631 from likel/master
Don't do format if it's unnecessary
2016-12-30 01:13:12 +01:00
Brian Goff
631f51015e Merge pull request #29684 from vdemeester/quick-unit
Enhance pkg/{httputils,integration}, distribution/xfer unit tests
2016-12-28 10:57:56 -05:00
Vincent Demeester
522bfd926b
Enhance pkg/{httputils,integration}, distribution/xfer unit tests
- Make it possible to define a shorter waiting time of httputils
- Make a small hack to reduce the waiting time on distribution/xfer

Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2016-12-28 11:54:05 +01:00
Brian Goff
bf6eb85539 Merge pull request #29666 from tonistiigi/client-deps
Clean up client binary dependencies
2016-12-27 15:01:24 -05:00
yuexiao-wang
08c6575caf Remove unused variable
Signed-off-by: yuexiao-wang <wang.yuexiao@zte.com.cn>
2016-12-28 03:20:41 +08:00
Ke Li
514adcf458 Remove redundant format
Signed-off-by: Ke Li <kel@splunk.com>

Add missing changes

Signed-off-by: Ke Li <kel@splunk.com>

User errors.New to create error

Signed-off-by: Ke Li <kel@splunk.com>
2016-12-27 21:46:52 +08:00
Tonis Tiigi
13222160e8 Define PushResult in api types
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2016-12-26 18:53:22 -08:00
Tonis Tiigi
3d86b0c79b Implement content addressability for plugins
Move plugins to shared distribution stack with images.

Create immutable plugin config that matches schema2 requirements.

Ensure data being pushed is same as pulled/created.

Store distribution artifacts in a blobstore.

Run init layer setup for every plugin start.

Fix breakouts from unsafe file accesses.

Add support for `docker plugin install --alias`

Uses normalized references for default names to avoid collisions when using default hosts/tags.

Some refactoring of the plugin manager to support the change, like removing the singleton manager and adding manager config struct.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2016-12-23 13:29:58 -08:00
Brian Goff
748a9a3f9c Merge pull request #29628 from liliVicky/my-feature4
modify some errors of plurality
2016-12-22 15:40:30 -05:00
Sebastiaan van Stijn
b9ee31ae02 Merge pull request #29314 from vdemeester/no-more-utils
Remove the utils package
2016-12-22 15:21:05 +01:00
liwenqi
128d07d349 modify some files
Signed-off-by: liwenqi <vikilwq@zju.edu.cn>

update some files in the folder of distribution/xfer

Signed-off-by: liwenqi <vikilwq@zju.edu.cn>

correct again

Signed-off-by: liwenqi <vikilwq@zju.edu.cn>
2016-12-22 08:38:37 +08:00
Derek McGowan
3c7676a057
Abstract distribution interfaces from image specific types
Move configurations into a single file.
Abstract download manager in pull config.
Add supports for schema2 only and schema2 type checking.
Add interface for providing push layers.
Abstract image store to generically handle configurations.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2016-12-19 10:55:00 -08:00
Anusha Ragunathan
8de2336f07 Merge pull request #29230 from vieux/remove_old_media_type_plugin
remove old media type compat for plugins
2016-12-12 12:43:32 -08:00
Vincent Demeester
0ab9320ab2
Move utils.TestDirectory to pkg/testutil/tempfile
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2016-12-12 09:28:32 +01:00
Victor Vieux
c1a1b381f9 Merge pull request #28459 from dmcgowan/plugin-repository-pinning
Plugin repository pinning
2016-12-08 11:28:52 -08:00
Victor Vieux
f644e758bd remove old media type compat for plugins
Signed-off-by: Victor Vieux <vieux@docker.com>
2016-12-07 17:35:09 -08:00
Alexander Morozov
523ad6ac71 Merge pull request #28966 from aaronlehmann/push-panic
distribution: Fix panic on push
2016-11-30 07:26:58 -08:00
Vincent Demeester
b5f4d3f716 Merge pull request #28954 from darrenstahlmsft/UnknownBlobRetry
Stop retrying pull on UnkownBlob error
2016-11-30 10:05:16 +01:00
Aaron Lehmann
e0702e9f37 distribution: Fix panic on push
When building a manifest during a push operation, all layers must have
an associated descriptor. If a layer is missing a descriptor, that leads
to a panic.

A break inside a switch in layerAlreadyExists meant to break from the
loop surrounding the switch, but instead breaks from the switch. This
causes the loop to continue, and can overwrite the descriptor with an
empty one, leading to the panic.

Also, fix layerAlreadyExists not to abort the push when a speculative
stat on a candidate layer digest fails with an error. This could happen
in situations like a potential cross-repository mount where the user
does not have permission to access the source repository.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-11-29 18:13:39 -08:00
Daehyeok Mun
d117a2f614 Refactoring functions
Modified newV1DependencyImage and generateDependencyImages not to return
errors instead of always return nil.

Signed-off-by: Daehyeok Mun <daehyeok@gmail.com>
2016-11-29 16:52:37 -07:00
Darren Stahl
dee2968560 Stop retrying pull on UnkownBlob error
Signed-off-by: Darren Stahl <darst@microsoft.com>
2016-11-29 15:06:25 -08:00
Derek McGowan
a12b466183
Add class to repository scope
Expose registry error translation for plugin distribution

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2016-11-21 22:18:50 -08:00
Vivek Goyal
f7f3d34210 Move CreateRWLayer() parameters in a struct
Move some of the optional parameters of CreateRWLayer() in a struct
called CreateRWLayerOpts. This will make it easy to add more options
arguments without having to change signature of CreateRWLayer().

Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
2016-11-16 16:31:23 -05:00
Derek McGowan
19a93a6e3d
Update pull error handling
Translate pull errors to provide a more consistent and user friendly
error message.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2016-11-10 17:34:12 -08:00
Derek McGowan
c85eb00841
Update for distribution vendor
Handle updates to reference package.
Updates for refactoring of challenge manager.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2016-11-10 17:34:12 -08:00
Victor Vieux
970b23db2e update media_type
Signed-off-by: Victor Vieux <vieux@docker.com>
2016-11-10 15:51:32 -08:00
Aaron Lehmann
99c59d5988 distribution: Remove misleading comment
Remove the following comment in pullV2Tag:

// NOTE: not using TagService.Get, since it uses HEAD requests
// against the manifests endpoint, which are not supported by
// all registry versions.

This is actually not an issue, because TagService.Get does a fallback to
GET if HEAD fails. It has done this ever since TagService was added to
the distribution API, so this comment was probably based on an early
version of TagService before it was merged, or was always a
misunderstanding.

However, we continue to use ManifestService.Get instead because it
saves a round trip. The manifest can be retrieved directly instead of
resolving the digest first.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-11-04 17:10:47 -07:00