Refactoring in Docker 1.7 changed the behavior to add this header where as Docker <= 1.6 wouldn't emit this Header on a HTTP 302 redirect.
This closes#13649
Signed-off-by: Jeffrey van Gogh <jvg@google.com>
It should not print to STDOUT so that it only prints the debugTransport
output if there was an error in one of the registry tests.
Signed-off-by: Tibor Vass <tibor@docker.com>
To ensure manifest integrity when pulling by digest, this changeset ensures
that not only the remote digest provided by the registry is verified but also
that the digest provided on the command line is checked, as well. If this check
fails, the pull is cancelled as with an error. Inspection also should that
while layers were being verified against their digests, the error was being
treated as tech preview image signing verification error. This, in fact, is not
a tech preview and opens up the docker daemon to man in the middle attacks that
can be avoided with the v2 registry protocol.
As a matter of cleanliness, the digest package from the distribution project
has been updated to latest version. There were some recent improvements in the
digest package.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
The v2 registry will act as a pull-through cache, and needs to be
handled differently by the client to the v1 registry mirror.
See docker/distribution#459 for details
Configuration
Only one v2 registry can be configured as a mirror. Acceptable configurations
in this chanage are: 0...n v1 mirrors or 1 v2 mirror. A mixture of v1 and v2
mirrors is considered an error.
Pull
If a v2 mirror is configured, all pulls are redirected to that mirror. The
mirror will serve the content locally or attempt a pull from the upstream mirror,
cache it locally, and then serve to the client.
Push
If an image is tagged to a mirror, it will be pushed to the mirror and be
stored locally there. Otherwise, images are pushed to the hub. This is
unchanged behavior.
Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
This patch removes the need for requestFactories and decorators
by implementing http.RoundTripper transports instead.
It refactors some challenging-to-read code.
NewSession now takes an *http.Client that can already have a
custom Transport, it will add its own auth transport by wrapping
it.
The idea is that callers of http.Client should not bother
setting custom headers for every handler but instead it should
be transparent to the callers of a same context.
This patch is needed for future refactorings of registry,
namely refactoring of the v1 client code.
Signed-off-by: Tibor Vass <tibor@docker.com>
Thanks to @dmcgowan for noticing.
Added a testcase to make sure Save() can create the dir and then
read from it.
Signed-off-by: Doug Davis <dug@us.ibm.com>
This PR does the following:
- migrated ~/.dockerfg to ~/.docker/config.json. The data is migrated
but the old file remains in case its needed
- moves the auth json in that fie into an "auth" property so we can add new
top-level properties w/o messing with the auth stuff
- adds support for an HttpHeaders property in ~/.docker/config.json
which adds these http headers to all msgs from the cli
In a follow-on PR I'll move the config file process out from under
"registry" since it not specific to that any more. I didn't do it here
because I wanted the diff to be smaller so people can make sure I didn't
break/miss any auth code during my edits.
Signed-off-by: Doug Davis <dug@us.ibm.com>
registry/SearchResults was missing the "is_automated" field.
I added it back in.
Pull this 'table' removal one from the others because it fixed
a bug too
Signed-off-by: Doug Davis <dug@us.ibm.com>
This makes `registry.Service` a first class type and does not use jobs
to interact with this type.
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
Update registry package to use the v2 registry api from distribution. Update interfaces to directly take in digests.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
No longer add the body to the error when a 404 is received on get repository data.
closes#11510
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Pushing a v2 image layer has two steps:
- POST to get a new upload URL
- PUT to that upload URL
We were previously not checking the response code of
the POST request and the PUT would fail in weird ways.
Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
Currently the http clients used by auth use the default tls config. The config needs to be updated to only support TLS1.0 and newer as well as respect registry insecure configuration.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
When docker push get response with unknown HTTP status, docker daemon
print:
"Error: Status XXX trying to push repository XXX: XXX"
But when docker pull meets response with unknown status code, it gives:
"HTTP code: XXX"
This commit helps docker pull print more detailed error info like push
does, so push and pull can behave consistently when error happens.
Signed-off-by: Zhang Wei <zhangwei555@huawei.com>
Add ability to refer to an image by repository name and digest using the
format repository@digest. Works for pull, push, run, build, and rmi.
Signed-off-by: Andy Goldstein <agoldste@redhat.com>
Currently when registry error strings contain new line characters only the last line is displayed to the client. Quote the string to ensure the client can see the entire body value.
fixes#11346
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Currently when the registry ping is sent, it creates the request directly from http.NewRequest instead of from the http request factory. The request factory adds useful header information such as user agent which is needed by the registry.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Private registries should support having images pushed with only a single name component (e.g. localhost:5000/myapp).
The public registry currently requires two name components, but this is already enforced in the registry code.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
This modifies the "docker help" text so that it is no wider than 80 chars
and each description fits on one line. This will also try to use ~ when
possible
Added a test to make sure we don't go over 80 chars again.
Added a test to make sure we use ~
Applied rules/tests to all docker commands - not just main help text
Closes#10214
Signed-off-by: Doug Davis <dug@us.ibm.com>
When requesting a token, the basic auth header is always being set even
if there is no username value. This patch corrects this and does not set
the basic auth header if the username is empty.
Also fixes an issue where pulling all tags from a v2 registry succeeds
when the image does not actually exist on the registry.
Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
When getting the URL from a v2 registry url builder, it does not
honor the scheme from the endpoint object and will cause an https
endpoint to return urls starting with http.
Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
Under certain cases, such as when putting a manifest or check for the existence
of a layer, the status code checks in session_v2.go were too narrow for their
purpose. In the case of putting a manifest, the handler only cares that an
error is not returned. Whether it is a 304 or 202 does not matter, as long as
the server reports success. Having the client only accept specific http codes
inhibits future protocol evolution.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
Since the Docker-Distribution-API-Version header value may contain multiple
space delimited versions as well as many instances of the header key, the
header value is now split on whitespace characters to iterate over all versions
that may be listed in one instance of the header.
Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
v2 ping now checks for a Docker-Distribution-API-Version
header that identifies the endpoint as "registry/2.0"
Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
Fixes#10129
Makes the .dockercfg more human parsable.
Also cleaned up the (technically) racey login test.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Token cache prevents the need to get a new token for every registry interaction.
Since the tokens are short lived, the cache expires after only a minute.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
While the v2 pull operation is writing the body of the layer blob to disk
it now computes the tarsum checksum of the archive before extracting it to
the backend storage driver. If the checksum does not match that from the
image manifest an error is raised.
Also adds more debug logging to the pull operation and fixes existing test
cases which were failing. Adds a reverse lookup constructor to the tarsum
package so that you can get a tarsum object using a checksum label.
Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
Because docker core cannot vendor non-master Go dependencies, we need to remove
dependencies on registry package. The definition of digest.Digest has been
changed to a string and the regular expressions have been ported from
docker-registry/common library.
We'll likely change this be dependent on the registry in the future when the
API stabilizies and use of the master branch becomes the norm.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
This package, ported from next-generation docker regsitry, includes route and
error definitions. These facilitate compliant V2 client implementation. The
portions of the HTTP API that are included in this package are considered to be
locked down and should only be changed through a careful change proposal.
Descriptor definitions package layout may change without affecting API behavior
until the exported Go API is ready to be locked down.
When the new registry stabilizes and becomes the master branch, this package
can be vendored from the registry.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
Registry authorization token is now taken from the response body rather than
the repsonse header.
Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
summary of changes:
registry/auth.go
- More logging around the login functions
- split Login() out to handle different code paths for v1 (unchanged logic)
and v2 (does not currently do account creation)
- handling for either basic or token based login attempts
registry/authchallenge.go
- New File
- credit to Brian Bland <brian.bland@docker.com> (github: BrianBland)
- handles parsing of WWW-Authenticate response headers
registry/endpoint.go
- EVEN MOAR LOGGING
- Many edits throught to make the coad less dense. Sparse code is more
readable code.
- slit Ping() out to handle different code paths for v1 (unchanged logic)
and v2.
- Updated Endpoint struct type to include an entry for authorization
challenges discovered during ping of a v2 registry.
- If registry endpoint version is unknown, v2 code path is first attempted,
then fallback to v1 upon failure.
registry/service.go
- STILL MOAR LOGGING
- simplified the logic around starting the 'auth' job.
registry/session.go
- updated use of a registry.Endpoint struct field.
registry/token.go
- New File
- Handles getting token from the parameters of a token auth challenge.
- Modified from function written by Brian Bland (see above credit).
registry/types.go
- Removed 'DefaultAPIVersion' in lieu of 'APIVersionUnknown = 0'`
Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
Passing RepositoryInfo to ResolveAuthConfig, pullRepository, and pushRepository
Moving --registry-mirror configuration to registry config
Created resolve_repository job
Repo names with 'index.docker.io' or 'docker.io' are now synonymous with omitting an index name.
Adding test for RepositoryInfo
Adding tests for opts.StringSetOpts and registry.ValidateMirror
Fixing search term use of repoInfo
Adding integration tests for registry mirror configuration
Normalizing LookupImage image name to match LocalName parsing rules
Normalizing repository LocalName to avoid multiple references to an official image
Removing errorOut use in tests
Removing TODO comment
gofmt changes
golint comments cleanup. renaming RegistryOptions => registry.Options, and RegistryServiceConfig => registry.ServiceConfig
Splitting out builtins.Registry and registry.NewService calls
Stray whitespace cleanup
Moving integration tests for Mirrors and InsecureRegistries into TestNewIndexInfo unit test
Factoring out ValidateRepositoryName from NewRepositoryInfo
Removing unused IndexServerURL
Allowing json marshaling of ServiceConfig. Exposing ServiceConfig in /info
Switching to CamelCase for json marshaling
PR cleanup; removing 'Is' prefix from boolean members. Removing unneeded json tags.
Removing non-cleanup related fix for 'localhost:[port]' in splitReposName
Merge fixes for gh9735
Fixing integration test
Reapplying #9754
Adding comment on config.IndexConfigs use from isSecureIndex
Remove unused error return value from isSecureIndex
Signed-off-by: Don Kjer <don.kjer@gmail.com>
Adding back comment in isSecureIndex
Signed-off-by: Don Kjer <don.kjer@gmail.com>
If `--insecure-registry mydomain.com` was specified, it would match a registry at mydomain.com on any port.
This was accidentally added in #9735 and is now being reverted.
Signed-off-by: Tibor Vass <teabee89@gmail.com>
This commit is patch for following comment
// TODO: This method should return the errors instead of masking them and returning false
Signed-off-by: Daehyeok Mun <daehyeok@gmail.com>
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
Incase of a 3xx redirect the var was being overshowed and ever changed
causing an infinite loop.
Fixes#9480
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
This ensures that ServerAddress is set, while previously it was getting
set after configFile.Configs.
Signed-off-by: Vaidas Jablonskis <jablonskis@gmail.com>
Fix issue with restoring the tag store and setting static configuration
from the daemon. i.e. the field on the TagStore struct must be made
internal or the json.Unmarshal in restore will overwrite the insecure
registries to be an empty struct.
Signed-off-by: Michael Crosby <michael@docker.com>
Conflicts:
graph/pull.go
graph/push.go
graph/tags.go
We do this to prevent leakage of information, we don't want people
to be able to probe for existing content.
According to RFC 2616, "This status code (404) is commonly used when the server does not
wish to reveal exactly why the request has been refused, or when no other response i
is applicable."
https://www.ietf.org/rfc/rfc2616.txt
10.4.4 403 Forbidden
The server understood the request, but is refusing to fulfill it.
Authorization will not help and the request SHOULD NOT be repeated.
If the request method was not HEAD and the server wishes to make
public why the request has not been fulfilled, it SHOULD describe the
reason for the refusal in the entity. If the server does not wish to
make this information available to the client, the status code 404
(Not Found) can be used instead.
10.4.5 404 Not Found
The server has not found anything matching the Request-URI. No
indication is given of whether the condition is temporary or
permanent. The 410 (Gone) status code SHOULD be used if the server
knows, through some internally configurable mechanism, that an old
resource is permanently unavailable and has no forwarding address.
This status code is commonly used when the server does not wish to
reveal exactly why the request has been refused, or when no other
response is applicable.
When docker is running through its certificates, it should continue
trying with a new certificate even if it gets back a 404 error code.
Docker-DCO-1.1-Signed-off-by: Dan Walsh <dwalsh@redhat.com> (github: rhatdan)
Add support for pulling signed images from a version 2 registry.
Only official images within the library namespace will be pull from the
new registry and check the build signature.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
This introduces Versions for TarSum checksums.
Fixes: https://github.com/docker/docker/issues/7526
It preserves current functionality and abstracts the interface for
future flexibility of hashing algorithms. As a POC, the VersionDev
Tarsum does not include the mtime in the checksum calculation, and would
solve https://github.com/docker/docker/issues/7387 though this is not a
settled Version is subject to change until a version number is assigned.
Signed-off-by: Vincent Batts <vbatts@redhat.com>
Per registry.doRequest, res and client might be nil in case of error
For example, dns resolution errors, /etc/docker/certs.d perms, failed
loading of x509 cert ...
This will make res.StatusCode and res.Body SEGFAULT.
Signed-off-by: Arthur Gautier <baloo@gandi.net>
To avoid conflicting with layer IDs, repository names must
not be tagged with names that collide with hexadecimal strings.
Signed-off-by: Eric Windisch <eric@windisch.us>
The cli interface works similar to other registry related commands:
docker search foo
... searches for foo on the official hub
docker search localhost:5000/foo
... does the same for the private reg at localhost:5000
Signed-off-by: Daniel Menet <membership@sontags.ch>
renaming this struct to more clearly be session, as that is what it
handles.
Splitting out files for easier readability.
Signed-off-by: Vincent Batts <vbatts@redhat.com>
functions to pkg/parsers/kernel, and parsing filters to
pkg/parsers/filter. Adjust imports and package references.
Docker-DCO-1.1-Signed-off-by: Erik Hollensbe <github@hollensbe.org> (github: erikh)
This lets you specify custom client TLS certificates and CA root for a
specific registry hostname. Docker will then verify the registry
against the CA and present the client cert when talking to that
registry. This allows the registry to verify that the client has a
proper key, indicating that the client is allowed to access the
images.
A custom cert is configured by creating a directory in
/etc/docker/certs.d with the same name as the registry hostname. Inside
this directory all *.crt files are added as CA Roots (if none exists,
the system default is used) and pair of files <filename>.key and
<filename>.cert indicate a custom certificate to present to the registry.
If there are multiple certificates each one will be tried in
alphabetical order, proceeding to the next if we get a 403 of 5xx
response.
So, an example setup would be:
/etc/docker/certs.d/
└── localhost
├── client.cert
├── client.key
└── localhost.crt
A simple way to test this setup is to use an apache server to host a
registry. Just copy a registry tree into the apache root, here is an
example one containing the busybox image:
http://people.gnome.org/~alexl/v1.tar.gz
Then add this conf file as /etc/httpd/conf.d/registry.conf:
# This must be in the root context, otherwise it causes a re-negotiation
# which is not supported by the tls implementation in go
SSLVerifyClient optional_no_ca
<Location /v1>
Action cert-protected /cgi-bin/cert.cgi
SetHandler cert-protected
Header set x-docker-registry-version "0.6.2"
SetEnvIf Host (.*) custom_host=$1
Header set X-Docker-Endpoints "%{custom_host}e"
</Location>
And this as /var/www/cgi-bin/cert.cgi
#!/bin/bash
if [ "$HTTPS" != "on" ]; then
echo "Status: 403 Not using SSL"
echo "x-docker-registry-version: 0.6.2"
echo
exit 0
fi
if [ "$SSL_CLIENT_VERIFY" == "NONE" ]; then
echo "Status: 403 Client certificate invalid"
echo "x-docker-registry-version: 0.6.2"
echo
exit 0
fi
echo "Content-length: $(stat --printf='%s' $PATH_TRANSLATED)"
echo "x-docker-registry-version: 0.6.2"
echo "X-Docker-Endpoints: $SERVER_NAME"
echo "X-Docker-Size: 0"
echo
cat $PATH_TRANSLATED
This will return 403 for all accessed to /v1 unless *any* client cert
is presented. Obviously a real implementation would verify more details
about the certificate.
Example client certs can be generated with:
openssl genrsa -out client.key 1024
openssl req -new -x509 -text -key client.key -out client.cert
Docker-DCO-1.1-Signed-off-by: Alexander Larsson <alexl@redhat.com> (github: alexlarsson)
These constants don't need to use time.Duration(). Fixup this file since
it seems to be the only one using this style.
Docker-DCO-1.1-Signed-off-by: Brandon Philips <brandon.philips@coreos.com> (github: philips)
This continues the effort to separate all registry logic from the
deprecated `Server` object.
* 'search' is exposed by `github.com/dotcloud/docker/registry/Service`
* Added proper documentation of Search while I was at it
Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
This is the first step towards separating the registry subsystem from
the deprecated `Server` object.
* New service `github.com/dotcloud/docker/registry/Service`
* The service is installed by default in `builtins`
* The service only exposes `auth` for now...
* ...Soon to be followed by `pull`, `push` and `search`.
Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
roll version and standalone information into the _ping. And to support
Headers they are checked after the JSON is loaded (if there is anything
to load). To stay backwards compatible, if the _ping contents are not
able to unmarshal to RegistryInfo, do not stop, but continue with the
same behavior.
Docker-DCO-1.1-Signed-off-by: Vincent Batts <vbatts@redhat.com> (github: vbatts)
For a pull-only, static registry, there only a couple of headers that
need to be optional (that are presently required.
* X-Docker-Registry-Version
* X-Docker-Size
* X-Docker-Endpoints
Docker-DCO-1.1-Signed-off-by: Vincent Batts <vbatts@redhat.com> (github: vbatts)
Since docker uses cookiejar it doesn't need to manage cookies manually
anymore.
Managing cookie was duplicating it.
Docker-DCO-1.1-Signed-off-by: Fabio Falci <fabiofalci@gmail.com> (github: fabiofalci)
- Added an argument to the call() method in order to control the auth sharing
- Enabled it only for search. Pulls and pushes were enabled already.
- Grouped a few variable declarations
Docker-DCO-1.1-Signed-off-by: Roberto Hashioka <roberto.hashioka@docker.com> (github: rogaha)
RequestFactory is no longer a singleton (can be different for different instances of Registry)
Registry now has an indexEndpoint member
Registry methods that needed the indexEndpoint parameter no longer do so
Registry methods will only use token auth where applicable if basic auth is not enabled.
To improve the use of docker with a private registry the login
command is extended with a parameter for the server address.
While implementing i noticed that two problems hindered authentication to a
private registry:
1. the resolve of the authentication did not match during push
because the looked up key was for example localhost:8080 but
the stored one would have been https://localhost:8080
Besides The lookup needs to still work if the https->http fallback
is used
2. During pull of an image no authentication is sent, which
means all repositories are expected to be private.
These points are fixed now. The changes are implemented in
a way to be compatible to existing behavior both in the
API as also with the private registry.
Update:
- login does not require the full url any more, you can login
to the repository prefix:
example:
docker logon localhost:8080
Fixed corner corner cases:
- When login is done during pull and push the registry endpoint is used and
not the central index
- When Remote sends a 401 during pull, it is now correctly delegating to
CmdLogin
- After a Login is done pull and push are using the newly entered login data,
and not the previous ones. This one seems to be also broken in master, too.
- Auth config is now transfered in a parameter instead of the body when
/images/create is called.
