Commit graph

571 commits

Author SHA1 Message Date
Sebastiaan van Stijn
cb76638699
registry/search: pass User-Agent through headers
Commit 3991faf464 moved search into the registry
package, which also made the `dockerversion` package a dependency for registry,
which brings additional (indirect) dependencies, such as `pkg/parsers/kernel`,
and `golang.org/x/sys/windows/registry`.

Client code, such as used in docker/cli may depend on the `registry` package,
but should not depend on those additional dependencies.

This patch moves setting the userAgent to the API router, and instead of
passing it as a separate argument, includes it into the "headers".

As these headers now not only contain the `X-Meta-...` headers, the variables
were renamed accordingly.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-03-21 14:41:29 +01:00
Cory Snider
7b3acdff5d registry: return concrete service type
Move interface definitions to the packages which use the registry
service.

https://github.com/golang/go/wiki/CodeReviewComments#interfaces

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-03-10 18:38:08 -05:00
Cory Snider
3991faf464 Move filtered registry search out of image service
SearchRegistryForImages does not make sense as part of the image
service interface. The implementation just wraps the search API of the
registry service to filter the results client-side. It has nothing to do
with local image storage, and the implementation of search does not need
to change when changing which backend (graph driver vs. containerd
snapshotter) is used for local image storage.

Filtering of the search results is an implementation detail: the
consumer of the results does not care which actor does the filtering so
long as the results are filtered as requested. Move filtering into the
exported API of the registry service to hide the implementation details.
Only one thing---the registry service implementation---would need to
change in order to support server-side filtering of search results if
Docker Hub or other registry servers were to add support for it to their
APIs.

Use a fake registry server in the search unit tests to avoid having to
mock out the registry API client.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-03-10 18:36:33 -05:00
Cory Snider
19ad4ac03d registry: acquire mutex in IsInsecureRegistry
The mutex needs to be held when accessing s.config to prevent data
races.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-02-28 19:42:49 -05:00
Paweł Gronowski
9032e6779d c8d/resolver: Fallback to http for insecure registries
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-01-11 17:00:27 +01:00
Sebastiaan van Stijn
72ea8cfb43
registry: use http.NewServeMux for tests instead of gorilla/mux
We don't need gorilla/mux for these tests, so let's use stdlib.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-19 13:25:43 +01:00
Sebastiaan van Stijn
a44f547343
registry: session: remove unused id
This removes the dependency on github.com/docker/docker/pkg/stringid

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-29 19:10:07 +01:00
Sebastiaan van Stijn
30e5333ce3
registry: allow "allow-nondistributable-artifacts" for Docker Hub
Previously, Docker Hub was excluded when configuring "allow-nondistributable-artifacts".
With the updated policy announced by Microsoft, we can remove this restriction;
https://techcommunity.microsoft.com/t5/containers/announcing-windows-container-base-image-redistribution-rights/ba-p/3645201

There are plans to deprecated support for foreign layers altogether in the OCI,
and we should consider to make this option the default, but as that requires
deprecating the option (and possibly keeping an "opt-out" option), we can look
at that separately.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-16 01:08:49 +02:00
Sebastiaan van Stijn
8a2e1245d4
runconfig, oci, image, layer, distribution: fix empty-lines (revive)
runconfig/config_test.go:23:46: empty-lines: extra empty line at the start of a block (revive)
    runconfig/config_test.go:75:55: empty-lines: extra empty line at the start of a block (revive)

    oci/devices_linux.go:57:34: empty-lines: extra empty line at the start of a block (revive)
    oci/devices_linux.go:60:69: empty-lines: extra empty line at the start of a block (revive)

    image/fs_test.go:53:38: empty-lines: extra empty line at the end of a block (revive)
    image/tarexport/save.go:88:29: empty-lines: extra empty line at the end of a block (revive)

    layer/layer_unix_test.go:21:34: empty-lines: extra empty line at the end of a block (revive)

    distribution/xfer/download.go:302:9: empty-lines: extra empty line at the end of a block (revive)
    distribution/manifest_test.go:154:99: empty-lines: extra empty line at the end of a block (revive)
    distribution/manifest_test.go:329:52: empty-lines: extra empty line at the end of a block (revive)
    distribution/manifest_test.go:354:59: empty-lines: extra empty line at the end of a block (revive)

    registry/config_test.go:323:42: empty-lines: extra empty line at the end of a block (revive)
    registry/config_test.go:350:33: empty-lines: extra empty line at the end of a block (revive)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-28 01:58:52 +02:00
Sebastiaan van Stijn
d817f4dcee
registry: use types/registry.AuthConfig
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-29 23:05:20 +02:00
Eng Zer Jun
7873c27cfb
all: replace strings.Replace with strings.ReplaceAll
strings.ReplaceAll(s, old, new) is a wrapper function for
strings.Replace(s, old, new, -1). But strings.ReplaceAll is more
readable and removes the hardcoded -1.

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2022-05-09 19:45:40 +08:00
Sebastiaan van Stijn
85572cac14
registry: remove dependency on rootlesskit, add SetCertsDir()
The registry package contained code to automatically set the CertsDir() path,
based on wether or not the daemon was running in rootlessmode. In doing so,
it made use of the `pkg/rootless.RunningWithRootlessKit()` utility.

A recent change in de6732a403 added additional
functionality in the `pkg/rootless` package, introducing a dependency on
`github.com/rootless-containers/rootlesskit`. Unfortunately, the extra
dependency also made its way into the docker cli, which also uses the
registry package.

This patch introduces a new `SetCertsDir()` function, which allows
the default certs-directory to be overridden, and updates the daemon
to configure this location during startup.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-25 16:21:45 +01:00
Sebastiaan van Stijn
d5d5f258df
Merge pull request #43394 from thaJeztah/cleanup_registry_step2
registry: remove more dead code
2022-03-18 19:35:02 +01:00
Sebastiaan van Stijn
5e9829b75d
registry: remove unused fallbackError
Nothing was emitting this error.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-18 15:09:16 +01:00
Sebastiaan van Stijn
8947730124
registry: un-export AuthTransport()
It's only used internally for v1 search

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-18 13:52:20 +01:00
Sebastiaan van Stijn
2f466a9f88
registry: remove unused Service.TLSConfig()
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-18 13:30:13 +01:00
Sebastiaan van Stijn
a5be5801e9
search: un-export registry.DefaultSearchLimit, and fix API status codes
Move the default to the service itself, and produce the correct status code
if an invalid limit was specified. The default is currently set both on the
cli and on the daemon side, and it should be only set on one of them.

There is a slight change in behavior; previously, searching with `--limit=0`
would produce an error, but with this change, it's considered the equivalent
of "no limit set" (and using the default).

We could keep the old behavior by passing a pointer (`nil` means "not set"),
but I left that for a follow-up exercise (we may want to pass an actual
config instead of separate arguments, as well as some other things that need
cleaning up).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-18 09:41:56 +01:00
Sebastiaan van Stijn
bee8f006fe
registry: serviceConfig: don't embed registry.ServiceConfig
Just define it as a type, which makes it slightly easier to consume.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-17 17:12:30 +01:00
Sebastiaan van Stijn
2b5dc81582
registry: move allowNondistributableArtifacts, isSecureIndex to config
This felt slightly more natural to make it a function of the config type itself.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-17 17:12:28 +01:00
Sebastiaan van Stijn
382b986520
registry: make defaultService.ServiceConfig() more idiomatic
The intent of this function is to return a copy of the service's configuration,
and to copy / dereference the options in its configuration.

The code was doing this in slightly complicated fashion. This patch;

- adds a `copy()` function to serviceConfig
- rewrites the code to use a slightly more idiomatic approach, using one of
  the approaches described in "golang SliceTricks" https://github.com/golang/go/wiki/SliceTricks#copy
- changes defaultService.ServiceConfig() to use this function, and updates
  its godoc to better describe that it returns a copy.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-17 17:12:27 +01:00
Sebastiaan van Stijn
18de76a420
registry: make serviceConfig.loadInsecureRegistries() more atomic
This removes the ugly hack where we stored the current config, tried to
reconfigure the service, and rolled back to the stored copy on failures.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-17 17:12:25 +01:00
Sebastiaan van Stijn
dae2173568
registry: defaultService: use sync.RWMutex
Most operations only require read access, so change this to use an RWMutex,
and some minor refactoring in lookupV2Endpoints() so that we are not
constructing tlsconfig multiple times in some cases.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-17 17:12:23 +01:00
Sebastiaan van Stijn
9cb0aa4c91
registry: minor improvements and cleanup
- registry: newIndexInfo(): minor refactor
- registry: loadAllowNondistributableArtifacts() minor refactor
  initialise the slices with a length.
- registry: defaultService.Search(): minor refactor
  Perform all manipulation earlier, so that it's not needed to scroll up
  to learn what's done.
- various other minor cleanups

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-17 17:12:21 +01:00
Sebastiaan van Stijn
d3c3e2c867
registry: remove unneeded alias for api/types/registry import
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-17 17:12:19 +01:00
Sebastiaan van Stijn
79aa65c1fa
registry: return "errdefs" compatible error types
Adding some small utility functions to make generating them easier.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-17 17:12:13 +01:00
Sebastiaan van Stijn
98202c86ad
registry: remove ErrInvalidRepositoryName
This error was only returned in a single location, and not used anywhere
as a specific type.

The error returned by `validateNoScheme()` also appeared to only be used in
one case; in all other cases, the error itself was ignored, and replaced with
a custom error. Because of this, this patch also replace `validateNoScheme()`
with a `hasScheme()` function that returns a boolean, to better match how it's
used.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-17 17:09:15 +01:00
Sebastiaan van Stijn
273dca4e3c
registry: remove unused error return from HostCertsDir()
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-17 17:09:13 +01:00
Sebastiaan van Stijn
6e3e657ea6
registry: un-export config.LoadXXX() functions
Un-export:

- config.LoadAllowNondistributableArtifacts()
- config.LoadInsecureRegistries()
- config.LoadMirrors()

The config type is already un-exported; this also un-exports these functions
to be explicit they're internal only.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-17 17:09:11 +01:00
Sebastiaan van Stijn
4ebb18479d
registry: un-export Session, remove NewSession()
It's only used internally for search.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-17 17:09:09 +01:00
Sebastiaan van Stijn
286992ef53
registry: un-export Ping(), PingResult, remove v1Endpoint.Path()
These are only used internally, and the v1Endpoint.Path() function was only
used to get the `_ping` URL, so let's inline that code instead.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-17 17:09:07 +01:00
Sebastiaan van Stijn
6a01a3cfa4
registry: un-export V1Endpoint and NewV1Endpoint()
They're deprecated, and only used internally for the v1 search.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-17 17:09:00 +01:00
Sebastiaan van Stijn
8b8bbbd445
registry: un-export HTTPClient() and NewTransport()
They're only used internally.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-17 17:08:05 +01:00
Sebastiaan van Stijn
569dc6d692
registry: un-export DefaultService
The DefaultService was not really meant to be used outside of the package, so
un-export it, and change NewService()'s signature to return a Service interface.

To un-export this type, a test in daemon/images was updated to not use DefaultService,
but now using the registry.Service interface itself.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-17 17:08:04 +01:00
Sebastiaan van Stijn
541ed077a6
registry: remove unused authConfig from Session
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-17 17:08:02 +01:00
Sebastiaan van Stijn
02ed265854
registry: remove unused RepositoryData and ImgData
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-17 17:08:00 +01:00
Sebastiaan van Stijn
2bcf4628a7
registry: remove TODO for removing localhost as insecure by default
While this was intended t be a stop-gap solution, it's been there for years and
users depend on this. It's also still complicated to secure _localhost_, so
by now, we'd probably have to be realistic, and consider this to be "permanent".

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-17 17:07:58 +01:00
Sebastiaan van Stijn
d9261561f9
registry: add TODO for removing ParseSearchIndexInfo()
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-17 17:07:56 +01:00
Sebastiaan van Stijn
746c51b54f
registry: tests: remove unused mock code and use gotest.tools
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-17 17:07:48 +01:00
Sebastiaan van Stijn
7d70d95d8e
registry: add DefaultRegistryHost const, and improve documentation
This is more in line with other consts that are used for defaults, and makes it
slightly easier to consume than DefaultV2Registry, e.g. see:
https://github.com/oras-project/oras-go/blob/v1.1.0/pkg/auth/docker/resolver.go#L81-L84

Note that both the "index.docker.io" and "registry-1.docker.io" domains
are here for historic reasons and backward-compatibility. These domains
are still supported by Docker Hub (and will continue to be supported), but
there are new domains already in use, and plans to consolidate all legacy
domains to new "canonical" domains. Once those domains are decided on, we
should update these consts (but making sure to preserve compatibility with
existing installs, clients, and user configuration).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-02-27 13:48:27 +01:00
Sebastiaan van Stijn
ff408210da
distribution: remove use of deprecated dial.DualStack
From the field's description [1]:

    DualStack previously enabled RFC 6555 Fast Fallback
    support, also known as "Happy Eyeballs", in which IPv4 is
    tried soon if IPv6 appears to be misconfigured and
    hanging.

    Deprecated: Fast Fallback is enabled by default. To
    disable, set FallbackDelay to a negative value.

This field was deprecated in efc185029b,
which is included in Go 1.12beta1 and up.

[1]: 2ebe77a2fd/src/net/dial.go (L54-L61)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-02-22 16:34:12 +01:00
Sebastiaan van Stijn
971e03d9bb
Merge pull request #42915 from thaJeztah/registry_cleanup
registry: clean up some v1 code
2021-10-15 10:01:22 +02:00
Sebastiaan van Stijn
9dbec13362
registry: EndPointV1.Ping() remove redundant "Standalone" and cleanup logs
Standalone is a boolean, so false by default; also cleanup some debug logs
(probably more logs can be removed)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-10-07 16:26:39 +02:00
Sebastiaan van Stijn
37dc2582d1
registry: remove use of iota for consts
I think it's a bit more readable to just use a literal value
for these; this also prevents having to use `_` to skip zero.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-10-05 21:14:17 +02:00
Sebastiaan van Stijn
4a52c46e37
registry: trimV1Address(): simplify trimming trailing slash
strings.TrimSuffix() does exactly the same as this code, but is
a bit more readable.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-10-05 21:12:10 +02:00
Sebastiaan van Stijn
542edf0c21
registry: inline newV1Endpoint() into newV1EndpointFromStr()
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-10-05 20:49:33 +02:00
Sebastiaan van Stijn
c8754f44d7
registry: remove unused ToV1Endpoint()
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-10-05 14:43:58 +02:00
Sebastiaan van Stijn
afbeec8bda
registry: remove tlsConfigForMirror()
This function was just a shallow wrapper around tlsConfig(), so remove
the abstraction.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-10-05 14:37:53 +02:00
Sebastiaan van Stijn
9f874e53b9
registry: remove unused registry.ErrAlreadyExists
This error was no longer in use after the v1 push code was removed
in 53dad9f027.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-10-05 14:15:32 +02:00
Eng Zer Jun
c55a4ac779
refactor: move from io/ioutil to io and os package
The io/ioutil package has been deprecated in Go 1.16. This commit
replaces the existing io/ioutil functions with their new definitions in
io and os packages.

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2021-08-27 14:56:57 +08:00
Sebastiaan van Stijn
686be57d0a
Update to Go 1.17.0, and gofmt with Go 1.17
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-08-24 23:33:27 +02:00