Commit graph

592 commits

Author SHA1 Message Date
Sebastiaan van Stijn
1148a24e64
migrate to new github.com/distribution/reference module
The "reference" package was moved to a separate module, which was extracted
from b9b19409cf

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-05 12:09:26 +02:00
Sebastiaan van Stijn
5da2dd98e9
registry: move search-related code to separate files
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-29 16:45:43 +02:00
Sebastiaan van Stijn
f1a7028900
registry: rename v1-endpoint tests to have a common prefix
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-29 16:45:43 +02:00
Sebastiaan van Stijn
19f6f0b3db
registry: move v1 endpoint tests to endpoint_test.go
Moves the TestPingRegistryEndpoint and TestEndpoint tests.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-29 16:45:43 +02:00
Sebastiaan van Stijn
47bf587f84
registry: ResolveAuthConfig: rename var that collided with import
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-29 16:45:18 +02:00
Sebastiaan van Stijn
2832886761
Merge pull request #46361 from thaJeztah/registry_update_godoc
registry: update some godoc
2023-08-29 14:45:51 +02:00
Sebastiaan van Stijn
3f8cbd8ec9
registry: update some godoc
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-29 13:22:14 +02:00
Sebastiaan van Stijn
d43e61758a
registry: deprecate APIEndpoint.Version and APIVersion type
This field was used when the code supported both "v1" and "v2" registries.
We no longer support v1 registries, and the only v1 endpoint that's still
used is for the legacy "search" endpoint, which does not use the APIEndpoint
type.

As no code is using this field, and the value will always be set to "v2",
we can deprecated the Version field.

I'm keeping this field for 1 release, to give notice to any potential
external consumer, after which we can delete it.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-29 12:49:46 +02:00
Sebastiaan van Stijn
aa59b0f5a2
registry: improve error for invalid search endpoints
Explain that search is not supported on v2 endpoints, and include the
offending endpoint in the error-message.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-29 10:47:29 +02:00
Sebastiaan van Stijn
14b53c6318
registry: simplify trimV1Address
First, remove the loop over `apiVersions`. The `apiVersions` map has two
entries (`APIVersion1 => "v1"`, and `APIVersion2 => "v2"`), and `APIVersion1`
is skipped, which means that the loop effectively translates to;

    if apiVersionStr == "v2" {
        return "", invalidParamf("unsupported V1 version path %s", apiVersionStr)
    }

Which leaves us with "anything else" being returned as-is.

This patch removes the loop, and replaces the remaining handling to check
for the "v2" suffix to produce an error, or to strip the "v1" suffix.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-29 10:45:39 +02:00
Sebastiaan van Stijn
062c80199f
registry: combine TestEndpointParse and TestEndpointParseInvalid
Combine the two tests into a TestV1EndpointParse function, and rewrite
them to use gotest.tools for asserting.

Also changing the test-cases to use "https://", as the scheme doesn't
matter for this test, but using "http://" may trip-up some linters,
so let's avoid that.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-29 10:36:46 +02:00
Sebastiaan van Stijn
69c9adb7d3
Merge pull request #45925 from thaJeztah/deprecate_is_automated
api: search: deprecate is_automated field, and is-automated filter
2023-08-01 16:27:00 +02:00
Sebastiaan van Stijn
971083d419
api: search: deprecate is_automated field, and is-automated filter
The is-automated field is being deprecated by Docker Hub's search API,
and will always be set to "false" in future.

This patch deprecates the field and related filter for the Engine's API.

In future, the `is-automated` filter will no longer yield any results
when searching for `is-automated=true`, and will be ignored when
searching for `is-automated=false`.

Given that this field is deprecated by an external API, the deprecation
will not be versioned, and will apply to any API version.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-01 13:45:44 +02:00
Sebastiaan van Stijn
5e2a1195d7
swap logrus types for their containerd/logs aliases
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-01 13:02:55 +02:00
Sebastiaan van Stijn
c5c977855d
registry: session.searchRepositories(): return typed error, and small cleanup
- return a errdefs.System if we fail to decode the registry's response
- use strconv.Itoa instead of fmt.Sprintf

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-11 09:47:02 +02:00
Sebastiaan van Stijn
68ebfa2f18
registry: session.searchRepositories(): log actual search URL
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-11 09:41:52 +02:00
Sebastiaan van Stijn
27698f1d13
registry: ParseSearchIndexInfo: remove redundant error-handling
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-10 17:48:10 +02:00
Sebastiaan van Stijn
784ba59a08
registry: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:31:49 +02:00
Brian Goff
74da6a6363 Switch all logging to use containerd log pkg
This unifies our logging and allows us to propagate logging and trace
contexts together.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2023-06-24 00:23:44 +00:00
Cory Snider
742ac6e275 daemon: make config reloading more transactional
Config reloading has interleaved validations and other fallible
operations with mutating the live daemon configuration. The daemon
configuration could be left in a partially-reloaded state if any of the
operations returns an error. Mutating a copy of the configuration and
atomically swapping the config struct on success is not currently an
option as config values are not copyable due to the presence of
sync.Mutex fields. Introduce a two-phase commit protocol to defer any
mutations of the daemon state until after all fallible operations have
succeeded.

Reload transactions are not yet entirely hermetic. The platform
reloading logic for custom runtimes on *nix could still leave the
directory of generated runtime wrapper scripts in an indeterminate state
if an error is encountered.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-06-01 14:45:24 -04:00
Sebastiaan van Stijn
ab35df454d
remove pre-go1.17 build-tags
Removed pre-go1.17 build-tags with go fix;

    go mod init
    go fix -mod=readonly ./...
    rm go.mod

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-19 20:38:51 +02:00
Sebastiaan van Stijn
cb76638699
registry/search: pass User-Agent through headers
Commit 3991faf464 moved search into the registry
package, which also made the `dockerversion` package a dependency for registry,
which brings additional (indirect) dependencies, such as `pkg/parsers/kernel`,
and `golang.org/x/sys/windows/registry`.

Client code, such as used in docker/cli may depend on the `registry` package,
but should not depend on those additional dependencies.

This patch moves setting the userAgent to the API router, and instead of
passing it as a separate argument, includes it into the "headers".

As these headers now not only contain the `X-Meta-...` headers, the variables
were renamed accordingly.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-03-21 14:41:29 +01:00
Cory Snider
7b3acdff5d registry: return concrete service type
Move interface definitions to the packages which use the registry
service.

https://github.com/golang/go/wiki/CodeReviewComments#interfaces

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-03-10 18:38:08 -05:00
Cory Snider
3991faf464 Move filtered registry search out of image service
SearchRegistryForImages does not make sense as part of the image
service interface. The implementation just wraps the search API of the
registry service to filter the results client-side. It has nothing to do
with local image storage, and the implementation of search does not need
to change when changing which backend (graph driver vs. containerd
snapshotter) is used for local image storage.

Filtering of the search results is an implementation detail: the
consumer of the results does not care which actor does the filtering so
long as the results are filtered as requested. Move filtering into the
exported API of the registry service to hide the implementation details.
Only one thing---the registry service implementation---would need to
change in order to support server-side filtering of search results if
Docker Hub or other registry servers were to add support for it to their
APIs.

Use a fake registry server in the search unit tests to avoid having to
mock out the registry API client.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-03-10 18:36:33 -05:00
Cory Snider
19ad4ac03d registry: acquire mutex in IsInsecureRegistry
The mutex needs to be held when accessing s.config to prevent data
races.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-02-28 19:42:49 -05:00
Paweł Gronowski
9032e6779d c8d/resolver: Fallback to http for insecure registries
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-01-11 17:00:27 +01:00
Sebastiaan van Stijn
72ea8cfb43
registry: use http.NewServeMux for tests instead of gorilla/mux
We don't need gorilla/mux for these tests, so let's use stdlib.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-19 13:25:43 +01:00
Sebastiaan van Stijn
a44f547343
registry: session: remove unused id
This removes the dependency on github.com/docker/docker/pkg/stringid

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-29 19:10:07 +01:00
Sebastiaan van Stijn
30e5333ce3
registry: allow "allow-nondistributable-artifacts" for Docker Hub
Previously, Docker Hub was excluded when configuring "allow-nondistributable-artifacts".
With the updated policy announced by Microsoft, we can remove this restriction;
https://techcommunity.microsoft.com/t5/containers/announcing-windows-container-base-image-redistribution-rights/ba-p/3645201

There are plans to deprecated support for foreign layers altogether in the OCI,
and we should consider to make this option the default, but as that requires
deprecating the option (and possibly keeping an "opt-out" option), we can look
at that separately.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-16 01:08:49 +02:00
Sebastiaan van Stijn
8a2e1245d4
runconfig, oci, image, layer, distribution: fix empty-lines (revive)
runconfig/config_test.go:23:46: empty-lines: extra empty line at the start of a block (revive)
    runconfig/config_test.go:75:55: empty-lines: extra empty line at the start of a block (revive)

    oci/devices_linux.go:57:34: empty-lines: extra empty line at the start of a block (revive)
    oci/devices_linux.go:60:69: empty-lines: extra empty line at the start of a block (revive)

    image/fs_test.go:53:38: empty-lines: extra empty line at the end of a block (revive)
    image/tarexport/save.go:88:29: empty-lines: extra empty line at the end of a block (revive)

    layer/layer_unix_test.go:21:34: empty-lines: extra empty line at the end of a block (revive)

    distribution/xfer/download.go:302:9: empty-lines: extra empty line at the end of a block (revive)
    distribution/manifest_test.go:154:99: empty-lines: extra empty line at the end of a block (revive)
    distribution/manifest_test.go:329:52: empty-lines: extra empty line at the end of a block (revive)
    distribution/manifest_test.go:354:59: empty-lines: extra empty line at the end of a block (revive)

    registry/config_test.go:323:42: empty-lines: extra empty line at the end of a block (revive)
    registry/config_test.go:350:33: empty-lines: extra empty line at the end of a block (revive)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-28 01:58:52 +02:00
Sebastiaan van Stijn
d817f4dcee
registry: use types/registry.AuthConfig
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-29 23:05:20 +02:00
Eng Zer Jun
7873c27cfb
all: replace strings.Replace with strings.ReplaceAll
strings.ReplaceAll(s, old, new) is a wrapper function for
strings.Replace(s, old, new, -1). But strings.ReplaceAll is more
readable and removes the hardcoded -1.

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2022-05-09 19:45:40 +08:00
Sebastiaan van Stijn
85572cac14
registry: remove dependency on rootlesskit, add SetCertsDir()
The registry package contained code to automatically set the CertsDir() path,
based on wether or not the daemon was running in rootlessmode. In doing so,
it made use of the `pkg/rootless.RunningWithRootlessKit()` utility.

A recent change in de6732a403 added additional
functionality in the `pkg/rootless` package, introducing a dependency on
`github.com/rootless-containers/rootlesskit`. Unfortunately, the extra
dependency also made its way into the docker cli, which also uses the
registry package.

This patch introduces a new `SetCertsDir()` function, which allows
the default certs-directory to be overridden, and updates the daemon
to configure this location during startup.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-25 16:21:45 +01:00
Sebastiaan van Stijn
d5d5f258df
Merge pull request #43394 from thaJeztah/cleanup_registry_step2
registry: remove more dead code
2022-03-18 19:35:02 +01:00
Sebastiaan van Stijn
5e9829b75d
registry: remove unused fallbackError
Nothing was emitting this error.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-18 15:09:16 +01:00
Sebastiaan van Stijn
8947730124
registry: un-export AuthTransport()
It's only used internally for v1 search

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-18 13:52:20 +01:00
Sebastiaan van Stijn
2f466a9f88
registry: remove unused Service.TLSConfig()
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-18 13:30:13 +01:00
Sebastiaan van Stijn
a5be5801e9
search: un-export registry.DefaultSearchLimit, and fix API status codes
Move the default to the service itself, and produce the correct status code
if an invalid limit was specified. The default is currently set both on the
cli and on the daemon side, and it should be only set on one of them.

There is a slight change in behavior; previously, searching with `--limit=0`
would produce an error, but with this change, it's considered the equivalent
of "no limit set" (and using the default).

We could keep the old behavior by passing a pointer (`nil` means "not set"),
but I left that for a follow-up exercise (we may want to pass an actual
config instead of separate arguments, as well as some other things that need
cleaning up).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-18 09:41:56 +01:00
Sebastiaan van Stijn
bee8f006fe
registry: serviceConfig: don't embed registry.ServiceConfig
Just define it as a type, which makes it slightly easier to consume.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-17 17:12:30 +01:00
Sebastiaan van Stijn
2b5dc81582
registry: move allowNondistributableArtifacts, isSecureIndex to config
This felt slightly more natural to make it a function of the config type itself.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-17 17:12:28 +01:00
Sebastiaan van Stijn
382b986520
registry: make defaultService.ServiceConfig() more idiomatic
The intent of this function is to return a copy of the service's configuration,
and to copy / dereference the options in its configuration.

The code was doing this in slightly complicated fashion. This patch;

- adds a `copy()` function to serviceConfig
- rewrites the code to use a slightly more idiomatic approach, using one of
  the approaches described in "golang SliceTricks" https://github.com/golang/go/wiki/SliceTricks#copy
- changes defaultService.ServiceConfig() to use this function, and updates
  its godoc to better describe that it returns a copy.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-17 17:12:27 +01:00
Sebastiaan van Stijn
18de76a420
registry: make serviceConfig.loadInsecureRegistries() more atomic
This removes the ugly hack where we stored the current config, tried to
reconfigure the service, and rolled back to the stored copy on failures.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-17 17:12:25 +01:00
Sebastiaan van Stijn
dae2173568
registry: defaultService: use sync.RWMutex
Most operations only require read access, so change this to use an RWMutex,
and some minor refactoring in lookupV2Endpoints() so that we are not
constructing tlsconfig multiple times in some cases.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-17 17:12:23 +01:00
Sebastiaan van Stijn
9cb0aa4c91
registry: minor improvements and cleanup
- registry: newIndexInfo(): minor refactor
- registry: loadAllowNondistributableArtifacts() minor refactor
  initialise the slices with a length.
- registry: defaultService.Search(): minor refactor
  Perform all manipulation earlier, so that it's not needed to scroll up
  to learn what's done.
- various other minor cleanups

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-17 17:12:21 +01:00
Sebastiaan van Stijn
d3c3e2c867
registry: remove unneeded alias for api/types/registry import
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-17 17:12:19 +01:00
Sebastiaan van Stijn
79aa65c1fa
registry: return "errdefs" compatible error types
Adding some small utility functions to make generating them easier.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-17 17:12:13 +01:00
Sebastiaan van Stijn
98202c86ad
registry: remove ErrInvalidRepositoryName
This error was only returned in a single location, and not used anywhere
as a specific type.

The error returned by `validateNoScheme()` also appeared to only be used in
one case; in all other cases, the error itself was ignored, and replaced with
a custom error. Because of this, this patch also replace `validateNoScheme()`
with a `hasScheme()` function that returns a boolean, to better match how it's
used.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-17 17:09:15 +01:00
Sebastiaan van Stijn
273dca4e3c
registry: remove unused error return from HostCertsDir()
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-17 17:09:13 +01:00
Sebastiaan van Stijn
6e3e657ea6
registry: un-export config.LoadXXX() functions
Un-export:

- config.LoadAllowNondistributableArtifacts()
- config.LoadInsecureRegistries()
- config.LoadMirrors()

The config type is already un-exported; this also un-exports these functions
to be explicit they're internal only.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-17 17:09:11 +01:00
Sebastiaan van Stijn
4ebb18479d
registry: un-export Session, remove NewSession()
It's only used internally for search.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-17 17:09:09 +01:00