moby/distribution/repository.go

52 lines
1.6 KiB
Go
Raw Permalink Normal View History

package distribution
import (
"context"
api: fix "GET /distribution" endpoint ignoring mirrors If the daemon is configured to use a mirror for the default (Docker Hub) registry, the endpoint did not fall back to querying the upstream if the mirror did not contain the given reference. If the daemon is configured to use a mirror for the default (Docker Hub) registry, did not fall back to querying the upstream if the mirror did not contain the given reference. For pull-through registry-mirrors, this was not a problem, as in that case the registry would forward the request, but for other mirrors, no fallback would happen. This was inconsistent with how "pulling" images handled this situation; when pulling images, both the mirror and upstream would be tried. This problem was caused by the logic used in GetRepository, which had an optimization to only return the first registry it was successfully able to configure (and connect to), with the assumption that the mirror either contained all images used, or to be configured as a pull-through mirror. This patch: - Introduces a GetRepositories method, which returns all candidates (both mirror(s) and upstream). - Updates the endpoint to try all Before this patch: # the daemon is configured to use a mirror for Docker Hub cat /etc/docker/daemon.json { "registry-mirrors": ["http://localhost:5000"]} # start the mirror (empty registry, not configured as pull-through mirror) docker run -d --name registry -p 127.0.0.1:5000:5000 registry:2 # querying the endpoint fails, because the image-manifest is not found in the mirror: curl -s --unix-socket /var/run/docker.sock http://localhost/v1.43/distribution/docker.io/library/hello-world:latest/json { "message": "manifest unknown: manifest unknown" } With this patch applied: # the daemon is configured to use a mirror for Docker Hub cat /etc/docker/daemon.json { "registry-mirrors": ["http://localhost:5000"]} # start the mirror (empty registry, not configured as pull-through mirror) docker run -d --name registry -p 127.0.0.1:5000:5000 registry:2 # querying the endpoint succeeds (manifest is fetched from the upstream Docker Hub registry): curl -s --unix-socket /var/run/docker.sock http://localhost/v1.43/distribution/docker.io/library/hello-world:latest/json | jq . { "Descriptor": { "mediaType": "application/vnd.oci.image.index.v1+json", "digest": "sha256:1b9844d846ce3a6a6af7013e999a373112c3c0450aca49e155ae444526a2c45e", "size": 3849 }, "Platforms": [ { "architecture": "amd64", "os": "linux" } ] } Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-04 12:20:35 +00:00
"github.com/containerd/log"
"github.com/distribution/reference"
"github.com/docker/distribution"
"github.com/docker/docker/errdefs"
)
api: fix "GET /distribution" endpoint ignoring mirrors If the daemon is configured to use a mirror for the default (Docker Hub) registry, the endpoint did not fall back to querying the upstream if the mirror did not contain the given reference. If the daemon is configured to use a mirror for the default (Docker Hub) registry, did not fall back to querying the upstream if the mirror did not contain the given reference. For pull-through registry-mirrors, this was not a problem, as in that case the registry would forward the request, but for other mirrors, no fallback would happen. This was inconsistent with how "pulling" images handled this situation; when pulling images, both the mirror and upstream would be tried. This problem was caused by the logic used in GetRepository, which had an optimization to only return the first registry it was successfully able to configure (and connect to), with the assumption that the mirror either contained all images used, or to be configured as a pull-through mirror. This patch: - Introduces a GetRepositories method, which returns all candidates (both mirror(s) and upstream). - Updates the endpoint to try all Before this patch: # the daemon is configured to use a mirror for Docker Hub cat /etc/docker/daemon.json { "registry-mirrors": ["http://localhost:5000"]} # start the mirror (empty registry, not configured as pull-through mirror) docker run -d --name registry -p 127.0.0.1:5000:5000 registry:2 # querying the endpoint fails, because the image-manifest is not found in the mirror: curl -s --unix-socket /var/run/docker.sock http://localhost/v1.43/distribution/docker.io/library/hello-world:latest/json { "message": "manifest unknown: manifest unknown" } With this patch applied: # the daemon is configured to use a mirror for Docker Hub cat /etc/docker/daemon.json { "registry-mirrors": ["http://localhost:5000"]} # start the mirror (empty registry, not configured as pull-through mirror) docker run -d --name registry -p 127.0.0.1:5000:5000 registry:2 # querying the endpoint succeeds (manifest is fetched from the upstream Docker Hub registry): curl -s --unix-socket /var/run/docker.sock http://localhost/v1.43/distribution/docker.io/library/hello-world:latest/json | jq . { "Descriptor": { "mediaType": "application/vnd.oci.image.index.v1+json", "digest": "sha256:1b9844d846ce3a6a6af7013e999a373112c3c0450aca49e155ae444526a2c45e", "size": 3849 }, "Platforms": [ { "architecture": "amd64", "os": "linux" } ] } Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-04 12:20:35 +00:00
// GetRepositories returns a list of repositories configured for the given
// reference. Multiple repositories can be returned if the reference is for
// the default (Docker Hub) registry and a mirror is configured, but it omits
// registries that were not reachable (pinging the /v2/ endpoint failed).
//
// It returns an error if it was unable to reach any of the registries for
// the given reference, or if the provided reference is invalid.
func GetRepositories(ctx context.Context, ref reference.Named, config *ImagePullConfig) ([]distribution.Repository, error) {
repoInfo, err := config.RegistryService.ResolveRepository(ref)
if err != nil {
return nil, errdefs.InvalidParameter(err)
}
// makes sure name is not empty or `scratch`
if err := validateRepoName(repoInfo.Name); err != nil {
return nil, errdefs.InvalidParameter(err)
}
endpoints, err := config.RegistryService.LookupPullEndpoints(reference.Domain(repoInfo.Name))
if err != nil {
return nil, err
}
api: fix "GET /distribution" endpoint ignoring mirrors If the daemon is configured to use a mirror for the default (Docker Hub) registry, the endpoint did not fall back to querying the upstream if the mirror did not contain the given reference. If the daemon is configured to use a mirror for the default (Docker Hub) registry, did not fall back to querying the upstream if the mirror did not contain the given reference. For pull-through registry-mirrors, this was not a problem, as in that case the registry would forward the request, but for other mirrors, no fallback would happen. This was inconsistent with how "pulling" images handled this situation; when pulling images, both the mirror and upstream would be tried. This problem was caused by the logic used in GetRepository, which had an optimization to only return the first registry it was successfully able to configure (and connect to), with the assumption that the mirror either contained all images used, or to be configured as a pull-through mirror. This patch: - Introduces a GetRepositories method, which returns all candidates (both mirror(s) and upstream). - Updates the endpoint to try all Before this patch: # the daemon is configured to use a mirror for Docker Hub cat /etc/docker/daemon.json { "registry-mirrors": ["http://localhost:5000"]} # start the mirror (empty registry, not configured as pull-through mirror) docker run -d --name registry -p 127.0.0.1:5000:5000 registry:2 # querying the endpoint fails, because the image-manifest is not found in the mirror: curl -s --unix-socket /var/run/docker.sock http://localhost/v1.43/distribution/docker.io/library/hello-world:latest/json { "message": "manifest unknown: manifest unknown" } With this patch applied: # the daemon is configured to use a mirror for Docker Hub cat /etc/docker/daemon.json { "registry-mirrors": ["http://localhost:5000"]} # start the mirror (empty registry, not configured as pull-through mirror) docker run -d --name registry -p 127.0.0.1:5000:5000 registry:2 # querying the endpoint succeeds (manifest is fetched from the upstream Docker Hub registry): curl -s --unix-socket /var/run/docker.sock http://localhost/v1.43/distribution/docker.io/library/hello-world:latest/json | jq . { "Descriptor": { "mediaType": "application/vnd.oci.image.index.v1+json", "digest": "sha256:1b9844d846ce3a6a6af7013e999a373112c3c0450aca49e155ae444526a2c45e", "size": 3849 }, "Platforms": [ { "architecture": "amd64", "os": "linux" } ] } Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-04 12:20:35 +00:00
var (
repositories []distribution.Repository
lastError error
)
for _, endpoint := range endpoints {
api: fix "GET /distribution" endpoint ignoring mirrors If the daemon is configured to use a mirror for the default (Docker Hub) registry, the endpoint did not fall back to querying the upstream if the mirror did not contain the given reference. If the daemon is configured to use a mirror for the default (Docker Hub) registry, did not fall back to querying the upstream if the mirror did not contain the given reference. For pull-through registry-mirrors, this was not a problem, as in that case the registry would forward the request, but for other mirrors, no fallback would happen. This was inconsistent with how "pulling" images handled this situation; when pulling images, both the mirror and upstream would be tried. This problem was caused by the logic used in GetRepository, which had an optimization to only return the first registry it was successfully able to configure (and connect to), with the assumption that the mirror either contained all images used, or to be configured as a pull-through mirror. This patch: - Introduces a GetRepositories method, which returns all candidates (both mirror(s) and upstream). - Updates the endpoint to try all Before this patch: # the daemon is configured to use a mirror for Docker Hub cat /etc/docker/daemon.json { "registry-mirrors": ["http://localhost:5000"]} # start the mirror (empty registry, not configured as pull-through mirror) docker run -d --name registry -p 127.0.0.1:5000:5000 registry:2 # querying the endpoint fails, because the image-manifest is not found in the mirror: curl -s --unix-socket /var/run/docker.sock http://localhost/v1.43/distribution/docker.io/library/hello-world:latest/json { "message": "manifest unknown: manifest unknown" } With this patch applied: # the daemon is configured to use a mirror for Docker Hub cat /etc/docker/daemon.json { "registry-mirrors": ["http://localhost:5000"]} # start the mirror (empty registry, not configured as pull-through mirror) docker run -d --name registry -p 127.0.0.1:5000:5000 registry:2 # querying the endpoint succeeds (manifest is fetched from the upstream Docker Hub registry): curl -s --unix-socket /var/run/docker.sock http://localhost/v1.43/distribution/docker.io/library/hello-world:latest/json | jq . { "Descriptor": { "mediaType": "application/vnd.oci.image.index.v1+json", "digest": "sha256:1b9844d846ce3a6a6af7013e999a373112c3c0450aca49e155ae444526a2c45e", "size": 3849 }, "Platforms": [ { "architecture": "amd64", "os": "linux" } ] } Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-04 12:20:35 +00:00
repo, err := newRepository(ctx, repoInfo, endpoint, nil, config.AuthConfig, "pull")
if err != nil {
log.G(ctx).WithFields(log.Fields{"endpoint": endpoint.URL.String(), "error": err}).Info("endpoint")
lastError = err
continue
}
api: fix "GET /distribution" endpoint ignoring mirrors If the daemon is configured to use a mirror for the default (Docker Hub) registry, the endpoint did not fall back to querying the upstream if the mirror did not contain the given reference. If the daemon is configured to use a mirror for the default (Docker Hub) registry, did not fall back to querying the upstream if the mirror did not contain the given reference. For pull-through registry-mirrors, this was not a problem, as in that case the registry would forward the request, but for other mirrors, no fallback would happen. This was inconsistent with how "pulling" images handled this situation; when pulling images, both the mirror and upstream would be tried. This problem was caused by the logic used in GetRepository, which had an optimization to only return the first registry it was successfully able to configure (and connect to), with the assumption that the mirror either contained all images used, or to be configured as a pull-through mirror. This patch: - Introduces a GetRepositories method, which returns all candidates (both mirror(s) and upstream). - Updates the endpoint to try all Before this patch: # the daemon is configured to use a mirror for Docker Hub cat /etc/docker/daemon.json { "registry-mirrors": ["http://localhost:5000"]} # start the mirror (empty registry, not configured as pull-through mirror) docker run -d --name registry -p 127.0.0.1:5000:5000 registry:2 # querying the endpoint fails, because the image-manifest is not found in the mirror: curl -s --unix-socket /var/run/docker.sock http://localhost/v1.43/distribution/docker.io/library/hello-world:latest/json { "message": "manifest unknown: manifest unknown" } With this patch applied: # the daemon is configured to use a mirror for Docker Hub cat /etc/docker/daemon.json { "registry-mirrors": ["http://localhost:5000"]} # start the mirror (empty registry, not configured as pull-through mirror) docker run -d --name registry -p 127.0.0.1:5000:5000 registry:2 # querying the endpoint succeeds (manifest is fetched from the upstream Docker Hub registry): curl -s --unix-socket /var/run/docker.sock http://localhost/v1.43/distribution/docker.io/library/hello-world:latest/json | jq . { "Descriptor": { "mediaType": "application/vnd.oci.image.index.v1+json", "digest": "sha256:1b9844d846ce3a6a6af7013e999a373112c3c0450aca49e155ae444526a2c45e", "size": 3849 }, "Platforms": [ { "architecture": "amd64", "os": "linux" } ] } Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-04 12:20:35 +00:00
repositories = append(repositories, repo)
}
if len(repositories) == 0 {
return nil, lastError
}
api: fix "GET /distribution" endpoint ignoring mirrors If the daemon is configured to use a mirror for the default (Docker Hub) registry, the endpoint did not fall back to querying the upstream if the mirror did not contain the given reference. If the daemon is configured to use a mirror for the default (Docker Hub) registry, did not fall back to querying the upstream if the mirror did not contain the given reference. For pull-through registry-mirrors, this was not a problem, as in that case the registry would forward the request, but for other mirrors, no fallback would happen. This was inconsistent with how "pulling" images handled this situation; when pulling images, both the mirror and upstream would be tried. This problem was caused by the logic used in GetRepository, which had an optimization to only return the first registry it was successfully able to configure (and connect to), with the assumption that the mirror either contained all images used, or to be configured as a pull-through mirror. This patch: - Introduces a GetRepositories method, which returns all candidates (both mirror(s) and upstream). - Updates the endpoint to try all Before this patch: # the daemon is configured to use a mirror for Docker Hub cat /etc/docker/daemon.json { "registry-mirrors": ["http://localhost:5000"]} # start the mirror (empty registry, not configured as pull-through mirror) docker run -d --name registry -p 127.0.0.1:5000:5000 registry:2 # querying the endpoint fails, because the image-manifest is not found in the mirror: curl -s --unix-socket /var/run/docker.sock http://localhost/v1.43/distribution/docker.io/library/hello-world:latest/json { "message": "manifest unknown: manifest unknown" } With this patch applied: # the daemon is configured to use a mirror for Docker Hub cat /etc/docker/daemon.json { "registry-mirrors": ["http://localhost:5000"]} # start the mirror (empty registry, not configured as pull-through mirror) docker run -d --name registry -p 127.0.0.1:5000:5000 registry:2 # querying the endpoint succeeds (manifest is fetched from the upstream Docker Hub registry): curl -s --unix-socket /var/run/docker.sock http://localhost/v1.43/distribution/docker.io/library/hello-world:latest/json | jq . { "Descriptor": { "mediaType": "application/vnd.oci.image.index.v1+json", "digest": "sha256:1b9844d846ce3a6a6af7013e999a373112c3c0450aca49e155ae444526a2c45e", "size": 3849 }, "Platforms": [ { "architecture": "amd64", "os": "linux" } ] } Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-04 12:20:35 +00:00
return repositories, nil
}