None of the implementations used return an error, so removing the error
return can simplify using these.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This argument was added for LCOW support, but it was only used to verify if
the passed platform (OS) matched the host. Given that all uses of this function
(except for one) passed runtime.GOOS, we may as well move the check to that
location.
We should do more cleaning up after this, and perform such validations early,
instead of passing platform around in too many places where it's only used for
similar validations. This is a first step in that direction.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This interface only had a single implementation (xfer.LayerDownloadManager),
and all places where it was used already imported the xfer package.
Removing the interface, also makes it a closer match to the "upload" part,
as `xfer.LayerUploadManager()` did not use an interface.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The io/ioutil package has been deprecated in Go 1.16. This commit
replaces the existing io/ioutil functions with their new definitions in
io and os packages.
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
Otherwise a malformed or empty digest may cause a panic.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit a7d4af84bd)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Some registries seem to be non-conformant and return a not found error
when pulling by digest (which docker now does all the time).
To work around this, fallback when all of the following are true:
1. Image reference is a tag
2. Tag->digest resolution succeeds
3. Fetch by resolved digest fails with a "not found" error.
This is intentionally not caching the manifests to reduce complexity for
this edge case.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
This allows us to cache manifests and avoid extra round trips to the
registry for content we already know about.
dockerd currently does not support containerd on Windows, so this does
not store manifests on Windows, yet.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
The tag service does a `HEAD` request to get the manifest digest, where
we can then do a `GET` against the digest.
The `GET` by tag is not cacheable, but the `GET` against the digest is.
This allows proxies to work way better.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
For CI, a temporary `DOCKER_ALLOW_SCHEMA1_PUSH_DONOTUSE` environment
variable was added while we work out a solution for testing schema 1
pulls (which currently require pushing them to a local registry first
for testing).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Currently we show this deprecation notice for any error returned by a
registry.
Registries can return an error for any number of reasons.
Instead let's show the deprecation notice only if the fallback was
successful.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
```
distribution/push_v2_test.go:552:29: SA4021: x = append(y) is equivalent to x = y (staticcheck)
return nil, errcode.Errors(append([]error{errcode.ErrorCodeUnauthorized.WithMessage("unauthorized")}))
^
```
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Previously we were re-using schema2.DeserializedManifest to handle oci
manifests. The issue lies in the fact that distribution started
validating the media type string during json deserialization. This
change broke our usage of that type.
Instead distribution now provides direct support for oci schemas, so use
that instead of our custom handlers.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
This commit adds the image variant to the image.(Image) type and
updates related functionality. Images built from another will
inherit the OS, architecture and variant.
Note that if a base image does not specify an architecture, the
local machine's architecture is used for inherited images. On the
other hand, the variant is set equal to the parent image's variant,
even when the parent image's variant is unset.
The legacy builder is also updated to allow the user to specify
a '--platform' argument on the command line when creating an image
FROM scratch. A complete platform specification, including variant,
is supported. The built image will include the variant, as will any
derived images.
Signed-off-by: Chris Price <chris.price@docker.com>
1.Change base.Dial to base.DailContext.
2.Remove proxyDialer that was previously used to configure a
net.Dialer to route proxy.Dialer which will route the connections
through the proxy using the connections through a SOCKS proxy.
SOCKS proxies are now supported by configuring only http.Transport.Proxy,
and no longer require changing http.Transport.Dial.
Signed-off-by: HuanHuan Ye <logindaveye@gmail.com>
Moby works perfectly when you are in a situation when one has a good and stable
internet connection. Operating in area's where internet connectivity is likely
to be lost in undetermined intervals, like a satellite connection or 4G/LTE in
rural area's, can become a problem when pulling a new image. When connection is
lost while image layers are being pulled, Moby will try to reconnect up to 5 times.
If this fails, the incompletely downloaded layers are lost will need to be completely
downloaded again during the next pull request. This means that we are using more
data than we might have to.
Pulling a layer multiple times from the start can become costly over a satellite
or 4G/LTE connection. As these techniques (especially 4G) quite common in IoT and
Moby is used to run Azure IoT Edge devices, I would like to add a settable maximum
download attempts. The maximum download attempts is currently set at 5
(distribution/xfer/download.go). I would like to change this constant to a variable
that the user can set. The default will still be 5, so nothing will change from
the current version unless specified when starting the daemon with the added flag
or in the config file.
I added a default value of 5 for DefaultMaxDownloadAttempts and a settable
max-download-attempts in the daemon config file. It is also added to the config
of dockerd so it can be set with a flag when starting the daemon. This value gets
stored in the imageService of the daemon when it is initiated and can be passed
to the NewLayerDownloadManager as a parameter. It will be stored in the
LayerDownloadManager when initiated. This enables us to set the max amount of
retries in makeDownoadFunc equal to the max download attempts.
I also added some tests that are based on maxConcurrentDownloads/maxConcurrentUploads.
You can pull this version and test in a development container. Either create a config
`file /etc/docker/daemon.json` with `{"max-download-attempts"=3}``, or use
`dockerd --max-download-attempts=3 -D &` to start up the dockerd. Start downloading
a container and disconnect from the internet whilst downloading. The result would
be that it stops pulling after three attempts.
Signed-off-by: Lukas Heeren <lukas-heeren@hotmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
staticcheck go linter warns:
> distribution/xfer/transfer_test.go:37:2: SA2002: the goroutine calls T.Fatalf, which must be called in the same goroutine as the test (staticcheck)
What it doesn't say is why. The reason is, t.Fatalf() calls t.FailNow(),
which is expected to stop test execution right now. It does so by
calling runtime.Goexit(), which, unless called from a main goroutine,
does not stop test execution.
Anyway, long story short, if we don't care much about stopping the test
case immediately, we can just replace t.Fatalf() with t.Errorf() which
still marks the test case as failed, but won't stop it immediately.
This patch was tested to check that the test fails if any of the
goroutines call t.Errorf():
1. Failure in DoFunc ("transfer function not started ...") was tested by
decreading the NewTransferManager() argument:
- tm := NewTransferManager(5)
+ tm := NewTransferManager(2)
2. Failure "got unexpected progress value" was tested by injecting a random:
- if present && p.Current <= val {
+ if present && p.Current <= val || rand.Intn(100) > 80 {
3. Failure in DoFunc ("too many jobs running") was tested by increasing
the NewTransferManager() argument:
- tm := NewTransferManager(concurrencyLimit)
+ tm := NewTransferManager(concurrencyLimit + 1)
While at it:
* fix/amend some error messages
* use _ for unused arguments of DoFunc
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Fixes linter warnings like this one:
> distribution/pull_v2.go:229:39: SA1019: os.SEEK_SET is deprecated: Use io.SeekStart, io.SeekCurrent, and io.SeekEnd. (staticcheck)
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
opts/env_test: suppress a linter warning
this one:
> opts/env_test.go:95:4: U1000: field `err` is unused (unused)
> err error
> ^
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Format the source according to latest goimports.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The warning on pull was incorrectly asking to contact registry admins.
It is kept on push however.
Pulling manifest lists with v2 schema1 manifests will not be supported thus
there is a warning for those, but wording changed to suggest repository author
to upgrade.
Finally, a milder warning on regular pull is kept ONLY for DockerHub users
in order to incite moving away from schema1.
Signed-off-by: Tibor Vass <tibor@docker.com>
This reverts commit 98fc09128b in order to
keep registry v2 schema1 handling and libtrust-key-based engine ID.
Because registry v2 schema1 was not officially deprecated and
registries are still relying on it, this patch puts its logic back.
However, registry v1 relics are not added back since v1 logic has been
removed a while ago.
This also fixes an engine upgrade issue in a swarm cluster. It was relying
on the Engine ID to be the same upon upgrade, but the mentioned commit
modified the logic to use UUID and from a different file.
Since the libtrust key is always needed to support v2 schema1 pushes,
that the old engine ID is based on the libtrust key, and that the engine ID
needs to be conserved across upgrades, adding a UUID-based engine ID logic
seems to add more complexity than it solves the problems.
Hence reverting the engine ID changes as well.
Signed-off-by: Tibor Vass <tibor@docker.com>
It has been declared deprecated by the author, and has a knack for
false-positives (as well as giving bad advice when it comes to APIs --
which is quite clear when looking at "nolint: interfacer" comments).
Signed-off-by: Aleksa Sarai <asarai@suse.de>
As people are using the UUID in `docker info` that was based on the v1 manifest signing key, replace
with a UUID instead.
Remove deprecated `--disable-legacy-registry` option that was scheduled to be removed in 18.03.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
