Commit 6ca3ec88ae deprecated the experimental
"--stream" option for the legacy builder, adding an error message is a client
attempted to use this feature.
However, the detection used the session-ID (`session=xxx` query parameter),
which happens to be set automatically by the CLI if it detects that the daemon
has session support. Because of this, builds fail when trying to perform them
on a daemon with the `--experimental` flag set.
This patch changes the detection to look for the `remote` query parameter, which
is set to "client-session" when using the `--stream` option with the classic
(non-Buildkit) builder.
Before this change, running `docker build` with an older (19.03 or older) cli
against a daemon with `--experimental` enabled caused an error:
$ dockerd --experimental &
$ docker pull docker:18.09
$ docker run -it --rm -v /var/run/docker.sock:/var/run/docker.sock docker:18.09 sh -c 'echo "FROM scratch" | docker build -'
Sending build context to Docker daemon 2.048kB
Error response from daemon: experimental session with v1 builder is no longer supported, use builder version v2 (BuildKit) instead
docker run -it --rm -v /var/run/docker.sock:/var/run/docker.sock -w /foo docker:18.09 sh -c 'echo "FROM scratch" > Dockerfile && docker build --stream .'
Error response from daemon: experimental session with v1 builder is no longer supported, use builder version v2 (BuildKit) instead
With this patch, the error only occurs when trying to use the experimental
`--stream` option:
$ dockerd --experimental &
$ docker pull docker:18.09
$ docker run -it --rm -v /var/run/docker.sock:/var/run/docker.sock docker:18.09 sh -c 'echo "FROM scratch" | docker build -'
Step 1/1 : FROM scratch
--->
No image was generated. Is your Dockerfile empty?
$ docker run -it --rm -v /var/run/docker.sock:/var/run/docker.sock -w /foo docker:18.09 sh -c 'echo "FROM scratch" > Dockerfile && docker build --stream .'
Error response from daemon: experimental session with v1 builder is no longer supported, use builder version v2 (BuildKit) instead
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Before this change, the error returned to the user would include the physical
path inside the tmp dir on the daemon host. These paths should be considered
an implementation detail, and provide no value to the user. Printing the tmp
path can confuse users, and will be even more confusing if the daemon is running
remotely (or in a VM, such as on Docker Desktop), in which case the path in the
error message does not exist on the local machine;
echo -e "FROM busybox\nCOPY /some/non-existing/file.txt ." | DOCKER_BUILDKIT=0 docker build -f- .
Sending build context to Docker daemon 1.57kB
Step 1/2 : FROM busybox
---> 1c35c4412082
Step 2/2 : COPY /some/non-existing/file.txt .
COPY failed: stat /var/lib/docker/tmp/docker-builder405687992/some/non-existing/file.txt: no such file or directory
When copying files from an image or a build stage, using `--from`, the error
is similarly confusing:
echo -e "FROM busybox\nCOPY --from=busybox /some/non-existing/file.txt ." | DOCKER_BUILDKIT=0 docker build -f- .
Sending build context to Docker daemon 4.671kB
Step 1/2 : FROM busybox
---> 018c9d7b792b
Step 2/2 : COPY --from=busybox /some/non-existing/file.txt .
COPY failed: stat /var/lib/docker/overlay2/ef34239c80526c779b7afaeaedbf11c1b201d7f7681d45613102c4541da0e156/merged/some/non-existing/file.txt: no such file or directory
This patch updates the error messages to be more user-friendly. Changes are slightly
different, depending on if the source was a local path, or an image (or build-stage),
using `--from`.
If `--from` is used, only the path is updated, and we print the relative path
instead of the full path;
echo -e "FROM busybox\nCOPY --from=busybox /some/non-existing/file.txt ." | DOCKER_BUILDKIT=0 docker build -f- .
Sending build context to Docker daemon 1.583kB
Step 1/2 : FROM busybox
---> 018c9d7b792b
Step 2/2 : COPY --from=busybox /some/non-existing/file.txt .
COPY failed: stat some/non-existing/file.txt: file does not exist
In other cases, additional information is added to mention "build context" and
".dockerignore", which could provide the user some hints to find the problem:
echo -e "FROM busybox\nCOPY /some/non-existing/file.txt ." | DOCKER_BUILDKIT=0 docker build -f- .
Sending build context to Docker daemon 1.583kB
Step 1/2 : FROM busybox
---> 018c9d7b792b
Step 2/2 : COPY /some/non-existing/file.txt .
COPY failed: file not found in build context or excluded by .dockerignore: stat some/non-existing/file.txt: file does not exist
echo -e "FROM busybox\nADD /some/non-existing/file.txt ." | DOCKER_BUILDKIT=0 docker build -f- .
Sending build context to Docker daemon 1.583kB
Step 1/2 : FROM busybox
---> 018c9d7b792b
Step 2/2 : ADD /some/non-existing/file.txt .
ADD failed: file not found in build context or excluded by .dockerignore: stat some/non-existing/file.txt: file does not exist
This patch only improves the error for the classic builder. Similar changes could
be made for BuildKit, which produces equally, or even more confusing errors;
echo -e "FROM busybox\nCOPY /some/non-existing/file.txt ." | DOCKER_BUILDKIT=1 docker build -f- .
[+] Building 1.2s (6/6) FINISHED
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 85B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load metadata for docker.io/library/busybox:latest 1.2s
=> [internal] load build context 0.0s
=> => transferring context: 2B 0.0s
=> CACHED [1/2] FROM docker.io/library/busybox@sha256:4f47c01... 0.0s
=> ERROR [2/2] COPY /some/non-existing/file.txt . 0.0s
------
> [2/2] COPY /some/non-existing/file.txt .:
------
failed to compute cache key: failed to walk /var/lib/docker/tmp/buildkit-mount181923793/some/non-existing:
lstat /var/lib/docker/tmp/buildkit-mount181923793/some/non-existing: no such file or directory
echo -e "FROM busybox\nCOPY --from=busybox /some/non-existing/file.txt ." | DOCKER_BUILDKIT=1 docker build -f- .
[+] Building 2.5s (6/6) FINISHED
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 100B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load metadata for docker.io/library/busybox:latest 1.2s
=> FROM docker.io/library/busybox:latest 1.2s
=> => resolve docker.io/library/busybox:latest 1.2s
=> CACHED [stage-0 1/2] FROM docker.io/library/busybox@sha256:4f47c01... 0.0s
=> ERROR [stage-0 2/2] COPY --from=busybox /some/non-existing/file.txt . 0.0s
------
> [stage-0 2/2] COPY --from=busybox /some/non-existing/file.txt .:
------
failed to compute cache key: failed to walk /var/lib/docker/overlay2/2a796d91e46fc038648c6010f062bdfd612ee62b0e8fe77bc632688e3fba32d9/merged/some/non-existing:
lstat /var/lib/docker/overlay2/2a796d91e46fc038648c6010f062bdfd612ee62b0e8fe77bc632688e3fba32d9/merged/some/non-existing: no such file or directory
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Errors should not be capitalized. This error was marked as
"being compatible" with the old error, However, looking at
the original error that was in place before d1faf3df27,
I noticed that the error was lowercase before, so it should
be ok to change it back to be lowercase.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
After switch to leases all ref IDs use generated IDs meaning that
EnsureLayer hack needs to save the chainID for pulled layers.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
This fixes an edge case where some images may not have a variant set
just because it didn't used to get set.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
This fixes an issue where if a build requests an image that already
exists in Docker's image store but does not match the specified build
platform, instead of building using the wrong platform go ahead and
check the remote for the correct platform.
Steps to reproduce issue:
```terminal
$ docker pull --platform=amd64 debian:buster
<output supressed>
$ cat Dockerfile
FROM debian:buster
RUN echo hello
$ docker build --platform=armhf -< Dockerfile
<output supressed>
```
Without this fix, the build invokcation will build using the amd64 image
since it is already tagged locally, but this is clearly not what we
want.
With the fix the local image is not used and instead we pull the correct
image.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
urlutil.IsUrl() was merely checking if the url had a http(s)://
prefix, which is just as well handled through using url.Parse()
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
