Commit graph

1195 commits

Author SHA1 Message Date
Cory Snider
a7c8fdc55b pkg/containerfs: make ResolveScopedPath a free fn
Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-09-23 16:25:22 -04:00
Cory Snider
fd16e003fb pkg/containerfs: Remove NewLocalDriver()
Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-09-23 16:25:22 -04:00
Cory Snider
4d48c00f94 pkg/containerfs: Trim away Driver interface part 1
The Driver interface was required for Linux Containers on Windows, which
is no longer supported.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-09-23 16:25:21 -04:00
Sebastiaan van Stijn
3cb933db9d
Update uses of Image platform fields in OCI image-spec
The OCI image spec is considering to change the Image struct and embedding the
Platform type (see opencontainers/image-spec#959) in the go implementation.
Moby currently uses some struct-literals to propagate the platform fields,
which will break once those changes in the OCI spec are merged.

Ideally (once that change arrives) we would update the code to set the Platform
information as a whole, instead of assigning related fields individually, but
in some cases in the code, image platform information is only partially set
(for example, OSVersion and OSFeatures are not preserved in all cases). This
may be on purpose, so needs to be reviewed.

This patch keeps the current behavior (assigning only specific fields), but
removes the use of struct-literals to make the code compatible with the
upcoming changes in the image-spec module.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-10 12:09:06 +02:00
Sebastiaan van Stijn
cddaa84777
pkg/archive: remove backward compat hack for go < 1.9
The fillGo18FileTypeBits func was added in 1a451d9a7b
to keep the tar headers consistent with headers created with go1.8 and older.

go1.8 and older incorrectly preserved all file-mode bits, including file-type,
instead of stripping those bits and only preserving the _permission_ bits, as
defined in;

- the GNU tar spec: https://www.gnu.org/software/tar/manual/html_node/Standard.html
- and POSIX: http://pubs.opengroup.org/onlinepubs/009695399/basedefs/tar.h.html

We decided at the time to copy the "wrong" behavior to prevent a cache-bust and
to keep the archives identical, however:

- It's not matching the standards, which causes differences between our tar
  implementation and the standard tar implementations, as well as implementations
  in other languages, such as Python (see docker/compose#883).
- BuildKit does not implement this hack.
- We don't _need_ this extra information (as it's already preserved in the
  type header; https://pkg.go.dev/archive/tar#pkg-constants

In short; let's remove this hack.

This reverts commit 1a451d9a7b.
This reverts commit 41eb61d5c2.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-04 21:44:02 +02:00
Sebastiaan van Stijn
7797480eb0
daemon, builder: rename variables that collided with imports
rename some variables that collided with imports or (upcoming)
changes, e.g. `ctx`, which is commonly used for `context.Context`.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-08 13:16:10 +02:00
Sebastiaan van Stijn
e0db8207f3
api: add BuildCache.Parents for API >= v1.42
This field was added to replace the deprecated "Parent" field.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-04 09:30:06 +02:00
Sebastiaan van Stijn
ebf339628a
api: deprecate BuildCache.Parent in API >= v1.42
This field has been deprecated in BuildKit, so this follows the deprecation
in the Engine API.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-04 09:27:34 +02:00
Sebastiaan van Stijn
52c1a2fae8
gofmt GoDoc comments with go1.19
Older versions of Go don't format comments, so committing this as
a separate commit, so that we can already make these changes before
we upgrade to Go 1.19.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-08 19:56:23 +02:00
Erik Sipsma
88916949ef
build: force unlazy of refs by calling extract
Signed-off-by: Erik Sipsma <erik@sipsma.dev>
2022-06-24 11:54:41 +02:00
CrazyMax
795ecf02ce
builder: fix empty reference
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-05-28 21:23:30 +02:00
Eng Zer Jun
7873c27cfb
all: replace strings.Replace with strings.ReplaceAll
strings.ReplaceAll(s, old, new) is a wrapper function for
strings.Replace(s, old, new, -1). But strings.ReplaceAll is more
readable and removes the hardcoded -1.

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2022-05-09 19:45:40 +08:00
Sebastiaan van Stijn
d733481399
daemon: daemon.ContainerKill() accept stop-signal as string
This allows the postContainersKill() handler to pass values as-is. As part of
the rewrite, I also moved the daemon.GetContainer(name) call later in the
function, so that we can fail early if an invalid signal is passed, before
doing the (heavier) fetching of the container.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-05-05 11:27:47 +02:00
Sebastiaan van Stijn
41b96bff55
update uses of container.ContainerCreateCreatedBody to CreateResponse
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-28 22:39:20 +02:00
Sebastiaan van Stijn
449250994f
builder/remotecontext/urlutil: simplify and improve documentation
Simplify some of the logic, and add documentation about the package,
as well as warnings that this package should not be used as a general-
purpose utility.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-12 19:58:09 +02:00
Sebastiaan van Stijn
5f89a6a78e
pkg/urlutil: deprecate, and move to builder/remotecontext/urlutil
pkg/urlutil (despite its poorly chosen name) is not really intended as a generic
utility to handle URLs, and should only be used by the builder to handle (remote)
build contexts.

- IsURL() only does a very rudimentary check for http(s):// prefixes, without any
  other validation, but due to its name may give incorrect expectations.
- IsGitURL() is written specifically with docker build remote git contexts in
  mind, and has handling for backward-compatibility, where strings that are
  not URLs, but start with "github.com/" are accepted.

Because of the above, this patch:

- moves the package inside builder/remotecontext, close to where it's intended
  to be used (ideally this would be part of build/remotecontext itself, but this
  package imports many other dependencies, which would introduce those as extra
  dependencies in the CLI).
- deprecates pkg/urlutil, but adds aliases as there are some external consumers.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-12 19:58:05 +02:00
Sebastiaan van Stijn
1ad9a09af8
Merge pull request #43369 from sestegra/stop
Allow STOPSIGNAL instruction in commit change
2022-03-26 14:40:32 +01:00
Sebastiaan van Stijn
0a3336fd7d
Merge pull request #43366 from corhere/finish-identitymapping-refactor
Finish refactor of UID/GID usage to a new struct
2022-03-25 14:51:05 +01:00
CrazyMax
fda0226a89
builder: add buildinfo for buildkit
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-03-23 16:07:03 +01:00
CrazyMax
b899db6423
builder: enable shm-size and ulimit for buildkit
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-03-23 16:07:03 +01:00
CrazyMax
aadb3bf766
builder: changes needed since buildkit 0.10.0
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-03-23 16:07:03 +01:00
CrazyMax
a2aaf4cc83
vendor buildkit v0.10.0
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-03-22 18:51:27 +01:00
Cory Snider
098a44c07f Finish refactor of UID/GID usage to a new struct
Finish the refactor which was partially completed with commit
34536c498d, passing around IdentityMapping structs instead of pairs of
[]IDMap slices.

Existing code which uses []IDMap relies on zero-valued fields to be
valid, empty mappings. So in order to successfully finish the
refactoring without introducing bugs, their replacement therefore also
needs to have a useful zero value which represents an empty mapping.
Change IdentityMapping to be a pass-by-value type so that there are no
nil pointers to worry about.

The functionality provided by the deprecated NewIDMappingsFromMaps
function is required by unit tests to to construct arbitrary
IdentityMapping values. And the daemon will always need to access the
mappings to pass them to the Linux kernel. Accommodate these use cases
by exporting the struct fields instead. BuildKit currently depends on
the UIDs and GIDs methods so we cannot get rid of them yet.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-03-14 16:28:57 -04:00
Stéphane Este-Gracias
a768652fd6
Support STOPSIGNAL instruction in commit change
Signed-off-by: Stéphane Este-Gracias <sestegra@gmail.com>
2022-03-12 17:10:35 +01:00
Sebastiaan van Stijn
b94a319fdf
remove deprecated builder/dockerignore
This package was deprecated in de56a90929, which
was part of the 20.10 release, so consumers of this package should've been
able to migrate to the new location.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-06 00:49:59 +01:00
Sebastiaan van Stijn
a0230f3d9a
remove unneeded "digest" alias for "go-digest"
I think this was there for historic reasons (may have been goimports expected
this, and we used to have a linter that wanted it), but it's not needed, so
let's remove it (to make my IDE less complaining about unneeded aliases).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-04 14:49:42 +01:00
Sebastiaan van Stijn
32e5fe5099
Merge pull request #43182 from thaJeztah/layer_remove_unused_error
layer: remove unused error return from .Size() and .DiffSize()
2022-02-17 20:51:45 +01:00
Brian Goff
047d58f007
Merge pull request #43187 from thaJeztah/remove_lcow_checks
Remove various leftover LCOW checks
2022-02-17 11:22:19 -08:00
Sebastiaan van Stijn
cfddecc3d2
builder/dockerfile: remove leftover LCOW platform checks
This removes some of the checks that were added in 0cba7740d4,
but should no longer be needed.

- `dockerfile.BuildFromConfig()` is used for `docker (container) commmit` and
  `docker (image) import`. For `docker import`, we're failing early already.
  For `commit`, it won't be possible to have a container that doesn't have the
  right operating-system, so there's no need to validate.
- `dispatchRequest.getImageOrStage()`: simplify the check; all checks resulted
  in an error on Windows, so it came down to "Windows does not support FROM scratch".
- `dispatchState.beginStage()`: `image.OperatingSystem()` already defaults to the
  `runtime.GOOS` if unset, so remove the local default fallback.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-25 15:23:11 +01:00
Sebastiaan van Stijn
e1ea911aba
layer: remove unused error return from .Size() and .DiffSize()
None of the implementations used return an error, so removing the error
return can simplify using these.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-24 18:45:47 +01:00
Sebastiaan van Stijn
bb66ebd621
distribution: xfer.LayerDownloadManager.Download(): remove "os" argument
This argument was added for LCOW support, but it was only used to verify if
the passed platform (OS) matched the host. Given that all uses of this function
(except for one) passed runtime.GOOS, we may as well move the check to that
location.

We should do more cleaning up after this, and perform such validations early,
instead of passing platform around in too many places where it's only used for
similar validations. This is a first step in that direction.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-21 15:07:55 +01:00
Sebastiaan van Stijn
0b0a995d9d
distribution: remove RootFSDownloadManager interface
This interface only had a single implementation (xfer.LayerDownloadManager),
and all places where it was used already imported the xfer package.
Removing the interface, also makes it a closer match to the "upload" part,
as `xfer.LayerUploadManager()` did not use an interface.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-21 13:53:36 +01:00
CrazyMax
b4e056d556 buildkit: normalize build target and local platform
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-10-19 17:11:06 +02:00
Eng Zer Jun
c55a4ac779
refactor: move from io/ioutil to io and os package
The io/ioutil package has been deprecated in Go 1.16. This commit
replaces the existing io/ioutil functions with their new definitions in
io and os packages.

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2021-08-27 14:56:57 +08:00
Sebastiaan van Stijn
686be57d0a
Update to Go 1.17.0, and gofmt with Go 1.17
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-08-24 23:33:27 +02:00
Aaron Lehmann
ba2adeebdb
Merge pull request #42676 from aaronlehmann/patternmatcher-doublestar-bug
fileutils: Fix incorrect handling of "**/foo" pattern
2021-08-16 19:58:55 -07:00
Aaron Lehmann
97ede9df26 Rename Matches to MatchesOrParentMatches
Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
2021-08-12 18:10:04 -07:00
Brian Goff
6a60efc39b
Merge pull request #42684 from thaJeztah/remove_lcow_step7
Remove LCOW (step 7): remove LCOW bits from builder/dockerfile (copy)
2021-08-05 15:16:24 -07:00
Sebastiaan van Stijn
e05fc8ca35
builder/dockerfile: make normalizeDest() platform-specific again
Removing various bits that were added in 7a7357dae1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-27 23:14:40 +02:00
Sebastiaan van Stijn
337500f374
builder/dockerfile: make containsWildcards() platform-specific again
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-27 23:14:38 +02:00
Sebastiaan van Stijn
131fac6eb4
builder/dockerfile: remove fromSlash() and separator() utils
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-27 23:14:36 +02:00
Sebastiaan van Stijn
190b6f64e3
builder/dockerfile: copy: remove LCOW code
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-27 23:14:28 +02:00
Sebastiaan van Stijn
0998c7e363
system: remove LCOWSupported() utility
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-27 13:36:19 +02:00
Sebastiaan van Stijn
28409ca6c7
replace pkg/signal with moby/sys/signal v0.5.0
This code was moved to the moby/sys repository

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-23 09:32:54 +02:00
Sebastiaan van Stijn
46c591b045
pkg/system: deprecate some consts and move them to pkg/idtools
These consts were used in combination with idtools utilities, which
makes it a more logical location for these consts to live.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-06-18 23:16:15 +02:00
Akihiro Suda
9e8cf1016e
Merge pull request #42473 from thaJeztah/unfork_buildkit
revendor BuildKit (master branch)
2021-06-17 10:56:25 +09:00
Sebastiaan van Stijn
3eb1257698
revendor BuildKit (master branch)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-06-16 01:17:48 +02:00
Sebastiaan van Stijn
bb17074119
reformat "nolint" comments
Unlike regular comments, nolint comments should not have a leading space.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-06-10 13:03:42 +02:00
Sebastiaan van Stijn
dd1374f7b2
if-return: redundant if ...; err != nil check (revive)
builder/builder-next/adapters/snapshot/snapshot.go:386:3: if-return: redundant if ...; err != nil check, just return error instead. (revive)
            if err := b.Put(keyIsCommitted, []byte{}); err != nil {
                return err
            }

    plugin/fetch_linux.go:112:2: if-return: redundant if ...; err != nil check, just return error instead. (revive)
        if err := images.Dispatch(ctx, images.Handlers(handlers...), nil, desc); err != nil {
            return err
        }

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-06-10 13:03:40 +02:00
Sebastiaan van Stijn
d13997b4ba
gosec: G601: Implicit memory aliasing in for loop
plugin/v2/plugin.go:141:50: G601: Implicit memory aliasing in for loop. (gosec)
                    updateSettingsEnv(&p.PluginObj.Settings.Env, &s)
                                                                 ^
    libcontainerd/remote/client.go:572:13: G601: Implicit memory aliasing in for loop. (gosec)
                cpDesc = &m
                         ^
    distribution/push_v2.go:400:34: G601: Implicit memory aliasing in for loop. (gosec)
                (metadata.CheckV2MetadataHMAC(&mountCandidate, pd.hmacKey) ||
                                              ^
    builder/dockerfile/builder.go:261:84: G601: Implicit memory aliasing in for loop. (gosec)
            currentCommandIndex = printCommand(b.Stdout, currentCommandIndex, totalCommands, &meta)
                                                                                             ^
    builder/dockerfile/builder.go:278:46: G601: Implicit memory aliasing in for loop. (gosec)
            if err := initializeStage(dispatchRequest, &stage); err != nil {
                                                       ^
    daemon/container.go:283:40: G601: Implicit memory aliasing in for loop. (gosec)
            if err := parser.ValidateMountConfig(&cfg); err != nil {
                                                 ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-06-10 13:03:29 +02:00
Sebastiaan van Stijn
e047d984dc
Remove LCOW code (step 1)
The LCOW implementation in dockerd has been deprecated in favor of re-implementation
in containerd (in progress). Microsoft started removing the LCOW V1 code from the
build dependencies we use in Microsoft/opengcs (soon to be part of Microsoft/hcshhim),
which means that we need to start removing this code.

This first step removes the lcow graphdriver, the LCOW initialization code, and
some LCOW-related utilities.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-06-03 21:16:21 +02:00
Brian Goff
4b981436fe Fixup libnetwork lint errors
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2021-06-01 23:48:32 +00:00
Brian Goff
a0a473125b Fix libnetwork imports
After moving libnetwork to this repo, we need to update all the import
paths for libnetwork to point to docker/docker/libnetwork instead of
docker/libnetwork.
This change implements that.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2021-06-01 21:51:23 +00:00
Tonis Tiigi
85167fc634 builder-next: relax second cache key requirements for schema1
Schema1 images can not have a config based cache key
before the layers are pulled. Avoid validation and reuse
manifest digest as a second key.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-05-12 22:44:07 -07:00
Sebastiaan van Stijn
5a5f8564ba
builder/dockerfile: add "ALL_PROXY" to list of default build args
Relates to a82fff6377/docs/packages.md (proxies)

> (..) the first four of these are the standard built-in build-arg options
> available for `docker build`
> (..) The last, `all_proxy`, is a standard var used for socks proxying. Since
> it is not built into `docker build`, if you want to use it, you will need to
> add the following line to the dockerfile:
>
> ARG all_proxy

Given the we support all other commonly known proxy env-vars by default, it makes
sense to add this one as well.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-04-24 18:25:16 +02:00
Sebastiaan van Stijn
a09c0276a2
builder: produce error when using unsupported Dockerfile option
With the promotion of the experimental Dockerfile syntax to "stable", the Dockerfile
syntax now includes some options that are supported by BuildKit, but not (yet)
supported in the classic builder.

As a result, parsing a Dockerfile may succeed, but any flag that's known to BuildKit,
but not supported by the classic builder is silently ignored;

    $ mkdir buildkit_flags && cd buildkit_flags
    $ touch foo.txt

For example, `RUN --mount`:

    DOCKER_BUILDKIT=0 docker build --no-cache -f- . <<EOF
    FROM busybox
    RUN --mount=type=cache,target=/foo echo hello
    EOF

    Sending build context to Docker daemon  2.095kB
    Step 1/2 : FROM busybox
     ---> 219ee5171f80
    Step 2/2 : RUN --mount=type=cache,target=/foo echo hello
     ---> Running in 022fdb856bc8
    hello
    Removing intermediate container 022fdb856bc8
     ---> e9f0988844d1
    Successfully built e9f0988844d1

Or `COPY --chmod` (same for `ADD --chmod`):

    DOCKER_BUILDKIT=0 docker build --no-cache -f- . <<EOF
    FROM busybox
    COPY --chmod=0777 /foo.txt /foo.txt
    EOF

    Sending build context to Docker daemon  2.095kB
    Step 1/2 : FROM busybox
     ---> 219ee5171f80
    Step 2/2 : COPY --chmod=0777 /foo.txt /foo.txt
     ---> 8b7117932a2a
    Successfully built 8b7117932a2a

Note that unknown flags still produce and error, for example, the below fails because `--hello` is an unknown flag;

    DOCKER_BUILDKIT=0 docker build -<<EOF
    FROM busybox
    RUN --hello echo hello
    EOF

    Sending build context to Docker daemon  2.048kB
    Error response from daemon: dockerfile parse error line 2: Unknown flag: hello

With this patch applied
----------------------------

With this patch applied, flags that are known in the Dockerfile spec, but are not
supported by the classic builder, produce an error, which includes a link to the
documentation how to enable BuildKit:

    DOCKER_BUILDKIT=0 docker build --no-cache -f- . <<EOF
    FROM busybox
    RUN --mount=type=cache,target=/foo echo hello
    EOF

    Sending build context to Docker daemon  2.048kB
    Step 1/2 : FROM busybox
     ---> b97242f89c8a
    Step 2/2 : RUN --mount=type=cache,target=/foo echo hello
    the --mount option requires BuildKit. Refer to https://docs.docker.com/go/buildkit/ to learn how to build images with BuildKit enabled

    DOCKER_BUILDKIT=0 docker build --no-cache -f- . <<EOF
    FROM busybox
    COPY --chmod=0777 /foo.txt /foo.txt
    EOF

    Sending build context to Docker daemon  2.095kB
    Step 1/2 : FROM busybox
     ---> b97242f89c8a
    Step 2/2 : COPY --chmod=0777 /foo.txt /foo.txt
    the --chmod option requires BuildKit. Refer to https://docs.docker.com/go/buildkit/ to learn how to build images with BuildKit enabled

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-03-14 14:11:08 +01:00
Tibor Vass
3c9b4a63b5
Merge pull request #42056 from thaJeztah/bump_buildkit
vendor: github.com/moby/buildkit v0.8.2
2021-02-24 22:26:53 -08:00
Tibor Vass
9bf93e90fa builder: fix incorrect cache match for inline cache with empty layers
See https://github.com/moby/buildkit/pull/1993

Signed-off-by: Tibor Vass <tibor@docker.com>
2021-02-25 01:49:22 +00:00
Tibor Vass
271b43c43f
Merge pull request #42020 from tonistiigi/lease-blobs-fixes
builder: fix blobs releasing via leases after pull
2021-02-23 08:36:38 -08:00
Tonis Tiigi
b53ea19c49 builder: fix pull synchronization regression
Config resolution was synchronized based on a wrong key as ref
variable is initialized only after in the same function. Using
the right key isn't fully correct either as the synchronized method
changes properties of the puller instance and can't be just skipped.
Added better error handling for the same case as well.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-02-16 22:48:37 -08:00
Tonis Tiigi
5c01d06f72 builder: fix blobs releasing via leases after pull
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-02-12 20:27:16 -08:00
Tibor Vass
8d3179546e
Merge pull request #41966 from thaJeztah/CVE-2021-21285_master
[master] prevent an invalid image from crashing docker daemon (CVE-2021-21285)
2021-02-02 09:16:18 -08:00
Brian Goff
c747d9f8ee
pull: Validate layer digest format
Otherwise a malformed or empty digest may cause a panic.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit a7d4af84bd)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-02-02 13:37:24 +01:00
Brian Goff
94c07441c2
buildkit: Apply apparmor profile
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit 611eb6ffb3)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-02-02 13:32:24 +01:00
Tibor Vass
d5209b29b9
Merge pull request #41927 from tiborvass/execabs
Use golang.org/x/sys/execabs
2021-01-26 09:15:54 -08:00
Tibor Vass
7ca0cb7ffa Use golang.org/x/sys/execabs
Signed-off-by: Tibor Vass <tibor@docker.com>
2021-01-25 19:13:12 +00:00
Sebastiaan van Stijn
f266f13965
Merge pull request #41636 from TBBle/37352-test-and-fix
Set 127GB default sandbox size for WCOW, and ensure storage-opts is honoured on all paths under WCOW and LCOW
2021-01-25 14:34:34 +01:00
Tonis Tiigi
7c7e168902 builder: ensure libnetwork state file do not leak
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-01-07 11:16:20 -08:00
Sebastiaan van Stijn
73571e4689
vendor: github.com/moby/buildkit v0.8.0-rc2
full diff: 6861f17f15...v0.8.0-rc2

- dockerfile: rename experimental channel to labs
- dockerfile build: fix not exit when meet error in load config metadata
- copy containerd.UnknownExitStatus to local const to reduce dependency graph in client
- executor: switch to docker seccomp profile
- add retry handlers to push/pull
- SSH-based auth for llb.Git operations
- Allow gateway exec-ing into a failed solve with an exec op
- Fix parsing ssh-based git sources
- Fix sshkeyscan to work with ipv6
- fix assumption that ssh port must be 2 digits
- vendor: github.com/Microsoft/go-winio v0.4.15
- vendor: github.com/tonistiigi/fsutil v0.0.0-20201103201449-0834f99b7b85
- vendor: containerd v1.4.1-0.20201117152358-0edc412565dc
- vendor: golang.org/x/crypto v0.0.0-20201117144127-c1f2f97bffc9

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-11-19 10:31:35 +01:00
Tibor Vass
dbffbe8f9b builder-next: fix platform-specific behavior
Signed-off-by: Tibor Vass <tibor@docker.com>
2020-11-17 02:33:01 +00:00
Tibor Vass
39f9c1b5d1 builder-next: fix --cache-from panic
Signed-off-by: Tibor Vass <tibor@docker.com>
2020-11-14 03:57:36 +00:00
Tibor Vass
8f464ae2ab builder-next: refactor to use flighcontrol and remove resolveOnce
Signed-off-by: Tibor Vass <tibor@docker.com>
2020-11-14 03:57:36 +00:00
Tibor Vass
1b1eb094cc builder-next: simple refactor
Signed-off-by: Tibor Vass <tibor@docker.com>
2020-11-14 03:57:36 +00:00
Tibor Vass
3756668adb builder-next: Refactor using buildkit's resolver pool
Signed-off-by: Tibor Vass <tibor@docker.com>
2020-11-14 03:57:36 +00:00
Tibor Vass
de56a90929 vendor buildkit 6861f17f15364de0fe1fd1e6e8da07598a485123
Signed-off-by: Tibor Vass <tibor@docker.com>
2020-11-14 03:57:36 +00:00
Paul "TBBle" Hampson
56d378a88f Apply a 127GB default WCOW Sandbox size globally
This applies the 127GB default WCOW Sandbox size to not just `RUN` under
`docker build` (as was previously the case) but to `COPY` and `ADD`
under `docker build` and also to `docker run`.

It also removes an inconsistency that the 127GB size was not applied
when `--platform windows` was not passed to `docker build`, but WCOW was
still used as a platform default, e.g. Docker Desktop for Windows in
Windows Containers mode.

Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
2020-11-10 19:51:46 +11:00
Sebastiaan van Stijn
1bb3491601
builder: fix detection of experimental --stream option (deprecated)
Commit 6ca3ec88ae deprecated the experimental
"--stream" option for the legacy builder, adding an error message is a client
attempted to use this feature.

However, the detection used the session-ID (`session=xxx` query parameter),
which happens to be set automatically by the CLI if it detects that the daemon
has session support. Because of this, builds fail when trying to perform them
on a daemon with the `--experimental` flag set.

This patch changes the detection to look for the `remote` query parameter, which
is set to "client-session" when using the `--stream` option with the classic
(non-Buildkit) builder.

Before this change, running `docker build` with an older (19.03 or older) cli
against a daemon with `--experimental` enabled caused an error:

    $ dockerd --experimental &
    $ docker pull docker:18.09
    $ docker run -it --rm -v /var/run/docker.sock:/var/run/docker.sock docker:18.09 sh -c 'echo "FROM scratch" | docker build -'

    Sending build context to Docker daemon  2.048kB
    Error response from daemon: experimental session with v1 builder is no longer supported, use builder version v2 (BuildKit) instead

    docker run -it --rm -v /var/run/docker.sock:/var/run/docker.sock -w /foo docker:18.09 sh -c 'echo "FROM scratch" > Dockerfile && docker build --stream .'
    Error response from daemon: experimental session with v1 builder is no longer supported, use builder version v2 (BuildKit) instead

With this patch, the error only occurs when trying to use the experimental
`--stream` option:

    $ dockerd --experimental &
    $ docker pull docker:18.09
    $ docker run -it --rm -v /var/run/docker.sock:/var/run/docker.sock docker:18.09 sh -c 'echo "FROM scratch" | docker build -'

    Step 1/1 : FROM scratch
     --->
    No image was generated. Is your Dockerfile empty?

    $ docker run -it --rm -v /var/run/docker.sock:/var/run/docker.sock -w /foo docker:18.09 sh -c 'echo "FROM scratch" > Dockerfile && docker build --stream .'

    Error response from daemon: experimental session with v1 builder is no longer supported, use builder version v2 (BuildKit) instead

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-11-06 21:13:28 +01:00
Sebastiaan van Stijn
dc3c382b34
replace pkg/symlink with github.com/moby/sys/symlink
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-11-03 11:17:12 +01:00
Brian Goff
bf6f0d82bc
Merge pull request #41360 from lzhfromustc/GL_5
Builder: Add 1 buffer to two channels to avoid blocking goroutine
2020-09-12 21:43:05 -07:00
lzhfromustc
d1c2f953c5 Test: Add buffer to two channels to avoid blocking goroutine
Signed-off-by: Ziheng Liu <lzhfromustc@gmail.com>
2020-08-27 18:53:43 -04:00
Sebastiaan van Stijn
010adeec55
Builder: print relative path if COPY/ADD source path was not found
Before this change, the error returned to the user would include the physical
path inside the tmp dir on the daemon host. These paths should be considered
an implementation detail, and provide no value to the user. Printing the tmp
path can confuse users, and will be even more confusing if the daemon is running
remotely (or in a VM, such as on Docker Desktop), in which case the path in the
error message does not exist on the local machine;

    echo -e "FROM busybox\nCOPY /some/non-existing/file.txt ." | DOCKER_BUILDKIT=0 docker build -f- .

    Sending build context to Docker daemon   1.57kB
    Step 1/2 : FROM busybox
     ---> 1c35c4412082
    Step 2/2 : COPY /some/non-existing/file.txt .
    COPY failed: stat /var/lib/docker/tmp/docker-builder405687992/some/non-existing/file.txt: no such file or directory

When copying files from an image or a build stage, using `--from`, the error
is similarly confusing:

    echo -e "FROM busybox\nCOPY --from=busybox /some/non-existing/file.txt ." | DOCKER_BUILDKIT=0 docker build -f- .
    Sending build context to Docker daemon  4.671kB
    Step 1/2 : FROM busybox
     ---> 018c9d7b792b
    Step 2/2 : COPY --from=busybox /some/non-existing/file.txt .
    COPY failed: stat /var/lib/docker/overlay2/ef34239c80526c779b7afaeaedbf11c1b201d7f7681d45613102c4541da0e156/merged/some/non-existing/file.txt: no such file or directory

This patch updates the error messages to be more user-friendly. Changes are slightly
different, depending on if the source was a local path, or an image (or build-stage),
using `--from`.

If `--from` is used, only the path is updated, and we print the relative path
instead of the full path;

    echo -e "FROM busybox\nCOPY --from=busybox /some/non-existing/file.txt ." | DOCKER_BUILDKIT=0 docker build -f- .
    Sending build context to Docker daemon  1.583kB
    Step 1/2 : FROM busybox
     ---> 018c9d7b792b
    Step 2/2 : COPY --from=busybox /some/non-existing/file.txt .
    COPY failed: stat some/non-existing/file.txt: file does not exist

In other cases, additional information is added to mention "build context" and
".dockerignore", which could provide the user some hints to find the problem:

    echo -e "FROM busybox\nCOPY /some/non-existing/file.txt ." | DOCKER_BUILDKIT=0 docker build -f- .
    Sending build context to Docker daemon  1.583kB
    Step 1/2 : FROM busybox
     ---> 018c9d7b792b
    Step 2/2 : COPY /some/non-existing/file.txt .
    COPY failed: file not found in build context or excluded by .dockerignore: stat some/non-existing/file.txt: file does not exist

    echo -e "FROM busybox\nADD /some/non-existing/file.txt ." | DOCKER_BUILDKIT=0 docker build -f- .
    Sending build context to Docker daemon  1.583kB
    Step 1/2 : FROM busybox
     ---> 018c9d7b792b
    Step 2/2 : ADD /some/non-existing/file.txt .
    ADD failed: file not found in build context or excluded by .dockerignore: stat some/non-existing/file.txt: file does not exist

This patch only improves the error for the classic builder. Similar changes could
be made for BuildKit, which produces equally, or even more confusing errors;

    echo -e "FROM busybox\nCOPY /some/non-existing/file.txt ." | DOCKER_BUILDKIT=1 docker build -f- .
    [+] Building 1.2s (6/6) FINISHED
     => [internal] load build definition from Dockerfile                 0.0s
     => => transferring dockerfile: 85B                                  0.0s
     => [internal] load .dockerignore                                    0.0s
     => => transferring context: 2B                                      0.0s
     => [internal] load metadata for docker.io/library/busybox:latest    1.2s
     => [internal] load build context                                    0.0s
     => => transferring context: 2B                                      0.0s
     => CACHED [1/2] FROM docker.io/library/busybox@sha256:4f47c01...    0.0s
     => ERROR [2/2] COPY /some/non-existing/file.txt .                   0.0s
    ------
     > [2/2] COPY /some/non-existing/file.txt .:
    ------
    failed to compute cache key: failed to walk /var/lib/docker/tmp/buildkit-mount181923793/some/non-existing:
    lstat /var/lib/docker/tmp/buildkit-mount181923793/some/non-existing: no such file or directory

    echo -e "FROM busybox\nCOPY --from=busybox /some/non-existing/file.txt ." | DOCKER_BUILDKIT=1 docker build -f- .
    [+] Building 2.5s (6/6) FINISHED
     => [internal] load build definition from Dockerfile                        0.0s
     => => transferring dockerfile: 100B                                        0.0s
     => [internal] load .dockerignore                                           0.0s
     => => transferring context: 2B                                             0.0s
     => [internal] load metadata for docker.io/library/busybox:latest           1.2s
     => FROM docker.io/library/busybox:latest                                   1.2s
     => => resolve docker.io/library/busybox:latest                             1.2s
     => CACHED [stage-0 1/2] FROM docker.io/library/busybox@sha256:4f47c01...   0.0s
     => ERROR [stage-0 2/2] COPY --from=busybox /some/non-existing/file.txt .   0.0s
    ------
     > [stage-0 2/2] COPY --from=busybox /some/non-existing/file.txt .:
    ------
    failed to compute cache key: failed to walk /var/lib/docker/overlay2/2a796d91e46fc038648c6010f062bdfd612ee62b0e8fe77bc632688e3fba32d9/merged/some/non-existing:
    lstat /var/lib/docker/overlay2/2a796d91e46fc038648c6010f062bdfd612ee62b0e8fe77bc632688e3fba32d9/merged/some/non-existing: no such file or directory

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-08-17 12:03:18 +02:00
Sebastiaan van Stijn
afd9a6c2b2
builder/remotecontext: use lowercase for error
Errors should not be capitalized. This error was marked as
"being compatible" with the old error, However, looking at
the original error that was in place before d1faf3df27,
I noticed that the error was lowercase before, so it should
be ok to change it back to be lowercase.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-08-11 10:46:18 +02:00
Wang Yumu
2b22a2e792 fix buildkit pull nil panic
Signed-off-by: Wang Yumu <37442693@qq.com>
2020-07-29 07:48:50 +08:00
Tibor Vass
0f41a77c69
Merge pull request #41207 from thaJeztah/remove_whitelist
Remove some outdated terminology where possible
2020-07-28 14:10:27 +02:00
Tonis Tiigi
8774804ca7 vendor: update buildkit to 4d1f260e8
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-07-20 11:34:22 -07:00
Sebastiaan van Stijn
3895dd585f
Replace uses of blacklist/whitelist
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-07-14 10:41:34 +02:00
Tonis Tiigi
9b28939345 vendor: update buildkit to df35e9818
Update to new buildkit and fix upgrade bugs

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-06-15 09:44:41 -07:00
Tonis Tiigi
a430bc5d5f builder-next: fix layer access bug
After switch to leases all ref IDs use generated IDs meaning that
EnsureLayer hack needs to save the chainID for pulled layers.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-05-12 20:24:07 -07:00
Tibor Vass
751d5f879a
Merge pull request #40646 from thaJeztah/builder_use_pkg_sys_windows
pkg/system: deprecate SetNamedSecurityInfo, GetSecurityDescriptorDacl
2020-05-08 08:21:17 -07:00
Sebastiaan van Stijn
07d60bc257
Replace errors.Cause() with errors.Is() / errors.As()
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-04-29 00:28:41 +02:00
Tonis Tiigi
0cdf6ba9c8 vendor: update buildkit to ae7ff174
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-04-14 08:26:07 -07:00
Sebastiaan van Stijn
08e3da51f7
Merge pull request #40790 from cpuguy83/fix_builder_variant_check
Only check variant if set on image.
2020-04-09 20:57:13 +02:00
Brian Goff
fe2aca0e39 Only check variant if set on image.
This fixes an edge case where some images may not have a variant set
just because it didn't used to get set.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2020-04-07 15:09:12 -07:00
Tonis Tiigi
5ad981640f builder: fix concurrent map write
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-04-03 15:33:15 -07:00
Sebastiaan van Stijn
70a4d886cd
builder: fixPermissionsWindows(): use golang.org/x/sys/windows
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-03-09 12:43:02 +01:00
Brian Goff
ed033adb2c Builder: Check remote when local img platform doesn't match
This fixes an issue where if a build requests an image that already
exists in Docker's image store but does not match the specified build
platform, instead of building using the wrong platform go ahead and
check the remote for the correct platform.

Steps to reproduce issue:

```terminal
$ docker pull --platform=amd64 debian:buster
<output supressed>
$ cat Dockerfile
FROM debian:buster
RUN echo hello
$ docker build --platform=armhf -< Dockerfile
<output supressed>
```

Without this fix, the build invokcation will build using the amd64 image
since it is already tagged locally, but this is clearly not what we
want.

With the fix the local image is not used and instead we pull the correct
image.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2020-03-05 22:46:46 -08:00
Akihiro Suda
9a82a9a8ea vendor containerd, BuildKit, protobuf, grpc, and golang.org/x
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-03-03 10:25:20 +09:00
Tonis Tiigi
33baa55353 vendor: update buildkit to d75ed2b68
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-02-24 15:38:51 -08:00
Sebastiaan van Stijn
d96f61cfe1
Merge pull request #40472 from thaJeztah/bump_gotest_tools_v3.0.1
bump gotest.tools v3.0.1 for compatibility with Go 1.14
2020-02-11 01:44:29 +01:00
Sebastiaan van Stijn
9f0b3f5609
bump gotest.tools v3.0.1 for compatibility with Go 1.14
full diff: https://github.com/gotestyourself/gotest.tools/compare/v2.3.0...v3.0.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-02-11 00:06:42 +01:00
wingkwong
544c766d97 Fix typos
Signed-off-by: wingkwong <wingkwong.code@gmail.com>
2020-02-10 20:59:04 +08:00
Brian Goff
db85ef598b
Merge pull request #40247 from thaJeztah/pkg_system_consts
pkg/system: deprecate constants in favor of golang.org/x/sys/windows
2019-12-19 11:35:00 -08:00
Brian Goff
b95fad8e51
Merge pull request #40263 from thaJeztah/normalize_comments
Normalize comment formatting
2019-12-12 12:06:22 -08:00
Sebastiaan van Stijn
b1ffda5cea
builder/remotecontext: allow ssh:// urls for remote context
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-12-05 21:22:49 +01:00
Brian Goff
3ce9258447
Merge pull request #40178 from thaJeztah/remote_context_refactor
builder/remotecontext: small refactor
2019-12-05 09:39:03 -08:00
Sebastiaan van Stijn
2deeb6ad96
builder/remotecontext: normalize comment formatting
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-11-27 15:44:31 +01:00
Sebastiaan van Stijn
4afda3bb7c
Merge pull request #40122 from tonistiigi/buildkit-leases2
Update buildkit to containerd leases
2019-11-26 23:35:24 +01:00
Sebastiaan van Stijn
f22ff19668
builder: remove use of deprecated pkg/system constants
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-11-25 15:40:17 +01:00
Tõnis Tiigi
d1d5f64766
Merge pull request #40021 from thaJeztah/carry_40017
Use newer x/sys/windows SecurityAttributes struct (carry 40017)
2019-11-21 08:57:22 -08:00
Sebastiaan van Stijn
86594739b0
builder/remotecontext: use net/url instead of urlutil
urlutil.IsUrl() was merely checking if the url had a http(s)://
prefix, which is just as well handled through using url.Parse()

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-11-05 18:04:01 -08:00
Sebastiaan van Stijn
5a74a736a8
TestParseRemoteURL: use subtests
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-11-05 14:00:18 -08:00
Tonis Tiigi
21dfcc730b builder-next: clear temp leases on startup
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-11-05 12:50:41 -08:00
Tonis Tiigi
f14c9d4df5 builder-next: track layers and graphdrivers with leases
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-11-05 10:14:33 -08:00
Tonis Tiigi
fe16d95dcd builder-next: update adapters to new buildkit interfaces
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-11-05 10:14:33 -08:00
Tibor Vass
c4cf72bad3
Merge pull request #39964 from thaJeztah/bump_golangci_lint
bump golangci-lint v1.20.0
2019-10-21 10:57:02 -07:00
Sebastiaan van Stijn
4addf7ab1c
Merge pull request #39979 from tiborvass/fix-buildkit-prunegc-filter-config
daemon/config: fix filter type in BuildKit GC config
2019-10-18 22:44:48 +02:00
Sebastiaan van Stijn
301a2fbeca
builder/dockerfile/mockbackend_test.go: suppress SA9005 (staticcheck)
```
builder/dockerfile/mockbackend_test.go:107:21: SA9005: struct doesn't have any exported fields, nor custom marshaling (staticcheck)
	return json.Marshal(rawImage(*i))
	                   ^
```

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-18 00:45:27 +02:00
Justin Cormack
f681590a25
Merge pull request #40081 from thaJeztah/http_constants
Use http constants for HTTP methods and status codes
2019-10-17 11:30:26 -07:00
Grant Millar
df7b8f458a daemon: Use short libnetwork ID in exec-root & update libnetwork
Signed-off-by: Grant Millar <rid@cylo.io>
2019-10-15 11:40:24 +01:00
Sebastiaan van Stijn
984d5bc30c
builder-next: use constants for http status codes
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-13 19:11:24 +02:00
Sebastiaan van Stijn
3b84875216
builder-next: use constants for http methods
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-13 19:11:22 +02:00
Derek McGowan
bc5484d2dd
bump moby/buildkit f7042823e340d38d1746aa675b83d1aca431cee3
full diff: 588c73e1e4...f7042823e3

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

fix daemon for changes in containerd registry configuration

Signed-off-by: Evan Hazlett <ejhazlett@gmail.com>

Update buildernext and daemon for buildkit update

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2019-10-04 15:05:35 -07:00
Tibor Vass
fbdd437d29 daemon/config: fix filter type in BuildKit GC config
For backwards compatibility, the old incorrect object format for
builder.GC.Rule.Filter still works but is deprecated in favor of array of
strings akin to what needs to be passed on the CLI.

Signed-off-by: Tibor Vass <tibor@docker.com>
2019-10-03 00:09:58 +00:00
Tibor Vass
6ca3ec88ae builder: remove legacy build's session handling
This feature was used by docker build --stream and it was kept experimental.

Users of this endpoint should enable BuildKit anyway by setting Version to BuilderBuildKit.

Signed-off-by: Tibor Vass <tibor@docker.com>
2019-10-02 20:29:15 +00:00
Jason A. Donenfeld
c3a0a37446
Use newer x/sys/windows SecurityAttributes struct
This struct now has a properly typed member, so use the properly typed
functions with it.

Also update the vendor directory and hope nothing explodes.

Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-02 21:12:23 +02:00
Tibor Vass
9732185e07
Merge pull request #39144 from kunalkushwaha/builder-entitilement-confg
builder entitlements configuration added.
2019-09-30 13:43:27 -07:00
Kunal Kushwaha
8b7bbf180f builder entitlements configutation added.
buildkit supports entitlements like network-host and security-insecure.
this patch aims to make it configurable through daemon.json file.
by default network-host is enabled & secuirty-insecure is disabled.

Signed-off-by: Kunal Kushwaha <kunal.kushwaha@gmail.com>
2019-09-26 16:59:44 +09:00
Chris Price
c21a3cf432 Add variant to image.Image and legacy builder
This commit adds the image variant to the image.(Image) type and
updates related functionality. Images built from another will
inherit the OS, architecture and variant.

Note that if a base image does not specify an architecture, the
local machine's architecture is used for inherited images. On the
other hand, the variant is set equal to the parent image's variant,
even when the parent image's variant is unset.

The legacy builder is also updated to allow the user to specify
a '--platform' argument on the command line when creating an image
FROM scratch. A complete platform specification, including variant,
is supported. The built image will include the variant, as will any
derived images.

Signed-off-by: Chris Price <chris.price@docker.com>
2019-09-24 22:18:16 +00:00
Sebastiaan van Stijn
e34c081354
builder-next: explicitely ignore unhandled errors (errcheck)
```
builder/builder-next/adapters/containerimage/pull.go:278:23: Error return value of `resolveProgressDone` is not checked (errcheck)
builder/builder-next/adapters/containerimage/pull.go:286:24: Error return value of `resolveProgressDone` is not checked (errcheck)
builder/builder-next/adapters/containerimage/pull.go:303:24: Error return value of `resolveProgressDone` is not checked (errcheck)
builder/builder-next/adapters/containerimage/pull.go:309:24: Error return value of `resolveProgressDone` is not checked (errcheck)
builder/builder-next/adapters/containerimage/pull.go:315:22: Error return value of `resolveProgressDone` is not checked (errcheck)
builder/builder-next/adapters/containerimage/pull.go:512:14: Error return value of `pw.Write` is not checked (errcheck)
builder/builder-next/adapters/containerimage/pull.go:675:13: Error return value of `pw.Write` is not checked (errcheck)
builder/builder-next/adapters/containerimage/pull.go:700:14: Error return value of `pw.Write` is not checked (errcheck)
builder/builder-next/adapters/containerimage/pull.go:786:10: Error return value of `pw.Write` is not checked (errcheck)
builder/builder-next/adapters/containerimage/pull.go:791:11: Error return value of `pw.Write` is not checked (errcheck)

builder/builder-next/exporter/export.go:133:13: Error return value of `layersDone` is not checked (errcheck)
builder/builder-next/exporter/export.go:163:12: Error return value of `configDone` is not checked (errcheck)
builder/builder-next/exporter/export.go:172:11: Error return value of `tagDone` is not checked (errcheck)
builder/builder-next/exporter/writer.go:207:10: Error return value of `pw.Write` is not checked (errcheck)
builder/builder-next/exporter/writer.go:212:11: Error return value of `pw.Write` is not checked (errcheck)
builder/builder-next/worker/worker.go:397:6: Error return value of `done` is not checked (errcheck)
builder/builder-next/worker/worker.go:446:10: Error return value of `pw.Write` is not checked (errcheck)
builder/builder-next/worker/worker.go:451:11: Error return value of `pw.Write` is not checked (errcheck)
```

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-18 12:57:47 +02:00
Sebastiaan van Stijn
3926b5f09d
structcheck: nolint for false positives
```
builder/fscache/fscache.go:505:2: `src` is unused (structcheck)
builder/fscache/fscache.go:507:2: `cached` is unused (structcheck)

daemon/graphdriver/vfs/quota_linux.go:9:2: `quotaCtl` is unused (structcheck)
daemon/graphdriver/vfs/quota_linux.go:10:2: `quotaOpt` is unused (structcheck)
```

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-18 12:57:44 +02:00
Sebastiaan van Stijn
9419024554
gosec: add ignore comments for reported issues that can be ignored
```
builder/remotecontext/remote.go:48:        G107: Potential HTTP request made with variable url (gosec)
builder/remotecontext/git/gitutils.go:145: G107: Potential HTTP request made with variable url (gosec)
builder/remotecontext/git/gitutils.go:147: G107: Potential HTTP request made with variable url (gosec)
pkg/fileutils/fileutils_test.go:185:       G303: File creation in shared tmp directory without using ioutil.Tempfile (gosec)
pkg/tarsum/tarsum_test.go:7:               G501: Blacklisted import `crypto/md5`: weak cryptographic primitive (gosec)
pkg/tarsum/tarsum_test.go:9:               G505: Blacklisted import `crypto/sha1`: weak cryptographic primitive (gosec)
```

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-18 12:57:43 +02:00
Kir Kolyshkin
ac827a7fff
builder-next/patchImageConfig: nitpicks
1. Do not introduce another dt variable.

2. Wrap the error similar to all the others.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2019-09-18 12:57:40 +02:00
Sebastiaan van Stijn
a46ad5e704
Builder: remove unused field
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-18 12:57:32 +02:00
Kir Kolyshkin
d8ec1ee57d
builder/copy-unix: fix filepath.Walk args
Mark unused WalkFunc arguments as such, to get rid of the
following linterwarning:

> builder/dockerfile/copy_unix.go:28:71: SA4009: argument err is overwritten before first use (staticcheck)
> 	return filepath.Walk(source, func(fullpath string, info os.FileInfo, err error) error {

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2019-09-18 12:57:18 +02:00
Sebastiaan van Stijn
07ff4f1de8
goimports: fix imports
Format the source according to latest goimports.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-18 12:56:54 +02:00
Sebastiaan van Stijn
a3f9cb5b63
TestDispatch: refactor to use subtests again, and fix linting (structcheck)
Instead of using a `initDispatchTestCases()` function, declare the test-table
inside `TestDispatch` itself, and run the tests as subtests.

```
[2019-08-27T15:14:51.072Z] builder/dockerfile/evaluator_test.go:18:2: `name` is unused (structcheck)
[2019-08-27T15:14:51.072Z] 	name, expectedError string
```

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-11 11:13:03 +02:00
Sebastiaan van Stijn
7e8f7efdd1
fix build after merge of outdated PR
this broke after e554ab5589,
was merged but the PR was outdated.

```
[2019-08-28T00:27:12.031Z] builder/dockerfile/copy.go:566:29: too many arguments in call to system.MkdirAll
[2019-08-28T00:27:12.031Z] 	have (string, number, string)
[2019-08-28T00:27:12.031Z] 	want (string, os.FileMode)
```

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-08-28 03:14:28 +02:00
Sebastiaan van Stijn
08bc39c8f9
Merge pull request #39781 from tonistiigi/update-buildkit
vendor: update buildkit to 588c73e1e4
2019-08-22 20:12:53 +02:00
Tonis Tiigi
27f1f2b5be builder-next: close progress on layer export error
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-08-21 17:06:09 -07:00
Tonis Tiigi
d495eeb365 builder-next: update mount signature
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-08-21 16:40:56 -07:00
Yong Tang
0c46a20f94
Merge pull request #39761 from tonistiigi/ctx-upload-cancel
builder-next: close build context upload on cancel
2019-08-17 19:05:26 -07:00
Yong Tang
4760db0402
Merge pull request #39752 from vikramhh/39695-fix
Fixups for 39695 - edit comments, redundant asserts
2019-08-17 12:53:42 -07:00
Tonis Tiigi
2c2cd9b86a builder-next: close build context upload on cancel
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-08-16 13:39:50 -07:00
Tonis Tiigi
9ea2cf320a builder-next: avoid double unmounting mountable
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-08-15 17:40:33 -07:00
Vikram bir Singh
b146b2382c Fixups for 39695 - edit comments, redundant lines
1. Modify comments added in 5858a99267
Windows Volume GUID path format is: \\?\Volume{<GUID Value>}\<path>
Rewrote the example given in comments to conform to the format..

2. Remove two redundant asserts[assert.NilError]. They are redundant
because the last statement will not change the value of err.

Signed-off-by: Vikram bir Singh <vikrambir.singh@docker.com>
2019-08-15 21:13:46 +00:00
Sebastiaan van Stijn
5858a99267
Builder: fix "COPY --from" to non-existing directory on Windows
This fixes a regression introduced in 6d87f19142,
causing `COPY --from` to fail if the target directory does not exist:

```
FROM mcr.microsoft.com/windows/servercore:ltsc2019 as s1
RUN echo "Hello World" > /hello

FROM mcr.microsoft.com/windows/servercore:ltsc2019
COPY --from=s1 /hello /hello/another/world
```

Would produce an error:

```
Step 4/4 : COPY --from=s1 /hello /hello/another/world
failed to copy files: mkdir \\?: The filename, directory name, or volume label syntax is incorrect.
```

The cause for this was that Go's `os.MkdirAll()` does not support/detect volume GUID paths
(`\\?\Volume{dae8d3ac-b9a1-11e9-88eb-e8554b2ba1db}\hello\another}`), and as a result
attempted to create the volume as a directory (`\\?`), causing it to fail.

This patch replaces `os.MkdirAll()` with our own `system.MkdirAll()` function, which
is capable of detecting GUID volumes.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-08-08 16:45:10 +02:00
Sebastiaan van Stijn
e4611b3e07
Merge pull request #39683 from tonistiigi/builder-metadata-timestamps
builder-next: ensure timestamps set for metadata commands
2019-08-08 01:13:34 +02:00
Tonis Tiigi
c60e53a274 vendor: update buildkit to f5a55a95
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-08-06 20:44:32 -07:00
Tonis Tiigi
1a2bd3cf7d builder-next: ensure timestamps set for metadata commands
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-08-06 17:42:46 -07:00
Tonis Tiigi
af2e82d054 atomic: patch 64bit alignment on 32bit systems
causes panic on armv7

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-07-16 16:57:58 -07:00