Commit graph

1085 commits

Author SHA1 Message Date
CrazyMax
795ecf02ce
builder: fix empty reference
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-05-28 21:23:30 +02:00
Eng Zer Jun
7873c27cfb
all: replace strings.Replace with strings.ReplaceAll
strings.ReplaceAll(s, old, new) is a wrapper function for
strings.Replace(s, old, new, -1). But strings.ReplaceAll is more
readable and removes the hardcoded -1.

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2022-05-09 19:45:40 +08:00
Sebastiaan van Stijn
d733481399
daemon: daemon.ContainerKill() accept stop-signal as string
This allows the postContainersKill() handler to pass values as-is. As part of
the rewrite, I also moved the daemon.GetContainer(name) call later in the
function, so that we can fail early if an invalid signal is passed, before
doing the (heavier) fetching of the container.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-05-05 11:27:47 +02:00
Sebastiaan van Stijn
41b96bff55
update uses of container.ContainerCreateCreatedBody to CreateResponse
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-28 22:39:20 +02:00
Sebastiaan van Stijn
449250994f
builder/remotecontext/urlutil: simplify and improve documentation
Simplify some of the logic, and add documentation about the package,
as well as warnings that this package should not be used as a general-
purpose utility.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-12 19:58:09 +02:00
Sebastiaan van Stijn
5f89a6a78e
pkg/urlutil: deprecate, and move to builder/remotecontext/urlutil
pkg/urlutil (despite its poorly chosen name) is not really intended as a generic
utility to handle URLs, and should only be used by the builder to handle (remote)
build contexts.

- IsURL() only does a very rudimentary check for http(s):// prefixes, without any
  other validation, but due to its name may give incorrect expectations.
- IsGitURL() is written specifically with docker build remote git contexts in
  mind, and has handling for backward-compatibility, where strings that are
  not URLs, but start with "github.com/" are accepted.

Because of the above, this patch:

- moves the package inside builder/remotecontext, close to where it's intended
  to be used (ideally this would be part of build/remotecontext itself, but this
  package imports many other dependencies, which would introduce those as extra
  dependencies in the CLI).
- deprecates pkg/urlutil, but adds aliases as there are some external consumers.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-12 19:58:05 +02:00
Sebastiaan van Stijn
1ad9a09af8
Merge pull request #43369 from sestegra/stop
Allow STOPSIGNAL instruction in commit change
2022-03-26 14:40:32 +01:00
Sebastiaan van Stijn
0a3336fd7d
Merge pull request #43366 from corhere/finish-identitymapping-refactor
Finish refactor of UID/GID usage to a new struct
2022-03-25 14:51:05 +01:00
CrazyMax
fda0226a89
builder: add buildinfo for buildkit
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-03-23 16:07:03 +01:00
CrazyMax
b899db6423
builder: enable shm-size and ulimit for buildkit
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-03-23 16:07:03 +01:00
CrazyMax
aadb3bf766
builder: changes needed since buildkit 0.10.0
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-03-23 16:07:03 +01:00
CrazyMax
a2aaf4cc83
vendor buildkit v0.10.0
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-03-22 18:51:27 +01:00
Cory Snider
098a44c07f Finish refactor of UID/GID usage to a new struct
Finish the refactor which was partially completed with commit
34536c498d, passing around IdentityMapping structs instead of pairs of
[]IDMap slices.

Existing code which uses []IDMap relies on zero-valued fields to be
valid, empty mappings. So in order to successfully finish the
refactoring without introducing bugs, their replacement therefore also
needs to have a useful zero value which represents an empty mapping.
Change IdentityMapping to be a pass-by-value type so that there are no
nil pointers to worry about.

The functionality provided by the deprecated NewIDMappingsFromMaps
function is required by unit tests to to construct arbitrary
IdentityMapping values. And the daemon will always need to access the
mappings to pass them to the Linux kernel. Accommodate these use cases
by exporting the struct fields instead. BuildKit currently depends on
the UIDs and GIDs methods so we cannot get rid of them yet.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-03-14 16:28:57 -04:00
Stéphane Este-Gracias
a768652fd6
Support STOPSIGNAL instruction in commit change
Signed-off-by: Stéphane Este-Gracias <sestegra@gmail.com>
2022-03-12 17:10:35 +01:00
Sebastiaan van Stijn
b94a319fdf
remove deprecated builder/dockerignore
This package was deprecated in de56a90929, which
was part of the 20.10 release, so consumers of this package should've been
able to migrate to the new location.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-06 00:49:59 +01:00
Sebastiaan van Stijn
a0230f3d9a
remove unneeded "digest" alias for "go-digest"
I think this was there for historic reasons (may have been goimports expected
this, and we used to have a linter that wanted it), but it's not needed, so
let's remove it (to make my IDE less complaining about unneeded aliases).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-04 14:49:42 +01:00
Sebastiaan van Stijn
32e5fe5099
Merge pull request #43182 from thaJeztah/layer_remove_unused_error
layer: remove unused error return from .Size() and .DiffSize()
2022-02-17 20:51:45 +01:00
Brian Goff
047d58f007
Merge pull request #43187 from thaJeztah/remove_lcow_checks
Remove various leftover LCOW checks
2022-02-17 11:22:19 -08:00
Sebastiaan van Stijn
cfddecc3d2
builder/dockerfile: remove leftover LCOW platform checks
This removes some of the checks that were added in 0cba7740d4,
but should no longer be needed.

- `dockerfile.BuildFromConfig()` is used for `docker (container) commmit` and
  `docker (image) import`. For `docker import`, we're failing early already.
  For `commit`, it won't be possible to have a container that doesn't have the
  right operating-system, so there's no need to validate.
- `dispatchRequest.getImageOrStage()`: simplify the check; all checks resulted
  in an error on Windows, so it came down to "Windows does not support FROM scratch".
- `dispatchState.beginStage()`: `image.OperatingSystem()` already defaults to the
  `runtime.GOOS` if unset, so remove the local default fallback.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-25 15:23:11 +01:00
Sebastiaan van Stijn
e1ea911aba
layer: remove unused error return from .Size() and .DiffSize()
None of the implementations used return an error, so removing the error
return can simplify using these.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-24 18:45:47 +01:00
Sebastiaan van Stijn
bb66ebd621
distribution: xfer.LayerDownloadManager.Download(): remove "os" argument
This argument was added for LCOW support, but it was only used to verify if
the passed platform (OS) matched the host. Given that all uses of this function
(except for one) passed runtime.GOOS, we may as well move the check to that
location.

We should do more cleaning up after this, and perform such validations early,
instead of passing platform around in too many places where it's only used for
similar validations. This is a first step in that direction.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-21 15:07:55 +01:00
Sebastiaan van Stijn
0b0a995d9d
distribution: remove RootFSDownloadManager interface
This interface only had a single implementation (xfer.LayerDownloadManager),
and all places where it was used already imported the xfer package.
Removing the interface, also makes it a closer match to the "upload" part,
as `xfer.LayerUploadManager()` did not use an interface.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-21 13:53:36 +01:00
CrazyMax
b4e056d556 buildkit: normalize build target and local platform
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-10-19 17:11:06 +02:00
Eng Zer Jun
c55a4ac779
refactor: move from io/ioutil to io and os package
The io/ioutil package has been deprecated in Go 1.16. This commit
replaces the existing io/ioutil functions with their new definitions in
io and os packages.

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2021-08-27 14:56:57 +08:00
Sebastiaan van Stijn
686be57d0a
Update to Go 1.17.0, and gofmt with Go 1.17
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-08-24 23:33:27 +02:00
Aaron Lehmann
ba2adeebdb
Merge pull request #42676 from aaronlehmann/patternmatcher-doublestar-bug
fileutils: Fix incorrect handling of "**/foo" pattern
2021-08-16 19:58:55 -07:00
Aaron Lehmann
97ede9df26 Rename Matches to MatchesOrParentMatches
Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
2021-08-12 18:10:04 -07:00
Brian Goff
6a60efc39b
Merge pull request #42684 from thaJeztah/remove_lcow_step7
Remove LCOW (step 7): remove LCOW bits from builder/dockerfile (copy)
2021-08-05 15:16:24 -07:00
Sebastiaan van Stijn
e05fc8ca35
builder/dockerfile: make normalizeDest() platform-specific again
Removing various bits that were added in 7a7357dae1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-27 23:14:40 +02:00
Sebastiaan van Stijn
337500f374
builder/dockerfile: make containsWildcards() platform-specific again
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-27 23:14:38 +02:00
Sebastiaan van Stijn
131fac6eb4
builder/dockerfile: remove fromSlash() and separator() utils
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-27 23:14:36 +02:00
Sebastiaan van Stijn
190b6f64e3
builder/dockerfile: copy: remove LCOW code
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-27 23:14:28 +02:00
Sebastiaan van Stijn
0998c7e363
system: remove LCOWSupported() utility
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-27 13:36:19 +02:00
Sebastiaan van Stijn
28409ca6c7
replace pkg/signal with moby/sys/signal v0.5.0
This code was moved to the moby/sys repository

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-23 09:32:54 +02:00
Sebastiaan van Stijn
46c591b045
pkg/system: deprecate some consts and move them to pkg/idtools
These consts were used in combination with idtools utilities, which
makes it a more logical location for these consts to live.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-06-18 23:16:15 +02:00
Akihiro Suda
9e8cf1016e
Merge pull request #42473 from thaJeztah/unfork_buildkit
revendor BuildKit (master branch)
2021-06-17 10:56:25 +09:00
Sebastiaan van Stijn
3eb1257698
revendor BuildKit (master branch)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-06-16 01:17:48 +02:00
Sebastiaan van Stijn
bb17074119
reformat "nolint" comments
Unlike regular comments, nolint comments should not have a leading space.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-06-10 13:03:42 +02:00
Sebastiaan van Stijn
dd1374f7b2
if-return: redundant if ...; err != nil check (revive)
builder/builder-next/adapters/snapshot/snapshot.go:386:3: if-return: redundant if ...; err != nil check, just return error instead. (revive)
            if err := b.Put(keyIsCommitted, []byte{}); err != nil {
                return err
            }

    plugin/fetch_linux.go:112:2: if-return: redundant if ...; err != nil check, just return error instead. (revive)
        if err := images.Dispatch(ctx, images.Handlers(handlers...), nil, desc); err != nil {
            return err
        }

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-06-10 13:03:40 +02:00
Sebastiaan van Stijn
d13997b4ba
gosec: G601: Implicit memory aliasing in for loop
plugin/v2/plugin.go:141:50: G601: Implicit memory aliasing in for loop. (gosec)
                    updateSettingsEnv(&p.PluginObj.Settings.Env, &s)
                                                                 ^
    libcontainerd/remote/client.go:572:13: G601: Implicit memory aliasing in for loop. (gosec)
                cpDesc = &m
                         ^
    distribution/push_v2.go:400:34: G601: Implicit memory aliasing in for loop. (gosec)
                (metadata.CheckV2MetadataHMAC(&mountCandidate, pd.hmacKey) ||
                                              ^
    builder/dockerfile/builder.go:261:84: G601: Implicit memory aliasing in for loop. (gosec)
            currentCommandIndex = printCommand(b.Stdout, currentCommandIndex, totalCommands, &meta)
                                                                                             ^
    builder/dockerfile/builder.go:278:46: G601: Implicit memory aliasing in for loop. (gosec)
            if err := initializeStage(dispatchRequest, &stage); err != nil {
                                                       ^
    daemon/container.go:283:40: G601: Implicit memory aliasing in for loop. (gosec)
            if err := parser.ValidateMountConfig(&cfg); err != nil {
                                                 ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-06-10 13:03:29 +02:00
Sebastiaan van Stijn
e047d984dc
Remove LCOW code (step 1)
The LCOW implementation in dockerd has been deprecated in favor of re-implementation
in containerd (in progress). Microsoft started removing the LCOW V1 code from the
build dependencies we use in Microsoft/opengcs (soon to be part of Microsoft/hcshhim),
which means that we need to start removing this code.

This first step removes the lcow graphdriver, the LCOW initialization code, and
some LCOW-related utilities.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-06-03 21:16:21 +02:00
Brian Goff
4b981436fe Fixup libnetwork lint errors
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2021-06-01 23:48:32 +00:00
Brian Goff
a0a473125b Fix libnetwork imports
After moving libnetwork to this repo, we need to update all the import
paths for libnetwork to point to docker/docker/libnetwork instead of
docker/libnetwork.
This change implements that.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2021-06-01 21:51:23 +00:00
Tonis Tiigi
85167fc634 builder-next: relax second cache key requirements for schema1
Schema1 images can not have a config based cache key
before the layers are pulled. Avoid validation and reuse
manifest digest as a second key.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-05-12 22:44:07 -07:00
Sebastiaan van Stijn
5a5f8564ba
builder/dockerfile: add "ALL_PROXY" to list of default build args
Relates to a82fff6377/docs/packages.md (proxies)

> (..) the first four of these are the standard built-in build-arg options
> available for `docker build`
> (..) The last, `all_proxy`, is a standard var used for socks proxying. Since
> it is not built into `docker build`, if you want to use it, you will need to
> add the following line to the dockerfile:
>
> ARG all_proxy

Given the we support all other commonly known proxy env-vars by default, it makes
sense to add this one as well.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-04-24 18:25:16 +02:00
Sebastiaan van Stijn
a09c0276a2
builder: produce error when using unsupported Dockerfile option
With the promotion of the experimental Dockerfile syntax to "stable", the Dockerfile
syntax now includes some options that are supported by BuildKit, but not (yet)
supported in the classic builder.

As a result, parsing a Dockerfile may succeed, but any flag that's known to BuildKit,
but not supported by the classic builder is silently ignored;

    $ mkdir buildkit_flags && cd buildkit_flags
    $ touch foo.txt

For example, `RUN --mount`:

    DOCKER_BUILDKIT=0 docker build --no-cache -f- . <<EOF
    FROM busybox
    RUN --mount=type=cache,target=/foo echo hello
    EOF

    Sending build context to Docker daemon  2.095kB
    Step 1/2 : FROM busybox
     ---> 219ee5171f80
    Step 2/2 : RUN --mount=type=cache,target=/foo echo hello
     ---> Running in 022fdb856bc8
    hello
    Removing intermediate container 022fdb856bc8
     ---> e9f0988844d1
    Successfully built e9f0988844d1

Or `COPY --chmod` (same for `ADD --chmod`):

    DOCKER_BUILDKIT=0 docker build --no-cache -f- . <<EOF
    FROM busybox
    COPY --chmod=0777 /foo.txt /foo.txt
    EOF

    Sending build context to Docker daemon  2.095kB
    Step 1/2 : FROM busybox
     ---> 219ee5171f80
    Step 2/2 : COPY --chmod=0777 /foo.txt /foo.txt
     ---> 8b7117932a2a
    Successfully built 8b7117932a2a

Note that unknown flags still produce and error, for example, the below fails because `--hello` is an unknown flag;

    DOCKER_BUILDKIT=0 docker build -<<EOF
    FROM busybox
    RUN --hello echo hello
    EOF

    Sending build context to Docker daemon  2.048kB
    Error response from daemon: dockerfile parse error line 2: Unknown flag: hello

With this patch applied
----------------------------

With this patch applied, flags that are known in the Dockerfile spec, but are not
supported by the classic builder, produce an error, which includes a link to the
documentation how to enable BuildKit:

    DOCKER_BUILDKIT=0 docker build --no-cache -f- . <<EOF
    FROM busybox
    RUN --mount=type=cache,target=/foo echo hello
    EOF

    Sending build context to Docker daemon  2.048kB
    Step 1/2 : FROM busybox
     ---> b97242f89c8a
    Step 2/2 : RUN --mount=type=cache,target=/foo echo hello
    the --mount option requires BuildKit. Refer to https://docs.docker.com/go/buildkit/ to learn how to build images with BuildKit enabled

    DOCKER_BUILDKIT=0 docker build --no-cache -f- . <<EOF
    FROM busybox
    COPY --chmod=0777 /foo.txt /foo.txt
    EOF

    Sending build context to Docker daemon  2.095kB
    Step 1/2 : FROM busybox
     ---> b97242f89c8a
    Step 2/2 : COPY --chmod=0777 /foo.txt /foo.txt
    the --chmod option requires BuildKit. Refer to https://docs.docker.com/go/buildkit/ to learn how to build images with BuildKit enabled

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-03-14 14:11:08 +01:00
Tibor Vass
3c9b4a63b5
Merge pull request #42056 from thaJeztah/bump_buildkit
vendor: github.com/moby/buildkit v0.8.2
2021-02-24 22:26:53 -08:00
Tibor Vass
9bf93e90fa builder: fix incorrect cache match for inline cache with empty layers
See https://github.com/moby/buildkit/pull/1993

Signed-off-by: Tibor Vass <tibor@docker.com>
2021-02-25 01:49:22 +00:00
Tibor Vass
271b43c43f
Merge pull request #42020 from tonistiigi/lease-blobs-fixes
builder: fix blobs releasing via leases after pull
2021-02-23 08:36:38 -08:00
Tonis Tiigi
b53ea19c49 builder: fix pull synchronization regression
Config resolution was synchronized based on a wrong key as ref
variable is initialized only after in the same function. Using
the right key isn't fully correct either as the synchronized method
changes properties of the puller instance and can't be just skipped.
Added better error handling for the same case as well.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-02-16 22:48:37 -08:00