Commit graph

1108 commits

Author SHA1 Message Date
Tonis Tiigi
1fcbc7cf93
builder-next: fix timing filter for default policy
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit 49d088d9ce)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-15 19:57:57 +01:00
CrazyMax
40dbcd6e11
builder: pass host-gateway IP as worker label
We missed a case when parsing extra hosts from the dockerfile
frontend so the build fails.

To handle this case we need to set a dedicated worker label
that contains the host gateway IP so clients like Buildx
can just set the proper host:ip when parsing extra hosts
that contain the special string "host-gateway".

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 21e50b89c9)
2023-06-22 16:30:07 +02:00
Kevin Alvarez
276ef743cb
build: use daemon id as worker id for the graph driver controller
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 6d139e5e95)

# Conflicts:
#	builder/builder-next/controller.go
2023-05-18 22:22:23 +02:00
Djordje Lukic
fe0a414613
Use the image service instead of the reference store for tagging
The image store sends events when a new image is created/tagged, using
it instead of the reference store makes sure we send the "tag" event
when a new image is built using buildx.

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-04-26 16:32:25 +02:00
Tonis Tiigi
d43958b50b
builder-next: disable mergeop and diffop
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
(cherry picked from commit 0ac3bf837b)
Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
2023-03-16 08:27:26 -06:00
Tonis Tiigi
e697c9365f
builder: define GetRemotes for the worker
The function signature has changed since v0.10.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit 335907d187)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-02-08 01:01:18 +01:00
Sebastiaan van Stijn
c93e1e9e66
libnetwork/config: merge DaemonCfg into Config
It was unclear what the distinction was between these configuration
structs, so merging them to simplify.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 528428919e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-01-18 21:24:31 +01:00
Sebastiaan van Stijn
c0c146fc82
Remove uses of deprecated go-digest.NewDigestFromHex, go-digest.Digest.Hex
Both of these were deprecated in 55f675811a,
but the format of the GoDoc comments didn't follow the correct format, which
caused them not being picked up by tools as "deprecated".

This patch updates uses in the codebase to use the alternatives.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 0f7c9cd27e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-09 00:17:18 +01:00
CrazyMax
1da1a3ea87
builder: handle host-gateway with extra hosts
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 521b8c02cc)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-03 14:39:41 +01:00
Sebastiaan van Stijn
80dc5186ec
builder/remotecontext/git: allow building on go1.18
cmd.Environ() is new in go1.19, and not needed for this specific case.
Without this, trying to use this package in code that uses go1.18 will fail;

    builder/remotecontext/git/gitutils.go:216:23: cmd.Environ undefined (type *exec.Cmd has no field or method Environ)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 4fdc1bb1fb)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-21 17:44:20 +02:00
Cory Snider
f056df579a builder: add missing doc comment
Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-10-20 16:46:23 -04:00
Cory Snider
c062238ea4 builder: fix running git commands on Windows
Setting cmd.Env overrides the default of passing through the parent
process' environment, which works out fine most of the time, except when
it doesn't. For whatever reason, leaving out all the environment causes
git-for-windows sh.exe subprocesses to enter an infinite loop of
access violations during Cygwin initialization in certain environments
(specifically, our very own dev container image).

Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-10-20 16:46:23 -04:00
Cory Snider
20ff8a2380 builder: make git config isolation opt-in
While it is undesirable for the system or user git config to be used
when the daemon clones a Git repo, it could break workflows if it was
unconditionally applied to docker/cli as well.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-10-20 16:46:23 -04:00
Cory Snider
ca99cab891 builder: isolate git from local system
Prevent git commands we run from reading the user or system
configuration, or cloning submodules from the local filesystem.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-10-20 16:46:23 -04:00
Cory Snider
5829b244ec builder: explicitly set CWD for all git commands
Keep It Simple! Set the working directory for git commands by...setting
the git process's working directory. Git commands can be run in the
parent process's working directory by passing the empty string.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-10-20 16:46:23 -04:00
Cory Snider
3bc8fccc1b builder: modernize TestCheckoutGit
Make the test more debuggable by logging all git command output and
running each table-driven test case as a subtest.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-10-20 16:46:23 -04:00
Sebastiaan van Stijn
44d42c2b16
replace pkg/fileutils Matching funcs with github.com/moby/patternmatcher
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 3c69b9f2c5)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-01 01:09:36 +02:00
Sebastiaan van Stijn
9d44956d8c
builder: fix empty-lines (revive)
builder/remotecontext/detect_test.go:64:66: empty-lines: extra empty line at the end of a block (revive)
    builder/remotecontext/detect_test.go:78:46: empty-lines: extra empty line at the end of a block (revive)
    builder/remotecontext/detect_test.go:91:51: empty-lines: extra empty line at the end of a block (revive)
    builder/dockerfile/internals_test.go:95:38: empty-lines: extra empty line at the end of a block (revive)
    builder/dockerfile/copy.go:86:112: empty-lines: extra empty line at the end of a block (revive)
    builder/dockerfile/dispatchers_test.go:286:39: empty-lines: extra empty line at the start of a block (revive)
    builder/dockerfile/builder.go:280:38: empty-lines: extra empty line at the end of a block (revive)
    builder/dockerfile/dispatchers.go:66:85: empty-lines: extra empty line at the start of a block (revive)
    builder/dockerfile/dispatchers.go:559:85: empty-lines: extra empty line at the start of a block (revive)
    builder/builder-next/adapters/localinlinecache/inlinecache.go:26:183: empty-lines: extra empty line at the start of a block (revive)
    builder/builder-next/adapters/containerimage/pull.go:441:9: empty-lines: extra empty line at the start of a block (revive)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit ecb4ed172b)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-30 23:59:29 +02:00
Sebastiaan van Stijn
95831246a2
Update uses of Image platform fields in OCI image-spec
The OCI image spec is considering to change the Image struct and embedding the
Platform type (see opencontainers/image-spec#959) in the go implementation.
Moby currently uses some struct-literals to propagate the platform fields,
which will break once those changes in the OCI spec are merged.

Ideally (once that change arrives) we would update the code to set the Platform
information as a whole, instead of assigning related fields individually, but
in some cases in the code, image platform information is only partially set
(for example, OSVersion and OSFeatures are not preserved in all cases). This
may be on purpose, so needs to be reviewed.

This patch keeps the current behavior (assigning only specific fields), but
removes the use of struct-literals to make the code compatible with the
upcoming changes in the image-spec module.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 3cb933db9d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-10 12:19:53 +02:00
Sebastiaan van Stijn
910d5c44fc
api: add BuildCache.Parents for API >= v1.42
This field was added to replace the deprecated "Parent" field.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit e0db8207f3)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-04 09:33:35 +02:00
Sebastiaan van Stijn
0e3d20cb20
api: deprecate BuildCache.Parent in API >= v1.42
This field has been deprecated in BuildKit, so this follows the deprecation
in the Engine API.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit ebf339628a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-04 09:33:33 +02:00
Sebastiaan van Stijn
cdbca4061b
gofmt GoDoc comments with go1.19
Older versions of Go don't format comments, so committing this as
a separate commit, so that we can already make these changes before
we upgrade to Go 1.19.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 52c1a2fae8)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-13 22:42:29 +02:00
Erik Sipsma
88916949ef
build: force unlazy of refs by calling extract
Signed-off-by: Erik Sipsma <erik@sipsma.dev>
2022-06-24 11:54:41 +02:00
CrazyMax
795ecf02ce
builder: fix empty reference
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-05-28 21:23:30 +02:00
Eng Zer Jun
7873c27cfb
all: replace strings.Replace with strings.ReplaceAll
strings.ReplaceAll(s, old, new) is a wrapper function for
strings.Replace(s, old, new, -1). But strings.ReplaceAll is more
readable and removes the hardcoded -1.

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2022-05-09 19:45:40 +08:00
Sebastiaan van Stijn
d733481399
daemon: daemon.ContainerKill() accept stop-signal as string
This allows the postContainersKill() handler to pass values as-is. As part of
the rewrite, I also moved the daemon.GetContainer(name) call later in the
function, so that we can fail early if an invalid signal is passed, before
doing the (heavier) fetching of the container.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-05-05 11:27:47 +02:00
Sebastiaan van Stijn
41b96bff55
update uses of container.ContainerCreateCreatedBody to CreateResponse
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-28 22:39:20 +02:00
Sebastiaan van Stijn
449250994f
builder/remotecontext/urlutil: simplify and improve documentation
Simplify some of the logic, and add documentation about the package,
as well as warnings that this package should not be used as a general-
purpose utility.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-12 19:58:09 +02:00
Sebastiaan van Stijn
5f89a6a78e
pkg/urlutil: deprecate, and move to builder/remotecontext/urlutil
pkg/urlutil (despite its poorly chosen name) is not really intended as a generic
utility to handle URLs, and should only be used by the builder to handle (remote)
build contexts.

- IsURL() only does a very rudimentary check for http(s):// prefixes, without any
  other validation, but due to its name may give incorrect expectations.
- IsGitURL() is written specifically with docker build remote git contexts in
  mind, and has handling for backward-compatibility, where strings that are
  not URLs, but start with "github.com/" are accepted.

Because of the above, this patch:

- moves the package inside builder/remotecontext, close to where it's intended
  to be used (ideally this would be part of build/remotecontext itself, but this
  package imports many other dependencies, which would introduce those as extra
  dependencies in the CLI).
- deprecates pkg/urlutil, but adds aliases as there are some external consumers.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-12 19:58:05 +02:00
Sebastiaan van Stijn
1ad9a09af8
Merge pull request #43369 from sestegra/stop
Allow STOPSIGNAL instruction in commit change
2022-03-26 14:40:32 +01:00
Sebastiaan van Stijn
0a3336fd7d
Merge pull request #43366 from corhere/finish-identitymapping-refactor
Finish refactor of UID/GID usage to a new struct
2022-03-25 14:51:05 +01:00
CrazyMax
fda0226a89
builder: add buildinfo for buildkit
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-03-23 16:07:03 +01:00
CrazyMax
b899db6423
builder: enable shm-size and ulimit for buildkit
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-03-23 16:07:03 +01:00
CrazyMax
aadb3bf766
builder: changes needed since buildkit 0.10.0
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-03-23 16:07:03 +01:00
CrazyMax
a2aaf4cc83
vendor buildkit v0.10.0
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-03-22 18:51:27 +01:00
Cory Snider
098a44c07f Finish refactor of UID/GID usage to a new struct
Finish the refactor which was partially completed with commit
34536c498d, passing around IdentityMapping structs instead of pairs of
[]IDMap slices.

Existing code which uses []IDMap relies on zero-valued fields to be
valid, empty mappings. So in order to successfully finish the
refactoring without introducing bugs, their replacement therefore also
needs to have a useful zero value which represents an empty mapping.
Change IdentityMapping to be a pass-by-value type so that there are no
nil pointers to worry about.

The functionality provided by the deprecated NewIDMappingsFromMaps
function is required by unit tests to to construct arbitrary
IdentityMapping values. And the daemon will always need to access the
mappings to pass them to the Linux kernel. Accommodate these use cases
by exporting the struct fields instead. BuildKit currently depends on
the UIDs and GIDs methods so we cannot get rid of them yet.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-03-14 16:28:57 -04:00
Stéphane Este-Gracias
a768652fd6
Support STOPSIGNAL instruction in commit change
Signed-off-by: Stéphane Este-Gracias <sestegra@gmail.com>
2022-03-12 17:10:35 +01:00
Sebastiaan van Stijn
b94a319fdf
remove deprecated builder/dockerignore
This package was deprecated in de56a90929, which
was part of the 20.10 release, so consumers of this package should've been
able to migrate to the new location.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-06 00:49:59 +01:00
Sebastiaan van Stijn
a0230f3d9a
remove unneeded "digest" alias for "go-digest"
I think this was there for historic reasons (may have been goimports expected
this, and we used to have a linter that wanted it), but it's not needed, so
let's remove it (to make my IDE less complaining about unneeded aliases).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-04 14:49:42 +01:00
Sebastiaan van Stijn
32e5fe5099
Merge pull request #43182 from thaJeztah/layer_remove_unused_error
layer: remove unused error return from .Size() and .DiffSize()
2022-02-17 20:51:45 +01:00
Brian Goff
047d58f007
Merge pull request #43187 from thaJeztah/remove_lcow_checks
Remove various leftover LCOW checks
2022-02-17 11:22:19 -08:00
Sebastiaan van Stijn
cfddecc3d2
builder/dockerfile: remove leftover LCOW platform checks
This removes some of the checks that were added in 0cba7740d4,
but should no longer be needed.

- `dockerfile.BuildFromConfig()` is used for `docker (container) commmit` and
  `docker (image) import`. For `docker import`, we're failing early already.
  For `commit`, it won't be possible to have a container that doesn't have the
  right operating-system, so there's no need to validate.
- `dispatchRequest.getImageOrStage()`: simplify the check; all checks resulted
  in an error on Windows, so it came down to "Windows does not support FROM scratch".
- `dispatchState.beginStage()`: `image.OperatingSystem()` already defaults to the
  `runtime.GOOS` if unset, so remove the local default fallback.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-25 15:23:11 +01:00
Sebastiaan van Stijn
e1ea911aba
layer: remove unused error return from .Size() and .DiffSize()
None of the implementations used return an error, so removing the error
return can simplify using these.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-24 18:45:47 +01:00
Sebastiaan van Stijn
bb66ebd621
distribution: xfer.LayerDownloadManager.Download(): remove "os" argument
This argument was added for LCOW support, but it was only used to verify if
the passed platform (OS) matched the host. Given that all uses of this function
(except for one) passed runtime.GOOS, we may as well move the check to that
location.

We should do more cleaning up after this, and perform such validations early,
instead of passing platform around in too many places where it's only used for
similar validations. This is a first step in that direction.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-21 15:07:55 +01:00
Sebastiaan van Stijn
0b0a995d9d
distribution: remove RootFSDownloadManager interface
This interface only had a single implementation (xfer.LayerDownloadManager),
and all places where it was used already imported the xfer package.
Removing the interface, also makes it a closer match to the "upload" part,
as `xfer.LayerUploadManager()` did not use an interface.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-21 13:53:36 +01:00
CrazyMax
b4e056d556 buildkit: normalize build target and local platform
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-10-19 17:11:06 +02:00
Eng Zer Jun
c55a4ac779
refactor: move from io/ioutil to io and os package
The io/ioutil package has been deprecated in Go 1.16. This commit
replaces the existing io/ioutil functions with their new definitions in
io and os packages.

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2021-08-27 14:56:57 +08:00
Sebastiaan van Stijn
686be57d0a
Update to Go 1.17.0, and gofmt with Go 1.17
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-08-24 23:33:27 +02:00
Aaron Lehmann
ba2adeebdb
Merge pull request #42676 from aaronlehmann/patternmatcher-doublestar-bug
fileutils: Fix incorrect handling of "**/foo" pattern
2021-08-16 19:58:55 -07:00
Aaron Lehmann
97ede9df26 Rename Matches to MatchesOrParentMatches
Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
2021-08-12 18:10:04 -07:00