Commit graph

1160 commits

Author SHA1 Message Date
Sebastiaan van Stijn
fed26d5b3c
vendor: github.com/moby/buildkit v0.11.7-dev
full diff: 0a15675913...616c3f613b

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 0ec73a7892)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-24 16:14:03 +02:00
Sebastiaan van Stijn
5bba60b1bb
builder/builder-next: use string-literals for easier grep'ing
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 2f61620339)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-15 00:49:08 +02:00
Sebastiaan van Stijn
632fc235d6
builder/dockerfile: use string-literals for easier grep'ing
Use string-literal for reduce escaped quotes, which makes for easier grepping.
While at it, also changed http -> https to keep some linters at bay.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 202907b14c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-15 00:48:58 +02:00
CrazyMax
a4b1a5aef4
vendor: github.com/moby/buildkit@v0.11 0a0807e
full diff 798ad6b...0a15675

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-07-12 19:45:44 +02:00
CrazyMax
35a29c7328
builder: pass host-gateway IP as worker label
We missed a case when parsing extra hosts from the dockerfile
frontend so the build fails.

To handle this case we need to set a dedicated worker label
that contains the host gateway IP so clients like Buildx
can just set the proper host:ip when parsing extra hosts
that contain the special string "host-gateway".

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 21e50b89c9)
2023-06-22 16:23:40 +02:00
Paweł Gronowski
647ba03224
builder-next: Set moby exporter as default
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit d63569c73d)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-06-02 16:06:11 +02:00
Jeyanthinath Muthuram
e370f224ae
fixing consistent aliases for OCI spec imports
Signed-off-by: Jeyanthinath Muthuram <jeyanthinath10@gmail.com>
(cherry picked from commit 307b09e7eb)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-01 17:22:49 +02:00
Sebastiaan van Stijn
ec8ec9056c
builder/remotecontext: deprecate CachableSource, NewCachableSource
This type (as well as TarsumBackup), was used for the experimental --stream
support for the classic builder. This feature was removed in commit
6ca3ec88ae, which also removed uses of
the CachableSource type.

As far as I could find, there's no external consumers of these types,
but let's deprecated it, to give potential users a heads-up that it
will be removed.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 37d4b0bee9)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-26 15:13:27 +02:00
Sebastiaan van Stijn
ae6e9333c0
vendor: github.com/moby/buildkit v0.11.7-0.20230525183624-798ad6b0ce9f
full diff: https://github.com/moby/buildkit/compare/v0.11.6...798ad6b0ce9f2fe86dfb2b0277e6770d0b545871

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 79ca6630d4)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-25 21:45:42 +02:00
Kevin Alvarez
07140c0eca
build: use daemon id as worker id for the graph driver controller
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 6d139e5e95)
2023-05-18 22:29:45 +02:00
Sebastiaan van Stijn
807e415260
Merge pull request #45526 from laurazard/backport-classic-builder
[24.0 backport] c8d: implement classic builder
2023-05-11 21:12:01 +02:00
Laura Brehm
9717369913
c8d: implement classic builder
Co-authored-by: Djordje Lukic <djordje.lukic@docker.com>
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
(cherry picked from commit e46674b6a7)
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
2023-05-11 19:13:13 +01:00
Cory Snider
f77a3274b4
[chore] clean up reexec.Init() calls
Now that most uses of reexec have been replaced with non-reexec
solutions, most of the reexec.Init() calls peppered throughout the test
suites are unnecessary. Furthermore, most of the reexec.Init() calls in
test code neglects to check the return value to determine whether to
exit, which would result in the reexec'ed subprocesses proceeding to run
the tests, which would reexec another subprocess which would proceed to
run the tests, recursively. (That would explain why every reexec
callback used to unconditionally call os.Exit() instead of returning...)

Remove unneeded reexec.Init() calls from test and example code which no
longer needs it, and fix the reexec.Init() calls which are not inert to
exit after a reexec callback is invoked.

Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit 4e0319c878)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-11 16:31:41 +02:00
Djordje Lukic
a1e1038736
Use the image service instead of the reference store for tagging
The image store sends events when a new image is created/tagged, using
it instead of the reference store makes sure we send the "tag" event
when a new image is built using buildx.

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-04-26 15:10:01 +02:00
Sebastiaan van Stijn
fd03a52468
Merge pull request #45294 from vvoland/builder-fix-prune-timestamp
builder-next/prune: Handle `until` filter timestamps
2023-04-25 11:19:27 +02:00
Paweł Gronowski
e502c69140
builder: use typed error for invalid "until" value
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-04-21 10:25:53 +02:00
Paweł Gronowski
54a125f677
builder-next/prune: Handle "until" filter timestamps
Fixes `docker system prune --filter until=<timestamp>`.
`docker system prune` claims to support "until" filter for timestamps,
but it doesn't work because builder "until" filter only supports
duration.
Use the same filter parsing logic and then convert the timestamp to a
relative "keep-duration" supported by buildkit.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-04-20 21:20:32 +02:00
Tonis Tiigi
38e791fb2f
vendor: update buildkit to v0.11.6
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2023-04-20 11:17:25 -07:00
Sebastiaan van Stijn
890959eea8
builder-next: fix buildkit version
Update the version manually (we don't have automation for this yet), and
add a comment to vendor.mod to help users remind to update it.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-14 10:44:27 +02:00
Sebastiaan van Stijn
81e62af94a
use consistent alias for containerd's errdefs package
The signatures of functions in containerd's errdefs packages are very
similar to those in our own, and it's easy to accidentally use the wrong
package.

This patch uses a consistent alias for all occurrences of this import.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-08 19:30:33 +02:00
Tonis Tiigi
0ac3bf837b
builder-next: disable mergeop and diffop
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
2023-03-16 08:23:22 -06:00
Paweł Gronowski
af677245fb
c8d/builder: Store untagged images as dangling
Set `dangling-name-prefix` exporter attribute  to `moby-dangling` which
makes it create an containerd image even when user didn't provide any
name for the new image.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-03-08 16:59:33 +01:00
Paweł Gronowski
8f716ca06a
builder/overrides: Add consts for attribute key names
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-03-08 16:59:26 +01:00
Tonis Tiigi
f8b468fda2
builder-next: enable more cache backends
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Co-authored-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-03-07 10:32:40 +01:00
Djordje Lukic
bba77163ff
c8d: Make build and buildx work
- Only use the image exporter in build if we don't use containerd
  Without this "docker build" fails with:

    Error response from daemon: exporter "image" could not be found

- let buildx know we support containerd snapshotter
- Pass the current snapshotter to the buildkit worker

  If buildkit uses a different snapshotter we can't list the images any
  more because we can't find the snapshot.

builder/builder-next: make ContainerdWorker a minimal wrapper

Note that this makes "Worker" a public field, so technically one could
overwrite it.

builder-next: reenable runc executor

Currently, without special CNI config the builder would
only create host network containers that is a security issue.

Using runc directly instead of shim is faster as well
as builder doesn’t need anything from shim. The overhead
of setting up network sandbox is much slower of course.

builder/builder-next: simplify options handling

Trying to simplify the logic;

- Use an early return if multiple outputs are provided
- Only construct the list of tags if we're using an image (or moby) exporter
- Combine some logic for snapshotter and non-snapshotter handling

Create a constant for the moby exporter

Pass a context when creating a router

The context has a 10 seconds timeout which should be more than enough to
get the answer from containerd.

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
Co-authored-by: Sebastiaan van Stijn <github@gone.nl>
Co-authored-by: Tonis Tiigi <tonistiigi@gmail.com>
Co-authored-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Co-authored-by: Paweł Gronowski <pawel.gronowski@docker.com>
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-03-07 10:01:47 +01:00
Paweł Gronowski
3e4c4df664
builder/exporter: Remove commented unused const
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-03-03 11:05:07 +01:00
Paweł Gronowski
6588b3eef0
builder: Remove buildkit's build information
Buildkit deprecated build information in v0.11 and will remove it in v0.12.
It's suggested to use provenance attestations instead.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-03-02 19:17:11 +01:00
Tonis Tiigi
335907d187
builder: define GetRemotes for the worker
The function signature has changed since v0.10.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2023-02-07 10:24:36 -08:00
Tonis Tiigi
666334bd48
vendor: update buildkit to v0.11.2
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2023-02-01 23:29:10 -08:00
Sebastiaan van Stijn
17b2c89116
Merge pull request #44818 from vvoland/c8d-commit-ctx-upstream
images: Pass context to commit related operations
2023-01-18 15:17:52 +01:00
Nicolas De Loof
9b5c21309b
images: Pass context to commit related operations
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-01-18 12:23:32 +01:00
Cory Snider
c71555f030 libnetwork: return concrete-typed *Endpoint
libnetwork.Endpoint is an interface with a single implementation.

https://github.com/golang/go/wiki/CodeReviewComments#interfaces

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-01-13 14:19:06 -05:00
Cory Snider
0e91d2e0e9 libnetwork: return concrete-typed *Sandbox
Basically every exported method which takes a libnetwork.Sandbox
argument asserts that the value's concrete type is *sandbox. Passing any
other implementation of the interface is a runtime error! This interface
is a footgun, and clearly not necessary. Export and use the concrete
type instead.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-01-13 14:19:06 -05:00
Cory Snider
f96b9bf761 libnetwork: return concrete-typed *Controller
libnetwork.NetworkController is an interface with a single
implementation.

https://github.com/golang/go/wiki/CodeReviewComments#interfaces

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-01-13 14:09:37 -05:00
Sebastiaan van Stijn
4f81f9d5ea
builder/dockerfile: use example.com for tests
Trying to remove the "docker.io" domain from locations where it's not relevant.
In these cases, this domain was used as a "random" domain for testing or example
purposes.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-01-10 15:28:09 +01:00
Sebastiaan van Stijn
59880cb855
builder/dockerfile: use strings.Cut()
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-21 11:09:00 +01:00
Sebastiaan van Stijn
604a2d67b5
builder/remotecontext/git: use strings.Cut()
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-21 11:09:00 +01:00
Sebastiaan van Stijn
c63ea32a17
pkg/ioutils: TempDir: move to pkg/longpath
This utility wasn't very related to all other utilities in pkg/ioutils.
Moving it to longpath to also make it more clear what it does.

It looks like there's only a single (public) external consumer of this
utility, and only used in a test, and it's not 100% clear if it was
intentional to use our package, of if it was a case of "I actually meant
`io/ioutil.MkdirTemp`" so we could consider skipping the alias.

While moving the package, I also renamed `TempDir` to `MkdirTemp`, which
is the signature it matches in "os" from stdlib.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-20 23:24:12 +01:00
Sebastiaan van Stijn
6371675bf9
Merge pull request #44275 from thaJeztah/move_pkg_system_funcs
pkg/system: move some functions to a new home
2022-12-16 15:25:41 +01:00
CrazyMax
521b8c02cc
builder: handle host-gateway with extra hosts
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-12-02 23:17:47 +01:00
Sebastiaan van Stijn
9f3e5eead5
pkg/system: deprecate DefaultPathEnv, move to oci
This patch:

- Deprecates pkg/system.DefaultPathEnv
- Moves the implementation inside oci
- Adds TODOs to align the default in the Builder with the one used elsewhere

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-29 17:02:50 +01:00
Sebastiaan van Stijn
de5ac5410d
Merge pull request #44432 from thaJeztah/no_execabs
switch back from golang.org/x/sys/execabs to os/exec
2022-11-11 00:37:19 +01:00
Sebastiaan van Stijn
b7d242f094
builder/remotecontext/gitutils: switch back to os/exec
This is a partial revert of 7ca0cb7ffa, which
switched from os/exec to the golang.org/x/sys/execabs package to mitigate
security issues (mainly on Windows) with lookups resolving to binaries in the
current directory.

from the go1.19 release notes https://go.dev/doc/go1.19#os-exec-path

> ## PATH lookups
>
> Command and LookPath no longer allow results from a PATH search to be found
> relative to the current directory. This removes a common source of security
> problems but may also break existing programs that depend on using, say,
> exec.Command("prog") to run a binary named prog (or, on Windows, prog.exe) in
> the current directory. See the os/exec package documentation for information
> about how best to update such programs.
>
> On Windows, Command and LookPath now respect the NoDefaultCurrentDirectoryInExePath
> environment variable, making it possible to disable the default implicit search
> of “.” in PATH lookups on Windows systems.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-09 12:28:17 +01:00
Sebastiaan van Stijn
0f7c9cd27e
Remove uses of deprecated go-digest.NewDigestFromHex, go-digest.Digest.Hex
Both of these were deprecated in 55f675811a,
but the format of the GoDoc comments didn't follow the correct format, which
caused them not being picked up by tools as "deprecated".

This patch updates uses in the codebase to use the alternatives.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-08 16:42:13 +01:00
Paweł Gronowski
66a0289081
builder: Don't store context in struct
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2022-11-03 12:22:46 +01:00
Nicolas De Loof
def549c8f6
imageservice: Add context to various methods
Co-authored-by: Paweł Gronowski <pawel.gronowski@docker.com>
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2022-11-03 12:22:40 +01:00
Sebastiaan van Stijn
4fdc1bb1fb
builder/remotecontext/git: allow building on go1.18
cmd.Environ() is new in go1.19, and not needed for this specific case.
Without this, trying to use this package in code that uses go1.18 will fail;

    builder/remotecontext/git/gitutils.go:216:23: cmd.Environ undefined (type *exec.Cmd has no field or method Environ)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-21 17:41:41 +02:00
Sebastiaan van Stijn
b9921a5560
Merge pull request #44273 from thaJeztah/use_walkdir
use filepath.WalkDir instead of filepath.Walk
2022-10-21 02:28:56 +02:00
Cory Snider
67d010bd2c builder: add missing doc comment
Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-10-20 16:47:18 -04:00
Cory Snider
94672c89cc builder: fix running git commands on Windows
Setting cmd.Env overrides the default of passing through the parent
process' environment, which works out fine most of the time, except when
it doesn't. For whatever reason, leaving out all the environment causes
git-for-windows sh.exe subprocesses to enter an infinite loop of
access violations during Cygwin initialization in certain environments
(specifically, our very own dev container image).

Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-10-20 16:47:18 -04:00