Commit graph

1188 commits

Author SHA1 Message Date
Bjorn Neergaard
34e71b374b
builder-next/a/ci/pull: strongly type flightcontrol.Group
Now that this is a generic, we can define a struct type at the package
level, and remove the casting logic necessary when we had to use
interface{}.

Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-09-21 14:18:57 -06:00
Bjorn Neergaard
c8773e10b1
builder-next/wrapper: use new exptypes constants
Introduced in 4fc2d7b5e7

Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-09-21 14:18:55 -06:00
Bjorn Neergaard
a728bd0ee9
builder-next/pull: use ResolveMode.String() helper
Introduced years ago in 6644e1b0da

Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-09-21 14:18:52 -06:00
Bjorn Neergaard
221b73e23d
builder-next: apply source policies on config resolve
SourcePolicy was accounted for in 330cf7ae7d

TODO: replace applySourcePolicies with BuildKit's implementation, which
is currently unexported.

Co-authored-by: Tonis Tiigi <tonistiigi@gmail.com>
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-09-21 14:18:50 -06:00
Bjorn Neergaard
2246297ae6
builder-next: adopt new wrapped types
With BuildKit 0.12, some existing types are now required to be wrapped
by new types:

* containerd's LeaseManager and ContentStore have to be a
  (namespace-aware) BuildKit type since f044e0a946
* BuildKit's solver.CacheManager is used instead of
  bboltstorage.CacheKeyStorage since 2b30693409
* The MaxAge config field is a bkconfig.Duration since e06c96274f

Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-09-21 14:18:48 -06:00
Bjorn Neergaard
c217e3c87a
vendor: github.com/moby/buildkit v0.12.2
The following changes were required:
* integration/build: progressui's signature changed in 6b8fbed01e
* builder-next: flightcontrol.Group has become a generic type in 8ffc03b8f0
* builder-next/executor: add github.com/moby/buildkit/executor/resources types, necessitated by 6e87e4b455
* builder-next: stub util/network/Namespace.Sample(), necessitated by 963f16179f

Co-authored-by: CrazyMax <crazy-max@users.noreply.github.com>
Co-authored-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-09-21 14:18:45 -06:00
Bjorn Neergaard
fd6dd6935b
vendor: github.com/containerd/containerd v1.7.6
The DeepEqual ignore required in the daemon tests is a bit ugly, but it
works given the new protoc output.

We also have to ignore lints related to schema1 deprecations; these do
not apply as we must continue to support this schema version.

Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-09-21 14:18:40 -06:00
Bjorn Neergaard
79a4cbbec9
builder-next: make stub executor generic
The current executor is only tested on Linux, so let's be honest about
that. Stubbing this correctly helps avoid incorrectly trying to call
into Linux-only code in e.g. libnetwork.

Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-09-21 14:18:27 -06:00
Sebastiaan van Stijn
06499c52e2
Merge pull request #46416 from thaJeztah/move_IsOSSupported
pkg/system: deprecate IsOSSupported() and ErrNotSupportedOperatingSystem, and implement image.CheckOS
2023-09-08 00:25:36 +02:00
Sebastiaan van Stijn
a3c97beee0
image: implement CheckOS, deprecate pkg/system IsOSSupported
Implement a function that returns an error to replace existing uses of
the IsOSSupported utility, where callers had to produce the error after
checking.

The IsOSSupported function was used in combination with images, so implementing
a utility in "image" to prevent having to import pkg/system (which contains many
unrelated functions)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-07 22:14:44 +02:00
Brian Goff
642e9917ff Add otel support
This uses otel standard environment variables to configure tracing in
the daemon.
It also adds support for propagating trace contexts in the client and
reading those from the API server.

See
https://opentelemetry.io/docs/specs/otel/configuration/sdk-environment-variables/
for details on otel environment variables.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2023-09-07 18:38:19 +00:00
Sebastiaan van Stijn
1148a24e64
migrate to new github.com/distribution/reference module
The "reference" package was moved to a separate module, which was extracted
from b9b19409cf

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-05 12:09:26 +02:00
Sebastiaan van Stijn
e9b09b49fd
builder/builder-next: format code with gofumpt
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-24 17:59:50 +02:00
Sebastiaan van Stijn
3553b4c684
replace dockerfile/dockerignore with patternmatcher/ignorefile
The BuildKit dockerignore package was integrated in the patternmatcher
repository / module. This patch updates our uses of the BuildKit package
with its new location.

A small local change was made to keep the format of the existing error message,
because the "ignorefile" package is slightly more agnostic in that respect
and doesn't include ".dockerignore" in the error message.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-23 00:53:18 +02:00
Sebastiaan van Stijn
5dc44f03f0
classic builder: align "removing intermediate container" output
This is something that stood out to me: removing the intermediate
container is part of a build step, but unlike the other output from
the build, wasn't indented (and prefixed with `--->`) to be shown
as part of the build.

This patch adds the `--->` prefix, to make it clearer what step the
removal was part of.

While at it, I also updated the message itself: this output is printed
_after_ the intermediate container has been removed, so we may as well
make it match reality, so I changed "removing" to "removed".

Before:

    echo -e 'FROM busybox\nRUN echo hello > /dev/null\nRUN echo world > /dev/null\n' | DOCKER_BUILDKIT=0 docker build --no-cache -
    Sending build context to Docker daemon  2.048kB
    Step 1/3 : FROM busybox
     ---> a416a98b71e2
    Step 2/3 : RUN echo hello > /dev/null
     ---> Running in a1a65b9365ac
    Removing intermediate container a1a65b9365ac
     ---> 8c6b57ebebdd
    Step 3/3 : RUN echo world > /dev/null
     ---> Running in 9fa977b763a5
    Removing intermediate container 9fa977b763a5
     ---> 795c1f2fc7b9
    Successfully built 795c1f2fc7b9

After:

    echo -e 'FROM busybox\nRUN echo hello > /dev/null\nRUN echo world > /dev/null\n' | DOCKER_BUILDKIT=0 docker build --no-cache -
    Sending build context to Docker daemon  2.048kB
    Step 1/3 : FROM busybox
     ---> fc9db2894f4e
    Step 2/3 : RUN echo hello > /dev/null
     ---> Running in 38d7c34c2178
     ---> Removed intermediate container 38d7c34c2178
     ---> 7c0dbc45111d
    Step 3/3 : RUN echo world > /dev/null
     ---> Running in 629620285d4c
     ---> Removed intermediate container 629620285d4c
     ---> b92f70f2e57d
    Successfully built b92f70f2e57d

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-18 21:27:56 +02:00
Sebastiaan van Stijn
64c6f72988
libnetwork: remove Network interface
There's only one implementation; drop the interface and use the
concrete type instead.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-22 11:56:41 +02:00
Jose Diaz-Gonzalez
079a9d4562 Sort unconsumed build arguments before usage
Golang map iteration order is not guaranteed, so in some cases the built slice has it's output of order as well. This means that testing for exact warning messages in docker build output would result in random test failures, making it more annoying for end-users to test against this functionality.

Signed-off-by: Jose Diaz-Gonzalez <email@josediazgonzalez.com>
2023-07-09 03:23:06 -04:00
Sebastiaan van Stijn
2f61620339
builder/builder-next: use string-literals for easier grep'ing
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-05 12:26:51 +02:00
Sebastiaan van Stijn
202907b14c
builder/dockerfile: use string-literals for easier grep'ing
Use string-literal for reduce escaped quotes, which makes for easier grepping.
While at it, also changed http -> https to keep some linters at bay.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-05 12:25:59 +02:00
Sebastiaan van Stijn
b9833a2059
builder: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:25:21 +02:00
Sebastiaan van Stijn
540d6539be
builder/remotecontext: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:25:21 +02:00
Sebastiaan van Stijn
3e0565e17c
builder/dockerfile: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:25:21 +02:00
Sebastiaan van Stijn
9d4abbbc30
builder/builder-next: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:25:21 +02:00
Brian Goff
74da6a6363 Switch all logging to use containerd log pkg
This unifies our logging and allows us to propagate logging and trace
contexts together.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2023-06-24 00:23:44 +00:00
CrazyMax
21e50b89c9
builder: pass host-gateway IP as worker label
We missed a case when parsing extra hosts from the dockerfile
frontend so the build fails.

To handle this case we need to set a dedicated worker label
that contains the host gateway IP so clients like Buildx
can just set the proper host:ip when parsing extra hosts
that contain the special string "host-gateway".

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-06-21 13:21:38 +02:00
Paweł Gronowski
d63569c73d
builder-next: Set moby exporter as default
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-06-02 15:38:53 +02:00
Sebastiaan van Stijn
2cd23ffeec
Merge pull request #45628 from thaJeztah/context_simplify
builder/remotecontext: remove mimeTypes struct, use consts
2023-05-30 13:40:42 +02:00
Sebastiaan van Stijn
44124ab6b0
builder/remotecontext: remove CachableSource, NewCachableSource
This type (as well as TarsumBackup), was used for the experimental --stream
support for the classic builder. This feature was removed in commit
6ca3ec88ae, which also removed uses of
the CachableSource type.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-29 16:35:42 +02:00
CrazyMax
fd72b134d5
update generated files
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-29 03:28:35 +02:00
Sebastiaan van Stijn
e410e27547
builder/remotecontext: remove mimeTypes struct, use consts
This struct was never modified; let's just use consts for these.

Also remove the args return from detectContentType(), as it was
not used anywhere.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-26 15:21:15 +02:00
Sebastiaan van Stijn
37d4b0bee9
builder/remotecontext: deprecate CachableSource, NewCachableSource
This type (as well as TarsumBackup), was used for the experimental --stream
support for the classic builder. This feature was removed in commit
6ca3ec88ae, which also removed uses of
the CachableSource type.

As far as I could find, there's no external consumers of these types,
but let's deprecated it, to give potential users a heads-up that it
will be removed.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-26 00:05:08 +02:00
Sebastiaan van Stijn
ab35df454d
remove pre-go1.17 build-tags
Removed pre-go1.17 build-tags with go fix;

    go mod init
    go fix -mod=readonly ./...
    rm go.mod

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-19 20:38:51 +02:00
Kevin Alvarez
6d139e5e95
build: use daemon id as worker id for the graph driver controller
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-18 21:17:29 +02:00
Akihiro Suda
9e6370819b
Merge pull request #45473 from zhangguanzhang/master-commentStr
[builder] make chownComment easy to read
2023-05-12 09:35:10 +09:00
Laura Brehm
e46674b6a7
c8d: implement classic builder
Co-authored-by: Djordje Lukic <djordje.lukic@docker.com>
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
2023-05-11 11:57:01 +01:00
Brian Goff
e58c267d66
Merge pull request #45304 from crazy-max/buildkit-version-buildinfo
Set BuildKit version using buildinfo
2023-05-10 16:58:18 -07:00
Sebastiaan van Stijn
3f7fde76c2
Merge pull request #45507 from corhere/cleanup-reexec-inits
[chore] clean up reexec.Init() calls
2023-05-10 11:49:11 +02:00
Cory Snider
4e0319c878 [chore] clean up reexec.Init() calls
Now that most uses of reexec have been replaced with non-reexec
solutions, most of the reexec.Init() calls peppered throughout the test
suites are unnecessary. Furthermore, most of the reexec.Init() calls in
test code neglects to check the return value to determine whether to
exit, which would result in the reexec'ed subprocesses proceeding to run
the tests, which would reexec another subprocess which would proceed to
run the tests, recursively. (That would explain why every reexec
callback used to unconditionally call os.Exit() instead of returning...)

Remove unneeded reexec.Init() calls from test and example code which no
longer needs it, and fix the reexec.Init() calls which are not inert to
exit after a reexec callback is invoked.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-05-09 19:13:17 -04:00
Jeyanthinath Muthuram
307b09e7eb
fixing consistent aliases for OCI spec imports
Signed-off-by: Jeyanthinath Muthuram <jeyanthinath10@gmail.com>
2023-05-08 15:27:52 +05:30
zhangguanzhang
43b1afb2cf make chownComment easy to read
Signed-off-by: zhangguanzhang <zhangguanzhang@qq.com>
2023-05-05 19:22:49 +08:00
Djordje Lukic
a1e1038736
Use the image service instead of the reference store for tagging
The image store sends events when a new image is created/tagged, using
it instead of the reference store makes sure we send the "tag" event
when a new image is built using buildx.

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-04-26 15:10:01 +02:00
CrazyMax
5400366b90
Set BuildKit version using buildinfo
This sets BuildKit version from the build information embedded
in running binary so we are aligned with the expected vendoring.

We iterate over all dependencies and find the BuildKit one
and set the right version. We also check if the module is
replaced and use it this case.

There is also additional checks if a pseudo version is
detected. See comments in code for more info.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-04-25 12:30:52 +02:00
Sebastiaan van Stijn
fd03a52468
Merge pull request #45294 from vvoland/builder-fix-prune-timestamp
builder-next/prune: Handle `until` filter timestamps
2023-04-25 11:19:27 +02:00
Paweł Gronowski
e502c69140
builder: use typed error for invalid "until" value
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-04-21 10:25:53 +02:00
Paweł Gronowski
54a125f677
builder-next/prune: Handle "until" filter timestamps
Fixes `docker system prune --filter until=<timestamp>`.
`docker system prune` claims to support "until" filter for timestamps,
but it doesn't work because builder "until" filter only supports
duration.
Use the same filter parsing logic and then convert the timestamp to a
relative "keep-duration" supported by buildkit.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-04-20 21:20:32 +02:00
Tonis Tiigi
38e791fb2f
vendor: update buildkit to v0.11.6
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2023-04-20 11:17:25 -07:00
Sebastiaan van Stijn
890959eea8
builder-next: fix buildkit version
Update the version manually (we don't have automation for this yet), and
add a comment to vendor.mod to help users remind to update it.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-14 10:44:27 +02:00
Sebastiaan van Stijn
81e62af94a
use consistent alias for containerd's errdefs package
The signatures of functions in containerd's errdefs packages are very
similar to those in our own, and it's easy to accidentally use the wrong
package.

This patch uses a consistent alias for all occurrences of this import.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-08 19:30:33 +02:00
Tonis Tiigi
0ac3bf837b
builder-next: disable mergeop and diffop
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
2023-03-16 08:23:22 -06:00
Paweł Gronowski
af677245fb
c8d/builder: Store untagged images as dangling
Set `dangling-name-prefix` exporter attribute  to `moby-dangling` which
makes it create an containerd image even when user didn't provide any
name for the new image.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-03-08 16:59:33 +01:00