Commit graph

960 commits

Author SHA1 Message Date
Grant Millar
d3d724e45a
daemon: Use short libnetwork ID in exec-root & update libnetwork
also updates libnetwork to d9a6682a4dbb13b1f0d8216c425fe9ae010a0f23
full diff:

3eb39382bf...d9a6682a4d

- docker/libnetwork#2482 [19.03 backport] Shorten controller ID in exec-root to not hit UNIX_PATH_MAX
- docker/libnetwork#2483 [19.03 backport] Fix panic in drivers/overlay/encryption.go

Signed-off-by: Grant Millar <rid@cylo.io>
(cherry picked from commit df7b8f458a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-11-21 14:30:33 +01:00
Andrew Hsu
370def6b30
Merge pull request #412 from thaJeztah/19.03_backport_builder_entitilement_confg
[19.03 backport] builder entitlements configuration added.
2019-10-28 10:53:19 -07:00
Tibor Vass
1e26b431c9
daemon/config: fix filter type in BuildKit GC config
For backwards compatibility, the old incorrect object format for
builder.GC.Rule.Filter still works but is deprecated in favor of array of
strings akin to what needs to be passed on the CLI.

Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit fbdd437d29)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-22 10:28:09 +02:00
Kunal Kushwaha
ce74774c09
builder entitlements configutation added.
buildkit supports entitlements like network-host and security-insecure.
this patch aims to make it configurable through daemon.json file.
by default network-host is enabled & secuirty-insecure is disabled.

Signed-off-by: Kunal Kushwaha <kunal.kushwaha@gmail.com>
(cherry picked from commit 8b7bbf180f)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-22 10:06:46 +02:00
Sebastiaan van Stijn
45f49fe5c3
TestDispatch: refactor to use subtests again, and fix linting (structcheck)
Instead of using a `initDispatchTestCases()` function, declare the test-table
inside `TestDispatch` itself, and run the tests as subtests.

```
[2019-08-27T15:14:51.072Z] builder/dockerfile/evaluator_test.go:18:2: `name` is unused (structcheck)
[2019-08-27T15:14:51.072Z] 	name, expectedError string
```

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit a3f9cb5b63)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-25 21:35:56 +02:00
Tonis Tiigi
776c2bd113
atomic: patch 64bit alignment on 32bit systems
causes panic on armv7

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit af2e82d054)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-16 15:32:14 +02:00
Tonis Tiigi
52cef4bbee builder-next: update mount signature
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit d495eeb365)
Signed-off-by: Tibor Vass <tibor@docker.com>
2019-08-22 18:10:43 +00:00
Tonis Tiigi
613a32482f builder-next: close progress on layer export error
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit 27f1f2b5be)
Signed-off-by: Tibor Vass <tibor@docker.com>
2019-08-22 18:08:51 +00:00
Tonis Tiigi
cad2cd71b7 builder-next: avoid double unmounting mountable
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit 9ea2cf320a)
2019-08-15 22:45:08 -07:00
Andrew Hsu
305b2416ea
Merge pull request #316 from thaJeztah/19.03_backport_buildkit_userns_remap_take2
[19.03 backport] builder-next: userns remap support and honor daemon's DNS config
2019-08-08 13:35:34 -07:00
Tonis Tiigi
b0ef7422b0
vendor: update buildkit to f5a55a95
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit c60e53a274)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-08-08 19:01:44 +02:00
Tibor Vass
dd85af0e12
build: buildkit now honors daemon's DNS config
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit a1cdd4bfcc)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-08-08 18:43:00 +02:00
Tonis Tiigi
3bbf7b0d4d
builder-next: reset identitymapping if empty
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit 0bdcc60c4c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-08-08 18:42:53 +02:00
Tonis Tiigi
47517880ec
builder-next: userns remap support
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit 07b3aac902)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-08-08 18:42:42 +02:00
Sebastiaan van Stijn
7b0cf8b16d
Revert "vendor: update buildkit to f5a55a95"
This reverts commit eaa83640fa.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-08-08 18:32:27 +02:00
Sebastiaan van Stijn
ff0a0e364b
Builder: fix "COPY --from" to non-existing directory on Windows
This fixes a regression introduced in 6d87f19142,
causing `COPY --from` to fail if the target directory does not exist:

```
FROM mcr.microsoft.com/windows/servercore:ltsc2019 as s1
RUN echo "Hello World" > /hello

FROM mcr.microsoft.com/windows/servercore:ltsc2019
COPY --from=s1 /hello /hello/another/world
```

Would produce an error:

```
Step 4/4 : COPY --from=s1 /hello /hello/another/world
failed to copy files: mkdir \\?: The filename, directory name, or volume label syntax is incorrect.
```

The cause for this was that Go's `os.MkdirAll()` does not support/detect volume GUID paths
(`\\?\Volume{dae8d3ac-b9a1-11e9-88eb-e8554b2ba1db}\hello\another}`), and as a result
attempted to create the volume as a directory (`\\?`), causing it to fail.

This patch replaces `os.MkdirAll()` with our own `system.MkdirAll()` function, which
is capable of detecting GUID volumes.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 5858a99267)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-08-08 17:05:11 +02:00
Tonis Tiigi
6efcd74c6b builder-next: ensure timestamps set for metadata commands
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit 1a2bd3cf7d)
2019-08-07 14:03:11 -07:00
Tonis Tiigi
eaa83640fa vendor: update buildkit to f5a55a95
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit c60e53a274)
2019-08-07 14:02:45 -07:00
Tonis Tiigi
2de4afdee5
vendor: update buildkit to 37d53758
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit 85bbbd4495)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-06-04 11:17:10 +02:00
Akihiro Suda
1c346f16a3 builder-next: support DOCKER_RAMDISK
For https://github.com/kubernetes/minikube/issues/4143

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit b4247b433e)
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2019-05-22 06:58:44 +09:00
Sebastiaan van Stijn
bff7e300e6
Merge pull request #215 from thaJeztah/19.03_backport_buildkit_fixes
[19.03 backport] BuildKit fixes
2019-05-13 20:16:34 -07:00
Tonis Tiigi
4784740273
builder-next: call stopprogress on download error
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit 91a57f3e7f)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-05-13 17:44:38 -07:00
Sebastiaan van Stijn
ea09008423
bump buildkit v0.5.0
full diff: 8818c67cff...v0.5.0

- moby/buildkit#909 exporter: support unpack opt for image exporter
- moby/buildkit#961 dockerfile: allow subdirs for remote contexts

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 3e4723cf33)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-05-13 17:25:19 -07:00
Tonis Tiigi
c47f2a4a1a
builder-next: fix gcr workaround token cache
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit cfce0acd33)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-05-13 15:45:27 -07:00
Tonis Tiigi
a3fdd5bf7e vendor: update buildkit to 8818c67c
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-04-23 19:02:06 -07:00
Sebastiaan van Stijn
c334a87aec
Remove deprecated filter functions
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-04-08 15:46:45 +02:00
Tonis Tiigi
b6aae9919f vendor: update buildkit to b3028967a
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-04-03 01:01:01 -07:00
Tibor Vass
05c5d20a2c grpc: register BuildKit controller to /grpc
Signed-off-by: Tibor Vass <tibor@docker.com>
2019-04-02 19:57:59 +00:00
Tonis Tiigi
a588898f99 gitutils: add validation for ref
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit 723b107ca4fba14580a6cd971e63d8af2e7d2bbe)
Signed-off-by: Andrew Hsu <andrewhsu@docker.com>
2019-03-26 22:05:46 +00:00
Tibor Vass
29de017df7
Merge pull request #38599 from thaJeztah/builder_fix_copy_permissions
builder: fix `COPY --from` should preserve ownership
2019-03-22 09:38:13 -07:00
Akihiro Suda
7c197c18d3
Merge pull request #38246 from tonistiigi/gcr-workaround
builder: add workaround for gcr auth issue
2019-03-22 15:55:55 +09:00
Derek McGowan
2447f67c52
Merge pull request #38897 from tonistiigi/schema1-fix
builder-next: fix second cachekey for schema1
2019-03-21 10:59:32 -07:00
Tonis Tiigi
e96d1c3754 vendor: update buildkit to e9aca5be
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-03-20 22:01:23 -07:00
John Howard
a3eda72f71
Merge pull request #38541 from Microsoft/jjh/containerd
Windows: Experimental: ContainerD runtime
2019-03-19 21:09:19 -07:00
Tonis Tiigi
768c6d7b29 builder-next: allow outputs configuration
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-03-19 10:28:30 -07:00
Tonis Tiigi
bcd8298c35 builder: add workaround for gcr auth issue
GCR does not currently support 401 response from blob endpoints.
This detects the case where no manifest requests have been
performed for the current resolver and does a dummy request
to enable authorization.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-03-18 19:19:15 -07:00
Tonis Tiigi
0037da0230 builder-next: fix second cachekey for schema1
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-03-18 14:03:01 -07:00
Sebastiaan van Stijn
38005cfc12
Merge pull request #38882 from tonistiigi/buildkit-update
builder-next: update buildkit to c3541087 (v0.4.0)
2019-03-16 23:03:45 +01:00
Tonis Tiigi
8b0a1ca8a5 builder: fix platform passing in image adapter
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-03-15 19:07:41 -07:00
Sebastiaan van Stijn
818d0dc5fd
builder: alias containerd errdefs to prevent confusion
the containerd errdefs functions have the same name as the
docker errdefs, but their types use a different signature;
use an alias to prevent them from being mistaken for the
docker errdefs equivalents.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-03-16 00:42:50 +01:00
Tonis Tiigi
0289aa436c builder-next: support runtime platform detection
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-03-15 00:33:47 -07:00
Tonis Tiigi
b8385c98e8 builder-next: support for inline cache from local images
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-03-15 00:33:47 -07:00
Tonis Tiigi
ddaba80467 builder-next: fixes for inline cache support
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-03-15 00:33:46 -07:00
Tonis Tiigi
7ac4e33e63 builder-next: fixes for updated buildkit
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-03-15 00:33:46 -07:00
Tonis Tiigi
5166013119 vendor: update buildkit and fsutil
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-03-15 00:04:15 -07:00
John Howard
20833b06a0 Windows: (WCOW) Generate OCI spec that remote runtime can escape
Signed-off-by: John Howard <jhoward@microsoft.com>

Also fixes https://github.com/moby/moby/issues/22874

This commit is a pre-requisite to moving moby/moby on Windows to using
Containerd for its runtime.

The reason for this is that the interface between moby and containerd
for the runtime is an OCI spec which must be unambigious.

It is the responsibility of the runtime (runhcs in the case of
containerd on Windows) to ensure that arguments are escaped prior
to calling into HCS and onwards to the Win32 CreateProcess call.

Previously, the builder was always escaping arguments which has
led to several bugs in moby. Because the local runtime in
libcontainerd had context of whether or not arguments were escaped,
it was possible to hack around in daemon/oci_windows.go with
knowledge of the context of the call (from builder or not).

With a remote runtime, this is not possible as there's rightly
no context of the caller passed across in the OCI spec. Put another
way, as I put above, the OCI spec must be unambigious.

The other previous limitation (which leads to various subtle bugs)
is that moby is coded entirely from a Linux-centric point of view.

Unfortunately, Windows != Linux. Windows CreateProcess uses a
command line, not an array of arguments. And it has very specific
rules about how to escape a command line. Some interesting reading
links about this are:

https://blogs.msdn.microsoft.com/twistylittlepassagesallalike/2011/04/23/everyone-quotes-command-line-arguments-the-wrong-way/
https://stackoverflow.com/questions/31838469/how-do-i-convert-argv-to-lpcommandline-parameter-of-createprocess
https://docs.microsoft.com/en-us/cpp/cpp/parsing-cpp-command-line-arguments?view=vs-2017

For this reason, the OCI spec has recently been updated to cater
for more natural syntax by including a CommandLine option in
Process.

What does this commit do?

Primary objective is to ensure that the built OCI spec is unambigious.

It changes the builder so that `ArgsEscaped` as commited in a
layer is only controlled by the use of CMD or ENTRYPOINT.

Subsequently, when calling in to create a container from the builder,
if follows a different path to both `docker run` and `docker create`
using the added `ContainerCreateIgnoreImagesArgsEscaped`. This allows
a RUN from the builder to control how to escape in the OCI spec.

It changes the builder so that when shell form is used for RUN,
CMD or ENTRYPOINT, it builds (for WCOW) a more natural command line
using the original as put by the user in the dockerfile, not
the parsed version as a set of args which loses fidelity.
This command line is put into args[0] and `ArgsEscaped` is set
to true for CMD or ENTRYPOINT. A RUN statement does not commit
`ArgsEscaped` to the commited layer regardless or whether shell
or exec form were used.
2019-03-12 18:41:55 -07:00
Sebastiaan van Stijn
6d87f19142
builder: fix COPY --from should preserve ownership
When copying between stages, or copying from an image,
ownership of the copied files should not be changed, unless
the `--chown` option is set (in which case ownership of copied
files should be updated to the specified user/group).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-03-13 00:55:04 +01:00
Tonis Tiigi
f9b9d5f584 builder-next: fixes for rootless mode
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-02-28 10:44:21 -08:00
linuxmercedes
64466b0cd9 Convert parse errors to more informative format
- Wrap parse errors in errdefs.InvalidParameters
- Include dockerfile in error names

Signed-off-by: Natasha Jarus <linuxmercedes@gmail.com>
2019-01-14 20:01:00 -06:00
Tonis Tiigi
bcf1967dd0 builder: delete sandbox in a goroutine for performance
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-11-28 22:55:39 -08:00