Commit graph

943 commits

Author SHA1 Message Date
Tonis Tiigi
07b3aac902 builder-next: userns remap support
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-06-10 21:49:17 -07:00
Tonis Tiigi
85bbbd4495 vendor: update buildkit to 37d53758
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-06-04 00:34:16 -07:00
Akihiro Suda
b4247b433e builder-next: support DOCKER_RAMDISK
For https://github.com/kubernetes/minikube/issues/4143

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2019-05-22 00:39:03 +09:00
Tonis Tiigi
91a57f3e7f builder-next: call stopprogress on download error
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-05-13 17:01:13 -07:00
Tibor Vass
df43bc329e
Merge pull request #39152 from thaJeztah/bump_buildkit
Bump buildkit v0.5.0
2019-05-10 16:22:48 -07:00
Tonis Tiigi
cfce0acd33 builder-next: fix gcr workaround token cache
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-05-07 10:33:04 -07:00
Sebastiaan van Stijn
3e4723cf33
bump buildkit v0.5.0
full diff: 8818c67cff...v0.5.0

- moby/buildkit#909 exporter: support unpack opt for image exporter
- moby/buildkit#961 dockerfile: allow subdirs for remote contexts

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-04-27 16:04:21 -07:00
Tonis Tiigi
a3fdd5bf7e vendor: update buildkit to 8818c67c
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-04-23 19:02:06 -07:00
Sebastiaan van Stijn
c334a87aec
Remove deprecated filter functions
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-04-08 15:46:45 +02:00
Tonis Tiigi
b6aae9919f vendor: update buildkit to b3028967a
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-04-03 01:01:01 -07:00
Tibor Vass
05c5d20a2c grpc: register BuildKit controller to /grpc
Signed-off-by: Tibor Vass <tibor@docker.com>
2019-04-02 19:57:59 +00:00
Tonis Tiigi
a588898f99 gitutils: add validation for ref
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit 723b107ca4fba14580a6cd971e63d8af2e7d2bbe)
Signed-off-by: Andrew Hsu <andrewhsu@docker.com>
2019-03-26 22:05:46 +00:00
Tibor Vass
29de017df7
Merge pull request #38599 from thaJeztah/builder_fix_copy_permissions
builder: fix `COPY --from` should preserve ownership
2019-03-22 09:38:13 -07:00
Akihiro Suda
7c197c18d3
Merge pull request #38246 from tonistiigi/gcr-workaround
builder: add workaround for gcr auth issue
2019-03-22 15:55:55 +09:00
Derek McGowan
2447f67c52
Merge pull request #38897 from tonistiigi/schema1-fix
builder-next: fix second cachekey for schema1
2019-03-21 10:59:32 -07:00
Tonis Tiigi
e96d1c3754 vendor: update buildkit to e9aca5be
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-03-20 22:01:23 -07:00
John Howard
a3eda72f71
Merge pull request #38541 from Microsoft/jjh/containerd
Windows: Experimental: ContainerD runtime
2019-03-19 21:09:19 -07:00
Tonis Tiigi
768c6d7b29 builder-next: allow outputs configuration
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-03-19 10:28:30 -07:00
Tonis Tiigi
bcd8298c35 builder: add workaround for gcr auth issue
GCR does not currently support 401 response from blob endpoints.
This detects the case where no manifest requests have been
performed for the current resolver and does a dummy request
to enable authorization.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-03-18 19:19:15 -07:00
Tonis Tiigi
0037da0230 builder-next: fix second cachekey for schema1
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-03-18 14:03:01 -07:00
Sebastiaan van Stijn
38005cfc12
Merge pull request #38882 from tonistiigi/buildkit-update
builder-next: update buildkit to c3541087 (v0.4.0)
2019-03-16 23:03:45 +01:00
Tonis Tiigi
8b0a1ca8a5 builder: fix platform passing in image adapter
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-03-15 19:07:41 -07:00
Sebastiaan van Stijn
818d0dc5fd
builder: alias containerd errdefs to prevent confusion
the containerd errdefs functions have the same name as the
docker errdefs, but their types use a different signature;
use an alias to prevent them from being mistaken for the
docker errdefs equivalents.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-03-16 00:42:50 +01:00
Tonis Tiigi
0289aa436c builder-next: support runtime platform detection
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-03-15 00:33:47 -07:00
Tonis Tiigi
b8385c98e8 builder-next: support for inline cache from local images
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-03-15 00:33:47 -07:00
Tonis Tiigi
ddaba80467 builder-next: fixes for inline cache support
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-03-15 00:33:46 -07:00
Tonis Tiigi
7ac4e33e63 builder-next: fixes for updated buildkit
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-03-15 00:33:46 -07:00
Tonis Tiigi
5166013119 vendor: update buildkit and fsutil
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-03-15 00:04:15 -07:00
John Howard
20833b06a0 Windows: (WCOW) Generate OCI spec that remote runtime can escape
Signed-off-by: John Howard <jhoward@microsoft.com>

Also fixes https://github.com/moby/moby/issues/22874

This commit is a pre-requisite to moving moby/moby on Windows to using
Containerd for its runtime.

The reason for this is that the interface between moby and containerd
for the runtime is an OCI spec which must be unambigious.

It is the responsibility of the runtime (runhcs in the case of
containerd on Windows) to ensure that arguments are escaped prior
to calling into HCS and onwards to the Win32 CreateProcess call.

Previously, the builder was always escaping arguments which has
led to several bugs in moby. Because the local runtime in
libcontainerd had context of whether or not arguments were escaped,
it was possible to hack around in daemon/oci_windows.go with
knowledge of the context of the call (from builder or not).

With a remote runtime, this is not possible as there's rightly
no context of the caller passed across in the OCI spec. Put another
way, as I put above, the OCI spec must be unambigious.

The other previous limitation (which leads to various subtle bugs)
is that moby is coded entirely from a Linux-centric point of view.

Unfortunately, Windows != Linux. Windows CreateProcess uses a
command line, not an array of arguments. And it has very specific
rules about how to escape a command line. Some interesting reading
links about this are:

https://blogs.msdn.microsoft.com/twistylittlepassagesallalike/2011/04/23/everyone-quotes-command-line-arguments-the-wrong-way/
https://stackoverflow.com/questions/31838469/how-do-i-convert-argv-to-lpcommandline-parameter-of-createprocess
https://docs.microsoft.com/en-us/cpp/cpp/parsing-cpp-command-line-arguments?view=vs-2017

For this reason, the OCI spec has recently been updated to cater
for more natural syntax by including a CommandLine option in
Process.

What does this commit do?

Primary objective is to ensure that the built OCI spec is unambigious.

It changes the builder so that `ArgsEscaped` as commited in a
layer is only controlled by the use of CMD or ENTRYPOINT.

Subsequently, when calling in to create a container from the builder,
if follows a different path to both `docker run` and `docker create`
using the added `ContainerCreateIgnoreImagesArgsEscaped`. This allows
a RUN from the builder to control how to escape in the OCI spec.

It changes the builder so that when shell form is used for RUN,
CMD or ENTRYPOINT, it builds (for WCOW) a more natural command line
using the original as put by the user in the dockerfile, not
the parsed version as a set of args which loses fidelity.
This command line is put into args[0] and `ArgsEscaped` is set
to true for CMD or ENTRYPOINT. A RUN statement does not commit
`ArgsEscaped` to the commited layer regardless or whether shell
or exec form were used.
2019-03-12 18:41:55 -07:00
Sebastiaan van Stijn
6d87f19142
builder: fix COPY --from should preserve ownership
When copying between stages, or copying from an image,
ownership of the copied files should not be changed, unless
the `--chown` option is set (in which case ownership of copied
files should be updated to the specified user/group).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-03-13 00:55:04 +01:00
Tonis Tiigi
f9b9d5f584 builder-next: fixes for rootless mode
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-02-28 10:44:21 -08:00
linuxmercedes
64466b0cd9 Convert parse errors to more informative format
- Wrap parse errors in errdefs.InvalidParameters
- Include dockerfile in error names

Signed-off-by: Natasha Jarus <linuxmercedes@gmail.com>
2019-01-14 20:01:00 -06:00
Tonis Tiigi
bcf1967dd0 builder: delete sandbox in a goroutine for performance
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-11-28 22:55:39 -08:00
Tonis Tiigi
1ad272c7e4 builder: set exernalkey option for faster hook processing
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-11-28 22:28:57 -08:00
Sebastiaan van Stijn
2483e03531
Merge pull request #38238 from tiborvass/fix-38208
builder: deprecate prune filter `unused-for` in favor of `until`
2018-11-21 23:36:14 +01:00
Tibor Vass
62923f29f5 builder: ignore label and label! prune filters
Signed-off-by: Tibor Vass <tibor@docker.com>
2018-11-21 01:39:53 +00:00
Tibor Vass
369da264ba builder: deprecate prune filter unused-for in favor of until
This is to keep the UX consistent. `unused-for` is still accepted and a synonym.

Signed-off-by: Tibor Vass <tibor@docker.com>
2018-11-21 01:39:53 +00:00
Tonis Tiigi
effb2bd9d2 builder: avoid unset credentials in containerd
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-11-19 17:40:08 -08:00
Tibor Vass
48620057be builder: fix bugs when pruning buildkit cache with filters
Only the filters the user specified should be added as cache filters to buildkit.
Make an AND operation of the provided filters.
ID filter now does prefix-matching.

Signed-off-by: Tibor Vass <tibor@docker.com>
2018-11-05 23:15:35 +00:00
Tonis Tiigi
2732fe527f builder: fix duplicate mount release
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-10-22 17:24:20 -07:00
Tonis Tiigi
c693d45acf builder: fix private pulls on buildkit
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-10-17 17:45:44 -07:00
Akihiro Suda
b5ed4ebe06
Merge pull request #36537 from Microsoft/jjh/lcow-log-stderr
LCOW: Log stderr on failures to ease diagnosis
2018-10-06 11:05:55 +09:00
Akihiro Suda
837b9c6214 bump up buildkit
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-10-05 18:33:03 +09:00
John Howard
63f9c7784b LCOW: Log stderr on failures
Signed-off-by: John Howard <jhoward@microsoft.com>
2018-09-26 13:23:04 -07:00
Tibor Vass
34eede0296 Remove 'docker-' prefix for containerd and runc binaries
This allows to run the daemon in environments that have upstream containerd installed.

Signed-off-by: Tibor Vass <tibor@docker.com>
2018-09-24 21:49:03 +00:00
Yong Tang
fb7883e980
Merge pull request #37846 from tiborvass/buildkit-gc
builder: use buildkit's GC for build cache
2018-09-22 10:00:06 -07:00
Tibor Vass
4a776d0ca7 builder: use buildkit's GC for build cache
This allows users to configure the buildkit GC.

The following enables the default GC:
```
{
  "builder": {
    "gc": {
      "enabled": true
    }
  }
}
```

The default GC policy has a simple config:
```
{
  "builder": {
    "gc": {
      "enabled": true,
      "defaultKeepStorage": "30GB"
    }
  }
}
```

A custom GC policy can be used instead by specifying a list of cache prune rules:
```
{
  "builder": {
    "gc": {
      "enabled": true,
      "policy": [
        {"keepStorage": "512MB", "filter": ["unused-for=1400h"]]},
        {"keepStorage": "30GB", "all": true}
      ]
    }
  }
}
```

Signed-off-by: Tibor Vass <tibor@docker.com>
2018-09-21 22:06:00 +00:00
Tibor Vass
d0f00bc1fb builder: vendor buildkit to 39404586a50d1b9d0fb1c578cf0f4de7bdb7afe5
Signed-off-by: Tibor Vass <tibor@docker.com>
2018-09-21 20:43:26 +00:00
Anda Xu
171d51c861 add support of registry-mirrors and insecure-registries to buildkit
Signed-off-by: Anda Xu <anda.xu@docker.com>
2018-09-20 11:53:02 -07:00
Akihiro Suda
5adee401d1
Merge pull request #37843 from Microsoft/jjh/boltdb
Revendor libnetwork @ 36d3bed0e9
2018-09-14 23:22:36 +09:00