Commit graph

36489 commits

Author SHA1 Message Date
Sebastiaan van Stijn
9c846b2fcc
Client: always call ensureReaderClosed
Unlike a plain `net/http/client.Do()`, requests made through client/request
use the `sendRequest` function, which parses the server response, and may
convert non-transport errors into errors (through `cli.checkResponseErr()`).

This means that we cannot assume that no reader was opened if an error is
returned.

This patch changes various locations where `ensureReaderClosed` was only
called in the non-error situation, and uses a `defer` to make sure it's
always called.

`ensureReaderClosed` itself already checks if the response's body was set,
so in situations where the error was due to a transport error, calling
`ensureReaderClosed` should be a no-op.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-03-18 15:26:21 +01:00
Sebastiaan van Stijn
38005cfc12
Merge pull request #38882 from tonistiigi/buildkit-update
builder-next: update buildkit to c3541087 (v0.4.0)
2019-03-16 23:03:45 +01:00
Brian Goff
50bfc8310c
Merge pull request #38886 from thaJeztah/fix_api_history
minor touch-up in api/version-history.md
2019-03-16 12:53:21 -07:00
Brian Goff
c579856835
Merge pull request #38404 from thaJeztah/bump_golang_1.12
Bump Golang to 1.12.1
2019-03-16 10:45:02 -07:00
Sebastiaan van Stijn
9d9d3fe4b4
minor touch-up in api/version-history.md
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-03-16 17:49:53 +01:00
Sebastiaan van Stijn
2925eb7a2a
Merge pull request #38777 from wk8/wk8/raw_cred_specs
Making it possible to pass Windows credential specs directly to the engine
2019-03-16 16:42:39 +01:00
Sebastiaan van Stijn
5635c2a498
Merge pull request #38689 from thaJeztah/add_errdefs_utils_take2
Make client return "rich" errors (take 2)
2019-03-16 15:15:13 +01:00
Jean Rouge
7fdac7eb0f Making it possible to pass Windows credential specs directly to the engine
Instead of having to go through files or registry values as is currently the
case.

While adding GMSA support to Kubernetes (https://github.com/kubernetes/kubernetes/pull/73726)
I stumbled upon the fact that Docker currently only allows passing Windows
credential specs through files or registry values, forcing the Kubelet
to perform a rather awkward dance of writing-then-deleting to either the
disk or the registry to be able to create a Windows container with cred
specs.

This patch solves this problem by making it possible to directly pass
whole base64-encoded cred specs to the engine's API. I took the opportunity
to slightly refactor the method responsible for Windows cred spec as it
seemed hard to read to me.

Added some unit tests on Windows credential specs handling, as there were
previously none.

Added/amended the relevant integration tests.

I have also tested it manually: given a Windows container using a cred spec
that you would normally start with e.g.
```powershell
docker run --rm --security-opt "credentialspec=file://win.json" mcr.microsoft.com/windows/servercore:ltsc2019 nltest /parentdomain
# output:
# my.ad.domain.com. (1)
# The command completed successfully
```
can now equivalently be started with
```powershell
$rawCredSpec = & cat 'C:\ProgramData\docker\credentialspecs\win.json'
$escaped = $rawCredSpec.Replace('"', '\"')
docker run --rm --security-opt "credentialspec=raw://$escaped" mcr.microsoft.com/windows/servercore:ltsc2019 nltest /parentdomain
# same output!
```

I'll do another PR on Swarmkit after this is merged to allow services to use
the same option.

(It's worth noting that @dperny faced the same problem adding GMSA support
to Swarmkit, to which he came up with an interesting solution - see
https://github.com/moby/moby/pull/38632 - but alas these tricks are not
available to the Kubelet.)

Signed-off-by: Jean Rouge <rougej+github@gmail.com>
2019-03-15 19:20:19 -07:00
Tonis Tiigi
8b0a1ca8a5 builder: fix platform passing in image adapter
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-03-15 19:07:41 -07:00
Sebastiaan van Stijn
818d0dc5fd
builder: alias containerd errdefs to prevent confusion
the containerd errdefs functions have the same name as the
docker errdefs, but their types use a different signature;
use an alias to prevent them from being mistaken for the
docker errdefs equivalents.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-03-16 00:42:50 +01:00
Sebastiaan van Stijn
0cafc84fb2
Use errdefs for handling errors in client
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-03-16 00:42:42 +01:00
Sebastiaan van Stijn
053c6f097a
Keep old "notfound" interface for backward compatibility
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-03-16 00:42:36 +01:00
Sebastiaan van Stijn
2a9c987e5a
Move httputils error helpers to errdefs package
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-03-16 00:42:23 +01:00
Sebastiaan van Stijn
ae875d4069
Update more tests to use new errors
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-03-16 00:41:02 +01:00
Sebastiaan van Stijn
161e0a90a6
Update tests to check returned errors
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-03-16 00:40:55 +01:00
Sebastiaan van Stijn
77c5668baf
client: remove special error handling for "no such image"
looks like we don't need this handling

Before this patch:

    Error: No such image: nosuchimage

After this patch:

    Error response from daemon: No such image: nosuchimage:latest
"

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-03-16 00:40:47 +01:00
Sebastiaan van Stijn
38e6d474af
client: return rich / errdefs errors
this patch makes the client return errors matching
the errdefs interface.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-03-16 00:40:39 +01:00
Sebastiaan van Stijn
5d8ece522b
Make client.notfound error match errdefs.notfound
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-03-16 00:40:26 +01:00
Sebastiaan van Stijn
421b66a486
Make client/errors helpers work with errdefs errors
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-03-16 00:40:07 +01:00
Sebastiaan van Stijn
1af30c50ca
Add httputils.FromStatusCode()
This utility allows a client to convert an API response
back to a typed error; allowing the client to perform
different actions based on the type of error, without
having to resort to string-matching the error.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-03-16 00:39:48 +01:00
Sebastiaan van Stijn
db22517fec
Bump Golang to 1.12.1
Also removed the `-stretch` suffix, because Debian Stretch
is the default base-image now, so there should be no need
to keep the suffix

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-03-15 21:39:15 +01:00
Yong Tang
87d593639c
Merge pull request #38860 from xichengliudui/fixtypos
Delete duplicate words
2019-03-15 10:01:54 -07:00
Tonis Tiigi
0289aa436c builder-next: support runtime platform detection
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-03-15 00:33:47 -07:00
Tonis Tiigi
b8385c98e8 builder-next: support for inline cache from local images
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-03-15 00:33:47 -07:00
Tonis Tiigi
ddaba80467 builder-next: fixes for inline cache support
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-03-15 00:33:46 -07:00
Tonis Tiigi
7ac4e33e63 builder-next: fixes for updated buildkit
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-03-15 00:33:46 -07:00
Tonis Tiigi
4337e97f25 vendor: update containerd to a15b6e2097
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-03-15 00:33:46 -07:00
Tonis Tiigi
586c964af2 vendor: update protobuf for new code compatibility
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-03-15 00:04:15 -07:00
Tonis Tiigi
5166013119 vendor: update buildkit and fsutil
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-03-15 00:04:15 -07:00
Brian Goff
05e7d000f2
Merge pull request #38873 from thaJeztah/update_containerd_1.2.5
Update containerd v1.2.5, runc 2b18fe1d885ee5083ef9f0838fee39b62d653e30
2019-03-14 19:12:41 -07:00
Sebastiaan van Stijn
ca0b64ee3b
Merge pull request #35621 from kolyshkin/ipc-private
daemon: use 'private' ipc mode by default
2019-03-14 19:27:30 +01:00
Tibor Vass
5b6771345a
Merge pull request #38875 from thaJeztah/revert_38518-docker_info_builder_version
Revert "Adding builder version"
2019-03-14 10:55:44 -07:00
xichengliudui
1d62807db3 Delete duplicate words
update pull request

Signed-off-by: “xichengliudui” <“liuduidui@beyondcent.com”>
2019-03-14 07:53:38 -04:00
Sebastiaan van Stijn
42987cab19
Merge pull request #38874 from thaJeztah/small_error_improvements
Minor error cleanups in projectquota
2019-03-14 09:58:08 +01:00
Sebastiaan van Stijn
aa51dcec94
Merge pull request #38868 from justincormack/google-uuid
Switch to google/uuid
2019-03-14 02:19:01 +01:00
Sebastiaan van Stijn
f73dd5fdad
Revert "Adding builder version"
This reverts commit f821f002e5.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-03-14 00:18:46 +01:00
Sebastiaan van Stijn
154d6c5207
Minor error cleanups in projectquota
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-03-13 23:39:38 +01:00
Sebastiaan van Stijn
42ad354e7a
Merge pull request #38870 from dmcgowan/quota-not-permitted-log
Update quota support to treat permission error as not supported
2019-03-13 23:38:37 +01:00
Sebastiaan van Stijn
386b06eacd
vendor containerd/cgroups dbea6f2bd41658b84b00417ceefa416b979cbf10
Relevant changes:

- containerd/containerd#51 Fix empty device type
- containerd/containerd#52 Remove call to unitName
  - Calling unitName incorrectly appends -slice onto the end of the slice cgroup we are looking for
  - addresses containerd/containerd#47 cgroups: cgroup deleted
- containerd/containerd#53 systemd-239+ no longer allows delegate slice
- containerd/containerd#54 Bugfix: can't write to cpuset cgroup
- containerd/containerd#63 Makes Load function more lenient on subsystems' checking
  - addresses containerd/containerd#58 Very strict checking of subsystems' existence while loading cgroup
- containerd/containerd#67 Add functionality for retrieving all tasks of a cgroup
- containerd/containerd#68 Fix net_prio typo
- containerd/containerd#69 Blkio weight/leafWeight pointer value
- containerd/containerd#77 Check for non-active/supported cgroups
  - addresses containerd/containerd#76 unable to find * in controller set: unknown
  - addresses docker/for-linux#545 Raspbian: Error response from daemon: unable to find "net_prio" in controller set: unknown
  - addresses docker/for-linux#552 Error response from daemon: unable to find "cpuacct" in controller set: unknown
  - addresses docker/for-linux#545 Raspbian: Error response from daemon: unable to find "net_prio" in controller set: unknown

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-03-13 21:39:49 +01:00
Sebastiaan van Stijn
69f7263795
vendor containerd client v1.2.5
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-03-13 21:22:13 +01:00
Sebastiaan van Stijn
79f5fbee01
Vendor runc 2b18fe1d885ee5083ef9f0838fee39b62d653e30
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-03-13 21:15:32 +01:00
Sebastiaan van Stijn
25cdae293f
Update containerd v1.2.5, runc 2b18fe1d885ee5083ef9f0838fee39b62d653e30
Notable Updates

- Fix an issue that non-existent parent directory in image layers is created with permission 0700. containerd#3017
- Fix an issue that snapshots of the base image can be deleted by mistake, when images built on top of it are deleted. containerd#3087
- Support for GC references to content from snapshot and container objects. containerd#3080
- cgroups updated to dbea6f2bd41658b84b00417ceefa416b97 to fix issues for systemd 420 and non-existent cgroups. containerd#3079
- runc updated to 2b18fe1d885ee5083ef9f0838fee39b62d653e30 to include the improved fix for CVE-2019-5736. containerd#3082
- cri: Fix a bug that pod can't get started when the same volume is defined differently in the image and the pod spec. cri#1059
- cri: Fix a bug that causes container start failure after in-place upgrade containerd to 1.2.4+ or 1.1.6+. cri#1082
- cri updated to a92c40017473cbe0239ce180125f12669757e44f. containerd#3084

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-03-13 21:00:50 +01:00
Derek McGowan
1217819f07
Update quota support to treat permission error as not supported
When initializing graphdrivers without root a permission warning
log is given due to lack of permission to create a device. This
error should be treated the same as quota not supported.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2019-03-13 11:22:13 -07:00
Justin Cormack
c435551ccc
Switch to google/uuid
pborman/uuid and google/uuid used to be different versions of
the same package, but now pborman/uuid is a compatibility wrapper
around google/uuid, maintained by the same person.

Clean up some of the usage as the functions differ slightly.

Not yet removed some uses of pborman/uuid in vendored code but
I have PRs in process for these.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2019-03-13 14:13:58 +00:00
Vincent Demeester
46036c2308
Merge pull request #37534 from thaJeztah/fix-distribution-500
Fix error 500 on distribution endpoint
2019-03-13 08:29:16 +01:00
Brian Goff
258edd715d
Merge pull request #38831 from thaJeztah/bump_swarmkit
bump swarmkit to 415dc72789e2b733ea884f09188c286ca187d8ec
2019-03-12 09:51:51 -07:00
Sebastiaan van Stijn
f58fa6e5c0
Merge pull request #38855 from thaJeztah/добро_пожаловать_Кир_как_сопровождающий
Add Kir as maintainer
2019-03-12 16:35:37 +01:00
Sebastiaan van Stijn
f196671db1
Add Kir as maintainer
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-03-12 13:36:55 +01:00
Kir Kolyshkin
596ca142e0 daemon: use 'private' ipc mode by default
This changes the default ipc mode of daemon/engine to be private,
meaning the containers will not have their /dev/shm bind-mounted
from the host by default. The benefits of doing this are:

 1. No leaked mounts. Eliminate a possibility to leak mounts into
    other namespaces (and therefore unfortunate errors like "Unable to
    remove filesystem for <ID>: remove /var/lib/docker/containers/<ID>/shm:
    device or resource busy").

 2. Working checkpoint/restore. Make `docker checkpoint`
    not lose the contents of `/dev/shm`, but save it to
    the dump, and be restored back upon `docker start --checkpoint`
    (currently it is lost -- while CRIU handles tmpfs mounts,
    the "shareable" mount is seen as external to container,
    and thus rightfully ignored).

3. Better security. Currently any container is opened to share
   its /dev/shm with any other container.

Obviously, this change will break the following usage scenario:

 $ docker run -d --name donor busybox top
 $ docker run --rm -it --ipc container:donor busybox sh
 Error response from daemon: linux spec namespaces: can't join IPC
 of container <ID>: non-shareable IPC (hint: use IpcMode:shareable
 for the donor container)

The soution, as hinted by the (amended) error message, is to
explicitly enable donor sharing by using --ipc shareable:

 $ docker run -d --name donor --ipc shareable busybox top

Compatibility notes:

1. This only applies to containers created _after_ this change.
   Existing containers are not affected and will work fine
   as their ipc mode is stored in HostConfig.

2. Old backward compatible behavior ("shareable" containers
   by default) can be enabled by either using
   `--default-ipc-mode shareable` daemon command line option,
   or by adding a `"default-ipc-mode": "shareable"`
   line in `/etc/docker/daemon.json` configuration file.

3. If an older client (API < 1.40) is used, a "shareable" container
   is created. A test to check that is added.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2019-03-09 18:57:42 -08:00
Kir Kolyshkin
ce7528ebdf postContainersCreate: minor nitpick
There are two if statements checking for exactly same conditions:

> if hostConfig != nil && versions.LessThan(version, "1.40")

Merge these.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2019-03-09 18:57:42 -08:00