Commit graph

7184 commits

Author SHA1 Message Date
Sebastiaan van Stijn
f337a8d21d
Do not require "experimental" for metrics API
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-04-20 22:19:00 +02:00
Akihiro Suda
55e6d7d36f
Merge pull request #37867 from mountkin/fix-ov2
enhance storage-opt validation logic in overlay2 driver
2020-04-19 23:02:19 +09:00
Akihiro Suda
f350b53241 cgroup2: implement docker info
ref: https://www.kernel.org/doc/html/latest/admin-guide/cgroup-v2.html

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-04-17 07:20:01 +09:00
Brian Goff
2200d938a2
Merge pull request #40796 from cpuguy83/log_reads_allocs
Reduce allocations for logfile reader
2020-04-16 12:09:41 -07:00
Sebastiaan van Stijn
c8e31dc2f2
Merge pull request #39882 from thaJeztah/swarm_pids_limit
Add API support for PidsLimit on services
2020-04-16 21:02:30 +02:00
Brian Goff
620bce847d
Merge pull request #40749 from DanielQujun/zombie_check_for_container
add zombie check for container when killing it
2020-04-16 12:01:58 -07:00
Sebastiaan van Stijn
54d88a7cd3
Merge pull request #40478 from cpuguy83/dont-prime-the-stats
Add stats options to not prime the stats
2020-04-16 20:57:06 +02:00
Sebastiaan van Stijn
157c53c8e0
Add API support for PidsLimit on services
Support for PidsLimit was added to SwarmKit in docker/swarmkit/pull/2415,
but never exposed through the Docker remove API.

This patch exposes the feature in the repote API.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-04-15 22:37:42 +02:00
Sebastiaan van Stijn
db669cd117
Merge pull request #40814 from tonistiigi/buildkit-update
vendor: update buildkit to ae7ff174
2020-04-15 21:18:51 +02:00
Akihiro Suda
2e5923c547
Merge pull request #39705 from thaJeztah/daemon_nits
daemon: various nits and small fixes
2020-04-15 09:11:25 +09:00
Tonis Tiigi
0cdf6ba9c8 vendor: update buildkit to ae7ff174
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-04-14 08:26:07 -07:00
Sebastiaan van Stijn
eb14d936bf
daemon: rename variables that collide with imported package names
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-04-14 17:22:23 +02:00
Sebastiaan van Stijn
797ec8e913
daemon: rename all receivers to "daemon"
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-04-14 17:22:21 +02:00
Sebastiaan van Stijn
5d040cbd16
daemon: fix capitalization of some functions
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-04-14 17:22:19 +02:00
Sebastiaan van Stijn
eeef12f469
daemon: address some minor linting issues and nits
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-04-14 17:22:17 +02:00
Brian Goff
ced91bee4b On startup, actually shutdown the container.
When a container is left running after the daemon exits (e.g. the daemon
is SIGKILL'd or crashes), it should stop any running containers when the
daemon starts back up.

What actually happens is the daemon only sends the container's
configured stop signal and does not check if it has exited.
If the container does not actually exit then it is left running.

This fixes this unexpected behavior by calling the same function to shut
down the container that the daemon shutdown process does.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2020-04-13 14:20:12 -07:00
Shijiang Wei
e9d785ce3f enhance storage-opt validation logic in overlay2 driver
Signed-off-by: Shijiang Wei <mountkin@gmail.com>
2020-04-12 23:22:30 +08:00
屈骏
f3c1eec99e add zombie check for container when killing it, alernative fix for #40735.
Signed-off-by: 屈骏 <qujun@tiduyun.com>
2020-04-10 16:46:31 +08:00
Brian Goff
933a87236f Reduce allocations for logfile reader
Before this change, the log decoder function provided by the log driver
to logfile would not be able to re-use buffers, causing undeeded
allocations and memory bloat for dockerd.

This change introduces an interface that allows the log driver to manage
it's memory usge more effectively.
This only affects json-file and local log drivers.

`json-file` still is not great just because of how the json decoder in the
stdlib works.
`local` is significantly improved.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2020-04-08 12:24:31 -07:00
Carlos de Paula
7ac638f86a Add support to riscv64 to the build scripts
Added riscv64 architecture support to the scripts used to build Docker
and it's dependencies.

Signed-off-by: Carlos de Paula <me@carlosedp.com>
2020-04-03 14:33:32 -03:00
Sebastiaan van Stijn
af0415257e
Merge pull request #40694 from kolyshkin/moby-sys-mount-part-II
switch to moby/sys/{mount,mountinfo} part II
2020-04-02 21:52:21 +02:00
Sebastiaan van Stijn
b6f07fad51
Merge pull request #40712 from vboulineau/bugfix-windows-user-cpu
Fix CPU Stat value UsageInUsermode on Windows
2020-04-02 21:37:57 +02:00
Akihiro Suda
3802830989 cgroup2: implement docker stats
The following fields are unsupported:

* BlkioStats: all fields other than IoServiceBytesRecursive
* CPUStats: CPUUsage.PercpuUsage
* MemoryStats: MaxUsage and Failcnt

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-04-02 17:51:34 +09:00
Kir Kolyshkin
5b658a0348 daemon.overlaySupportsSelinux: simplify check
1. Sscanf is very slow, and we don't use the first two fields -- get rid of it.

2. Since the field we search for is at the end of line and prepended by
   a space, we can just use strings.HaveSuffix.

3. Error checking for bufio.Scanner should be done after the Scan()
   loop, not inside it.

Fixes: 885b29df09
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2020-03-31 14:32:42 -07:00
vboulineau
ec16053ccf
Fix UsageInUsermode value on Windows
Looks like a wrong copy-paste using `RuntimeKernel100ns` twice instead of `RuntimeUser100ns`

Signed-off-by: Vincent Boulineau <vincent.boulineau@datadoghq.com>
2020-03-27 16:43:22 +01:00
Brian Goff
7a9cb29fb9 Accept platform spec on container create
This enables image lookup when creating a container to fail when the
reference exists but it is for the wrong platform. This prevents trying
to run an image for the wrong platform, as can be the case with, for
example binfmt_misc+qemu.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2020-03-20 16:10:36 -07:00
Kir Kolyshkin
39048cf656 Really switch to moby/sys/mount*
Switch to moby/sys/mount and mountinfo. Keep the pkg/mount for potential
outside users.

This commit was generated by the following bash script:

```
set -e -u -o pipefail

for file in $(git grep -l 'docker/docker/pkg/mount"' | grep -v ^pkg/mount); do
	sed -i -e 's#/docker/docker/pkg/mount"#/moby/sys/mount"#' \
		-e 's#mount\.\(GetMounts\|Mounted\|Info\|[A-Za-z]*Filter\)#mountinfo.\1#g' \
		$file
	goimports -w $file
done
```

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2020-03-20 09:46:25 -07:00
Brian Goff
714cba6740
Merge pull request #38788 from AkihiroSuda/bind-nonrecursive-swarm
service: support --mount type=bind,bind-nonrecursive
2020-03-13 15:42:46 -07:00
Sebastiaan van Stijn
1e078e1ac5
Merge pull request #40666 from AkihiroSuda/prohibit-rootless-systemd-cgroup1
daemon: fail early if rootless && cgroupdriver == "systemd" && cgroup v1
2020-03-13 22:29:04 +01:00
Akihiro Suda
745fa04e52 service: support --mount type=bind,bind-nonrecursive
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-03-13 19:01:28 +09:00
Akihiro Suda
f073ed4187
Merge pull request #39816 from tao12345666333/rm-systeminfo-error-handler
Remove `SystemInfo()` error handling.
2020-03-13 01:59:13 +09:00
Brian Goff
470d32d32a
Merge pull request #40483 from AkihiroSuda/fuse-overlayfs
new storage driver: fuse-overlayfs
2020-03-11 11:32:37 -07:00
Akihiro Suda
92e7f8f67c daemon: fail early if rootless && cgroupdriver == "systemd" && cgroup v1
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-03-11 12:49:03 +09:00
Jintao Zhang
18c22f5bc1 fix backingFs assignment
Signed-off-by: Jintao Zhang <zhangjintao9020@gmail.com>
2020-03-10 00:03:59 +08:00
Akihiro Suda
5ddbe511a1
Merge pull request #40614 from thaJeztah/more_deprecating
Add warnings for deprecated "cluster" functions, and deprecate API fields
2020-03-06 16:35:42 +09:00
Sebastiaan van Stijn
616e64b42f
API: deprecate /info "ClusterStore" and "ClusterAdvertise" fields
These fields will now be omitted when empty.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-03-03 18:10:47 +01:00
Sebastiaan van Stijn
a5538c06f9
Add warning about deprecated "cluster" options to "docker info"
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-03-03 18:10:31 +01:00
Akihiro Suda
9a82a9a8ea vendor containerd, BuildKit, protobuf, grpc, and golang.org/x
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-03-03 10:25:20 +09:00
Brian Goff
76e3a49933
Merge pull request #40486 from AkihiroSuda/rootless-cgroup2-systemd
rootless: support `--exec-opt native.cgroupdriver=systemd`
2020-03-02 16:11:21 -08:00
Brian Goff
ce1ceeb257 Add stats options to not prime the stats
Metrics collectors generally don't need the daemon to prime the stats
with something to compare since they already have something to compare
with.
Before this change, the API does 2 collection cycles (which takes
roughly 2s) in order to provide comparison for CPU usage over 1s. This
was primarily added so that `docker stats --no-stream` had something to
compare against.

Really the CLI should have just made a 2nd call and done the comparison
itself rather than forcing it on all API consumers.
That ship has long sailed, though.

With this change, clients can set an option to just pull a single stat,
which is *at least* a full second faster:

Old:
```
time curl --unix-socket
/go/src/github.com/docker/docker/bundles/test-integration-shell/docker.sock
http://./containers/test/stats?stream=false\&one-shot=false > /dev/null
2>&1

real0m1.864s
user0m0.005s
sys0m0.007s

time curl --unix-socket
/go/src/github.com/docker/docker/bundles/test-integration-shell/docker.sock
http://./containers/test/stats?stream=false\&one-shot=false > /dev/null
2>&1

real0m1.173s
user0m0.010s
sys0m0.006s
```

New:
```
time curl --unix-socket
/go/src/github.com/docker/docker/bundles/test-integration-shell/docker.sock
http://./containers/test/stats?stream=false\&one-shot=true > /dev/null
2>&1
real0m0.680s
user0m0.008s
sys0m0.004s

time curl --unix-socket
/go/src/github.com/docker/docker/bundles/test-integration-shell/docker.sock
http://./containers/test/stats?stream=false\&one-shot=true > /dev/null
2>&1

real0m0.156s
user0m0.007s
sys0m0.007s
```

This fixes issues with downstreams ability to use the stats API to
collect metrics.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2020-02-28 09:54:37 -08:00
Brian Goff
40b2b4b083
Merge pull request #40594 from sfzhu93/Mis_Unlock
daemon/cluster: add a missing Unlock
2020-02-28 09:47:53 -08:00
Sebastiaan van Stijn
39679991f4
Merge pull request #40543 from SamWhited/upstream_logging
Upstream logging changes from Enterprise Edition
2020-02-27 13:54:14 +01:00
Ziheng Liu
83c0bedba9 daemon/cluster: add a missing Unlock
Signed-off-by: Ziheng Liu <lzhfromustc@gmail.com>
2020-02-25 13:55:11 -05:00
Brian Goff
62bd5a33f7
Merge pull request #40137 from fuweid/me-wait-for-remote-containerd-before-reload
daemon: add grpc.WithBlock option
2020-02-21 10:11:10 -08:00
Akihiro Suda
0c6b85717b
Merge pull request #40481 from cpuguy83/stats_use_cond_var
Use condition variable to wake stats collector.
2020-02-21 13:37:14 +09:00
Brian Goff
750f0d1648 Support configuration of log cacher.
Configuration over the API per container is intentionally left out for
the time being, but is supported to configure the default from the
daemon config.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit cbecf48bc352e680a5390a7ca9cff53098cd16d7)
Signed-off-by: Madhu Venugopal <madhu@docker.com>
2020-02-19 17:02:34 -05:00
Brian Goff
e2ceb83a53 Support reads for all log drivers.
This supplements any log driver which does not support reads with a
custom read implementation that uses a local file cache.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit d675e2bf2b75865915c7a4552e00802feeb0847f)
Signed-off-by: Madhu Venugopal <madhu@docker.com>
2020-02-19 17:01:44 -05:00
Akihiro Suda
ca4b51868a rootless: support --exec-opt native.cgroupdriver=systemd
Support cgroup as in Rootless Podman.

Requires cgroup v2 host with crun.
Tested with Ubuntu 19.10 (kernel 5.3, systemd 242), crun v0.12.1.

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-02-14 15:32:31 +09:00
Sebastiaan van Stijn
3af8d484b1
Merge pull request #40394 from tao12345666333/reserved-namespace-labels
enforce reserve internal labels.
2020-02-13 01:47:05 +01:00
Sebastiaan van Stijn
58c2615208
Merge pull request #40497 from arkodg/fix-bip-subnet-config
Set the bip network value as the subnet
2020-02-12 12:41:29 +01:00