Commit graph

6513 commits

Author SHA1 Message Date
Sebastiaan van Stijn
eeef12f469
daemon: address some minor linting issues and nits
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-04-14 17:22:17 +02:00
Carlos de Paula
7ac638f86a Add support to riscv64 to the build scripts
Added riscv64 architecture support to the scripts used to build Docker
and it's dependencies.

Signed-off-by: Carlos de Paula <me@carlosedp.com>
2020-04-03 14:33:32 -03:00
Sebastiaan van Stijn
af0415257e
Merge pull request #40694 from kolyshkin/moby-sys-mount-part-II
switch to moby/sys/{mount,mountinfo} part II
2020-04-02 21:52:21 +02:00
Sebastiaan van Stijn
b6f07fad51
Merge pull request #40712 from vboulineau/bugfix-windows-user-cpu
Fix CPU Stat value UsageInUsermode on Windows
2020-04-02 21:37:57 +02:00
Akihiro Suda
3802830989 cgroup2: implement docker stats
The following fields are unsupported:

* BlkioStats: all fields other than IoServiceBytesRecursive
* CPUStats: CPUUsage.PercpuUsage
* MemoryStats: MaxUsage and Failcnt

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-04-02 17:51:34 +09:00
vboulineau
ec16053ccf
Fix UsageInUsermode value on Windows
Looks like a wrong copy-paste using `RuntimeKernel100ns` twice instead of `RuntimeUser100ns`

Signed-off-by: Vincent Boulineau <vincent.boulineau@datadoghq.com>
2020-03-27 16:43:22 +01:00
Kir Kolyshkin
39048cf656 Really switch to moby/sys/mount*
Switch to moby/sys/mount and mountinfo. Keep the pkg/mount for potential
outside users.

This commit was generated by the following bash script:

```
set -e -u -o pipefail

for file in $(git grep -l 'docker/docker/pkg/mount"' | grep -v ^pkg/mount); do
	sed -i -e 's#/docker/docker/pkg/mount"#/moby/sys/mount"#' \
		-e 's#mount\.\(GetMounts\|Mounted\|Info\|[A-Za-z]*Filter\)#mountinfo.\1#g' \
		$file
	goimports -w $file
done
```

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2020-03-20 09:46:25 -07:00
Brian Goff
714cba6740
Merge pull request #38788 from AkihiroSuda/bind-nonrecursive-swarm
service: support --mount type=bind,bind-nonrecursive
2020-03-13 15:42:46 -07:00
Sebastiaan van Stijn
1e078e1ac5
Merge pull request #40666 from AkihiroSuda/prohibit-rootless-systemd-cgroup1
daemon: fail early if rootless && cgroupdriver == "systemd" && cgroup v1
2020-03-13 22:29:04 +01:00
Akihiro Suda
745fa04e52 service: support --mount type=bind,bind-nonrecursive
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-03-13 19:01:28 +09:00
Akihiro Suda
f073ed4187
Merge pull request #39816 from tao12345666333/rm-systeminfo-error-handler
Remove `SystemInfo()` error handling.
2020-03-13 01:59:13 +09:00
Brian Goff
470d32d32a
Merge pull request #40483 from AkihiroSuda/fuse-overlayfs
new storage driver: fuse-overlayfs
2020-03-11 11:32:37 -07:00
Akihiro Suda
92e7f8f67c daemon: fail early if rootless && cgroupdriver == "systemd" && cgroup v1
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-03-11 12:49:03 +09:00
Jintao Zhang
18c22f5bc1 fix backingFs assignment
Signed-off-by: Jintao Zhang <zhangjintao9020@gmail.com>
2020-03-10 00:03:59 +08:00
Akihiro Suda
5ddbe511a1
Merge pull request #40614 from thaJeztah/more_deprecating
Add warnings for deprecated "cluster" functions, and deprecate API fields
2020-03-06 16:35:42 +09:00
Sebastiaan van Stijn
616e64b42f
API: deprecate /info "ClusterStore" and "ClusterAdvertise" fields
These fields will now be omitted when empty.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-03-03 18:10:47 +01:00
Sebastiaan van Stijn
a5538c06f9
Add warning about deprecated "cluster" options to "docker info"
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-03-03 18:10:31 +01:00
Akihiro Suda
9a82a9a8ea vendor containerd, BuildKit, protobuf, grpc, and golang.org/x
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-03-03 10:25:20 +09:00
Brian Goff
76e3a49933
Merge pull request #40486 from AkihiroSuda/rootless-cgroup2-systemd
rootless: support `--exec-opt native.cgroupdriver=systemd`
2020-03-02 16:11:21 -08:00
Brian Goff
40b2b4b083
Merge pull request #40594 from sfzhu93/Mis_Unlock
daemon/cluster: add a missing Unlock
2020-02-28 09:47:53 -08:00
Sebastiaan van Stijn
39679991f4
Merge pull request #40543 from SamWhited/upstream_logging
Upstream logging changes from Enterprise Edition
2020-02-27 13:54:14 +01:00
Ziheng Liu
83c0bedba9 daemon/cluster: add a missing Unlock
Signed-off-by: Ziheng Liu <lzhfromustc@gmail.com>
2020-02-25 13:55:11 -05:00
Brian Goff
62bd5a33f7
Merge pull request #40137 from fuweid/me-wait-for-remote-containerd-before-reload
daemon: add grpc.WithBlock option
2020-02-21 10:11:10 -08:00
Akihiro Suda
0c6b85717b
Merge pull request #40481 from cpuguy83/stats_use_cond_var
Use condition variable to wake stats collector.
2020-02-21 13:37:14 +09:00
Brian Goff
750f0d1648 Support configuration of log cacher.
Configuration over the API per container is intentionally left out for
the time being, but is supported to configure the default from the
daemon config.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit cbecf48bc352e680a5390a7ca9cff53098cd16d7)
Signed-off-by: Madhu Venugopal <madhu@docker.com>
2020-02-19 17:02:34 -05:00
Brian Goff
e2ceb83a53 Support reads for all log drivers.
This supplements any log driver which does not support reads with a
custom read implementation that uses a local file cache.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit d675e2bf2b75865915c7a4552e00802feeb0847f)
Signed-off-by: Madhu Venugopal <madhu@docker.com>
2020-02-19 17:01:44 -05:00
Akihiro Suda
ca4b51868a rootless: support --exec-opt native.cgroupdriver=systemd
Support cgroup as in Rootless Podman.

Requires cgroup v2 host with crun.
Tested with Ubuntu 19.10 (kernel 5.3, systemd 242), crun v0.12.1.

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-02-14 15:32:31 +09:00
Sebastiaan van Stijn
3af8d484b1
Merge pull request #40394 from tao12345666333/reserved-namespace-labels
enforce reserve internal labels.
2020-02-13 01:47:05 +01:00
Sebastiaan van Stijn
58c2615208
Merge pull request #40497 from arkodg/fix-bip-subnet-config
Set the bip network value as the subnet
2020-02-12 12:41:29 +01:00
Jintao Zhang
35d6c1870f enforce reserve internal labels.
The namespaces com.docker.*, io.docker.*, org.dockerproject.*
have been documented to be reserved for Docker's internal use.

Co-Authored-By: Sebastiaan van Stijn <thaJeztah@users.noreply.github.com>
Signed-off-by: Jintao Zhang <zhangjintao9020@gmail.com>
2020-02-12 12:03:35 +08:00
Sebastiaan van Stijn
562880b276
Fix more goimports
```
daemon/logger/splunk/splunk_test.go:33: File is not `goimports`-ed (goimports)
        envKey:      "a",
        envRegexKey: "^foo",
        labelsKey:   "b",
        tagKey:      "c",
integration/build/build_test.go:41: File is not `goimports`-ed (goimports)
            rm:      false,
            forceRm: false,
integration/image/remove_unix_test.go:49: File is not `goimports`-ed (goimports)
        Root: d.Root,
```

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-02-11 18:56:25 +01:00
Arko Dasgupta
f800d5f786 Set the bip network value as the subnet
Dont assign the --bip value directly to the subnet
for the default bridge. Instead use the network value
from the ParseCIDR output

Addresses: https://github.com/moby/moby/issues/40392

Signed-off-by: Arko Dasgupta <arko.dasgupta@docker.com>
2020-02-10 17:38:54 -08:00
Sebastiaan van Stijn
008fc67974
Fluentd: add fluentd-request-ack option
This adds a new `fluentd-request-ack` logging option for the Fluentd
logging driver. If enabled, the server will respond with an acknowledgement.
This option improves the reliability of the message transmission. This
change is not versioned, and affects all API versions.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-02-11 02:13:24 +01:00
Sebastiaan van Stijn
cc1f3c750e
Fluentd: add fluentd-async option, deprecate fluentd-async-connect
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-02-11 02:13:22 +01:00
Sebastiaan van Stijn
a1d4a081dd
Fluentd: extract parsing config, and validate early
This extracts parsing the driver's configuration to a
function, and uses the same function both when initializing
the driver, and when validating logging options.

Doing so allows validating if the provided options are in
the correct format when calling `ValidateOpts`, instead
of resulting in an error when initializing the logging driver.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-02-11 02:13:20 +01:00
Sebastiaan van Stijn
8bd4aedb02
Fluentd: sort consts alphabetically
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-02-11 02:13:18 +01:00
Sebastiaan van Stijn
ad13a2a4ba
Fluentd: return "invalid parameter" for invalid config options
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-02-11 02:13:15 +01:00
Sebastiaan van Stijn
9f0b3f5609
bump gotest.tools v3.0.1 for compatibility with Go 1.14
full diff: https://github.com/gotestyourself/gotest.tools/compare/v2.3.0...v3.0.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-02-11 00:06:42 +01:00
Akihiro Suda
7418745001 new storage driver: fuse-overlayfs
`fuse-overlayfs` provides rootless overlayfs functionality without depending
on any kernel patch.

Aside from rootless, `fuse-overlayfs` could be potentially used for eliminating
`chown()` calls that happen in userns-remap mode, because `fuse-overlayfs` also
provides shiftfs functionality.

System requirements:
* fuse-overlayfs needs to be installed. Tested with 0.7.6.
* kernel >= 4.18

Unit test: `go test -exec sudo -v ./daemon/graphdriver/fuse-overlayfs`

The implementation is based on Podman's `overlay` driver which supports
both kernel-mode overlayfs and fuse-overlayfs in the single driver instance:
https://github.com/containers/storage/blob/39a8d5ed/drivers/overlay/overlay.go

However, Moby's implementation aims to decouple `fuse-overlayfs` driver from the
kernel-mode driver (`overlay2`) for simplicity.

Fix #40218

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-02-10 23:48:52 +09:00
Brian Goff
e75e6b0e31 Use condition variable to wake stats collector.
Before the collection goroutine wakes up every 1 second (as configured).
This sleep interval is in case there are no stats to collect we don't
end up in a tight loop.

Instead use a condition variable to signal that a collection is needed.
This prevents us from waking the goroutine needlessly when there is no
one looking for stats.

For now I've kept the sleep just moved it to the end of the loop, which
gives some space between collections.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2020-02-08 11:06:34 -08:00
Brian Goff
f464c31668 Check tmpfs mounts before create anon volume
This makes sure that things like `--tmpfs` mounts over an anonymous
volume don't create volumes uneccessarily.
One method only checks mountpoints, the other checks both mountpoints
and tmpfs... the usage of these should likely be consolidated.

Ideally, processing for `--tmpfs` mounts would get merged in with the
rest of the mount parsing. I opted not to do that for this change so the
fix is minimal and can potentially be backported with fewer changes of
breaking things.
Merging the mount processing for tmpfs can be handled in a followup.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2020-02-04 10:12:05 -08:00
Sebastiaan van Stijn
ca20bc4214
Merge pull request #40007 from arkodg/add-host-docker-internal
Support host.docker.internal in dockerd on Linux
2020-01-27 13:42:26 +01:00
Arko Dasgupta
92e809a680 Support host.docker.internal in dockerd on Linux
Docker Desktop (on MAC and Windows hosts) allows containers
running inside a Linux VM to connect to the host using
the host.docker.internal DNS name, which is implemented by
VPNkit (DNS proxy on the host)

This PR allows containers to connect to Linux hosts
by appending a special string "host-gateway" to --add-host
e.g. "--add-host=host.docker.internal:host-gateway" which adds
host.docker.internal DNS entry in /etc/hosts and maps it to host-gateway-ip

This PR also add a daemon flag call host-gateway-ip which defaults to
the default bridge IP
Docker Desktop will need to set this field to the Host Proxy IP
so DNS requests for host.docker.internal can be routed to VPNkit

Addresses: https://github.com/docker/for-linux/issues/264

Signed-off-by: Arko Dasgupta <arko.dasgupta@docker.com>
2020-01-22 13:30:00 -08:00
Akihiro Suda
f9d136b6c6
Merge pull request #40307 from dperny/swarm-jobs
Add support for swarm jobs
2020-01-20 12:57:05 +09:00
Sebastiaan van Stijn
be095a1859
Merge pull request #40366 from arkodg/check-cidr-ipv6
Handle the error case when fixed-cidr-ipv6 is empty and ipv6 is enabled
2020-01-14 13:53:45 +01:00
Drew Erny
30d9fe30b1 Add swarm jobs
Adds support for ReplicatedJob and GlobalJob service modes. These modes
allow running service which execute tasks that exit upon success,
instead of daemon-type tasks.

Signed-off-by: Drew Erny <drew.erny@docker.com>
2020-01-13 13:21:12 -06:00
Arko Dasgupta
bdad16b0ee Handle error case when fixed-cidr-ipv6 is empty
When IPv6 is enabled, make sure fixed-cidr-ipv6 is set
by the user since there is no default IPv6 local subnet
in the IPAM

Signed-off-by: Arko Dasgupta <arko.dasgupta@docker.com>
2020-01-13 09:56:41 -08:00
Sebastiaan van Stijn
339fb74cbc
prevent panic if TINI_COMMIT isn't set during build
If TINI_COMMIT isn't set, .go-autogen sets an empty value
as the "expected" commit. Attempting to truncate the value
caused a panic in that situation.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-01-10 15:27:20 +01:00
Sebastiaan van Stijn
e6c1820ef5
Merge pull request #40174 from AkihiroSuda/cgroup2
support cgroup2
2020-01-09 20:09:11 +01:00
Brian Goff
03163f6825
Merge pull request #40291 from akhilerm/privileged-device
35991- make `--device` works at privileged mode
2020-01-02 10:09:31 -08:00