Commit graph

7184 commits

Author SHA1 Message Date
Jintao Zhang
35d6c1870f enforce reserve internal labels.
The namespaces com.docker.*, io.docker.*, org.dockerproject.*
have been documented to be reserved for Docker's internal use.

Co-Authored-By: Sebastiaan van Stijn <thaJeztah@users.noreply.github.com>
Signed-off-by: Jintao Zhang <zhangjintao9020@gmail.com>
2020-02-12 12:03:35 +08:00
Sebastiaan van Stijn
562880b276
Fix more goimports
```
daemon/logger/splunk/splunk_test.go:33: File is not `goimports`-ed (goimports)
        envKey:      "a",
        envRegexKey: "^foo",
        labelsKey:   "b",
        tagKey:      "c",
integration/build/build_test.go:41: File is not `goimports`-ed (goimports)
            rm:      false,
            forceRm: false,
integration/image/remove_unix_test.go:49: File is not `goimports`-ed (goimports)
        Root: d.Root,
```

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-02-11 18:56:25 +01:00
Arko Dasgupta
f800d5f786 Set the bip network value as the subnet
Dont assign the --bip value directly to the subnet
for the default bridge. Instead use the network value
from the ParseCIDR output

Addresses: https://github.com/moby/moby/issues/40392

Signed-off-by: Arko Dasgupta <arko.dasgupta@docker.com>
2020-02-10 17:38:54 -08:00
Sebastiaan van Stijn
008fc67974
Fluentd: add fluentd-request-ack option
This adds a new `fluentd-request-ack` logging option for the Fluentd
logging driver. If enabled, the server will respond with an acknowledgement.
This option improves the reliability of the message transmission. This
change is not versioned, and affects all API versions.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-02-11 02:13:24 +01:00
Sebastiaan van Stijn
cc1f3c750e
Fluentd: add fluentd-async option, deprecate fluentd-async-connect
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-02-11 02:13:22 +01:00
Sebastiaan van Stijn
a1d4a081dd
Fluentd: extract parsing config, and validate early
This extracts parsing the driver's configuration to a
function, and uses the same function both when initializing
the driver, and when validating logging options.

Doing so allows validating if the provided options are in
the correct format when calling `ValidateOpts`, instead
of resulting in an error when initializing the logging driver.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-02-11 02:13:20 +01:00
Sebastiaan van Stijn
8bd4aedb02
Fluentd: sort consts alphabetically
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-02-11 02:13:18 +01:00
Sebastiaan van Stijn
ad13a2a4ba
Fluentd: return "invalid parameter" for invalid config options
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-02-11 02:13:15 +01:00
Sebastiaan van Stijn
9f0b3f5609
bump gotest.tools v3.0.1 for compatibility with Go 1.14
full diff: https://github.com/gotestyourself/gotest.tools/compare/v2.3.0...v3.0.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-02-11 00:06:42 +01:00
Akihiro Suda
7418745001 new storage driver: fuse-overlayfs
`fuse-overlayfs` provides rootless overlayfs functionality without depending
on any kernel patch.

Aside from rootless, `fuse-overlayfs` could be potentially used for eliminating
`chown()` calls that happen in userns-remap mode, because `fuse-overlayfs` also
provides shiftfs functionality.

System requirements:
* fuse-overlayfs needs to be installed. Tested with 0.7.6.
* kernel >= 4.18

Unit test: `go test -exec sudo -v ./daemon/graphdriver/fuse-overlayfs`

The implementation is based on Podman's `overlay` driver which supports
both kernel-mode overlayfs and fuse-overlayfs in the single driver instance:
https://github.com/containers/storage/blob/39a8d5ed/drivers/overlay/overlay.go

However, Moby's implementation aims to decouple `fuse-overlayfs` driver from the
kernel-mode driver (`overlay2`) for simplicity.

Fix #40218

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-02-10 23:48:52 +09:00
Brian Goff
e75e6b0e31 Use condition variable to wake stats collector.
Before the collection goroutine wakes up every 1 second (as configured).
This sleep interval is in case there are no stats to collect we don't
end up in a tight loop.

Instead use a condition variable to signal that a collection is needed.
This prevents us from waking the goroutine needlessly when there is no
one looking for stats.

For now I've kept the sleep just moved it to the end of the loop, which
gives some space between collections.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2020-02-08 11:06:34 -08:00
Brian Goff
f464c31668 Check tmpfs mounts before create anon volume
This makes sure that things like `--tmpfs` mounts over an anonymous
volume don't create volumes uneccessarily.
One method only checks mountpoints, the other checks both mountpoints
and tmpfs... the usage of these should likely be consolidated.

Ideally, processing for `--tmpfs` mounts would get merged in with the
rest of the mount parsing. I opted not to do that for this change so the
fix is minimal and can potentially be backported with fewer changes of
breaking things.
Merging the mount processing for tmpfs can be handled in a followup.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2020-02-04 10:12:05 -08:00
Sebastiaan van Stijn
ca20bc4214
Merge pull request #40007 from arkodg/add-host-docker-internal
Support host.docker.internal in dockerd on Linux
2020-01-27 13:42:26 +01:00
Arko Dasgupta
92e809a680 Support host.docker.internal in dockerd on Linux
Docker Desktop (on MAC and Windows hosts) allows containers
running inside a Linux VM to connect to the host using
the host.docker.internal DNS name, which is implemented by
VPNkit (DNS proxy on the host)

This PR allows containers to connect to Linux hosts
by appending a special string "host-gateway" to --add-host
e.g. "--add-host=host.docker.internal:host-gateway" which adds
host.docker.internal DNS entry in /etc/hosts and maps it to host-gateway-ip

This PR also add a daemon flag call host-gateway-ip which defaults to
the default bridge IP
Docker Desktop will need to set this field to the Host Proxy IP
so DNS requests for host.docker.internal can be routed to VPNkit

Addresses: https://github.com/docker/for-linux/issues/264

Signed-off-by: Arko Dasgupta <arko.dasgupta@docker.com>
2020-01-22 13:30:00 -08:00
Akihiro Suda
f9d136b6c6
Merge pull request #40307 from dperny/swarm-jobs
Add support for swarm jobs
2020-01-20 12:57:05 +09:00
Sebastiaan van Stijn
be095a1859
Merge pull request #40366 from arkodg/check-cidr-ipv6
Handle the error case when fixed-cidr-ipv6 is empty and ipv6 is enabled
2020-01-14 13:53:45 +01:00
Drew Erny
30d9fe30b1 Add swarm jobs
Adds support for ReplicatedJob and GlobalJob service modes. These modes
allow running service which execute tasks that exit upon success,
instead of daemon-type tasks.

Signed-off-by: Drew Erny <drew.erny@docker.com>
2020-01-13 13:21:12 -06:00
Arko Dasgupta
bdad16b0ee Handle error case when fixed-cidr-ipv6 is empty
When IPv6 is enabled, make sure fixed-cidr-ipv6 is set
by the user since there is no default IPv6 local subnet
in the IPAM

Signed-off-by: Arko Dasgupta <arko.dasgupta@docker.com>
2020-01-13 09:56:41 -08:00
Sebastiaan van Stijn
339fb74cbc
prevent panic if TINI_COMMIT isn't set during build
If TINI_COMMIT isn't set, .go-autogen sets an empty value
as the "expected" commit. Attempting to truncate the value
caused a panic in that situation.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-01-10 15:27:20 +01:00
Sebastiaan van Stijn
e6c1820ef5
Merge pull request #40174 from AkihiroSuda/cgroup2
support cgroup2
2020-01-09 20:09:11 +01:00
Brian Goff
03163f6825
Merge pull request #40291 from akhilerm/privileged-device
35991- make `--device` works at privileged mode
2020-01-02 10:09:31 -08:00
Akhil Mohan
86ebbe16de
remove host directory check
Signed-off-by: Akhil Mohan <akhil.mohan@mayadata.io>
2020-01-02 14:28:51 +05:30
Akihiro Suda
491531c12b cgroup2: mark cpu-rt-{period,runtime} unimplemented
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-01-01 02:58:40 +09:00
Akihiro Suda
19baeaca26 cgroup2: enable cgroup namespace by default
For cgroup v1, we were unable to change the default because of
compatibility issue.

For cgroup v2, we should change the default right now because switching
to cgroup v2 is already breaking change.

See also containers/libpod#4363 containers/libpod#4374

Privileged containers also use cgroupns=private by default.
https://github.com/containers/libpod/pull/4374#issuecomment-549776387

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-01-01 02:58:40 +09:00
Akihiro Suda
612343618d cgroup2: use shim V2
* Requires containerd binaries from containerd/containerd#3799 . Metrics are unimplemented yet.
* Works with crun v0.10.4, but `--security-opt seccomp=unconfined` is needed unless using master version of libseccomp
  ( containers/crun#156, seccomp/libseccomp#177 )
* Doesn't work with master runc yet
* Resource limitations are unimplemented

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-01-01 02:58:40 +09:00
Brian Goff
8d2456d1e6
Merge pull request #40246 from thaJeztah/system_windows_cleanup
pkg/system: minor cleanups and remove use of deprecated system.GetOSVersion()
2019-12-19 11:36:29 -08:00
Brian Goff
b95fad8e51
Merge pull request #40263 from thaJeztah/normalize_comments
Normalize comment formatting
2019-12-12 12:06:22 -08:00
Akhil Mohan
35b9e6989f
Make --device flag work in privileged mode
When a container is started in privileged mode, the device mappings
provided by `--device` flag was ignored. Now the device mappings will
be considered even in privileged mode.

Signed-off-by: Akhil Mohan <akhil.mohan@mayadata.io>
2019-12-06 18:43:56 +05:30
wenlxie
03b3ec1dd5
make --device works at privileged mode
Signed-off-by: wenlxie <wenlxie@ebay.com>
2019-12-06 18:17:03 +05:30
Brian Goff
e25754b80c
Merge pull request #40186 from pradipd/default-nat-subnet
Dockerd won't start if a network with the default subnet prefix already exists in HNS.
2019-12-03 09:31:29 -08:00
Sebastiaan van Stijn
f4f56b1197
daemon: normalize comment formatting
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-11-27 15:43:53 +01:00
Sebastiaan van Stijn
ec4bc83258
daemon/graphdriver: normalize comment formatting
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-11-27 15:43:23 +01:00
Sebastiaan van Stijn
6625fa6103
daemon/cluster: normalize comment formatting
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-11-27 15:42:53 +01:00
Sebastiaan van Stijn
ba6bbca89a
daemon/logger: normalize comment formatting
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-11-27 15:42:27 +01:00
Akihiro Suda
a4a0429eec
Merge pull request #40234 from Jonas-Heinrich/40232-comply-with-gelf-spec
Change gelf logger to comply with gelf spec
2019-11-26 12:06:47 +09:00
Akihiro Suda
7bd40b56ef
Merge pull request #40249 from thaJeztah/os_seek_deprecated
remove use of deprecated os.SEEK_END
2019-11-26 12:06:29 +09:00
Akihiro Suda
d2d8e96f51
Merge pull request #39940 from carlosedp/runtime-version
Change version parsing to support alternate runtimes
2019-11-26 09:55:53 +09:00
Sebastiaan van Stijn
6ee536b4a0
daemon: remove use of deprecated os.SEEK_END
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-11-25 18:53:03 +01:00
Sebastiaan van Stijn
044b74e33b
daemon: remove use of deprecated system.GetOSVersion()
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-11-25 13:39:50 +01:00
Jonas Heinrich
5c6b913ff1
logger/gelf: Skip empty lines to comply with spec
The [gelf payload specification](http://docs.graylog.org/en/2.4/pages/gelf.html#gelf-payload-specification)
demands that the field `short_message` *MUST* be set by the client library.
Since docker logging via the gelf driver sends messages line by line, it can happen that messages with an empty
`short_message` are passed on. This causes strict downstream processors (like graylog) to raise an exception.

The logger now skips messages with an empty line.

Resolves: #40232
See also: #37572

Signed-off-by: Jonas Heinrich <Jonas@JonasHeinrich.com>
2019-11-25 11:55:15 +01:00
Tõnis Tiigi
d1d5f64766
Merge pull request #40021 from thaJeztah/carry_40017
Use newer x/sys/windows SecurityAttributes struct (carry 40017)
2019-11-21 08:57:22 -08:00
Sebastiaan van Stijn
1086671441
Merge pull request #40212 from olljanat/default-capabilities-to-caps
Move DefaultCapabilities() to caps package
2019-11-17 14:00:06 +01:00
Akihiro Suda
4124e78d57
Merge pull request #40100 from thaJeztah/test_fixes
Some small (test) fixes/improvements
2019-11-16 14:13:35 +09:00
Akihiro Suda
4f8070c84d
Merge pull request #40210 from kolyshkin/ovr-rm-checks
overlay[2]: rm extra checks in init
2019-11-15 22:43:52 +09:00
Olli Janatuinen
1308a3a99f Move DefaultCapabilities() to caps package
Signed-off-by: Olli Janatuinen <olli.janatuinen@gmail.com>
2019-11-14 21:13:16 +02:00
Kir Kolyshkin
e226aea280 overlay[2]: rm fs checks
Now that we do check if overlay is working by performing an actual
overlayfs mount, there's no need in extra checks for the kernel version
or the filesystem type. Actual mount check is sufficient.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2019-11-14 08:13:12 -08:00
Olli Janatuinen
447a840254 Windows: Use system specific parallelism value on containers restart
Signed-off-by: Olli Janatuinen <olli.janatuinen@gmail.com>
2019-11-11 15:44:47 +02:00
Kir Kolyshkin
649e4c8889 Fix/improve overlay support check
Before this commit, overlay check was performed by looking for
`overlay` in /proc/filesystem. This obviously might not work
for rootless Docker (fs is there, but one can't use it as non-root).

This commit changes the check to perform the actual mount, by reusing
the code previously written to check for multiple lower dirs support.

The old check is removed from both drivers, as well as the additional
check for the multiple lower dirs support in overlay2 since it's now
a part of the main check.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2019-11-08 11:49:39 -08:00
Kir Kolyshkin
d5687079ad overlay: move supportsMultipleLowerDir to utils
This moves supportsMultipleLowerDir() to overlayutils
so it can be used from both overlay and overlay2.

The only changes made were:
 * replace logger with logrus
 * don't use workDirName mergedDirName constants
 * add mnt var to improve readability a bit

This is a preparation for the next commit.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2019-11-08 11:48:47 -08:00
Yong Tang
f09dc2f4fc Fix docker crash when creating namespaces with UID in /etc/subuid and /etc/subgid
This fix tries to address the issue raised in 39353 where
docker crash when creating namespaces with UID in /etc/subuid and /etc/subgid.

The issue was that, mapping to `/etc/sub[u,g]id` in docker does not
allow numeric ID.

This fix fixes the issue by probing other combinations (uid:groupname, username:gid, uid:gid)
when normal username:groupname fails.

This fix fixes 39353.

Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
2019-11-07 20:17:11 +00:00