Commit graph

6076 commits

Author SHA1 Message Date
Sebastiaan van Stijn
813a7da526
Revert "Add limit to page size used by overlay2 driver"
This reverts commit 520034e35b.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-11-29 23:02:18 +01:00
Sebastiaan van Stijn
6f70946a27
Add warning to /info if KernelMemoryTCP is not supported
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-11-27 22:47:39 +01:00
Sebastiaan van Stijn
d3e75e4220
Merge pull request #37043 from yongtang/37038-kernelTCP
Add memory.kernelTCP support for linux
2018-11-27 22:36:10 +01:00
Vincent Demeester
6fa149805c
Merge pull request #37638 from jterry75/devices_windows
Add --device support for Windows
2018-11-27 15:03:17 +01:00
Yong Tang
f023816608 Add memory.kernelTCP support for linux
This fix tries to address the issue raised in 37038 where
there were no memory.kernelTCP support for linux.

This fix add MemoryKernelTCP to HostConfig, and pass
the config to runtime-spec.

Additional test case has been added.

This fix fixes 37038.

Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
2018-11-26 21:03:08 +00:00
Sebastiaan van Stijn
0b7cb16dde
Merge pull request #38102 from selansen/master
VXLAN UDP Port configuration support
2018-11-24 11:50:10 +01:00
selansen
32180ac0c7 VXLAN UDP Port configuration support
This commit contains changes to configure DataPathPort
option. By default we use 4789 port number. But this commit
will allow user to configure port number during swarm init.
DataPathPort can't be modified after swarm init.
Signed-off-by: selansen <elango.siva@docker.com>
2018-11-22 17:35:02 -05:00
zhangyue
5007c36d71 cli: fix images filter when use multi reference filter
Signed-off-by: zhangyue <zy675793960@yeah.net>
2018-11-22 10:33:45 +08:00
Justin Terry (VM)
b2d99865ea Add --device support for Windows
Implements the --device forwarding for Windows daemons. This maps the physical
device into the container at runtime.

Ex:

docker run --device="class/<clsid>" <image> <cmd>

Signed-off-by: Justin Terry (VM) <juterry@microsoft.com>
2018-11-21 15:31:17 -08:00
Sebastiaan van Stijn
758255791e
Merge pull request #38177 from mooncak/fix_duplicate
Cleanup duplication in daemon files
2018-11-13 09:55:51 +01:00
mooncake
345d1fd089 Cleanup duplication in daemon files
Signed-off-by: Bily Zhang <xcoder@tenxcloud.com>
2018-11-13 10:42:57 +08:00
Akihiro Suda
596cdffb9f mount: add BindOptions.NonRecursive (API v1.40)
This allows non-recursive bind-mount, i.e. mount(2) with "bind" rather than "rbind".

Swarm-mode will be supported in a separate PR because of mutual vendoring.

Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-11-06 17:51:58 +09:00
Sebastiaan van Stijn
12bba16306
Merge pull request #38029 from lifubang/checkpointrm
fixes checkpoint rm fail
2018-11-05 19:09:35 +01:00
Tonis Tiigi
489b8eda66 cluster: set bigger grpc limit for array requests
4MB client side limit was introduced in vendoring go-grpc#1165 (v1.4.0)
making these requests likely to produce errors

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-10-30 16:02:34 -07:00
Sebastiaan van Stijn
13ef0ebd2b
Deprecate AuFS storage driver, and add warning
The `aufs` storage driver is deprecated in favor of `overlay2`, and will
be removed in a future release. Users of the `aufs` storage driver are
recommended to migrate to a different storage driver, such as `overlay2`, which
is now the default storage driver.

The `aufs` storage driver facilitates running Docker on distros that have no
support for OverlayFS, such as Ubuntu 14.04 LTS, which originally shipped with
a 3.14 kernel.

Now that Ubuntu 14.04 is no longer a supported distro for Docker, and `overlay2`
is available to all supported distros (as they are either on kernel 4.x, or have
support for multiple lowerdirs backported), there is no reason to continue
maintenance of the `aufs` storage driver.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-10-26 18:41:46 +02:00
Sebastiaan van Stijn
1527a67212
Merge pull request #37999 from Microsoft/jjh/tar2vhd
LCOW: ApplyDiff() use tar2ext4, not SVM
2018-10-24 22:35:34 +02:00
Sebastiaan van Stijn
b48bf39a79
Merge pull request #37944 from IRCody/awslogs_error_context
Return more context on awslogs create failure
2018-10-24 21:00:15 +02:00
Anusha Ragunathan
6611ab1c6f
Merge pull request #37986 from samuelkarp/moby/moby-37747
awslogs: account for UTF-8 normalization in limits
2018-10-18 10:17:24 -07:00
Yong Tang
533e07afbe
Merge pull request #38032 from RohitK89/21497-log-image-name
Add IMAGE_NAME attribute to journald log events
2018-10-17 12:18:05 -07:00
Colin Panisset
5cd2bb315a Only add CONTAINER_PARTIAL_MESSAGE if not the last partial
Addresses #38045

Signed-off-by: Colin Panisset <colin.panisset@cevo.com.au>
2018-10-17 07:51:59 +11:00
Cody Roseborough
7a5c813d9c Return more context on awslogs create failure
Signed-off-by: Cody Roseborough <crrosebo@amazon.com>
2018-10-16 11:36:52 -07:00
Yong Tang
ee6fc90b2c
Merge pull request #37993 from kolyshkin/ovr2-index
overlay2: use index=off if possible (fix EBUSY on mount)
2018-10-13 08:28:10 -07:00
Yong Tang
9d4ac4b8d2
Merge pull request #38019 from thaJeztah/skip_deprecated_drivers_in_autoselect
Skip deprecated storage-drivers in auto-selection
2018-10-13 08:26:03 -07:00
Rohit Kapur
5f7e102df7 Add IMAGE_NAME as a key to journald log messages
Signed-off-by: Rohit Kapur <rkapur@flatiron.com>
2018-10-12 16:16:31 -04:00
Vincent Demeester
10ebe6381e
Merge pull request #38025 from thaJeztah/itsy_bitsy_teeny_weeny
Remove redundant nil checks
2018-10-12 18:43:11 +02:00
Lifubang
99a7a4dcd0 checkpoint rm fail
Signed-off-by: Lifubang <lifubang@acmcoder.com>
2018-10-12 19:08:28 +08:00
Kir Kolyshkin
16d822bba8 btrfs: ensure graphdriver home is bind mount
For some reason, shared mount propagation between the host
and a container does not work for btrfs, unless container
root directory (i.e. graphdriver home) is a bind mount.

The above issue was reproduced on SLES 12sp3 + btrfs using
the following script:

	#!/bin/bash
	set -eux -o pipefail

	# DIR should not be under a subvolume
	DIR=${DIR:-/lib}
	MNT=$DIR/my-mnt
	FILE=$MNT/file

	ID=$(docker run -d --privileged -v $DIR:$DIR:rshared ubuntu sleep 24h)
	docker exec $ID mkdir -p $MNT
	docker exec $ID mount -t tmpfs tmpfs $MNT
	docker exec $ID touch $FILE
	ls -l $FILE
	umount $MNT
	docker rm -f $ID

which fails this way:

	+ ls -l /lib/my-mnt/file
	ls: cannot access '/lib/my-mnt/file': No such file or directory

meaning the mount performed inside a priviledged container is not
propagated back to the host (even if all the mounts have "shared"
propagation mode).

The remedy to the above is to make graphdriver home a bind mount.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2018-10-11 23:45:00 -07:00
Sebastiaan van Stijn
de1094bc95
Remove redundant nil checks
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-10-11 23:19:01 +02:00
Kir Kolyshkin
8422d85087 overlay2: use index=off if possible
As pointed out in https://github.com/moby/moby/issues/37970,
Docker overlay driver can't work with index=on feature of
the Linux kernel "overlay" filesystem. In case the global
default is set to "yes", Docker will fail with EBUSY when
trying to mount, like this:

> error creating overlay mount to ...../merged: device or resource busy

and the kernel log should contain something like:

> overlayfs: upperdir is in-use by another mount, mount with
> '-o index=off' to override exclusive upperdir protection.

A workaround is to set index=off in overlay kernel module
parameters, or even recompile the kernel with
CONFIG_OVERLAY_FS_INDEX=n in .config. Surely this is not
always practical or even possible.

The solution, as pointed out my Amir Goldstein (as well as
the above kernel message:) is to use 'index=off' option
when mounting.

NOTE since older (< 4.13rc1) kernels do not support "index="
overlayfs parameter, try to figure out whether the option
is supported. In case it's not possible to figure out,
assume it is not.

NOTE the default can be changed anytime (by writing to
/sys/module/overlay/parameters/index) so we need to always
use index=off.

[v2: move the detection code to Init()]
[v3: don't set index=off if stat() failed]

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2018-10-11 12:52:57 -07:00
Kir Kolyshkin
a55d32546a overlay2: use global logger instance
This simplifies the code a lot.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2018-10-11 12:50:45 -07:00
Sebastiaan van Stijn
b72db8b82c
Skip deprecated storage-drivers in auto-selection
Discourage users from using deprecated storage-drivers
by skipping them when automatically selecting a storage-
driver.

This change does not affect existing installations, because
existing state will take precedence.

Users can still use deprecated drivers by manually configuring
the daemon to use a specific driver.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-10-11 15:52:19 +02:00
Sebastiaan van Stijn
31be4e0ba1
Deprecate legacy overlay storage driver, and add warning
The `overlay` storage driver is deprecated in favor of the `overlay2` storage
driver, which has all the benefits of `overlay`, without its limitations (excessive
inode consumption). The legacy `overlay` storage driver will be removed in a future
release. Users of the `overlay` storage driver should migrate to the `overlay2`
storage driver.

The legacy `overlay` storage driver allowed using overlayFS-backed filesystems
on pre 4.x kernels. Now that all supported distributions are able to run `overlay2`
(as they are either on kernel 4.x, or have support for multiple lowerdirs
backported), there is no reason to keep maintaining the `overlay` storage driver.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-10-11 15:49:15 +02:00
Sebastiaan van Stijn
06fcabbaa0
Deprecate "devicemapper" storage driver, and add warning
The `devicemapper` storage driver is deprecated in favor of `overlay2`, and will
be removed in a future release. Users of the `devicemapper` storage driver are
recommended to migrate to a different storage driver, such as `overlay2`, which
is now the default storage driver.

The `devicemapper` storage driver facilitates running Docker on older (3.x) kernels
that have no support for other storage drivers (such as overlay2, or AUFS).

Now that support for `overlay2` is added to all supported distros (as they are
either on kernel 4.x, or have support for multiple lowerdirs backported), there
is no reason to continue maintenance of the `devicemapper` storage driver.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-10-11 15:46:26 +02:00
Samuel Karp
1e8ef38627 awslogs: account for UTF-8 normalization in limits
The CloudWatch Logs API defines its limits in terms of bytes, but its
inputs in terms of UTF-8 encoded strings.  Byte-sequences which are not
valid UTF-8 encodings are normalized to the Unicode replacement
character U+FFFD, which is a 3-byte sequence in UTF-8.  This replacement
can cause the input to grow, exceeding the API limit and causing failed
API calls.

This commit adds logic for counting the effective byte length after
normalization and splitting input without splitting valid UTF-8
byte-sequences into two invalid byte-sequences.

Fixes https://github.com/moby/moby/issues/37747

Signed-off-by: Samuel Karp <skarp@amazon.com>
2018-10-10 14:45:06 -07:00
Sebastiaan van Stijn
6efa2767d4
Merge pull request #38000 from Microsoft/jjh/processandiot
Windows: Client: Allow process isolation [RS5+]
2018-10-10 19:29:23 +02:00
John Howard
bde9996065 LCOW: ApplyDiff() use tar2ext4, not SVM
Signed-off-by: John Howard <jhoward@microsoft.com>

This removes the need for an SVM in the LCOW driver to ApplyDiff.

This change relates to a fix for https://github.com/moby/moby/issues/36353

However, it found another issue, tracked by https://github.com/moby/moby/issues/37955
2018-10-09 16:10:46 -07:00
Yong Tang
82a4797499
Merge pull request #37988 from mirake/fix-typos
Fix typo: adapater -> adapter
2018-10-09 12:47:18 -07:00
John Howard
c907c2486c Windows:Allow process isolation
Signed-off-by: John Howard <jhoward@microsoft.com>
2018-10-09 11:58:26 -07:00
Yong Tang
2cc338c100
Merge pull request #37967 from thaJeztah/upstream_dos_fix
Fix denial of service with large numbers in cpuset-cpus and cpuset-mems
2018-10-08 13:23:03 +00:00
Sebastiaan van Stijn
fddefa72c4
Merge pull request #37983 from IRCody/tar_id_logging
Add layer id to NaiveDiffDriver untar timing log
2018-10-08 14:17:41 +02:00
Rui Cao
d3e155d926 Fix typo: adapater -> adapter
Signed-off-by: Rui Cao <ruicao@alauda.io>
2018-10-08 19:15:38 +08:00
Vincent Demeester
2bbd0bd8ef
Merge pull request #37802 from Microsoft/jjh/37687-docker-system-df
Fix docker system df when LCOW and WCOW images loaded
2018-10-08 09:26:35 +02:00
Akihiro Suda
b5ed4ebe06
Merge pull request #36537 from Microsoft/jjh/lcow-log-stderr
LCOW: Log stderr on failures to ease diagnosis
2018-10-06 11:05:55 +09:00
Cody Roseborough
3b4df3d146 Add layer id to NaiveDiffDriver untar timing log
Signed-off-by: Cody Roseborough <crrosebo@amazon.com>
2018-10-05 16:28:40 -07:00
mooncake
ea60a87fcf Remove duplicated words in daemon files
Signed-off-by: mooncake <xcoder@tenxcloud.com>
2018-10-06 00:06:38 +08:00
Justin Cormack
f8e876d761
Fix denial of service with large numbers in cpuset-cpus and cpuset-mems
Using a value such as `--cpuset-mems=1-9223372036854775807` would cause
`dockerd` to run out of memory allocating a map of the values in the
validation code. Set limits to the normal limit of the number of CPUs,
and improve the error handling.

Reported by Huawei PSIRT.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-10-05 15:09:02 +02:00
Vincent Demeester
e3b712152d
Merge pull request #37968 from thaJeztah/no_more_version_mismatch
Remove version-checks for containerd and runc
2018-10-05 12:07:44 +02:00
Sebastiaan van Stijn
c65f0bd13c
Remove version-checks for containerd and runc
With containerd reaching 1.0, the runtime now
has a stable API, so there's no need to do a check
if the installed version matches the expected version.

Current versions of Docker now also package containerd
and runc separately, and can be _updated_ separately.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-10-04 23:17:13 +02:00
Sebastiaan van Stijn
192ff56d87
Merge pull request #37949 from selansen/master
Fix for default-addr-pool-mask-length param max value check
2018-10-04 22:00:42 +02:00
Sebastiaan van Stijn
9c4982685e
Merge pull request #37934 from dani-docker/esc-879
Masking proxy credentials from URL when displayed in system info
2018-10-04 19:37:58 +02:00