For some reason, shared mount propagation between the host
and a container does not work for btrfs, unless container
root directory (i.e. graphdriver home) is a bind mount.
The above issue was reproduced on SLES 12sp3 + btrfs using
the following script:
#!/bin/bash
set -eux -o pipefail
# DIR should not be under a subvolume
DIR=${DIR:-/lib}
MNT=$DIR/my-mnt
FILE=$MNT/file
ID=$(docker run -d --privileged -v $DIR:$DIR:rshared ubuntu sleep 24h)
docker exec $ID mkdir -p $MNT
docker exec $ID mount -t tmpfs tmpfs $MNT
docker exec $ID touch $FILE
ls -l $FILE
umount $MNT
docker rm -f $ID
which fails this way:
+ ls -l /lib/my-mnt/file
ls: cannot access '/lib/my-mnt/file': No such file or directory
meaning the mount performed inside a priviledged container is not
propagated back to the host (even if all the mounts have "shared"
propagation mode).
The remedy to the above is to make graphdriver home a bind mount.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
As pointed out in https://github.com/moby/moby/issues/37970,
Docker overlay driver can't work with index=on feature of
the Linux kernel "overlay" filesystem. In case the global
default is set to "yes", Docker will fail with EBUSY when
trying to mount, like this:
> error creating overlay mount to ...../merged: device or resource busy
and the kernel log should contain something like:
> overlayfs: upperdir is in-use by another mount, mount with
> '-o index=off' to override exclusive upperdir protection.
A workaround is to set index=off in overlay kernel module
parameters, or even recompile the kernel with
CONFIG_OVERLAY_FS_INDEX=n in .config. Surely this is not
always practical or even possible.
The solution, as pointed out my Amir Goldstein (as well as
the above kernel message:) is to use 'index=off' option
when mounting.
NOTE since older (< 4.13rc1) kernels do not support "index="
overlayfs parameter, try to figure out whether the option
is supported. In case it's not possible to figure out,
assume it is not.
NOTE the default can be changed anytime (by writing to
/sys/module/overlay/parameters/index) so we need to always
use index=off.
[v2: move the detection code to Init()]
[v3: don't set index=off if stat() failed]
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Discourage users from using deprecated storage-drivers
by skipping them when automatically selecting a storage-
driver.
This change does not affect existing installations, because
existing state will take precedence.
Users can still use deprecated drivers by manually configuring
the daemon to use a specific driver.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The `overlay` storage driver is deprecated in favor of the `overlay2` storage
driver, which has all the benefits of `overlay`, without its limitations (excessive
inode consumption). The legacy `overlay` storage driver will be removed in a future
release. Users of the `overlay` storage driver should migrate to the `overlay2`
storage driver.
The legacy `overlay` storage driver allowed using overlayFS-backed filesystems
on pre 4.x kernels. Now that all supported distributions are able to run `overlay2`
(as they are either on kernel 4.x, or have support for multiple lowerdirs
backported), there is no reason to keep maintaining the `overlay` storage driver.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The `devicemapper` storage driver is deprecated in favor of `overlay2`, and will
be removed in a future release. Users of the `devicemapper` storage driver are
recommended to migrate to a different storage driver, such as `overlay2`, which
is now the default storage driver.
The `devicemapper` storage driver facilitates running Docker on older (3.x) kernels
that have no support for other storage drivers (such as overlay2, or AUFS).
Now that support for `overlay2` is added to all supported distros (as they are
either on kernel 4.x, or have support for multiple lowerdirs backported), there
is no reason to continue maintenance of the `devicemapper` storage driver.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: John Howard <jhoward@microsoft.com>
This is a fix for a few related scenarios where it's impossible to remove layers or containers
until the host is rebooted. Generally (or at least easiest to repro) through a forced daemon kill
while a container is running.
Possibly slightly worse than that, as following a host reboot, the scratch layer would possibly be leaked and
left on disk under the dataroot\windowsfilter directory after the container is removed.
One such example of a failure:
1. run a long running container with the --rm flag
docker run --rm -d --name test microsoft/windowsservercore powershell sleep 30
2. Force kill the daemon not allowing it to cleanup. Simulates a crash or a host power-cycle.
3. (re-)Start daemon
4. docker ps -a
PS C:\control> docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
7aff773d782b malloc "powershell start-sl…" 11 seconds ago Removal In Progress malloc
5. Try to remove
PS C:\control> docker rm 7aff
Error response from daemon: container 7aff773d782bbf35d95095369ffcb170b7b8f0e6f8f65d5aff42abf61234855d: driver "windowsfilter" failed to remove root filesystem: rename C:\control\windowsfilter\7aff773d782bbf35d95095369ffcb170b7b8f0e6f8f65d5aff42abf61234855d C:\control\windowsfilter\7aff773d782bbf35d95095369ffcb170b7b8f0e6f8f65d5aff42abf61234855d-removing: Access is denied.
PS C:\control>
Step 5 fails.
Signed-off-by: John Howard <jhoward@microsoft.com>
Fixes#36764
@johnstep PTAL. @jterry75 FYI.
There are two commits in this PR. The first ensure that errors are actually returned to the caller - it was being thrown away.
The second commit changes the LCOW driver to map, on a per service VM basis, "long" container paths such as `/tmp/c8fa0ae1b348f505df2707060f6a49e63280d71b83b7936935c827e2e9bde16d` to much shorter paths, based on a per-service VM counter, so something more like /tmp/d3. This means that the root cause of the failure where the mount call to create the overlay was failing due to command line length becomes something much shorter such as below.
`mount -t overlay overlay -olowerdir=/tmp/d3:/tmp/d4:/tmp/d5:/tmp/d6:/tmp/d7:/tmp/d8:/tmp/d9:/tmp/d10:/tmp/d11:/tmp/d12:/tmp/d13:/tmp/d14:/tmp/d15:/tmp/d16:/tmp/d17:/tmp/d18:/tmp/d19:/tmp/d20:/tmp/d21:/tmp/d22:/tmp/d23:/tmp/d24:/tmp/d25:/tmp/d26:/tmp/d27:/tmp/d28:/tmp/d29:/tmp/d30:/tmp/d31:/tmp/d32:/tmp/d33:/tmp/d34:/tmp/d35:/tmp/d36:/tmp/d37:/tmp/d38:/tmp/d39:/tmp/d40:/tmp/d41:/tmp/d42:/tmp/d43:/tmp/d44:/tmp/d45:/tmp/d46:/tmp/d47:/tmp/d48:/tmp/d49:/tmp/d50:/tmp/d51:/tmp/d52:/tmp/d53:/tmp/d54:/tmp/d55:/tmp/d56:/tmp/d57:/tmp/d58:/tmp/d59:/tmp/d60:/tmp/d61:/tmp/d62,upperdir=/tmp/d2/upper,workdir=/tmp/d2/work /tmp/c8fa0ae1b348f505df2707060f6a49e63280d71b83b7936935c827e2e9bde16d-mount`
For those worrying about overflow (which I'm sure @thaJeztah will mention...): It's safe to use a counter here as SVMs are disposable in the default configuration. The exception is when running the daemon in unsafe LCOW "global" mode (ie `--storage-opt lcow.globalmode=1`) where the SVMs aren't disposed of, but a single one is reused. However, to overflow the command line length, it would require several hundred-thousand trillion (conservative, I should sit down and work it out accurately if I get -really- bored) of SCSI hot-add operations, and even to hit that would be hard as just running containers normally uses the VPMEM path for the containers UVM, not to the global SVM on SCSI. It gets incremented by one per build step (commit more accurately) as a general rule. Hence it would be necessary to have to be doing automated builds without restarting the daemon for literally years on end in unsafe mode. 😇
Note that in reality, the previous limit of ~47 layers before hitting the command line length limit is close to what is possible in the platform, at least as of RS5/Windows Server 2019 where, in the HCS v1 schema, a single SCSI controller is used, and that can only support 64 disks per controller per the Hyper-V VDEV. And remember we have one slot taken up for the SVMs scratch, and another for the containers scratch when committing a layer. So the best you can architecturally get on the platform is around the following (it's also different by 1 depending on whether in unsafe or default mode)
```
PS E:\docker\build\36764\short> docker build --no-cache .
Sending build context to Docker daemon 2.048kB
Step 1/4 : FROM alpine as first
---> 11cd0b38bc3c
Step 2/4 : RUN echo test > /test
---> Running in 8ddfe20e5bfb
Removing intermediate container 8ddfe20e5bfb
---> b0103a00b1c9
Step 3/4 : FROM alpine
---> 11cd0b38bc3c
Step 4/4 : COPY --from=first /test /test
---> 54bfae391eba
Successfully built 54bfae391eba
PS E:\docker\build\36764\short> cd ..
PS E:\docker\build\36764> docker build --no-cache .
Sending build context to Docker daemon 4.689MB
Step 1/61 : FROM alpine as first
---> 11cd0b38bc3c
Step 2/61 : RUN echo test > /test
---> Running in 02597ff870db
Removing intermediate container 02597ff870db
---> 3096de6fc454
Step 3/61 : RUN echo test > /test
---> Running in 9a8110f4ff19
Removing intermediate container 9a8110f4ff19
---> 7691808cf28e
Step 4/61 : RUN echo test > /test
---> Running in 9afb8f51510b
Removing intermediate container 9afb8f51510b
---> e42a0df2bb1c
Step 5/61 : RUN echo test > /test
---> Running in fe977ed6804e
Removing intermediate container fe977ed6804e
---> 55850c9b0479
Step 6/61 : RUN echo test > /test
---> Running in be65cbfad172
Removing intermediate container be65cbfad172
---> 0cf8acba70f0
Step 7/61 : RUN echo test > /test
---> Running in fd5b0907b6a9
Removing intermediate container fd5b0907b6a9
---> 257a4493d85d
Step 8/61 : RUN echo test > /test
---> Running in f7ca0ffd9076
Removing intermediate container f7ca0ffd9076
---> 3baa6f4fa2d5
Step 9/61 : RUN echo test > /test
---> Running in 5146814d4727
Removing intermediate container 5146814d4727
---> 485b9d5cf228
Step 10/61 : RUN echo test > /test
---> Running in a090eec1b743
Removing intermediate container a090eec1b743
---> a7eb10155b51
Step 11/61 : RUN echo test > /test
---> Running in 942660b288df
Removing intermediate container 942660b288df
---> 9d286a1e2133
Step 12/61 : RUN echo test > /test
---> Running in c3d369aa91df
Removing intermediate container c3d369aa91df
---> f78be4788992
Step 13/61 : RUN echo test > /test
---> Running in a03c3ac6888f
Removing intermediate container a03c3ac6888f
---> 6504363f61ab
Step 14/61 : RUN echo test > /test
---> Running in 0c3c2fca3f90
Removing intermediate container 0c3c2fca3f90
---> fe3448b8bb29
Step 15/61 : RUN echo test > /test
---> Running in 828d51c76d3b
Removing intermediate container 828d51c76d3b
---> 870684e3aea0
Step 16/61 : RUN echo test > /test
---> Running in 59a2f7c5f3ad
Removing intermediate container 59a2f7c5f3ad
---> cf84556ca5c0
Step 17/61 : RUN echo test > /test
---> Running in bfb4e088eeb3
Removing intermediate container bfb4e088eeb3
---> 9c8f9f652cef
Step 18/61 : RUN echo test > /test
---> Running in f1b88bb5a2d7
Removing intermediate container f1b88bb5a2d7
---> a6233ad21648
Step 19/61 : RUN echo test > /test
---> Running in 45f70577d709
Removing intermediate container 45f70577d709
---> 1b5cc52d370d
Step 20/61 : RUN echo test > /test
---> Running in 2ce231d5043d
Removing intermediate container 2ce231d5043d
---> 4a0e17cbebaa
Step 21/61 : RUN echo test > /test
---> Running in 52e4b0928f1f
Removing intermediate container 52e4b0928f1f
---> 99b50e989bcb
Step 22/61 : RUN echo test > /test
---> Running in f7ba3da7460d
Removing intermediate container f7ba3da7460d
---> bfa3cad88285
Step 23/61 : RUN echo test > /test
---> Running in 60180bf60f88
Removing intermediate container 60180bf60f88
---> fe7271988bcb
Step 24/61 : RUN echo test > /test
---> Running in 20324d396531
Removing intermediate container 20324d396531
---> e930bc039128
Step 25/61 : RUN echo test > /test
---> Running in b3ac70fd4404
Removing intermediate container b3ac70fd4404
---> 39d0a11ea6d8
Step 26/61 : RUN echo test > /test
---> Running in 0193267d3787
Removing intermediate container 0193267d3787
---> 8062d7aab0a5
Step 27/61 : RUN echo test > /test
---> Running in f41f45fb7985
Removing intermediate container f41f45fb7985
---> 1f5f18f2315b
Step 28/61 : RUN echo test > /test
---> Running in 90dd09c63d6e
Removing intermediate container 90dd09c63d6e
---> 02f0a1141f11
Step 29/61 : RUN echo test > /test
---> Running in c557e5386e0a
Removing intermediate container c557e5386e0a
---> dbcd6fb1f6f4
Step 30/61 : RUN echo test > /test
---> Running in 65369385d855
Removing intermediate container 65369385d855
---> e6e9058a0650
Step 31/61 : RUN echo test > /test
---> Running in d861fcc388fd
Removing intermediate container d861fcc388fd
---> 6e4c2c0f741f
Step 32/61 : RUN echo test > /test
---> Running in 1483962b7e1c
Removing intermediate container 1483962b7e1c
---> cf8f142aa055
Step 33/61 : RUN echo test > /test
---> Running in 5868934816c1
Removing intermediate container 5868934816c1
---> d5ff87cdc204
Step 34/61 : RUN echo test > /test
---> Running in e057f3201f3a
Removing intermediate container e057f3201f3a
---> b4031b7ab4ac
Step 35/61 : RUN echo test > /test
---> Running in 22b769b9079c
Removing intermediate container 22b769b9079c
---> 019d898510b6
Step 36/61 : RUN echo test > /test
---> Running in f1d364ef4ff8
Removing intermediate container f1d364ef4ff8
---> 9525cafdf04d
Step 37/61 : RUN echo test > /test
---> Running in 5bf505b8bdcc
Removing intermediate container 5bf505b8bdcc
---> cd5002b33bfd
Step 38/61 : RUN echo test > /test
---> Running in be24a921945c
Removing intermediate container be24a921945c
---> 8675db44d1b7
Step 39/61 : RUN echo test > /test
---> Running in 352dc6beef3d
Removing intermediate container 352dc6beef3d
---> 0ab0ece43c71
Step 40/61 : RUN echo test > /test
---> Running in eebde33e5d9b
Removing intermediate container eebde33e5d9b
---> 46ca4b0dfc03
Step 41/61 : RUN echo test > /test
---> Running in f920313a1e85
Removing intermediate container f920313a1e85
---> 7f3888414d58
Step 42/61 : RUN echo test > /test
---> Running in 10e2f4dc1ac7
Removing intermediate container 10e2f4dc1ac7
---> 14db9e15f2dc
Step 43/61 : RUN echo test > /test
---> Running in c849d6e89aa5
Removing intermediate container c849d6e89aa5
---> fdb770494dd6
Step 44/61 : RUN echo test > /test
---> Running in 419d1a8353db
Removing intermediate container 419d1a8353db
---> d12e9cf078be
Step 45/61 : RUN echo test > /test
---> Running in 0f1805263e4c
Removing intermediate container 0f1805263e4c
---> cd005e7b08a4
Step 46/61 : RUN echo test > /test
---> Running in 5bde05b46441
Removing intermediate container 5bde05b46441
---> 05aa426a3d4a
Step 47/61 : RUN echo test > /test
---> Running in 01ebc84bd1bc
Removing intermediate container 01ebc84bd1bc
---> 35d371fa4342
Step 48/61 : RUN echo test > /test
---> Running in 49f6c2f51dd4
Removing intermediate container 49f6c2f51dd4
---> 1090b5dfa130
Step 49/61 : RUN echo test > /test
---> Running in f8a9089cd725
Removing intermediate container f8a9089cd725
---> b2d0eec0716d
Step 50/61 : RUN echo test > /test
---> Running in a1697a0b2db0
Removing intermediate container a1697a0b2db0
---> 10d96ac8f497
Step 51/61 : RUN echo test > /test
---> Running in 33a2332c06eb
Removing intermediate container 33a2332c06eb
---> ba5bf5609c1c
Step 52/61 : RUN echo test > /test
---> Running in e8920392be0d
Removing intermediate container e8920392be0d
---> 5b3a95685c7e
Step 53/61 : RUN echo test > /test
---> Running in 4b9298587c65
Removing intermediate container 4b9298587c65
---> d4961a349141
Step 54/61 : RUN echo test > /test
---> Running in 8a0c960c2ba1
Removing intermediate container 8a0c960c2ba1
---> b413197fcfa2
Step 55/61 : RUN echo test > /test
---> Running in 536ee3b9596b
Removing intermediate container 536ee3b9596b
---> fc16b69b224a
Step 56/61 : RUN echo test > /test
---> Running in 8b817b8d7b59
Removing intermediate container 8b817b8d7b59
---> 2f0896400ff9
Step 57/61 : RUN echo test > /test
---> Running in ab0ed79ec3d4
Removing intermediate container ab0ed79ec3d4
---> b4fb420e736c
Step 58/61 : RUN echo test > /test
---> Running in 8548d7eead1f
Removing intermediate container 8548d7eead1f
---> 745103fd5a38
Step 59/61 : RUN echo test > /test
---> Running in 1980559ad5d6
Removing intermediate container 1980559ad5d6
---> 08c1c74a5618
Step 60/61 : FROM alpine
---> 11cd0b38bc3c
Step 61/61 : COPY --from=first /test /test
---> 67f053c66c27
Successfully built 67f053c66c27
PS E:\docker\build\36764>
```
Note also that subsequent error messages once you go beyond current platform limitations kind of suck (such as insufficient resources with a bunch of spew which is incomprehensible to most) and we could do better to detect this earlier in the daemon. That'll be for a (reasonably low-priority) follow-up though as and when I have time. Theoretically we *may*, if the platform doesn't require additional changes for RS5, be able to have bigger platform limits using the v2 schema with up to 127 VPMem devices, and the possibility to have multiple SCSI controllers per SVM/UVM. However, currently LCOW is using HCS v1 schema calls, and there's no plans to rewrite the graphdriver/libcontainerd components outside of the moving LCOW fully over to the containerd runtime/snapshotter using HCS v2 schema, which is still some time off fruition.
PS OK, while waiting for a full run to complete, I did get bored. Turns out it won't overflow line length as max(uint64) is 18446744073709551616 which would still be short enough at 127 layers, double the current platform limit. And I could always change it to hex or base36 to make it even shorter, or remove the 'd' from /tmp/dN. IOW, pretty sure no-one is going to hit the limit even if we could get the platform to 256 which is the current Hyper-V SCSI limit per VM (4x64), although PMEM at 127 would be the next immediate limit.
This implements chown support on Windows. Built-in accounts as well
as accounts included in the SAM database of the container are supported.
NOTE: IDPair is now named Identity and IDMappings is now named
IdentityMapping.
The following are valid examples:
ADD --chown=Guest . <some directory>
COPY --chown=Administrator . <some directory>
COPY --chown=Guests . <some directory>
COPY --chown=ContainerUser . <some directory>
On Windows an owner is only granted the permission to read the security
descriptor and read/write the discretionary access control list. This
fix also grants read/write and execute permissions to the owner.
Signed-off-by: Salahuddin Khan <salah@docker.com>
When go-1.11beta1 is used for building, the following error is
reported:
> 14:56:20 daemon\graphdriver\lcow\lcow.go:236: Debugf format %s reads
> arg #2, but call has 1 arg
While fixing this, let's also fix a few other things in this
very function (startServiceVMIfNotRunning):
1. Do not use fmt.Printf when not required.
2. Use `title` whenever possible.
3. Don't add `id` to messages as `title` already has it.
4. Remove duplicated colons.
5. Try to unify style of messages.
6. s/startservicevmifnotrunning/startServiceVMIfNotRunning/
...
In general, logging/debugging here is a mess and requires much more
love than I can give it at the moment. Areas for improvement:
1. Add a global var logger = logrus.WithField("storage-driver", "lcow")
and use it everywhere else in the code.
2. Use logger.WithField("id", id) whenever possible (same for "context"
and other similar fields).
3. Revise all the errors returned to be uniform.
4. Make use of errors.Wrap[f] whenever possible.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Fix the following go-1.11beta1 build error:
> daemon/graphdriver/aufs/aufs.go:376: Wrapf format %s reads arg #1, but call has 0 args
While at it, change '%s' to %q.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This makes it a bit simpler to remove this interface for v2 plugins
and not break external projects (libnetwork and swarmkit).
Note that before we remove the `Client()` interface from `CompatPlugin`
libnetwork and swarmkit must be updated to explicitly check for the v1
client interface as is done int his PR.
This is just a minor tweak that I realized is needed after trying to
implement the needed changes on libnetwork.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
In case aufs driver is not supported because supportsAufs() said so,
it is not possible to get a real reason from the logs.
To fix, log the error returned.
Note we're not using WithError here as the error message itself is the
sole message we want to print (i.e. there's nothing to add to it).
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
ext4 support d_type by default, but filetype feature is a tunable so
there is still a chance to disable it for some reasons. In this case,
print additional message to explicitly tell how to support d_type.
Signed-off-by: Chengguang Xu <cgxu519@gmx.com>
The overlay storage driver currently does not support any option, but was silently
ignoring any option that was passed.
This patch verifies that no options are passed, and if they are passed will produce
an error.
Before this change:
dockerd --storage-driver=overlay --storage-opt dm.thinp_percent=95
INFO[2018-05-11T11:40:40.996597152Z] libcontainerd: started new docker-containerd process pid=256
....
INFO[2018-05-11T11:40:41.135392535Z] Daemon has completed initialization
INFO[2018-05-11T11:40:41.141035093Z] API listen on /var/run/docker.sock
After this change:
dockerd --storage-driver=overlay --storage-opt dm.thinp_percent=95
INFO[2018-05-11T11:39:21.632610319Z] libcontainerd: started new docker-containerd process pid=233
....
Error starting daemon: error initializing graphdriver: overlay: unknown option dm.thinp_percent
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Functions `GetMounts()` and `parseMountTable()` return all the entries
as read and parsed from /proc/self/mountinfo. In many cases the caller
is only interested only one or a few entries, not all of them.
One good example is `Mounted()` function, which looks for a specific
entry only. Another example is `RecursiveUnmount()` which is only
interested in mount under a specific path.
This commit adds `filter` argument to `GetMounts()` to implement
two things:
1. filter out entries a caller is not interested in
2. stop processing if a caller is found what it wanted
`nil` can be passed to get a backward-compatible behavior, i.e. return
all the entries.
A few filters are implemented:
- `PrefixFilter`: filters out all entries not under `prefix`
- `SingleEntryFilter`: looks for a specific entry
Finally, `Mounted()` is modified to use `SingleEntryFilter()`, and
`RecursiveUnmount()` is using `PrefixFilter()`.
Unit tests are added to check filters are working.
[v2: ditch NoFilter, use nil]
[v3: ditch GetMountsFiltered()]
[v4: add unit test for filters]
[v5: switch to gotestyourself]
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
commit 617c352e92 "Don't create devices if in a user namespace"
introduced check, which meant to skip mknod operation when run
in user namespace, but instread skipped FIFO and socket files
copy.
Signed-off-by: Maxim Ivanov <ivanov.maxim@gmail.com>
