With both rootless and live restore enabled, there's some race condition
which causes the container to be `Unmount`ed before the refcount is
restored.
This makes sure we don't underflow the refcount (uint64) when
decrementing it.
The root cause of this race condition still needs to be investigated and
fixed, but at least this unflakies the `TestLiveRestore`.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
I made a mistake in the last commit - after resolving the IP from the
passed `addr` for CIFS it would still resolve the `device` part.
Apply only one name resolution
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Commit 8ae94cafa5 added a DNS resolution
of the `device` part of the volume option.
The previous way to resolve the passed hostname was to use `addr`
option, which was handled by the same code path as the `nfs` mount type.
The issue is that `addr` is also an SMB module option handled by kernel
and passing a hostname as `addr` produces an invalid argument error.
To fix that, restore the old behavior to handle `addr` the same way as
before, and only perform the new DNS resolution of `device` if there is
no `addr` passed.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
When creating a CIFS volume, generate an error if the device URL
includes a port number, for example:
--opt device="//some.server.com:2345/thepath"
The port must be specified in the port option instead, for example:
--opt o=username=USERNAME,password=PASSWORD,vers=3,sec=ntlmsspi,port=1234
Signed-off-by: Rob Murray <rob.murray@docker.com>
The github.com/containerd/containerd/log package was moved to a separate
module, which will also be used by upcoming (patch) releases of containerd.
This patch moves our own uses of the package to use the new module.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
On startup all local volumes were unmounted as a cleanup mechanism for
the non-clean exit of the last engine process.
This caused live-restored volumes that used special volume opt mount
flags to be broken. While the refcount was restored, the _data directory
was just unmounted, so all new containers mounting this volume would
just have the access to the empty _data directory instead of the real
volume.
With this patch, the mountpoint isn't unmounted. Instead, if the volume
is already mounted, just mark it as mounted, so the next time Mount is
called only the ref count is incremented, but no second attempt to mount
it is performed.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
When live-restoring a container the volume driver needs be notified that
there is an active mount for the volume.
Before this change the count is zero until the container stops and the
uint64 overflows pretty much making it so the volume can never be
removed until another daemon restart.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
The CreatedAt date was determined from the volume's `_data`
directory (`/var/lib/docker/volumes/<volumename>/_data`).
However, when initializing a volume, this directory is updated,
causing the date to change.
Instead of using the `_data` directory, use its parent directory,
which is not updated afterwards, and should reflect the time that
the volume was created.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
volume/mounts/parser_test.go:42:39: empty-lines: extra empty line at the end of a block (revive)
volume/mounts/windows_parser.go:129:24: empty-lines: extra empty line at the end of a block (revive)
volume/local/local_test.go:16:35: empty-lines: extra empty line at the end of a block (revive)
volume/local/local_unix.go:145:3: early-return: if c {...} else {... return } can be simplified to if !c { ... return } ... (revive)
volume/service/service_test.go:18:38: empty-lines: extra empty line at the end of a block (revive)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Note that Windows does not support options, so strictly doesn't need
to have this code, but keeping it in case we're adding support.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Unmounting does not depend on wether or not loading options failed.
This code-path seemed to be used as a "hack" to prevent hitting the
unmount on Windows (which does not support unmounting).
Moving it outside of the "if" to make more clear that it's independent
of loading the options.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Differentiate between Windows and Linux, as Windows doesn't support
options, so there's no need to save options to disk,
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Instead of evaluating these paths each time (appending `_data`, or using
`filepath.Dir()` to find the root path from the `_data_` path).
This also removes the `root.DataPath()` utility, which is now no longer needed.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This way we can validate if Root supports quotaCtl, allowing us to
fail early, before creating any of the directories.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This moves validation of options to the start of the Create function
to prevent hitting the filesystem and having to remove the volume
from disk.
Also addressing some minor nits w.r.t. errors.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
While the current code is correct (as errors.Wrap() returns nil if
err is nil), relying on this behavior has caused some confusion in
the past, resulting in regressions.
This patch makes the error-handling code slightly more idiomatic and
defensive against such regressions.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
In managed environment (such as Nomad clusters), users are not always
supposed to see credentials used to mount volumes.
However, if errors occur (most commonly, misspelled mount paths), the
error messages will output the full mount command -- which might contain
a username and a password in the case of CIFS mounts.
This PR detects password=... when error messages are wrapped and masks
them with ********.
Closes https://github.com/fsouza/go-dockerclient/issues/905.
Closes https://github.com/hashicorp/nomad/issues/12296.
Closes https://github.com/moby/moby/issues/43596.
Signed-off-by: Sebastian Höffner <sebastian.hoeffner@mevis.fraunhofer.de>
Now that there's no differentiation between Linux and Windows
for this check, we can remove the two implementations and move
the code inline as it's only used in a single location and moving
it inline makes it more transparent on what's being checked.
As part of this change, the now unused "scope" field is also removed.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This was added in bd9814f0db to support downgrading
docker 1.7 to 1.6.
The related migration code was removed in 0023abbad3
(Docker 18.05), which was also the last consumer of VolumeDataPathName outside
of the package, so that const can be un-exported.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The io/ioutil package has been deprecated in Go 1.16. This commit
replaces the existing io/ioutil functions with their new definitions in
io and os packages.
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
Various dirs in /var/lib/docker contain data that needs to be mounted
into a container. For this reason, these dirs are set to be owned by the
remapped root user, otherwise there can be permissions issues.
However, this uneccessarily exposes these dirs to an unprivileged user
on the host.
Instead, set the ownership of these dirs to the real root (or rather the
UID/GID of dockerd) with 0701 permissions, which allows the remapped
root to enter the directories but not read/write to them.
The remapped root needs to enter these dirs so the container's rootfs
can be configured... e.g. to mount /etc/resolve.conf.
This prevents an unprivileged user from having read/write access to
these dirs on the host.
The flip side of this is now any user can enter these directories.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit e908cc3901)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
full diff: https://github.com/moby/sys/compare/mountinfo/v0.1.3...mountinfo/v0.4.0
> Note that this dependency uses submodules, providing "github.com/moby/sys/mount"
> and "github.com/moby/sys/mountinfo". Our vendoring tool (vndr) currently doesn't
> support submodules, so we vendor the top-level moby/sys repository (which contains
> both) and pick the most recent tag, which could be either `mountinfo/vXXX` or
> `mount/vXXX`.
github.com/moby/sys/mountinfo v0.4.0
--------------------------------------------------------------------------------
Breaking changes:
- `PidMountInfo` is now deprecated and will be removed before v1.0; users should switch to `GetMountsFromReader`
Fixes and improvements:
- run filter after all fields are parsed
- correct handling errors from bufio.Scan
- documentation formatting fixes
github.com/moby/sys/mountinfo v0.3.1
--------------------------------------------------------------------------------
- mount: use MNT_* flags from golang.org/x/sys/unix on freebsd
- various godoc and CI fixes
- mountinfo: make GetMountinfoFromReader Linux-specific
- Add support for OpenBSD in addition to FreeBSD
- mountinfo: use idiomatic naming for fields
github.com/moby/sys/mountinfo v0.2.0
--------------------------------------------------------------------------------
Bug fixes:
- Fix path unescaping for paths with double quotes
Improvements:
- Mounted: speed up by adding fast paths using openat2 (Linux-only) and stat
- Mounted: relax path requirements (allow relative, non-cleaned paths, symlinks)
- Unescape fstype and source fields
- Documentation improvements
Testing/CI:
- Unit tests: exclude darwin
- CI: run tests under Fedora 32 to test openat2
- TestGetMounts: fix for Ubuntu build system
- Makefile: fix ignoring test failures
- CI: add cross build
github.com/moby/sys/mount v0.1.1
--------------------------------------------------------------------------------
https://github.com/moby/sys/releases/tag/mount%2Fv0.1.1
Improvements:
- RecursiveUnmount: add a fast path (#26)
- Unmount: improve doc
- fix CI linter warning on Windows
Testing/CI:
- Unit tests: exclude darwin
- Makefile: fix ignoring test failures
- CI: add cross build
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Switch to moby/sys/mount and mountinfo. Keep the pkg/mount for potential
outside users.
This commit was generated by the following bash script:
```
set -e -u -o pipefail
for file in $(git grep -l 'docker/docker/pkg/mount"' | grep -v ^pkg/mount); do
sed -i -e 's#/docker/docker/pkg/mount"#/moby/sys/mount"#' \
-e 's#mount\.\(GetMounts\|Mounted\|Info\|[A-Za-z]*Filter\)#mountinfo.\1#g' \
$file
goimports -w $file
done
```
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
This comes from an old suggestion (https://github.com/docker/cli/issues/706#issuecomment-371157691) on an issue we were having and has since popped up again. For NFS volumes, Docker will do an IP lookup on the volume name. This is not done for CIFS volumes, which forces you to add the volume via IP address instead. This change will enable the IP lookup also for CIFS volumes.
Signed-off-by: Shu-Wai Chow <shu-wai.chow@seattlechildrens.org>
Using `errors.Errorf()` passes the error with the stack trace for
debugging purposes.
Also using `errdefs.InvalidParameter` for Windows, so that the API
will return a 4xx status, instead of a 5xx, and added tests for
both validations.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
