The tricks performed by EnsureRemoveAll only make sense for Linux and
other Unices, so separate it out, and make EnsureRemoveAll for Windows
just an alias of os.RemoveAll.
This makes sure RecursiveUnmount is not called on Windows.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
As it turns out, we call this function every time someone calls `docker
info`, every time a contianer is created, and every time a container is
started.
Certainly this should be refactored as a whole, but for now, memoize the
seccomp value.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
The package used a lot of string-formatting, followed by string-splitting.
This looked to originate from attempts to use templating to allow future
extensibility (9a3ab0358e).
Looking at the history of the package, only a single update was made to
these templates, 5 years go, which makes it unlikely that more templating
will be needed.
This patch simplifies the handling of arguments to use `[]string` instead
of a single `string` (and splitting to a `[]string`). This both simplifies
the code somewhat, and prevents user/group-names containing spaces to be
splitted (causing, e.g. `getent` to fail).
Note that user/group-names containing spaces are invalid (or at least
discouraged), there are situations where such names may be used, so we
should avoid breaking on such names.
Before this change, a user/group name with a space in its name would fail;
dockerd --userns-remap="user:domain users"
INFO[2020-08-19T10:26:59.288868661+02:00] Starting up
Error during groupname lookup for "domain users": getent unable to find entry "domain" in group database
With this change:
# Add some possibly problematic usernames for testing
# need to do this manually, as `adduser` / `useradd` won't accept these names
echo 'user name❌1002:1002::/home/one:/bin/false' >> /etc/passwd; \
echo 'user name❌1002:' >> /etc/group; \
echo 'user name:1266401166:65536' >> /etc/subuid; \
echo 'user name:1266401153:65536' >> /etc/subgid; \
echo 'user$HOME❌1003:1003::/home/one:/bin/false' >> /etc/passwd; \
echo 'user$HOME❌1003:' >> /etc/group; \
echo 'user$HOME:1266401166:65536' >> /etc/subuid; \
echo 'user$HOME:1266401153:65536' >> /etc/subgid; \
echo 'user'"'"'name❌1004:1004::/home/one:/bin/false' >> /etc/passwd; \
echo 'user'"'"'name❌1004:' >> /etc/group; \
echo 'user'"'"'name:1266401166:65536' >> /etc/subuid; \
echo 'user'"'"'name:1266401153:65536' >> /etc/subgid; \
echo 'user"name❌1005:1005::/home/one:/bin/false' >> /etc/passwd; \
echo 'user"name❌1005:' >> /etc/group; \
echo 'user"name:1266401166:65536' >> /etc/subuid; \
echo 'user"name:1266401153:65536' >> /etc/subgid;
# Start the daemon using those users
dockerd --userns-remap="user name:user name"
dockerd --userns-remap='user$HOME:user$HOME'
dockerd --userns-remap="user'name":"user'name"
dockerd --userns-remap='user"name':'user"name'
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Seeing this `ErrNotImplements` in some of our logs and it's not very
helpful because we don't know what plugin is causing it or even what the
requested interface is.
```
{"message":"legacy plugin: Plugin does not implement the requested driver"}
```
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
The CPU CFS cgroup-aware scheduler is one single kernel feature, not
two, so it does not make sense to have two separate booleans
(CPUCfsQuota and CPUCfsPeriod). Merge these into CPUCfs.
Same for CPU realtime.
For compatibility reasons, /info stays the same for now.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
update test to fix go 1.15 linting failure:
pkg/chrootarchive/archive_test.go:103:32: conversion from int to string yields a string of one rune
relates to golang/go 32479
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The implementation in libcontainer/system is quite complicated,
and we only use it to detect if user-namespaces are enabled.
In addition, the implementation in containerd uses a sync.Once,
so that detection (and reading/parsing `/proc/self/uid_map`) is
only performed once.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The implementation in libcontainer/system is quite complicated,
and we only use it to detect if user-namespaces are enabled.
In addition, the implementation in containerd uses a sync.Once,
so that detection (and reading/parsing `/proc/self/uid_map`) is
only performed once.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The implementation in libcontainer/system is quite complicated,
and we only use it to detect if user-namespaces are enabled.
In addition, the implementation in containerd uses a sync.Once,
so that detection (and reading/parsing `/proc/self/uid_map`) is
only performed once.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
NewIdentityMapping took group name as an argument, and used
the group name also to parse the /etc/sub{uid,gui}. But as per
linux man pages, the sub{uid,gid} file maps username or uid,
not a group name.
Therefore, all occurrences where mapping is used need to
consider only username and uid. Code trying to map using gid
and group name in the daemon is also removed.
Signed-off-by: Akhil Mohan <akhil.mohan@mayadata.io>
compile error the "stat.Rdev" variable and "s.Rdev" mismatched types on mips64el
convert "stat.Rdev" type to uint64 explicitly
Signed-off-by: Xiaodong Liu <liuxiaodong@loongson.cn>
For some reason, commit 69cf03700f chose not to use information
already fetched, and called cgroups.FindCgroupMountpoint() instead.
This is not a cheap call, as it has to parse the whole nine yards
of /proc/self/mountinfo, and the info which it tries to get (whether
the pids controller is present) is already available from cgMounts map.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
`func init()` is evil here, and the logrus calls are being made before
the logger is even setup.
It also means in order to use pigz you have to restart the daemon.
Instead this takes a small hit and resolves pigz on each extraction.
In the grand scheme of decompressing this is a very small hit.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
On macOS, unit tests were failing with
root@c4101a75c792:/go/src/github.com/docker/docker/pkg/authorization# go test .
--- FAIL: TestAuthZRequestPluginError (0.00s)
authz_unix_test.go:295: listen unix authz-test-plugin.sock: bind: file name too long
--- FAIL: TestAuthZRequestPlugin (0.00s)
authz_unix_test.go:295: listen unix authz-test-plugin.sock: bind: file name too long
--- FAIL: TestAuthZResponsePlugin (0.00s)
authz_unix_test.go:295: listen unix authz-test-plugin.sock: bind: file name too long
time="2020-04-07T10:07:04Z" level=warning msg="Request body is larger than: '1048576' skipping body"
--- FAIL: TestMiddlewareWrapHandler (0.00s)
authz_unix_test.go:295: listen unix authz-test-plugin.sock: bind: file name too long
FAIL
FAIL github.com/docker/docker/pkg/authorization 0.120s
This change moves the socket creation from a working test directory to a tmp directory,
so the path is shorter.
Change-type: patch
Signed-off-by: Roman Mazur <roman@balena.io>
We do our CI via golangci-lint, which understands nolint: annotations.
A standalone linter tool, golint, does not, and it insists on
documenting these:
> pkg/mount/deprecated.go:47:1: comment on exported var MergeTmpfsOptions should be of the form "MergeTmpfsOptions ..."
> pkg/mount/deprecated.go:51:1: comment on exported type FilterFunc should be of the form "FilterFunc ..." (with optional leading article)
> pkg/mount/deprecated.go:51:1: comment on exported type Info should be of the form "Info ..." (with optional leading article)
For `MergeTmpfsOptions`, the workaround is to put it inside a
`var ( ... )` block.
For the other two warnings, we have to provide the "actual"
documentation (or something that looks like it).
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Add a deprecation/removal notice, pointing out to appropriate
replacement packages.
I was not sure if a package-level deprecation is enough, so
I also added notices around each block.
Note that `nolint:golint` annotations are left as is, otherwise
golint complains like this:
> pkg/mount/deprecated.go:45:1: comment on exported var `MergeTmpfsOptions` should be of the form `MergeTmpfsOptions ...` (golint)
> // Deprecated: use github.com/moby/sys/mount instead.
> ^
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Switch to moby/sys/mount and mountinfo. Keep the pkg/mount for potential
outside users.
This commit was generated by the following bash script:
```
set -e -u -o pipefail
for file in $(git grep -l 'docker/docker/pkg/mount"' | grep -v ^pkg/mount); do
sed -i -e 's#/docker/docker/pkg/mount"#/moby/sys/mount"#' \
-e 's#mount\.\(GetMounts\|Mounted\|Info\|[A-Za-z]*Filter\)#mountinfo.\1#g' \
$file
goimports -w $file
done
```
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Make pkg/mount a shallow package, relying on
github.com/moby/sys/mount and github.com/moby/sys/mountinfo
The plan is to
- switch the rest of this repo to use moby/sys directly
- add deprecation notice to pkg/mount
- (eventually) remove pkg/mount
The nolint:golint annotation is added to suppress warnings like
"exported XXX should have comment or be unexported".
The ForceMount function is deprecated (and is identical to Mount),
so let's not use it (but still provide an alias for those who do).
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Currently, the escapeProxy works under the assumption that the
underlying reader will always return 1 byte at a time. Even though this
is usually true, it is not always the case, for example when using a pty
and writing multiple bytes to the master before flushing it.
In such cases the proxy reader doesn't work properly. For example with
an escape sequence being `ctrl-p,ctrl-q`, when the underlying reader
returns `ctrl-p,ctrl-q` at once, the escape sequence isn't detected.
This updates the reader to support this use-case and adds unit tests.
Signed-off-by: Bilal Amarni <bilal.amarni@gmail.com>
error log :
signal_test.go:20: assertion failed: error is not nil: Invalid signal: SIGEMT
signal_test.go:22: assertion failed:
When "ParseSignal" function parse sigStr from SignalMap, it find the signal object with key ("SIG"+sigStr). But EMT signal named "SIGEMT" in SignalMap structrue, so the real key is "SIGSIGEMT" , and cannot find the target signal.
modify "SIGEMT" to "EMT" in SignalMap structrue.
Signed-off-by: liuxiaodong <liuxiaodong@loongson.cn>
We recently updated golangci-lint, which is checking for some
additional linting rules, causing a failure in code that was
just merged to master; 5bd02b8a86
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
It makes sense to use mount package here because
- it no longer requires /proc to be mounted
- it provides verbose errors so the caller doesn't have to
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
This was added in PR #6669 (commit f87afda123) because it was
otherwise impossible to do a re-mount of already mounted file system.
It is way better to just remove the Mounted() check altogether.
This change might potentially lead to multiple mounts to the same
mount point, so I audited all the users (except tests) and it looks
like no one is doing that:
* volume/local maintains 'mounted' flag for every volume
* pkg/chrootarchive already calls Mounted() before Mount()
(so it actually parsed /proc/self/mountinfo twice, oops!)
* daemon.mountVolumes() is called for docker cp only, and
it is called once
* daemon/graphdriver/zfs keeps track of 'mounted' status
* daemon/graphdriver/devmapper: ditto
* daemon.createSecretsDir() is only called once during container start
Surely I might have easily missed something so this needs a careful
review.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Calling mount.Mounted() after an error from Unmount() is
questionable -- if umount failed, the mount is probably
still there anyway, it doesn't make sense to check it.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
1. Call to mount.Mounted() is very expensive and it's redundant
to call it before Unmount().
2. Calling mount.Mounted() after an error from Unmount() is
questionable -- if umount failed, the mount is probably
still there anyway, it doesn't make sense to check it.
This should result in faster code with no change in functionality.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
This function was added in 9c4570a958,
but appears to never have been used.
Removing it, as it's not used in the codebase and, from a quick
search on GitHub, also doesn't look to be used by other projects.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
follow-up to 069fdc8a08, replacing
more uses of the syscall package in favor of their "windows"
equivalents in golang.org/x/sys.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
lgetxattr(2) man page says:
> If size is specified as zero, these calls return the current size of
> the named extended attribute (and leave value unchanged). This can be
> used to determine the size of the buffer that should be supplied in a
> subsequent call. (But, bear in mind that there is a possibility that
> the attribute value may change between the two calls, so that it is
> still necessary to check the return status from the second call.)
The current code does not handle the case when the size changes between
the two calls, and the new size is larger.
Fix the above problem, and slightly simplify the code.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
enable resource limitation by disabling cgroup v1 warnings
resource limitation still doesn't work with rootless mode (even with systemd mode)
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
If `unix.Lgetxattr` returns an error, then `sz == -1` which will cause a
runtime panic if `errno == unix.ERANGE`.
Signed-off-by: Sascha Grunert <sgrunert@suse.com>
This clarifies comments about static linking made in commit a8608b5b67.
1. There are two ways to create a static binary, one is to disable
cgo, the other is to set linker flags. When cgo is disabled,
there is no need to use osusergo build tag.
2. osusergo only needs to be set when linking against glibc.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
TL;DR: there is no way to do this right.
We do know that in some combination of build tags set (or unset),
linker flags, environment variables, and libc implementation,
this package won't work right. In fact, there is one specific
combination:
1. `CGO_ENABLED=1` (or unset)
2. static binary is being built (e.g. `go build` is run with `-extldflags -static`)
3. `go build` links the binary against glibc
4. `osusergo` is not set
This particular combination results in the following legitimate linker warning:
> cgo_lookup_unix.go: warning: Using 'getpwuid_r' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
If this warning is ignored and the resulting binary is used on a system
with files from a different glibc version (or without those files), it
could result in a segfault.
The commit being reverted tried to guard against such possibility,
but the problem is, we can only use build tags to account for items
1 and 4 from the above list, while items 2 and 3 do not result in
any build tags being set or unset, making this guard excessive.
Remove it.
This reverts commit 023b072288.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
```
pkg/mount/mountinfo_linux.go:93:5: SA4011: ineffective break statement. Did you mean to break out of the outer loop? (staticcheck)
break
^
```
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Also fixed some incorrectly formatted comments
```
pkg/jsonmessage/jsonmessage.go:180:20: SA1006: printf-style function with dynamic format string and no further arguments should use print-style function instead (staticcheck)
fmt.Fprintf(out, endl)
^
```
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This struct now has a properly typed member, so use the properly typed
functions with it.
Also update the vendor directory and hope nothing explodes.
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This is to ensure that users of the homedir package cannot
compile statically (CGO_ENABLED=0) without also setting the osusergo
build tag.
Signed-off-by: Tibor Vass <tibor@docker.com>
About github.com/opencontainers/runc/libcontainer/user:
According to 195d8d544a
this package has two functions:
- Have a static implementation of user lookup, which is now supported in the
os/user stdlib package with the osusergo build tag, but wasn't at the time.
- Have extra functions that os/user doesn't have, but none of those are used
in homedir.
Since https://github.com/moby/moby/pull/11287, homedir depended directly on
libcontainer's user package for CurrentUser().
This is being replaced with os/user.Current(), because all of our static
binaries are compiled with the osusergo tag, and for dynamic libraries it
is more correct to use libc's implementation than parsing /etc/passwd.
About github.com/docker/docker/pkg/idtools:
Only dependency was from GetStatic() which uses idtools.LookupUID(uid).
The implementation of idtools.LookupUID just calls to
github.com/opencontainers/runc/libcontainer/user.LookupUid or fallbacks
to exec-ing to getent (since https://github.com/moby/moby/pull/27599).
This patch replaces calls to homedir.GetStatic by homedir.Get(), opting out
of supporting nss lookups in static binaries via exec-ing to getent for
the homedir package.
If homedir package users need to support nss lookups, they are advised
to compile dynamically instead.
Signed-off-by: Tibor Vass <tibor@docker.com>
Staticcheck reported:
SA9002: file mode '600' evaluates to 01130; did you mean '0600'? (staticcheck)
But fixing that caused the test to fail:
=== Failed
=== FAIL: pkg/filenotify TestPollerEvent (0.80s)
poller_test.go:75: timeout waiting for event CHMOD
The problem turned out to be that the file was created with `0644`. However,
after umask, the file created actually had `0600` filemode. Running the `os.Chmod`
with `0600` therefore was a no-op, causing the test to fail (because no
CHMOD event would fire).
This patch changes the test to;
- create the file with mode `0600`
- assert that the file has the expected mode
- change the chmod to `0644`
- assert that it has the correct mode, before testing the event.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
```
pkg/devicemapper/devmapper_wrapper.go:209:206: SA4000: identical expressions on the left and right side of the '==' operator (staticcheck)
```
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
As pointed out by govet,
> pkg/jsonmessage/jsonmessage_test.go:231:94: nilness: nil dereference in dynamic method call (govet)
> if err := DisplayJSONMessagesStream(reader, data, inFd, false, nil); err == nil && err.Error()[:17] != "invalid character" {
> ^
The nil deref never happened as err was always non-nil, and so the check
for error message text was not performed.
Fix this, and while at it, refactor the code a bit.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Fix the following warnings from staticcheck linter:
```
pkg/term/term_linux_test.go:34:2: SA5001: should check returned error before deferring tty.Close() (staticcheck)
defer tty.Close()
^
pkg/term/term_linux_test.go:52:2: SA5001: should check returned error before deferring tty.Close() (staticcheck)
defer tty.Close()
^
pkg/term/term_linux_test.go:67:2: SA5001: should check returned error before deferring tty.Close() (staticcheck)
defer tty.Close()
^
....
```
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
opts/env_test: suppress a linter warning
this one:
> opts/env_test.go:95:4: U1000: field `err` is unused (unused)
> err error
> ^
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Format the source according to latest goimports.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
```
13:06:14 pkg/directory/directory_test.go:182:2: S1032: should use sort.Strings(...) instead of sort.Sort(sort.StringSlice(...)) (gosimple)
13:06:14 sort.Sort(sort.StringSlice(results))
```
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
```
13:06:14 pkg/pools/pools.go:75:13: SA6002: argument should be pointer-like to avoid allocations (staticcheck)
13:06:14 bp.pool.Put(b)
```
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
There is a race condition in pkg/archive when using `cmd.Start` for pigz
and xz where the `*bufio.Reader` could be returned to the pool while the
command is still writing to it, and then picked up and used by a new
command.
The command is wrapped in a `CommandContext` where the process will be
killed when the context is cancelled, however this is not instantaneous,
so there's a brief window while the command is still running but the
`*bufio.Reader` was already returned to the pool.
wrapReadCloser calls `cancel()`, and then `readBuf.Close()` which
eventually returns the buffer to the pool. However, because cmdStream
runs `cmd.Wait` in a go routine that we never wait for to finish, it is
not safe to return the reader to the pool yet. We need to ensure we
wait for `cmd.Wait` to finish!
Signed-off-by: Stephen Benjamin <stephen@redhat.com>
CheckSystemDriveAndRemoveDriveLetter depends on pathdriver.PathDriver
unnecessarily. This depends on the minimal interface that it actually
needs, to avoid callers from unnecessarily bringing in a
containerd/continuity dependency.
Signed-off-by: Jon Johnson <jonjohnson@google.com>
Support for GOOS=solaris was removed in PR #35373. Remove two leftover
*_solaris.go files missed in this PR.
Signed-off-by: Tobias Klauser <tklauser@distanz.ch>
The `ioctl` interface for the `LOOP_CTL_GET_FREE` request on
`/dev/loop-control` is a little different from what `unix.IoctlGetInt`
expects: the first index is the returned status in `r1`, not an `int`
pointer as the first parameter.
Unfortunately we have to go a little lower level to get the appropriate
loop device index out, using `unix.Syscall` directly to read from
`r1`. Internally, the index is returned as a signed integer to match the
internal `ioctl` expectations of interpreting a negative signed integer
as an error at the userspace ABI boundary, so the direct interface of
`ioctlLoopCtlGetFree` can remain as-is.
[@kolyshkin: it still worked before this fix because of
/dev scan fallback in ioctlLoopCtlGetFree()]
Signed-off-by: Daniel Sweet <danieljsweet@icloud.com>
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
- use subtests to make it clearer what the individual test-cases
are, and to prevent tests from depending on values set by the
previous test(s).
- remove redundant messages in assert (gotest.tools already prints
a useful message if assertions fail).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
also renamed the non-windows variant of this file to be
consistent with other files in this package
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
environment not in the chroot from untrusted files.
See also OpenVZ a3f732ef75/src/enter.c (L227-L234)
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
(cherry picked from commit cea6dca993c2b4cfa99b1e7a19ca134c8ebc236b)
Signed-off-by: Tibor Vass <tibor@docker.com>
This fix was added in 8e71b1e210 to work around
a go issue (https://github.com/golang/go/issues/20506).
That issue was fixed in
66c03d39f3,
which is part of Go 1.10 and up. This reverts the changes that were made in
8e71b1e210, and are no longer needed.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Not really bullet-proof, users can still create cgroups with name like
"foo:/init.scope" or "\nfoo" to bypass the detection. However, solving
these cases will require kernel to provide a better interface.
Signed-off-by: Robert Wang <robert@arctic.tw>
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
This can happen when you have --config-only network
Such attempt will fail anyway and it will create 15s delay in container
startup
Signed-off-by: Pavel Matěja <pavel@verotel.cz>
Use Getpid and SchedGetaffinity from golang.org/x/sys/unix to get the
number of CPUs in numCPU on Linux.
Signed-off-by: Tobias Klauser <tklauser@distanz.ch>
