Commit graph

2984 commits

Author SHA1 Message Date
Akihiro Suda
c1e7924f7c
archive: do not use overlayWhiteoutConverter for UserNS
overlay2 no longer sets `archive.OverlayWhiteoutFormat` when
running in UserNS, so we can remove the complicated logic in the
archive package.

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit 6322dfc217)
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-04-01 19:00:42 +09:00
Brian Goff
60aa0f2f6b
Merge pull request #42079 from thaJeztah/20.10_backport_update_docs_links
[20.10 backport] Update documentation links
2021-03-25 12:48:52 -07:00
Shengjing Zhu
a0670c6d3d
pkg/archive: TestUntarParentPathPermissions requires root
=== RUN   TestUntarParentPathPermissions
    archive_unix_test.go:171: assertion failed: error is not nil: chown /tmp/TestUntarParentPathPermissions694189715/foo: operation not permitted

Signed-off-by: Shengjing Zhu <zhsj@debian.org>
(cherry picked from commit f23c1c297d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-03-01 22:05:09 +01:00
Sebastiaan van Stijn
04d9b581e9
Update documentation links
- Using "/go/" redirects for some topics, which allows us to
  redirect to new locations if topics are moved around in the
  documentation.
- Updated some old URLs to their new location.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 328de0b8d9)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-02-25 21:54:39 +01:00
Sebastiaan van Stijn
148e6c9514
Merge pull request #42017 from thaJeztah/20.10_backport_build_fixes
[20.10 backport]: avoid creating parent dirs for XGlobalHeader, and fix permissions
2021-02-22 20:04:04 +01:00
Grant Millar
94d2467613
Fix userns-remap option when username & UID match
Signed-off-by: Grant Millar <rid@cylo.io>
(cherry picked from commit 2ad187fd4a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-02-17 21:14:40 +01:00
Sebastiaan van Stijn
5d442b1cb7
pkg/archive: Unpack() use 0755 permissions for missing directories
Commit edb62a3ace fixed a bug in MkdirAllAndChown()
that caused the specified permissions to not be applied correctly. As a result
of that bug, the configured umask would be applied.

When extracting archives, Unpack() used 0777 permissions when creating missing
parent directories for files that were extracted.
Before edb62a3ace, this resulted in actual
permissions of those directories to be 0755 on most configurations (using a
default 022 umask).

Creating these directories should not depend on the host's umask configuration.
This patch changes the permissions to 0755 to match the previous behavior,
and to reflect the original intent of using 0755 as default.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 25ada76437)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-02-17 21:12:57 +01:00
Tonis Tiigi
5db18e0aba
archive: avoid creating parent dirs for XGlobalHeader
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit ba7906aef3)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-02-17 21:12:55 +01:00
Akihiro Suda
a287e76e15
pkg/archive: allow mknodding FIFO inside userns
Fix #41803

Also attempt to mknod devices.
Mknodding devices are likely to fail, but still worth trying when
running with a seccomp user notification.

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit d5d5cccb7e)
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-02-02 14:29:49 +09:00
Tibor Vass
46229ca1d8 Use golang.org/x/sys/execabs
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 7ca0cb7ffa)
Signed-off-by: Tibor Vass <tibor@docker.com>
2021-01-28 21:33:12 +00:00
Brian Goff
edb62a3ace Ensure MkdirAllAndChown also sets perms
Generally if we ever need to change perms of a dir, between versions,
this ensures the permissions actually change when we think it should
change without having to handle special cases if it already existed.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2021-01-26 17:23:32 +00:00
Arnaud Rebillout
c7e7426796 pkg/chrootarchive: Skip privileged tests when non-root
These tests fail when run by a non-root user

  === RUN   TestUntarWithMaliciousSymlinks
      archive_unix_test.go:63: assertion failed: expected error to contain "open /safe/host-file: no such file or directory", got "Error processing tar file(exit status 1): Error creating mount namespace before pivot: operation not permitted"
  --- FAIL: TestUntarWithMaliciousSymlinks (0.00s)
  === RUN   TestTarWithMaliciousSymlinks
      archive_unix_test.go:90: /tmp/TestTarWithMaliciousSymlinks400408188
  === RUN   TestTarWithMaliciousSymlinks//tmp/TestTarWithMaliciousSymlinks400408188/root/safe_host-file
      archive_unix_test.go:155: assertion failed: error is not nil: error processing tar file: Error after fallback to chroot: operation not permitted: exit status 1
  === RUN   TestTarWithMaliciousSymlinks//tmp/TestTarWithMaliciousSymlinks400408188/root/safe/_host-file
      archive_unix_test.go:155: assertion failed: error is not nil: error processing tar file: Error after fallback to chroot: operation not permitted: exit status 1
  === RUN   TestTarWithMaliciousSymlinks//tmp/TestTarWithMaliciousSymlinks400408188/root/safe_
      archive_unix_test.go:155: assertion failed: error is not nil: error processing tar file: Error after fallback to chroot: operation not permitted: exit status 1
  === RUN   TestTarWithMaliciousSymlinks//tmp/TestTarWithMaliciousSymlinks400408188/root/safe/_
      archive_unix_test.go:155: assertion failed: error is not nil: error processing tar file: Error after fallback to chroot: operation not permitted: exit status 1
  === RUN   TestTarWithMaliciousSymlinks//tmp/TestTarWithMaliciousSymlinks400408188/root_safe/host-file
      archive_unix_test.go:155: assertion failed: error is not nil: error processing tar file: Error after fallback to chroot: operation not permitted: exit status 1
  === RUN   TestTarWithMaliciousSymlinks//tmp/TestTarWithMaliciousSymlinks400408188/root_/safe/host-file
      archive_unix_test.go:155: assertion failed: error is not nil: error processing tar file: Error after fallback to chroot: operation not permitted: exit status 1
  === RUN   TestTarWithMaliciousSymlinks//tmp/TestTarWithMaliciousSymlinks400408188/root_
      archive_unix_test.go:155: assertion failed: error is not nil: error processing tar file: Error after fallback to chroot: operation not permitted: exit status 1
  --- FAIL: TestTarWithMaliciousSymlinks (0.02s)
      --- FAIL: TestTarWithMaliciousSymlinks//tmp/TestTarWithMaliciousSymlinks400408188/root/safe_host-file (0.00s)
      --- FAIL: TestTarWithMaliciousSymlinks//tmp/TestTarWithMaliciousSymlinks400408188/root/safe/_host-file (0.00s)
      --- FAIL: TestTarWithMaliciousSymlinks//tmp/TestTarWithMaliciousSymlinks400408188/root/safe_ (0.00s)
      --- FAIL: TestTarWithMaliciousSymlinks//tmp/TestTarWithMaliciousSymlinks400408188/root/safe/_ (0.00s)
      --- FAIL: TestTarWithMaliciousSymlinks//tmp/TestTarWithMaliciousSymlinks400408188/root_safe/host-file (0.00s)
      --- FAIL: TestTarWithMaliciousSymlinks//tmp/TestTarWithMaliciousSymlinks400408188/root_/safe/host-file (0.00s)
      --- FAIL: TestTarWithMaliciousSymlinks//tmp/TestTarWithMaliciousSymlinks400408188/root_ (0.00s)
  FAIL
  FAIL	github.com/docker/docker/pkg/chrootarchive	0.042s

Signed-off-by: Arnaud Rebillout <elboulangero@gmail.com>
2020-12-15 09:47:48 +07:00
Sebastiaan van Stijn
be67385209
Merge pull request #41626 from cuonglm/master
pkg/devicemapper: fix invalid usage of reflect.SliceHeader
2020-11-16 23:05:50 +01:00
Sebastiaan van Stijn
2b41e84719
pkg/system: fix compile on darwin (macOS)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-11-10 20:49:09 +01:00
Sebastiaan van Stijn
0e8023ddea
Merge pull request #41638 from TBBle/37352-fix-Windows-CI-pipeline
Reinstate Windows CI Pipeline usefulness
2020-11-10 02:52:54 +01:00
Sebastiaan van Stijn
6458f750e1
use containerd/cgroups to detect cgroups v2
libcontainer does not guarantee a stable API, and is not intended
for external consumers.

this patch replaces some uses of libcontainer/cgroups with
containerd/cgroups.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-11-09 15:00:32 +01:00
Paul "TBBle" Hampson
a158b53d86 Separate non-Windows mount code from common code
Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
2020-11-08 23:15:15 +11:00
Sebastiaan van Stijn
dc3c382b34
replace pkg/symlink with github.com/moby/sys/symlink
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-11-03 11:17:12 +01:00
Cuong Manh Le
c208f03fbd
pkg/devicemapper: fix invalid usage of reflect.SliceHeader
The current usage of reflect.SliceHeader violates rule 6th of
unsafe.Pointer conversion. In short, reflect.SliceHeader could not be
used as plain struct.

See https://golang.org/pkg/unsafe/#Pointer

Signed-off-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
2020-11-03 14:20:52 +07:00
Sebastiaan van Stijn
e766361271
pkg/mount: update deprecated wrappers
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-10-29 23:04:20 +01:00
Sebastiaan van Stijn
32d506b394
vendor: moby/sys mountinfo/v0.4.0
full diff: https://github.com/moby/sys/compare/mountinfo/v0.1.3...mountinfo/v0.4.0

> Note that this dependency uses submodules, providing "github.com/moby/sys/mount"
> and "github.com/moby/sys/mountinfo". Our vendoring tool (vndr) currently doesn't
> support submodules, so we vendor the top-level moby/sys repository (which contains
> both) and pick the most recent tag, which could be either `mountinfo/vXXX` or
> `mount/vXXX`.

github.com/moby/sys/mountinfo v0.4.0
--------------------------------------------------------------------------------

Breaking changes:

- `PidMountInfo` is now deprecated and will be removed before v1.0; users should switch to `GetMountsFromReader`

Fixes and improvements:

- run filter after all fields are parsed
- correct handling errors from bufio.Scan
- documentation formatting fixes

github.com/moby/sys/mountinfo v0.3.1
--------------------------------------------------------------------------------

- mount: use MNT_* flags from golang.org/x/sys/unix on freebsd
- various godoc and CI fixes
- mountinfo: make GetMountinfoFromReader Linux-specific
- Add support for OpenBSD in addition to FreeBSD
- mountinfo: use idiomatic naming for fields

github.com/moby/sys/mountinfo v0.2.0
--------------------------------------------------------------------------------

Bug fixes:

- Fix path unescaping for paths with double quotes

Improvements:

- Mounted: speed up by adding fast paths using openat2 (Linux-only) and stat
- Mounted: relax path requirements (allow relative, non-cleaned paths, symlinks)
- Unescape fstype and source fields
- Documentation improvements

Testing/CI:

- Unit tests: exclude darwin
- CI: run tests under Fedora 32 to test openat2
- TestGetMounts: fix for Ubuntu build system
- Makefile: fix ignoring test failures
- CI: add cross build

github.com/moby/sys/mount v0.1.1
--------------------------------------------------------------------------------

https://github.com/moby/sys/releases/tag/mount%2Fv0.1.1

Improvements:

- RecursiveUnmount: add a fast path (#26)
- Unmount: improve doc
- fix CI linter warning on Windows

Testing/CI:

- Unit tests: exclude darwin
- Makefile: fix ignoring test failures
- CI: add cross build

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-10-29 23:02:28 +01:00
fanjiyun
1b8826beee docker stats: fix 'panic: close of closed channel'
Signed-off-by: fanjiyun <fan.jiyun@zte.com.cn>
2020-10-24 11:48:56 +08:00
Tianon Gravi
654cad4d9d Also trim "~..." from AppArmor versions
Signed-off-by: Tianon Gravi <admwiggin@gmail.com>
2020-10-08 17:03:51 -07:00
Akihiro Suda
ee079e4692
pkg/aaparser: support parsing version like "3.0.0-beta1"
Fix #41517

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-10-02 17:33:52 +09:00
Kir Kolyshkin
2502db66d0 pkg/system: make EnsureRemoveAll unix-specific
The tricks performed by EnsureRemoveAll only make sense for Linux and
other Unices, so separate it out, and make EnsureRemoveAll for Windows
just an alias of os.RemoveAll.

This makes sure RecursiveUnmount is not called on Windows.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2020-09-21 01:23:28 -07:00
Brian Goff
f99814d749
Merge pull request #41448 from iamleot/stat_bsd
Unify same fromStatT() implementation for FreeBSD and NetBSD
2020-09-15 16:53:08 -07:00
Sebastiaan van Stijn
1155b6bc7a
Merge pull request #41395 from cpuguy83/no_libseccomp
Remove dependency in dockerd on libseccomp
2020-09-15 17:37:04 +02:00
Leonardo Taccari
3d9120f744 Unify same fromStatT() implementation for FreeBSD and NetBSD
Suggested by Sebastiaan van Stijn, thanks!

Signed-off-by: Leonardo Taccari <leot@NetBSD.org>
2020-09-14 18:33:04 +02:00
Leonardo Taccari
6efa9e0575 Add support for NetBSD
Signed-off-by: Leonardo Taccari <leot@NetBSD.org>
2020-09-12 14:28:29 +02:00
Brian Goff
df7031b669 Memoize seccomp value for SysInfo
As it turns out, we call this function every time someone calls `docker
info`, every time a contianer is created, and every time a container is
started.
Certainly this should be refactored as a whole, but for now, memoize the
seccomp value.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2020-09-11 22:48:46 +00:00
Sebastiaan van Stijn
5ca758199d
replace pkg/locker with github.com/moby/locker
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-09-10 22:15:40 +02:00
Sebastiaan van Stijn
ea9886cec4
pkg/idtools: refactor to avoid string-splitting
The package used a lot of string-formatting, followed by string-splitting.
This looked to originate from attempts to use templating to allow future
extensibility (9a3ab0358e).

Looking at the history of the package, only a single update was made to
these templates, 5 years go, which makes it unlikely that more templating
will be needed.

This patch simplifies the handling of arguments to use `[]string` instead
of a single `string` (and splitting to a `[]string`). This both simplifies
the code somewhat, and prevents user/group-names containing spaces to be
splitted (causing, e.g. `getent` to fail).

Note that user/group-names containing spaces are invalid (or at least
discouraged), there are situations where such names may be used, so we
should avoid breaking on such names.

Before this change, a user/group name with a space in its name would fail;

    dockerd --userns-remap="user:domain users"
    INFO[2020-08-19T10:26:59.288868661+02:00] Starting up
    Error during groupname lookup for "domain users": getent unable to find entry "domain" in group database

With this change:

    # Add some possibly problematic usernames for testing
    # need to do this manually, as `adduser` / `useradd` won't accept these names
    echo 'user name1002:1002::/home/one:/bin/false' >> /etc/passwd; \
    echo 'user name1002:' >> /etc/group; \
    echo 'user name:1266401166:65536' >> /etc/subuid; \
    echo 'user name:1266401153:65536' >> /etc/subgid; \
    echo 'user$HOME1003:1003::/home/one:/bin/false' >> /etc/passwd; \
    echo 'user$HOME1003:' >> /etc/group; \
    echo 'user$HOME:1266401166:65536' >> /etc/subuid; \
    echo 'user$HOME:1266401153:65536' >> /etc/subgid; \
    echo 'user'"'"'name1004:1004::/home/one:/bin/false' >> /etc/passwd; \
    echo 'user'"'"'name1004:' >> /etc/group; \
    echo 'user'"'"'name:1266401166:65536' >> /etc/subuid; \
    echo 'user'"'"'name:1266401153:65536' >> /etc/subgid; \
    echo 'user"name1005:1005::/home/one:/bin/false' >> /etc/passwd; \
    echo 'user"name1005:' >> /etc/group; \
    echo 'user"name:1266401166:65536' >> /etc/subuid; \
    echo 'user"name:1266401153:65536' >> /etc/subgid;

    # Start the daemon using those users
    dockerd --userns-remap="user name:user name"
    dockerd --userns-remap='user$HOME:user$HOME'
    dockerd --userns-remap="user'name":"user'name"
    dockerd --userns-remap='user"name':'user"name'

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-08-20 12:24:38 +02:00
Till Wegmüller
e3ab236309 Add solaris support to lstat
Signed-off-by: Till Wegmüller <toasterson@gmail.com>
2020-08-12 11:40:58 +02:00
Brian Goff
cadd72d37d
Merge pull request #41208 from thaJeztah/remove_shellwords 2020-07-17 15:46:24 -07:00
Brian Goff
260c26b7be
Merge pull request #41016 from kolyshkin/cgroup-init 2020-07-16 11:26:52 -07:00
Sebastiaan van Stijn
3f36764844
Merge pull request #40994 from gaurav1086/rm_test_fix_goroutine_leak
rm_test: fix goroutine leak
2020-07-14 17:41:43 +02:00
Sebastiaan van Stijn
d4b7b4b0a7
Merge pull request #40941 from cpuguy83/down_with_the_func_init
Don't use init() to set unpigz path
2020-07-14 15:44:08 +02:00
Sebastiaan van Stijn
c9c9e0ab90
pkg/parsers: remove uses of mattn/go-shellwords
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-07-14 14:03:10 +02:00
Sebastiaan van Stijn
818e0b2fcf
pkg/parsers: add unit test for Darwin
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-07-14 14:02:54 +02:00
Kir Kolyshkin
afbeaf6f29 pkg/sysinfo: rm duplicates
The CPU CFS cgroup-aware scheduler is one single kernel feature, not
two, so it does not make sense to have two separate booleans
(CPUCfsQuota and CPUCfsPeriod). Merge these into CPUCfs.

Same for CPU realtime.

For compatibility reasons, /info stays the same for now.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2020-06-26 16:19:52 -07:00
Sebastiaan van Stijn
ab6b92b6b2
chrootarchive: fix "conversion from int to string yields a string of one rune"
update test to fix go 1.15 linting failure:

    pkg/chrootarchive/archive_test.go:103:32: conversion from int to string yields a string of one rune

relates to golang/go 32479

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-06-25 13:38:42 +02:00
Sebastiaan van Stijn
66bb1c4644
pkg/sysinfo: use containerd/sys to detect UserNamespaces
The implementation in libcontainer/system is quite complicated,
and we only use it to detect if user-namespaces are enabled.

In addition, the implementation in containerd uses a sync.Once,
so that detection (and reading/parsing `/proc/self/uid_map`) is
only performed once.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-06-15 13:07:48 +02:00
Sebastiaan van Stijn
9ff990a2b9
pkg/chrootarchive: use containerd/sys to detect UserNamespaces
The implementation in libcontainer/system is quite complicated,
and we only use it to detect if user-namespaces are enabled.

In addition, the implementation in containerd uses a sync.Once,
so that detection (and reading/parsing `/proc/self/uid_map`) is
only performed once.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-06-15 13:07:23 +02:00
Sebastiaan van Stijn
f7d5d70e44
pkg/archive: use containerd/sys to detect UserNamespaces
The implementation in libcontainer/system is quite complicated,
and we only use it to detect if user-namespaces are enabled.

In addition, the implementation in containerd uses a sync.Once,
so that detection (and reading/parsing `/proc/self/uid_map`) is
only performed once.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-06-15 13:06:47 +02:00
Sebastiaan van Stijn
3aac5f0bbb
Merge pull request #41018 from akhilerm/identity-mapping
remove group name from identity mapping
2020-06-08 15:15:05 +02:00
Akhil Mohan
7ad0da7051
remove group name from identity mapping
NewIdentityMapping took group name as an argument, and used
the group name also to parse the /etc/sub{uid,gui}. But as per
linux man pages, the sub{uid,gid} file maps username or uid,
not a group name.

Therefore, all occurrences where mapping is used need to
consider only username and uid. Code trying to map using gid
and group name in the daemon is also removed.

Signed-off-by: Akhil Mohan <akhil.mohan@mayadata.io>
2020-06-03 20:04:42 +05:30
Akihiro Suda
41ac6bef8d
Merge pull request #41026 from XiaodongLoong/master
fix "stat.Rdev" invalid operation mismatched types on mips64el
2020-05-26 16:51:31 +09:00
Xiaodong Liu
a366fe41cb fix "stat.Rdev" invalid operation mismatched types on mips64el
compile error the "stat.Rdev" variable and "s.Rdev" mismatched types on mips64el
convert "stat.Rdev" type to uint64 explicitly

Signed-off-by: Xiaodong Liu <liuxiaodong@loongson.cn>
2020-05-26 14:50:06 +08:00
Kir Kolyshkin
d5da7e5330 pkg/sysinfo/sysinfo_linux.go: fix some comments
Some were misleading or vague, some were plain wrong.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2020-05-22 13:13:27 -07:00
Kir Kolyshkin
f02a53d6b9 pkg/sysinfo.applyPIDSCgroupInfo: optimize
For some reason, commit 69cf03700f chose not to use information
already fetched, and called cgroups.FindCgroupMountpoint() instead.
This is not a cheap call, as it has to parse the whole nine yards
of /proc/self/mountinfo, and the info which it tries to get (whether
the pids controller is present) is already available from cgMounts map.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2020-05-22 13:13:27 -07:00