Commit graph

116 commits

Author SHA1 Message Date
Brian Goff
74da6a6363 Switch all logging to use containerd log pkg
This unifies our logging and allows us to propagate logging and trace
contexts together.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2023-06-24 00:23:44 +00:00
Sebastiaan van Stijn
ab35df454d
remove pre-go1.17 build-tags
Removed pre-go1.17 build-tags with go fix;

    go mod init
    go fix -mod=readonly ./...
    rm go.mod

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-19 20:38:51 +02:00
Tobias Klauser
4ec063fade
pkg/sysinfo: omit Getpid call in numCPU
The man page for sched_setaffinity(2) states the following about the pid
argument [1]:

> If pid is zero, then the mask of the calling thread is returned.

Thus the additional call to unix.Getpid can be omitted and pid = 0
passed to unix.SchedGetaffinity.

[1] https://man7.org/linux/man-pages/man2/sched_setaffinity.2.html#DESCRIPTION

Signed-off-by: Tobias Klauser <tklauser@distanz.ch>
2023-04-25 10:05:20 +02:00
Sebastiaan van Stijn
2d49080056
pkg/sysinfo: move MemInfo and ReadMemInfo to a separate package
Commit 6a516acb2e moved the MemInfo type and
ReadMemInfo() function into the pkg/sysinfo package. In an attempt to assist
consumers of these to migrate to the new location, an alias was added.

Unfortunately, the side effect of this alias is that pkg/system now depends
on pkg/sysinfo, which means that consumers of this (such as docker/cli) now
get all (indirect) dependencies of that package as dependency, which includes
many dependencies that should only be needed for the daemon / runtime;

- github.com/cilium/ebpf
- github.com/containerd/cgroups
- github.com/coreos/go-systemd/v22
- github.com/godbus/dbus/v5
- github.com/moby/sys/mountinfo
- github.com/opencontainers/runtime-spec

This patch moves the MemInfo related code to its own package. As the previous move
was not yet part of a release, we're not adding new aliases in pkg/sysinfo.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-03-15 17:52:45 +01:00
Akihiro Suda
e807ae4f2e
vendor: github.com/containerd/cgroups/v3 v3.0.1
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2023-03-08 20:15:17 +09:00
Sebastiaan van Stijn
ec878a3d89
pkg/sysinfo: unify ReadMemInfo implementation
Use a single exported implementation, so that we can maintain the
GoDoc string in one place, and use non-exported functions for the
actual implementation.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-21 10:53:45 +01:00
Sebastiaan van Stijn
09ea6744fe
pkg/sysinfo: remove github.com/docker/go-units dependency
It was only used in a test, and only for a single const; define
it locally.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-21 10:53:45 +01:00
Sebastiaan van Stijn
6a516acb2e
pkg/system: move memory-info types to pkg/systeminfo
These types and functions are more closely related to the functionality
provided by pkg/systeminfo, and used in conjunction with the other functions
in that package, so moving them there.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-21 10:53:45 +01:00
Sebastiaan van Stijn
6919b9879b
pkg/sysinfo: unify NumCPU implementation
Use a single exported implementation, so that we can maintain the
GoDoc string in one place, and use non-exported functions for the
actual implementation (which were already in place).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-21 10:53:42 +01:00
Sebastiaan van Stijn
f73d72bfdc
pkg: replace some README's with GoDoc package descriptions
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-30 17:11:37 +02:00
Sebastiaan van Stijn
412c650e05
pkg/*: fix "empty-lines" (revive)
pkg/directory/directory.go:9:49: empty-lines: extra empty line at the start of a block (revive)
    pkg/pubsub/publisher.go:8:48: empty-lines: extra empty line at the start of a block (revive)
    pkg/loopback/attach_loopback.go:96:69: empty-lines: extra empty line at the start of a block (revive)
    pkg/devicemapper/devmapper_wrapper.go:136:48: empty-lines: extra empty line at the start of a block (revive)
    pkg/devicemapper/devmapper.go:391:35: empty-lines: extra empty line at the end of a block (revive)
    pkg/devicemapper/devmapper.go:676:35: empty-lines: extra empty line at the end of a block (revive)
    pkg/archive/changes_posix_test.go:15:38: empty-lines: extra empty line at the end of a block (revive)
    pkg/devicemapper/devmapper.go:241:51: empty-lines: extra empty line at the start of a block (revive)
    pkg/fileutils/fileutils_test.go:17:47: empty-lines: extra empty line at the end of a block (revive)
    pkg/fileutils/fileutils_test.go:34:48: empty-lines: extra empty line at the end of a block (revive)
    pkg/fileutils/fileutils_test.go:318:32: empty-lines: extra empty line at the end of a block (revive)
    pkg/tailfile/tailfile.go:171:6: empty-lines: extra empty line at the end of a block (revive)
    pkg/tarsum/fileinfosums_test.go:16:41: empty-lines: extra empty line at the end of a block (revive)
    pkg/tarsum/tarsum_test.go:198:42: empty-lines: extra empty line at the start of a block (revive)
    pkg/tarsum/tarsum_test.go:294:25: empty-lines: extra empty line at the start of a block (revive)
    pkg/tarsum/tarsum_test.go:407:34: empty-lines: extra empty line at the end of a block (revive)
    pkg/ioutils/fswriters_test.go:52:45: empty-lines: extra empty line at the end of a block (revive)
    pkg/ioutils/writers_test.go:24:39: empty-lines: extra empty line at the end of a block (revive)
    pkg/ioutils/bytespipe_test.go:78:26: empty-lines: extra empty line at the end of a block (revive)
    pkg/sysinfo/sysinfo_linux_test.go:13:37: empty-lines: extra empty line at the end of a block (revive)
    pkg/archive/archive_linux_test.go:57:64: empty-lines: extra empty line at the end of a block (revive)
    pkg/archive/changes.go:248:72: empty-lines: extra empty line at the start of a block (revive)
    pkg/archive/changes_posix_test.go:15:38: empty-lines: extra empty line at the end of a block (revive)
    pkg/archive/copy.go:248:124: empty-lines: extra empty line at the end of a block (revive)
    pkg/archive/diff_test.go:198:44: empty-lines: extra empty line at the end of a block (revive)
    pkg/archive/archive.go:304:12: empty-lines: extra empty line at the end of a block (revive)
    pkg/archive/archive.go:749:37: empty-lines: extra empty line at the end of a block (revive)
    pkg/archive/archive.go:812:81: empty-lines: extra empty line at the start of a block (revive)
    pkg/archive/copy_unix_test.go:347:34: empty-lines: extra empty line at the end of a block (revive)
    pkg/system/path.go:11:39: empty-lines: extra empty line at the end of a block (revive)
    pkg/system/meminfo_linux.go:29:21: empty-lines: extra empty line at the end of a block (revive)
    pkg/plugins/plugins.go:135:32: empty-lines: extra empty line at the end of a block (revive)
    pkg/authorization/response.go:71:48: empty-lines: extra empty line at the start of a block (revive)
    pkg/authorization/api_test.go:18:51: empty-lines: extra empty line at the end of a block (revive)
    pkg/authorization/middleware_test.go:23:44: empty-lines: extra empty line at the end of a block (revive)
    pkg/authorization/middleware_unix_test.go:17:46: empty-lines: extra empty line at the end of a block (revive)
    pkg/authorization/api_test.go:57:45: empty-lines: extra empty line at the end of a block (revive)
    pkg/authorization/response.go:83:50: empty-lines: extra empty line at the start of a block (revive)
    pkg/authorization/api_test.go:66:47: empty-lines: extra empty line at the end of a block (revive)
    pkg/authorization/middleware_unix_test.go:45:48: empty-lines: extra empty line at the end of a block (revive)
    pkg/authorization/response.go:145:75: empty-lines: extra empty line at the start of a block (revive)
    pkg/authorization/middleware_unix_test.go:56:51: empty-lines: extra empty line at the end of a block (revive)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-28 01:58:49 +02:00
Akihiro Suda
a04e3326e7
Fix constant WARNING: No swap limit support on cgroup v2 hosts
Fix issue 43646

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2022-05-27 10:51:54 +09:00
Sebastiaan van Stijn
5d10c6ec67
Update handling of deprecated kernel (tcp) memory options
- Omit `KernelMemory` and `KernelMemoryTCP` fields in `/info` response if they're
  not supported, or when using API v1.42 or up.
- Re-enable detection of `KernelMemory` (as it's still needed for older API versions)
- Remove warning about kernel memory TCP in daemon logs (a warning is still returned
  by the `/info` endpoint, but we can consider removing that).
- Prevent incorrect "Minimum kernel memory limit allowed" error if the value was
  reset because it's not supported by the host.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-17 09:56:39 +01:00
aiordache
af6307fbda
Remove KernelMemory option from /containers/create and /update endpoints
- remove KernelMemory option from `v1.42` api docs
 - remove KernelMemory warning on `/info`
 - update changes for `v1.42`
 - remove `KernelMemory` field from endpoints docs

Signed-off-by: aiordache <anca.iordache@docker.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-17 09:55:36 +01:00
Cory Snider
b0b71dbe1c pkg/sysinfo: remove libcontainer dependency
Reimplement GetCgroupMounts using the github.com/containerd/cgroups and
github.com/moby/sys/mountinfo packages.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-03-07 18:09:09 -05:00
Akihiro Suda
fecf45b09a
Merge pull request #42796 from thaJeztah/containerd_seccomp_check
pkg/sysinfo: use containerd/pkg/seccomp.IsEnabled()
2021-08-29 03:05:59 +09:00
Sebastiaan van Stijn
accec292c1
pkg/sysinfo: use containerd/pkg/seccomp.IsEnabled()
This replaces the local SeccompSupported() utility for the implementation in containerd,
which performs the same check.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-08-27 15:21:52 +02:00
Eng Zer Jun
c55a4ac779
refactor: move from io/ioutil to io and os package
The io/ioutil package has been deprecated in Go 1.16. This commit
replaces the existing io/ioutil functions with their new definitions in
io and os packages.

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2021-08-27 14:56:57 +08:00
Sebastiaan van Stijn
686be57d0a
Update to Go 1.17.0, and gofmt with Go 1.17
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-08-24 23:33:27 +02:00
Sebastiaan van Stijn
4ace1998e5
pkg/sysinfo: use correct name for AppArmor in t.Skip()
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-14 23:10:13 +02:00
Sebastiaan van Stijn
9b795c3e50
pkg/sysinfo.New(), daemon.RawSysInfo(): remove "quiet" argument
The "quiet" argument was only used in a single place (at daemon startup), and
every other use had to pass "false" to prevent this function from logging
warnings.

Now that SysInfo contains the warnings that occurred when collecting the
system information, we can make leave it up to the caller to use those
warnings (and log them if wanted).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-14 23:10:07 +02:00
Sebastiaan van Stijn
1fb62f455c
pkg/sysinfo: collect warnings in SysInfo struct
This allows the warnings to be consumed in other locations.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-14 17:28:25 +02:00
Sebastiaan van Stijn
208d3c6efb
pkg/sysinfo: move cg2Controllers to be a field in SysInfo and unify v1/v2
We pass the SysInfo struct to all functions. Adding cg2Controllers as a
(non-exported) field makes passing around this information easier.

Now that infoCollector and infoCollectorV2 have the same signature, we can
simplify some bits and use a single slice for all "collectors".

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-14 16:39:44 +02:00
Sebastiaan van Stijn
5cc20ad9e5
pkg/sysinfo: adjust Opt to set new field
This removes the need to have the opts type.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-14 16:39:26 +02:00
Sebastiaan van Stijn
ca27b473cc
pkg/sysinfo: move cg2GroupPath to be a field in SysInfo
We pass the SysInfo struct to all functions. Adding cg2GroupPath as a
(non-exported) field makes passing around this information easier.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-14 16:37:03 +02:00
Sebastiaan van Stijn
e70c5ea1a9
pkg/sysinfo.newV2() remove redundant path.Clean()
path.Join() already does path.Clean(), and the opts.cg2GroupPath
field is already cleaned as part of WithCgroup2GroupPath()

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-14 16:37:02 +02:00
Sebastiaan van Stijn
6677ab6a63
pkg/sysinfo: move cgMounts to be a field in SysInfo
We pass the SysInfo struct to all functions. Adding cgMounts as a
(non-exported) field makes passing around this information easier.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-14 16:37:00 +02:00
Sebastiaan van Stijn
10ce0d84c2
pkg/sysinfo.New() move v1 code to a newV1() function
This makes it clearer that this code is the cgroups v1 equivalent of newV2().

Also moves the "options" handling to newV2() because it's currently only used
for cgroupsv2.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-14 16:36:56 +02:00
Sebastiaan van Stijn
472f21b923
replace uses of deprecated containerd/sys.RunningInUserNS()
This utility was moved to a separate package, which has no
dependencies.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-06-18 11:01:24 +02:00
Jakub Guzik
7ef6ece774 Fix setting swaplimit=true without checking
Signed-off-by: Jakub Guzik <jakubmguzik@gmail.com>
2021-06-03 22:54:10 +02:00
Sebastiaan van Stijn
6458f750e1
use containerd/cgroups to detect cgroups v2
libcontainer does not guarantee a stable API, and is not intended
for external consumers.

this patch replaces some uses of libcontainer/cgroups with
containerd/cgroups.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-11-09 15:00:32 +01:00
Brian Goff
df7031b669 Memoize seccomp value for SysInfo
As it turns out, we call this function every time someone calls `docker
info`, every time a contianer is created, and every time a container is
started.
Certainly this should be refactored as a whole, but for now, memoize the
seccomp value.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2020-09-11 22:48:46 +00:00
Kir Kolyshkin
afbeaf6f29 pkg/sysinfo: rm duplicates
The CPU CFS cgroup-aware scheduler is one single kernel feature, not
two, so it does not make sense to have two separate booleans
(CPUCfsQuota and CPUCfsPeriod). Merge these into CPUCfs.

Same for CPU realtime.

For compatibility reasons, /info stays the same for now.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2020-06-26 16:19:52 -07:00
Sebastiaan van Stijn
66bb1c4644
pkg/sysinfo: use containerd/sys to detect UserNamespaces
The implementation in libcontainer/system is quite complicated,
and we only use it to detect if user-namespaces are enabled.

In addition, the implementation in containerd uses a sync.Once,
so that detection (and reading/parsing `/proc/self/uid_map`) is
only performed once.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-06-15 13:07:48 +02:00
Kir Kolyshkin
d5da7e5330 pkg/sysinfo/sysinfo_linux.go: fix some comments
Some were misleading or vague, some were plain wrong.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2020-05-22 13:13:27 -07:00
Kir Kolyshkin
f02a53d6b9 pkg/sysinfo.applyPIDSCgroupInfo: optimize
For some reason, commit 69cf03700f chose not to use information
already fetched, and called cgroups.FindCgroupMountpoint() instead.
This is not a cheap call, as it has to parse the whole nine yards
of /proc/self/mountinfo, and the info which it tries to get (whether
the pids controller is present) is already available from cgMounts map.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2020-05-22 13:13:27 -07:00
Akihiro Suda
f350b53241 cgroup2: implement docker info
ref: https://www.kernel.org/doc/html/latest/admin-guide/cgroup-v2.html

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-04-17 07:20:01 +09:00
Sebastiaan van Stijn
9f0b3f5609
bump gotest.tools v3.0.1 for compatibility with Go 1.14
full diff: https://github.com/gotestyourself/gotest.tools/compare/v2.3.0...v3.0.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-02-11 00:06:42 +01:00
Akihiro Suda
409bbdc321 cgroup2: enable resource limitation
enable resource limitation by disabling cgroup v1 warnings

resource limitation still doesn't work with rootless mode (even with systemd mode)

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-01-01 02:58:40 +09:00
Tobias Klauser
ba8a15694a Use functions from x/sys/unix to get number of CPUs on Linux
Use Getpid and SchedGetaffinity from golang.org/x/sys/unix to get the
number of CPUs in numCPU on Linux.

Signed-off-by: Tobias Klauser <tklauser@distanz.ch>
2019-06-18 19:26:56 +02:00
Rob Gulewich
256eb04d69 Start containers in their own cgroup namespaces
This is enabled for all containers that are not run with --privileged,
if the kernel supports it.

Fixes #38332

Signed-off-by: Rob Gulewich <rgulewich@netflix.com>
2019-05-07 10:22:16 -07:00
Sebastiaan van Stijn
53460047e4
Refactor pkg/sysinfo
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-02-04 00:38:12 +01:00
Akihiro Suda
ec87479b7e allow running dockerd in an unprivileged user namespace (rootless mode)
Please refer to `docs/rootless.md`.

TLDR:
 * Make sure `/etc/subuid` and `/etc/subgid` contain the entry for you
 * `dockerd-rootless.sh --experimental`
 * `docker -H unix://$XDG_RUNTIME_DIR/docker.sock run ...`

Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2019-02-04 00:24:27 +09:00
Andrew Hsu
78045a5419
use empty string as cgroup path to grab first find
Signed-off-by: Andrew Hsu <andrewhsu@docker.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-12-07 18:44:00 +01:00
Yong Tang
f023816608 Add memory.kernelTCP support for linux
This fix tries to address the issue raised in 37038 where
there were no memory.kernelTCP support for linux.

This fix add MemoryKernelTCP to HostConfig, and pass
the config to runtime-spec.

Additional test case has been added.

This fix fixes 37038.

Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
2018-11-26 21:03:08 +00:00
Justin Cormack
f8e876d761
Fix denial of service with large numbers in cpuset-cpus and cpuset-mems
Using a value such as `--cpuset-mems=1-9223372036854775807` would cause
`dockerd` to run out of memory allocating a map of the values in the
validation code. Set limits to the normal limit of the number of CPUs,
and improve the error handling.

Reported by Huawei PSIRT.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-10-05 15:09:02 +02:00
Vincent Demeester
3845728524
Update tests to use gotest.tools 👼
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2018-06-13 09:04:30 +02:00
Daniel Nephin
6be0f70983 Automated migration using
gty-migrate-from-testify --ignore-build-tags

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2018-03-16 11:03:43 -04:00
Daniel Nephin
4f0d95fa6e Add canonical import comment
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2018-02-05 16:51:57 -05:00
Sebastiaan van Stijn
6ed1163c98
Remove redundant build-tags
Files that are suffixed with `_linux.go` or `_windows.go` are
already only built on Linux / Windows, so these build-tags
were redundant.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2017-12-18 17:41:53 +01:00