Commit graph

3332 commits

Author SHA1 Message Date
Cory Snider
9cee34bc94 pkg/plugins: make unit test less time sensitive
TestClientWithRequestTimeout has been observed to flake in CI. The
timing in the test is quite tight, only giving the client a 10ms window
to time out, which could potentially be missed if the host is under
load and the goroutine scheduling is unlucky. Give the client a full
five seconds of grace to time out before failing the test.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-07-05 12:09:37 -04:00
Sebastiaan van Stijn
ec11aea880
pkg/jsonmessage: use string-literals for easier grep'ing
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-05 12:27:02 +02:00
Sebastiaan van Stijn
fded42c3bd
pkg/ioutils: use string-literals for easier grep'ing
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-05 12:27:02 +02:00
Sebastiaan van Stijn
1da079f211
pkg/idtools: use string-literals for easier grep'ing
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-05 12:27:01 +02:00
Sebastiaan van Stijn
cffe563d4d
pkg/system: remove windows stubs for MkDev/MkNod
They're not used anywhere, so let's remove them; better to have
a compile error than a panic at runtime.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-03 18:50:30 +02:00
Sebastiaan van Stijn
ec79d0fc05
pkg/fileutils: GetTotalUsedFds(): use fast-path for Kernel 6.2 and up
Linux 6.2 and up (commit [f1f1f2569901ec5b9d425f2e91c09a0e320768f3][1])
provides a fast path for the number of open files for the process.

From the [Linux docs][2]:

> The number of open files for the process is stored in 'size' member of
> `stat()` output for /proc/<pid>/fd for fast access.

[1]: f1f1f25699
[2]: https://docs.kernel.org/filesystems/proc.html#proc-pid-fd-list-of-symlinks-to-open-files

This patch adds a fast-path for Kernels that support this, and falls back
to the slow path if the Size fields is zero.

Comparing on a Fedora 38 (kernel 6.2.9-300.fc38.x86_64):

Before/After:

    go test -bench ^BenchmarkGetTotalUsedFds$ -run ^$ ./pkg/fileutils/
    BenchmarkGetTotalUsedFds        57264     18595 ns/op     408 B/op      10 allocs/op
    BenchmarkGetTotalUsedFds       370392      3271 ns/op      40 B/op       3 allocs/op

Note that the slow path has 1 more file-descriptor, due to the open
file-handle for /proc/<pid>/fd during the calculation.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-30 01:31:58 +02:00
Sebastiaan van Stijn
eaa9494b71
pkg/fileutils: GetTotalUsedFds: reduce allocations
Use File.Readdirnames instead of os.ReadDir, as we're only interested in
the number of files, and results don't have to be sorted.

Before:

    BenchmarkGetTotalUsedFds-5   	  149272	      7896 ns/op	     945 B/op	      20 allocs/op

After:

    BenchmarkGetTotalUsedFds-5   	  153517	      7644 ns/op	     408 B/op	      10 allocs/op

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-30 01:31:58 +02:00
Sebastiaan van Stijn
03390be5fa
pkg/fileutils: add BenchmarkGetTotalUsedFds
go test -bench ^BenchmarkGetTotalUsedFds$ -run ^$ ./pkg/fileutils/
    goos: linux
    goarch: arm64
    pkg: github.com/docker/docker/pkg/fileutils
    BenchmarkGetTotalUsedFds-5   	  149272	      7896 ns/op	     945 B/op	      20 allocs/op

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-30 01:31:57 +02:00
Sebastiaan van Stijn
252e94f499
pkg/fileutils: GetTotalUsedFds(): don't pretend to support FreeBSD
Commit 8d56108ffb moved this function from
the generic (no build-tags) fileutils.go to a unix file, adding "freebsd"
to the build-tags.

This likely was a wrong assumption (as other files had freebsd build-tags).
FreeBSD's procfs does not mention `/proc/<pid>/fd` in the manpage, and
we don't test FreeBSD in CI, so let's drop it, and make this a Linux-only
file.

While updating also dropping the import-tag, as we're planning to move
this file internal to the daemon.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-30 01:31:51 +02:00
Sebastiaan van Stijn
a764cd5a40
pkg/chrootarchive: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:33:03 +02:00
Sebastiaan van Stijn
f930559076
pkg/stdcopy: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:31:52 +02:00
Sebastiaan van Stijn
1216328c2d
pkg/plugins: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:31:52 +02:00
Sebastiaan van Stijn
651086d1d9
pkg/jsonmessage: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:31:51 +02:00
Sebastiaan van Stijn
1291bbfa50
pkg/capabilities: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:31:51 +02:00
Sebastiaan van Stijn
79af23e1ea
pkg/ioutils: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:31:51 +02:00
Sebastiaan van Stijn
316c7331eb
pkg/parsers: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:26:03 +02:00
Sebastiaan van Stijn
fb017754e1
pkg/system: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:26:02 +02:00
Sebastiaan van Stijn
5e9a354ff1
pkg/tarsum: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:25:20 +02:00
Sebastiaan van Stijn
2aac054f52
pkg/authorization: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:25:20 +02:00
Sebastiaan van Stijn
f7552f1de4
pkg/archive: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:25:09 +02:00
Bjorn Neergaard
8805e38398
Merge pull request #45799 from cpuguy83/containerd_logrus
Switch all logging to use containerd log pkg
2023-06-26 11:51:44 -06:00
Sebastiaan van Stijn
3c1de2e667
pkg/homedir: deprecate Key() utility
This utility was only used in tests, and internally, and no longer
used since we switch to using os.UserHomeDir() from stdlib.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-26 14:46:03 +02:00
Sebastiaan van Stijn
ddd9665289
pkg/homedir: deprecate GetShortcutString() utility
This function was last used in the pkg/mflag package, which was removed
in 14712f9ff0, and is no longer used in
libnetwork code since e6de8aec2f

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-26 14:45:21 +02:00
Sebastiaan van Stijn
6876e45f9e
pkg/homedir: unify implementations
Unify some of the exported functions instead of maintaining separate functions
per platform.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-26 14:43:26 +02:00
Sebastiaan van Stijn
0aca7f2a0d
pkg/homedir: use os.UserHomeDir()
Use os.UserHomeDir() instead of writing our own, but keep the fallback
on Linux.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-26 14:30:42 +02:00
Brian Goff
74da6a6363 Switch all logging to use containerd log pkg
This unifies our logging and allows us to propagate logging and trace
contexts together.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2023-06-24 00:23:44 +00:00
Sebastiaan van Stijn
a9788886e6
Merge pull request #45720 from ndeloof/copy_uidgid
fix `docker cp -a` failing to access `/` in container
2023-06-14 01:11:36 +02:00
Sebastiaan van Stijn
59b5c6075f
pkg/rootless: remove GetRootlessKitClient, and move to daemon
This utility was only used in a single location (as part of `docker info`),
but the `pkg/rootless` package is imported in various locations, causing
rootlesskit to be a dependency for consumers of that package.

Move GetRootlessKitClient to the daemon code, which is the only location
it was used.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-12 13:44:30 +02:00
Nicolas De Loof
3cc5d62f8a
run getent with a noop stdin
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2023-06-12 09:35:40 +02:00
Cory Snider
8a094fe609 daemon: ensure OCI options play nicely together
Audit the OCI spec options used for Linux containers to ensure they are
less order-dependent. Ensure they don't assume that any pointer fields
are non-nil and that they don't unintentionally clobber mutations to the
spec applied by other options.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-06-06 13:10:05 -04:00
Sebastiaan van Stijn
ab35df454d
remove pre-go1.17 build-tags
Removed pre-go1.17 build-tags with go fix;

    go mod init
    go fix -mod=readonly ./...
    rm go.mod

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-19 20:38:51 +02:00
Cory Snider
6690d2969c pkg/archive: bail if setting xattrs is unsupported
Extended attributes are set on files in container images for a reason.
Fail to unpack if extended attributes are present in a layer and setting
the attributes on the unpacked files fails for any reason.

Add an option to the vfs graph driver to opt into the old behaviour
where ENOTSUPP and EPERM errors encountered when setting extended
attributes are ignored. Make it abundantly clear to users and anyone
triaging their bug reports that they are shooting themselves in the
foot by enabling this option.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-05-18 17:21:12 -04:00
Cory Snider
0cdfd5f275 pkg/system: return rich errors from L(g|s)etxattr
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-05-18 16:31:01 -04:00
Sebastiaan van Stijn
50d2c94bd6
Merge pull request #43637 from thaJeztah/remove_deprecated_storage_drivers
Remove deprecated devicemapper storage driver
2023-05-18 21:23:41 +02:00
Sebastiaan van Stijn
bafcfdf8c5
Merge pull request #45484 from thaJeztah/remove_deprecated_stubs
remove deprecated types, fields, and functions
2023-05-12 14:03:26 +01:00
Cory Snider
4e0319c878 [chore] clean up reexec.Init() calls
Now that most uses of reexec have been replaced with non-reexec
solutions, most of the reexec.Init() calls peppered throughout the test
suites are unnecessary. Furthermore, most of the reexec.Init() calls in
test code neglects to check the return value to determine whether to
exit, which would result in the reexec'ed subprocesses proceeding to run
the tests, which would reexec another subprocess which would proceed to
run the tests, recursively. (That would explain why every reexec
callback used to unconditionally call os.Exit() instead of returning...)

Remove unneeded reexec.Init() calls from test and example code which no
longer needs it, and fix the reexec.Init() calls which are not inert to
exit after a reexec callback is invoked.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-05-09 19:13:17 -04:00
Sebastiaan van Stijn
e3e715666f
pkg/aaparser: deprecate GetVersion, as it's no longer used
Our templates no longer contain version-specific rules, so this function
is no longer used. This patch deprecates it.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-09 16:47:03 +02:00
Sebastiaan van Stijn
55608d8381
pkg/system: remove deprecated IsProcessAlive, IsProcessZombie, KillProcess
These were deprecated in 9d5e754caa, which
is part of the v24.0.0 release, so we can remove it from master.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-06 16:36:17 +02:00
Sebastiaan van Stijn
a3b12ec420
pkg/system: remove deprecated DefaultPathEnv()
This was deprecated in 9f3e5eead5, which
is part of the v24.0.0 release, so we can remove it from master.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-06 16:36:17 +02:00
Sebastiaan van Stijn
dc76b90705
pkg/system: remove deprecated MemInfo, ReadMemInfo()
These were deprecated in 2d49080056, which
is part of the v24.0.0 release, so we can remove it from master.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-06 16:36:17 +02:00
Sebastiaan van Stijn
7d0488b11e
pkg/ioutils: remove deprecated TempDir() alias
This function was deprecated in c63ea32a17, which
is part of the v24.0.0 release, so we can remove it from master.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-06 16:36:16 +02:00
Sebastiaan van Stijn
152c482fee
pkg/platform: remove deprecated OSType const
This const was deprecated in 5c78cbd3be, which
is part of the v24.0.0 release, so we can remove it from master.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-06 16:36:16 +02:00
Sebastiaan van Stijn
3e6092f5b2
pkg/jsonmessage: touch-up GoDoc
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-04 13:46:11 +02:00
Sebastiaan van Stijn
05cb0e5a7d
pkg/jsonmessage: JSONProgress.String(), JSONProgress.now(): simplify
- inline an intermediate variable
- remove a redundant intermediate function

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-04 13:46:10 +02:00
Sebastiaan van Stijn
4cadee65b3
pkg/jsonmessage: JSONMessage.Display(): remove special case for 401 errors
This special case was added in 3043c26419 as
a sentinel error (`AuthRequiredError`) to check whether authentication
is required (and to prompt the users to authenticate). A later refactor
(946bbee39a) removed the `AuthRequiredError`,
but kept the error-message and logic.

Starting with fcee6056dc, it looks like we
no longer depend on this specific error, so we can return the registry's
error message instead.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-04 13:46:10 +02:00
Sebastiaan van Stijn
c80f205f86
Merge pull request #45298 from thaJeztah/pkg_pkatform_cleanup
pkg/platform: cleanup, and deprecate OSType
2023-04-27 02:02:41 +02:00
Tobias Klauser
4ec063fade
pkg/sysinfo: omit Getpid call in numCPU
The man page for sched_setaffinity(2) states the following about the pid
argument [1]:

> If pid is zero, then the mask of the calling thread is returned.

Thus the additional call to unix.Getpid can be omitted and pid = 0
passed to unix.SchedGetaffinity.

[1] https://man7.org/linux/man-pages/man2/sched_setaffinity.2.html#DESCRIPTION

Signed-off-by: Tobias Klauser <tklauser@distanz.ch>
2023-04-25 10:05:20 +02:00
Sebastiaan van Stijn
dc11d2a2d8
remove deprecated devicemapper storage-driver
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-20 23:51:54 +02:00
Sebastiaan van Stijn
5c78cbd3be
pkg/platform: deprecate OSType in favor or runtime.GOOS
This const looks to only be there for "convenience", or _possibly_ was created
with future normalization or special handling in mind.

In either case, currently it is just a direct copy (alias) for runtime.GOOS,
and defining our own type for this gives the impression that it's more than
that. It's only used in a single place, and there's no external consumers, so
let's deprecate this const, and use runtime.GOOS instead.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-08 14:57:33 +02:00
Sebastiaan van Stijn
cf9a76fe8f
pkg/platform: un-export consts that are only for internal use
These consts are only used internally, and never returned to the user.
Un-export to make it clear these are not for external consumption.

While looking at the code, I also noticed that we may be using the wrong
Windows API to collect this information (and found an implementation elsewhere
that does use the correct API). I did not yet update the code, in cases there
are specific reasons.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-08 14:57:30 +02:00