Commit graph

3291 commits

Author SHA1 Message Date
Brian Goff
6eab4f55fa
Merge pull request #44210 from corhere/chrootarchive-without-reexec
Fix 'docker cp' mount table explosion, take four
2022-11-11 10:47:09 -08:00
Sebastiaan van Stijn
67b9f120d5
pkg/archive: switch back to os/exec
This is a partial revert of 7ca0cb7ffa, which
switched from os/exec to the golang.org/x/sys/execabs package to mitigate
security issues (mainly on Windows) with lookups resolving to binaries in the
current directory.

from the go1.19 release notes https://go.dev/doc/go1.19#os-exec-path

> ## PATH lookups
>
> Command and LookPath no longer allow results from a PATH search to be found
> relative to the current directory. This removes a common source of security
> problems but may also break existing programs that depend on using, say,
> exec.Command("prog") to run a binary named prog (or, on Windows, prog.exe) in
> the current directory. See the os/exec package documentation for information
> about how best to update such programs.
>
> On Windows, Command and LookPath now respect the NoDefaultCurrentDirectoryInExePath
> environment variable, making it possible to disable the default implicit search
> of “.” in PATH lookups on Windows systems.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-09 12:28:33 +01:00
Sebastiaan van Stijn
7493debe26
pkg/pidfile: implement Read()
This allows consumers to read back the pidFile and to check if the
PID is still running.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-04 01:50:26 +01:00
Sebastiaan van Stijn
7d3e1ad943
pkg/pidfile: Write(): don't automatically create parent directories
While this was convenient for our use, it's somewhat unexpected for a function
that writes a file to also create all parent directories; even more because
this function may be executed as root.

This patch makes the package more "safe" to use as a generic package by removing
this functionality, and leaving it up to the caller to create parent directories,
if needed.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-04 01:50:26 +01:00
Sebastiaan van Stijn
81945da0ac
pkg/pidfile: Write(): take pid as argument
This allows it to be used for processes other than the daemon itself.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-04 01:50:26 +01:00
Sebastiaan van Stijn
735e250326
pkg/process: Alive(): fix PID 0, -1, negative values
unix.Kill() does not produce an error for PID 0, -1. As a result, checking
process.Alive() would return "true" for both 0 and -1 on macOS (and previously
on Linux as well).

Let's shortcut these values to consider them "not alive", to prevent someone
trying to kill them.

A basic test was added to check the behavior.

Given that the intent of these functions is to handle single processes, this patch
also prevents 0 and negative values to be used.

From KILL(2): https://man7.org/linux/man-pages/man2/kill.2.html

    If pid is positive, then signal sig is sent to the process with
    the ID specified by pid.

    If pid equals 0, then sig is sent to every process in the process
    group of the calling process.

    If pid equals -1, then sig is sent to every process for which the
    calling process has permission to send signals, except for
    process 1 (init), but see below.

    If pid is less than -1, then sig is sent to every process in the
    process group whose ID is -pid.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-04 01:50:26 +01:00
Sebastiaan van Stijn
55d15e9d05
pkg/pidfile, pkg/process: use single implementation for process alive
Using the implementation from pkg/pidfile for windows, as that implementation
looks to be handling more cases to check if a process is still alive (or to be
considered alive).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-04 01:50:26 +01:00
Sebastiaan van Stijn
9d5e754caa
move pkg/system: process to a separate package
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-04 01:50:23 +01:00
Sebastiaan van Stijn
0040fb93d6
pkg/system: IsProcessZombie() skip conversion to string, use bytes instead
bytes.SplitN() is more performant, and skips having to do the conversion.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-04 01:49:54 +01:00
Sebastiaan van Stijn
970ad4e3c7
pkg/system: IsProcessZombie() ignore "os.ErrNotExist" errors
If the file doesn't exist, the process isn't running, so we should be able
to ignore that.

Also remove an intermediate variable.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-04 01:49:49 +01:00
Sebastiaan van Stijn
8d6da1e100
pkg/system: IsProcessAlive() remove redundant type-cast
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-04 01:48:54 +01:00
Sebastiaan van Stijn
781fd745ed
Merge pull request #44393 from thaJeztah/pkg_kernel_duplicate_docs
pkg/parsers/kernel: remove duplicate GoDoc and un-export Utsname
2022-11-02 23:49:48 +01:00
Sebastiaan van Stijn
e1743888ed
Merge pull request #44384 from thaJeztah/idtools_cleanup4
pkg/idtools: remove execCmd() utility
2022-11-02 18:46:56 +01:00
Sebastiaan van Stijn
4386e3f7c0
pkg/parsers/kernel: un-export Utsname
It's only used internally to allow the "unsupported" stub.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-02 10:55:54 +01:00
Sebastiaan van Stijn
775dcab7a2
pkg/parsers/kernel: remove duplicate Package godoc
It was present both in kernel.go and kernel_unix.go.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-02 10:44:50 +01:00
Tiago Seabra
952e1e62c5 Sort entries in pkg/namesgenerator
Signed-off-by: Tiago Seabra <tlgs@users.noreply.github.com>
2022-11-01 23:13:34 +00:00
Sebastiaan van Stijn
1e88fe578e
pkg/idtools: remove execCmd() utility
The `execCmd()` utility was a basic wrapper around `exec.Command()`. Inlining it
makes the code more transparent.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-01 21:13:10 +01:00
Cory Snider
60ee6f739f Add reusable chroot and unshare utilities
Refactor pkg/chrootarchive in terms of those utilities.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-10-26 12:06:31 -04:00
Cory Snider
317d3d10b8 Revert "Use real chroot if daemon is running in a user namespace"
This change was introduced early in the development of rootless support,
before all the kinks were worked out and rootlesskit was built. The
author was testing the daemon by inside a user namespace set up by runc,
observed that the unshare(2) syscall was returning EPERM, and assumed
that it was a fundamental limitation of user namespaces. Seeing as the
kernel documentation (of today) disagrees with that assessment and that
unshare demonstrably works inside user namespaces, I can only assume
that the EPERM was due to a quirk of their test environment, such as a
seccomp filter set up by runc blocking the unshare syscall.
https://github.com/moby/moby/pull/20902#issuecomment-236409406

Mount namespaces are necessary to address #38995 and #43390. Revert the
special-casing so those issues can also be fixed for rootless daemons.

This reverts commit dc950567c1.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-10-26 12:05:20 -04:00
Cory Snider
5de229644f pkg/chrootarchive: stop reexec'ing before chroot
Unshare the thread's file system attributes and, if applicable, mount
namespace so that the chroot operation does not affect the rest of the
process.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-10-26 12:05:13 -04:00
Cory Snider
f2f884a92f pkg/archive: create whiteout temp dir under dest
The applyLayer implementation in pkg/chrootarchive has to set the TMPDIR
environment variable so that archive.UnpackLayer() can successfully
create the whiteout-file temp directory. Change UnpackLayer to create
the temporary directory under the destination path so that environment
variables do not need to be touched.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-10-26 12:04:37 -04:00
Brian Goff
ada6ddc794
Merge pull request #44306 from thaJeztah/chrootarchive_mkdir
pkg/chrootarchive: replace system.MkdirAll for os.Mkdir, use t.TempDir()
2022-10-25 09:29:19 -07:00
Sebastiaan van Stijn
b9921a5560
Merge pull request #44273 from thaJeztah/use_walkdir
use filepath.WalkDir instead of filepath.Walk
2022-10-21 02:28:56 +02:00
Sebastiaan van Stijn
0f2956ab5d
Merge pull request #44302 from thaJeztah/sys_windows
pkg/system: optimize and refactor MkdirAllWithACL()
2022-10-21 00:36:58 +02:00
Sebastiaan van Stijn
1c550c36b3
Merge pull request #44268 from thaJeztah/idtools_cleanup3
pkg/idtools: remove CanAccess(), and move to daemon
2022-10-20 21:58:17 +02:00
Sebastiaan van Stijn
4fa853f5de
pkg/fileutils: ReadSymlinkedDirectory: preserve underlying error
We were discarding the underlying error, which made it impossible for
callers to detect (e.g.) an os.ErrNotExist.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-16 20:15:08 +02:00
Sebastiaan van Stijn
24e371c812
pkg/fileutils: improve tests
- use t.TempDir() to make sure we're testing from a clean state
- improve checks for errors to have the correct error-type where possible

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-16 20:15:08 +02:00
Sebastiaan van Stijn
d4d242ba76
pkg/chrootarchive: gofumpt test files
Excluding non-test files, as a large refactor of those files is
being worked on.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-16 14:09:06 +02:00
Sebastiaan van Stijn
dee3f716b3
pkg/chrootarchive: replace system.MkdirAll for os.Mkdir
system.MkdirAll is a special version of os.Mkdir to handle creating directories
using Windows volume paths ("\\?\Volume{4c1b02c1-d990-11dc-99ae-806e6f6e6963}").
This may be important when MkdirAll is used, which traverses all parent paths to
create them if missing (ultimately landing on the "volume" path).

Commit 62f648b061 introduced the system.MkdirAll
calls, as a change was made in applyLayer() for Windows to use Windows volume
paths as an alternative for chroot (which is not supported on Windows). Later
iteractions changed this to regular Windows long-paths (`\\?\<path>`) in
230cfc6ed2, and 9b648dfac6.
Such paths are handled by the `os` package.

However, in these tests, the parent path already exists (all paths created are
a direct subdirectory within `tmpDir`). It looks like `MkdirAll` here is used
out of convenience to not have to handle `os.ErrExist` errors. As all these
tests are running in a fresh temporary directory, there should be no need to
handle those, and it's actually desirable to produce an error in that case, as
the directory already existing would be unexpected.

Because of the above, this test changes `system.MkdirAll` to `os.Mkdir`. As we
are changing these lines, this patch also changes the legacy octal notation
(`0700`) to the now preferred `0o700`.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-16 13:58:50 +02:00
Sebastiaan van Stijn
8a8202fcdc
pkg/chrootarchive: TestChrootTarUntar fix copy/paste mistake
Introduced in 3ac6394b80, which makes no mention
of a reason for extracting to the same directory as we created the archive from,
so I assume this was a copy/paste mistake and the path was meant to be "dest",
not "src".

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-16 13:15:23 +02:00
Sebastiaan van Stijn
0955c88c2e
pkg/chrootarchive: use t.TempDir()
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-16 13:02:45 +02:00
Sebastiaan van Stijn
d006242d73
Merge pull request #44251 from thaJeztah/pkg_dir_cleanup
pkg/directory: remove unused MoveToSubdir() utility, and some refactoring
2022-10-15 22:48:19 +02:00
Sebastiaan van Stijn
69f72417f4
pkg/idtools: remove CanAccess(), and move to daemon
The implementation of CanAccess() is very rudimentary, and should
not be used for anything other than a basic check (and maybe not
even for that). It's only used in a single location in the daemon,
so move it there, and un-export it to not encourage others to use
it out of context.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-15 22:42:39 +02:00
Sebastiaan van Stijn
ee34a8ac29
pkg/idtools: setPermissions() accept Identity as argument
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-15 22:42:39 +02:00
Sebastiaan van Stijn
bca90530fa
pkg/idtools: simplify if-statement
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-15 22:42:38 +02:00
Sebastiaan van Stijn
d68fa0382d
pkg/idtools: don't use system.Stat() on unix
Looks like we don't need the abstraction, so we can reduce the
dependency on pkg/system.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-15 22:42:35 +02:00
Sebastiaan van Stijn
1311687d0d
Merge pull request #44254 from thaJeztah/idtools_cleanup2
pkg/idtools: various cleanups
2022-10-15 22:42:09 +02:00
Sebastiaan van Stijn
081c00c7df
Merge pull request #44265 from thaJeztah/pkg_system_move_init_step1
pkg/system: cleanup, test-fixes and improvements and minor fixes
2022-10-15 21:28:13 +02:00
Sebastiaan van Stijn
43d6eb7173
pkg/pidfile: remove PIDFile type, rename New() to Write()
This type felt really redundant; `pidfile.New()` takes the path of the file to
create as an argument, so this is already known. The only thing the PIDFile
type provided was a `Remove()` method, which was just calling `os.Remove()` on
the path of the file.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-15 16:40:16 +02:00
Sebastiaan van Stijn
dd8983f96c
pkg/pidfile: reduce cyclomatic complexity, and small optimisation
Use bytes.TrimSpace instead of using the strings package, which is
more performant, and allows us to skip the intermediate variable.

Also combined some "if" statements to reduce cyclomatic complexity.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-15 15:11:40 +02:00
Sebastiaan van Stijn
4917bcc039
pkg/pidfile: don't ignore all errors when reading file
It's ok to ignore if the file doesn't exist, or if the file doesn't
have a PID in it, but we should produce an error if the file exists,
but we're unable to read it for other reasons.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-15 15:11:40 +02:00
Sebastiaan van Stijn
3ce2a7d026
pkg/pidfile: pkg/pidfile: use strconv instead of fmt.Sprintf(), and unconvert
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-15 15:11:40 +02:00
Sebastiaan van Stijn
f058afc861
pkg/system: synchronize mkdirall() with latest os.MkDirAll()
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-15 13:24:43 +02:00
Sebastiaan van Stijn
2e66c0b6f0
pkg/system: create SecurityAttribute only once (Windows)
The same attribute was generated for each path that was created, but always
the same, so instead of generating it in each iteration, generate it once,
and pass it to our mkdirall() implementation.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-15 13:24:43 +02:00
Sebastiaan van Stijn
55ceb5047c
pkg/system: update volumePath regex to allow returning earlier
The regex only matched volume paths without a trailing path-separator. In cases
where a path would be passed with a trailing path-separator, it would depend on
further code in mkdirall to strip the trailing slash, then to perform the regex
again in the next iteration.

While regexes aren't ideal, we're already executing this one, so we may as well
use it to match those situations as well (instead of executing it twice), to
allow us to return early.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-15 13:24:43 +02:00
Sebastiaan van Stijn
cfef1b11e5
pkg/system: compile volume-path regex once, and update GoDoc
Ideally, we would construct this lazily, but adding a function and a
sync.Once felt like a bit "too much".

Also updated the GoDoc for some functions to better describe what they do.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-15 13:24:40 +02:00
Sebastiaan van Stijn
ec3c08d618
Merge pull request #44261 from thaJeztah/remove_deprecated_pkgs
remove aliases for deprecated pkg/urlutil, pkg/fsutils, pkg/pubsub
2022-10-14 01:35:01 +02:00
Brian Goff
058010187b
Merge pull request #44271 from thaJeztah/pidfile_windows_cleanup
pkg/pidfile: windows: replace magic consts for golang.org/x/sys consts
2022-10-10 09:30:06 -07:00
Samuel Karp
c9d2b7df77
Merge pull request #44222 from thaJeztah/godoc_instead_of_readme 2022-10-10 00:06:17 -07:00
Sebastiaan van Stijn
ec000ce555
pkg/archive: use filepath.WalkDir instead of filepath.Walk
WalkDir is more performant as it doesn't perform an os.Lstat on every visited
file or directory.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-09 17:12:04 +02:00