Commit graph

44982 commits

Author SHA1 Message Date
Sebastiaan van Stijn
8f1bc0bccc
daemon/graphdriver/btrfs: use filepath.WalkDir instead of filepath.Walk
WalkDir is more performant as it doesn't perform an os.Lstat on every visited
file or directory.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-09 17:04:16 +02:00
Sebastiaan van Stijn
1870d5f4aa
builder: use filepath.WalkDir instead of filepath.Walk
WalkDir is more performant as it doesn't perform an os.Lstat on every visited
file or directory.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-09 17:00:23 +02:00
Sebastiaan van Stijn
d33428f0bf
pkg/system: remove solaris left-over
It was removed everywhere else, so we may as well remove it here.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-09 14:59:19 +02:00
Sebastiaan van Stijn
a35bcd01c5
pkg/pidfile: replace uses of windows.Close() with windows.CloseHandle()
Since https://golang.org/cl/4600042, Close is a straight wrapper of CloseHandle.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-09 13:27:58 +02:00
Sebastiaan van Stijn
89de943401
pkg/pidfile: windows: replace magic consts for golang.org/x/sys consts
These consts were defined locally, but are now defined in golang.org/x/sys, so
we can use those.

Also added some documentation about how this function works, taking the description
from the GetExitCodeProcess function (processthreadsapi.h) API reference:
https://learn.microsoft.com/en-us/windows/win32/api/processthreadsapi/nf-processthreadsapi-getexitcodeprocess

> The GetExitCodeProcess function returns a valid error code defined by the
> application only after the thread terminates. Therefore, an application should
> not use `STILL_ACTIVE` (259) as an error code (`STILL_ACTIVE` is a macro for
> `STATUS_PENDING` (minwinbase.h)). If a thread returns `STILL_ACTIVE` (259) as
> an error code, then applications that test for that value could interpret it
> to mean that the thread is still running, and continue to test for the
> completion of the thread after the thread has terminated, which could put
> the application into an infinite loop.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-09 12:48:35 +02:00
Samuel Karp
9582dd10e1
Merge pull request #44267 from thaJeztah/pkg_loopback_xsysunix 2022-10-09 01:20:06 -07:00
Sebastiaan van Stijn
11cceea58a
pkg/idtools: cleanup errors
Most of the package was using stdlib's errors package, so replacing two calls
to pkg/errors with stdlib. Also fixing capitalization of error strings.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-08 21:56:28 +02:00
Sebastiaan van Stijn
8d5b17e939
pkg/idtools: don't use system.MkdirAll() where not needed
On unix, it's an alias for os.MkdirAll, so remove its use to be
more transparent what's being used.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-08 21:55:18 +02:00
Sebastiaan van Stijn
2e74e307d0
pkg/idtools: format code with gofumpt
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-08 21:55:04 +02:00
Sebastiaan van Stijn
0fc13104e7
pkg/idtools: CanAccess(): reorder checks to allow early return
Merge the accessible() function into CanAccess, and check world-
readable permissions first, before checking owner and group.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-08 21:54:23 +02:00
Sebastiaan van Stijn
c7c02eea81
pkg/loopback: use ioctl helpers from x/sys/unix
Use the IoctlRetInt, IoctlSetInt and IoctlLoopSetStatus64 helper
functions defined in the golang.org/x/sys/unix package instead of
manually wrapping these using a locally defined function.

Inspired by 3cc3d8a560

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-08 21:20:29 +02:00
Sebastiaan van Stijn
6742f74e0e
vendor: golang.org/x/sys v0.0.0-20221006211917-84dc82d7e875
full diff: 3c1f35247d...84dc82d7e8

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-08 21:19:33 +02:00
Sebastiaan van Stijn
1ffdac4502
layer: gofumpt
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-08 19:15:28 +02:00
Sebastiaan van Stijn
c3a5641132
layer: use strconv instead of fmt.Sprintf()
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-08 19:15:28 +02:00
Sebastiaan van Stijn
1fccb39316
pkg/idtools: remove unused CanAccess() stub for Windows
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-08 18:46:05 +02:00
Sebastiaan van Stijn
3b9b5842b3
pkg/idtools: mkdirAs(): move var and comment to where it's used
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-08 18:45:33 +02:00
Sebastiaan van Stijn
838fc976c8
integration(-cli): remove some redundant formatting
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-08 17:41:39 +02:00
Sebastiaan van Stijn
07b2e4cb79
client: use strconv instead of fmt.Sprintf()
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-08 17:41:39 +02:00
Sebastiaan van Stijn
145817a9cf
libnetwork: use strconv instead of fmt.Sprintf()
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-08 17:41:39 +02:00
Sebastiaan van Stijn
56e64270f3
daemon: use strconv instead of fmt.Sprintf()
Also cleaning up some errors

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-08 17:41:39 +02:00
Sebastiaan van Stijn
533ecb44b1
daemon/logger: use strconv instead of fmt.Sprintf()
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-08 17:41:38 +02:00
Sebastiaan van Stijn
7fbf321c2a
daemon/graphdriver: use strconv instead of fmt.Sprintf
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-08 17:41:35 +02:00
Sebastiaan van Stijn
27aea4956c
pkg/idtools: mkdirAs() be more explicit about ignored args on Windows
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-08 17:11:05 +02:00
Sebastiaan van Stijn
a19ee75bd1
pkg/system: fix missing assertions and use sub-tests for ChTimes
These tests were effectively doing "subtests", using comments to describe each,
however;

- due to the use of `t.Fatal()` would terminate before completing all "subtests"
- The error returned by the function being tested (`Chtimes`), was not checked,
  and the test used "indirect" checks to verify if it worked correctly. Adding
  assertions to check if the function didn't produce an error.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-07 22:57:01 +02:00
Sebastiaan van Stijn
a9c5a40087
pkg/system: rename some tests to be more descriptive
Removing the "Linux" suffix from one test, which should probably be
rewritten to be run on "unix", to provide test-coverage for those
implementations.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-07 18:36:25 +02:00
Sebastiaan van Stijn
ab7bc6b7d2
pkg/system: use t.TempDir(), remove some test-utils
With t.TempDir(), some of the test-utilities became so small that
it was more transparent to inline them. This also helps separating
concenrs, as we're in the process of thinning out and decoupling
some packages.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-07 18:36:13 +02:00
Sebastiaan van Stijn
7bd051eeec
pkg/system: windows: setCTime(): remove redundant conversion
It looks like this function was converting the time (`windows.NsecToTimespec()`),
only to convert it back (`windows.TimespecToNsec()`). This became clear when
moving the lines together:

```go
ctimespec := windows.NsecToTimespec(ctime.UnixNano())
c := windows.NsecToFiletime(windows.TimespecToNsec(ctimespec))
```

And looking at the Golang code, it looks like they're indeed the exact reverse:

```go
func TimespecToNsec(ts Timespec) int64 { return int64(ts.Sec)*1e9 + int64(ts.Nsec) }

func NsecToTimespec(nsec int64) (ts Timespec) {
    ts.Sec = nsec / 1e9
    ts.Nsec = nsec % 1e9
    return
}
```

While modifying this code, also renaming the `e` variable to a more common `err`.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-07 18:34:44 +02:00
Sebastiaan van Stijn
0b8444aa0c
pkg/system: rename maxTime and re-use, define unixEpochTime, update GoDoc
This more closely matches to how it's used everywhere. Also move the comment
describing "what" ChTimes() does inside its GoDoc.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-07 18:34:44 +02:00
Sebastiaan van Stijn
2c9684e35c
pkg/system: add note about maxTime
This code caused me some head-scratches, and initially I wondered
if this was a bug, but it looks to be intentional to set nsec, not
sec, as time.Unix() internally divides nsec, and sets sec accordingly;
https://github.com/golang/go/blob/go1.19.2/src/time/time.go#L1364-L1380

    // Unix returns the local Time corresponding to the given Unix time,
    // sec seconds and nsec nanoseconds since January 1, 1970 UTC.
    // It is valid to pass nsec outside the range [0, 999999999].
    // Not all sec values have a corresponding time value. One such
    // value is 1<<63-1 (the largest int64 value).
    func Unix(sec int64, nsec int64) Time {
        if nsec < 0 || nsec >= 1e9 {
            n := nsec / 1e9
            sec += n
            nsec -= n * 1e9
            if nsec < 0 {
                nsec += 1e9
                sec--
            }
        }
        return unixTime(sec, int32(nsec))
    }

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-07 18:34:34 +02:00
Sebastiaan van Stijn
247f90c82e
pkg/system: move maxTime init() back to Chtimes code
This code was moved to a separate file in fe5b34ba88,
but it's unclear why it was moved (as this file is not excluded on Windows).

Moving the code back into the chtimes file, to move it closer to where it's used.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-07 18:13:05 +02:00
Sebastiaan van Stijn
c218211012
Merge pull request #44255 from thaJeztah/pkg_system_remove_GetExitCode
pkg/system: move GetExitCode() to pkg/idtools, and un-export
2022-10-07 18:10:03 +02:00
Sebastiaan van Stijn
e034b7b04d
Merge pull request #44253 from thaJeztah/pkg_sys_umask
pkg/system: remove Umask() utility
2022-10-07 17:25:52 +02:00
Samuel Karp
968a0bcd63
Merge pull request #43055 from thaJeztah/use_containerd_oci_devices_part2 2022-10-07 00:36:40 -07:00
Sebastiaan van Stijn
4347080b46
pkg/system: remove Umask() utility
It was only used in a couple of places, and in most places shouldn't be used
as those locations were in unix/linux-only files, so didn't need the wrapper.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-06 22:22:29 +02:00
Sebastiaan van Stijn
59c77c8f5b
Merge pull request #44250 from thaJeztah/fix_pkg_dir
pkg/directory: Size(): add back type-casts to account for platform differences
2022-10-06 21:44:46 +02:00
Sebastiaan van Stijn
1515e02c8a
Merge pull request #44215 from corhere/fix-unlockosthread-pdeathsig
Stop subprocesses from getting unexpectedly killed
2022-10-06 20:08:53 +02:00
Sebastiaan van Stijn
76ce3fd9c9
remove aliases for deprecated pkg/pubsub
The aliases are included in the 22 release branch, so we can remove them
from master.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-06 16:41:00 +02:00
tockn
89a9d7d650 fix typo
fix typo

Signed-off-by: Takuto Sato <tockn.jp@gmail.com>
2022-10-06 11:09:06 +09:00
Sebastiaan van Stijn
c1729f876c
remove aliases for deprecated pkg/fsutils
The aliases are included in the 22 release branch, so we can remove them
from master.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-06 02:51:24 +02:00
Sebastiaan van Stijn
ce2abb82c0
remove aliases for deprecated pkg/urlutil
The aliases are included in the 22 release branch, so we can remove them
from master.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-06 02:48:42 +02:00
Akihiro Suda
88c6ec40a4
Merge pull request #44258 from thaJeztah/bump_selinux
vendor: github.com/opencontainers/selinux v1.10.2
2022-10-06 07:36:12 +09:00
Sebastiaan van Stijn
a5e1baf3ab
vendor: github.com/opencontainers/selinux v1.10.2
full diff: https://github.com/opencontainers/selinux/compare/v1.10.1...v1.10.2

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-05 22:36:16 +02:00
Sebastiaan van Stijn
49940ab5ee
Merge pull request #44216 from cpuguy83/volume_unnamed_label
Volume prune: only prune anonymous volumes by default
2022-10-05 19:34:47 +02:00
Cory Snider
1f22b15030 Lock OS threads when exec'ing with Pdeathsig
On Linux, when (os/exec.Cmd).SysProcAttr.Pdeathsig is set, the signal
will be sent to the process when the OS thread on which cmd.Start() was
executed dies. The runtime terminates an OS thread when a goroutine
exits after being wired to the thread with runtime.LockOSThread(). If
other goroutines are allowed to be scheduled onto a thread which called
cmd.Start(), an unrelated goroutine could cause the thread to be
terminated and prematurely signal the command. See
https://github.com/golang/go/issues/27505 for more information.

Prevent started subprocesses with Pdeathsig from getting signaled
prematurely by wiring the starting goroutine to the OS thread until the
subprocess has exited. No other goroutines can be scheduled onto a
locked thread so it will remain alive until unlocked or the daemon
process exits.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-10-05 12:18:03 -04:00
Brian Goff
a8b20ed105
Merge pull request #44252 from thaJeztah/deprecate_pkg_fsutil
pkg/fsutils: deprecate in favor of containerd/continuity/fs
2022-10-05 08:37:21 -07:00
Sebastiaan van Stijn
ab677c41ea
pkg/system: unconvert
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-05 16:21:04 +02:00
Sebastiaan van Stijn
07b1aa822c
pkg/system: move GetExitCode() to pkg/idtools, and un-export
This utility was only used in a single place, and had no external consumers.
Move it to where it's used.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-05 16:20:33 +02:00
Sebastiaan van Stijn
5b6b42162b
pkg/fsutils: deprecate in favor of containerd/continuity/fs
The pkg/fsutils package was forked in containerd, and later moved to
containerd/continuity/fs. As we're moving more bits to containerd, let's also
use the same implementation to reduce code-duplication and to prevent them from
diverging.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-05 11:36:04 +02:00
Sebastiaan van Stijn
26659d5eb8
pkg/directory: remove unused MoveToSubdir() utility
This utility was added in 442b45628e as part of
user-namespaces, and first used in 44e1023a93 to
set up the daemon root, and move the existing content;
44e1023a93/daemon/daemon_experimental.go (L68-L71)

A later iteration no longer _moved_ the existing root directory, and removed the
use of `directory.MoveToSubdir()` e8532023f2

It looks like there's no external consumers of this utility, so we should be
save to remove it.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-05 11:12:11 +02:00
Sebastiaan van Stijn
bd6217bb74
pkg/directory: minor refactor of Size()
- separate exported function from implementation, to allow for GoDoc to be
  maintained in a single location.
- don't use named return variables (no "bare" return, and potentially shadowing
  variables)
- reverse the `os.IsNotExist(err) && d != dir` condition, putting  the "lighter"
  `d != dir` first.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-05 11:12:11 +02:00