Commit graph

1047 commits

Author SHA1 Message Date
Sebastiaan van Stijn
ac1a867282
vendor: github.com/mistifyio/go-zfs/v3 v3.0.1
Switching to the v3 version, which was renamed to be compatible with
go modules.

Full diff: https://github.com/mistifyio/go-zfs/compare/f784269be439...v3.0.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 86108812b8)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-01 17:14:33 +02:00
Cory Snider
f77a3274b4
[chore] clean up reexec.Init() calls
Now that most uses of reexec have been replaced with non-reexec
solutions, most of the reexec.Init() calls peppered throughout the test
suites are unnecessary. Furthermore, most of the reexec.Init() calls in
test code neglects to check the return value to determine whether to
exit, which would result in the reexec'ed subprocesses proceeding to run
the tests, which would reexec another subprocess which would proceed to
run the tests, recursively. (That would explain why every reexec
callback used to unconditionally call os.Exit() instead of returning...)

Remove unneeded reexec.Init() calls from test and example code which no
longer needs it, and fix the reexec.Init() calls which are not inert to
exit after a reexec callback is invoked.

Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit 4e0319c878)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-11 16:31:41 +02:00
Cory Snider
2b6761fd3e d/graphdriver/copy: support src fs w/o xattr support
Treat copying extended attributes from a source filesystem which does
not support extended attributes as a no-op, same as if the file did not
possess the extended attribute. Only fail copying extended attributes if
the source file has the attribute and the destination filesystem does
not support xattrs.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-05-04 12:30:59 -04:00
Sebastiaan van Stijn
9791756284
overlay2: remove deprecated overlay2.override_kernel_check option
This option was deprecated in e35700eb50
(and backported to v23.0 through 43ce8f7d24).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-20 23:57:45 +02:00
Sebastiaan van Stijn
f72548956f
remove deprecated legacy "overlay" storage-driver
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-19 17:06:45 +02:00
Sebastiaan van Stijn
3903f16cd6
daemon: remove deprecated AuFS storage driver
There's still some locations refering to AuFS;

- pkg/archive: I suspect most of that code is because the whiteout-files
  are modelled after aufs (but possibly some code is only relevant to
  images created with AuFS as storage driver; to be looked into).
- contrib/apparmor/template: likely some rules can be removed
- contrib/dockerize-disk.sh: very old contribution, and unlikely used
  by anyone, but perhaps could be updated if we want to (or just removed).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-15 01:27:16 +02:00
Sebastiaan van Stijn
3b569cc686
daemon/graphdriver/windows: cleanup errors
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-03-14 23:00:48 +01:00
Sebastiaan van Stijn
bbeaeee3c7
daemon/graphdriver/windows: remove some intermediate variables
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-03-14 23:00:48 +01:00
Sebastiaan van Stijn
9db5dc9a46
daemon/graphdriver/windows: use strings.EqualFold()
Saves some allocations

    BenchmarkTolower
    BenchmarkTolower-5     7917788       150.4 ns/op      16 B/op       3 allocs/op
    BenchmarkEqualFold
    BenchmarkEqualFold-5   8248605       143.5 ns/op       8 B/op       1 allocs/op

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-03-14 23:00:48 +01:00
Sebastiaan van Stijn
d742188e3b
daemon/graphdriver/windows: remove fileFlagSequentialScan const
Replace it with the const that's now defined in golang.org/x/sys/windows

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-03-14 23:00:47 +01:00
Sebastiaan van Stijn
3a8c97be45
daemon/graphdriver/windows: rename vars that collided with imports
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-03-14 23:00:47 +01:00
Sebastiaan van Stijn
605e8f53b1
daemon/graphdriver/windows: InitFilter() don't use idtools.MkdirAllAndChown()
idtools.MkdirAllAndChown on Windows does not chown directories, which makes
idtools.MkdirAllAndChown() just an alias for system.MkDirAll().

Also setting the filemode to `0`, as changing filemode is a no-op on Windows as
well; both of these changes should make it more transparent that no chown'ing,
nor changing filemode takes place.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-03-14 23:00:44 +01:00
Bjorn Neergaard
3bcb350711
graphdriver/overlay2: usingMetacopy ENOTSUP is non-fatal
Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
2023-02-03 05:10:53 -07:00
Akihiro Suda
89fb8b32f6
graphdriver/btrfs: needs kernel headers >= 4.12, not >= 4.7
`linux/btrfs_tree.h` was not installed to `/usr/include` until kernel 4.12
fcc8487d47

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2023-01-08 05:04:47 +09:00
Bjorn Neergaard
d3778d65fa
graphdriver/btrfs: use free wrapper consistently
While the Cgo in this entire file is quite questionable, that is a task
for another day.

Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
2023-01-06 13:23:05 -07:00
Bjorn Neergaard
3208dcabdc
graphdriver/btrfs: use kernel UAPI headers
By relying on the kernel UAPI (userspace API), we can drop a dependency
and simplify building Moby, while also ensuring that we are using a
stable/supported source of the C types and defines we need.

btrfs-progs mirrors the kernel headers, but the headers it ships with
are not the canonical source and as [we have seen before][44698], could
be subject to changes.

Depending on the canonical headers from the kernel both is more
idiomatic, and ensures we are protected by the kernel's promise to not
break userspace.

  [44698]: https://github.com/moby/moby/issues/44698

Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
2023-01-06 13:22:58 -07:00
Bjorn Neergaard
1449c82484
graphdriver/btrfs: drop version information
This is actually quite meaningless as we are reporting the libbtrfs
version, but we do not use libbtrfs. We only use the kernel interface to
btrfs instead.

While we could report the version of the kernel headers in play, they're
rather all-or-nothing: they provide the structures and defines we need,
or they don't. As such, drop all version information as the host kernel
version is the only thing that matters.

Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
2023-01-06 11:13:23 -07:00
Sebastiaan van Stijn
a5ebd28797
daemon/graphdriver/devicemapper: simplify Udev log, and update link
Simplify the error message so that we don't have to distinguish between static-
and non-static builds. Also update the link to the storage-driver section to
use a "/go/" redirect in the docs, as the anchor link was no longer correct.
Using a "/go/" redirect  makes sure the link remains functional if docs is  moving
around.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-31 14:36:57 +01:00
Shengjing Zhu
ffbbe3d103 daemon/graphdriver/btrfs: workaround field rename in btrfs-progs 6.1
Closes: #44698

Signed-off-by: Shengjing Zhu <zhsj@debian.org>
2022-12-29 03:47:12 +08:00
Sebastiaan van Stijn
3f935d0e2c
daemon/graphdriver/zfs: use strings.Cut, and refactor
Fixes a (theoretical?) panic if ID would be shorter than 12
characters. Also trim the ID _after_ cutting off the suffix.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-21 11:09:03 +01:00
Sebastiaan van Stijn
6db4af7bd5
graphdriver: windows: use strings.Cut()
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-21 11:08:59 +01:00
Sebastiaan van Stijn
542c735926
Merge pull request #44256 from thaJeztah/redundant_sprintfs
replace redundant fmt.Sprintf() with strconv
2022-10-25 16:48:15 -04:00
Sebastiaan van Stijn
b9921a5560
Merge pull request #44273 from thaJeztah/use_walkdir
use filepath.WalkDir instead of filepath.Walk
2022-10-21 02:28:56 +02:00
Sebastiaan van Stijn
2400bc66ef
Merge pull request #44285 from cpuguy83/nix_ov2_reexec
Replace overlay2 mount reexec with in-proc impl
2022-10-18 14:39:05 +02:00
Tianon Gravi
6b2328e5cc
Merge pull request #44281 from thaJeztah/windows_filter_defer_in_loop
daemon/graphdriver/windows: Remove() don't use defer() in a loop
2022-10-14 18:14:32 +00:00
Tianon Gravi
82d563243f
Merge pull request #44280 from thaJeztah/windows_filter
daemon/graphdriver/windows: use go-winio.GetFileSystemType()
2022-10-14 18:12:42 +00:00
Brian Goff
34f459423a Replace overlay2 mount reexec with in-proc impl
Building off insights from the great work Cory Snider has been doing,
this replaces a reexec with a much lower overhead implementation which
performs the `Chddir` in a new goroutine that is locked to a specific
thread with CLONE_FS unshared.
The thread is thrown away afterwards and the Chdir does effectively the
same thing as what the reexec was being used for.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2022-10-12 00:38:50 +00:00
Sebastiaan van Stijn
75bdbf02a6
daemon/graphdriver/windows: Remove() don't use defer() in a loop
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-10 17:48:45 +02:00
Sebastiaan van Stijn
90431d1857
daemon/graphdriver/windows: use go-winio.GetFileSystemType()
go-winio now defines this function, so we can consume that.

Note that there's a difference between the old implementation and the original
one (added in 1cb9e9b44e). The old implementation
had special handling for win32 error codes, which was removed in the go-winio
implementation in 0966e1ad56

As `go-winio.GetFileSystemType()` calls `filepath.VolumeName(path)` internally,
this patch also removes the `string(home[0])`, which is redundant, and could
potentially panic if an empty string would be passed.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-10 16:22:05 +02:00
Sebastiaan van Stijn
e35700eb50
daemon/graphdriver/overlay2: remove deprecated overrideKernelCheck
Commit 955c1f881a (Docker v17.12.0) replaced
detection of support for multiple lowerdirs (as required by overlay2) to not
depend on the kernel version. The `overlay2.override_kernel_check` was still
used to print a warning that older kernel versions may not have full support.

After this, commit e226aea280 (Docker v20.10.0,
backported to v19.03.7) removed uses of the `overlay2.override_kernel_check`
option altogether, but we were still parsing it.

This patch changes the `parseOptions()` function to not parse the option,
printing a deprecation warning instead. We should change this to be an error,
but the  `overlay2.override_kernel_check` option was not deprecated in the
documentation, so keeping it around for one more release.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-10 14:58:40 +02:00
Sebastiaan van Stijn
f595434abc
daemon/graphdriver/devmapper: use filepath.WalkDir instead of filepath.Walk
WalkDir is more performant as it doesn't perform an os.Lstat on every visited
file or directory.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-09 17:04:44 +02:00
Sebastiaan van Stijn
8f1bc0bccc
daemon/graphdriver/btrfs: use filepath.WalkDir instead of filepath.Walk
WalkDir is more performant as it doesn't perform an os.Lstat on every visited
file or directory.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-09 17:04:16 +02:00
Sebastiaan van Stijn
7fbf321c2a
daemon/graphdriver: use strconv instead of fmt.Sprintf
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-08 17:41:35 +02:00
Sebastiaan van Stijn
1515e02c8a
Merge pull request #44215 from corhere/fix-unlockosthread-pdeathsig
Stop subprocesses from getting unexpectedly killed
2022-10-06 20:08:53 +02:00
Cory Snider
1f22b15030 Lock OS threads when exec'ing with Pdeathsig
On Linux, when (os/exec.Cmd).SysProcAttr.Pdeathsig is set, the signal
will be sent to the process when the OS thread on which cmd.Start() was
executed dies. The runtime terminates an OS thread when a goroutine
exits after being wired to the thread with runtime.LockOSThread(). If
other goroutines are allowed to be scheduled onto a thread which called
cmd.Start(), an unrelated goroutine could cause the thread to be
terminated and prematurely signal the command. See
https://github.com/golang/go/issues/27505 for more information.

Prevent started subprocesses with Pdeathsig from getting signaled
prematurely by wiring the starting goroutine to the OS thread until the
subprocess has exited. No other goroutines can be scheduled onto a
locked thread so it will remain alive until unlocked or the daemon
process exits.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-10-05 12:18:03 -04:00
Sebastiaan van Stijn
5b6b42162b
pkg/fsutils: deprecate in favor of containerd/continuity/fs
The pkg/fsutils package was forked in containerd, and later moved to
containerd/continuity/fs. As we're moving more bits to containerd, let's also
use the same implementation to reduce code-duplication and to prevent them from
diverging.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-05 11:36:04 +02:00
Sebastiaan van Stijn
9d9cca49b4
daemon/graphdriver: fix empty-lines (revive)
daemon/graphdriver/aufs/aufs.go:239:80: empty-lines: extra empty line at the start of a block (revive)
    daemon/graphdriver/graphtest/graphbench_unix.go:249:27: empty-lines: extra empty line at the start of a block (revive)
    daemon/graphdriver/graphtest/testutil.go:271:30: empty-lines: extra empty line at the end of a block (revive)
    daemon/graphdriver/graphtest/graphbench_unix.go:179:32: empty-block: this block is empty, you can remove it (revive)
    daemon/graphdriver/zfs/zfs.go:375:48: empty-lines: extra empty line at the end of a block (revive)
    daemon/graphdriver/overlay/overlay.go:248:89: empty-lines: extra empty line at the start of a block (revive)
    daemon/graphdriver/devmapper/deviceset.go:636:21: empty-lines: extra empty line at the end of a block (revive)
    daemon/graphdriver/devmapper/deviceset.go:1150:70: empty-lines: extra empty line at the start of a block (revive)
    daemon/graphdriver/devmapper/deviceset.go:1613:30: empty-lines: extra empty line at the end of a block (revive)
    daemon/graphdriver/devmapper/deviceset.go:1645:65: empty-lines: extra empty line at the start of a block (revive)
    daemon/graphdriver/btrfs/btrfs.go:53:101: empty-lines: extra empty line at the start of a block (revive)
    daemon/graphdriver/devmapper/deviceset.go:1944:89: empty-lines: extra empty line at the start of a block (revive)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-28 01:58:51 +02:00
Sebastiaan van Stijn
762fc76cf9
Merge pull request #44089 from thaJeztah/update_golangci_lint
golangci-lint: update to v1.49.0
2022-09-27 18:24:15 +02:00
Sebastiaan van Stijn
2f1c382a6d
golangci-lint: update to v1.49.0
Remove the "deadcode", "structcheck", and "varcheck" linters, as they are
deprecated:

    WARN [runner] The linter 'deadcode' is deprecated (since v1.49.0) due to: The owner seems to have abandoned the linter.  Replaced by unused.
    WARN [runner] The linter 'structcheck' is deprecated (since v1.49.0) due to: The owner seems to have abandoned the linter.  Replaced by unused.
    WARN [runner] The linter 'varcheck' is deprecated (since v1.49.0) due to: The owner seems to have abandoned the linter.  Replaced by unused.
    WARN [linters context] structcheck is disabled because of generics. You can track the evolution of the generics support by following the https://github.com/golangci/golangci-lint/issues/2649.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-23 23:31:27 +02:00
Cory Snider
9ce2b30b81 pkg/containerfs: drop ContainerFS type alias
Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-09-23 16:56:53 -04:00
Cory Snider
e332c41e9d pkg/containerfs: alias ContainerFS to string
Drop the constructor and redundant string() type-casts.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-09-23 16:56:52 -04:00
Cory Snider
95824f2b5f pkg/containerfs: simplify ContainerFS type
Iterate towards dropping the type entirely.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-09-23 16:56:49 -04:00
Cory Snider
be4f4644a8 pkg/containerfs: drop Driver abstraction
The Driver abstraction was needed for Linux Containers on Windows,
support for which has since been removed.

There is no direct equivalent to Lchmod() in the standard library so
continue to use the containerd/continuity version.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-09-23 16:25:22 -04:00
Cory Snider
7014c0d65d pkg/containerfs: drop PathDriver abstraction
With LCOW support removed, there is no need to support non-native file
paths any longer.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-09-23 16:25:22 -04:00
Illo Abdulrahim
31f654a704
Fix file capabilities droping in Dockerfile
doCopyXattrs() never reached due to copyXattrs boolean being false, as
a result file capabilities not being copied.

moved copyXattr() out of doCopyXattrs()

Signed-off-by: Illo Abdulrahim <abdulrahim.illo@nokia.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-27 17:21:02 +02:00
Sebastiaan van Stijn
52c1a2fae8
gofmt GoDoc comments with go1.19
Older versions of Go don't format comments, so committing this as
a separate commit, so that we can already make these changes before
we upgrade to Go 1.19.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-08 19:56:23 +02:00
Sebastiaan van Stijn
553b0edb4c
fix unclosed file-handles in tests
These seemed to prevent cleaning up directories;

On arm64:

    === RUN   TestSysctlOverride
        testing.go:1090: TempDir RemoveAll cleanup: unlinkat /tmp/TestSysctlOverride2860094781/001/mounts/shm: device or resource busy
    --- FAIL: TestSysctlOverride (0.00s)

On Windows:

    === Failed
    === FAIL: github.com/docker/docker/daemon TestLoadOrCreateTrustKeyInvalidKeyFile (0.00s)
        testing.go:1090: TempDir RemoveAll cleanup: remove C:\Users\CONTAI~1\AppData\Local\Temp\TestLoadOrCreateTrustKeyInvalidKeyFile2014634395\001\keyfile4156691647: The process cannot access the file because it is being used by another process.

    === FAIL: github.com/docker/docker/daemon/graphdriver TestIsEmptyDir (0.01s)
        testing.go:1090: TempDir RemoveAll cleanup: remove C:\Users\CONTAI~1\AppData\Local\Temp\TestIsEmptyDir1962964337\001\dir-with-empty-file\file2523853824: The process cannot access the file because it is being used by another process.

    === FAIL: github.com/docker/docker/pkg/directory TestSizeEmptyFile (0.00s)
        testing.go:1090: TempDir RemoveAll cleanup: remove C:\Users\CONTAI~1\AppData\Local\Temp\TestSizeEmptyFile1562416712\001\file16507846: The process cannot access the file because it is being used by another process.

    === FAIL: github.com/docker/docker/pkg/directory TestSizeNonemptyFile (0.00s)
        testing.go:1090: TempDir RemoveAll cleanup: remove C:\Users\CONTAI~1\AppData\Local\Temp\TestSizeNonemptyFile1240832785\001\file3265662846: The process cannot access the file because it is being used by another process.

    === FAIL: github.com/docker/docker/pkg/directory TestSizeFileAndNestedDirectoryEmpty (0.00s)
        testing.go:1090: TempDir RemoveAll cleanup: remove C:\Users\CONTAI~1\AppData\Local\Temp\TestSizeFileAndNestedDirectoryEmpty2163416550\001\file3715413181: The process cannot access the file because it is being used by another process.

    === FAIL: github.com/docker/docker/pkg/directory TestSizeFileAndNestedDirectoryNonempty (0.00s)
        testing.go:1090: TempDir RemoveAll cleanup: remove C:\Users\CONTAI~1\AppData\Local\Temp\TestSizeFileAndNestedDirectoryNonempty878205470\001\file3280422273: The process cannot access the file because it is being used by another process.

    === FAIL: github.com/docker/docker/volume/service TestSetGetMeta (0.01s)
        testing.go:1090: TempDir RemoveAll cleanup: remove C:\Users\CONTAI~1\AppData\Local\Temp\TestSetGetMeta3332268057\001\db: The process cannot access the file because it is being used by another process.

    === FAIL: github.com/docker/docker/volume/service TestList (0.03s)
        testing.go:1090: TempDir RemoveAll cleanup: remove C:\Users\CONTAI~1\AppData\Local\Temp\TestList2846947953\001\volumes\metadata.db: The process cannot access the file because it is being used by another process.

    === FAIL: github.com/docker/docker/volume/service TestRestore (0.02s)
        testing.go:1090: TempDir RemoveAll cleanup: remove C:\Users\CONTAI~1\AppData\Local\Temp\TestRestore3368254142\001\volumes\metadata.db: The process cannot access the file because it is being used by another process.

    === FAIL: github.com/docker/docker/daemon/graphdriver TestIsEmptyDir (0.00s)
        testing.go:1090: TempDir RemoveAll cleanup: remove C:\Users\CONTAI~1\AppData\Local\Temp\TestIsEmptyDir2823795693\001\dir-with-empty-file\file2625561089: The process cannot access the file because it is being used by another process.

    === FAIL: github.com/docker/docker/pkg/directory TestSizeFileAndNestedDirectoryNonempty (0.00s)
        testing.go:1090: TempDir RemoveAll cleanup: remove C:\Users\CONTAI~1\AppData\Local\Temp\TestSizeFileAndNestedDirectoryNonempty4246252950\001\nested3442260313\file21164327: The process cannot access the file because it is being used by another process.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-05-31 21:53:38 +02:00
Bjorn Neergaard
ce3e2d1955 overlay2: account for UserNS/userxattr in metacopy test
Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
2022-05-17 06:58:50 -06:00
Bjorn Neergaard
2c3d1f7b4b overlay2: test for and report metacopy status
This is a first, naive implementation, that does not account for
userxattr/UserNS.

Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
2022-05-13 07:37:20 -06:00
Sebastiaan van Stijn
2bc07370ec
daemon/graphdriver: remove unused graphdriver.IsInitialized()
It's no longer used, and has no external consumers.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-07 16:21:31 +02:00