Commit graph

3103 commits

Author SHA1 Message Date
CrazyMax
5d9e99ead3
missing windows arm64 arch detection
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-04-09 16:10:42 +02:00
Sebastiaan van Stijn
0a3336fd7d
Merge pull request #43366 from corhere/finish-identitymapping-refactor
Finish refactor of UID/GID usage to a new struct
2022-03-25 14:51:05 +01:00
Sebastiaan van Stijn
54eeff6eb3
Merge pull request #43385 from thaJeztah/move_IsWindowsClient
pkg/system: remove deprecated/unused consts and move IsWindowsClient()
2022-03-18 15:29:32 +01:00
Sebastiaan van Stijn
9bf40d7edd
pkg/system: move IsWindowsClient to pkg/parsers/operatingsystem
This function was only used in a single place, and pkg/parsers/operatingsystem
already copied the `verNTWorkstation` const, so we might as well move this function
there as well to "unclutter" pkg/system.

The function had no external users, so not adding an alias / stub.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-17 10:26:50 +01:00
Sebastiaan van Stijn
5d10c6ec67
Update handling of deprecated kernel (tcp) memory options
- Omit `KernelMemory` and `KernelMemoryTCP` fields in `/info` response if they're
  not supported, or when using API v1.42 or up.
- Re-enable detection of `KernelMemory` (as it's still needed for older API versions)
- Remove warning about kernel memory TCP in daemon logs (a warning is still returned
  by the `/info` endpoint, but we can consider removing that).
- Prevent incorrect "Minimum kernel memory limit allowed" error if the value was
  reset because it's not supported by the host.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-17 09:56:39 +01:00
aiordache
af6307fbda
Remove KernelMemory option from /containers/create and /update endpoints
- remove KernelMemory option from `v1.42` api docs
 - remove KernelMemory warning on `/info`
 - update changes for `v1.42`
 - remove `KernelMemory` field from endpoints docs

Signed-off-by: aiordache <anca.iordache@docker.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-17 09:55:36 +01:00
Akihiro Suda
7f99438ce7
Merge pull request #43194 from thaJeztah/bump_golang_1.18
Update golang to 1.18.0
2022-03-17 12:52:16 +09:00
Sebastiaan van Stijn
85c4d633db
pkg/system: remove deprecated (and unused) windows consts
These consts were deprecated in 46c591b045, and
although that has not been in a release yet (we usually deprecate for at least
one release before removing), doing a search showed that there were no external
consumers of these consts, so it should be fine to remove them.

This patch removes the consts that were moded to pkg/idtools;

- SeTakeOwnershipPrivilege
- ContainerAdministratorSidString
- ContainerUserSidString

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-16 12:42:57 +01:00
Sebastiaan van Stijn
1133d55770
Merge pull request #43321 from corhere/43284-report-displayversion
pkg/parsers: support Windows 11: report DisplayVersion; drop ProductName
2022-03-16 12:41:50 +01:00
Sebastiaan van Stijn
4203a97aad
staticcheck: ignore "SA1019: strings.Title is deprecated"
This function is marked deprecated in Go 1.18; however, the suggested replacement
brings in a large amount of new code, and most strings we generate will be ASCII,
so this would only be in case it's used for some user-provided string. We also
don't have a language to use, so would be using the "default".

Adding a `//nolint` comment to suppress the linting failure instead.

    daemon/logger/templates/templates.go:23:14: SA1019: strings.Title is deprecated: The rule Title uses for word boundaries does not handle Unicode punctuation properly. Use golang.org/x/text/cases instead. (staticcheck)
        "title":    strings.Title,
                    ^
    pkg/plugins/pluginrpc-gen/template.go:67:9: SA1019: strings.Title is deprecated: The rule Title uses for word boundaries does not handle Unicode punctuation properly. Use golang.org/x/text/cases instead. (staticcheck)
        return strings.Title(s)
               ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-16 12:11:54 +01:00
Cory Snider
098a44c07f Finish refactor of UID/GID usage to a new struct
Finish the refactor which was partially completed with commit
34536c498d, passing around IdentityMapping structs instead of pairs of
[]IDMap slices.

Existing code which uses []IDMap relies on zero-valued fields to be
valid, empty mappings. So in order to successfully finish the
refactoring without introducing bugs, their replacement therefore also
needs to have a useful zero value which represents an empty mapping.
Change IdentityMapping to be a pass-by-value type so that there are no
nil pointers to worry about.

The functionality provided by the deprecated NewIDMappingsFromMaps
function is required by unit tests to to construct arbitrary
IdentityMapping values. And the daemon will always need to access the
mappings to pass them to the Linux kernel. Accommodate these use cases
by exporting the struct fields instead. BuildKit currently depends on
the UIDs and GIDs methods so we cannot get rid of them yet.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-03-14 16:28:57 -04:00
Sebastiaan van Stijn
8539d06209
Merge pull request #43347 from corhere/42452-sysinfo-remove-libcontainer
pkg/sysinfo: remove libcontainer dependency
2022-03-09 18:25:00 +01:00
Cory Snider
b0b71dbe1c pkg/sysinfo: remove libcontainer dependency
Reimplement GetCgroupMounts using the github.com/containerd/cgroups and
github.com/moby/sys/mountinfo packages.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-03-07 18:09:09 -05:00
Sebastiaan van Stijn
d89bfac728
remove deprecated pkg/symlink
This package was deprecated in dc3c382b34, which
was part of the 20.10 release, so consumers of this package should've been
able to migrate to the new location.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-06 00:24:22 +01:00
Sebastiaan van Stijn
7449ca078b
remove deprecated pkg/locker
This package was deprecated in 5ca758199d, which
was part of the 20.10 release, so consumers of this package should've been
able to migrate to the new location.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-06 00:20:46 +01:00
Sebastiaan van Stijn
4a6dc85e40
remove deprecated pkg/term, pkg/term/windows
This package was deprecated in 41d4112e89, which
was part of the 20.10 release, so consumers of this package should've been
able to migrate to the new location.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-06 00:18:46 +01:00
Sebastiaan van Stijn
97a235196e
remove deprecated pkg/mount
This package was deprecated in 99beb2ca02, which
was part of the 20.10 release, so consumers of this package should've been
able to migrate to the new location.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-06 00:13:05 +01:00
Cory Snider
9aacaeb667 pkg/parsers: support Windows 11; drop ProductName
Microsoft has stopped updating the ProductName registry value in Windows
11; it reads as Windows 10. And Microsoft has made it very difficult to
look up the real product name programmatically so that applications do
not attempt to parse it. (Ever wonder why they skipped Windows 9?) The
only documented and supported mechanisms require WMI or WinRT. The
product name has no bearing on application compatibility so it is not
worth doing any heroics to display the correct name. The build number
and Update Build Revision is sufficient information to identify a
specific build of Windows. Stop displaying the ProductName so as not to
confuse users with incorrect information.

Microsoft has frozen the ReleaseId registry value at 2009 when they
switched to semi-annual releases and alpha-numeric versions. The release
version as displayed by winver.exe and Settings -> System -> About on
Windows 20H2 and newer can be found in the new DisplayVersion registry
value. Replicate the way winver.exe displays the version by
preferentially reporting the DisplayVersion if present and reporting if
it is a Windows Server edition.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-03-04 16:08:34 -05:00
Sebastiaan van Stijn
dee9f422c8
pkg/system: remove github.com/docker/go-units dependency
This is not "very" important, but this dependency was only used
for a single const, which could be satisfied with a comment.

Not very urgent, as github.com/docker/go-units is likely imported
through other ways already (but it's nice to have the package be
more isolated).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-03 00:22:32 +01:00
Sebastiaan van Stijn
25ee00c494
pkg/system: move EnsureRemoveAll() to pkg/containerfs
pkg/system historically has been a bit of a kitchen-sink of things that were
somewhat "system" related, but didn't have a good place for. EnsureRemoveAll()
is one of those utilities. EnsureRemoveAll() is used to both unmount and remove
a path, for which it depends on both github.com/moby/sys/mount, which in turn
depends on github.com/moby/sys/mountinfo.

pkg/system is imported in the CLI, but neither EnsureRemoveAll(), nor any of its
moby/sys dependencies are used on the client side, so let's move this function
somewhere else, to remove those dependencies from the CLI.

I looked for plausible locations that were related; it's used in:

- daemon
- daemon/graphdriver/XXX/
- plugin

I considered moving it into a (e.g.) "utils" package within graphdriver (but not
a huge fan of "utils" packages), and given that it felt (mostly) related to
cleaning up container filesystems, I decided to move it there.

Some things to follow-up on after this:

- Verify if this function is still needed (it feels a bit like a big hammer in
  a "YOLO, let's try some things just in case it fails")
- Perhaps it should be integrated in `containerfs.Remove()` (so that it's used
  automatically)
- Look if there's other implementations (and if they should be consolidated),
  although (e.g.) the one in containerd is a copy of ours:
  https://github.com/containerd/containerd/blob/v1.5.9/pkg/cri/server/helpers_linux.go#L200

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-03 00:22:26 +01:00
Sebastiaan van Stijn
c72c1ca62c
Merge pull request #43185 from corhere/42402-safer-fileinfo
Remove local fork of archive/tar package
2022-02-24 10:36:14 +01:00
Sebastiaan van Stijn
705f9b68cc
some cleaning up of isolation checks, and platform information
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-02-18 22:58:37 +01:00
Cory Snider
833139f390 pkg/archive: audit gosec file-traversal lints
The recently-upgraded gosec linter has a rule for archive extraction
code which may be vulnerable to directory traversal attacks, a.k.a. Zip
Slip. Gosec's detection is unfortunately prone to false positives,
however: it flags any filepath.Join call with an argument derived from a
tar.Header value, irrespective of whether the resultant path is used for
filesystem operations or if directory traversal attacks are guarded
against.

All of the lint errors reported by gosec appear to be false positives.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-02-18 15:42:22 -05:00
Cory Snider
e9bbc41dd1 Remove local fork of archive/tar package
A copy of Go's archive/tar packge was vendored with a patch applied to
mitigate CVE-2019-14271. Vendoring standard library packages is not
supported by Go in module-aware mode, which is getting in the way of
maintenance. A different approach to mitigate the vulnerability is
needed which does not involve vendoring parts of the standard library.

glibc implements name service lookups such as users, groups and DNS
using a scheme known as Name Service Switch. The services are
implemented as modules, shared libraries which glibc dynamically links
into the process the first time a function requiring the module is
called. This is the crux of the vulnerability: if a process linked
against glibc chroots, then calls one of the functions implemented with
NSS for the first time, glibc may load NSS modules out of the chrooted
filesystem.

The API underlying the `docker cp` command is implemented by forking a
new process which chroots into the container's rootfs and writes a tar
stream of files from the container over standard output. It utilizes the
Go standard library's archive/tar package to write the tar stream. It
makes use of the tar.FileInfoHeader function to construct a tar.Header
value from an fs.FileInfo value. In modern versions of Go on *nix
platforms, FileInfoHeader will attempt to resolve the file's UID and GID
to their respective user and group names by calling the os/user
functions LookupId and LookupGroupId. The cgo implementation of os/user
on *nix performs lookups by calling the corresponding libc functions. So
when linked against glibc, calls to tar.FileInfoHeader after the
process has chrooted into the container's rootfs can have the side
effect of loading NSS modules from the container! Without any
mitigations, a malicious container image author can trivially get
arbitrary code execution by leveraging this vulnerability and escape the
chroot (which is not a sandbox) into the host.

Mitigate the vulnerability without patching or forking archive/tar by
hiding the OS-dependent file info from tar.FileInfoHeader which it needs
to perform the lookups. Without that information available it falls back
to populating the tar.Header with only the information obtainable
directly from the FileInfo value without making any calls into os/user.

Fixes #42402

Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-02-18 13:40:19 -05:00
Sebastiaan van Stijn
5c8d50a132
Merge pull request #43210 from tianon/frozen-names-generator
Freeze the namesgenerator package against new additions
2022-02-10 20:38:44 +01:00
Tianon Gravi
624b3cfbe8 Freeze the namesgenerator package against new additions
See the added comment/documentation within the package for more details.

Signed-off-by: Tianon Gravi <admwiggin@gmail.com>
2022-02-10 09:56:08 -08:00
Sebastiaan van Stijn
347a85787b
pkg/namesgenerator: remove names-generator binary
This binary was added in 0800650665, but no motivation,
other than "this could be nice". Searching if it's used by anyone, I found only
two occurrences, both of which were just examples, or indexing results;

- eae845e40e/tree-examples/docker.txt (L1229)
- 8b1e0b89c7/sync/cache/librariesioGoGo.json (L2200)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-02-04 10:37:19 +01:00
Sebastiaan van Stijn
ab9796b0e9
pkg/system: rewrite IsWindowsClient() using golang.org/x/sys/windows
Looks like we should be able to use the implementation from x/sys/windows.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-26 13:39:03 +01:00
Sebastiaan van Stijn
be463cbd6c
pkg/system: remove unused HasWin32KSupport()
This was added in 194eaa5c0f to check image
compatibility based on Platform.Features;

    // For now, hard code that all base images except nanoserver depend on win32k support
    if imageData.Name != "nanoserver" {
        imageData.OSFeatures = append(imageData.OSFeatures, "win32k")
    }

But no longer used since 1f59bc8c03 and
d231260868

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-26 13:11:58 +01:00
Aaron Lehmann
40bb983175
Merge pull request #43095 from aaronlehmann/avoid-regexp-simple-cases
fileutils: Avoid compiling a regexp for simple patterns
2022-01-20 17:43:07 -08:00
Aaron Lehmann
38e2e756c9 Avoid platform-specific NewPatternMatcher function in TestCompile
Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
2022-01-20 09:08:16 -08:00
Aaron Lehmann
8d250be008 Add unit test for (*Pattern).compile
Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
2022-01-12 10:31:35 -08:00
Sebastiaan van Stijn
e202ac3f38
remove pkg/discovery as it's now unused
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-06 18:28:28 +01:00
Tobias Klauser
cfd26afabe
Use syscall.Timespec.Unix
Use the syscall method instead of repeating the type conversions for
the syscall.Stat_t Atim/Mtim members. This also allows to drop the
//nolint: unconvert comments.

Signed-off-by: Tobias Klauser <tklauser@distanz.ch>
2022-01-03 16:51:02 +01:00
Aaron Lehmann
dd66dcad9c fileutils: Avoid compiling a regexp for simple patterns
If we detect that a pattern is either an exact match, prefix match, or
suffix match, use an optimized code path instead of compiling a regexp.

Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
2021-12-20 16:32:13 -08:00
Tonis Tiigi
26dafe43a4 system: unbreak build for darwin
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-12-15 22:15:07 -08:00
Aaron Lehmann
55da5245de Fix missing parent info case in MatchesUsingParentResults
Unfortunately, this check was missing in the original version. It could
cause a positive match to be overwritten by checking parent dirs.

Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
2021-11-26 10:22:19 -08:00
Aaron Lehmann
0f1b68df16 Adjust deprecation comments
Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
2021-11-24 07:53:10 -08:00
Aaron Lehmann
4555d3aa54 pkg/fileutils: Track incremental pattern match results against each pattern
The existing code does not correctly handle the case where a file
matches one of the patterns, but should not match overall because of an
exclude pattern that applied to a parent directory (see
https://github.com/docker/buildx/issues/850).

Fix this by independently tracking the results of matching against each
pattern. A file should be considered to match any pattern that matched a
parent dir.

Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
2021-11-19 15:58:13 -08:00
Sebastiaan van Stijn
89a3c427aa
Merge pull request #42961 from milas/fix-path-match-escape
pkg/fileutils: escape additional regex meta characters
2021-10-28 20:25:21 +02:00
Akihiro Suda
4283e93e64
Merge pull request #41215 from cpuguy83/better_plugin_errors
Add more error details on plugin get
2021-10-24 12:37:07 +09:00
Milas Bowman
195558de5a pkg/fileutils: escape additional regex meta characters
There were a couple characters being explicitly escaped, but it
wasn't comprehensive.

This is now the set difference between the Golang regex meta
characters and the `filepath` match meta characters with the
exception of `\`, which already has special logic due to being
the path separator on Windows.

Signed-off-by: Milas Bowman <milasb@gmail.com>
2021-10-22 11:31:11 -04:00
Sebastiaan van Stijn
872c64cd66
Merge pull request #42862 from dkkb/feature/zstd_with_skippable_frame
compression: support zstd with skippable frame
2021-10-21 20:29:12 +02:00
Da McGrady
23abee412b
compression: support zstd with skippable frame
As a matter of fact, there are two frame formats defined by Zstandard: Zstandard frames and Skippable frames.
So we should probably support zstd algorithms with skippable frames.
See https://tools.ietf.org/id/draft-kucherawy-dispatch-zstd-00.html#rfc.section.2 for more details.

Signed-off-by: Da McGrady <dabkb@aol.com>
2021-10-15 17:23:55 +08:00
Tonis Tiigi
cec4e69813
chrootarchive: don't create parent dirs outside of chroot
If chroot is used with a special root directory then create
destination directory within chroot. This works automatically
already due to extractor creating parent paths and is only
used currently with cp where parent paths are actually required
and error will be shown to user before reaching this point.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit 52d285184068998c22632bfb869f6294b5613a58)
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit 80f1169eca)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-10-05 09:57:04 +02:00
Artem Khramov
f3d3994a4b
Port pkg/system/mknod.go to FreeBSD
Because FreeBSD uses 64-bit device nodes (see
https://reviews.freebsd.org/rS318736), Linux implementation of
`system.Mknod` & `system.Mkdev` is not sufficient.

This change adds freebsd-specific implementations for `Mknod` and
Mkdev`.

Signed-off-by: Artem Khramov <akhramov@pm.me>
2021-09-22 09:47:35 +03:00
Akihiro Suda
6014c1e29d
Merge pull request #41759 from giuseppe/zstd-compression
compression: add support for the zstd algorithm
2021-09-17 17:01:45 +09:00
Giuseppe Scrivano
e187eb2bb5
compression: add support for the zstd algorithm
zstd is a compression algorithm that has a very fast decoder, while
providing also good compression ratios.  The fast decoder makes it
suitable for container images, as decompressing the tarballs is a very
expensive operation.

https://github.com/opencontainers/image-spec/pull/788 added support
for zstd to the OCI image specs.

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2021-09-16 17:03:47 +02:00
Sebastiaan van Stijn
f586a473cf
pkg/namesgenerator: replace uses of fmt.Sprintf()
Looks like we don't need sprintf for how it's used. Replacing sprintf makes it
more performant (~2.4x as fast), and less memory, allocations:

    BenchmarkGetRandomName-8      	 8203230	       142.4 ns/op	      37 B/op	       2 allocs/op
    BenchmarkGetRandomNameOld-8   	 3499509	       342.9 ns/op	      85 B/op	       5 allocs/op

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-09-14 15:15:29 +02:00
Sebastiaan van Stijn
8fa6126f75
Merge pull request #42543 from rainrambler/patch-1
use defer to unlock mutex
2021-09-01 13:26:30 +02:00