Commit graph

3070 commits

Author SHA1 Message Date
Sebastiaan van Stijn
e202ac3f38
remove pkg/discovery as it's now unused
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-06 18:28:28 +01:00
Tobias Klauser
cfd26afabe
Use syscall.Timespec.Unix
Use the syscall method instead of repeating the type conversions for
the syscall.Stat_t Atim/Mtim members. This also allows to drop the
//nolint: unconvert comments.

Signed-off-by: Tobias Klauser <tklauser@distanz.ch>
2022-01-03 16:51:02 +01:00
Tonis Tiigi
26dafe43a4 system: unbreak build for darwin
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-12-15 22:15:07 -08:00
Aaron Lehmann
55da5245de Fix missing parent info case in MatchesUsingParentResults
Unfortunately, this check was missing in the original version. It could
cause a positive match to be overwritten by checking parent dirs.

Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
2021-11-26 10:22:19 -08:00
Aaron Lehmann
0f1b68df16 Adjust deprecation comments
Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
2021-11-24 07:53:10 -08:00
Aaron Lehmann
4555d3aa54 pkg/fileutils: Track incremental pattern match results against each pattern
The existing code does not correctly handle the case where a file
matches one of the patterns, but should not match overall because of an
exclude pattern that applied to a parent directory (see
https://github.com/docker/buildx/issues/850).

Fix this by independently tracking the results of matching against each
pattern. A file should be considered to match any pattern that matched a
parent dir.

Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
2021-11-19 15:58:13 -08:00
Sebastiaan van Stijn
89a3c427aa
Merge pull request #42961 from milas/fix-path-match-escape
pkg/fileutils: escape additional regex meta characters
2021-10-28 20:25:21 +02:00
Akihiro Suda
4283e93e64
Merge pull request #41215 from cpuguy83/better_plugin_errors
Add more error details on plugin get
2021-10-24 12:37:07 +09:00
Milas Bowman
195558de5a pkg/fileutils: escape additional regex meta characters
There were a couple characters being explicitly escaped, but it
wasn't comprehensive.

This is now the set difference between the Golang regex meta
characters and the `filepath` match meta characters with the
exception of `\`, which already has special logic due to being
the path separator on Windows.

Signed-off-by: Milas Bowman <milasb@gmail.com>
2021-10-22 11:31:11 -04:00
Sebastiaan van Stijn
872c64cd66
Merge pull request #42862 from dkkb/feature/zstd_with_skippable_frame
compression: support zstd with skippable frame
2021-10-21 20:29:12 +02:00
Da McGrady
23abee412b
compression: support zstd with skippable frame
As a matter of fact, there are two frame formats defined by Zstandard: Zstandard frames and Skippable frames.
So we should probably support zstd algorithms with skippable frames.
See https://tools.ietf.org/id/draft-kucherawy-dispatch-zstd-00.html#rfc.section.2 for more details.

Signed-off-by: Da McGrady <dabkb@aol.com>
2021-10-15 17:23:55 +08:00
Tonis Tiigi
cec4e69813
chrootarchive: don't create parent dirs outside of chroot
If chroot is used with a special root directory then create
destination directory within chroot. This works automatically
already due to extractor creating parent paths and is only
used currently with cp where parent paths are actually required
and error will be shown to user before reaching this point.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit 52d285184068998c22632bfb869f6294b5613a58)
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit 80f1169eca)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-10-05 09:57:04 +02:00
Artem Khramov
f3d3994a4b
Port pkg/system/mknod.go to FreeBSD
Because FreeBSD uses 64-bit device nodes (see
https://reviews.freebsd.org/rS318736), Linux implementation of
`system.Mknod` & `system.Mkdev` is not sufficient.

This change adds freebsd-specific implementations for `Mknod` and
Mkdev`.

Signed-off-by: Artem Khramov <akhramov@pm.me>
2021-09-22 09:47:35 +03:00
Akihiro Suda
6014c1e29d
Merge pull request #41759 from giuseppe/zstd-compression
compression: add support for the zstd algorithm
2021-09-17 17:01:45 +09:00
Giuseppe Scrivano
e187eb2bb5
compression: add support for the zstd algorithm
zstd is a compression algorithm that has a very fast decoder, while
providing also good compression ratios.  The fast decoder makes it
suitable for container images, as decompressing the tarballs is a very
expensive operation.

https://github.com/opencontainers/image-spec/pull/788 added support
for zstd to the OCI image specs.

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2021-09-16 17:03:47 +02:00
Sebastiaan van Stijn
f586a473cf
pkg/namesgenerator: replace uses of fmt.Sprintf()
Looks like we don't need sprintf for how it's used. Replacing sprintf makes it
more performant (~2.4x as fast), and less memory, allocations:

    BenchmarkGetRandomName-8      	 8203230	       142.4 ns/op	      37 B/op	       2 allocs/op
    BenchmarkGetRandomNameOld-8   	 3499509	       342.9 ns/op	      85 B/op	       5 allocs/op

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-09-14 15:15:29 +02:00
Sebastiaan van Stijn
8fa6126f75
Merge pull request #42543 from rainrambler/patch-1
use defer to unlock mutex
2021-09-01 13:26:30 +02:00
Brian Goff
94e00d09da
Merge pull request #42797 from thaJeztah/go116_compat
pkg/plugins: fix compatibility with go1.16
2021-08-31 08:19:59 -07:00
Sebastiaan van Stijn
303ea8e820
pkg/plugins: fix compatibility with go1.16
commit c55a4ac779 changed the ioutil utilities
to use the new os variants, per recommendation from the go 1.16 release notes:
https://golang.org/doc/go1.16#ioutil

> we encourage new code to use the new definitions in the io and os packages.
> Here is a list of the new locations of the names exported by io/ioutil:

However, the devil is in the detail, and io.ReadDir() is not a direct
replacement for ioutil.ReadDir();

> ReadDir => os.ReadDir (note: returns a slice of os.DirEntry rather than a slice of fs.FileInfo)

go1.16 added a io.FileInfoToDirEntry() utility to concert a DirEntry to
a FileInfo, but it's not available in go1.16

This patch copies the FileInfoToDirEntry code, and uses it for go1.16.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-08-31 15:42:54 +02:00
Akihiro Suda
fecf45b09a
Merge pull request #42796 from thaJeztah/containerd_seccomp_check
pkg/sysinfo: use containerd/pkg/seccomp.IsEnabled()
2021-08-29 03:05:59 +09:00
Sebastiaan van Stijn
accec292c1
pkg/sysinfo: use containerd/pkg/seccomp.IsEnabled()
This replaces the local SeccompSupported() utility for the implementation in containerd,
which performs the same check.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-08-27 15:21:52 +02:00
rainrambler
7ce810c771
use defer to unlock mutex
use defer to unlock mutex (clean up)

Signed-off-by: Anyu Wang <wanganyu@outlook.com>
Signed-off-by: rainrambler <wanganyu@outlook.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-08-27 14:39:46 +02:00
Eng Zer Jun
c55a4ac779
refactor: move from io/ioutil to io and os package
The io/ioutil package has been deprecated in Go 1.16. This commit
replaces the existing io/ioutil functions with their new definitions in
io and os packages.

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2021-08-27 14:56:57 +08:00
Sebastiaan van Stijn
686be57d0a
Update to Go 1.17.0, and gofmt with Go 1.17
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-08-24 23:33:27 +02:00
Sebastiaan van Stijn
8207c05cfc
Merge pull request #41479 from olljanat/ci-win-containerd-support
Windows CI: Add support for testing with containerd
2021-08-24 22:29:14 +02:00
Brian Goff
a44a8e54ce
Merge pull request #42717 from thaJeztah/move_defaults 2021-08-24 09:33:22 -07:00
Sebastiaan van Stijn
f68260ba22
Dockerfile: frozen images: update to bullseye, remove buildpack-dep
Update the frozen images to also be based on Debian bullseye. Using the "slim"
variant (which looks to have all we're currently using),  and remove the
buildpack-dep frozen image.

The buildpack-dep image is quite large, and it looks like we only use it to
compile some C binaries, which should work fine on a regular debian image;

    docker build -t debian:bullseye-slim-gcc -<<EOF
    FROM debian:bullseye-slim
    RUN apt-get update && apt-get install -y gcc libc6-dev --no-install-recommends
    EOF

    docker image ls

    REPOSITORY        TAG                 IMAGE ID       CREATED          SIZE
    debian            bullseye-slim-gcc   1851750242af   About a minute ago   255MB
    buildpack-deps    bullseye            fe8fece98de2   2 days ago           834MB

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-08-20 18:23:06 +02:00
Sebastiaan van Stijn
98caf09f0f
fix go-digest to make sure the algorithms are registered
Previously, ioutils imported the crypty/sha256 package, because it was
used by the HashData() utility. As a side-effect of that import, the
sha256 algorithm was registered through its `init()` function.

Now that the HashData() utility is removed, the import is no longer needed
in this package, but some parts of our code depended on the side-effect, and
without this, it fail to recognise the algorithms, unless something else
happens to import crypto/sha256 / crypto/sha512, which made our
tests fail:

```
=== Failed
=== FAIL: reference TestLoad (0.00s)
    store_test.go:53: failed to parse reference: unsupported digest algorithm

=== FAIL: reference TestSave (0.00s)
    store_test.go:82: failed to parse reference: unsupported digest algorithm

=== FAIL: reference TestAddDeleteGet (0.00s)
    store_test.go:174: could not parse reference: unsupported digest algorithm

=== FAIL: reference TestInvalidTags (0.00s)
    store_test.go:355: assertion failed: error is not nil: unsupported digest algorithm
```

While it would be better to do the import in the actual locations where it's
expected, there may be code-paths we overlook, so instead adding the import
here temporarily. Until the PR in go-digest has been merged and released.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-08-20 00:35:41 +02:00
Sebastiaan van Stijn
572498be56
move pkg/ioutils.HashData() to libnetwork/resolvconf
It's the only location it's used, so we might as well move it there.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-08-20 00:35:39 +02:00
Olli Janatuinen
1285c6d125 Windows CI: Add support for testing with containerd
Signed-off-by: Olli Janatuinen <olli.janatuinen@gmail.com>
2021-08-17 07:09:40 -07:00
Aaron Lehmann
ba2adeebdb
Merge pull request #42676 from aaronlehmann/patternmatcher-doublestar-bug
fileutils: Fix incorrect handling of "**/foo" pattern
2021-08-16 19:58:55 -07:00
Aaron Lehmann
c44b90f3bf Test fix for Windows compatibility
Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
2021-08-12 20:03:41 -07:00
Aaron Lehmann
97ede9df26 Rename Matches to MatchesOrParentMatches
Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
2021-08-12 18:10:04 -07:00
Aaron Lehmann
9bae4f2f24 Add more optimal MatchesUsingParentResult method, use it in pkg/archive
Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
2021-08-12 13:57:50 -07:00
Sebastiaan van Stijn
e53f65a916
pkg/signal: remove DefaultStopSignal const
This const was previously living in pkg/signal, but with that package
being moved to its own module, it didn't make much sense to put docker's
defaults in a generic module.

The const from the "signal" package is currenlty used *both* by the CLI
and the daemon as a default value when creating containers. This put up
some questions:

a. should the default be non-exported, and private to the container
   package? After all, it's a _default_ (so should be used if _NOT_ set).
b. should the client actually setting a default, or instead just omit
   the value, unless specified by the user? having the client set a
   default also means that the daemon cannot change the default value
   because the client (or older clients) will override it.
c. consider defaults from the client and defaults of the daemon to be
   separate things, and create a default const in the CLI.

This patch implements option "a" (option "b" will be done separately,
as it involves the CLI code). This still leaves "c" open as an option,
if the CLI wants to set its own default.

Unfortunately, this change means we'll have to drop the alias for the
deprecated pkg/signal.DefaultStopSignal const, but a comment was left
instead, which can assist consumers of the const to find why it's no
longer there (a search showed the Docker CLI as the only consumer though).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-08-11 10:31:29 +02:00
Samuel Karp
8b43104937
Merge pull request #42672 from thaJeztah/pkg_archive_debuglogs 2021-08-05 12:04:59 -07:00
Brian Goff
51b06c6795
Merge pull request #42683 from thaJeztah/remove_lcow_step6
Remove LCOW (step 6)
2021-07-29 11:34:29 -07:00
Brian Goff
ad268e79c4
Merge pull request #42193 from lzhfromustc/3_23
discovery & test: Fix goroutine leaks by adding 1 buffer to channel
2021-07-28 15:25:37 -07:00
Sebastiaan van Stijn
e3491ab56a
pkg/archive: gzDecompress(): use local vars for MOBY_DISABLE_PIGZ
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-28 00:11:48 +02:00
Sebastiaan van Stijn
b321474747
pkg/archive: DetectCompression(): use bytes.HasPrefix()
The existing code was the exact equivalent of bytes.HasPrefix();

    // HasPrefix tests whether the byte slice s begins with prefix.
    func HasPrefix(s, prefix []byte) bool {
    	return len(s) >= len(prefix) && Equal(s[0:len(prefix)], prefix)
    }

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-28 00:11:46 +02:00
Sebastiaan van Stijn
ebeda658bc
pkg/archive: remove some debug logs
These seemed fairly redundant

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-28 00:11:43 +02:00
Sebastiaan van Stijn
13cb04e57c
remove various LCOW bits (container, image, pkg/containerfs)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-27 13:36:21 +02:00
Sebastiaan van Stijn
0998c7e363
system: remove LCOWSupported() utility
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-27 13:36:19 +02:00
Aaron Lehmann
90f8d1b675 fileutils: Fix incorrect handling of "**/foo" pattern
(*PatternMatcher).Matches includes a special case for when the pattern
matches a parent dir, even though it doesn't match the current path.
However, it assumes that the parent dir which would match the pattern
must have the same number of separators as the pattern itself. This
doesn't hold true with a patern like "**/foo". A file foo/bar would have
len(parentPathDirs) == 1, which is less than the number of path
len(pattern.dirs) == 2... therefore this check would be skipped.

Given that "**/foo" matches "foo", I think it's a bug that the "parent
subdir matches" check is being skipped in this case.

It seems safer to loop over the parent subdirs and check each against
the pattern. It's possible there is a safe optimization to check only a
certain subset, but the existing logic seems unsafe.

Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
2021-07-26 11:28:10 -07:00
Sebastiaan van Stijn
28409ca6c7
replace pkg/signal with moby/sys/signal v0.5.0
This code was moved to the moby/sys repository

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-23 09:32:54 +02:00
Brian Goff
9a6ff685a8
Merge pull request #42641 from thaJeztah/make_signal_selfcontained 2021-07-19 14:46:15 -07:00
Justin Cormack
b337c70bdc
Merge pull request #42639 from thaJeztah/system_info_clean
pkg/sysinfo: assorted cleanup/refactoring for handling warnings and logging
2021-07-19 15:17:07 +01:00
Justin Cormack
fb21a1e474
Merge pull request #42580 from thaJeztah/reduce_TestClientWithRequestTimeout_flakiness
Reduce TestClientWithRequestTimeout flakiness
2021-07-19 15:11:43 +01:00
Sebastiaan van Stijn
6ff6913ac4
pkg/signal: remove gotest.tools dependency
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-15 18:24:23 +02:00
Sebastiaan van Stijn
0880df4644
pkg/signal: move Trap() to cmd/dockerd
It's the only location where this is used, and it's quite specific
to dockerd (not really a reusable function for external use), so
moving it into that package.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-15 18:11:00 +02:00