Commit graph

43556 commits

Author SHA1 Message Date
Akihiro Suda
d809ad98e3
Merge pull request #43183 from thaJeztah/cleanup_distribution
distribution/xfer: refactor to reduce public api/interface
2022-02-26 23:50:03 +09:00
Tianon Gravi
c549116f32
Merge pull request #43280 from thaJeztah/remove_deprecated_dualstack
distribution: remove use of deprecated dial.DualStack
2022-02-25 16:22:49 -08:00
Tianon Gravi
c78f6963a1
Merge pull request #43289 from thaJeztah/bump_hcsshim
vendor: github.com/Microsoft/hcsshim v0.9.2
2022-02-24 14:24:38 -08:00
Sebastiaan van Stijn
9d6382f2b3
vendor: github.com/Microsoft/hcsshim v0.9.2
full diff: https://github.com/Microsoft/hcsshim/compare/v0.8.23...v0.9.2

diff is hard to compare on github, because Microsoft/opengcs was merged into
hcsshim; https://github.com/microsoft/hcsshim/pull/973

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-02-24 21:28:18 +01:00
Brian Goff
5c36bb7206
Merge pull request #43287 from thaJeztah/bump_dependencies_for_buildkit
vendor: update various dependencies in preparation of BuildKit update
2022-02-24 11:05:48 -08:00
Sebastiaan van Stijn
c72c1ca62c
Merge pull request #43185 from corhere/42402-safer-fileinfo
Remove local fork of archive/tar package
2022-02-24 10:36:14 +01:00
Akihiro Suda
48d08a7b77
Merge pull request #43288 from thaJeztah/bump_aws_sdk
vendor: github.com/aws/aws-sdk-go v1.31.6
2022-02-24 11:56:59 +09:00
Brian Goff
7c764a29b0
Merge pull request #43235 from thaJeztah/api_wait_fixes
Assorted (documentation) fixes for `POST /containers/{id}/wait`
2022-02-23 15:13:18 -08:00
Sebastiaan van Stijn
ace606fd75
vendor: github.com/aws/aws-sdk-go v1.31.6
full diff: https://github.com/aws/aws-sdk-go/compare/v1.28.11...v1.31.6

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-02-23 19:49:51 +01:00
Sebastiaan van Stijn
0809bd6859
vendor: github.com/klauspost/compress v1.14.2
full diff: https://github.com/klauspost/compress/compare/v1.12.3...v1.14.2

Relevant changes affecting vendor:

- docs: Add combined LICENSE file
- Add snappy replacement package
- tests: Remove snappy dependency for tests
- huff0: Add size estimation function
- huff0: Improve 4X decompression speed
- huff0: Improve 4X decompression speed 5-10%
- huff0: Faster 1X Decompression
- zstd: Spawn decoder goroutine only if needed
- zstd: Detect short invalid signatures
- zstd: Add configurable Decoder window size
- zstd: Add stream content size
- zstd: Simplify hashing functions
- zstd: use SpeedBestCompression for level >= 10
- zstd: Fix WriteTo error forwarding
- zstd: Improve Best compression
- zstd: Fix incorrect encoding in best mode
- zstd: pooledZipWriter should return Writers to the same pool
- zstd: Upgrade xxhash
- zstd: Improve block encoding speed
- zstd: add arm64 xxhash assembly
- zstd: Minor decoder improvements
- zstd: Minor performance tweaks
- zstd: Add bigger default blocks
- zstd: Remove unused decompression buffer
- zstd: fix logically dead code
- zstd: Add noasm tag for xxhash
- zstd: improve header decoder

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-02-23 19:41:14 +01:00
Sebastiaan van Stijn
20e5d6b3e3
vendor: github.com/pelletier/go-toml v1.9.4
full diff: https://github.com/pelletier/go-toml/compare/v1.9.1...v1.9.4

- Add Encoder.CompactComments to omit extra new line
- Clarify license and comply with Apache 2.0
- feat(tomll): add multiLineArray flag to linter

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-02-23 19:25:30 +01:00
Sebastiaan van Stijn
812dacb676
vendor: github.com/containerd/continuity v0.2.2
full diff: https://github.com/containerd/continuity/compare/v0.1.0...v0.2.2

- fs/stat: add FreeBSD, and cleanup some nolint-comments
- go.mod: bazil.org/fuse v0.0.0-20200407214033-5883e5a4b5125
- Fix darwin issues
- Remove direct dependency on github.com/pkg/errors
- Do not log errors before returning them
- Build containerd/continuity on multiple Unix OSes
- Update CI Go version to 1.17
- fs: use syscall.Timespec.Unix

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-02-23 19:22:36 +01:00
Sebastiaan van Stijn
461845bfbc
vendor: github.com/containerd/cgroups v1.0.3
full diff: https://github.com/containerd/cgroups/compare/v1.0.1...v1.0.3

- cgroup v1: implement AddProc()
- cgroup v1: reduce duplicated code
- cgroup v2: Fix potential dirfd leak
- cgroup v2: remove unimplemented errors and ErrorHandler, IgnoreNotExist
- cgroup v2: v2: Fix inotify fd leak when cgroup is deleted
- cgroup.go: avoid panic on nil interface
- cgroup: Optionally add process and task to a subsystems subset
- fix Implicit memory aliasing in for loop
- go.mod: coreos/go-systemd/v22 v22.3.2 to prepare for deprecations
- Improvements on cgroup v2 support
- replace pkg/errors from vendor
- Use /proc/partitions to get device names
- utils: export ParseCgroupFile()

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-02-23 19:04:43 +01:00
Sebastiaan van Stijn
7876c53424
vendor: golang.org/x/tools v0.1.5
full diff: https://github.com/golang/tools/compare/v0.1.0...v0.1.5

It's not used, but one of our dependencies has a `tools.go` file that forces
it to be vendored; vendor/cloud.google.com/go/tools.go

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-02-23 18:42:17 +01:00
Sebastiaan van Stijn
1b829c2a6a
vendor: golang.org/x/mod v0.4.2
full diff: https://github.com/golang/mod/compare/v0.4.1...v0.4.2

It's not used, but one of our dependencies has a `tools.go` file that forces
it to be vendored; vendor/cloud.google.com/go/tools.go

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-02-23 18:37:58 +01:00
Sebastiaan van Stijn
6be521ccb8
vendor: golang.org/x/oauth2 v0.0.0-20210805134026-6f1e6394065a
full diff: f6687ab280...6f1e639406

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-02-23 18:30:08 +01:00
Sebastiaan van Stijn
8f106d4576
vendor: github.com/google/go-cmp v0.5.6
full diff: http://github.com/google/go-cmp/compare/v0.5.5...v0.5.6

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-02-23 18:29:02 +01:00
Sebastiaan van Stijn
9144d1705c
Merge pull request #43282 from Cornul11/43265-fix-docker-ce-mentions
Removed mentions of the deprecated Docker CE repo
2022-02-23 18:25:28 +01:00
Dan Plamadeala
9393cf73f6 removed mentions of the archived Docker CE repo
Signed-off-by: Dan Plamadeala <cornul11@gmail.com>
2022-02-23 12:13:12 +01:00
Sebastiaan van Stijn
ff408210da
distribution: remove use of deprecated dial.DualStack
From the field's description [1]:

    DualStack previously enabled RFC 6555 Fast Fallback
    support, also known as "Happy Eyeballs", in which IPv4 is
    tried soon if IPv6 appears to be misconfigured and
    hanging.

    Deprecated: Fast Fallback is enabled by default. To
    disable, set FallbackDelay to a negative value.

This field was deprecated in efc185029b,
which is included in Go 1.12beta1 and up.

[1]: 2ebe77a2fd/src/net/dial.go (L54-L61)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-02-22 16:34:12 +01:00
Sebastiaan van Stijn
dc8fb8f03b
Merge pull request #43254 from thaJeztah/windows_rs5_minimum
Windows: require Windows Server RS5 / ltsc2019 (build 17763) as minimum
2022-02-19 00:35:45 +01:00
Sebastiaan van Stijn
705f9b68cc
some cleaning up of isolation checks, and platform information
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-02-18 22:58:37 +01:00
Sebastiaan van Stijn
1b3fef5333
Windows: require Windows Server RS5 / ltsc2019 (build 17763) as minimum
Windows Server 2016 (RS1) reached end of support, and Docker Desktop requires
Windows 10 V19H2 (version 1909, build 18363) as a minimum.

This patch makes Windows Server RS5 /  ltsc2019 (build 17763) the minimum version
to run the daemon, and removes some hacks for older versions of Windows.

There is one check remaining that checks for Windows RS3 for a workaround
on older versions, but recent changes in Windows seemed to have regressed
on the same issue, so I kept that code for now to check if we may need that
workaround (again);

085c6a98d5/daemon/graphdriver/windows/windows.go (L319-L341)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-02-18 22:58:28 +01:00
Cory Snider
833139f390 pkg/archive: audit gosec file-traversal lints
The recently-upgraded gosec linter has a rule for archive extraction
code which may be vulnerable to directory traversal attacks, a.k.a. Zip
Slip. Gosec's detection is unfortunately prone to false positives,
however: it flags any filepath.Join call with an argument derived from a
tar.Header value, irrespective of whether the resultant path is used for
filesystem operations or if directory traversal attacks are guarded
against.

All of the lint errors reported by gosec appear to be false positives.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-02-18 15:42:22 -05:00
Cory Snider
e9bbc41dd1 Remove local fork of archive/tar package
A copy of Go's archive/tar packge was vendored with a patch applied to
mitigate CVE-2019-14271. Vendoring standard library packages is not
supported by Go in module-aware mode, which is getting in the way of
maintenance. A different approach to mitigate the vulnerability is
needed which does not involve vendoring parts of the standard library.

glibc implements name service lookups such as users, groups and DNS
using a scheme known as Name Service Switch. The services are
implemented as modules, shared libraries which glibc dynamically links
into the process the first time a function requiring the module is
called. This is the crux of the vulnerability: if a process linked
against glibc chroots, then calls one of the functions implemented with
NSS for the first time, glibc may load NSS modules out of the chrooted
filesystem.

The API underlying the `docker cp` command is implemented by forking a
new process which chroots into the container's rootfs and writes a tar
stream of files from the container over standard output. It utilizes the
Go standard library's archive/tar package to write the tar stream. It
makes use of the tar.FileInfoHeader function to construct a tar.Header
value from an fs.FileInfo value. In modern versions of Go on *nix
platforms, FileInfoHeader will attempt to resolve the file's UID and GID
to their respective user and group names by calling the os/user
functions LookupId and LookupGroupId. The cgo implementation of os/user
on *nix performs lookups by calling the corresponding libc functions. So
when linked against glibc, calls to tar.FileInfoHeader after the
process has chrooted into the container's rootfs can have the side
effect of loading NSS modules from the container! Without any
mitigations, a malicious container image author can trivially get
arbitrary code execution by leveraging this vulnerability and escape the
chroot (which is not a sandbox) into the host.

Mitigate the vulnerability without patching or forking archive/tar by
hiding the OS-dependent file info from tar.FileInfoHeader which it needs
to perform the lookups. Without that information available it falls back
to populating the tar.Header with only the information obtainable
directly from the FileInfo value without making any calls into os/user.

Fixes #42402

Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-02-18 13:40:19 -05:00
Sebastiaan van Stijn
79ea1b16b1
distribution/xfer: un-export DoFunc type
It's only used internally and by non-exported functions, so it doesn't
have to be exported.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-02-18 16:58:42 +01:00
Sebastiaan van Stijn
4d6c6a5000
distribution/xfer: smaller interface, rename to DigestRegisterer
All uses of this interface already accept a DownloadDescriptor; keeping the
interface small to allow this functionality to be used by other download-descriptors,
while still being able to check for the actual functionality (to be able to register
a digest).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-02-18 16:58:40 +01:00
Sebastiaan van Stijn
047e032461
distribution/xfer: define DownloadOption type
This is mostly for documentation purposes; defining a type makes
the option(s) show up grouped on pkg.go.dev (and in godoc).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-02-18 16:58:38 +01:00
Sebastiaan van Stijn
69b0913e1f
distribution/xfer: add TODO for return error on Registered()
There's currently four implementations in our code: one in distribution, two in
BuildKit-related code, and one "mock" for testing:

- 65b8bcc321/builder/builder-next/worker/worker.go (L452-L455)
- 65b8bcc321/builder/builder-next/adapters/containerimage/pull.go (L648-L651)
- 65b8bcc321/distribution/pull_v2.go (L318-L321)
- 65b8bcc321/distribution/xfer/download_test.go (L186-L188)

All of these call a metadata-service to register the DiffID, and all of those
may return an error, which currently gets ignored.

We should consider changing the signature to return possible errors, so that
the callsite can decide how to handle them (which could be just "log" the error
or handle them in other ways).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-02-18 16:58:36 +01:00
Sebastiaan van Stijn
3309ea9419
distribution/xfer: slightly more idomatic type assertion
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-02-18 16:58:35 +01:00
Sebastiaan van Stijn
4c7dc9806c
distribution/xfer: un-export transfer interface accessors
This interface is internal to the package, so there's no need to export
it's methods.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-02-18 16:58:33 +01:00
Sebastiaan van Stijn
849d8c2d02
distribution/xfer: un-export Watcher
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-02-18 16:58:31 +01:00
Sebastiaan van Stijn
96a750aab4
distribution/xfer: un-export Transfer interface
The only implementations and uses are internal to this package.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-02-18 16:58:29 +01:00
Sebastiaan van Stijn
765844e419
distribution/xfer: un-export NewTransfer()
This is also only used internally, so no need to export.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-02-18 16:58:28 +01:00
Sebastiaan van Stijn
587c474b57
distribution/xfer: use "transferManager" in godoc
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-02-18 16:58:26 +01:00
Sebastiaan van Stijn
874b11495b
distribution/xfer: un-export transferManager.setConcurrency() and .transfer()
They're only used within the package itself, so no need to have them public.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-02-18 16:58:24 +01:00
Sebastiaan van Stijn
d746a836fc
distribution/xfer: remove TransferManager interface, un-export newTransferManager
The `TransferManager` interface only had a single implementation, and neither
`LayerDownloadManager`, nor `LayerUploadManager` currently had an option to
provide a custom implementation, so we can un-export this.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-02-18 16:58:22 +01:00
Sebastiaan van Stijn
cf31aa0fa0
distribution/xfer: fix some nits in tests
- use const instead of a variable
- remove unneeded type-conversion
- use keys in literals

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-02-18 16:58:19 +01:00
Akihiro Suda
54d35c071d
Merge pull request #43130 from thaJeztah/daemon_cache_sysinfo
daemon: load and cache sysInfo on initialization
2022-02-18 13:46:15 +09:00
Sebastiaan van Stijn
32e5fe5099
Merge pull request #43182 from thaJeztah/layer_remove_unused_error
layer: remove unused error return from .Size() and .DiffSize()
2022-02-17 20:51:45 +01:00
Brian Goff
047d58f007
Merge pull request #43187 from thaJeztah/remove_lcow_checks
Remove various leftover LCOW checks
2022-02-17 11:22:19 -08:00
Brian Goff
81db56f0a9
Merge pull request #43253 from thaJeztah/remove_kernel_version_check
daemon: remove kernel version check and DOCKER_NOWARN_KERNEL_VERSION
2022-02-17 11:17:15 -08:00
Tianon Gravi
d666358dcd
Merge pull request #43170 from thaJeztah/api_docs_fixes
docs: update API v1.41 and v1.40 docs with fixes from api/swagger
2022-02-17 11:09:51 -08:00
Sebastiaan van Stijn
dd4cf4b641
daemon: remove some unused stubs on Windows
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-02-17 17:57:51 +01:00
Sebastiaan van Stijn
1240f8b41d
daemon: remove kernel version check and DOCKER_NOWARN_KERNEL_VERSION
All regular, non-EOL Linux distros now come with more recent kernels
out of the box. There may still be users trying to run on kernel 3.10
or older (some embedded systems, e.g.), but those should be a rare
exception, which we don't have to take into account.

This patch removes the kernel version check on Linux, and the corresponding
DOCKER_NOWARN_KERNEL_VERSION environment that was there to skip this
check.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-02-17 17:47:22 +01:00
Tianon Gravi
c94596abc9
Merge pull request #43142 from lebauce/fix-panic-on-empty-quoted-strings
Do not panic on empty quoted string argument
2022-02-16 11:54:34 -08:00
Akihiro Suda
dca8689cd1
Merge pull request #43240 from thaJeztah/remove_more_replaces
vendor.mod: google/go-cmp v0.5.5, golang.org/x/oauth2 v0.0.0-20210514164344-f6687ab2804c, containerd v1.5.9
2022-02-16 13:17:58 +09:00
Akihiro Suda
bdaf94c28a
Merge pull request #43245 from thaJeztah/info_counters_rwmutex
daemon: use RWMutex for stateCounter
2022-02-16 13:17:31 +09:00
Akihiro Suda
12a5f54ad3
Merge pull request #43241 from thaJeztah/update_moby_sys_fsnotify
vendor: fsnotify v1.5.1, moby/sys/mountinfo v0.6.0, moby/sys/signal v0.7.0
2022-02-16 13:17:13 +09:00
Akihiro Suda
c41b752e70
Merge pull request #43242 from thaJeztah/update_go_1.17.7
Update Go to 1.17.7
2022-02-16 12:03:01 +09:00