Commit graph

43796 commits

Author SHA1 Message Date
Cory Snider
90c54320c8 daemon/logger: fix data race in LogFile
The log message's timestamp was being read after it was returned to the
pool. By coincidence the timestamp field happened to not be zeroed on
reset so much of the time things would work as expected. But if the
message value was to be taken back out of the pool before WriteLogEntry
returned, the timestamp recorded in the gzip header of compressed
rotated log files would be incorrect.

Make future use-after-put bugs fail fast by zeroing all fields of the
Message value, including the timestamp, when it is put into the pool.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-03-03 14:56:25 -05:00
Cory Snider
9080e5a1f7 daemon/logger: add test to detect data races
Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-03-03 14:56:25 -05:00
Sebastiaan van Stijn
c8cf4517fc
Merge pull request #43309 from thaJeztah/daemon_refactor_statecounter
daemon: SystemInfo() extract collecting data to more helper functions
2022-03-03 20:49:27 +01:00
Sebastiaan van Stijn
61c6a4792a
Merge pull request #43318 from thaJeztah/layer_remove_unused_error
layer: remove unused ErrActiveMount, ErrNotMounted, ErrNotSupported
2022-03-03 20:45:49 +01:00
Brian Goff
c8eff73fff
Merge pull request #43297 from thaJeztah/registry_add_DefaultRegistryHost_const
registry: add DefaultRegistryHost const, and improve documentation
2022-03-03 11:19:29 -08:00
Sebastiaan van Stijn
5263bea70f
daemon: move check for CPU-realtime daemon options
Perform the validation when the daemon starts instead of performing these
validations for each individual container, so that we can fail early.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-03 19:50:27 +01:00
Sebastiaan van Stijn
36ec581e5c
Merge pull request #43277 from thaJeztah/fix_kernelmem_docs_and_error
api: add missing docs for KernelMemoryTCP, and fix error message
2022-03-03 19:33:49 +01:00
Sebastiaan van Stijn
4b9931f58f
Merge pull request #43320 from thaJeztah/move_EnsureRemoveAll
pkg/system: move EnsureRemoveAll() to pkg/containerfs
2022-03-03 07:39:42 +01:00
Sebastiaan van Stijn
dee9f422c8
pkg/system: remove github.com/docker/go-units dependency
This is not "very" important, but this dependency was only used
for a single const, which could be satisfied with a comment.

Not very urgent, as github.com/docker/go-units is likely imported
through other ways already (but it's nice to have the package be
more isolated).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-03 00:22:32 +01:00
Sebastiaan van Stijn
25ee00c494
pkg/system: move EnsureRemoveAll() to pkg/containerfs
pkg/system historically has been a bit of a kitchen-sink of things that were
somewhat "system" related, but didn't have a good place for. EnsureRemoveAll()
is one of those utilities. EnsureRemoveAll() is used to both unmount and remove
a path, for which it depends on both github.com/moby/sys/mount, which in turn
depends on github.com/moby/sys/mountinfo.

pkg/system is imported in the CLI, but neither EnsureRemoveAll(), nor any of its
moby/sys dependencies are used on the client side, so let's move this function
somewhere else, to remove those dependencies from the CLI.

I looked for plausible locations that were related; it's used in:

- daemon
- daemon/graphdriver/XXX/
- plugin

I considered moving it into a (e.g.) "utils" package within graphdriver (but not
a huge fan of "utils" packages), and given that it felt (mostly) related to
cleaning up container filesystems, I decided to move it there.

Some things to follow-up on after this:

- Verify if this function is still needed (it feels a bit like a big hammer in
  a "YOLO, let's try some things just in case it fails")
- Perhaps it should be integrated in `containerfs.Remove()` (so that it's used
  automatically)
- Look if there's other implementations (and if they should be consolidated),
  although (e.g.) the one in containerd is a copy of ours:
  https://github.com/containerd/containerd/blob/v1.5.9/pkg/cri/server/helpers_linux.go#L200

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-03 00:22:26 +01:00
Sebastiaan van Stijn
d492101172
daemon: SystemInfo() extract collecting debugging information to a helper
This makes it more inline with other data we collect, and can be used to make
some info optional at some point.

fillDebugInfo sets the current debugging state of the daemon, and additional
debugging information, such as the number of Go-routines, and file descriptors.

Note that this currently always collects the information, but the CLI only
prints it if the daemon has debug enabled. We should consider to either make
this information optional (cli to request "with debugging information"), or
only collect it if the daemon has debug enabled. For the CLI code, see
https://github.com/docker/cli/blob/v20.10.12/cli/command/system/info.go#L239-L244

Additional note: the CLI considers info.SystemTime debugging information. This
felt a bit "odd" (daemon time could be useful for standard use), so I left this
out of this function.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-02 22:19:24 +01:00
Sebastiaan van Stijn
ac2cd5a8f2
daemon: unexport Daemon.ID and Daemon.RegistryService
These are used internally only, and set by daemon.NewDaemon(). If they're
used externally, we should add an accessor added (which may be something
we want to do for daemon.registryService (which should be its own backend)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-02 22:19:22 +01:00
Sebastiaan van Stijn
a27f8aecad
daemon: SystemInfo() extract container counts to a helper function
This makes it more inline with other data we collect, and can be used to
make some info optional at some point.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-02 22:19:20 +01:00
Sebastiaan van Stijn
82f96da04d
layer: remove unused ErrActiveMount, ErrNotMounted, ErrNotSupported
These errors were added in 500e77bad0, but were
never used.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-02 21:30:41 +01:00
Sebastiaan van Stijn
e8a0a545e7
Merge pull request #43308 from thaJeztah/bump_more_buildkit_deps
vendor: update more dependencies in preparation of buildkit and containerd update
2022-03-02 20:08:33 +01:00
Sebastiaan van Stijn
de0eabbd66
vendor: github.com/vishvananda/netns v0.0.0-20210104183010-2eb08e3e575f
full diff: db3c7e526a...2eb08e3e57

- Add support for detecting netns for all possible QoS in Kubernetes
- Add go1.10 build constraint

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-01 22:58:47 +01:00
Sebastiaan van Stijn
d8e1746466
vendor: github.com/tonistiigi/fsutil v0.0.0-20220115021204-b19f7f9cb274
full diff: d72af97c0e...b19f7f9cb2

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-01 22:41:22 +01:00
Sebastiaan van Stijn
931b455f27
vendor: github.com/hashicorp/errwrap v1.1.0
deprecates `errwrap.Wrapf()`

That function appears to be still used by `go-multierror.Prefix()`);
https://github.com/hashicorp/go-multierror/blob/v1.1.1/prefix.go#L30-L35
which itself is only used in a single place in `containerd/pkg/process`:
https://github.com/containerd/containerd/blob/v1.5.9/pkg/process/io.go#L438

full diff: https://github.com/hashicorp/errwrap/compare/v1.0.0...v1.1.0
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-01 18:06:12 +01:00
Sebastiaan van Stijn
4d1c323796
vendor: golang.org/x/text v0.3.7
full diff: https://github.com/golang/text/compare/v0.3.6...v0.3.7

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-01 18:04:19 +01:00
Sebastiaan van Stijn
7f9c77b2fe
vendor: golang.org/x/oauth2 v0.0.0-20210819190943-2bc19b11175f
full diff: 6f1e639406...2bc19b1117

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-01 18:03:34 +01:00
Sebastiaan van Stijn
a69cda092b
vendor: golang.org/x/lint v0.0.0-20210508222113-6edffad5e616
no changes in vendored code

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-01 18:02:03 +01:00
Sebastiaan van Stijn
c03ae0b726
vendor: github.com/cespare/xxhash/v2 v2.1.2
full diff: https://github.com/cespare/xxhash/compare/v2.1.1...v2.1.2

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-01 18:00:17 +01:00
Sebastiaan van Stijn
2634edec6e
vendor: github.com/klauspost/compress v1.14.3
full diff: https://github.com/klauspost/compress/compare/v1.14.2...v1.14.3

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-01 17:57:38 +01:00
Sebastiaan van Stijn
8bf694b427
vendor: github.com/google/go-cmp v0.5.7
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-01 17:04:32 +01:00
Sebastiaan van Stijn
89d39e5e77
vendor: gotest.tools/v3 v3.1.0
full diff: https://github.com/gotestyourself/gotest.tools/compare/v3.0.3...v3.1.0

noteworthy changes:

- ci: add go1.16
- ci: add go1.17, remove go1.13
- golden: only create dir if update flag is set
- icmd: replace all usages of os/exec with golang.org/x/sys/execabs
- assert: ErrorIs
- fs: add DirFromPath
- Stop creating directory outside of testdata
- fs: Fix comparing symlink permissions

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-01 17:02:53 +01:00
Sebastiaan van Stijn
c35143f92e
vendor: github.com/moby/sys/mount v0.3.1
full diff: https://github.com/moby/sys/compare/mount/v0.3.0...mount/v0.3.1

- mount: fix unused/deadcode warnings on Mac
- mount: bump mountinfo to v0.6.0
- Makefile: rm .SHELLFLAGS, add set -e

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-01 17:02:07 +01:00
Sebastiaan van Stijn
b7310e6d13
Merge pull request #43303 from cuishuang/master
all: fix typos
2022-03-01 13:39:48 +01:00
cuishuang
9efa8b3500 all: fix typos
Signed-off-by: cuishuang <imcusg@gmail.com>
2022-03-01 14:26:35 +08:00
Sebastiaan van Stijn
203fcd6997
layers: remove layerStore.getWithoutLock()
This function was abstracting things a bit too much; the layerStore had a
exported `.Get()` which called `.getWithoutLock()`, but also a non-exported
`.get()`, which also called `.getWithoutLock()`.

While it's common to have a non-exported variant (without locking), the naming
of `.get()` could easily be confused for that variant (which it wasn't).

All locations where `.get()` was called were already handling locks for
`releaseLayer()`, so moving the actual locking inline for `.get()` makes it
more visible where locking happens.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-02-28 19:57:37 +01:00
Sebastiaan van Stijn
7d70d95d8e
registry: add DefaultRegistryHost const, and improve documentation
This is more in line with other consts that are used for defaults, and makes it
slightly easier to consume than DefaultV2Registry, e.g. see:
https://github.com/oras-project/oras-go/blob/v1.1.0/pkg/auth/docker/resolver.go#L81-L84

Note that both the "index.docker.io" and "registry-1.docker.io" domains
are here for historic reasons and backward-compatibility. These domains
are still supported by Docker Hub (and will continue to be supported), but
there are new domains already in use, and plans to consolidate all legacy
domains to new "canonical" domains. Once those domains are decided on, we
should update these consts (but making sure to preserve compatibility with
existing installs, clients, and user configuration).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-02-27 13:48:27 +01:00
Akihiro Suda
d809ad98e3
Merge pull request #43183 from thaJeztah/cleanup_distribution
distribution/xfer: refactor to reduce public api/interface
2022-02-26 23:50:03 +09:00
Tianon Gravi
c549116f32
Merge pull request #43280 from thaJeztah/remove_deprecated_dualstack
distribution: remove use of deprecated dial.DualStack
2022-02-25 16:22:49 -08:00
Tianon Gravi
c78f6963a1
Merge pull request #43289 from thaJeztah/bump_hcsshim
vendor: github.com/Microsoft/hcsshim v0.9.2
2022-02-24 14:24:38 -08:00
Sebastiaan van Stijn
9d6382f2b3
vendor: github.com/Microsoft/hcsshim v0.9.2
full diff: https://github.com/Microsoft/hcsshim/compare/v0.8.23...v0.9.2

diff is hard to compare on github, because Microsoft/opengcs was merged into
hcsshim; https://github.com/microsoft/hcsshim/pull/973

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-02-24 21:28:18 +01:00
Brian Goff
5c36bb7206
Merge pull request #43287 from thaJeztah/bump_dependencies_for_buildkit
vendor: update various dependencies in preparation of BuildKit update
2022-02-24 11:05:48 -08:00
Sebastiaan van Stijn
c72c1ca62c
Merge pull request #43185 from corhere/42402-safer-fileinfo
Remove local fork of archive/tar package
2022-02-24 10:36:14 +01:00
Akihiro Suda
48d08a7b77
Merge pull request #43288 from thaJeztah/bump_aws_sdk
vendor: github.com/aws/aws-sdk-go v1.31.6
2022-02-24 11:56:59 +09:00
Brian Goff
7c764a29b0
Merge pull request #43235 from thaJeztah/api_wait_fixes
Assorted (documentation) fixes for `POST /containers/{id}/wait`
2022-02-23 15:13:18 -08:00
Sebastiaan van Stijn
ace606fd75
vendor: github.com/aws/aws-sdk-go v1.31.6
full diff: https://github.com/aws/aws-sdk-go/compare/v1.28.11...v1.31.6

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-02-23 19:49:51 +01:00
Sebastiaan van Stijn
0809bd6859
vendor: github.com/klauspost/compress v1.14.2
full diff: https://github.com/klauspost/compress/compare/v1.12.3...v1.14.2

Relevant changes affecting vendor:

- docs: Add combined LICENSE file
- Add snappy replacement package
- tests: Remove snappy dependency for tests
- huff0: Add size estimation function
- huff0: Improve 4X decompression speed
- huff0: Improve 4X decompression speed 5-10%
- huff0: Faster 1X Decompression
- zstd: Spawn decoder goroutine only if needed
- zstd: Detect short invalid signatures
- zstd: Add configurable Decoder window size
- zstd: Add stream content size
- zstd: Simplify hashing functions
- zstd: use SpeedBestCompression for level >= 10
- zstd: Fix WriteTo error forwarding
- zstd: Improve Best compression
- zstd: Fix incorrect encoding in best mode
- zstd: pooledZipWriter should return Writers to the same pool
- zstd: Upgrade xxhash
- zstd: Improve block encoding speed
- zstd: add arm64 xxhash assembly
- zstd: Minor decoder improvements
- zstd: Minor performance tweaks
- zstd: Add bigger default blocks
- zstd: Remove unused decompression buffer
- zstd: fix logically dead code
- zstd: Add noasm tag for xxhash
- zstd: improve header decoder

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-02-23 19:41:14 +01:00
Sebastiaan van Stijn
20e5d6b3e3
vendor: github.com/pelletier/go-toml v1.9.4
full diff: https://github.com/pelletier/go-toml/compare/v1.9.1...v1.9.4

- Add Encoder.CompactComments to omit extra new line
- Clarify license and comply with Apache 2.0
- feat(tomll): add multiLineArray flag to linter

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-02-23 19:25:30 +01:00
Sebastiaan van Stijn
812dacb676
vendor: github.com/containerd/continuity v0.2.2
full diff: https://github.com/containerd/continuity/compare/v0.1.0...v0.2.2

- fs/stat: add FreeBSD, and cleanup some nolint-comments
- go.mod: bazil.org/fuse v0.0.0-20200407214033-5883e5a4b5125
- Fix darwin issues
- Remove direct dependency on github.com/pkg/errors
- Do not log errors before returning them
- Build containerd/continuity on multiple Unix OSes
- Update CI Go version to 1.17
- fs: use syscall.Timespec.Unix

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-02-23 19:22:36 +01:00
Sebastiaan van Stijn
461845bfbc
vendor: github.com/containerd/cgroups v1.0.3
full diff: https://github.com/containerd/cgroups/compare/v1.0.1...v1.0.3

- cgroup v1: implement AddProc()
- cgroup v1: reduce duplicated code
- cgroup v2: Fix potential dirfd leak
- cgroup v2: remove unimplemented errors and ErrorHandler, IgnoreNotExist
- cgroup v2: v2: Fix inotify fd leak when cgroup is deleted
- cgroup.go: avoid panic on nil interface
- cgroup: Optionally add process and task to a subsystems subset
- fix Implicit memory aliasing in for loop
- go.mod: coreos/go-systemd/v22 v22.3.2 to prepare for deprecations
- Improvements on cgroup v2 support
- replace pkg/errors from vendor
- Use /proc/partitions to get device names
- utils: export ParseCgroupFile()

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-02-23 19:04:43 +01:00
Sebastiaan van Stijn
7876c53424
vendor: golang.org/x/tools v0.1.5
full diff: https://github.com/golang/tools/compare/v0.1.0...v0.1.5

It's not used, but one of our dependencies has a `tools.go` file that forces
it to be vendored; vendor/cloud.google.com/go/tools.go

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-02-23 18:42:17 +01:00
Sebastiaan van Stijn
1b829c2a6a
vendor: golang.org/x/mod v0.4.2
full diff: https://github.com/golang/mod/compare/v0.4.1...v0.4.2

It's not used, but one of our dependencies has a `tools.go` file that forces
it to be vendored; vendor/cloud.google.com/go/tools.go

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-02-23 18:37:58 +01:00
Sebastiaan van Stijn
6be521ccb8
vendor: golang.org/x/oauth2 v0.0.0-20210805134026-6f1e6394065a
full diff: f6687ab280...6f1e639406

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-02-23 18:30:08 +01:00
Sebastiaan van Stijn
8f106d4576
vendor: github.com/google/go-cmp v0.5.6
full diff: http://github.com/google/go-cmp/compare/v0.5.5...v0.5.6

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-02-23 18:29:02 +01:00
Sebastiaan van Stijn
9144d1705c
Merge pull request #43282 from Cornul11/43265-fix-docker-ce-mentions
Removed mentions of the deprecated Docker CE repo
2022-02-23 18:25:28 +01:00
Dan Plamadeala
9393cf73f6 removed mentions of the archived Docker CE repo
Signed-off-by: Dan Plamadeala <cornul11@gmail.com>
2022-02-23 12:13:12 +01:00
Sebastiaan van Stijn
ff408210da
distribution: remove use of deprecated dial.DualStack
From the field's description [1]:

    DualStack previously enabled RFC 6555 Fast Fallback
    support, also known as "Happy Eyeballs", in which IPv4 is
    tried soon if IPv6 appears to be misconfigured and
    hanging.

    Deprecated: Fast Fallback is enabled by default. To
    disable, set FallbackDelay to a negative value.

This field was deprecated in efc185029b,
which is included in Go 1.12beta1 and up.

[1]: 2ebe77a2fd/src/net/dial.go (L54-L61)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-02-22 16:34:12 +01:00