Commit graph

44933 commits

Author SHA1 Message Date
Sebastiaan van Stijn
7493debe26
pkg/pidfile: implement Read()
This allows consumers to read back the pidFile and to check if the
PID is still running.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-04 01:50:26 +01:00
Sebastiaan van Stijn
7d3e1ad943
pkg/pidfile: Write(): don't automatically create parent directories
While this was convenient for our use, it's somewhat unexpected for a function
that writes a file to also create all parent directories; even more because
this function may be executed as root.

This patch makes the package more "safe" to use as a generic package by removing
this functionality, and leaving it up to the caller to create parent directories,
if needed.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-04 01:50:26 +01:00
Sebastiaan van Stijn
81945da0ac
pkg/pidfile: Write(): take pid as argument
This allows it to be used for processes other than the daemon itself.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-04 01:50:26 +01:00
Sebastiaan van Stijn
735e250326
pkg/process: Alive(): fix PID 0, -1, negative values
unix.Kill() does not produce an error for PID 0, -1. As a result, checking
process.Alive() would return "true" for both 0 and -1 on macOS (and previously
on Linux as well).

Let's shortcut these values to consider them "not alive", to prevent someone
trying to kill them.

A basic test was added to check the behavior.

Given that the intent of these functions is to handle single processes, this patch
also prevents 0 and negative values to be used.

From KILL(2): https://man7.org/linux/man-pages/man2/kill.2.html

    If pid is positive, then signal sig is sent to the process with
    the ID specified by pid.

    If pid equals 0, then sig is sent to every process in the process
    group of the calling process.

    If pid equals -1, then sig is sent to every process for which the
    calling process has permission to send signals, except for
    process 1 (init), but see below.

    If pid is less than -1, then sig is sent to every process in the
    process group whose ID is -pid.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-04 01:50:26 +01:00
Sebastiaan van Stijn
55d15e9d05
pkg/pidfile, pkg/process: use single implementation for process alive
Using the implementation from pkg/pidfile for windows, as that implementation
looks to be handling more cases to check if a process is still alive (or to be
considered alive).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-04 01:50:26 +01:00
Sebastiaan van Stijn
9d5e754caa
move pkg/system: process to a separate package
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-04 01:50:23 +01:00
Sebastiaan van Stijn
0040fb93d6
pkg/system: IsProcessZombie() skip conversion to string, use bytes instead
bytes.SplitN() is more performant, and skips having to do the conversion.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-04 01:49:54 +01:00
Sebastiaan van Stijn
970ad4e3c7
pkg/system: IsProcessZombie() ignore "os.ErrNotExist" errors
If the file doesn't exist, the process isn't running, so we should be able
to ignore that.

Also remove an intermediate variable.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-04 01:49:49 +01:00
Sebastiaan van Stijn
8d6da1e100
pkg/system: IsProcessAlive() remove redundant type-cast
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-04 01:48:54 +01:00
Sebastiaan van Stijn
6c829007cc
Merge pull request #44356 from corhere/libnetwork-namespace-correctness
libnetwork: fix restoring thread network namespaces
2022-11-03 22:33:29 +01:00
Sebastiaan van Stijn
e7f034fe4e
Merge pull request #44365 from vvoland/c8d-contexts
imageservice: Add context to various methods
2022-11-03 19:51:39 +01:00
Sebastiaan van Stijn
6bb909f152
Merge pull request #44367 from vvoland/oci-artifacts-error
distribution: Error when pulling OCI artifacts
2022-11-03 19:39:22 +01:00
Cory Snider
2006d9f7d1
cmd/dockerd: Rewrite shutdownDaemon to use context timeout
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2022-11-03 14:31:03 +01:00
Paweł Gronowski
66a0289081
builder: Don't store context in struct
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2022-11-03 12:22:46 +01:00
Paweł Gronowski
a181a825c8
daemon/start: Revert passing ctx to ctr.Start
This caused integration tests to timeout in the CI

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2022-11-03 12:22:44 +01:00
Nicolas De Loof
def549c8f6
imageservice: Add context to various methods
Co-authored-by: Paweł Gronowski <pawel.gronowski@docker.com>
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2022-11-03 12:22:40 +01:00
Sebastiaan van Stijn
4c07d58592
Merge pull request #44401 from neersighted/swarmkit_revendor
vendor: github.com/moby/swarmkit/v2 v2.0.0-20221102165002-6341884e5fc9
2022-11-03 01:01:25 +01:00
Sebastiaan van Stijn
98f36bba9a
Merge pull request #44398 from thaJeztah/daemon_logger_godoc
daemon/logger: fix godoc
2022-11-02 23:50:29 +01:00
Sebastiaan van Stijn
781fd745ed
Merge pull request #44393 from thaJeztah/pkg_kernel_duplicate_docs
pkg/parsers/kernel: remove duplicate GoDoc and un-export Utsname
2022-11-02 23:49:48 +01:00
Bjorn Neergaard
57c2545cd5
vendor: github.com/moby/swarmkit/v2 v2.0.0-20221102165002-6341884e5fc9
full diff: 48dd89375d...6341884e5f

Pulls in a set of fixes to SwarmKit's nascent Cluster Volumes support
discovered during subsequent development and testing.

Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
2022-11-02 15:10:08 -06:00
Sebastiaan van Stijn
2a478f9215
Merge pull request #44399 from corhere/fix-task-delete-on-failed-start
Fix containerd task deletion after failed start
2022-11-02 21:48:56 +01:00
Cory Snider
1bef9e3fbf Fix containerd task deletion after failed start
Deleting a containerd task whose status is Created fails with a
"precondition failed" error. This is because (aside from Windows)
a process is spawned when the task is created, and deleting the task
while the process is running would leak the process if it was allowed.
libcontainerd and the containerd plugin executor mistakenly try to clean
up from a failed start by deleting the created task, which will always
fail with the aforementined error. Change them to pass the
`WithProcessKill` delete option so the cleanup has a chance to succeed.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-11-02 13:48:13 -04:00
Sebastiaan van Stijn
be1829f63d
daemon/logger: fix godoc
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-02 18:47:50 +01:00
Sebastiaan van Stijn
e1743888ed
Merge pull request #44384 from thaJeztah/idtools_cleanup4
pkg/idtools: remove execCmd() utility
2022-11-02 18:46:56 +01:00
Tianon Gravi
b76ccfef5f
Merge pull request #44289 from thaJeztah/windows_refactor_etwlogs
daemon/logger/etwlogs: rewrite to use go-winio/pkg/etw
2022-11-02 10:43:55 -07:00
Sebastiaan van Stijn
9899820a17
Merge pull request #44389 from tianon/debug-typo
Remove errant "runtime.GOARCH" from debug message
2022-11-02 18:23:11 +01:00
Sebastiaan van Stijn
4112481170
Merge pull request #44390 from tianon/rm-deprecated-arm-fallback
Remove long-deprecated "arm" fallback
2022-11-02 18:20:43 +01:00
Sebastiaan van Stijn
5a01c1dad1
Merge pull request #44383 from thaJeztah/improve_example
api/types/filters: add output to example
2022-11-02 17:29:37 +01:00
Brian Goff
8d15e7d742
Merge pull request #44073 from ndeloof/upstream_29
implement docker system df
2022-11-02 08:05:17 -07:00
Paweł Gronowski
407e3a4552
distribution: Error when pulling OCI artifacts
Currently an attempt to pull a reference which resolves to an OCI
artifact (Helm chart for example), results in a bit unrelated error
message `invalid rootfs in image configuration`.

This provides a more meaningful error in case a user attempts to
download a media type which isn't image related.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2022-11-02 12:51:03 +01:00
Sebastiaan van Stijn
4386e3f7c0
pkg/parsers/kernel: un-export Utsname
It's only used internally to allow the "unsupported" stub.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-02 10:55:54 +01:00
Sebastiaan van Stijn
775dcab7a2
pkg/parsers/kernel: remove duplicate Package godoc
It was present both in kernel.go and kernel_unix.go.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-02 10:44:50 +01:00
Nicolas De Loof
8d0dc69027
implement docker system df
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2022-11-02 09:35:18 +01:00
Samuel Karp
5dd50a381a
Merge pull request #44391 from tlgs/sort-names 2022-11-01 21:37:37 -07:00
Brian Goff
9fa38435aa
Merge pull request #44316 from thaJeztah/plugin_improve_test_errors
integration-cli: DockerPluginSuite: use gotest.tools compare utilities
2022-11-01 17:07:22 -07:00
Tiago Seabra
952e1e62c5 Sort entries in pkg/namesgenerator
Signed-off-by: Tiago Seabra <tlgs@users.noreply.github.com>
2022-11-01 23:13:34 +00:00
Tianon Gravi
5bc17c3e54 Remove long-deprecated "arm" fallback
This fallback is used when we filter the manifest list by the user-provided platform and find no matches such that we match the previous Docker behavior (before it supported variant matching).  This has been deprecated long enough that I think it's time we finally stop supporting this weird fallback, especially since it makes for buggy behavior like `docker pull --platform linux/arm/v5 alpine:3.16` leading to a `linux/arm/v6` image being pulled (I specified a variant, every manifest list entry specifies a variant, so clearly the only behavior I as a user could reasonably expect is an error that `linux/arm/v5` is not supported, but instead I get an explicitly incompatible image despite doing everything I as a user can to prevent that situation).

Signed-off-by: Tianon Gravi <admwiggin@gmail.com>
2022-11-01 15:31:07 -07:00
Tianon Gravi
3d71555a47 Remove errant "runtime.GOARCH" from debug message
This debug message already includes a full platform string, so this ends up being something like `linux/arm/v7/amd64` in the end result.

Signed-off-by: Tianon Gravi <admwiggin@gmail.com>
2022-11-01 15:28:29 -07:00
Sebastiaan van Stijn
6509ebc594
Merge pull request #44388 from corhere/bump-go1.19.3
Update to Go 1.19.3 to address CVE-2022-41716
2022-11-01 23:19:02 +01:00
Cory Snider
f9d4589976 Update to Go 1.19.3 to address CVE-2022-41716
On Windows, syscall.StartProcess and os/exec.Cmd did not properly
    check for invalid environment variable values. A malicious
    environment variable value could exploit this behavior to set a
    value for a different environment variable. For example, the
    environment variable string "A=B\x00C=D" set the variables "A=B" and
    "C=D".

    Thanks to RyotaK (https://twitter.com/ryotkak) for reporting this
    issue.

    This is CVE-2022-41716 and Go issue https://go.dev/issue/56284.

This Go release also fixes https://github.com/golang/go/issues/56309, a
runtime bug which can cause random memory corruption when a goroutine
exits with runtime.LockOSThread() set. This fix is necessary to unblock
work to replace certain uses of pkg/reexec with unshared OS threads.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-11-01 16:54:48 -04:00
Sebastiaan van Stijn
1e88fe578e
pkg/idtools: remove execCmd() utility
The `execCmd()` utility was a basic wrapper around `exec.Command()`. Inlining it
makes the code more transparent.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-01 21:13:10 +01:00
Sebastiaan van Stijn
3d73d32499
api/types/filters: add output to example
Make the example actually do something, and include the output, so that it
shows up in the documentation.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-01 18:16:22 +01:00
Sebastiaan van Stijn
c207947508
integration-cli: DockerPluginSuite: use gotest.tools compare utilities
Some of these tests are failing (but not enabled in CI), but the current output
doesn't provide any details on the failure, so this patch is just to improve the
test output to allow debugging the actual failure.

Before this, tests would fail like:

    make BIND_DIR=. TEST_FILTER=TestPluginInstallImage test-integration
    ...
    === FAIL: amd64.integration-cli TestDockerPluginSuite/TestPluginInstallImage (15.22s)
        docker_cli_plugins_test.go:220: assertion failed: expression is false: strings.Contains(out, `Encountered remote "application/vnd.docker.container.image.v1+json"(image) when fetching`)
        --- FAIL: TestDockerPluginSuite/TestPluginInstallImage (15.22s)

With this patch, tests provide more useful output:

    make BIND_DIR=. TEST_FILTER=TestPluginInstallImage test-integration
    ...
    === FAIL: amd64.integration-cli TestDockerPluginSuite/TestPluginInstallImage (1.15s)
    time="2022-10-18T10:21:22Z" level=warning msg="reference for unknown type: application/vnd.docker.plugin.v1+json"
    time="2022-10-18T10:21:22Z" level=warning msg="reference for unknown type: application/vnd.docker.plugin.v1+json" digest="sha256:bee151d3fef5c1f787e7846efe4fa42b25a02db4e7543e54e8c679cf19d78598"
mediatype=application/vnd.docker.plugin.v1+json size=522
    time="2022-10-18T10:21:22Z" level=warning msg="reference for unknown type: application/vnd.docker.plugin.v1+json"
    time="2022-10-18T10:21:22Z" level=warning msg="reference for unknown type: application/vnd.docker.plugin.v1+json" digest="sha256:bee151d3fef5c1f787e7846efe4fa42b25a02db4e7543e54e8c679cf19d78598"
mediatype=application/vnd.docker.plugin.v1+json size=522
        docker_cli_plugins_test.go:221: assertion failed: string "Error response from daemon: application/vnd.docker.distribution.manifest.v1+prettyjws not supported\n" does not contain "Encountered remote
\"application/vnd.docker.container.image.v1+json\"(image) when fetching"
        --- FAIL: TestDockerPluginSuite/TestPluginInstallImage (1.15s)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-01 17:27:18 +01:00
Brian Goff
6743bf3173
Merge pull request #44375 from thaJeztah/improve_TestParseLink
opts: TestParseLink(): add test-case for legacy format
2022-11-01 09:16:26 -07:00
Sebastiaan van Stijn
e7904c5faa
Merge pull request #44309 from thaJeztah/daemon_check_requirements
daemon: NewDaemon(): check system requirements early
2022-11-01 13:42:44 +01:00
Sebastiaan van Stijn
ef7e4ec3c6
Merge pull request #44317 from thaJeztah/daemon_mkdir
daemon: NewDaemon(): replace system.MkdirAll for os.Mkdir where possible
2022-11-01 13:41:16 +01:00
Sebastiaan van Stijn
346a438da8
opts: TestParseLink(): add test-case for legacy format
The ParseLink() function has special handling for legacy formats;

> This is kept because we can actually get a HostConfig with links
> from an already created container and the format is not `foo:bar`
> but `/foo:/c1/bar`

This patch adds a test-case for this format. While updating, also switching
to use gotest.tools assertions.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-01 11:05:19 +01:00
Sebastiaan van Stijn
aeafa2a28f
Merge pull request #44363 from luismulinari/fix_max_concurrent_downloads_uploads_docs
Fix the max-concurrent-downloads and max-concurrent-uploads configs documentation
2022-10-28 21:17:24 -04:00
Cory Snider
ad4073edc1 daemon: fix docs for config-default constants
Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-10-28 15:52:57 -04:00
Luis Henrique Mulinari
6c0aa5b00a
Fix the max-concurrent-downloads and max-concurrent-uploads configs documentation
This fix tries to address issues raised in #44346.
The max-concurrent-downloads and max-concurrent-uploads limits are applied for the whole engine and not for each pull/push command.

Signed-off-by: Luis Henrique Mulinari <luis.mulinari@gmail.com>
2022-10-26 11:10:00 +01:00