Commit graph

7031 commits

Author SHA1 Message Date
Brian Goff
4e025b54d5 Remove mount spec backport
This was added in 1.13 to "upgrade" old mount specs to the new format.
This is no longer needed.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2022-05-13 23:14:43 +00:00
Samuel Karp
a75620086f
Merge pull request #43580 from thaJeztah/remove_initlayer_stub 2022-05-13 01:09:01 -07:00
Brian Goff
f32b304a8f
Merge pull request #42501 from tianon/always-seccomp
Remove "seccomp" build tag
2022-05-12 19:12:15 -07:00
Sebastiaan van Stijn
34e02d9b04
Merge pull request #43524 from thaJeztah/daemon_fix_hosts_validation_step2
opts: ParseTCPAddr(): extract parsing logic, consistent errors
2022-05-13 02:42:40 +02:00
Drew Erny
240a9fcb83
Add Swarm cluster volume supports
Adds code to support Cluster Volumes in Swarm using CSI drivers.

Signed-off-by: Drew Erny <derny@mirantis.com>
2022-05-13 00:55:44 +02:00
Tianon Gravi
c9e19a2aa1 Remove "seccomp" build tag
Similar to the (now removed) `apparmor` build tag, this build-time toggle existed for users who needed to build without the `libseccomp` library.  That's no longer necessary, and given the importance of seccomp to the overall default security profile of Docker containers, it makes sense that any binary built for Linux should support (and use by default) seccomp if the underlying host does.

Signed-off-by: Tianon Gravi <admwiggin@gmail.com>
2022-05-12 14:48:35 -07:00
Nicolas De Loof
af5d83a641
Make it explicit raw|multiplexed stream implementation being used
fix #35761

Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2022-05-12 11:36:31 +02:00
Sebastiaan van Stijn
61fec7b36e
daemon/initlayer: Init(): remove unused stub for Windows
This package is only used in unix/linux files, so we don't
need a stub for Windows.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-05-11 01:27:47 +02:00
Sebastiaan van Stijn
3228dbaaa9
Merge pull request #43555 from thaJeztah/separate_engine_id
daemon: separate daemon ID from trust-key, and disable generating
2022-05-10 14:27:42 +02:00
Eng Zer Jun
7873c27cfb
all: replace strings.Replace with strings.ReplaceAll
strings.ReplaceAll(s, old, new) is a wrapper function for
strings.Replace(s, old, new, -1). But strings.ReplaceAll is more
readable and removes the hardcoded -1.

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2022-05-09 19:45:40 +08:00
Sebastiaan van Stijn
6b4696e18d
Merge pull request #43544 from thaJeztah/daemon_fix_hosts_validation_step1h
daemon/config: remove uses of pointers for ints
2022-05-06 17:52:52 +02:00
Sebastiaan van Stijn
d6115b8f40
daemon: fix some minor nits
- remove isErrNoSuchProcess() in favor of a plain errors.As()
- errNoSuchProcess.Error(): remove punctuation

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-05-05 11:27:59 +02:00
Sebastiaan van Stijn
d733481399
daemon: daemon.ContainerKill() accept stop-signal as string
This allows the postContainersKill() handler to pass values as-is. As part of
the rewrite, I also moved the daemon.GetContainer(name) call later in the
function, so that we can fail early if an invalid signal is passed, before
doing the (heavier) fetching of the container.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-05-05 11:27:47 +02:00
Sebastiaan van Stijn
21df9a04e0
container: StopSignal(): return syscall.Signal
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-05-05 00:53:53 +02:00
Sebastiaan van Stijn
ea1eb449b7
daemon: killWithSignal, killPossiblyDeadProcess: accept syscall.Signal
This helps reducing some type-juggling / conversions further up
the stack.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-05-05 00:53:52 +02:00
Sebastiaan van Stijn
2ec2b65e45
libcontainerd: SignalProcess(): accept syscall.Signal
This helps reducing some type-juggling / conversions further up
the stack.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-05-05 00:53:49 +02:00
Sebastiaan van Stijn
070da63310
daemon: only create trust-key if DOCKER_ALLOW_SCHEMA1_PUSH_DONOTUSE is set
The libtrust trust-key is only used for pushing legacy image manifests;
pushing these images has been deprecated, and we only need to be able
to push them in our CI.

This patch disables generating the trust-key (and related paths) unless
the DOCKER_ALLOW_SCHEMA1_PUSH_DONOTUSE env-var is set (which we do in
our CI).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-05-04 20:18:08 +02:00
Sebastiaan van Stijn
bb1208639b
daemon: separate daemon ID from trust-key
This change is in preparation of deprecating support for old manifests.
Currently the daemon's ID is based on the trust-key ID, which will be
removed once we fully deprecate support for old manifests (the trust
key is currently only used in tests).

This patch:

- looks if a trust-key is present; if so, it migrates the trust-key
  ID to the new "engine-id" file within the daemon's root.
- if no trust-key is present (so in case it's a "fresh" install), we
  generate a UUID instead and use that as ID.

The migration is to prevent engines from getting a new ID on upgrades;
while we don't provide any guarantees on the engine's ID, users may
expect the ID to be "stable" (not change) between upgrades.

A test has been added, which can be ran with;

    make DOCKER_GRAPHDRIVER=vfs TEST_FILTER='TestConfigDaemonID' test-integration

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-05-04 20:17:18 +02:00
Sebastiaan van Stijn
a3ae9a5956
opts: ParseTCPAddr(): extract parsing logic, consistent errors
Make sure we validate the default address given before using it, and
combine the parsing/validation logic so that it can be reused.

This patch also makes the errors more consistent, and uses pkg/errors
for generating them.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-05-01 19:53:40 +02:00
Sebastiaan van Stijn
545cf195e2
Merge pull request #43480 from corhere/mitigate-slow-health-check-start
Mitigate the impact of slow exec starts on health checks
2022-04-29 15:07:31 +02:00
Sebastiaan van Stijn
5486146943
Merge pull request #43525 from thaJeztah/daemon_fix_hosts_validation_step1e
daemon: daemon.initNetworkController(): dont return the controller
2022-04-29 14:12:56 +02:00
Sebastiaan van Stijn
bf04690bbc
Merge pull request #43530 from thaJeztah/api_cleanup_definitions
api/types: cleanup to use more idiomatic names
2022-04-29 11:35:43 +02:00
Sebastiaan van Stijn
e62382d014
daemon/config: remove uses of pointers for ints
Use the default (0) value to indicate "not set", which simplifies
working with these configuration options, preventing the need to
use intermediate variables etc.

While changing this code, also making some small cleanups, such
as replacing "fmt.Sprintf()" for "strconv" variants.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-29 09:39:34 +02:00
Sebastiaan van Stijn
4d22584432
Merge pull request #43536 from thaJeztah/daemon_fix_hosts_validation_step1g
daemon: improvements to config (re)loading
2022-04-29 09:39:11 +02:00
Sebastiaan van Stijn
dbd575ef91
daemon: daemon.initNetworkController(): dont return the controller
This method returned the network controller, only to set it on the daemon.

While making this change, also;

- update some error messages to be in the correct format
- use errors.Wrap() where possible
- extract configuring networks into a separate function to make the flow
  slightly easier to follow.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-29 09:08:49 +02:00
Cory Snider
bdc6473d2d health: Start probe timeout after exec starts
Starting an exec can take a significant amount of time while under heavy
container operation load. In extreme cases the time to start the process
can take upwards of a second, which is a significant fraction of the
default health probe timeout (30s). With a shorter timeout, the exec
start delay could make the difference between a successful probe and a
probe timeout! Mitigate the impact of excessive exec start latencies by
only starting the probe timeout timer after the exec'ed process has
started.

Add a metric to sample the latency of starting health-check exec probes.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-04-28 17:21:03 -04:00
Sebastiaan van Stijn
41b96bff55
update uses of container.ContainerCreateCreatedBody to CreateResponse
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-28 22:39:20 +02:00
Sebastiaan van Stijn
64e96932bd
api: rename volume.VolumeCreateBody to volume.CreateOptions
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-28 22:39:14 +02:00
Sebastiaan van Stijn
3cae9fef16
imports: remove "volumetypes" aliases for api/types/volume
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-28 22:39:04 +02:00
Brian Goff
b3332b851a
Merge pull request #43517 from Juneezee/test/t.Setenv
test: use `T.Setenv` to set env vars in tests
2022-04-28 12:02:01 -07:00
Sebastiaan van Stijn
647aede6ad
Merge pull request #43515 from corhere/swarmkit-v2
Bump swarmkit to v2
2022-04-28 20:08:42 +02:00
Sebastiaan van Stijn
4cf904494e
daemon: reloadMaxDownloadAttempts() remove validation
reloadMaxDownloadAttempts() is used to reload the configuration,
but validation happened before merging the config with the defaults.

This removes the validation from this function, instead centralizing
validation in config.Validate().

NOTE:
Currently this validation is "ok", as it checks for "nil" values;
I am working on changes to reduce the use of pointers in the config,
and instead provide a mechanism to fill in defaults. This change is
in preparation of that.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-28 09:30:06 +02:00
Sebastiaan van Stijn
c46e2e85ee
daemon/config: Reload(): add TODO for config reload logic
The Reload logic is problematic and needs a rewrite.

Currently, config.Reload() is validating newConfig before the reload callback
is executed. At that point, newConfig may be a partial configuration, yet to be
merged with the existing configuration (in the "reload()" callback). Validating
this config before it's merged can result in incorrect validation errors.

However, the current "reload()" callback we use is DaemonCli.reloadConfig(),
which includes a call to Daemon.Reload(), which both performs "merging" and
validation, as well as actually updating the daemon configuration. Calling
DaemonCli.reloadConfig() *before* validation, could thus lead to a failure in
that function (making the reload non-atomic).

While *some* errors could always occur when applying/updating the config, we
should make it more atomic, and;

1. get (a copy of) the active configuration
2. get the new configuration
3. apply the (reloadable) options from the new configuration
4. validate the merged results
5. apply the new configuration.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-28 09:30:04 +02:00
Sebastiaan van Stijn
9a54dadc44
daemon/config: MergeDaemonConfigurations() don't validate intermediates
MergeDaemonConfigurations was validating the configs before and after
merging. However, the "fileConfig" configuration may contain only a
"partial" configuration (options to apply to / override the existing
config). This means that some options may not be set and contain default
or empty values.

Validating such partial configurations can produce validation failures,
so to prevent those, we should validate the configuration _after_
merging, to validate the "final" state.

There's more cleaning up / improvements to be made in this area; for
example, we currently use our "self crafted" `getConflictFreeConfiguration()`
function, which is used to detect options that are not allowed to
be overridden, and which could potentially be handled by mergo.Merge(),
but leaving those changes for a future exercise.

This patch removes the first validation step, changing the function
to only validate the resulting configuration after merging.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-28 09:30:02 +02:00
Sebastiaan van Stijn
b819480899
daemon/config: Reload(): normalize labels before validation
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-28 09:30:00 +02:00
Brian Goff
f7d070b58c
Merge pull request #43538 from corhere/dry-metrics-definitions
metrics: DRY metric definitions
2022-04-27 10:26:25 -07:00
Sebastiaan van Stijn
390c7d6871
daemon/config: Validate(): validate log-level
Log-level validation was previously performed when configuring the daemon-logs;
this moves the validation to config.Validate() so that we can catch invalid
settings when running dockerd --validate.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-27 00:34:14 +02:00
Sebastiaan van Stijn
787257f767
Merge pull request #43332 from thaJeztah/api_swagger_move_definitions
api: swagger: use explicit definitions for some response types, and move examples per-field
2022-04-26 23:46:49 +02:00
Sebastiaan van Stijn
9184f0b5e4
Merge pull request #43365 from thaJeztah/cleanup_distribution
distribution: remove v1 leftovers, and refactor to reduce public api/interface
2022-04-26 23:45:38 +02:00
Sebastiaan van Stijn
3b56c0663d
daemon: daemon.networkOptions(): don't pass Config as argument
This is a method on the daemon, which itself holds the Config, so
there's no need to pass the same configuration as an argument.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-23 23:34:13 +02:00
Eng Zer Jun
36049a04d2
test: use T.Setenv to set env vars in tests
This commit replaces `os.Setenv` with `t.Setenv` in tests. The
environment variable is automatically restored to its original value
when the test and all its subtests complete.

Reference: https://pkg.go.dev/testing#T.Setenv
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2022-04-23 17:44:16 +08:00
Cory Snider
1c129103b4 Bump swarmkit to v2
Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-04-21 17:33:07 -04:00
Sebastiaan van Stijn
566c8db66d
distribution: add GetRepository(), un-export NewV2Repository, ValidateRepoName
These were only exported to facilitate ImageService.GetRepository() (used for
the `GET /distribution/{name:.*}/json` endpoint.

Moving the core functionality of that to the distribution package makes it
more consistent with (e.g.) "pull" operations, and allows us to keep more things
internal.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-21 23:12:02 +02:00
Sebastiaan van Stijn
fb5485f5d0
distribution: un-export ImageTypes, make ImagePullConfig.Schema2Types optional
Use the default list of accepted mediaTypes if none were passed.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-21 22:36:56 +02:00
Samuel Karp
ccb691a427
Merge pull request #43511 from thaJeztah/no_logrus_fatal 2022-04-21 11:33:43 -07:00
Sebastiaan van Stijn
176f66df9c
api/types: replace uses of deprecated types.Volume with volume.Volume
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-21 19:50:59 +02:00
Sebastiaan van Stijn
df650a1aeb
panic() instead of logrus.Fatal() in init funcs
Some packages were using `logrus.Fatal()` in init functions (which logs the error,
and (by default) calls `os.Exit(1)` after logging).

Given that logrus formatting and outputs have not yet been configured during the
initialization stage, it does not provide much benefits over a plain `panic()`.

This patch replaces some instances of `logrus.Fatal()` with `panic()`, which has
the added benefits of not introducing logrus as a dependency in some of these
packages, and also produces a stacktrace, which could help locating the problem
in the unlikely event an `init()` fails.

Before this change, an error would look like:

    $ dockerd
    FATA[0000] something bad happened

After this change, the same error looks like:

    $ dockerd
    panic: something bad happened

    goroutine 1 [running]:
      github.com/docker/docker/daemon/logger/awslogs.init.0()
        /go/src/github.com/docker/docker/daemon/logger/awslogs/cloudwatchlogs.go:128 +0x89

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-21 12:15:20 +02:00
Sebastiaan van Stijn
83a185897d
API: add "signal" parameter to container stop and restart endpoints
Containers can have a default stop-signal (`--stop-signal` / `STOPSIGNAL`) and
timeout (`--stop-timeout`). It is currently not possible to update either of
these after the container is created (`docker update` does not allow updating
them), and while either of these can be overridden through some commands, we
currently do not have a command that can override *both*:

command         | stop-signal | stop-timeout | notes
----------------|-------------|--------------|----------------------------
docker kill     | yes         | DNA          | only sends a single signal
docker restart  | no          | yes          |
docker stop     | no          | yes          |

As a result, if a user wants to stop a container with a custom signal and
timeout, the only option is to do this manually:

    docker kill -s <custom signal> mycontainer
    # wait <desired timeout>
    # press ^C to cancel the graceful stop
    # forcibly kill the container
    docker kill mycontainer

This patch adds a new `signal` query parameter to the container "stop" and
"restart" endpoints. This parameter can be added as a new flag on the CLI,
which would allow stopping and restarting with a custom timeout and signal,
for example:

    docker stop --signal=SIGWINCH --time=120 mycontainer

    docker restart --signal=SIGWINCH --time=120 mycontainer

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-20 21:29:31 +02:00
Sebastiaan van Stijn
90de570cfa
backend: add StopOptions to ContainerRestart and ContainerStop
While we're modifying the interface, also add a context to both.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-20 21:29:30 +02:00
Sebastiaan van Stijn
952902efbc
daemon: containerStop(): use a regular "defer" to log container event
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-20 21:29:27 +02:00