Commit graph

201 commits

Author SHA1 Message Date
Rob Murray
6c68be24a2 Windows DNS resolver forwarding
Make the internal DNS resolver for Windows containers forward requests
to upsteam DNS servers when it cannot respond itself, rather than
returning SERVFAIL.

Windows containers are normally configured with the internal resolver
first for service discovery (container name lookup), then external
resolvers from '--dns' or the host's networking configuration.

When a tool like ping gets a SERVFAIL from the internal resolver, it
tries the other nameservers. But, nslookup does not, and with this
change it does not need to.

The internal resolver learns external server addresses from the
container's HNSEndpoint configuration, so it will use the same DNS
servers as processes in the container.

The internal resolver for Windows containers listens on the network's
gateway address, and each container may have a different set of external
DNS servers. So, the resolver uses the source address of the DNS request
to select external resolvers.

On Windows, daemon.json feature option 'windows-no-dns-proxy' can be used
to prevent the internal resolver from forwarding requests (restoring the
old behaviour).

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-04-16 18:57:28 +01:00
Sebastiaan van Stijn
6b01719ffb
api: add MinSupportedAPIVersion const
This const contains the minimum API version that can be supported by the
API server. The daemon is currently configured to use the same version,
but we may increment the _configured_ minimum version when deprecating
old API versions in future.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-06 18:44:44 +01:00
Sebastiaan van Stijn
19a04efa2f
api: remove API < v1.24
Commit 08e4e88482 (Docker Engine v25.0.0)
deprecated API version v1.23 and lower, but older API versions could be
enabled through the DOCKER_MIN_API_VERSION environment variable.

This patch removes all support for API versions < v1.24.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-06 18:44:44 +01:00
Albin Kerouanton
f07c45e4f2
daemon: remove --oom-score-adjust flag
This flag was marked deprecated in commit 5a922dc16 (released in v24.0)
and to be removed in the next release.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-01-20 00:40:28 +01:00
Sebastiaan van Stijn
92175dd78c
vendor: containerd v1.7.12, and switch to dario.cat/mergo v1.0.0
Switch github.com/imdario/mergo to dario.cat/mergo v1.0.0, because
the module was renamed, and reached v1.0.0

full diff: https://github.com/imdario/mergo/compare/v0.3.13...v1.0.0

vendor: github.com/containerd/containerd v1.7.12

- full diff: https://github.com/containerd/containerd/compare/v1.7.11...v1.7.12
- release notes: https://github.com/containerd/containerd/releases/tag/v1.7.12

Welcome to the v1.7.12 release of containerd!

The twelfth patch release for containerd 1.7 contains various fixes and updates.

Notable Updates

- Fix on dialer function for Windows
- Improve `/etc/group` handling when appending groups
- Update shim pidfile permissions to 0644
- Update runc binary to v1.1.11
- Allow import and export to reference missing content
- Remove runc import
- Update Go version to 1.20.13

Deprecation Warnings

- Emit deprecation warning for `containerd.io/restart.logpath` label usage

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-12 18:09:24 +01:00
Sebastiaan van Stijn
e8e20c0897
daemon/config: setPlatformDefaults: use debug for missing userland-proxy
commit 4f9db655ed moved looking up the
userland-proxy binary to early in the startup process, and introduced
a log-message if the binary was missing.

However, a side-effect of this was this message would also be printed
when running "--version";

    dockerd --version
    time="2024-01-09T09:18:53.705271292Z" level=warning msg="failed to lookup default userland-proxy binary" error="exec: \"docker-proxy\": executable file not found in $PATH"
    Docker version v25.0.0-rc.1, build 9cebefa717

We should look if we can avoid this, but let's change the message to be
a debug message as a short-term workaround.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-09 13:18:04 +01:00
Sebastiaan van Stijn
4f9db655ed
portmapper: move userland-proxy lookup to daemon config
When mapping a port with the userland-proxy enabled, the daemon would
perform an "exec.LookPath" for every mapped port (which, in case of
a range of ports, would be for every port in the range).

This was both inefficient (looking up the binary for each port), inconsistent
(when running in rootless-mode, the binary was looked-up once), as well as
inconvenient, because a missing binary, or a mis-configureed userland-proxy-path
would not be detected daeemon startup, and not produce an error until starting
the container;

    docker run -d -P nginx:alpine
    4f7b6589a1680f883d98d03db12203973387f9061e7a963331776170e4414194
    docker: Error response from daemon: driver failed programming external connectivity on endpoint romantic_wiles (7cfdc361821f75cbc665564cf49856cf216a5b09046d3c22d5b9988836ee088d): fork/exec docker-proxy: no such file or directory.

However, the container would still be created (but invalid);

    docker ps -a
    CONTAINER ID   IMAGE          COMMAND                  CREATED          STATUS    PORTS     NAMES
    869f41d7e94f   nginx:alpine   "/docker-entrypoint.…"   10 seconds ago   Created             romantic_wiles

This patch changes how the userland-proxy is configured;

- The path of the userland-proxy is now looked up / configured at daemon
  startup; this is similar to how the proxy is configured in rootless-mode.
- A warning is logged when failing to lookup the binary.
- If the daemon is configured with "userland-proxy" enabled, an error is
  produced, and the daemon will refuse to start.
- The "proxyPath" argument for newProxyCommand() (in libnetwork/portmapper)
  is now required to be set. It no longer looks up the executable, and
  produces an error if no path was provided. While this change was not
  required, it makes the daemon config the canonical source of truth, instead
  of logic spread accross multiplee locations.

Some of this logic is a change of behavior, but these changes were made with
the assumption that we don't want to support;

- installing the userland proxy _after_ the daemon was started
- moving the userland proxy (or installing a proxy with a higher
  preference in PATH)

With this patch:

Validating the config produces an error if the binary is not found:

    dockerd --validate
    WARN[2023-12-29T11:36:39.748699591Z] failed to lookup default userland-proxy binary       error="exec: \"docker-proxy\": executable file not found in $PATH"
    userland-proxy is enabled, but userland-proxy-path is not set

Disabling userland-proxy prints a warning, but validates as "OK":

    dockerd --userland-proxy=false --validate
    WARN[2023-12-29T11:38:30.752523879Z] ffailed to lookup default userland-proxy binary       error="exec: \"docker-proxy\": executable file not found in $PATH"
    configuration OK

Speficying a non-absolute path produces an error:

    dockerd --userland-proxy-path=docker-proxy --validate
    invalid userland-proxy-path: must be an absolute path: docker-proxy

Befor this patch, we would not validate this path, which would allow the daemon
to start, but fail to map a port;

    docker run -d -P nginx:alpine
    4f7b6589a1680f883d98d03db12203973387f9061e7a963331776170e4414194
    docker: Error response from daemon: driver failed programming external connectivity on endpoint romantic_wiles (7cfdc361821f75cbc665564cf49856cf216a5b09046d3c22d5b9988836ee088d): fork/exec docker-proxy: no such file or directory.

Specifying an invalid userland-proxy-path produces an error as well:

    dockerd --userland-proxy-path=/usr/local/bin/no-such-binary --validate
    userland-proxy-path is invalid: stat /usr/local/bin/no-such-binary: no such file or directory

    mkdir -p /usr/local/bin/not-a-file
    dockerd --userland-proxy-path=/usr/local/bin/not-a-file --validate
    userland-proxy-path is invalid: exec: "/usr/local/bin/not-a-file": is a directory

    touch /usr/local/bin/not-an-executable
    dockerd --userland-proxy-path=/usr/local/bin/not-an-executable --validate
    userland-proxy-path is invalid: exec: "/usr/local/bin/not-an-executable": permission denied

Same when using the daemon.json config-file;

    echo '{"userland-proxy-path":"no-such-binary"}' > /etc/docker/daemon.json
    dockerd --validate
    unable to configure the Docker daemon with file /etc/docker/daemon.json: merged configuration validation from file and command line flags failed: invalid userland-proxy-path: must be an absolute path: no-such-binary

    dockerd --userland-proxy-path=hello --validate
    unable to configure the Docker daemon with file /etc/docker/daemon.json: the following directives are specified both as a flag and in the configuration file: userland-proxy-path: (from flag: hello, from file: /usr/local/bin/docker-proxy)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-29 16:23:18 +01:00
Sebastiaan van Stijn
388216fc45
Merge pull request #46850 from robmry/46829-allow_ipv6_subnet_change
Allow overlapping change in bridge's IPv6 network.
2023-12-19 18:35:13 +01:00
Rob Murray
27f3abd893 Allow overlapping change in bridge's IPv6 network.
Calculate the IPv6 addreesses needed on a bridge, then reconcile them
with the addresses on an existing bridge by deleting then adding as
required.

(Previously, required addresses were added one-by-one, then unwanted
addresses were removed. This meant the daemon failed to start if, for
example, an existing bridge had address '2000:db8::/64' and the config
was changed to '2000:db8::/80'.)

IPv6 addresses are now calculated and applied in one go, so there's no
need for setupVerifyAndReconcile() to check the set of IPv6 addresses on
the bridge. And, it was guarded by !config.InhibitIPv4, which can't have
been right. So, removed its IPv6 parts, and added IPv4 to its name.

Link local addresses, the example given in the original ticket, are now
released when containers are stopped. Not releasing them meant that
when using an LL subnet on the default bridge, no container could be
started after a container was stopped (because the calculated address
could not be re-allocated). In non-default bridge networks using an
LL subnet, addresses leaked.

Linux always uses the standard 'fe80::/64' LL network. So, if a bridge
is configured with an LL subnet prefix that overlaps with it, a config
error is reported. Non-overlapping LL subnet prefixes are allowed.

Signed-off-by: Rob Murray <rob.murray@docker.com>
2023-12-18 16:10:41 +00:00
Sebastiaan van Stijn
08e4e88482
daemon: raise default minimum API version to v1.24
The daemon currently provides support for API versions all the way back
to v1.12, which is the version of the API that shipped with docker 1.0. On
Windows, the minimum supported version is v1.24.

Such old versions of the client are rare, and supporting older API versions
has accumulated significant amounts of code to remain backward-compatible
(which is largely untested, and a "best-effort" at most).

This patch updates the minimum API version to v1.24, which is the fallback
API version used when API-version negotiation fails. The intent is to start
deprecating older API versions, but no code is removed yet as part of this
patch, and a DOCKER_MIN_API_VERSION environment variable is added, which
allows overriding the minimum version (to allow restoring the behavior from
before this patch).

With this patch the daemon defaults to API v1.24 as minimum:

    docker version
    Client:
     Version:           24.0.2
     API version:       1.43
     Go version:        go1.20.4
     Git commit:        cb74dfc
     Built:             Thu May 25 21:50:49 2023
     OS/Arch:           linux/arm64
     Context:           default

    Server:
     Engine:
      Version:          dev
      API version:      1.44 (minimum version 1.24)
      Go version:       go1.21.3
      Git commit:       0322a29b9ef8806aaa4b45dc9d9a2ebcf0244bf4
      Built:            Mon Dec  4 15:22:17 2023
      OS/Arch:          linux/arm64
      Experimental:     false
     containerd:
      Version:          v1.7.9
      GitCommit:        4f03e100cb967922bec7459a78d16ccbac9bb81d
     runc:
      Version:          1.1.10
      GitCommit:        v1.1.10-0-g18a0cb0
     docker-init:
      Version:          0.19.0
      GitCommit:        de40ad0

Trying to use an older version of the API produces an error:

    DOCKER_API_VERSION=1.23 docker version
    Client:
     Version:           24.0.2
     API version:       1.23 (downgraded from 1.43)
     Go version:        go1.20.4
     Git commit:        cb74dfc
     Built:             Thu May 25 21:50:49 2023
     OS/Arch:           linux/arm64
     Context:           default
    Error response from daemon: client version 1.23 is too old. Minimum supported API version is 1.24, please upgrade your client to a newer version

To restore the previous minimum, users can start the daemon with the
DOCKER_MIN_API_VERSION environment variable set:

    DOCKER_MIN_API_VERSION=1.12 dockerd

API 1.12 is the oldest supported API version on Linux;

    docker version
    Client:
     Version:           24.0.2
     API version:       1.43
     Go version:        go1.20.4
     Git commit:        cb74dfc
     Built:             Thu May 25 21:50:49 2023
     OS/Arch:           linux/arm64
     Context:           default

    Server:
     Engine:
      Version:          dev
      API version:      1.44 (minimum version 1.12)
      Go version:       go1.21.3
      Git commit:       0322a29b9ef8806aaa4b45dc9d9a2ebcf0244bf4
      Built:            Mon Dec  4 15:22:17 2023
      OS/Arch:          linux/arm64
      Experimental:     false
     containerd:
      Version:          v1.7.9
      GitCommit:        4f03e100cb967922bec7459a78d16ccbac9bb81d
     runc:
      Version:          1.1.10
      GitCommit:        v1.1.10-0-g18a0cb0
     docker-init:
      Version:          0.19.0
      GitCommit:        de40ad0

When using the `DOCKER_MIN_API_VERSION` with a version of the API that
is not supported, an error is produced when starting the daemon;

    DOCKER_MIN_API_VERSION=1.11 dockerd --validate
    invalid DOCKER_MIN_API_VERSION: minimum supported API version is 1.12: 1.11

    DOCKER_MIN_API_VERSION=1.45 dockerd --validate
    invalid DOCKER_MIN_API_VERSION: maximum supported API version is 1.44: 1.45

Specifying a malformed API version also produces the same error;

    DOCKER_MIN_API_VERSION=hello dockerd --validate
    invalid DOCKER_MIN_API_VERSION: minimum supported API version is 1.12: hello

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-05 23:11:02 +01:00
Sebastiaan van Stijn
84036d3e18
daemon/config: change DNSConfig.DNS to a []net.IP
Use a strong type for the DNS IP-addresses so that we can use flags.IPSliceVar,
instead of implementing our own option-type and validation.

Behavior should be the same, although error-messages have slightly changed:

Before this patch:

    dockerd --dns 1.1.1.1oooo --validate
    Status: invalid argument "1.1.1.1oooo" for "--dns" flag: 1.1.1.1oooo is not an ip address
    See 'dockerd --help'., Code: 125

    cat /etc/docker/daemon.json
    {"dns": ["1.1.1.1"]}

    dockerd --dns 2.2.2.2 --validate
    unable to configure the Docker daemon with file /etc/docker/daemon.json: the following directives are specified both as a flag and in the configuration file: dns: (from flag: [2.2.2.2], from file: [1.1.1.1])

    cat /etc/docker/daemon.json
    {"dns": ["1.1.1.1oooo"]}

    dockerd --validate
    unable to configure the Docker daemon with file /etc/docker/daemon.json: merged configuration validation from file and command line flags failed: 1.1.1.1ooooo is not an ip address

With this patch:

    dockerd --dns 1.1.1.1oooo --validate
    Status: invalid argument "1.1.1.1oooo" for "--dns" flag: invalid string being converted to IP address: 1.1.1.1oooo
    See 'dockerd --help'., Code: 125

    cat /etc/docker/daemon.json
    {"dns": ["1.1.1.1"]}

    dockerd --dns 2.2.2.2 --validate
    unable to configure the Docker daemon with file /etc/docker/daemon.json: the following directives are specified both as a flag and in the configuration file: dns: (from flag: [2.2.2.2], from file: [1.1.1.1])

    cat /etc/docker/daemon.json
    {"dns": ["1.1.1.1oooo"]}

    dockerd --validate
    unable to configure the Docker daemon with file /etc/docker/daemon.json: invalid IP address: 1.1.1.1oooo

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-11-13 12:22:51 +01:00
Sebastiaan van Stijn
1eadfb0e28
opts: ValidateIPAddress: improve error, godoc, and tests
- document accepted values
- add test-coverage for the function's behavior (including whitespace handling),
  and use sub-tests.
- improve error-message to use uppercase for "IP", and to use a common prefix.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-11-11 15:31:19 +01:00
Albin Kerouanton
d5d41c2849
daemon/config: Put params for the default network into a dedicated struct
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-11-03 14:10:41 +01:00
Sebastiaan van Stijn
cff4f20c44
migrate to github.com/containerd/log v0.1.0
The github.com/containerd/containerd/log package was moved to a separate
module, which will also be used by upcoming (patch) releases of containerd.

This patch moves our own uses of the package to use the new module.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-11 17:52:23 +02:00
Sebastiaan van Stijn
3614749b55
Merge pull request #45966 from neersighted/buildkit_0.12
Update to BuildKit 0.12
2023-09-22 02:13:15 +02:00
Bjorn Neergaard
2246297ae6
builder-next: adopt new wrapped types
With BuildKit 0.12, some existing types are now required to be wrapped
by new types:

* containerd's LeaseManager and ContentStore have to be a
  (namespace-aware) BuildKit type since f044e0a946
* BuildKit's solver.CacheManager is used instead of
  bboltstorage.CacheKeyStorage since 2b30693409
* The MaxAge config field is a bkconfig.Duration since e06c96274f

Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-09-21 14:18:48 -06:00
Bjorn Neergaard
0e80073e01
daemon: strongly type containerd log.OutputFormat
This type was introduced in
0a79e67e4f

Make use of it throughout our log-format handling code, and convert back
to a string before we pass it to the containerd client.

Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-09-21 05:40:17 -06:00
Sebastiaan van Stijn
bd523abd44
remove more direct uses of logrus
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-15 20:12:27 +02:00
Sebastiaan van Stijn
1ffbbe798a
vendor: github.com/containerd/containerd v1.6.24
full diff: https://github.com/containerd/containerd/compare/v1.6.22...v1.6.24

v1.6.24 release notes:

full diff: https://github.com/containerd/containerd/compare/v1.6.23...v1.6.24

The twenty-fourth patch release for containerd 1.6 contains various fixes
and updates.

Notable Updates

- CRI: fix leaked shim caused by high IO pressure
- Update to go1.20.8
- Update runc to v1.1.9
- Backport: add configurable mount options to overlay snapshotter
- log: cleanups and improvements to decouple more from logrus

v1.6.23 release notes:

full diff: https://github.com/containerd/containerd/compare/v1.6.22...v1.6.23

The twenty-third patch release for containerd 1.6 contains various fixes
and updates.

Notable Updates

- Add stable ABI support in windows platform matcher + update hcsshim tag
- cri: Don't use rel path for image volumes
- Upgrade GitHub actions packages in release workflow
- update to go1.19.12
- backport: ro option for userxattr mount check + cherry-pick: Fix ro mount option being passed

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-15 17:34:58 +02:00
Sebastiaan van Stijn
ce01d779df
daemon: format code with gofumpt
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-24 17:59:22 +02:00
Evan Lezar
3b71197fb8 Set default CDI spec dirs after parsing args
Signed-off-by: Evan Lezar <elezar@nvidia.com>
2023-08-03 12:21:44 +02:00
Cory Snider
0c2699da27
Merge pull request #45737 from pkwarren/pkw/issue-44940-dockerd-json-logs
Update dockerd to support JSON logging format
2023-07-13 19:00:31 -04:00
Sebastiaan van Stijn
c90229ed9a
api/types: move system info types to api/types/system
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-07 13:01:36 +02:00
Sebastiaan van Stijn
b8220f5d0d
daemon/config: move MTU to BridgeConfig
This option is only used for the default bridge network; let's move the
field to that struct to make it clearer what it's used for.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-05 14:43:35 +02:00
Sebastiaan van Stijn
11abd0742e
dockerd: "--mtu": update description, hide on Windows and warn if set
The --mtu option is only used for the default "bridge" network on Linux.
On Windows, the flag is available, but ignored. As this option has been
available for a long time, and was always silently ignored, deprecating
or removing it would be a breaking change (and perhaps it's possible to
support it in future).

This patch:

- hides the option on Windows binaries
- logs a warning if the option is set to any non-zero value other than
  the default on a Windows binary

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-05 14:43:27 +02:00
Sebastiaan van Stijn
13472c6cf1
daemon/config: remove unused CommonConfig.Context
This field was added in f0e6e135a8, and
from that change I suspect it was intended to store the default SELinux
mount-labels to be set on containers.

However, it was never used, so let's remove it.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-30 16:44:53 +02:00
Sebastiaan van Stijn
8d923c60b9
daemon/config: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:31:11 +02:00
Philip K. Warren
a08abec9f8 Update dockerd to support JSON logging format
Update docker to support a '--log-format' option, which accepts either
'text' (default) or 'json'. Propagate the log format to containerd as
well, to ensure that everything will be logged consistently.

Signed-off-by: Philip K. Warren <pkwarren@gmail.com>
2023-06-28 12:46:28 -05:00
Brian Goff
74da6a6363 Switch all logging to use containerd log pkg
This unifies our logging and allows us to propagate logging and trace
contexts together.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2023-06-24 00:23:44 +00:00
Cory Snider
0f6eeecac0 daemon: consolidate runtimes config validation
The daemon has made a habit of mutating the DefaultRuntime and Runtimes
values in the Config struct to merge defaults. This would be fine if it
was a part of the regular configuration loading and merging process,
as is done with other config options. The trouble is it does so in
surprising places, such as in functions with 'verify' or 'validate' in
their name. It has been necessary in order to validate that the user has
not defined a custom runtime named "runc" which would shadow the
built-in runtime of the same name. Other daemon code depends on the
runtime named "runc" always being defined in the config, but merging it
with the user config at the same time as the other defaults are merged
would trip the validation. The root of the issue is that the daemon has
used the same config values for both validating the daemon runtime
configuration as supplied by the user and for keeping track of which
runtimes have been set up by the daemon. Now that a completely separate
value is used for the latter purpose, surprising contortions are no
longer required to make the validation work as intended.

Consolidate the validation of the runtimes config and merging of the
built-in runtimes into the daemon.setupRuntimes() function. Set the
result of merging the built-in runtimes config and default default
runtime on the returned runtimes struct, without back-propagating it
onto the config.Config argument.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-06-01 14:45:25 -04:00
Cory Snider
d222bf097c daemon: reload runtimes w/o breaking containers
The existing runtimes reload logic went to great lengths to replace the
directory containing runtime wrapper scripts as atomically as possible
within the limitations of the Linux filesystem ABI. Trouble is,
atomically swapping the wrapper scripts directory solves the wrong
problem! The runtime configuration is "locked in" when a container is
started, including the path to the runC binary. If a container is
started with a runtime which requires a daemon-managed wrapper script
and then the daemon is reloaded with a config which no longer requires
the wrapper script (i.e. some args -> no args, or the runtime is dropped
from the config), that container would become unmanageable. Any attempts
to stop, exec or otherwise perform lifecycle management operations on
the container are likely to fail due to the wrapper script no longer
existing at its original path.

Atomically swapping the wrapper scripts is also incompatible with the
read-copy-update paradigm for reloading configuration. A handler in the
daemon could retain a reference to the pre-reload configuration for an
indeterminate amount of time after the daemon configuration has been
reloaded and updated. It is possible for the daemon to attempt to start
a container using a deleted wrapper script if a request to run a
container races a reload.

Solve the problem of deleting referenced wrapper scripts by ensuring
that all wrapper scripts are *immutable* for the lifetime of the daemon
process. Any given runtime wrapper script must always exist with the
same contents, no matter how many times the daemon config is reloaded,
or what changes are made to the config. This is accomplished by using
everyone's favourite design pattern: content-addressable storage. Each
wrapper script file name is suffixed with the SHA-256 digest of its
contents to (probabilistically) guarantee immutability without needing
any concurrency control. Stale runtime wrapper scripts are only cleaned
up on the next daemon restart.

Split the derived runtimes configuration from the user-supplied
configuration to have a place to store derived state without mutating
the user-supplied configuration or exposing daemon internals in API
struct types. Hold the derived state and the user-supplied configuration
in a single struct value so that they can be updated as an atomic unit.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-06-01 14:45:25 -04:00
Cory Snider
0b592467d9 daemon: read-copy-update the daemon config
Ensure data-race-free access to the daemon configuration without
locking by mutating a deep copy of the config and atomically storing
a pointer to the copy into the daemon-wide configStore value. Any
operations which need to read from the daemon config must capture the
configStore value only once and pass it around to guarantee a consistent
view of the config.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-06-01 14:45:24 -04:00
Evan Lezar
7ec9561a77 Add support for CDI devices to docker daemon under linux
These changes add basic CDI integration to the docker daemon.

A cdi driver is added to handle cdi device requests. This
is gated by an experimental feature flag and is only supported on linux

This change also adds a CDISpecDirs (cdi-spec-dirs) option to the config.
This allows the default values of `/etc/cdi`, /var/run/cdi` to be overridden
which is useful for testing.

Signed-off-by: Evan Lezar <elezar@nvidia.com>
2023-05-16 17:07:57 +02:00
Sebastiaan van Stijn
fb96b94ed0
daemon: remove handling for deprecated "oom-score-adjust", and produce error
This option was deprecated in 5a922dc162, which
is part of the v24.0.0 release, so we can remove it from master.

This patch;

- adds a check to ValidatePlatformConfig, and produces a fatal error
  if oom-score-adjust is set
- removes the deprecated libcontainerd/supervisor.WithOOMScore
- removes the warning from docker info

With this patch:

    dockerd --oom-score-adjust=-500 --validate
    Flag --oom-score-adjust has been deprecated, and will be removed in the next release.
    unable to configure the Docker daemon with file /etc/docker/daemon.json: merged configuration validation from file and command line flags failed: DEPRECATED: The "oom-score-adjust" config parameter and the dockerd "--oom-score-adjust" options have been removed.

And when using `daemon.json`:

    dockerd --validate
    unable to configure the Docker daemon with file /etc/docker/daemon.json: merged configuration validation from file and command line flags failed: DEPRECATED: The "oom-score-adjust" config parameter and the dockerd "--oom-score-adjust" options have been removed.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-06 16:36:17 +02:00
Sebastiaan van Stijn
5a922dc162
daemon: deprecate --oom-score-adjust for the daemon
The `oom-score-adjust` option was added in a894aec8d8,
to prevent the daemon from being OOM-killed before other processes. This
option was mostly added as a "convenience", as running the daemon as a
systemd unit was not yet common.

Having the daemon set its own limits is not best-practice, and something
better handled by the process-manager starting the daemon.

Commit cf7a5be0f2 fixed this option to allow
disabling it, and 2b8e68ef06 removed the default
score adjust.

This patch deprecates the option altogether, recommending users to set these
limits through the process manager used, such as the "OOMScoreAdjust" option
in systemd units.

With this patch:

    dockerd --oom-score-adjust=-500 --validate
    Flag --oom-score-adjust has been deprecated, and will be removed in the next release.
    configuration OK

    echo '{"oom-score-adjust":-500}' > /etc/docker/daemon.json
    dockerd
    INFO[2023-04-12T21:34:51.133389627Z] Starting up
    INFO[2023-04-12T21:34:51.135607544Z] containerd not running, starting managed containerd
    WARN[2023-04-12T21:34:51.135629086Z] DEPRECATED: The "oom-score-adjust" config parameter and the dockerd "--oom-score-adjust" option will be removed in the next release.

    docker info
    Client:
      Context:    default
      Debug Mode: false
    ...
    DEPRECATED: The "oom-score-adjust" config parameter and the dockerd "--oom-score-adjust" option will be removed in the next release

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-13 00:02:39 +02:00
Cory Snider
750218c5e5 d/config: fix default-network-opts in daemon.json
Prevent the daemon from erroring out if daemon.json contains default
network options for network drivers aside from bridge. Configuring
defaults for the bridge driver previously worked by coincidence because
the unrelated CLI flag '--bridge' exists.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-04-03 15:59:53 -04:00
Tianon Gravi
6caaa8cadc Prefer loading docker-init from an appropriate "libexec" directory
The `docker-init` binary is not intended to be a user-facing command, and as such it is more appropriate for it to be found in `/usr/libexec` (or similar) than in `PATH` (see the FHS, especially https://refspecs.linuxfoundation.org/FHS_3.0/fhs/ch04s07.html and https://refspecs.linuxfoundation.org/FHS_2.3/fhs-2.3.html#USRLIBLIBRARIESFORPROGRAMMINGANDPA).

This adjusts the logic for using that configuration option to take this into account and appropriately search for `docker-init` (or the user's configured alternative) in these directories before falling back to the existing `PATH` lookup behavior.

This behavior _used_ to exist for the old `dockerinit` binary (of a similar name and used in a similar way but for an alternative purpose), but that behavior was removed in 4357ed4a73 when that older `dockerinit` was also removed.

Most of this reasoning _also_ applies to `docker-proxy` (and various `containerd-xxx` binaries such as the shims), but this change does not affect those.  It would be relatively straightforward to adapt `LookupInitPath` to be a more generic function such as `libexecLookupPath` or similar if we wanted to explore that.

See 14482589df/cli-plugins/manager/manager_unix.go for the related path list in the CLI which loads CLI plugins from a similar set of paths (with a similar rationale - plugin binaries are not typically intended to be run directly by users but rather invoked _via_ the CLI binary).

Signed-off-by: Tianon Gravi <admwiggin@gmail.com>
2023-03-24 14:25:12 -07:00
Akihiro Suda
e807ae4f2e
vendor: github.com/containerd/cgroups/v3 v3.0.1
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2023-03-08 20:15:17 +09:00
Sebastiaan van Stijn
2323f9deda
Merge pull request #43197 from dajudge/default-bridge-mtu
Introduce config option for default generic network options of newly created networks
2023-03-02 20:21:47 +01:00
Cory Snider
a9e7360775 daemon/config: remove AuthzMiddleware field
The authorization.Middleware contains a sync.Mutex field, making it
non-copyable. Remove one of the barriers to allowing deep copies of
config.Config values.

Inject the middleware into Daemon as a constructor argument instead.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-03-01 09:43:39 -05:00
Alex Stockinger
91c2b12205 Make default options for newly created networks configurable
Signed-off-by: Alex Stockinger <alex@atomicjar.com>
Co-authored-by: Sergei Egorov <bsideup@gmail.com>
Co-authored-by: Cory Snider <corhere@gmail.com>
2023-03-01 07:58:26 +01:00
Tonis Tiigi
666334bd48
vendor: update buildkit to v0.11.2
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2023-02-01 23:29:10 -08:00
Bjorn Neergaard
8dbc5df952
daemon/config: inform the user when the input JSON contains invalid UTF-8
This helps ensure that users are not surprised by unexpected tokens in
the JSON parser, or fallout later in the daemon.

Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
2023-01-10 17:27:21 -07:00
Bjorn Neergaard
d42495033e
daemon/config: support alternate (common) unicode encodings using a BOM
This is a pragmatic but impure choice, in order to better support the
default tools available on Windows Server, and reduce user confusion due
to otherwise inscrutable-to-the-uninitiated errors like the following:

> invalid character 'þ' looking for beginning of value
> invalid character 'ÿ' looking for beginning of value

While meaningful to those who are familiar with and are equipped to
diagnose encoding issues, these characters will be hidden when the file
is edited with a BOM-aware text editor, and further confuse the user.

Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
2023-01-10 15:49:15 -07:00
Bjorn Neergaard
1dcf7d5b03
daemon/config: clean up tests to use common helper
Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
2023-01-10 13:54:17 -07:00
Bjorn Neergaard
62227e1bba
Merge pull request #44733 from jg-public/fix-rootless-specspaths--T43111
Use user data path for plugin discovery in rootless mode
2023-01-09 17:32:11 -07:00
Bjorn Neergaard
cba3edbc37
daemon/config: group JSON preprocessing steps
Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
2023-01-09 13:17:32 -07:00
Bjorn Neergaard
bb19265ba8
daemon/config: ignore UTF-8 BOM in config JSON
[RFC 8259] allows for JSON implementations to optionally ignore a BOM
when it helps with interoperability; do so in Moby as Notepad (the only
text editor available out of the box in many versions of Windows Server)
insists on writing UTF-8 with a BOM.

  [RFC 8259]: https://tools.ietf.org/html/rfc8259#section-8.1

Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
2023-01-09 13:16:57 -07:00
Jan Garcia
6ab12ec8f4 rootless: move ./rootless to ./pkg/rootless
Signed-off-by: Jan Garcia <github-public@n-garcia.com>
2023-01-09 16:26:06 +01:00
Sebastiaan van Stijn
e19dff9285
daemon/config: remove deprecated RootDeprecated field
Since b58de39ca7, this option was now only used
to produce a fatal error when starting the daemon. That change is in the 23.0
release, so we can remove it from the master branch.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-15 10:44:32 +01:00