Commit graph

512 commits

Author SHA1 Message Date
Cory Snider
0f3c5d3893 cmd/dockerd/trap: don't force exit after cleanup
Always calling os.Exit() on clean shutdown may not always be desirable
as deferred functions are not run. Let the cleanup callback decide
whether or not to call os.Exit() itself. Allow the process to exit the
normal way, by returning from func main().

Simplify the trap.Trap implementation. The signal notifications are
buffered in a channel so there is little need to spawn a new goroutine
for each received signal. With all signals being handled in the same
goroutine, there are no longer any concurrency concerns around the
interrupt counter.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-04-26 09:51:36 -04:00
Djordje Lukic
a1e1038736
Use the image service instead of the reference store for tagging
The image store sends events when a new image is created/tagged, using
it instead of the reference store makes sure we send the "tag" event
when a new image is built using buildx.

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-04-26 15:10:01 +02:00
Cory Snider
16d5d4b6e1 cmd/dockerd: ignore SIGPIPE using signal.Ignore
The fix to ignore SIGPIPE signals was originally added in the Go 1.4
era. signal.Ignore was first added in Go 1.5.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-04-25 17:50:12 -04:00
Sebastiaan van Stijn
61656464d8
Merge pull request #45315 from thaJeztah/deprecate_oom_score_adjust
daemon: deprecate --oom-score-adjust for the daemon
2023-04-14 00:06:58 +02:00
Sebastiaan van Stijn
03fc3aad12
daemon: touch-up flag description for --seccomp-profile
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-13 13:39:20 +02:00
Sebastiaan van Stijn
5a922dc162
daemon: deprecate --oom-score-adjust for the daemon
The `oom-score-adjust` option was added in a894aec8d8,
to prevent the daemon from being OOM-killed before other processes. This
option was mostly added as a "convenience", as running the daemon as a
systemd unit was not yet common.

Having the daemon set its own limits is not best-practice, and something
better handled by the process-manager starting the daemon.

Commit cf7a5be0f2 fixed this option to allow
disabling it, and 2b8e68ef06 removed the default
score adjust.

This patch deprecates the option altogether, recommending users to set these
limits through the process manager used, such as the "OOMScoreAdjust" option
in systemd units.

With this patch:

    dockerd --oom-score-adjust=-500 --validate
    Flag --oom-score-adjust has been deprecated, and will be removed in the next release.
    configuration OK

    echo '{"oom-score-adjust":-500}' > /etc/docker/daemon.json
    dockerd
    INFO[2023-04-12T21:34:51.133389627Z] Starting up
    INFO[2023-04-12T21:34:51.135607544Z] containerd not running, starting managed containerd
    WARN[2023-04-12T21:34:51.135629086Z] DEPRECATED: The "oom-score-adjust" config parameter and the dockerd "--oom-score-adjust" option will be removed in the next release.

    docker info
    Client:
      Context:    default
      Debug Mode: false
    ...
    DEPRECATED: The "oom-score-adjust" config parameter and the dockerd "--oom-score-adjust" option will be removed in the next release

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-13 00:02:39 +02:00
Sebastiaan van Stijn
a5d46a15f5
split GetRepository from ImageService
The GetRepository method interacts directly with the registry, and does
not depend on the snapshotter, but is used for two purposes;

For the GET /distribution/{name:.*}/json route;
dd3b71d17c/api/server/router/distribution/backend.go (L11-L15)

And to satisfy the "executor.ImageBackend" interface as used by Swarm;
58c027ac8b/daemon/cluster/executor/backend.go (L77)

This patch removes the method from the ImageService interface, and instead
implements it through an composite struct that satisfies both interfaces,
and an ImageBackend() method is added to the daemon.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

remove GetRepository from ImageService

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-09 12:07:57 +02:00
Brian Goff
c7ccc68b15 Silence GRPC logs unless our log level is debug
GRPC is logging a *lot* of garbage at info level.
This configures the GRPC logger such that it is only giving us logs when
at debug level and also adds a log field indicating where the logs are
coming from.

containerd is still currently spewing these same log messages and needs
a separate update.

Without this change `docker build` is extremely noisy in the daemon
logs.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2023-03-21 21:01:10 +00:00
Cory Snider
3991faf464 Move filtered registry search out of image service
SearchRegistryForImages does not make sense as part of the image
service interface. The implementation just wraps the search API of the
registry service to filter the results client-side. It has nothing to do
with local image storage, and the implementation of search does not need
to change when changing which backend (graph driver vs. containerd
snapshotter) is used for local image storage.

Filtering of the search results is an implementation detail: the
consumer of the results does not care which actor does the filtering so
long as the results are filtered as requested. Move filtering into the
exported API of the registry service to hide the implementation details.
Only one thing---the registry service implementation---would need to
change in order to support server-side filtering of search results if
Docker Hub or other registry servers were to add support for it to their
APIs.

Use a fake registry server in the search unit tests to avoid having to
mock out the registry API client.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-03-10 18:36:33 -05:00
Brian Goff
37a9d6aabe Fix listener parsering regression when no addr set
5008409b5c introduced the usage of
`strings.Cut` to help parse listener addresses.
Part of that also made it error out if no addr is specified after the
protocol spec (e.g. `tcp://`).

Before the change a proto spec without an address just used the default
address for that proto.
e.g. `tcp://` would be `tcp://127.0.0.1:2375`, `unix://` would be
`unix:///var/run/docker.sock`.
Critically, socket activation (`fd://`) never has an address.

This change brings back the old behavior but keeps the usage of
`strings.Cut`.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2023-03-09 23:53:05 +00:00
Akihiro Suda
e807ae4f2e
vendor: github.com/containerd/cgroups/v3 v3.0.1
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2023-03-08 20:15:17 +09:00
Djordje Lukic
bba77163ff
c8d: Make build and buildx work
- Only use the image exporter in build if we don't use containerd
  Without this "docker build" fails with:

    Error response from daemon: exporter "image" could not be found

- let buildx know we support containerd snapshotter
- Pass the current snapshotter to the buildkit worker

  If buildkit uses a different snapshotter we can't list the images any
  more because we can't find the snapshot.

builder/builder-next: make ContainerdWorker a minimal wrapper

Note that this makes "Worker" a public field, so technically one could
overwrite it.

builder-next: reenable runc executor

Currently, without special CNI config the builder would
only create host network containers that is a security issue.

Using runc directly instead of shim is faster as well
as builder doesn’t need anything from shim. The overhead
of setting up network sandbox is much slower of course.

builder/builder-next: simplify options handling

Trying to simplify the logic;

- Use an early return if multiple outputs are provided
- Only construct the list of tags if we're using an image (or moby) exporter
- Combine some logic for snapshotter and non-snapshotter handling

Create a constant for the moby exporter

Pass a context when creating a router

The context has a 10 seconds timeout which should be more than enough to
get the answer from containerd.

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
Co-authored-by: Sebastiaan van Stijn <github@gone.nl>
Co-authored-by: Tonis Tiigi <tonistiigi@gmail.com>
Co-authored-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Co-authored-by: Paweł Gronowski <pawel.gronowski@docker.com>
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-03-07 10:01:47 +01:00
Sebastiaan van Stijn
2323f9deda
Merge pull request #43197 from dajudge/default-bridge-mtu
Introduce config option for default generic network options of newly created networks
2023-03-02 20:21:47 +01:00
Brian Goff
a026f3be4b
Merge pull request #45091 from corhere/remove-authz-middleware-from-config
daemon/config: remove AuthzMiddleware field
2023-03-02 17:51:06 +00:00
Cory Snider
fb5df9722b
Merge pull request #45088 from corhere/make-apiserver-less-weird
api/server: delete Wait method
2023-03-01 10:14:09 -05:00
Cory Snider
a9e7360775 daemon/config: remove AuthzMiddleware field
The authorization.Middleware contains a sync.Mutex field, making it
non-copyable. Remove one of the barriers to allowing deep copies of
config.Config values.

Inject the middleware into Daemon as a constructor argument instead.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-03-01 09:43:39 -05:00
Cory Snider
be39be87f6 api/server: delete Wait method
It's surprising that the method to begin serving requests is named Wait.
And it is unidiomatic: it is a synchronous call, but it sends its return
value to the channel passed in as an argument instead of just returning
the value. And ultimately it is just a trivial wrapper around serveAPI.
Export the ServeAPI method instead so callers can decide how to call and
synchronize around it.

Call ServeAPI synchronously on the main goroutine in cmd/dockerd. The
goroutine and channel which the Wait() API demanded are superfluous
after all. The notifyReady() call was always concurrent and asynchronous
with respect to serving the API (its implementation spawns a goroutine)
so it makes no difference whether it is called before ServeAPI() or
after `go ServeAPI()`.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-03-01 09:03:34 -05:00
Alex Stockinger
91c2b12205 Make default options for newly created networks configurable
Signed-off-by: Alex Stockinger <alex@atomicjar.com>
Co-authored-by: Sergei Egorov <bsideup@gmail.com>
Co-authored-by: Cory Snider <corhere@gmail.com>
2023-03-01 07:58:26 +01:00
Cory Snider
0a6a726d26 api/server: drop unused Config struct
The Server.cfg field is never referenced by any code in package
"./api/server". "./api/server".Config struct values are used by
DaemonCli code, but only to pass around configuration copied out of the
daemon config within the "./cmd/dockerd" package. Delete the
"./api/server".Config struct definition and refactor the "./cmd/dockerd"
package to pull configuration directly from cli.Config.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-02-28 13:49:53 -05:00
Cory Snider
0867d3173c cmd/dockerd: use default SIGQUIT behaviour
dockerd handles SIGQUIT by dumping all goroutine stacks to standard
error and exiting. In contrast, the Go runtime's default SIGQUIT
behaviour... dumps all goroutine stacks to standard error and exits.
The default SIGQUIT behaviour is implemented directly in the runtime's
signal handler, and so is both more robust to bugs in the Go runtime and
does not perturb the state of the process to anywhere near same degree
as dumping goroutine stacks from a user goroutine. The only notable
difference from a user's perspective is that the process exits with
status 2 instead of 128+SIGQUIT.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-01-10 17:11:54 -05:00
Jan Garcia
6ab12ec8f4 rootless: move ./rootless to ./pkg/rootless
Signed-off-by: Jan Garcia <github-public@n-garcia.com>
2023-01-09 16:26:06 +01:00
Sebastiaan van Stijn
12df6024db
cmd/docker-proxy: add "-v / --version" flag
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-27 22:26:57 +01:00
Sebastiaan van Stijn
ab78900e29
Merge pull request #44649 from thaJeztah/config_cleanup
daemon/config: rename CommonTLSOptions, and remove RootDeprecated field
2022-12-27 13:06:16 +01:00
Sebastiaan van Stijn
5008409b5c
cmd/dockerd: use strings.Cut()
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-21 11:09:01 +01:00
Sebastiaan van Stijn
e19dff9285
daemon/config: remove deprecated RootDeprecated field
Since b58de39ca7, this option was now only used
to produce a fatal error when starting the daemon. That change is in the 23.0
release, so we can remove it from the master branch.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-15 10:44:32 +01:00
Sebastiaan van Stijn
32b299f63e
daemon/config: rename CommonTLSOptions to TLSOptions
This type was added in 677a6b3506, and named
"common", because at the time, the "docker" and "dockerd" (daemon) code
were still in the same repository, and shared this type. Renaming it, now
that's no longer the case.

As there are no external consumers of this type, I'm not adding an alias.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-15 10:44:24 +01:00
Sebastiaan van Stijn
ef7ab7bde8
cli: remove cli/config package, integrate into cmd/dockerd
The configDir (and "DOCKER_CONFIG" environment variable) is now only used
for the default location for TLS certificates to secure the daemon API.
It is a leftover from when the "docker" and "dockerd" CLI shared the same
binary, allowing the DOCKER_CONFIG environment variable to set the location
for certificates to be used by both.

This patch merges it into cmd/dockerd, which is where it was used.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-14 12:52:40 +01:00
Sebastiaan van Stijn
6a90113e68
cli: move cobra customizations into cmd/dockerd
The cli package is a leftover from when the "docker" and "dockerd" cli
were both maintained in this repository; the only consumer of this is
now the dockerd CLI, so we can move this code (it should not be imported
by anyone).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-14 11:37:51 +01:00
Sebastiaan van Stijn
8feeaecb84
use ad-hoc libtrust key
This is only used for tests, and the key is not verified anymore, so
instead of creating a key and storing it, we can just use an ad-hoc
one.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-28 20:29:19 +01:00
Sebastiaan van Stijn
5cdd6ab7cd
daemon/config: remove TrustKeyPath, and local utilities
Turned out that the loadOrCreateTrustKey() utility was doing exactly the
same as libtrust.LoadOrCreateTrustKey(), so making it a thin wrapped. I kept
the tests to verify the behavior, but we could remove them as we only need this
for our integration tests.

The storage location for the generated key was changed (again as we only need
this for some integration tests), so we can remove the TrustKeyPath from the
config.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-28 20:29:19 +01:00
Sebastiaan van Stijn
14da1bfe40
cmd/dockerd: adjust error message for pidfile changes
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-04 01:50:26 +01:00
Sebastiaan van Stijn
7d3e1ad943
pkg/pidfile: Write(): don't automatically create parent directories
While this was convenient for our use, it's somewhat unexpected for a function
that writes a file to also create all parent directories; even more because
this function may be executed as root.

This patch makes the package more "safe" to use as a generic package by removing
this functionality, and leaving it up to the caller to create parent directories,
if needed.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-04 01:50:26 +01:00
Sebastiaan van Stijn
81945da0ac
pkg/pidfile: Write(): take pid as argument
This allows it to be used for processes other than the daemon itself.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-04 01:50:26 +01:00
Cory Snider
2006d9f7d1
cmd/dockerd: Rewrite shutdownDaemon to use context timeout
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2022-11-03 14:31:03 +01:00
Nicolas De Loof
def549c8f6
imageservice: Add context to various methods
Co-authored-by: Paweł Gronowski <pawel.gronowski@docker.com>
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2022-11-03 12:22:40 +01:00
Luis Henrique Mulinari
6c0aa5b00a
Fix the max-concurrent-downloads and max-concurrent-uploads configs documentation
This fix tries to address issues raised in #44346.
The max-concurrent-downloads and max-concurrent-uploads limits are applied for the whole engine and not for each pull/push command.

Signed-off-by: Luis Henrique Mulinari <luis.mulinari@gmail.com>
2022-10-26 11:10:00 +01:00
Sebastiaan van Stijn
43d6eb7173
pkg/pidfile: remove PIDFile type, rename New() to Write()
This type felt really redundant; `pidfile.New()` takes the path of the file to
create as an argument, so this is already known. The only thing the PIDFile
type provided was a `Remove()` method, which was just calling `os.Remove()` on
the path of the file.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-15 16:40:16 +02:00
Tianon Gravi
af10b35ba9
Merge pull request #44287 from thaJeztah/use_SetRecoveryActions
cmd/dockerd: use x/sys Service.SetRecoveryActions(), and update restart delay
2022-10-14 18:18:47 +00:00
Tianon Gravi
96155f626d
Merge pull request #44286 from thaJeztah/use_SetStdHandle
cmd/dockerd: use golang.org/x/sys/windows.SetStdHandle()
2022-10-14 18:15:52 +00:00
Sebastiaan van Stijn
624daf8d9e
Change restart delay for Windows service to 15s
Previously we waited for 60 seconds after the service faults to restart
it. However, there isn't much benefit to waiting this long. We expect
15 seconds to be a more reasonable delay.

Co-Authored-by: Kevin Parsons <kevpar@microsoft.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-12 09:52:51 +02:00
Sebastiaan van Stijn
3c585e6567
cmd/dockerd: use golang.org/x/sys Service.SetRecoveryActions()
This is the equivalent of the local implementation.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-12 09:52:10 +02:00
Sebastiaan van Stijn
6176ab5901
cmd/dockerd: use golang.org/x/sys/windows.SetStdHandle()
golang.org/x/sys/windows now implements this, so we can use that
instead of a local implementation.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-12 01:50:44 +02:00
Sebastiaan van Stijn
ffcddc908e
cmd/dockerd: replace deprecated windows.IsAnInteractiveSession()
The `IsAnInteractiveSession` was deprecated, and `IsWindowsService` is marked
as the recommended replacement.

For details, see 280f808b4a

> CL 244958 includes isWindowsService function that determines if a
> process is running as a service. The code of the function is based on
> public .Net implementation.
>
> IsAnInteractiveSession function implements similar functionality, but
> is based on an old Stackoverflow post., which is not as authoritative
> as code written by Microsoft for their official product.
>
> This change copies CL 244958 isWindowsService function into svc package
> and makes it public. The intention is that future users will prefer
> IsWindowsService to IsAnInteractiveSession.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-11 22:54:40 +02:00
Sebastiaan van Stijn
f63dea4337
cmd/dockerd: fix empty-lines (revive)
cmd/dockerd/trap/trap_linux_test.go:29:29: empty-lines: extra empty line at the end of a block (revive)
    cmd/dockerd/daemon.go:327:35: empty-lines: extra empty line at the start of a block (revive)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-28 01:58:52 +02:00
Sebastiaan van Stijn
55fd77f724
set ReadHeaderTimeout to address G112: Potential Slowloris Attack (gosec)
After discussing in the maintainers meeting, we concluded that Slowloris attacks
are not a real risk other than potentially having some additional goroutines
lingering around, so setting a long timeout to satisfy the linter, and to at
least have "some" timeout.

    libnetwork/diagnostic/server.go:96:10: G112: Potential Slowloris Attack because ReadHeaderTimeout is not configured in the http.Server (gosec)
        srv := &http.Server{
            Addr:    net.JoinHostPort(ip, strconv.Itoa(port)),
            Handler: s,
        }
    api/server/server.go:60:10: G112: Potential Slowloris Attack because ReadHeaderTimeout is not configured in the http.Server (gosec)
                srv: &http.Server{
                    Addr: addr,
                },
    daemon/metrics_unix.go:34:13: G114: Use of net/http serve function that has no support for setting timeouts (gosec)
            if err := http.Serve(l, mux); err != nil && !strings.Contains(err.Error(), "use of closed network connection") {
                      ^
    cmd/dockerd/metrics.go:27:13: G114: Use of net/http serve function that has no support for setting timeouts (gosec)
            if err := http.Serve(l, mux); err != nil && !strings.Contains(err.Error(), "use of closed network connection") {
                      ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-22 12:13:28 +02:00
Sebastiaan van Stijn
61e5cf083a
Merge pull request #43971 from thaJeztah/daemon_fix_hosts_validation_step1c3
daemon: use initialized config struct in more tests
2022-08-18 21:30:07 +02:00
Sebastiaan van Stijn
b58de39ca7
daemon: complete the "--graph" / "-g" deprecation
The `-g` / `--graph` options were soft deprecated in favor of `--data-root` in
261ef1fa27 (v17.05.0) and at the time considered
to not be removed. However, with the move towards containerd snapshotters, having
these options around adds additional complexity to handle fallbacks for deprecated
(and hidden) flags, so completing the deprecation.

With this patch:

    dockerd --graph=/var/lib/docker --validate
    Flag --graph has been deprecated, Use --data-root instead
    unable to configure the Docker daemon with file /etc/docker/daemon.json: merged configuration validation from file and command line flags failed: the "graph" config file option is deprecated; use "data-root" instead

    mkdir -p /etc/docker
    echo '{"graph":"/var/lib/docker"}' > /etc/docker/daemon.json

    dockerd --validate
    unable to configure the Docker daemon with file /etc/docker/daemon.json: merged configuration validation from file and command line flags failed: the "graph" config file option is deprecated; use "data-root" instead

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-18 14:57:16 +02:00
Sebastiaan van Stijn
2d7d81bc46
cmd/dockerd: use initialized config in tests
Makes sure that tests use a config struct that's more representative
to how it's used in the code.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-17 14:07:03 +02:00
Sebastiaan van Stijn
b28e66cf4f
daemon/config: New(): initialize config with platform-specific defaults
This centralizes more defaults, to be part of the config struct that's
created, instead of interweaving the defaults with other code in various
places.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-17 08:54:32 +02:00
Tianon Gravi
3a7fda78a2
Merge pull request #43942 from thaJeztah/rundaemon_windows_nit
cmd/dockerd: fix some minor issues in Windows implementation
2022-08-11 08:30:44 -07:00
Sebastiaan van Stijn
226e07144e
cmd/dockerd: initContainerd() use early return
- return early if we're expecting a system-containerd
- rename `initContainerD` to `initContainerd` ':)
- remove .Config to reduce verbosity

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-11 14:11:10 +02:00
Sebastiaan van Stijn
6560e0b136
cmd/dockerd: initContainerD(): clean-up some logs
Change the log-level for messages about starting the managed containerd instance
to be the same as for the main API. And remove a redundant debug-log.

With this patch:

    dockerd
    INFO[2022-08-11T11:46:32.573299176Z] Starting up
    INFO[2022-08-11T11:46:32.574304409Z] containerd not running, starting managed containerd
    INFO[2022-08-11T11:46:32.575289181Z] started new containerd process                address=/var/run/docker/containerd/containerd.sock module=libcontainerd pid=5370
cmd/dockerd: initContainerD(): clean-up some logs

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-11 14:11:08 +02:00
Sebastiaan van Stijn
b6b0b0a05f
libcontainerd/supervisor: don't write log-level to config file
the `--log-level` flag overrides whatever is in the containerd configuration file;
f033f6ff85/cmd/containerd/command/main.go (L339-L352)

Given that we set that flag when we start the containerd binary, there is no need
to write it both to the generated config-file and pass it as flag.

This patch also slightly changes the behavior; as both dockerd and containerd use
"info" as default log-level, don't set the log-level if it's the default.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-11 14:11:06 +02:00
Sebastiaan van Stijn
89ab14a115
libcontainerd/supervisor: make supervisor adjust OOM score for containerd
Containerd, like dockerd has a OOMScore configuration option to adjust its own
OOM score. In dockerd, this option was added when default installations were not
yet running the daemon as a systemd unit, which made it more complicated to set
the score, and adding a daemon option was convenient.

A binary adjusting its own score has been frowned upon, as it's more logical to
make that the responsibility of the process manager _starting_ the daemon, which
is what we did for dockerd in 21578530d7.

There have been discussions on deprecating the daemon flag for dockerd, and
similar discussions have been happening for containerd.

This patch changes how we set the OOM score for the containerd child process,
and to have dockerd (supervisor) set the OOM score, as it's acting as process
manager in this case (performing a role similar to systemd otherwise).

With this patch, the score is still adjusted as usual, but not written to the
containerd configuration file;

    dockerd --oom-score-adjust=-123
    cat /proc/$(pidof containerd)/oom_score_adj
    -123

As a follow-up, we may consider to adjust the containerd OOM score based on the
daemon's own score instead of on the `cli.OOMScoreAdjust` configuration so that
we will also adjust the score in situations where dockerd's OOM score was set
through other ways (systemd or manually adjusting the cgroup). A TODO was added
for this.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-11 14:11:01 +02:00
Sebastiaan van Stijn
37a241768d
cmd/dockerd: fix some minor issues in Windows implementation
- properly use filepath.Join() for Windows paths
- use getDaemonConfDir() to get the path for storing daemon.json
- return error instead of logrus.Fatal(). The logrus.Fatal() was a left-over
  from when Cobra was not used, and appears to have been missed in commit
  fb83394714, which did the conversion to Cobra.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-09 22:50:41 +02:00
Sebastiaan van Stijn
051e604adc
libcontainerd/supervisor: simplify logic for disabling CRI plugin
The existing implementation used a `nil` value for the CRI plugin's configuration
to indicate that the plugin had to be disabled. Effectively, the `Plugins` value
was only used as an intermediate step, only to be removed later on, and to instead
add the given plugin to `DisabledPlugins` in the containerd configuration.

This patch removes the intermediate step; as a result we also don't need to mask
the containerd `Plugins` field, which was added to allow serializing the toml.

A code comment was added as well to explain why we're (currently) disabling the
CRI plugin by default, which may help future visitors of the code to determin
if that default is still needed.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-03 10:39:20 +02:00
Sebastiaan van Stijn
ba2ff69894
libcontainerd: switch generated containerd.toml to v2 (v1 is deprecated)
Before this patch:

    INFO[2022-07-27T14:30:06.188762628Z] Starting up
    INFO[2022-07-27T14:30:06.190750725Z] libcontainerd: started new containerd process  pid=2028
    ...
    WARN[0000] containerd config version `1` has been deprecated and will be removed in containerd v2.0, please switch to version `2`, see https://github.com/containerd/containerd/blob/main/docs/PLUGINS.md#version-header
    INFO[2022-07-27T14:30:06.220024286Z] starting containerd                           revision=10c12954828e7c7c9b6e0ea9b0c02b01407d3ae1 version=v1.6.6

With this patch:

    INFO[2022-07-27T14:28:04.025543517Z] Starting up
    INFO[2022-07-27T14:28:04.027447105Z] libcontainerd: started new containerd process  pid=1377
    ...
    INFO[2022-07-27T14:28:04.054483270Z] starting containerd                           revision=10c12954828e7c7c9b6e0ea9b0c02b01407d3ae1 version=v1.6.6

And the generated /var/run/docker/containerd/containerd.toml:

```toml
disabled_plugins = ["io.containerd.grpc.v1.cri"]
imports = []
oom_score = 0
plugin_dir = ""
required_plugins = []
root = "/var/lib/docker/containerd/daemon"
state = "/var/run/docker/containerd/daemon"
temp = ""
version = 2

[cgroup]
  path = ""

[debug]
  address = "/var/run/docker/containerd/containerd-debug.sock"
  format = ""
  gid = 0
  level = "debug"
  uid = 0

[grpc]
  address = "/var/run/docker/containerd/containerd.sock"
  gid = 0
  max_recv_message_size = 16777216
  max_send_message_size = 16777216
  tcp_address = ""
  tcp_tls_ca = ""
  tcp_tls_cert = ""
  tcp_tls_key = ""
  uid = 0

[metrics]
  address = ""
  grpc_histogram = false

[plugins]

[proxy_plugins]

[stream_processors]

[timeouts]

[ttrpc]
  address = ""
  gid = 0
  uid = 0
```

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-27 16:33:00 +02:00
Djordje Lukic
7d74269c0d
Create the containerd image service
Initial pull/ls works
Build is deactivated if the feature is active

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-14 16:46:11 +02:00
Sebastiaan van Stijn
68bf777ece
daemon: remove deprecated fields and flags for cluster-advertise
Removes:

- CommonConfig.ClusterStore
- CommonConfig.ClusterOpts
- CommonConfig.ClusterAdvertise
- The deprecated --cluster-advertise, --cluster-store, and
  --cluster-store-opt flags

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-12 15:39:20 +02:00
Sebastiaan van Stijn
52c1a2fae8
gofmt GoDoc comments with go1.19
Older versions of Go don't format comments, so committing this as
a separate commit, so that we can already make these changes before
we upgrade to Go 1.19.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-08 19:56:23 +02:00
Sebastiaan van Stijn
f73aadb230
daemon/config: New(): set more defaults
Set the defaults when constructing the config, instead of setting them
indirectly through the command-line flags.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-06-29 19:59:18 +02:00
Sebastiaan van Stijn
a0d0db126c
daemon/config: set default MTU when initializing config
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-06-29 19:59:16 +02:00
Sebastiaan van Stijn
fce7ebdaa5
cmd/dockerd: loadDaemonCliConfig() safeguard for unparsed flags
This function depends on flags having been parsed before it's used;
add a safety-net in case this function would be called before that.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-06-29 19:55:06 +02:00
Sebastiaan van Stijn
0281f50efa
cmd/dockerd: move setting defaults to loadDaemonCliConfig()
Move changes to follow, but this moves the code to a more logical place.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-06-29 19:55:05 +02:00
Sebastiaan van Stijn
717a503590
api: add Hosts to API Config
This makes the API configuration more self-contained.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-06-29 19:55:03 +02:00
Sebastiaan van Stijn
fee8a6a5c4
cmd/dockerd: make newAPIServerConfig() more idiomatic
Construct the TLSConfig if needed, before constructing and returning the whole config.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-06-29 19:55:01 +02:00
Sebastiaan van Stijn
b053376741
cmd/dockerd: reserve port before creating sockets/listeners
This prevents creating a socket and touching the filesystem before
trying to use a port that was already in use by a container.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-06-29 19:55:00 +02:00
Sebastiaan van Stijn
986725b466
daemon: improve some errors
use pkg/errors for all errors in this file, and wrap some errors
to provide context.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-06-29 19:54:57 +02:00
Djordje Lukic
b4ffe3a9fb Move the inspect code away from the image service
The LoopkupImage method is only used by the inspect image route and
returns an api/type struct. The depenency to api/types of the
daemon/images package is wrong, the daemon doesn't need to know about
the api types.

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2022-06-22 15:08:55 +02:00
Drew Erny
240a9fcb83
Add Swarm cluster volume supports
Adds code to support Cluster Volumes in Swarm using CSI drivers.

Signed-off-by: Drew Erny <derny@mirantis.com>
2022-05-13 00:55:44 +02:00
Sebastiaan van Stijn
e62382d014
daemon/config: remove uses of pointers for ints
Use the default (0) value to indicate "not set", which simplifies
working with these configuration options, preventing the need to
use intermediate variables etc.

While changing this code, also making some small cleanups, such
as replacing "fmt.Sprintf()" for "strconv" variants.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-29 09:39:34 +02:00
Sebastiaan van Stijn
2fa315c656
Merge pull request #43535 from thaJeztah/daemon_fix_hosts_validation_step1f
cmd/dockerd: unify flags, and remove opt.IPOpt()
2022-04-29 09:20:42 +02:00
Sebastiaan van Stijn
647aede6ad
Merge pull request #43515 from corhere/swarmkit-v2
Bump swarmkit to v2
2022-04-28 20:08:42 +02:00
Sebastiaan van Stijn
ccb75439ff
cmd/dockerd: use flags.IPVar() instead of custom type
spf13/pflag now provides this out of the box, so no need to implement
and use our own value-type for this.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-27 11:14:10 +02:00
Sebastiaan van Stijn
5d352f6a87
cmd/dockerd: remove config_common_unix.go
This file was originally part of the work to support Solaris, and
there's nothing "not common unix" anymmore, so merging the files.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-27 11:14:08 +02:00
Sebastiaan van Stijn
92d9e892b5
cmd/dockerd: combine installUnixConfigFlags with installConfigFlags
installConfigFlags already has separate implementations for Linux and
Windows, so no need to further differentiate.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-27 11:14:07 +02:00
Sebastiaan van Stijn
83f67c63a6
cmd/dockerd: combine installRegistryServiceFlags with installCommonConfigFlags
There's no compelling reason to keep it in a separate function

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-27 11:14:05 +02:00
Sebastiaan van Stijn
1483905024
cmd/dockerd: remove some intermediate variables
use the Config itself to hold the "default" values.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-27 11:14:03 +02:00
Sebastiaan van Stijn
782de17bd0
cmd/dockerd: installCommonConfigFlags() re-group some flags
move deprecated options and some network-related options together

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-27 11:14:01 +02:00
Sebastiaan van Stijn
0ff0b8693e
cmd/dockerd: move storage-driver flag to unix-only file
The installCommonConfigFlags() function is meant for flags that are
supported by all platforms, so removing it from that function.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-27 11:13:58 +02:00
Sebastiaan van Stijn
0603f87fab
cmd/dockerd: validate API configuration as part of --validate
Previously, the API server configuration would be initialized and
validated when starting the API. Because of this, invalid configuration
(e.g. missing or invalid TLS certificates) would not be detected
when using `dockerd --validate`.

This patch moves creation of the validation earlier, so that it's
validated as part of `dockerd --validate`.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-27 00:34:24 +02:00
Sebastiaan van Stijn
e16c3616e2
cmd/dockerd: change newAPIServerConfig() to only receive config.Config
This function took the whole daemon cli as argument but only needed the config

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-27 00:34:22 +02:00
Sebastiaan van Stijn
57c20c1b79
cmd/dockerd: normalize hosts when loading config
Previously, hosts were de-duplicated and normalized when starting
the API server (in `loadListeners()`), which meant that errors could
occur in that step (but not detected when using `dockerd --validate`),
as well as the list of hosts in the config not matching what would
actually be used (i.e., if duplicates were present).

This patch extracts the de-duplicating to a separate function, and
executes it as part of loading the daemon configuration, so that we
can fail early.

Moving this code also showed that some of this validation depended
on `newAPIServerConfig()` modifying the configuration (adding an
empty host if none was set) in order to have the parsing set a
default. This code was moved elsewhere, but a TODO comment added
as this logic is somewhat sketchy.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-27 00:34:20 +02:00
Sebastiaan van Stijn
7b3463f2c5
cmd/dockerd: loadDaemonCliConfig(): minor cleanup
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-27 00:34:19 +02:00
Sebastiaan van Stijn
d9d0683862
cmd/dockerd: un-export config methods, and don't pass flags "twice"
- un-export `daemonOptions.InstallFlags()`; `daemonOptions` itself isn't exported,
  not exported, and `InstallFlags()` isn't matching any interface and only used
  internally.
- un-export `daemonOptions.SetDefaultOptions()` and remove the `flags` argument
  as we were passing `daemonOptions.flags` as argument on a method attached to
  `daemonOptions`, which was somewhat backwards. While at it, also removing an
  intermediate variable that wasn't needed.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-27 00:34:17 +02:00
Sebastiaan van Stijn
390c7d6871
daemon/config: Validate(): validate log-level
Log-level validation was previously performed when configuring the daemon-logs;
this moves the validation to config.Validate() so that we can catch invalid
settings when running dockerd --validate.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-27 00:34:14 +02:00
Sebastiaan van Stijn
104ac2ca49
Merge pull request #43521 from thaJeztah/rootless_fixups
minor rootless fix-ups
2022-04-26 11:00:28 +02:00
Sebastiaan van Stijn
63ea9eb594
cmd/dockerd: don't call registry.SetCertsDir() twice
This was introduced in 85572cac14, where I
probably forgot to remove this code from an earlier iteration (I decided
that having an explicit `configureCertsDir()` function call for this would
make it more transparent that we're re-configuring a default).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-25 14:09:25 +02:00
Sebastiaan van Stijn
cd054983ff
api/server: remove "Logging" from config
The Logging boolean was unconditionally set to true and ignored in all locations,
except for enabling the debugging middleware, which was also gated by the active
logrus logging level.

While it could make sense to have a Loglevel option configured on the API server,
we don't have this currently, and to make that actually useful, that config would
need to be tollerated by all locations that produce logs (which isn't the case
either).

Looking at the history of this option; a boolean to disable logging was originally
added in commit c423a790d6, which hard-coded it to
"disabled" in a test, and "enabled" for the API server outside of tests (before
that commit, logging was always enabled).

02ddaad5d9 and 5c42b2b512
changed the hard-coded values to be configurable through a `Logging` env-var (env-
vars were used _internally_ at the time to pass on options), which later became
a configuration struct in a0bf80fe03.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-23 23:38:30 +02:00
Cory Snider
1c129103b4 Bump swarmkit to v2
Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-04-21 17:33:07 -04:00
Sebastiaan van Stijn
79becebd3d
Merge pull request #43449 from thaJeztah/daemon_config_cleanup
daemon: move more consts for default configuration values to daemon/config
2022-04-20 17:01:19 +02:00
Sebastiaan van Stijn
83969fa3dd
daemon: move DefaultShutdownTimeout to daemon/config
Unifying defaults to the daemon/config package

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-17 13:11:03 +02:00
Sebastiaan van Stijn
690a6fddf9
daemon: move default namespaces to daemon/config
Keeping the defaults in a single location, which also reduces
the list of imports needed.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-17 13:10:57 +02:00
CrazyMax
1efda78f2b
use go-winres for windows build and cleanup autogen and winresources
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-04-14 19:52:36 +02:00
CrazyMax
fd2143e0b0
use go-winres for cross to create Windows resources
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-04-14 19:52:35 +02:00
Sebastiaan van Stijn
846af8e413
cmd/dockerd: update --ip6tables description to include "experimental"
This feature requires experimental mode to be enabled, so mentioning that
in the flag description.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-04 11:54:01 +02:00
Sebastiaan van Stijn
85572cac14
registry: remove dependency on rootlesskit, add SetCertsDir()
The registry package contained code to automatically set the CertsDir() path,
based on wether or not the daemon was running in rootlessmode. In doing so,
it made use of the `pkg/rootless.RunningWithRootlessKit()` utility.

A recent change in de6732a403 added additional
functionality in the `pkg/rootless` package, introducing a dependency on
`github.com/rootless-containers/rootlesskit`. Unfortunately, the extra
dependency also made its way into the docker cli, which also uses the
registry package.

This patch introduces a new `SetCertsDir()` function, which allows
the default certs-directory to be overridden, and updates the daemon
to configure this location during startup.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-25 16:21:45 +01:00
Sebastiaan van Stijn
1d5405cb6f
Merge pull request #43250 from tianon/rm-quotedstring
Remove opts.QuotedString implementation
2022-03-16 16:55:39 +01:00
Sebastiaan van Stijn
2c97295ad8
daemon: remove v1 shim configuration for containerd
This removes the plugin section from the containerd configuration file
(`/var/run/docker/containerd/containerd.toml`) that is generated when
starting containerd as child process;

```toml
[plugins]
  [plugins.linux]
    shim = "containerd-shim"
    runtime = "runc"
    runtime_root = "/var/lib/docker/runc"
    no_shim = false
    shim_debug = true
```

This configuration doesn't appear to be used since commit:
0b14c2b67a, which switched the default runtime
to to io.containerd.runc.v2.

Note that containerd itself uses `containerd-shim` and `runc` as default
for `shim` and `runtime` v1, so omitting that configuration doesn't seem
to make a difference.

I'm slightly confused if any of the other options in this configuration were
actually used: for example, even though `runtime_root` was configured to be
`/var/lib/docker/runc`, when starting a container with that coniguration set
on docker 19.03, `/var/lib/docker/runc` doesn't appear to exist:

```console
$ docker ps
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS               NAMES
098baa4cb0e7        nginx:alpine        "/docker-entrypoint.…"   59 minutes ago      Up 59 minutes       80/tcp              foo

$ ls /var/lib/docker/runc
ls: /var/lib/docker/runc: No such file or directory

$ ps auxf
PID   USER     TIME  COMMAND
    1 root      0:00 sh
   16 root      0:11 dockerd --debug
   26 root      0:09 containerd --config /var/run/docker/containerd/containerd.toml --log-level debug
  234 root      0:00 containerd-shim -namespace moby -workdir /var/lib/docker/containerd/daemon/io.containerd.runtime.v1.linux/moby/09
  251 root      0:00 nginx: master process nginx -g daemon off;
  304 101       0:00 nginx: worker process
...

```

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-07 19:31:24 +01:00
Sebastiaan van Stijn
5263bea70f
daemon: move check for CPU-realtime daemon options
Perform the validation when the daemon starts instead of performing these
validations for each individual container, so that we can fail early.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-03 19:50:27 +01:00