Commit graph

7160 commits

Author SHA1 Message Date
Sebastiaan van Stijn
ce550fa9c2
Merge pull request #43968 from thaJeztah/implement_GetImageOpts
introduce GetImageOpts to manage image inspect data in backend
2022-08-24 19:07:30 +02:00
Sebastiaan van Stijn
500c026818
Merge pull request #43983 from thaJeztah/daemon_configurable_snapshotter
daemon: make the snapshotter configurable
2022-08-24 14:02:27 +02:00
Cory Snider
4b84a33217
daemon: kill exec process on ctx cancel
Terminating the exec process when the context is canceled has been
broken since Docker v17.11 so nobody has been able to depend upon that
behaviour in five years of releases. We are thus free from backwards-
compatibility constraints.

Co-authored-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Co-authored-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Signed-off-by: Cory Snider <csnider@mirantis.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-23 15:35:30 +02:00
Sebastiaan van Stijn
774e25de13
info: add driver-type
With this patch:

    mkdir -p /etc/docker/
    echo '{"features":{"containerd-snapshotter":true}}' > /etc/docker/daemon.json
    dockerd

    docker info
    ...
    Storage Driver: overlayfs
     driver-type: io.containerd.snapshotter.v1
    Logging Driver: json-file

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-23 10:09:37 +02:00
Djordje Lukic
d8d990f2e3
daemon: make the snapshotter configurable
Treat (storage/graph)Driver as snapshotter

Also moved some layerStore related initialization to the non-c8d case
because otherwise they get treated as a graphdriver plugins.

Co-authored-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-22 18:57:42 +02:00
Tianon Gravi
464882e398
Merge pull request #43982 from thaJeztah/daemon_remove_graphdriver_field
ImageService: rename GraphDriverName to StorageDriver, remove daemon.graphdriver
2022-08-19 13:08:36 -07:00
Sebastiaan van Stijn
d082bbcc05
Merge pull request #43978 from cpuguy83/default_runtime_shim
Allow containerd shim refs in default-runtime
2022-08-18 22:59:57 +02:00
Sebastiaan van Stijn
61e5cf083a
Merge pull request #43971 from thaJeztah/daemon_fix_hosts_validation_step1c3
daemon: use initialized config struct in more tests
2022-08-18 21:30:07 +02:00
Brian Goff
e6ee27a541 Allow containerd shim refs in default-runtime
Since runtimes can now just be containerd shims, we need to check if the
reference is possibly a containerd shim.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2022-08-18 18:41:03 +00:00
Sebastiaan van Stijn
b58de39ca7
daemon: complete the "--graph" / "-g" deprecation
The `-g` / `--graph` options were soft deprecated in favor of `--data-root` in
261ef1fa27 (v17.05.0) and at the time considered
to not be removed. However, with the move towards containerd snapshotters, having
these options around adds additional complexity to handle fallbacks for deprecated
(and hidden) flags, so completing the deprecation.

With this patch:

    dockerd --graph=/var/lib/docker --validate
    Flag --graph has been deprecated, Use --data-root instead
    unable to configure the Docker daemon with file /etc/docker/daemon.json: merged configuration validation from file and command line flags failed: the "graph" config file option is deprecated; use "data-root" instead

    mkdir -p /etc/docker
    echo '{"graph":"/var/lib/docker"}' > /etc/docker/daemon.json

    dockerd --validate
    unable to configure the Docker daemon with file /etc/docker/daemon.json: merged configuration validation from file and command line flags failed: the "graph" config file option is deprecated; use "data-root" instead

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-18 14:57:16 +02:00
Sebastiaan van Stijn
d2276ff3f2
daemon: remove daemon.graphdriver
It was only used as an intermediate variable to store what's returned
by layerstore.DriverName() / ImageService.StorageDriver()

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-18 13:20:48 +02:00
Sebastiaan van Stijn
c6eab4077a
daemon: info: fillDriverInfo() get driver-name from ImageService
Make the ImageService the source of truth for the storage-driver
that's used.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-18 09:44:56 +02:00
Sebastiaan van Stijn
8dd14509d7
ImageService: rename GraphDriverName to StorageDriver
Make the function name more generic, as it's no longer used only
for graphdrivers but also for snapshotters.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-18 09:44:51 +02:00
Brian Goff
031c731c68
Merge pull request #43940 from thaJeztah/remove_legacy_graphdriver_fallback
daemon: remove some legacy fallbacks, and small refactor
2022-08-17 14:00:27 -07:00
Sebastiaan van Stijn
327c8fc52f
daemon/config: use initialized config in more tests
Makes sure that tests use a config struct that's more representative
to how it's used in the code.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-17 14:31:25 +02:00
Sebastiaan van Stijn
b28e66cf4f
daemon/config: New(): initialize config with platform-specific defaults
This centralizes more defaults, to be part of the config struct that's
created, instead of interweaving the defaults with other code in various
places.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-17 08:54:32 +02:00
Sebastiaan van Stijn
9d74c7ab99
daemon: Mount(): use container's driver information for error-message
Use the information stored as part of the container for the error-message,
instead of querying the current storage driver from the daemon.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-16 23:25:25 +02:00
Sebastiaan van Stijn
239d9c5eda
daemon: restore(): remove fallback for legacy containers
The check was accounting for old containers that did not have a storage-driver
set in their config, and was added in 4908d7f81d
for docker v0.7.0-rc6 - nearly 9 Years ago, so very likely nobody is still
depending on this ;-)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-16 23:25:16 +02:00
Sebastiaan van Stijn
6ccda5a041
daemon: restore(): remove platform-check (was used for LCOW)
This was added in 0cba7740d4, as part of
the LCOW implementation. LCOW support has been removed, so we can remove
this check.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-16 23:17:23 +02:00
Nicolas De Loof
9a849cc83a
introduce GetImageOpts to manage image inspect data in backend
Currently only provides the existing "platform" option, but more
options will be added in follow-ups.

Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-16 16:49:46 +02:00
Sebastiaan van Stijn
cff2b5b338
Merge pull request #43912 from thaJeztah/containerd_emptyfs_modern
emptyfs: Use newer image layout
2022-08-16 14:43:01 +02:00
Sebastiaan van Stijn
301fed8a45
Merge pull request #43817 from thaJeztah/pull_auth
containerd integration: containerd/pull: Use authorization
2022-08-16 13:18:29 +02:00
Paweł Gronowski
502d3a1fcb
containerd/load: Load all platforms
To make it possible to load emptyfs which is amd64 only

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2022-08-16 13:00:18 +02:00
Brian Goff
54de892d1e
Merge pull request #43833 from thaJeztah/compute_sharedsize
containerd integration: compute image's shared size
2022-08-15 09:25:13 -07:00
Sebastiaan van Stijn
8925f735a1
daemon/config: fix GoDoc for StockRuntimeName
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-11 18:16:50 +02:00
Sebastiaan van Stijn
9861dd069b
vendor: github.com/moby/swarmkit/v2 v2.0.0-20220721174824-48dd89375d0a
full diff: 6068d1894d...48dd89375d

Finishes off the work to change references to cluster volumes in the API
from using "csi" as the magic word to "cluster". This reflects that the
volumes are "cluster volumes", not "csi volumes".

Notably, there is no change to the plugin definitions being "csinode"
and "csicontroller". This terminology is appropriate with regards to
plugins because it accurates reflects what the plugin is.

Signed-off-by: Drew Erny <derny@mirantis.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-09 14:03:50 +02:00
Sebastiaan van Stijn
7797480eb0
daemon, builder: rename variables that collided with imports
rename some variables that collided with imports or (upcoming)
changes, e.g. `ctx`, which is commonly used for `context.Context`.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-08 13:16:10 +02:00
Paweł Gronowski
5c96365c94
containerd: Unpack loaded images
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-04 11:07:12 +02:00
Nicolas De Loof
50fb999bb1
add image load/save support
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2022-08-04 09:41:19 +02:00
Paweł Gronowski
d4579a983e
containerd/pull: Use authorization
- containerd/pull: Use authorization
- containerd/auth: Check if registry hostname matches

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2022-08-04 01:05:41 +02:00
Brian Goff
7e8df0e2c9
Merge pull request #43820 from thaJeztah/image_delete
containerd integration: Implement ImageDelete for containerd
2022-08-03 15:47:07 -07:00
Tianon Gravi
e60bddcc60
Merge pull request #43885 from thaJeztah/auth_header_refactor
Move AuthConfig to types/registry, and implement utilities for encoding/decoding
2022-08-03 11:31:26 -07:00
Sebastiaan van Stijn
9100e3c087
Merge pull request #43219 from corhere/sdjournal
Rewrite journald LogReader and unit test it
2022-08-02 23:47:35 +02:00
Sebastiaan van Stijn
360d38adb1
daemon/containerd: use types/registry.AuthConfig
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-29 23:05:21 +02:00
Sebastiaan van Stijn
a02abca49a
daemon: use types/registry.AuthConfig
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-29 23:05:18 +02:00
Sebastiaan van Stijn
39f3adc079
daemon/images: use types/registry.AuthConfig
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-29 23:05:17 +02:00
Sebastiaan van Stijn
05eacf92fe
daemon/cluster: use types/registry.AuthConfig
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-29 23:05:16 +02:00
Brian Goff
6e21389917
Merge pull request #43800 from corhere/implicit-runtime-config
daemon: support alternative runtimes MVP
2022-07-29 09:35:22 -07:00
Sebastiaan van Stijn
f34567bf41
Merge pull request #43751 from vvoland/fix-exitcode-wait
state/Wait: Fix race when reading exit status
2022-07-29 16:01:39 +02:00
Djordje Lukic
26c65447df
Implement ImageDelete for containerd
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-28 22:56:34 +02:00
Cory Snider
547da0d575 daemon: support other containerd runtimes (MVP)
Contrary to popular belief, the OCI Runtime specification does not
specify the command-line API for runtimes. Looking at containerd's
architecture from the lens of the OCI Runtime spec, the _shim_ is the
OCI Runtime and runC is "just" an implementation detail of the
io.containerd.runc.v2 runtime. When one configures a non-default runtime
in Docker, what they're really doing is instructing Docker to create
containers using the io.containerd.runc.v2 runtime with a configuration
option telling the runtime that the runC binary is at some non-default
path. Consequently, only OCI runtimes which are compatible with the
io.containerd.runc.v2 shim, such as crun, can be used in this manner.
Other OCI runtimes, including kata-containers v2, come with their own
containerd shim and are not compatible with io.containerd.runc.v2.
As Docker has not historically provided a way to select a non-default
runtime which requires its own shim, runtimes such as kata-containers v2
could not be used with Docker.

Allow other containerd shims to be used with Docker; no daemon
configuration required. If the daemon is instructed to create a
container with a runtime name which does not match any of the configured
or stock runtimes, it passes the name along to containerd verbatim. A
user can start a container with the kata-containers runtime, for
example, simply by calling

    docker run --runtime io.containerd.kata.v2

Runtime names which containerd would interpret as a path to an arbitrary
binary are disallowed. While handy for development and testing it is not
strictly necessary and would allow anyone with Engine API access to
trivially execute any binary on the host as root, so we have decided it
would be safest for our users if it was not allowed.

It is not yet possible to set an alternative containerd shim as the
default runtime; it can only be configured per-container.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-07-27 14:22:49 -04:00
Illo Abdulrahim
31f654a704
Fix file capabilities droping in Dockerfile
doCopyXattrs() never reached due to copyXattrs boolean being false, as
a result file capabilities not being copied.

moved copyXattr() out of doCopyXattrs()

Signed-off-by: Illo Abdulrahim <abdulrahim.illo@nokia.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-27 17:21:02 +02:00
Cory Snider
ef5b279887 logger/journald: implement --follow correctly
Implement --follow entirely correctly for the journald log reader, such
that it exits immediately upon reading back the last log message written
to the journal before the logger was closed. The impossibility of doing
so has been slightly exaggerated.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-07-25 16:41:38 -04:00
Cory Snider
e278d3f185 logger/*: fix reading w/ non-monotonic timestamps
Fix journald and logfile-powered (jsonfile, local) log readers
incorrectly filtering out messages with timestamps < Since which were
preceded by a message with a timestamp >= Since.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-07-25 16:41:38 -04:00
Cory Snider
342b44bf20 logger/journald: rewrite reader w/o cursors
Careful management of the journal read pointer is sufficient to ensure
that no entry is read more than once.

Unit test the journald logger without requiring a running journald by
using the systemd-journal-remote command to write arbitrary entries to
journal files.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-07-25 16:41:38 -04:00
Cory Snider
3e8405aa82 logger/journald: refactor cgo out of business logic
Wrap the libsystemd journal reading functionality in a more idiomatic Go
API and refactor the journald logging driver's ReadLogs implementation
to use the wrapper. Rewrite the parts of the ReadLogs implementation in
Go which were previously implemented in C as part of the cgo preamble.
Separating the business logic from the cgo minutiae should hopefully
make the code more accessible to a wider audience of developers for
reviewing the code and contributing improvements.

The structure of the ReadLogs implementation is retained with few
modifications. Any ignored errors were also ignored before the refactor;
the explicit error return values afforded by the sdjournal wrapper makes
this more obvious.

The package github.com/coreos/go-systemd/v22/sdjournal also provides a
more idiomatic Go wrapper around libsystemd. It is unsuitable for our
needs as it does not expose wrappers for the sd_journal_process and
sd_journal_get_fd functions.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-07-25 16:41:09 -04:00
Cory Snider
7ff4b64319 logger/journald: simplify control flow
Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-07-25 16:37:59 -04:00
Cory Snider
f7fe2c2290 logger/journald: follow the thread-safety rules
Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-07-25 16:37:59 -04:00
Cory Snider
12744335c0 logger/journald: simplify build constraints
Ensure the package can be imported, no matter the build constratints, by
adding an unconstrained doc.go containing a package statement.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-07-25 16:37:59 -04:00
Nicolas De Loof
7bdc0a6291
compute image's shared size
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2022-07-25 12:19:48 +02:00