moby/daemon/runtime_unix_test.go

434 lines
11 KiB
Go
Raw Normal View History

daemon: support other containerd runtimes (MVP) Contrary to popular belief, the OCI Runtime specification does not specify the command-line API for runtimes. Looking at containerd's architecture from the lens of the OCI Runtime spec, the _shim_ is the OCI Runtime and runC is "just" an implementation detail of the io.containerd.runc.v2 runtime. When one configures a non-default runtime in Docker, what they're really doing is instructing Docker to create containers using the io.containerd.runc.v2 runtime with a configuration option telling the runtime that the runC binary is at some non-default path. Consequently, only OCI runtimes which are compatible with the io.containerd.runc.v2 shim, such as crun, can be used in this manner. Other OCI runtimes, including kata-containers v2, come with their own containerd shim and are not compatible with io.containerd.runc.v2. As Docker has not historically provided a way to select a non-default runtime which requires its own shim, runtimes such as kata-containers v2 could not be used with Docker. Allow other containerd shims to be used with Docker; no daemon configuration required. If the daemon is instructed to create a container with a runtime name which does not match any of the configured or stock runtimes, it passes the name along to containerd verbatim. A user can start a container with the kata-containers runtime, for example, simply by calling docker run --runtime io.containerd.kata.v2 Runtime names which containerd would interpret as a path to an arbitrary binary are disallowed. While handy for development and testing it is not strictly necessary and would allow anyone with Engine API access to trivially execute any binary on the host as root, so we have decided it would be safest for our users if it was not allowed. It is not yet possible to set an alternative containerd shim as the default runtime; it can only be configured per-container. Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-07-20 20:12:01 +00:00
//go:build !windows
package daemon
import (
daemon: reload runtimes w/o breaking containers The existing runtimes reload logic went to great lengths to replace the directory containing runtime wrapper scripts as atomically as possible within the limitations of the Linux filesystem ABI. Trouble is, atomically swapping the wrapper scripts directory solves the wrong problem! The runtime configuration is "locked in" when a container is started, including the path to the runC binary. If a container is started with a runtime which requires a daemon-managed wrapper script and then the daemon is reloaded with a config which no longer requires the wrapper script (i.e. some args -> no args, or the runtime is dropped from the config), that container would become unmanageable. Any attempts to stop, exec or otherwise perform lifecycle management operations on the container are likely to fail due to the wrapper script no longer existing at its original path. Atomically swapping the wrapper scripts is also incompatible with the read-copy-update paradigm for reloading configuration. A handler in the daemon could retain a reference to the pre-reload configuration for an indeterminate amount of time after the daemon configuration has been reloaded and updated. It is possible for the daemon to attempt to start a container using a deleted wrapper script if a request to run a container races a reload. Solve the problem of deleting referenced wrapper scripts by ensuring that all wrapper scripts are *immutable* for the lifetime of the daemon process. Any given runtime wrapper script must always exist with the same contents, no matter how many times the daemon config is reloaded, or what changes are made to the config. This is accomplished by using everyone's favourite design pattern: content-addressable storage. Each wrapper script file name is suffixed with the SHA-256 digest of its contents to (probabilistically) guarantee immutability without needing any concurrency control. Stale runtime wrapper scripts are only cleaned up on the next daemon restart. Split the derived runtimes configuration from the user-supplied configuration to have a place to store derived state without mutating the user-supplied configuration or exposing daemon internals in API struct types. Hold the derived state and the user-supplied configuration in a single struct value so that they can be updated as an atomic unit. Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-08-31 20:12:30 +00:00
"io/fs"
daemon: support other containerd runtimes (MVP) Contrary to popular belief, the OCI Runtime specification does not specify the command-line API for runtimes. Looking at containerd's architecture from the lens of the OCI Runtime spec, the _shim_ is the OCI Runtime and runC is "just" an implementation detail of the io.containerd.runc.v2 runtime. When one configures a non-default runtime in Docker, what they're really doing is instructing Docker to create containers using the io.containerd.runc.v2 runtime with a configuration option telling the runtime that the runC binary is at some non-default path. Consequently, only OCI runtimes which are compatible with the io.containerd.runc.v2 shim, such as crun, can be used in this manner. Other OCI runtimes, including kata-containers v2, come with their own containerd shim and are not compatible with io.containerd.runc.v2. As Docker has not historically provided a way to select a non-default runtime which requires its own shim, runtimes such as kata-containers v2 could not be used with Docker. Allow other containerd shims to be used with Docker; no daemon configuration required. If the daemon is instructed to create a container with a runtime name which does not match any of the configured or stock runtimes, it passes the name along to containerd verbatim. A user can start a container with the kata-containers runtime, for example, simply by calling docker run --runtime io.containerd.kata.v2 Runtime names which containerd would interpret as a path to an arbitrary binary are disallowed. While handy for development and testing it is not strictly necessary and would allow anyone with Engine API access to trivially execute any binary on the host as root, so we have decided it would be safest for our users if it was not allowed. It is not yet possible to set an alternative containerd shim as the default runtime; it can only be configured per-container. Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-07-20 20:12:01 +00:00
"os"
daemon: reload runtimes w/o breaking containers The existing runtimes reload logic went to great lengths to replace the directory containing runtime wrapper scripts as atomically as possible within the limitations of the Linux filesystem ABI. Trouble is, atomically swapping the wrapper scripts directory solves the wrong problem! The runtime configuration is "locked in" when a container is started, including the path to the runC binary. If a container is started with a runtime which requires a daemon-managed wrapper script and then the daemon is reloaded with a config which no longer requires the wrapper script (i.e. some args -> no args, or the runtime is dropped from the config), that container would become unmanageable. Any attempts to stop, exec or otherwise perform lifecycle management operations on the container are likely to fail due to the wrapper script no longer existing at its original path. Atomically swapping the wrapper scripts is also incompatible with the read-copy-update paradigm for reloading configuration. A handler in the daemon could retain a reference to the pre-reload configuration for an indeterminate amount of time after the daemon configuration has been reloaded and updated. It is possible for the daemon to attempt to start a container using a deleted wrapper script if a request to run a container races a reload. Solve the problem of deleting referenced wrapper scripts by ensuring that all wrapper scripts are *immutable* for the lifetime of the daemon process. Any given runtime wrapper script must always exist with the same contents, no matter how many times the daemon config is reloaded, or what changes are made to the config. This is accomplished by using everyone's favourite design pattern: content-addressable storage. Each wrapper script file name is suffixed with the SHA-256 digest of its contents to (probabilistically) guarantee immutability without needing any concurrency control. Stale runtime wrapper scripts are only cleaned up on the next daemon restart. Split the derived runtimes configuration from the user-supplied configuration to have a place to store derived state without mutating the user-supplied configuration or exposing daemon internals in API struct types. Hold the derived state and the user-supplied configuration in a single struct value so that they can be updated as an atomic unit. Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-08-31 20:12:30 +00:00
"strings"
daemon: support other containerd runtimes (MVP) Contrary to popular belief, the OCI Runtime specification does not specify the command-line API for runtimes. Looking at containerd's architecture from the lens of the OCI Runtime spec, the _shim_ is the OCI Runtime and runC is "just" an implementation detail of the io.containerd.runc.v2 runtime. When one configures a non-default runtime in Docker, what they're really doing is instructing Docker to create containers using the io.containerd.runc.v2 runtime with a configuration option telling the runtime that the runC binary is at some non-default path. Consequently, only OCI runtimes which are compatible with the io.containerd.runc.v2 shim, such as crun, can be used in this manner. Other OCI runtimes, including kata-containers v2, come with their own containerd shim and are not compatible with io.containerd.runc.v2. As Docker has not historically provided a way to select a non-default runtime which requires its own shim, runtimes such as kata-containers v2 could not be used with Docker. Allow other containerd shims to be used with Docker; no daemon configuration required. If the daemon is instructed to create a container with a runtime name which does not match any of the configured or stock runtimes, it passes the name along to containerd verbatim. A user can start a container with the kata-containers runtime, for example, simply by calling docker run --runtime io.containerd.kata.v2 Runtime names which containerd would interpret as a path to an arbitrary binary are disallowed. While handy for development and testing it is not strictly necessary and would allow anyone with Engine API access to trivially execute any binary on the host as root, so we have decided it would be safest for our users if it was not allowed. It is not yet possible to set an alternative containerd shim as the default runtime; it can only be configured per-container. Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-07-20 20:12:01 +00:00
"testing"
"github.com/containerd/containerd/plugin"
daemon: support other containerd runtimes (MVP) Contrary to popular belief, the OCI Runtime specification does not specify the command-line API for runtimes. Looking at containerd's architecture from the lens of the OCI Runtime spec, the _shim_ is the OCI Runtime and runC is "just" an implementation detail of the io.containerd.runc.v2 runtime. When one configures a non-default runtime in Docker, what they're really doing is instructing Docker to create containers using the io.containerd.runc.v2 runtime with a configuration option telling the runtime that the runC binary is at some non-default path. Consequently, only OCI runtimes which are compatible with the io.containerd.runc.v2 shim, such as crun, can be used in this manner. Other OCI runtimes, including kata-containers v2, come with their own containerd shim and are not compatible with io.containerd.runc.v2. As Docker has not historically provided a way to select a non-default runtime which requires its own shim, runtimes such as kata-containers v2 could not be used with Docker. Allow other containerd shims to be used with Docker; no daemon configuration required. If the daemon is instructed to create a container with a runtime name which does not match any of the configured or stock runtimes, it passes the name along to containerd verbatim. A user can start a container with the kata-containers runtime, for example, simply by calling docker run --runtime io.containerd.kata.v2 Runtime names which containerd would interpret as a path to an arbitrary binary are disallowed. While handy for development and testing it is not strictly necessary and would allow anyone with Engine API access to trivially execute any binary on the host as root, so we have decided it would be safest for our users if it was not allowed. It is not yet possible to set an alternative containerd shim as the default runtime; it can only be configured per-container. Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-07-20 20:12:01 +00:00
v2runcoptions "github.com/containerd/containerd/runtime/v2/runc/options"
daemon: consolidate runtimes config validation The daemon has made a habit of mutating the DefaultRuntime and Runtimes values in the Config struct to merge defaults. This would be fine if it was a part of the regular configuration loading and merging process, as is done with other config options. The trouble is it does so in surprising places, such as in functions with 'verify' or 'validate' in their name. It has been necessary in order to validate that the user has not defined a custom runtime named "runc" which would shadow the built-in runtime of the same name. Other daemon code depends on the runtime named "runc" always being defined in the config, but merging it with the user config at the same time as the other defaults are merged would trip the validation. The root of the issue is that the daemon has used the same config values for both validating the daemon runtime configuration as supplied by the user and for keeping track of which runtimes have been set up by the daemon. Now that a completely separate value is used for the latter purpose, surprising contortions are no longer required to make the validation work as intended. Consolidate the validation of the runtimes config and merging of the built-in runtimes into the daemon.setupRuntimes() function. Set the result of merging the built-in runtimes config and default default runtime on the returned runtimes struct, without back-propagating it onto the config.Config argument. Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-08-31 21:24:22 +00:00
"github.com/imdario/mergo"
daemon: support other containerd runtimes (MVP) Contrary to popular belief, the OCI Runtime specification does not specify the command-line API for runtimes. Looking at containerd's architecture from the lens of the OCI Runtime spec, the _shim_ is the OCI Runtime and runC is "just" an implementation detail of the io.containerd.runc.v2 runtime. When one configures a non-default runtime in Docker, what they're really doing is instructing Docker to create containers using the io.containerd.runc.v2 runtime with a configuration option telling the runtime that the runC binary is at some non-default path. Consequently, only OCI runtimes which are compatible with the io.containerd.runc.v2 shim, such as crun, can be used in this manner. Other OCI runtimes, including kata-containers v2, come with their own containerd shim and are not compatible with io.containerd.runc.v2. As Docker has not historically provided a way to select a non-default runtime which requires its own shim, runtimes such as kata-containers v2 could not be used with Docker. Allow other containerd shims to be used with Docker; no daemon configuration required. If the daemon is instructed to create a container with a runtime name which does not match any of the configured or stock runtimes, it passes the name along to containerd verbatim. A user can start a container with the kata-containers runtime, for example, simply by calling docker run --runtime io.containerd.kata.v2 Runtime names which containerd would interpret as a path to an arbitrary binary are disallowed. While handy for development and testing it is not strictly necessary and would allow anyone with Engine API access to trivially execute any binary on the host as root, so we have decided it would be safest for our users if it was not allowed. It is not yet possible to set an alternative containerd shim as the default runtime; it can only be configured per-container. Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-07-20 20:12:01 +00:00
"gotest.tools/v3/assert"
is "gotest.tools/v3/assert/cmp"
"github.com/docker/docker/api/types"
"github.com/docker/docker/daemon/config"
"github.com/docker/docker/errdefs"
)
daemon: consolidate runtimes config validation The daemon has made a habit of mutating the DefaultRuntime and Runtimes values in the Config struct to merge defaults. This would be fine if it was a part of the regular configuration loading and merging process, as is done with other config options. The trouble is it does so in surprising places, such as in functions with 'verify' or 'validate' in their name. It has been necessary in order to validate that the user has not defined a custom runtime named "runc" which would shadow the built-in runtime of the same name. Other daemon code depends on the runtime named "runc" always being defined in the config, but merging it with the user config at the same time as the other defaults are merged would trip the validation. The root of the issue is that the daemon has used the same config values for both validating the daemon runtime configuration as supplied by the user and for keeping track of which runtimes have been set up by the daemon. Now that a completely separate value is used for the latter purpose, surprising contortions are no longer required to make the validation work as intended. Consolidate the validation of the runtimes config and merging of the built-in runtimes into the daemon.setupRuntimes() function. Set the result of merging the built-in runtimes config and default default runtime on the returned runtimes struct, without back-propagating it onto the config.Config argument. Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-08-31 21:24:22 +00:00
func TestSetupRuntimes(t *testing.T) {
cases := []struct {
name string
daemon: consolidate runtimes config validation The daemon has made a habit of mutating the DefaultRuntime and Runtimes values in the Config struct to merge defaults. This would be fine if it was a part of the regular configuration loading and merging process, as is done with other config options. The trouble is it does so in surprising places, such as in functions with 'verify' or 'validate' in their name. It has been necessary in order to validate that the user has not defined a custom runtime named "runc" which would shadow the built-in runtime of the same name. Other daemon code depends on the runtime named "runc" always being defined in the config, but merging it with the user config at the same time as the other defaults are merged would trip the validation. The root of the issue is that the daemon has used the same config values for both validating the daemon runtime configuration as supplied by the user and for keeping track of which runtimes have been set up by the daemon. Now that a completely separate value is used for the latter purpose, surprising contortions are no longer required to make the validation work as intended. Consolidate the validation of the runtimes config and merging of the built-in runtimes into the daemon.setupRuntimes() function. Set the result of merging the built-in runtimes config and default default runtime on the returned runtimes struct, without back-propagating it onto the config.Config argument. Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-08-31 21:24:22 +00:00
config *config.Config
expectErr string
}{
{
daemon: consolidate runtimes config validation The daemon has made a habit of mutating the DefaultRuntime and Runtimes values in the Config struct to merge defaults. This would be fine if it was a part of the regular configuration loading and merging process, as is done with other config options. The trouble is it does so in surprising places, such as in functions with 'verify' or 'validate' in their name. It has been necessary in order to validate that the user has not defined a custom runtime named "runc" which would shadow the built-in runtime of the same name. Other daemon code depends on the runtime named "runc" always being defined in the config, but merging it with the user config at the same time as the other defaults are merged would trip the validation. The root of the issue is that the daemon has used the same config values for both validating the daemon runtime configuration as supplied by the user and for keeping track of which runtimes have been set up by the daemon. Now that a completely separate value is used for the latter purpose, surprising contortions are no longer required to make the validation work as intended. Consolidate the validation of the runtimes config and merging of the built-in runtimes into the daemon.setupRuntimes() function. Set the result of merging the built-in runtimes config and default default runtime on the returned runtimes struct, without back-propagating it onto the config.Config argument. Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-08-31 21:24:22 +00:00
name: "Empty",
config: &config.Config{
Runtimes: map[string]types.Runtime{
"myruntime": {},
},
},
expectErr: "either a runtimeType or a path must be configured",
},
{
daemon: consolidate runtimes config validation The daemon has made a habit of mutating the DefaultRuntime and Runtimes values in the Config struct to merge defaults. This would be fine if it was a part of the regular configuration loading and merging process, as is done with other config options. The trouble is it does so in surprising places, such as in functions with 'verify' or 'validate' in their name. It has been necessary in order to validate that the user has not defined a custom runtime named "runc" which would shadow the built-in runtime of the same name. Other daemon code depends on the runtime named "runc" always being defined in the config, but merging it with the user config at the same time as the other defaults are merged would trip the validation. The root of the issue is that the daemon has used the same config values for both validating the daemon runtime configuration as supplied by the user and for keeping track of which runtimes have been set up by the daemon. Now that a completely separate value is used for the latter purpose, surprising contortions are no longer required to make the validation work as intended. Consolidate the validation of the runtimes config and merging of the built-in runtimes into the daemon.setupRuntimes() function. Set the result of merging the built-in runtimes config and default default runtime on the returned runtimes struct, without back-propagating it onto the config.Config argument. Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-08-31 21:24:22 +00:00
name: "ArgsOnly",
config: &config.Config{
Runtimes: map[string]types.Runtime{
"myruntime": {Args: []string{"foo", "bar"}},
},
},
expectErr: "either a runtimeType or a path must be configured",
},
{
daemon: consolidate runtimes config validation The daemon has made a habit of mutating the DefaultRuntime and Runtimes values in the Config struct to merge defaults. This would be fine if it was a part of the regular configuration loading and merging process, as is done with other config options. The trouble is it does so in surprising places, such as in functions with 'verify' or 'validate' in their name. It has been necessary in order to validate that the user has not defined a custom runtime named "runc" which would shadow the built-in runtime of the same name. Other daemon code depends on the runtime named "runc" always being defined in the config, but merging it with the user config at the same time as the other defaults are merged would trip the validation. The root of the issue is that the daemon has used the same config values for both validating the daemon runtime configuration as supplied by the user and for keeping track of which runtimes have been set up by the daemon. Now that a completely separate value is used for the latter purpose, surprising contortions are no longer required to make the validation work as intended. Consolidate the validation of the runtimes config and merging of the built-in runtimes into the daemon.setupRuntimes() function. Set the result of merging the built-in runtimes config and default default runtime on the returned runtimes struct, without back-propagating it onto the config.Config argument. Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-08-31 21:24:22 +00:00
name: "OptionsOnly",
config: &config.Config{
Runtimes: map[string]types.Runtime{
"myruntime": {Options: map[string]interface{}{"hello": "world"}},
},
},
expectErr: "either a runtimeType or a path must be configured",
},
{
daemon: consolidate runtimes config validation The daemon has made a habit of mutating the DefaultRuntime and Runtimes values in the Config struct to merge defaults. This would be fine if it was a part of the regular configuration loading and merging process, as is done with other config options. The trouble is it does so in surprising places, such as in functions with 'verify' or 'validate' in their name. It has been necessary in order to validate that the user has not defined a custom runtime named "runc" which would shadow the built-in runtime of the same name. Other daemon code depends on the runtime named "runc" always being defined in the config, but merging it with the user config at the same time as the other defaults are merged would trip the validation. The root of the issue is that the daemon has used the same config values for both validating the daemon runtime configuration as supplied by the user and for keeping track of which runtimes have been set up by the daemon. Now that a completely separate value is used for the latter purpose, surprising contortions are no longer required to make the validation work as intended. Consolidate the validation of the runtimes config and merging of the built-in runtimes into the daemon.setupRuntimes() function. Set the result of merging the built-in runtimes config and default default runtime on the returned runtimes struct, without back-propagating it onto the config.Config argument. Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-08-31 21:24:22 +00:00
name: "PathAndType",
config: &config.Config{
Runtimes: map[string]types.Runtime{
"myruntime": {Path: "/bin/true", Type: "io.containerd.runsc.v1"},
},
},
expectErr: "cannot configure both",
},
{
daemon: consolidate runtimes config validation The daemon has made a habit of mutating the DefaultRuntime and Runtimes values in the Config struct to merge defaults. This would be fine if it was a part of the regular configuration loading and merging process, as is done with other config options. The trouble is it does so in surprising places, such as in functions with 'verify' or 'validate' in their name. It has been necessary in order to validate that the user has not defined a custom runtime named "runc" which would shadow the built-in runtime of the same name. Other daemon code depends on the runtime named "runc" always being defined in the config, but merging it with the user config at the same time as the other defaults are merged would trip the validation. The root of the issue is that the daemon has used the same config values for both validating the daemon runtime configuration as supplied by the user and for keeping track of which runtimes have been set up by the daemon. Now that a completely separate value is used for the latter purpose, surprising contortions are no longer required to make the validation work as intended. Consolidate the validation of the runtimes config and merging of the built-in runtimes into the daemon.setupRuntimes() function. Set the result of merging the built-in runtimes config and default default runtime on the returned runtimes struct, without back-propagating it onto the config.Config argument. Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-08-31 21:24:22 +00:00
name: "PathAndOptions",
config: &config.Config{
Runtimes: map[string]types.Runtime{
"myruntime": {Path: "/bin/true", Options: map[string]interface{}{"a": "b"}},
},
},
expectErr: "options cannot be used with a path runtime",
},
{
daemon: consolidate runtimes config validation The daemon has made a habit of mutating the DefaultRuntime and Runtimes values in the Config struct to merge defaults. This would be fine if it was a part of the regular configuration loading and merging process, as is done with other config options. The trouble is it does so in surprising places, such as in functions with 'verify' or 'validate' in their name. It has been necessary in order to validate that the user has not defined a custom runtime named "runc" which would shadow the built-in runtime of the same name. Other daemon code depends on the runtime named "runc" always being defined in the config, but merging it with the user config at the same time as the other defaults are merged would trip the validation. The root of the issue is that the daemon has used the same config values for both validating the daemon runtime configuration as supplied by the user and for keeping track of which runtimes have been set up by the daemon. Now that a completely separate value is used for the latter purpose, surprising contortions are no longer required to make the validation work as intended. Consolidate the validation of the runtimes config and merging of the built-in runtimes into the daemon.setupRuntimes() function. Set the result of merging the built-in runtimes config and default default runtime on the returned runtimes struct, without back-propagating it onto the config.Config argument. Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-08-31 21:24:22 +00:00
name: "TypeAndArgs",
config: &config.Config{
Runtimes: map[string]types.Runtime{
"myruntime": {Type: "io.containerd.runsc.v1", Args: []string{"--version"}},
},
},
expectErr: "args cannot be used with a runtimeType runtime",
},
{
name: "PathArgsOptions",
daemon: consolidate runtimes config validation The daemon has made a habit of mutating the DefaultRuntime and Runtimes values in the Config struct to merge defaults. This would be fine if it was a part of the regular configuration loading and merging process, as is done with other config options. The trouble is it does so in surprising places, such as in functions with 'verify' or 'validate' in their name. It has been necessary in order to validate that the user has not defined a custom runtime named "runc" which would shadow the built-in runtime of the same name. Other daemon code depends on the runtime named "runc" always being defined in the config, but merging it with the user config at the same time as the other defaults are merged would trip the validation. The root of the issue is that the daemon has used the same config values for both validating the daemon runtime configuration as supplied by the user and for keeping track of which runtimes have been set up by the daemon. Now that a completely separate value is used for the latter purpose, surprising contortions are no longer required to make the validation work as intended. Consolidate the validation of the runtimes config and merging of the built-in runtimes into the daemon.setupRuntimes() function. Set the result of merging the built-in runtimes config and default default runtime on the returned runtimes struct, without back-propagating it onto the config.Config argument. Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-08-31 21:24:22 +00:00
config: &config.Config{
Runtimes: map[string]types.Runtime{
"myruntime": {
Path: "/bin/true",
Args: []string{"--version"},
Options: map[string]interface{}{"hmm": 3},
},
},
},
expectErr: "options cannot be used with a path runtime",
},
{
name: "TypeOptionsArgs",
daemon: consolidate runtimes config validation The daemon has made a habit of mutating the DefaultRuntime and Runtimes values in the Config struct to merge defaults. This would be fine if it was a part of the regular configuration loading and merging process, as is done with other config options. The trouble is it does so in surprising places, such as in functions with 'verify' or 'validate' in their name. It has been necessary in order to validate that the user has not defined a custom runtime named "runc" which would shadow the built-in runtime of the same name. Other daemon code depends on the runtime named "runc" always being defined in the config, but merging it with the user config at the same time as the other defaults are merged would trip the validation. The root of the issue is that the daemon has used the same config values for both validating the daemon runtime configuration as supplied by the user and for keeping track of which runtimes have been set up by the daemon. Now that a completely separate value is used for the latter purpose, surprising contortions are no longer required to make the validation work as intended. Consolidate the validation of the runtimes config and merging of the built-in runtimes into the daemon.setupRuntimes() function. Set the result of merging the built-in runtimes config and default default runtime on the returned runtimes struct, without back-propagating it onto the config.Config argument. Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-08-31 21:24:22 +00:00
config: &config.Config{
Runtimes: map[string]types.Runtime{
"myruntime": {
Type: "io.containerd.kata.v2",
Options: map[string]interface{}{"a": "b"},
Args: []string{"--help"},
},
},
},
expectErr: "args cannot be used with a runtimeType runtime",
},
{
name: "PathArgsTypeOptions",
daemon: consolidate runtimes config validation The daemon has made a habit of mutating the DefaultRuntime and Runtimes values in the Config struct to merge defaults. This would be fine if it was a part of the regular configuration loading and merging process, as is done with other config options. The trouble is it does so in surprising places, such as in functions with 'verify' or 'validate' in their name. It has been necessary in order to validate that the user has not defined a custom runtime named "runc" which would shadow the built-in runtime of the same name. Other daemon code depends on the runtime named "runc" always being defined in the config, but merging it with the user config at the same time as the other defaults are merged would trip the validation. The root of the issue is that the daemon has used the same config values for both validating the daemon runtime configuration as supplied by the user and for keeping track of which runtimes have been set up by the daemon. Now that a completely separate value is used for the latter purpose, surprising contortions are no longer required to make the validation work as intended. Consolidate the validation of the runtimes config and merging of the built-in runtimes into the daemon.setupRuntimes() function. Set the result of merging the built-in runtimes config and default default runtime on the returned runtimes struct, without back-propagating it onto the config.Config argument. Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-08-31 21:24:22 +00:00
config: &config.Config{
Runtimes: map[string]types.Runtime{
"myruntime": {
Path: "/bin/true",
Args: []string{"foo"},
Type: "io.containerd.runsc.v1",
Options: map[string]interface{}{"a": "b"},
},
},
},
expectErr: "cannot configure both",
},
daemon: consolidate runtimes config validation The daemon has made a habit of mutating the DefaultRuntime and Runtimes values in the Config struct to merge defaults. This would be fine if it was a part of the regular configuration loading and merging process, as is done with other config options. The trouble is it does so in surprising places, such as in functions with 'verify' or 'validate' in their name. It has been necessary in order to validate that the user has not defined a custom runtime named "runc" which would shadow the built-in runtime of the same name. Other daemon code depends on the runtime named "runc" always being defined in the config, but merging it with the user config at the same time as the other defaults are merged would trip the validation. The root of the issue is that the daemon has used the same config values for both validating the daemon runtime configuration as supplied by the user and for keeping track of which runtimes have been set up by the daemon. Now that a completely separate value is used for the latter purpose, surprising contortions are no longer required to make the validation work as intended. Consolidate the validation of the runtimes config and merging of the built-in runtimes into the daemon.setupRuntimes() function. Set the result of merging the built-in runtimes config and default default runtime on the returned runtimes struct, without back-propagating it onto the config.Config argument. Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-08-31 21:24:22 +00:00
{
name: "CannotOverrideStockRuntime",
config: &config.Config{
Runtimes: map[string]types.Runtime{
config.StockRuntimeName: {},
},
},
expectErr: `runtime name 'runc' is reserved`,
},
{
name: "SetStockRuntimeAsDefault",
config: &config.Config{
CommonConfig: config.CommonConfig{
DefaultRuntime: config.StockRuntimeName,
},
},
},
{
name: "SetLinuxRuntimeAsDefault",
config: &config.Config{
CommonConfig: config.CommonConfig{
DefaultRuntime: linuxV2RuntimeName,
},
},
},
{
name: "CannotSetBogusRuntimeAsDefault",
config: &config.Config{
CommonConfig: config.CommonConfig{
DefaultRuntime: "notdefined",
},
},
expectErr: "specified default runtime 'notdefined' does not exist",
},
{
name: "SetDefinedRuntimeAsDefault",
config: &config.Config{
Runtimes: map[string]types.Runtime{
"some-runtime": {
Path: "/usr/local/bin/file-not-found",
},
},
CommonConfig: config.CommonConfig{
DefaultRuntime: "some-runtime",
},
},
},
}
daemon: consolidate runtimes config validation The daemon has made a habit of mutating the DefaultRuntime and Runtimes values in the Config struct to merge defaults. This would be fine if it was a part of the regular configuration loading and merging process, as is done with other config options. The trouble is it does so in surprising places, such as in functions with 'verify' or 'validate' in their name. It has been necessary in order to validate that the user has not defined a custom runtime named "runc" which would shadow the built-in runtime of the same name. Other daemon code depends on the runtime named "runc" always being defined in the config, but merging it with the user config at the same time as the other defaults are merged would trip the validation. The root of the issue is that the daemon has used the same config values for both validating the daemon runtime configuration as supplied by the user and for keeping track of which runtimes have been set up by the daemon. Now that a completely separate value is used for the latter purpose, surprising contortions are no longer required to make the validation work as intended. Consolidate the validation of the runtimes config and merging of the built-in runtimes into the daemon.setupRuntimes() function. Set the result of merging the built-in runtimes config and default default runtime on the returned runtimes struct, without back-propagating it onto the config.Config argument. Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-08-31 21:24:22 +00:00
for _, tc := range cases {
tc := tc
t.Run(tc.name, func(t *testing.T) {
cfg, err := config.New()
assert.NilError(t, err)
cfg.Root = t.TempDir()
daemon: consolidate runtimes config validation The daemon has made a habit of mutating the DefaultRuntime and Runtimes values in the Config struct to merge defaults. This would be fine if it was a part of the regular configuration loading and merging process, as is done with other config options. The trouble is it does so in surprising places, such as in functions with 'verify' or 'validate' in their name. It has been necessary in order to validate that the user has not defined a custom runtime named "runc" which would shadow the built-in runtime of the same name. Other daemon code depends on the runtime named "runc" always being defined in the config, but merging it with the user config at the same time as the other defaults are merged would trip the validation. The root of the issue is that the daemon has used the same config values for both validating the daemon runtime configuration as supplied by the user and for keeping track of which runtimes have been set up by the daemon. Now that a completely separate value is used for the latter purpose, surprising contortions are no longer required to make the validation work as intended. Consolidate the validation of the runtimes config and merging of the built-in runtimes into the daemon.setupRuntimes() function. Set the result of merging the built-in runtimes config and default default runtime on the returned runtimes struct, without back-propagating it onto the config.Config argument. Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-08-31 21:24:22 +00:00
assert.NilError(t, mergo.Merge(cfg, tc.config, mergo.WithOverride))
daemon: reload runtimes w/o breaking containers The existing runtimes reload logic went to great lengths to replace the directory containing runtime wrapper scripts as atomically as possible within the limitations of the Linux filesystem ABI. Trouble is, atomically swapping the wrapper scripts directory solves the wrong problem! The runtime configuration is "locked in" when a container is started, including the path to the runC binary. If a container is started with a runtime which requires a daemon-managed wrapper script and then the daemon is reloaded with a config which no longer requires the wrapper script (i.e. some args -> no args, or the runtime is dropped from the config), that container would become unmanageable. Any attempts to stop, exec or otherwise perform lifecycle management operations on the container are likely to fail due to the wrapper script no longer existing at its original path. Atomically swapping the wrapper scripts is also incompatible with the read-copy-update paradigm for reloading configuration. A handler in the daemon could retain a reference to the pre-reload configuration for an indeterminate amount of time after the daemon configuration has been reloaded and updated. It is possible for the daemon to attempt to start a container using a deleted wrapper script if a request to run a container races a reload. Solve the problem of deleting referenced wrapper scripts by ensuring that all wrapper scripts are *immutable* for the lifetime of the daemon process. Any given runtime wrapper script must always exist with the same contents, no matter how many times the daemon config is reloaded, or what changes are made to the config. This is accomplished by using everyone's favourite design pattern: content-addressable storage. Each wrapper script file name is suffixed with the SHA-256 digest of its contents to (probabilistically) guarantee immutability without needing any concurrency control. Stale runtime wrapper scripts are only cleaned up on the next daemon restart. Split the derived runtimes configuration from the user-supplied configuration to have a place to store derived state without mutating the user-supplied configuration or exposing daemon internals in API struct types. Hold the derived state and the user-supplied configuration in a single struct value so that they can be updated as an atomic unit. Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-08-31 20:12:30 +00:00
assert.Assert(t, initRuntimesDir(cfg))
daemon: reload runtimes w/o breaking containers The existing runtimes reload logic went to great lengths to replace the directory containing runtime wrapper scripts as atomically as possible within the limitations of the Linux filesystem ABI. Trouble is, atomically swapping the wrapper scripts directory solves the wrong problem! The runtime configuration is "locked in" when a container is started, including the path to the runC binary. If a container is started with a runtime which requires a daemon-managed wrapper script and then the daemon is reloaded with a config which no longer requires the wrapper script (i.e. some args -> no args, or the runtime is dropped from the config), that container would become unmanageable. Any attempts to stop, exec or otherwise perform lifecycle management operations on the container are likely to fail due to the wrapper script no longer existing at its original path. Atomically swapping the wrapper scripts is also incompatible with the read-copy-update paradigm for reloading configuration. A handler in the daemon could retain a reference to the pre-reload configuration for an indeterminate amount of time after the daemon configuration has been reloaded and updated. It is possible for the daemon to attempt to start a container using a deleted wrapper script if a request to run a container races a reload. Solve the problem of deleting referenced wrapper scripts by ensuring that all wrapper scripts are *immutable* for the lifetime of the daemon process. Any given runtime wrapper script must always exist with the same contents, no matter how many times the daemon config is reloaded, or what changes are made to the config. This is accomplished by using everyone's favourite design pattern: content-addressable storage. Each wrapper script file name is suffixed with the SHA-256 digest of its contents to (probabilistically) guarantee immutability without needing any concurrency control. Stale runtime wrapper scripts are only cleaned up on the next daemon restart. Split the derived runtimes configuration from the user-supplied configuration to have a place to store derived state without mutating the user-supplied configuration or exposing daemon internals in API struct types. Hold the derived state and the user-supplied configuration in a single struct value so that they can be updated as an atomic unit. Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-08-31 20:12:30 +00:00
_, err = setupRuntimes(cfg)
daemon: consolidate runtimes config validation The daemon has made a habit of mutating the DefaultRuntime and Runtimes values in the Config struct to merge defaults. This would be fine if it was a part of the regular configuration loading and merging process, as is done with other config options. The trouble is it does so in surprising places, such as in functions with 'verify' or 'validate' in their name. It has been necessary in order to validate that the user has not defined a custom runtime named "runc" which would shadow the built-in runtime of the same name. Other daemon code depends on the runtime named "runc" always being defined in the config, but merging it with the user config at the same time as the other defaults are merged would trip the validation. The root of the issue is that the daemon has used the same config values for both validating the daemon runtime configuration as supplied by the user and for keeping track of which runtimes have been set up by the daemon. Now that a completely separate value is used for the latter purpose, surprising contortions are no longer required to make the validation work as intended. Consolidate the validation of the runtimes config and merging of the built-in runtimes into the daemon.setupRuntimes() function. Set the result of merging the built-in runtimes config and default default runtime on the returned runtimes struct, without back-propagating it onto the config.Config argument. Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-08-31 21:24:22 +00:00
if tc.expectErr == "" {
assert.NilError(t, err)
} else {
assert.ErrorContains(t, err, tc.expectErr)
}
})
}
}
daemon: support other containerd runtimes (MVP) Contrary to popular belief, the OCI Runtime specification does not specify the command-line API for runtimes. Looking at containerd's architecture from the lens of the OCI Runtime spec, the _shim_ is the OCI Runtime and runC is "just" an implementation detail of the io.containerd.runc.v2 runtime. When one configures a non-default runtime in Docker, what they're really doing is instructing Docker to create containers using the io.containerd.runc.v2 runtime with a configuration option telling the runtime that the runC binary is at some non-default path. Consequently, only OCI runtimes which are compatible with the io.containerd.runc.v2 shim, such as crun, can be used in this manner. Other OCI runtimes, including kata-containers v2, come with their own containerd shim and are not compatible with io.containerd.runc.v2. As Docker has not historically provided a way to select a non-default runtime which requires its own shim, runtimes such as kata-containers v2 could not be used with Docker. Allow other containerd shims to be used with Docker; no daemon configuration required. If the daemon is instructed to create a container with a runtime name which does not match any of the configured or stock runtimes, it passes the name along to containerd verbatim. A user can start a container with the kata-containers runtime, for example, simply by calling docker run --runtime io.containerd.kata.v2 Runtime names which containerd would interpret as a path to an arbitrary binary are disallowed. While handy for development and testing it is not strictly necessary and would allow anyone with Engine API access to trivially execute any binary on the host as root, so we have decided it would be safest for our users if it was not allowed. It is not yet possible to set an alternative containerd shim as the default runtime; it can only be configured per-container. Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-07-20 20:12:01 +00:00
func TestGetRuntime(t *testing.T) {
// Configured runtimes can have any arbitrary name, including names
// which would not be allowed as implicit runtime names. Explicit takes
// precedence over implicit.
const configuredRtName = "my/custom.runtime.v1"
daemon: support other containerd runtimes (MVP) Contrary to popular belief, the OCI Runtime specification does not specify the command-line API for runtimes. Looking at containerd's architecture from the lens of the OCI Runtime spec, the _shim_ is the OCI Runtime and runC is "just" an implementation detail of the io.containerd.runc.v2 runtime. When one configures a non-default runtime in Docker, what they're really doing is instructing Docker to create containers using the io.containerd.runc.v2 runtime with a configuration option telling the runtime that the runC binary is at some non-default path. Consequently, only OCI runtimes which are compatible with the io.containerd.runc.v2 shim, such as crun, can be used in this manner. Other OCI runtimes, including kata-containers v2, come with their own containerd shim and are not compatible with io.containerd.runc.v2. As Docker has not historically provided a way to select a non-default runtime which requires its own shim, runtimes such as kata-containers v2 could not be used with Docker. Allow other containerd shims to be used with Docker; no daemon configuration required. If the daemon is instructed to create a container with a runtime name which does not match any of the configured or stock runtimes, it passes the name along to containerd verbatim. A user can start a container with the kata-containers runtime, for example, simply by calling docker run --runtime io.containerd.kata.v2 Runtime names which containerd would interpret as a path to an arbitrary binary are disallowed. While handy for development and testing it is not strictly necessary and would allow anyone with Engine API access to trivially execute any binary on the host as root, so we have decided it would be safest for our users if it was not allowed. It is not yet possible to set an alternative containerd shim as the default runtime; it can only be configured per-container. Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-07-20 20:12:01 +00:00
configuredRuntime := types.Runtime{Path: "/bin/true"}
const rtWithArgsName = "withargs"
rtWithArgs := types.Runtime{
Path: "/bin/false",
Args: []string{"--version"},
}
const shimWithOptsName = "shimwithopts"
shimWithOpts := types.Runtime{
Type: plugin.RuntimeRuncV2,
Options: map[string]interface{}{"IoUid": 42},
}
const shimAliasName = "wasmedge"
shimAlias := types.Runtime{Type: "io.containerd.wasmedge.v1"}
const configuredShimByPathName = "shimwithpath"
configuredShimByPath := types.Runtime{Type: "/path/to/my/shim"}
cfg, err := config.New()
assert.NilError(t, err)
cfg.Root = t.TempDir()
cfg.Runtimes = map[string]types.Runtime{
configuredRtName: configuredRuntime,
rtWithArgsName: rtWithArgs,
shimWithOptsName: shimWithOpts,
shimAliasName: shimAlias,
configuredShimByPathName: configuredShimByPath,
daemon: support other containerd runtimes (MVP) Contrary to popular belief, the OCI Runtime specification does not specify the command-line API for runtimes. Looking at containerd's architecture from the lens of the OCI Runtime spec, the _shim_ is the OCI Runtime and runC is "just" an implementation detail of the io.containerd.runc.v2 runtime. When one configures a non-default runtime in Docker, what they're really doing is instructing Docker to create containers using the io.containerd.runc.v2 runtime with a configuration option telling the runtime that the runC binary is at some non-default path. Consequently, only OCI runtimes which are compatible with the io.containerd.runc.v2 shim, such as crun, can be used in this manner. Other OCI runtimes, including kata-containers v2, come with their own containerd shim and are not compatible with io.containerd.runc.v2. As Docker has not historically provided a way to select a non-default runtime which requires its own shim, runtimes such as kata-containers v2 could not be used with Docker. Allow other containerd shims to be used with Docker; no daemon configuration required. If the daemon is instructed to create a container with a runtime name which does not match any of the configured or stock runtimes, it passes the name along to containerd verbatim. A user can start a container with the kata-containers runtime, for example, simply by calling docker run --runtime io.containerd.kata.v2 Runtime names which containerd would interpret as a path to an arbitrary binary are disallowed. While handy for development and testing it is not strictly necessary and would allow anyone with Engine API access to trivially execute any binary on the host as root, so we have decided it would be safest for our users if it was not allowed. It is not yet possible to set an alternative containerd shim as the default runtime; it can only be configured per-container. Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-07-20 20:12:01 +00:00
}
daemon: reload runtimes w/o breaking containers The existing runtimes reload logic went to great lengths to replace the directory containing runtime wrapper scripts as atomically as possible within the limitations of the Linux filesystem ABI. Trouble is, atomically swapping the wrapper scripts directory solves the wrong problem! The runtime configuration is "locked in" when a container is started, including the path to the runC binary. If a container is started with a runtime which requires a daemon-managed wrapper script and then the daemon is reloaded with a config which no longer requires the wrapper script (i.e. some args -> no args, or the runtime is dropped from the config), that container would become unmanageable. Any attempts to stop, exec or otherwise perform lifecycle management operations on the container are likely to fail due to the wrapper script no longer existing at its original path. Atomically swapping the wrapper scripts is also incompatible with the read-copy-update paradigm for reloading configuration. A handler in the daemon could retain a reference to the pre-reload configuration for an indeterminate amount of time after the daemon configuration has been reloaded and updated. It is possible for the daemon to attempt to start a container using a deleted wrapper script if a request to run a container races a reload. Solve the problem of deleting referenced wrapper scripts by ensuring that all wrapper scripts are *immutable* for the lifetime of the daemon process. Any given runtime wrapper script must always exist with the same contents, no matter how many times the daemon config is reloaded, or what changes are made to the config. This is accomplished by using everyone's favourite design pattern: content-addressable storage. Each wrapper script file name is suffixed with the SHA-256 digest of its contents to (probabilistically) guarantee immutability without needing any concurrency control. Stale runtime wrapper scripts are only cleaned up on the next daemon restart. Split the derived runtimes configuration from the user-supplied configuration to have a place to store derived state without mutating the user-supplied configuration or exposing daemon internals in API struct types. Hold the derived state and the user-supplied configuration in a single struct value so that they can be updated as an atomic unit. Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-08-31 20:12:30 +00:00
assert.NilError(t, initRuntimesDir(cfg))
runtimes, err := setupRuntimes(cfg)
assert.NilError(t, err)
daemon: reload runtimes w/o breaking containers The existing runtimes reload logic went to great lengths to replace the directory containing runtime wrapper scripts as atomically as possible within the limitations of the Linux filesystem ABI. Trouble is, atomically swapping the wrapper scripts directory solves the wrong problem! The runtime configuration is "locked in" when a container is started, including the path to the runC binary. If a container is started with a runtime which requires a daemon-managed wrapper script and then the daemon is reloaded with a config which no longer requires the wrapper script (i.e. some args -> no args, or the runtime is dropped from the config), that container would become unmanageable. Any attempts to stop, exec or otherwise perform lifecycle management operations on the container are likely to fail due to the wrapper script no longer existing at its original path. Atomically swapping the wrapper scripts is also incompatible with the read-copy-update paradigm for reloading configuration. A handler in the daemon could retain a reference to the pre-reload configuration for an indeterminate amount of time after the daemon configuration has been reloaded and updated. It is possible for the daemon to attempt to start a container using a deleted wrapper script if a request to run a container races a reload. Solve the problem of deleting referenced wrapper scripts by ensuring that all wrapper scripts are *immutable* for the lifetime of the daemon process. Any given runtime wrapper script must always exist with the same contents, no matter how many times the daemon config is reloaded, or what changes are made to the config. This is accomplished by using everyone's favourite design pattern: content-addressable storage. Each wrapper script file name is suffixed with the SHA-256 digest of its contents to (probabilistically) guarantee immutability without needing any concurrency control. Stale runtime wrapper scripts are only cleaned up on the next daemon restart. Split the derived runtimes configuration from the user-supplied configuration to have a place to store derived state without mutating the user-supplied configuration or exposing daemon internals in API struct types. Hold the derived state and the user-supplied configuration in a single struct value so that they can be updated as an atomic unit. Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-08-31 20:12:30 +00:00
stockRuntime, ok := runtimes.configured[config.StockRuntimeName]
daemon: support other containerd runtimes (MVP) Contrary to popular belief, the OCI Runtime specification does not specify the command-line API for runtimes. Looking at containerd's architecture from the lens of the OCI Runtime spec, the _shim_ is the OCI Runtime and runC is "just" an implementation detail of the io.containerd.runc.v2 runtime. When one configures a non-default runtime in Docker, what they're really doing is instructing Docker to create containers using the io.containerd.runc.v2 runtime with a configuration option telling the runtime that the runC binary is at some non-default path. Consequently, only OCI runtimes which are compatible with the io.containerd.runc.v2 shim, such as crun, can be used in this manner. Other OCI runtimes, including kata-containers v2, come with their own containerd shim and are not compatible with io.containerd.runc.v2. As Docker has not historically provided a way to select a non-default runtime which requires its own shim, runtimes such as kata-containers v2 could not be used with Docker. Allow other containerd shims to be used with Docker; no daemon configuration required. If the daemon is instructed to create a container with a runtime name which does not match any of the configured or stock runtimes, it passes the name along to containerd verbatim. A user can start a container with the kata-containers runtime, for example, simply by calling docker run --runtime io.containerd.kata.v2 Runtime names which containerd would interpret as a path to an arbitrary binary are disallowed. While handy for development and testing it is not strictly necessary and would allow anyone with Engine API access to trivially execute any binary on the host as root, so we have decided it would be safest for our users if it was not allowed. It is not yet possible to set an alternative containerd shim as the default runtime; it can only be configured per-container. Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-07-20 20:12:01 +00:00
assert.Assert(t, ok, "stock runtime could not be found (test needs to be updated)")
daemon: reload runtimes w/o breaking containers The existing runtimes reload logic went to great lengths to replace the directory containing runtime wrapper scripts as atomically as possible within the limitations of the Linux filesystem ABI. Trouble is, atomically swapping the wrapper scripts directory solves the wrong problem! The runtime configuration is "locked in" when a container is started, including the path to the runC binary. If a container is started with a runtime which requires a daemon-managed wrapper script and then the daemon is reloaded with a config which no longer requires the wrapper script (i.e. some args -> no args, or the runtime is dropped from the config), that container would become unmanageable. Any attempts to stop, exec or otherwise perform lifecycle management operations on the container are likely to fail due to the wrapper script no longer existing at its original path. Atomically swapping the wrapper scripts is also incompatible with the read-copy-update paradigm for reloading configuration. A handler in the daemon could retain a reference to the pre-reload configuration for an indeterminate amount of time after the daemon configuration has been reloaded and updated. It is possible for the daemon to attempt to start a container using a deleted wrapper script if a request to run a container races a reload. Solve the problem of deleting referenced wrapper scripts by ensuring that all wrapper scripts are *immutable* for the lifetime of the daemon process. Any given runtime wrapper script must always exist with the same contents, no matter how many times the daemon config is reloaded, or what changes are made to the config. This is accomplished by using everyone's favourite design pattern: content-addressable storage. Each wrapper script file name is suffixed with the SHA-256 digest of its contents to (probabilistically) guarantee immutability without needing any concurrency control. Stale runtime wrapper scripts are only cleaned up on the next daemon restart. Split the derived runtimes configuration from the user-supplied configuration to have a place to store derived state without mutating the user-supplied configuration or exposing daemon internals in API struct types. Hold the derived state and the user-supplied configuration in a single struct value so that they can be updated as an atomic unit. Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-08-31 20:12:30 +00:00
stockRuntime.Features = nil
daemon: support other containerd runtimes (MVP) Contrary to popular belief, the OCI Runtime specification does not specify the command-line API for runtimes. Looking at containerd's architecture from the lens of the OCI Runtime spec, the _shim_ is the OCI Runtime and runC is "just" an implementation detail of the io.containerd.runc.v2 runtime. When one configures a non-default runtime in Docker, what they're really doing is instructing Docker to create containers using the io.containerd.runc.v2 runtime with a configuration option telling the runtime that the runC binary is at some non-default path. Consequently, only OCI runtimes which are compatible with the io.containerd.runc.v2 shim, such as crun, can be used in this manner. Other OCI runtimes, including kata-containers v2, come with their own containerd shim and are not compatible with io.containerd.runc.v2. As Docker has not historically provided a way to select a non-default runtime which requires its own shim, runtimes such as kata-containers v2 could not be used with Docker. Allow other containerd shims to be used with Docker; no daemon configuration required. If the daemon is instructed to create a container with a runtime name which does not match any of the configured or stock runtimes, it passes the name along to containerd verbatim. A user can start a container with the kata-containers runtime, for example, simply by calling docker run --runtime io.containerd.kata.v2 Runtime names which containerd would interpret as a path to an arbitrary binary are disallowed. While handy for development and testing it is not strictly necessary and would allow anyone with Engine API access to trivially execute any binary on the host as root, so we have decided it would be safest for our users if it was not allowed. It is not yet possible to set an alternative containerd shim as the default runtime; it can only be configured per-container. Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-07-20 20:12:01 +00:00
daemon: reload runtimes w/o breaking containers The existing runtimes reload logic went to great lengths to replace the directory containing runtime wrapper scripts as atomically as possible within the limitations of the Linux filesystem ABI. Trouble is, atomically swapping the wrapper scripts directory solves the wrong problem! The runtime configuration is "locked in" when a container is started, including the path to the runC binary. If a container is started with a runtime which requires a daemon-managed wrapper script and then the daemon is reloaded with a config which no longer requires the wrapper script (i.e. some args -> no args, or the runtime is dropped from the config), that container would become unmanageable. Any attempts to stop, exec or otherwise perform lifecycle management operations on the container are likely to fail due to the wrapper script no longer existing at its original path. Atomically swapping the wrapper scripts is also incompatible with the read-copy-update paradigm for reloading configuration. A handler in the daemon could retain a reference to the pre-reload configuration for an indeterminate amount of time after the daemon configuration has been reloaded and updated. It is possible for the daemon to attempt to start a container using a deleted wrapper script if a request to run a container races a reload. Solve the problem of deleting referenced wrapper scripts by ensuring that all wrapper scripts are *immutable* for the lifetime of the daemon process. Any given runtime wrapper script must always exist with the same contents, no matter how many times the daemon config is reloaded, or what changes are made to the config. This is accomplished by using everyone's favourite design pattern: content-addressable storage. Each wrapper script file name is suffixed with the SHA-256 digest of its contents to (probabilistically) guarantee immutability without needing any concurrency control. Stale runtime wrapper scripts are only cleaned up on the next daemon restart. Split the derived runtimes configuration from the user-supplied configuration to have a place to store derived state without mutating the user-supplied configuration or exposing daemon internals in API struct types. Hold the derived state and the user-supplied configuration in a single struct value so that they can be updated as an atomic unit. Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-08-31 20:12:30 +00:00
configdOpts := *stockRuntime.Opts.(*v2runcoptions.Options)
daemon: support other containerd runtimes (MVP) Contrary to popular belief, the OCI Runtime specification does not specify the command-line API for runtimes. Looking at containerd's architecture from the lens of the OCI Runtime spec, the _shim_ is the OCI Runtime and runC is "just" an implementation detail of the io.containerd.runc.v2 runtime. When one configures a non-default runtime in Docker, what they're really doing is instructing Docker to create containers using the io.containerd.runc.v2 runtime with a configuration option telling the runtime that the runC binary is at some non-default path. Consequently, only OCI runtimes which are compatible with the io.containerd.runc.v2 shim, such as crun, can be used in this manner. Other OCI runtimes, including kata-containers v2, come with their own containerd shim and are not compatible with io.containerd.runc.v2. As Docker has not historically provided a way to select a non-default runtime which requires its own shim, runtimes such as kata-containers v2 could not be used with Docker. Allow other containerd shims to be used with Docker; no daemon configuration required. If the daemon is instructed to create a container with a runtime name which does not match any of the configured or stock runtimes, it passes the name along to containerd verbatim. A user can start a container with the kata-containers runtime, for example, simply by calling docker run --runtime io.containerd.kata.v2 Runtime names which containerd would interpret as a path to an arbitrary binary are disallowed. While handy for development and testing it is not strictly necessary and would allow anyone with Engine API access to trivially execute any binary on the host as root, so we have decided it would be safest for our users if it was not allowed. It is not yet possible to set an alternative containerd shim as the default runtime; it can only be configured per-container. Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-07-20 20:12:01 +00:00
configdOpts.BinaryName = configuredRuntime.Path
daemon: reload runtimes w/o breaking containers The existing runtimes reload logic went to great lengths to replace the directory containing runtime wrapper scripts as atomically as possible within the limitations of the Linux filesystem ABI. Trouble is, atomically swapping the wrapper scripts directory solves the wrong problem! The runtime configuration is "locked in" when a container is started, including the path to the runC binary. If a container is started with a runtime which requires a daemon-managed wrapper script and then the daemon is reloaded with a config which no longer requires the wrapper script (i.e. some args -> no args, or the runtime is dropped from the config), that container would become unmanageable. Any attempts to stop, exec or otherwise perform lifecycle management operations on the container are likely to fail due to the wrapper script no longer existing at its original path. Atomically swapping the wrapper scripts is also incompatible with the read-copy-update paradigm for reloading configuration. A handler in the daemon could retain a reference to the pre-reload configuration for an indeterminate amount of time after the daemon configuration has been reloaded and updated. It is possible for the daemon to attempt to start a container using a deleted wrapper script if a request to run a container races a reload. Solve the problem of deleting referenced wrapper scripts by ensuring that all wrapper scripts are *immutable* for the lifetime of the daemon process. Any given runtime wrapper script must always exist with the same contents, no matter how many times the daemon config is reloaded, or what changes are made to the config. This is accomplished by using everyone's favourite design pattern: content-addressable storage. Each wrapper script file name is suffixed with the SHA-256 digest of its contents to (probabilistically) guarantee immutability without needing any concurrency control. Stale runtime wrapper scripts are only cleaned up on the next daemon restart. Split the derived runtimes configuration from the user-supplied configuration to have a place to store derived state without mutating the user-supplied configuration or exposing daemon internals in API struct types. Hold the derived state and the user-supplied configuration in a single struct value so that they can be updated as an atomic unit. Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-08-31 20:12:30 +00:00
wantConfigdRuntime := &shimConfig{
Shim: stockRuntime.Shim,
Opts: &configdOpts,
}
daemon: support other containerd runtimes (MVP) Contrary to popular belief, the OCI Runtime specification does not specify the command-line API for runtimes. Looking at containerd's architecture from the lens of the OCI Runtime spec, the _shim_ is the OCI Runtime and runC is "just" an implementation detail of the io.containerd.runc.v2 runtime. When one configures a non-default runtime in Docker, what they're really doing is instructing Docker to create containers using the io.containerd.runc.v2 runtime with a configuration option telling the runtime that the runC binary is at some non-default path. Consequently, only OCI runtimes which are compatible with the io.containerd.runc.v2 shim, such as crun, can be used in this manner. Other OCI runtimes, including kata-containers v2, come with their own containerd shim and are not compatible with io.containerd.runc.v2. As Docker has not historically provided a way to select a non-default runtime which requires its own shim, runtimes such as kata-containers v2 could not be used with Docker. Allow other containerd shims to be used with Docker; no daemon configuration required. If the daemon is instructed to create a container with a runtime name which does not match any of the configured or stock runtimes, it passes the name along to containerd verbatim. A user can start a container with the kata-containers runtime, for example, simply by calling docker run --runtime io.containerd.kata.v2 Runtime names which containerd would interpret as a path to an arbitrary binary are disallowed. While handy for development and testing it is not strictly necessary and would allow anyone with Engine API access to trivially execute any binary on the host as root, so we have decided it would be safest for our users if it was not allowed. It is not yet possible to set an alternative containerd shim as the default runtime; it can only be configured per-container. Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-07-20 20:12:01 +00:00
for _, tt := range []struct {
name, runtime string
daemon: reload runtimes w/o breaking containers The existing runtimes reload logic went to great lengths to replace the directory containing runtime wrapper scripts as atomically as possible within the limitations of the Linux filesystem ABI. Trouble is, atomically swapping the wrapper scripts directory solves the wrong problem! The runtime configuration is "locked in" when a container is started, including the path to the runC binary. If a container is started with a runtime which requires a daemon-managed wrapper script and then the daemon is reloaded with a config which no longer requires the wrapper script (i.e. some args -> no args, or the runtime is dropped from the config), that container would become unmanageable. Any attempts to stop, exec or otherwise perform lifecycle management operations on the container are likely to fail due to the wrapper script no longer existing at its original path. Atomically swapping the wrapper scripts is also incompatible with the read-copy-update paradigm for reloading configuration. A handler in the daemon could retain a reference to the pre-reload configuration for an indeterminate amount of time after the daemon configuration has been reloaded and updated. It is possible for the daemon to attempt to start a container using a deleted wrapper script if a request to run a container races a reload. Solve the problem of deleting referenced wrapper scripts by ensuring that all wrapper scripts are *immutable* for the lifetime of the daemon process. Any given runtime wrapper script must always exist with the same contents, no matter how many times the daemon config is reloaded, or what changes are made to the config. This is accomplished by using everyone's favourite design pattern: content-addressable storage. Each wrapper script file name is suffixed with the SHA-256 digest of its contents to (probabilistically) guarantee immutability without needing any concurrency control. Stale runtime wrapper scripts are only cleaned up on the next daemon restart. Split the derived runtimes configuration from the user-supplied configuration to have a place to store derived state without mutating the user-supplied configuration or exposing daemon internals in API struct types. Hold the derived state and the user-supplied configuration in a single struct value so that they can be updated as an atomic unit. Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-08-31 20:12:30 +00:00
want *shimConfig
daemon: support other containerd runtimes (MVP) Contrary to popular belief, the OCI Runtime specification does not specify the command-line API for runtimes. Looking at containerd's architecture from the lens of the OCI Runtime spec, the _shim_ is the OCI Runtime and runC is "just" an implementation detail of the io.containerd.runc.v2 runtime. When one configures a non-default runtime in Docker, what they're really doing is instructing Docker to create containers using the io.containerd.runc.v2 runtime with a configuration option telling the runtime that the runC binary is at some non-default path. Consequently, only OCI runtimes which are compatible with the io.containerd.runc.v2 shim, such as crun, can be used in this manner. Other OCI runtimes, including kata-containers v2, come with their own containerd shim and are not compatible with io.containerd.runc.v2. As Docker has not historically provided a way to select a non-default runtime which requires its own shim, runtimes such as kata-containers v2 could not be used with Docker. Allow other containerd shims to be used with Docker; no daemon configuration required. If the daemon is instructed to create a container with a runtime name which does not match any of the configured or stock runtimes, it passes the name along to containerd verbatim. A user can start a container with the kata-containers runtime, for example, simply by calling docker run --runtime io.containerd.kata.v2 Runtime names which containerd would interpret as a path to an arbitrary binary are disallowed. While handy for development and testing it is not strictly necessary and would allow anyone with Engine API access to trivially execute any binary on the host as root, so we have decided it would be safest for our users if it was not allowed. It is not yet possible to set an alternative containerd shim as the default runtime; it can only be configured per-container. Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-07-20 20:12:01 +00:00
}{
{
daemon: reload runtimes w/o breaking containers The existing runtimes reload logic went to great lengths to replace the directory containing runtime wrapper scripts as atomically as possible within the limitations of the Linux filesystem ABI. Trouble is, atomically swapping the wrapper scripts directory solves the wrong problem! The runtime configuration is "locked in" when a container is started, including the path to the runC binary. If a container is started with a runtime which requires a daemon-managed wrapper script and then the daemon is reloaded with a config which no longer requires the wrapper script (i.e. some args -> no args, or the runtime is dropped from the config), that container would become unmanageable. Any attempts to stop, exec or otherwise perform lifecycle management operations on the container are likely to fail due to the wrapper script no longer existing at its original path. Atomically swapping the wrapper scripts is also incompatible with the read-copy-update paradigm for reloading configuration. A handler in the daemon could retain a reference to the pre-reload configuration for an indeterminate amount of time after the daemon configuration has been reloaded and updated. It is possible for the daemon to attempt to start a container using a deleted wrapper script if a request to run a container races a reload. Solve the problem of deleting referenced wrapper scripts by ensuring that all wrapper scripts are *immutable* for the lifetime of the daemon process. Any given runtime wrapper script must always exist with the same contents, no matter how many times the daemon config is reloaded, or what changes are made to the config. This is accomplished by using everyone's favourite design pattern: content-addressable storage. Each wrapper script file name is suffixed with the SHA-256 digest of its contents to (probabilistically) guarantee immutability without needing any concurrency control. Stale runtime wrapper scripts are only cleaned up on the next daemon restart. Split the derived runtimes configuration from the user-supplied configuration to have a place to store derived state without mutating the user-supplied configuration or exposing daemon internals in API struct types. Hold the derived state and the user-supplied configuration in a single struct value so that they can be updated as an atomic unit. Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-08-31 20:12:30 +00:00
name: "StockRuntime",
runtime: config.StockRuntimeName,
want: stockRuntime,
daemon: support other containerd runtimes (MVP) Contrary to popular belief, the OCI Runtime specification does not specify the command-line API for runtimes. Looking at containerd's architecture from the lens of the OCI Runtime spec, the _shim_ is the OCI Runtime and runC is "just" an implementation detail of the io.containerd.runc.v2 runtime. When one configures a non-default runtime in Docker, what they're really doing is instructing Docker to create containers using the io.containerd.runc.v2 runtime with a configuration option telling the runtime that the runC binary is at some non-default path. Consequently, only OCI runtimes which are compatible with the io.containerd.runc.v2 shim, such as crun, can be used in this manner. Other OCI runtimes, including kata-containers v2, come with their own containerd shim and are not compatible with io.containerd.runc.v2. As Docker has not historically provided a way to select a non-default runtime which requires its own shim, runtimes such as kata-containers v2 could not be used with Docker. Allow other containerd shims to be used with Docker; no daemon configuration required. If the daemon is instructed to create a container with a runtime name which does not match any of the configured or stock runtimes, it passes the name along to containerd verbatim. A user can start a container with the kata-containers runtime, for example, simply by calling docker run --runtime io.containerd.kata.v2 Runtime names which containerd would interpret as a path to an arbitrary binary are disallowed. While handy for development and testing it is not strictly necessary and would allow anyone with Engine API access to trivially execute any binary on the host as root, so we have decided it would be safest for our users if it was not allowed. It is not yet possible to set an alternative containerd shim as the default runtime; it can only be configured per-container. Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-07-20 20:12:01 +00:00
},
{
daemon: reload runtimes w/o breaking containers The existing runtimes reload logic went to great lengths to replace the directory containing runtime wrapper scripts as atomically as possible within the limitations of the Linux filesystem ABI. Trouble is, atomically swapping the wrapper scripts directory solves the wrong problem! The runtime configuration is "locked in" when a container is started, including the path to the runC binary. If a container is started with a runtime which requires a daemon-managed wrapper script and then the daemon is reloaded with a config which no longer requires the wrapper script (i.e. some args -> no args, or the runtime is dropped from the config), that container would become unmanageable. Any attempts to stop, exec or otherwise perform lifecycle management operations on the container are likely to fail due to the wrapper script no longer existing at its original path. Atomically swapping the wrapper scripts is also incompatible with the read-copy-update paradigm for reloading configuration. A handler in the daemon could retain a reference to the pre-reload configuration for an indeterminate amount of time after the daemon configuration has been reloaded and updated. It is possible for the daemon to attempt to start a container using a deleted wrapper script if a request to run a container races a reload. Solve the problem of deleting referenced wrapper scripts by ensuring that all wrapper scripts are *immutable* for the lifetime of the daemon process. Any given runtime wrapper script must always exist with the same contents, no matter how many times the daemon config is reloaded, or what changes are made to the config. This is accomplished by using everyone's favourite design pattern: content-addressable storage. Each wrapper script file name is suffixed with the SHA-256 digest of its contents to (probabilistically) guarantee immutability without needing any concurrency control. Stale runtime wrapper scripts are only cleaned up on the next daemon restart. Split the derived runtimes configuration from the user-supplied configuration to have a place to store derived state without mutating the user-supplied configuration or exposing daemon internals in API struct types. Hold the derived state and the user-supplied configuration in a single struct value so that they can be updated as an atomic unit. Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-08-31 20:12:30 +00:00
name: "ShimName",
runtime: "io.containerd.my-shim.v42",
want: &shimConfig{Shim: "io.containerd.my-shim.v42"},
daemon: support other containerd runtimes (MVP) Contrary to popular belief, the OCI Runtime specification does not specify the command-line API for runtimes. Looking at containerd's architecture from the lens of the OCI Runtime spec, the _shim_ is the OCI Runtime and runC is "just" an implementation detail of the io.containerd.runc.v2 runtime. When one configures a non-default runtime in Docker, what they're really doing is instructing Docker to create containers using the io.containerd.runc.v2 runtime with a configuration option telling the runtime that the runC binary is at some non-default path. Consequently, only OCI runtimes which are compatible with the io.containerd.runc.v2 shim, such as crun, can be used in this manner. Other OCI runtimes, including kata-containers v2, come with their own containerd shim and are not compatible with io.containerd.runc.v2. As Docker has not historically provided a way to select a non-default runtime which requires its own shim, runtimes such as kata-containers v2 could not be used with Docker. Allow other containerd shims to be used with Docker; no daemon configuration required. If the daemon is instructed to create a container with a runtime name which does not match any of the configured or stock runtimes, it passes the name along to containerd verbatim. A user can start a container with the kata-containers runtime, for example, simply by calling docker run --runtime io.containerd.kata.v2 Runtime names which containerd would interpret as a path to an arbitrary binary are disallowed. While handy for development and testing it is not strictly necessary and would allow anyone with Engine API access to trivially execute any binary on the host as root, so we have decided it would be safest for our users if it was not allowed. It is not yet possible to set an alternative containerd shim as the default runtime; it can only be configured per-container. Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-07-20 20:12:01 +00:00
},
{
// containerd is pretty loose about the format of runtime names. Perhaps too
// loose. The only requirements are that the name contain a dot and (depending
// on the containerd version) not start with a dot. It does not enforce any
// particular format of the dot-delimited components of the name.
daemon: reload runtimes w/o breaking containers The existing runtimes reload logic went to great lengths to replace the directory containing runtime wrapper scripts as atomically as possible within the limitations of the Linux filesystem ABI. Trouble is, atomically swapping the wrapper scripts directory solves the wrong problem! The runtime configuration is "locked in" when a container is started, including the path to the runC binary. If a container is started with a runtime which requires a daemon-managed wrapper script and then the daemon is reloaded with a config which no longer requires the wrapper script (i.e. some args -> no args, or the runtime is dropped from the config), that container would become unmanageable. Any attempts to stop, exec or otherwise perform lifecycle management operations on the container are likely to fail due to the wrapper script no longer existing at its original path. Atomically swapping the wrapper scripts is also incompatible with the read-copy-update paradigm for reloading configuration. A handler in the daemon could retain a reference to the pre-reload configuration for an indeterminate amount of time after the daemon configuration has been reloaded and updated. It is possible for the daemon to attempt to start a container using a deleted wrapper script if a request to run a container races a reload. Solve the problem of deleting referenced wrapper scripts by ensuring that all wrapper scripts are *immutable* for the lifetime of the daemon process. Any given runtime wrapper script must always exist with the same contents, no matter how many times the daemon config is reloaded, or what changes are made to the config. This is accomplished by using everyone's favourite design pattern: content-addressable storage. Each wrapper script file name is suffixed with the SHA-256 digest of its contents to (probabilistically) guarantee immutability without needing any concurrency control. Stale runtime wrapper scripts are only cleaned up on the next daemon restart. Split the derived runtimes configuration from the user-supplied configuration to have a place to store derived state without mutating the user-supplied configuration or exposing daemon internals in API struct types. Hold the derived state and the user-supplied configuration in a single struct value so that they can be updated as an atomic unit. Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-08-31 20:12:30 +00:00
name: "VersionlessShimName",
runtime: "io.containerd.my-shim",
want: &shimConfig{Shim: "io.containerd.my-shim"},
daemon: support other containerd runtimes (MVP) Contrary to popular belief, the OCI Runtime specification does not specify the command-line API for runtimes. Looking at containerd's architecture from the lens of the OCI Runtime spec, the _shim_ is the OCI Runtime and runC is "just" an implementation detail of the io.containerd.runc.v2 runtime. When one configures a non-default runtime in Docker, what they're really doing is instructing Docker to create containers using the io.containerd.runc.v2 runtime with a configuration option telling the runtime that the runC binary is at some non-default path. Consequently, only OCI runtimes which are compatible with the io.containerd.runc.v2 shim, such as crun, can be used in this manner. Other OCI runtimes, including kata-containers v2, come with their own containerd shim and are not compatible with io.containerd.runc.v2. As Docker has not historically provided a way to select a non-default runtime which requires its own shim, runtimes such as kata-containers v2 could not be used with Docker. Allow other containerd shims to be used with Docker; no daemon configuration required. If the daemon is instructed to create a container with a runtime name which does not match any of the configured or stock runtimes, it passes the name along to containerd verbatim. A user can start a container with the kata-containers runtime, for example, simply by calling docker run --runtime io.containerd.kata.v2 Runtime names which containerd would interpret as a path to an arbitrary binary are disallowed. While handy for development and testing it is not strictly necessary and would allow anyone with Engine API access to trivially execute any binary on the host as root, so we have decided it would be safest for our users if it was not allowed. It is not yet possible to set an alternative containerd shim as the default runtime; it can only be configured per-container. Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-07-20 20:12:01 +00:00
},
{
name: "IllformedShimName",
runtime: "myshim",
},
{
name: "EmptyString",
runtime: "",
daemon: consolidate runtimes config validation The daemon has made a habit of mutating the DefaultRuntime and Runtimes values in the Config struct to merge defaults. This would be fine if it was a part of the regular configuration loading and merging process, as is done with other config options. The trouble is it does so in surprising places, such as in functions with 'verify' or 'validate' in their name. It has been necessary in order to validate that the user has not defined a custom runtime named "runc" which would shadow the built-in runtime of the same name. Other daemon code depends on the runtime named "runc" always being defined in the config, but merging it with the user config at the same time as the other defaults are merged would trip the validation. The root of the issue is that the daemon has used the same config values for both validating the daemon runtime configuration as supplied by the user and for keeping track of which runtimes have been set up by the daemon. Now that a completely separate value is used for the latter purpose, surprising contortions are no longer required to make the validation work as intended. Consolidate the validation of the runtimes config and merging of the built-in runtimes into the daemon.setupRuntimes() function. Set the result of merging the built-in runtimes config and default default runtime on the returned runtimes struct, without back-propagating it onto the config.Config argument. Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-08-31 21:24:22 +00:00
want: stockRuntime,
daemon: support other containerd runtimes (MVP) Contrary to popular belief, the OCI Runtime specification does not specify the command-line API for runtimes. Looking at containerd's architecture from the lens of the OCI Runtime spec, the _shim_ is the OCI Runtime and runC is "just" an implementation detail of the io.containerd.runc.v2 runtime. When one configures a non-default runtime in Docker, what they're really doing is instructing Docker to create containers using the io.containerd.runc.v2 runtime with a configuration option telling the runtime that the runC binary is at some non-default path. Consequently, only OCI runtimes which are compatible with the io.containerd.runc.v2 shim, such as crun, can be used in this manner. Other OCI runtimes, including kata-containers v2, come with their own containerd shim and are not compatible with io.containerd.runc.v2. As Docker has not historically provided a way to select a non-default runtime which requires its own shim, runtimes such as kata-containers v2 could not be used with Docker. Allow other containerd shims to be used with Docker; no daemon configuration required. If the daemon is instructed to create a container with a runtime name which does not match any of the configured or stock runtimes, it passes the name along to containerd verbatim. A user can start a container with the kata-containers runtime, for example, simply by calling docker run --runtime io.containerd.kata.v2 Runtime names which containerd would interpret as a path to an arbitrary binary are disallowed. While handy for development and testing it is not strictly necessary and would allow anyone with Engine API access to trivially execute any binary on the host as root, so we have decided it would be safest for our users if it was not allowed. It is not yet possible to set an alternative containerd shim as the default runtime; it can only be configured per-container. Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-07-20 20:12:01 +00:00
},
{
name: "PathToShim",
runtime: "/path/to/runc",
},
{
name: "PathToShimName",
runtime: "/path/to/io.containerd.runc.v2",
},
{
name: "RelPathToShim",
runtime: "my/io.containerd.runc.v2",
},
{
daemon: reload runtimes w/o breaking containers The existing runtimes reload logic went to great lengths to replace the directory containing runtime wrapper scripts as atomically as possible within the limitations of the Linux filesystem ABI. Trouble is, atomically swapping the wrapper scripts directory solves the wrong problem! The runtime configuration is "locked in" when a container is started, including the path to the runC binary. If a container is started with a runtime which requires a daemon-managed wrapper script and then the daemon is reloaded with a config which no longer requires the wrapper script (i.e. some args -> no args, or the runtime is dropped from the config), that container would become unmanageable. Any attempts to stop, exec or otherwise perform lifecycle management operations on the container are likely to fail due to the wrapper script no longer existing at its original path. Atomically swapping the wrapper scripts is also incompatible with the read-copy-update paradigm for reloading configuration. A handler in the daemon could retain a reference to the pre-reload configuration for an indeterminate amount of time after the daemon configuration has been reloaded and updated. It is possible for the daemon to attempt to start a container using a deleted wrapper script if a request to run a container races a reload. Solve the problem of deleting referenced wrapper scripts by ensuring that all wrapper scripts are *immutable* for the lifetime of the daemon process. Any given runtime wrapper script must always exist with the same contents, no matter how many times the daemon config is reloaded, or what changes are made to the config. This is accomplished by using everyone's favourite design pattern: content-addressable storage. Each wrapper script file name is suffixed with the SHA-256 digest of its contents to (probabilistically) guarantee immutability without needing any concurrency control. Stale runtime wrapper scripts are only cleaned up on the next daemon restart. Split the derived runtimes configuration from the user-supplied configuration to have a place to store derived state without mutating the user-supplied configuration or exposing daemon internals in API struct types. Hold the derived state and the user-supplied configuration in a single struct value so that they can be updated as an atomic unit. Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-08-31 20:12:30 +00:00
name: "ConfiguredRuntime",
runtime: configuredRtName,
want: wantConfigdRuntime,
},
{
daemon: reload runtimes w/o breaking containers The existing runtimes reload logic went to great lengths to replace the directory containing runtime wrapper scripts as atomically as possible within the limitations of the Linux filesystem ABI. Trouble is, atomically swapping the wrapper scripts directory solves the wrong problem! The runtime configuration is "locked in" when a container is started, including the path to the runC binary. If a container is started with a runtime which requires a daemon-managed wrapper script and then the daemon is reloaded with a config which no longer requires the wrapper script (i.e. some args -> no args, or the runtime is dropped from the config), that container would become unmanageable. Any attempts to stop, exec or otherwise perform lifecycle management operations on the container are likely to fail due to the wrapper script no longer existing at its original path. Atomically swapping the wrapper scripts is also incompatible with the read-copy-update paradigm for reloading configuration. A handler in the daemon could retain a reference to the pre-reload configuration for an indeterminate amount of time after the daemon configuration has been reloaded and updated. It is possible for the daemon to attempt to start a container using a deleted wrapper script if a request to run a container races a reload. Solve the problem of deleting referenced wrapper scripts by ensuring that all wrapper scripts are *immutable* for the lifetime of the daemon process. Any given runtime wrapper script must always exist with the same contents, no matter how many times the daemon config is reloaded, or what changes are made to the config. This is accomplished by using everyone's favourite design pattern: content-addressable storage. Each wrapper script file name is suffixed with the SHA-256 digest of its contents to (probabilistically) guarantee immutability without needing any concurrency control. Stale runtime wrapper scripts are only cleaned up on the next daemon restart. Split the derived runtimes configuration from the user-supplied configuration to have a place to store derived state without mutating the user-supplied configuration or exposing daemon internals in API struct types. Hold the derived state and the user-supplied configuration in a single struct value so that they can be updated as an atomic unit. Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-08-31 20:12:30 +00:00
name: "ShimWithOpts",
runtime: shimWithOptsName,
want: &shimConfig{
Shim: shimWithOpts.Type,
Opts: &v2runcoptions.Options{IoUid: 42},
},
},
{
daemon: reload runtimes w/o breaking containers The existing runtimes reload logic went to great lengths to replace the directory containing runtime wrapper scripts as atomically as possible within the limitations of the Linux filesystem ABI. Trouble is, atomically swapping the wrapper scripts directory solves the wrong problem! The runtime configuration is "locked in" when a container is started, including the path to the runC binary. If a container is started with a runtime which requires a daemon-managed wrapper script and then the daemon is reloaded with a config which no longer requires the wrapper script (i.e. some args -> no args, or the runtime is dropped from the config), that container would become unmanageable. Any attempts to stop, exec or otherwise perform lifecycle management operations on the container are likely to fail due to the wrapper script no longer existing at its original path. Atomically swapping the wrapper scripts is also incompatible with the read-copy-update paradigm for reloading configuration. A handler in the daemon could retain a reference to the pre-reload configuration for an indeterminate amount of time after the daemon configuration has been reloaded and updated. It is possible for the daemon to attempt to start a container using a deleted wrapper script if a request to run a container races a reload. Solve the problem of deleting referenced wrapper scripts by ensuring that all wrapper scripts are *immutable* for the lifetime of the daemon process. Any given runtime wrapper script must always exist with the same contents, no matter how many times the daemon config is reloaded, or what changes are made to the config. This is accomplished by using everyone's favourite design pattern: content-addressable storage. Each wrapper script file name is suffixed with the SHA-256 digest of its contents to (probabilistically) guarantee immutability without needing any concurrency control. Stale runtime wrapper scripts are only cleaned up on the next daemon restart. Split the derived runtimes configuration from the user-supplied configuration to have a place to store derived state without mutating the user-supplied configuration or exposing daemon internals in API struct types. Hold the derived state and the user-supplied configuration in a single struct value so that they can be updated as an atomic unit. Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-08-31 20:12:30 +00:00
name: "ShimAlias",
runtime: shimAliasName,
want: &shimConfig{Shim: shimAlias.Type},
},
{
daemon: reload runtimes w/o breaking containers The existing runtimes reload logic went to great lengths to replace the directory containing runtime wrapper scripts as atomically as possible within the limitations of the Linux filesystem ABI. Trouble is, atomically swapping the wrapper scripts directory solves the wrong problem! The runtime configuration is "locked in" when a container is started, including the path to the runC binary. If a container is started with a runtime which requires a daemon-managed wrapper script and then the daemon is reloaded with a config which no longer requires the wrapper script (i.e. some args -> no args, or the runtime is dropped from the config), that container would become unmanageable. Any attempts to stop, exec or otherwise perform lifecycle management operations on the container are likely to fail due to the wrapper script no longer existing at its original path. Atomically swapping the wrapper scripts is also incompatible with the read-copy-update paradigm for reloading configuration. A handler in the daemon could retain a reference to the pre-reload configuration for an indeterminate amount of time after the daemon configuration has been reloaded and updated. It is possible for the daemon to attempt to start a container using a deleted wrapper script if a request to run a container races a reload. Solve the problem of deleting referenced wrapper scripts by ensuring that all wrapper scripts are *immutable* for the lifetime of the daemon process. Any given runtime wrapper script must always exist with the same contents, no matter how many times the daemon config is reloaded, or what changes are made to the config. This is accomplished by using everyone's favourite design pattern: content-addressable storage. Each wrapper script file name is suffixed with the SHA-256 digest of its contents to (probabilistically) guarantee immutability without needing any concurrency control. Stale runtime wrapper scripts are only cleaned up on the next daemon restart. Split the derived runtimes configuration from the user-supplied configuration to have a place to store derived state without mutating the user-supplied configuration or exposing daemon internals in API struct types. Hold the derived state and the user-supplied configuration in a single struct value so that they can be updated as an atomic unit. Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-08-31 20:12:30 +00:00
name: "ConfiguredShimByPath",
runtime: configuredShimByPathName,
want: &shimConfig{Shim: configuredShimByPath.Type},
daemon: support other containerd runtimes (MVP) Contrary to popular belief, the OCI Runtime specification does not specify the command-line API for runtimes. Looking at containerd's architecture from the lens of the OCI Runtime spec, the _shim_ is the OCI Runtime and runC is "just" an implementation detail of the io.containerd.runc.v2 runtime. When one configures a non-default runtime in Docker, what they're really doing is instructing Docker to create containers using the io.containerd.runc.v2 runtime with a configuration option telling the runtime that the runC binary is at some non-default path. Consequently, only OCI runtimes which are compatible with the io.containerd.runc.v2 shim, such as crun, can be used in this manner. Other OCI runtimes, including kata-containers v2, come with their own containerd shim and are not compatible with io.containerd.runc.v2. As Docker has not historically provided a way to select a non-default runtime which requires its own shim, runtimes such as kata-containers v2 could not be used with Docker. Allow other containerd shims to be used with Docker; no daemon configuration required. If the daemon is instructed to create a container with a runtime name which does not match any of the configured or stock runtimes, it passes the name along to containerd verbatim. A user can start a container with the kata-containers runtime, for example, simply by calling docker run --runtime io.containerd.kata.v2 Runtime names which containerd would interpret as a path to an arbitrary binary are disallowed. While handy for development and testing it is not strictly necessary and would allow anyone with Engine API access to trivially execute any binary on the host as root, so we have decided it would be safest for our users if it was not allowed. It is not yet possible to set an alternative containerd shim as the default runtime; it can only be configured per-container. Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-07-20 20:12:01 +00:00
},
} {
tt := tt
t.Run(tt.name, func(t *testing.T) {
daemon: reload runtimes w/o breaking containers The existing runtimes reload logic went to great lengths to replace the directory containing runtime wrapper scripts as atomically as possible within the limitations of the Linux filesystem ABI. Trouble is, atomically swapping the wrapper scripts directory solves the wrong problem! The runtime configuration is "locked in" when a container is started, including the path to the runC binary. If a container is started with a runtime which requires a daemon-managed wrapper script and then the daemon is reloaded with a config which no longer requires the wrapper script (i.e. some args -> no args, or the runtime is dropped from the config), that container would become unmanageable. Any attempts to stop, exec or otherwise perform lifecycle management operations on the container are likely to fail due to the wrapper script no longer existing at its original path. Atomically swapping the wrapper scripts is also incompatible with the read-copy-update paradigm for reloading configuration. A handler in the daemon could retain a reference to the pre-reload configuration for an indeterminate amount of time after the daemon configuration has been reloaded and updated. It is possible for the daemon to attempt to start a container using a deleted wrapper script if a request to run a container races a reload. Solve the problem of deleting referenced wrapper scripts by ensuring that all wrapper scripts are *immutable* for the lifetime of the daemon process. Any given runtime wrapper script must always exist with the same contents, no matter how many times the daemon config is reloaded, or what changes are made to the config. This is accomplished by using everyone's favourite design pattern: content-addressable storage. Each wrapper script file name is suffixed with the SHA-256 digest of its contents to (probabilistically) guarantee immutability without needing any concurrency control. Stale runtime wrapper scripts are only cleaned up on the next daemon restart. Split the derived runtimes configuration from the user-supplied configuration to have a place to store derived state without mutating the user-supplied configuration or exposing daemon internals in API struct types. Hold the derived state and the user-supplied configuration in a single struct value so that they can be updated as an atomic unit. Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-08-31 20:12:30 +00:00
shim, opts, err := runtimes.Get(tt.runtime)
if tt.want != nil {
daemon: support other containerd runtimes (MVP) Contrary to popular belief, the OCI Runtime specification does not specify the command-line API for runtimes. Looking at containerd's architecture from the lens of the OCI Runtime spec, the _shim_ is the OCI Runtime and runC is "just" an implementation detail of the io.containerd.runc.v2 runtime. When one configures a non-default runtime in Docker, what they're really doing is instructing Docker to create containers using the io.containerd.runc.v2 runtime with a configuration option telling the runtime that the runC binary is at some non-default path. Consequently, only OCI runtimes which are compatible with the io.containerd.runc.v2 shim, such as crun, can be used in this manner. Other OCI runtimes, including kata-containers v2, come with their own containerd shim and are not compatible with io.containerd.runc.v2. As Docker has not historically provided a way to select a non-default runtime which requires its own shim, runtimes such as kata-containers v2 could not be used with Docker. Allow other containerd shims to be used with Docker; no daemon configuration required. If the daemon is instructed to create a container with a runtime name which does not match any of the configured or stock runtimes, it passes the name along to containerd verbatim. A user can start a container with the kata-containers runtime, for example, simply by calling docker run --runtime io.containerd.kata.v2 Runtime names which containerd would interpret as a path to an arbitrary binary are disallowed. While handy for development and testing it is not strictly necessary and would allow anyone with Engine API access to trivially execute any binary on the host as root, so we have decided it would be safest for our users if it was not allowed. It is not yet possible to set an alternative containerd shim as the default runtime; it can only be configured per-container. Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-07-20 20:12:01 +00:00
assert.Check(t, err)
daemon: reload runtimes w/o breaking containers The existing runtimes reload logic went to great lengths to replace the directory containing runtime wrapper scripts as atomically as possible within the limitations of the Linux filesystem ABI. Trouble is, atomically swapping the wrapper scripts directory solves the wrong problem! The runtime configuration is "locked in" when a container is started, including the path to the runC binary. If a container is started with a runtime which requires a daemon-managed wrapper script and then the daemon is reloaded with a config which no longer requires the wrapper script (i.e. some args -> no args, or the runtime is dropped from the config), that container would become unmanageable. Any attempts to stop, exec or otherwise perform lifecycle management operations on the container are likely to fail due to the wrapper script no longer existing at its original path. Atomically swapping the wrapper scripts is also incompatible with the read-copy-update paradigm for reloading configuration. A handler in the daemon could retain a reference to the pre-reload configuration for an indeterminate amount of time after the daemon configuration has been reloaded and updated. It is possible for the daemon to attempt to start a container using a deleted wrapper script if a request to run a container races a reload. Solve the problem of deleting referenced wrapper scripts by ensuring that all wrapper scripts are *immutable* for the lifetime of the daemon process. Any given runtime wrapper script must always exist with the same contents, no matter how many times the daemon config is reloaded, or what changes are made to the config. This is accomplished by using everyone's favourite design pattern: content-addressable storage. Each wrapper script file name is suffixed with the SHA-256 digest of its contents to (probabilistically) guarantee immutability without needing any concurrency control. Stale runtime wrapper scripts are only cleaned up on the next daemon restart. Split the derived runtimes configuration from the user-supplied configuration to have a place to store derived state without mutating the user-supplied configuration or exposing daemon internals in API struct types. Hold the derived state and the user-supplied configuration in a single struct value so that they can be updated as an atomic unit. Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-08-31 20:12:30 +00:00
got := &shimConfig{Shim: shim, Opts: opts}
assert.Check(t, is.DeepEqual(got, tt.want))
daemon: support other containerd runtimes (MVP) Contrary to popular belief, the OCI Runtime specification does not specify the command-line API for runtimes. Looking at containerd's architecture from the lens of the OCI Runtime spec, the _shim_ is the OCI Runtime and runC is "just" an implementation detail of the io.containerd.runc.v2 runtime. When one configures a non-default runtime in Docker, what they're really doing is instructing Docker to create containers using the io.containerd.runc.v2 runtime with a configuration option telling the runtime that the runC binary is at some non-default path. Consequently, only OCI runtimes which are compatible with the io.containerd.runc.v2 shim, such as crun, can be used in this manner. Other OCI runtimes, including kata-containers v2, come with their own containerd shim and are not compatible with io.containerd.runc.v2. As Docker has not historically provided a way to select a non-default runtime which requires its own shim, runtimes such as kata-containers v2 could not be used with Docker. Allow other containerd shims to be used with Docker; no daemon configuration required. If the daemon is instructed to create a container with a runtime name which does not match any of the configured or stock runtimes, it passes the name along to containerd verbatim. A user can start a container with the kata-containers runtime, for example, simply by calling docker run --runtime io.containerd.kata.v2 Runtime names which containerd would interpret as a path to an arbitrary binary are disallowed. While handy for development and testing it is not strictly necessary and would allow anyone with Engine API access to trivially execute any binary on the host as root, so we have decided it would be safest for our users if it was not allowed. It is not yet possible to set an alternative containerd shim as the default runtime; it can only be configured per-container. Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-07-20 20:12:01 +00:00
} else {
daemon: reload runtimes w/o breaking containers The existing runtimes reload logic went to great lengths to replace the directory containing runtime wrapper scripts as atomically as possible within the limitations of the Linux filesystem ABI. Trouble is, atomically swapping the wrapper scripts directory solves the wrong problem! The runtime configuration is "locked in" when a container is started, including the path to the runC binary. If a container is started with a runtime which requires a daemon-managed wrapper script and then the daemon is reloaded with a config which no longer requires the wrapper script (i.e. some args -> no args, or the runtime is dropped from the config), that container would become unmanageable. Any attempts to stop, exec or otherwise perform lifecycle management operations on the container are likely to fail due to the wrapper script no longer existing at its original path. Atomically swapping the wrapper scripts is also incompatible with the read-copy-update paradigm for reloading configuration. A handler in the daemon could retain a reference to the pre-reload configuration for an indeterminate amount of time after the daemon configuration has been reloaded and updated. It is possible for the daemon to attempt to start a container using a deleted wrapper script if a request to run a container races a reload. Solve the problem of deleting referenced wrapper scripts by ensuring that all wrapper scripts are *immutable* for the lifetime of the daemon process. Any given runtime wrapper script must always exist with the same contents, no matter how many times the daemon config is reloaded, or what changes are made to the config. This is accomplished by using everyone's favourite design pattern: content-addressable storage. Each wrapper script file name is suffixed with the SHA-256 digest of its contents to (probabilistically) guarantee immutability without needing any concurrency control. Stale runtime wrapper scripts are only cleaned up on the next daemon restart. Split the derived runtimes configuration from the user-supplied configuration to have a place to store derived state without mutating the user-supplied configuration or exposing daemon internals in API struct types. Hold the derived state and the user-supplied configuration in a single struct value so that they can be updated as an atomic unit. Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-08-31 20:12:30 +00:00
assert.Check(t, is.Equal(shim, ""))
assert.Check(t, is.Nil(opts))
assert.Check(t, errdefs.IsInvalidParameter(err), "[%T] %[1]v", err)
daemon: support other containerd runtimes (MVP) Contrary to popular belief, the OCI Runtime specification does not specify the command-line API for runtimes. Looking at containerd's architecture from the lens of the OCI Runtime spec, the _shim_ is the OCI Runtime and runC is "just" an implementation detail of the io.containerd.runc.v2 runtime. When one configures a non-default runtime in Docker, what they're really doing is instructing Docker to create containers using the io.containerd.runc.v2 runtime with a configuration option telling the runtime that the runC binary is at some non-default path. Consequently, only OCI runtimes which are compatible with the io.containerd.runc.v2 shim, such as crun, can be used in this manner. Other OCI runtimes, including kata-containers v2, come with their own containerd shim and are not compatible with io.containerd.runc.v2. As Docker has not historically provided a way to select a non-default runtime which requires its own shim, runtimes such as kata-containers v2 could not be used with Docker. Allow other containerd shims to be used with Docker; no daemon configuration required. If the daemon is instructed to create a container with a runtime name which does not match any of the configured or stock runtimes, it passes the name along to containerd verbatim. A user can start a container with the kata-containers runtime, for example, simply by calling docker run --runtime io.containerd.kata.v2 Runtime names which containerd would interpret as a path to an arbitrary binary are disallowed. While handy for development and testing it is not strictly necessary and would allow anyone with Engine API access to trivially execute any binary on the host as root, so we have decided it would be safest for our users if it was not allowed. It is not yet possible to set an alternative containerd shim as the default runtime; it can only be configured per-container. Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-07-20 20:12:01 +00:00
}
})
}
daemon: reload runtimes w/o breaking containers The existing runtimes reload logic went to great lengths to replace the directory containing runtime wrapper scripts as atomically as possible within the limitations of the Linux filesystem ABI. Trouble is, atomically swapping the wrapper scripts directory solves the wrong problem! The runtime configuration is "locked in" when a container is started, including the path to the runC binary. If a container is started with a runtime which requires a daemon-managed wrapper script and then the daemon is reloaded with a config which no longer requires the wrapper script (i.e. some args -> no args, or the runtime is dropped from the config), that container would become unmanageable. Any attempts to stop, exec or otherwise perform lifecycle management operations on the container are likely to fail due to the wrapper script no longer existing at its original path. Atomically swapping the wrapper scripts is also incompatible with the read-copy-update paradigm for reloading configuration. A handler in the daemon could retain a reference to the pre-reload configuration for an indeterminate amount of time after the daemon configuration has been reloaded and updated. It is possible for the daemon to attempt to start a container using a deleted wrapper script if a request to run a container races a reload. Solve the problem of deleting referenced wrapper scripts by ensuring that all wrapper scripts are *immutable* for the lifetime of the daemon process. Any given runtime wrapper script must always exist with the same contents, no matter how many times the daemon config is reloaded, or what changes are made to the config. This is accomplished by using everyone's favourite design pattern: content-addressable storage. Each wrapper script file name is suffixed with the SHA-256 digest of its contents to (probabilistically) guarantee immutability without needing any concurrency control. Stale runtime wrapper scripts are only cleaned up on the next daemon restart. Split the derived runtimes configuration from the user-supplied configuration to have a place to store derived state without mutating the user-supplied configuration or exposing daemon internals in API struct types. Hold the derived state and the user-supplied configuration in a single struct value so that they can be updated as an atomic unit. Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-08-31 20:12:30 +00:00
t.Run("RuntimeWithArgs", func(t *testing.T) {
shim, opts, err := runtimes.Get(rtWithArgsName)
assert.Check(t, err)
assert.Check(t, is.Equal(shim, stockRuntime.Shim))
runcopts, ok := opts.(*v2runcoptions.Options)
if assert.Check(t, ok, "runtimes.Get() opts = type %T, want *v2runcoptions.Options", opts) {
wrapper, err := os.ReadFile(runcopts.BinaryName)
if assert.Check(t, err) {
assert.Check(t, is.Contains(string(wrapper),
strings.Join(append([]string{rtWithArgs.Path}, rtWithArgs.Args...), " ")))
}
}
})
}
func TestGetRuntime_PreflightCheck(t *testing.T) {
cfg, err := config.New()
assert.NilError(t, err)
cfg.Root = t.TempDir()
cfg.Runtimes = map[string]types.Runtime{
"path-only": {
Path: "/usr/local/bin/file-not-found",
},
"with-args": {
Path: "/usr/local/bin/file-not-found",
Args: []string{"--arg"},
},
}
assert.NilError(t, initRuntimesDir(cfg))
runtimes, err := setupRuntimes(cfg)
assert.NilError(t, err, "runtime paths should not be validated during setupRuntimes()")
t.Run("PathOnly", func(t *testing.T) {
_, _, err := runtimes.Get("path-only")
assert.NilError(t, err, "custom runtimes without wrapper scripts should not have pre-flight checks")
})
t.Run("WithArgs", func(t *testing.T) {
_, _, err := runtimes.Get("with-args")
assert.ErrorIs(t, err, fs.ErrNotExist)
})
}
// TestRuntimeWrapping checks that reloading runtime config does not delete or
// modify existing wrapper scripts, which could break lifecycle management of
// existing containers.
func TestRuntimeWrapping(t *testing.T) {
cfg, err := config.New()
assert.NilError(t, err)
cfg.Root = t.TempDir()
cfg.Runtimes = map[string]types.Runtime{
"change-args": {
Path: "/bin/true",
Args: []string{"foo", "bar"},
},
"dupe": {
Path: "/bin/true",
Args: []string{"foo", "bar"},
},
"change-path": {
Path: "/bin/true",
Args: []string{"baz"},
},
"drop-args": {
Path: "/bin/true",
Args: []string{"some", "arguments"},
},
"goes-away": {
Path: "/bin/true",
Args: []string{"bye"},
},
}
assert.NilError(t, initRuntimesDir(cfg))
rt, err := setupRuntimes(cfg)
assert.Check(t, err)
type WrapperInfo struct{ BinaryName, Content string }
wrappers := make(map[string]WrapperInfo)
for name := range cfg.Runtimes {
_, opts, err := rt.Get(name)
if assert.Check(t, err, "rt.Get(%q)", name) {
binary := opts.(*v2runcoptions.Options).BinaryName
content, err := os.ReadFile(binary)
assert.Check(t, err, "could not read wrapper script contents for runtime %q", binary)
wrappers[name] = WrapperInfo{BinaryName: binary, Content: string(content)}
}
}
cfg.Runtimes["change-args"] = types.Runtime{
Path: cfg.Runtimes["change-args"].Path,
Args: []string{"baz", "quux"},
}
cfg.Runtimes["change-path"] = types.Runtime{
Path: "/bin/false",
Args: cfg.Runtimes["change-path"].Args,
}
cfg.Runtimes["drop-args"] = types.Runtime{
Path: cfg.Runtimes["drop-args"].Path,
}
delete(cfg.Runtimes, "goes-away")
_, err = setupRuntimes(cfg)
assert.Check(t, err)
for name, info := range wrappers {
t.Run(name, func(t *testing.T) {
content, err := os.ReadFile(info.BinaryName)
assert.NilError(t, err)
assert.DeepEqual(t, info.Content, string(content))
})
}
daemon: support other containerd runtimes (MVP) Contrary to popular belief, the OCI Runtime specification does not specify the command-line API for runtimes. Looking at containerd's architecture from the lens of the OCI Runtime spec, the _shim_ is the OCI Runtime and runC is "just" an implementation detail of the io.containerd.runc.v2 runtime. When one configures a non-default runtime in Docker, what they're really doing is instructing Docker to create containers using the io.containerd.runc.v2 runtime with a configuration option telling the runtime that the runC binary is at some non-default path. Consequently, only OCI runtimes which are compatible with the io.containerd.runc.v2 shim, such as crun, can be used in this manner. Other OCI runtimes, including kata-containers v2, come with their own containerd shim and are not compatible with io.containerd.runc.v2. As Docker has not historically provided a way to select a non-default runtime which requires its own shim, runtimes such as kata-containers v2 could not be used with Docker. Allow other containerd shims to be used with Docker; no daemon configuration required. If the daemon is instructed to create a container with a runtime name which does not match any of the configured or stock runtimes, it passes the name along to containerd verbatim. A user can start a container with the kata-containers runtime, for example, simply by calling docker run --runtime io.containerd.kata.v2 Runtime names which containerd would interpret as a path to an arbitrary binary are disallowed. While handy for development and testing it is not strictly necessary and would allow anyone with Engine API access to trivially execute any binary on the host as root, so we have decided it would be safest for our users if it was not allowed. It is not yet possible to set an alternative containerd shim as the default runtime; it can only be configured per-container. Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-07-20 20:12:01 +00:00
}