The Moby Project - a collaborative project for the container ecosystem to assemble container-based systems
Find a file
Cory Snider d222bf097c daemon: reload runtimes w/o breaking containers
The existing runtimes reload logic went to great lengths to replace the
directory containing runtime wrapper scripts as atomically as possible
within the limitations of the Linux filesystem ABI. Trouble is,
atomically swapping the wrapper scripts directory solves the wrong
problem! The runtime configuration is "locked in" when a container is
started, including the path to the runC binary. If a container is
started with a runtime which requires a daemon-managed wrapper script
and then the daemon is reloaded with a config which no longer requires
the wrapper script (i.e. some args -> no args, or the runtime is dropped
from the config), that container would become unmanageable. Any attempts
to stop, exec or otherwise perform lifecycle management operations on
the container are likely to fail due to the wrapper script no longer
existing at its original path.

Atomically swapping the wrapper scripts is also incompatible with the
read-copy-update paradigm for reloading configuration. A handler in the
daemon could retain a reference to the pre-reload configuration for an
indeterminate amount of time after the daemon configuration has been
reloaded and updated. It is possible for the daemon to attempt to start
a container using a deleted wrapper script if a request to run a
container races a reload.

Solve the problem of deleting referenced wrapper scripts by ensuring
that all wrapper scripts are *immutable* for the lifetime of the daemon
process. Any given runtime wrapper script must always exist with the
same contents, no matter how many times the daemon config is reloaded,
or what changes are made to the config. This is accomplished by using
everyone's favourite design pattern: content-addressable storage. Each
wrapper script file name is suffixed with the SHA-256 digest of its
contents to (probabilistically) guarantee immutability without needing
any concurrency control. Stale runtime wrapper scripts are only cleaned
up on the next daemon restart.

Split the derived runtimes configuration from the user-supplied
configuration to have a place to store derived state without mutating
the user-supplied configuration or exposing daemon internals in API
struct types. Hold the derived state and the user-supplied configuration
in a single struct value so that they can be updated as an atomic unit.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-06-01 14:45:25 -04:00
.github hack: generated files update and validation 2023-05-29 03:28:35 +02:00
api daemon: reload runtimes w/o breaking containers 2023-06-01 14:45:25 -04:00
builder Merge pull request #45628 from thaJeztah/context_simplify 2023-05-30 13:40:42 +02:00
cli cli: remove cli/config package, integrate into cmd/dockerd 2022-12-14 12:52:40 +01:00
client remove pre-go1.17 build-tags 2023-05-19 20:38:51 +02:00
cmd daemon: reload runtimes w/o breaking containers 2023-06-01 14:45:25 -04:00
container Support recursively read-only (RRO) mounts 2023-05-26 01:58:24 +09:00
contrib Merge pull request #45548 from tianon/sysvinit-debian 2023-05-22 16:08:55 +02:00
daemon daemon: reload runtimes w/o breaking containers 2023-06-01 14:45:25 -04:00
distribution remove pre-go1.17 build-tags 2023-05-19 20:38:51 +02:00
dockerversion fix some comments 2023-04-25 13:39:28 +08:00
docs Support recursively read-only (RRO) mounts 2023-05-26 01:58:24 +09:00
errdefs errdefs: FromStatusCode() don't log "FIXME" debug message 2022-12-20 16:03:46 +01:00
hack Update blogpost URL 2023-05-29 22:37:09 +02:00
image docs: fix JSON format error 2023-02-03 08:16:48 +00:00
integration vendor: github.com/opencontainers/image-spec v1.1.0-rc3 2023-05-26 02:34:50 +02:00
integration-cli remove pre-go1.17 build-tags 2023-05-19 20:38:51 +02:00
internal replace gogofast with gogofaster extension 2023-05-29 03:28:35 +02:00
layer remove pre-go1.17 build-tags 2023-05-19 20:38:51 +02:00
libcontainerd libcontainerd: work around exec start bug in c8d 2023-05-25 16:00:29 -04:00
libnetwork Merge pull request #45657 from corhere/libn/setup-resolver-with-verbose-iptables 2023-05-30 21:44:14 +02:00
oci remove pre-go1.17 build-tags 2023-05-19 20:38:51 +02:00
opts remove pre-go1.17 build-tags 2023-05-19 20:38:51 +02:00
pkg remove pre-go1.17 build-tags 2023-05-19 20:38:51 +02:00
plugin remove pre-go1.17 build-tags 2023-05-19 20:38:51 +02:00
profiles remove pre-go1.17 build-tags 2023-05-19 20:38:51 +02:00
project remove deprecated devicemapper storage-driver 2023-04-20 23:51:54 +02:00
quota remove pre-go1.17 build-tags 2023-05-19 20:38:51 +02:00
reference reference: add test-coverage for digested references 2023-03-06 13:01:35 +01:00
registry daemon: make config reloading more transactional 2023-06-01 14:45:24 -04:00
reports Fix typos 2018-05-16 09:15:43 +08:00
restartmanager restartmanager: remove RestartManager interface 2022-12-28 09:36:58 +01:00
runconfig remove pre-go1.17 build-tags 2023-05-19 20:38:51 +02:00
testutil integration: Add TestImageInspectEmptyTagsAndDigests 2023-05-23 15:50:36 +02:00
vendor daemon: read-copy-update the daemon config 2023-06-01 14:45:24 -04:00
volume Support recursively read-only (RRO) mounts 2023-05-26 01:58:24 +09:00
.dockerignore ignorefiles: cleanup 2022-11-21 18:43:42 -07:00
.gitattributes chore: fix linguist for Dockerfile 2022-04-27 06:38:41 +02:00
.gitignore hack: introduce validate/no-module 2022-12-12 18:39:06 -07:00
.mailmap .mailmap: update with my new email 2023-05-09 15:14:40 -06:00
AUTHORS AUTHORS: regenerate 2023-05-09 15:14:50 -06:00
codecov.yml codecov: disable "patch" status 2022-06-07 17:24:46 +02:00
CONTRIBUTING.md CONTRIBUTING.md: drop reference to the Derek GitHub bot 2023-01-10 15:42:09 -07:00
docker-bake.hcl ci: bin-image workflow 2023-05-11 15:52:41 +02:00
Dockerfile Dockerfile: use COPY --link to copy artifacts from build-stages 2023-05-31 11:52:18 +02:00
Dockerfile.e2e hack: Rename .ensure-emptyfs to .build-empty-images 2023-05-23 15:50:35 +02:00
Dockerfile.simple update go to go1.20.4 2023-05-03 20:42:33 +02:00
Dockerfile.windows update containerd binary to v1.7.1 2023-05-15 12:52:50 +01:00
Jenkinsfile Jenkinsfile: Fix dev image build fox ppc64le/s390x archs 2022-11-29 11:59:09 +01:00
LICENSE Update LICENSE 2018-09-12 14:27:53 +01:00
MAINTAINERS Add akerouanton as curator 2023-03-28 11:01:28 +02:00
Makefile hack: generated files update and validation 2023-05-29 03:28:35 +02:00
NOTICE switch kr/pty to creack/pty v1.1.7 2019-07-29 16:59:08 -07:00
README.md Fix grammar in README.md 2022-11-10 19:49:41 +00:00
ROADMAP.md Fix some typos in ROADMAP.md 2019-01-25 14:27:13 +08:00
SECURITY.md Update SECURITY.md to have an option to keep name anonymous if requested. 2019-06-18 16:37:16 +00:00
TESTING.md TESTING.md: note that integration-cli is deprecated 2020-12-18 07:51:46 +01:00
vendor.mod daemon: read-copy-update the daemon config 2023-06-01 14:45:24 -04:00
vendor.sum daemon: read-copy-update the daemon config 2023-06-01 14:45:24 -04:00
VENDORING.md fix the bare url and the Summary of http://semver.org 2017-01-17 16:20:11 +08:00

The Moby Project

Moby Project logo

Moby is an open-source project created by Docker to enable and accelerate software containerization.

It provides a "Lego set" of toolkit components, the framework for assembling them into custom container-based systems, and a place for all container enthusiasts and professionals to experiment and exchange ideas. Components include container build tools, a container registry, orchestration tools, a runtime and more, and these can be used as building blocks in conjunction with other tools and projects.

Principles

Moby is an open project guided by strong principles, aiming to be modular, flexible and without too strong an opinion on user experience. It is open to the community to help set its direction.

  • Modular: the project includes lots of components that have well-defined functions and APIs that work together.
  • Batteries included but swappable: Moby includes enough components to build fully featured container systems, but its modular architecture ensures that most of the components can be swapped by different implementations.
  • Usable security: Moby provides secure defaults without compromising usability.
  • Developer focused: The APIs are intended to be functional and useful to build powerful tools. They are not necessarily intended as end user tools but as components aimed at developers. Documentation and UX is aimed at developers not end users.

Audience

The Moby Project is intended for engineers, integrators and enthusiasts looking to modify, hack, fix, experiment, invent and build systems based on containers. It is not for people looking for a commercially supported system, but for people who want to work and learn with open source code.

Relationship with Docker

The components and tools in the Moby Project are initially the open source components that Docker and the community have built for the Docker Project. New projects can be added if they fit with the community goals. Docker is committed to using Moby as the upstream for the Docker Product. However, other projects are also encouraged to use Moby as an upstream, and to reuse the components in diverse ways, and all these uses will be treated in the same way. External maintainers and contributors are welcomed.

The Moby project is not intended as a location for support or feature requests for Docker products, but as a place for contributors to work on open source code, fix bugs, and make the code more useful. The releases are supported by the maintainers, community and users, on a best efforts basis only, and are not intended for customers who want enterprise or commercial support; Docker EE is the appropriate product for these use cases.


Legal

Brought to you courtesy of our legal counsel. For more context, please see the NOTICE document in this repo.

Use and transfer of Moby may be subject to certain restrictions by the United States and other governments.

It is your responsibility to ensure that your use and/or transfer does not violate applicable laws.

For more information, please see https://www.bis.doc.gov

Licensing

Moby is licensed under the Apache License, Version 2.0. See LICENSE for the full license text.