The Moby Project - a collaborative project for the container ecosystem to assemble container-based systems
Find a file
Kir Kolyshkin 7120976d74 Implement none, private, and shareable ipc modes
Since the commit d88fe447df ("Add support for sharing /dev/shm/ and
/dev/mqueue between containers") container's /dev/shm is mounted on the
host first, then bind-mounted inside the container. This is done that
way in order to be able to share this container's IPC namespace
(and the /dev/shm mount point) with another container.

Unfortunately, this functionality breaks container checkpoint/restore
(even if IPC is not shared). Since /dev/shm is an external mount, its
contents is not saved by `criu checkpoint`, and so upon restore any
application that tries to access data under /dev/shm is severily
disappointed (which usually results in a fatal crash).

This commit solves the issue by introducing new IPC modes for containers
(in addition to 'host' and 'container:ID'). The new modes are:

 - 'shareable':	enables sharing this container's IPC with others
		(this used to be the implicit default);

 - 'private':	disables sharing this container's IPC.

In 'private' mode, container's /dev/shm is truly mounted inside the
container, without any bind-mounting from the host, which solves the
issue.

While at it, let's also implement 'none' mode. The motivation, as
eloquently put by Justin Cormack, is:

> I wondered a while back about having a none shm mode, as currently it is
> not possible to have a totally unwriteable container as there is always
> a /dev/shm writeable mount. It is a bit of a niche case (and clearly
> should never be allowed to be daemon default) but it would be trivial to
> add now so maybe we should...

...so here's yet yet another mode:

 - 'none':	no /dev/shm mount inside the container (though it still
		has its own private IPC namespace).

Now, to ultimately solve the abovementioned checkpoint/restore issue, we'd
need to make 'private' the default mode, but unfortunately it breaks the
backward compatibility. So, let's make the default container IPC mode
per-daemon configurable (with the built-in default set to 'shareable'
for now). The default can be changed either via a daemon CLI option
(--default-shm-mode) or a daemon.json configuration file parameter
of the same name.

Note one can only set either 'shareable' or 'private' IPC modes as a
daemon default (i.e. in this context 'host', 'container', or 'none'
do not make much sense).

Some other changes this patch introduces are:

1. A mount for /dev/shm is added to default OCI Linux spec.

2. IpcMode.Valid() is simplified to remove duplicated code that parsed
   'container:ID' form. Note the old version used to check that ID does
   not contain a semicolon -- this is no longer the case (tests are
   modified accordingly). The motivation is we should either do a
   proper check for container ID validity, or don't check it at all
   (since it is checked in other places anyway). I chose the latter.

3. IpcMode.Container() is modified to not return container ID if the
   mode value does not start with "container:", unifying the check to
   be the same as in IpcMode.IsContainer().

3. IPC mode unit tests (runconfig/hostconfig_test.go) are modified
   to add checks for newly added values.

[v2: addressed review at https://github.com/moby/moby/pull/34087#pullrequestreview-51345997]
[v3: addressed review at https://github.com/moby/moby/pull/34087#pullrequestreview-53902833]
[v4: addressed the case of upgrading from older daemon, in this case
     container.HostConfig.IpcMode is unset and this is valid]
[v5: document old and new IpcMode values in api/swagger.yaml]
[v6: add the 'none' mode, changelog entry to docs/api/version-history.md]

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2017-08-14 10:50:39 +03:00
.github Add an initial CODEOWNERS 2017-07-26 16:04:23 -04:00
api Implement none, private, and shareable ipc modes 2017-08-14 10:50:39 +03:00
builder Merge pull request #34272 from dmcgowan/update-logrus 2017-08-01 18:53:33 -07:00
cli Update logrus to v1.0.1 2017-07-31 13:16:46 -07:00
client fix client.Transport 2017-08-03 17:14:26 +08:00
cmd/dockerd Implement none, private, and shareable ipc modes 2017-08-14 10:50:39 +03:00
container Implement none, private, and shareable ipc modes 2017-08-14 10:50:39 +03:00
contrib Update tmLanguage file to not be case sensitive 2017-08-05 19:45:54 +09:00
daemon Implement none, private, and shareable ipc modes 2017-08-14 10:50:39 +03:00
distribution Update logrus to v1.0.1 2017-07-31 13:16:46 -07:00
dockerversion Spelling fixes 2017-07-03 13:13:09 -07:00
docs Implement none, private, and shareable ipc modes 2017-08-14 10:50:39 +03:00
hack Update logrus to v1.0.1 2017-07-31 13:16:46 -07:00
image Update logrus to v1.0.1 2017-07-31 13:16:46 -07:00
integration-cli hide swarm plugins behind experimental flag 2017-08-08 18:33:25 -07:00
layer Update logrus to v1.0.1 2017-07-31 13:16:46 -07:00
libcontainerd jhoward/opengcs --> Microsoft/opengcs 2017-08-08 14:43:43 -07:00
migrate/v1 Update logrus to v1.0.1 2017-07-31 13:16:46 -07:00
oci Implement none, private, and shareable ipc modes 2017-08-14 10:50:39 +03:00
opts Added support for Generic Resources 2017-07-24 17:49:56 -07:00
pkg Merge pull request #34276 from keloyang/testcase-docker-cp-hardlink 2017-08-09 07:29:09 -07:00
plugin Make plugins dir private. 2017-08-02 16:58:07 -04:00
profiles Merge pull request #34445 from pmoust/f-seccomp-quotacl 2017-08-09 11:53:13 -07:00
project Merge pull request #31461 from jpetazzo/announce-all-the-rcs 2017-04-13 10:11:57 -07:00
reference Spelling fixes 2017-07-03 13:13:09 -07:00
registry Update logrus to v1.0.1 2017-07-31 13:16:46 -07:00
reports Add builder dev report for 2017-07-17 2017-07-18 18:31:42 -07:00
restartmanager Adding support for docker max restart time 2017-02-04 03:54:47 +00:00
runconfig Implement none, private, and shareable ipc modes 2017-08-14 10:50:39 +03:00
vendor Merge pull request #34378 from fcrisciani/ln-vendoring 2017-08-10 17:07:14 -07:00
volume Windows: Add named pipe mount support 2017-08-07 11:34:36 -07:00
.dockerignore [EXPERIMENTAL] Integration Test on Swarm 2017-02-28 02:10:09 +00:00
.gitignore Cleanup gitignore 2017-06-23 23:04:39 -07:00
.mailmap Update authors and mailmap 2017-07-18 10:36:11 +02:00
AUTHORS Update authors and mailmap 2017-07-18 10:36:11 +02:00
CHANGELOG.md Spelling fixes 2017-07-03 13:13:09 -07:00
CONTRIBUTING.md update url for issue 2017-06-29 17:10:02 +08:00
Dockerfile vendor: add archive/tar 2017-07-13 19:08:19 -07:00
Dockerfile.aarch64 archive: add test for prefix header 2017-07-14 10:20:48 -07:00
Dockerfile.armhf archive: add test for prefix header 2017-07-14 10:20:48 -07:00
Dockerfile.ppc64le archive: add test for prefix header 2017-07-14 10:20:48 -07:00
Dockerfile.s390x archive: add test for prefix header 2017-07-14 10:20:48 -07:00
Dockerfile.simple archive: add test for prefix header 2017-07-14 10:20:48 -07:00
Dockerfile.solaris Remove pkcs11, libltdl-dev, and clang for osx. 2017-06-21 11:18:44 -04:00
Dockerfile.windows Bump go to go1.8.3 2017-05-30 10:15:59 -04:00
LICENSE Update copyright dates 2017-02-24 19:41:59 +00:00
MAINTAINERS Add Boaz Shuster ("ripcurld0") as curator 2017-07-27 13:10:21 +02:00
Makefile make: add dynbinary target 2017-07-01 04:54:57 +10:00
NOTICE Update LICENSE date 2017-02-15 17:34:33 +01:00
poule.yml Disable auto-assigning PR's to milestones 2017-06-13 03:32:48 +02:00
README.md Add link to docker.com in README.md 2017-07-06 23:09:54 +05:30
ROADMAP.md Header has incorrect punctuation. 2016-12-19 21:38:59 +08:00
vendor.conf Merge pull request #34378 from fcrisciani/ln-vendoring 2017-08-10 17:07:14 -07:00
VENDORING.md fix the bare url and the Summary of http://semver.org 2017-01-17 16:20:11 +08:00
VERSION bump API and version 2017-04-11 11:21:55 -07:00

Docker users, see Moby and Docker to clarify the relationship between the projects

Docker maintainers and contributors, see Transitioning to Moby for more details

The Moby Project

Moby Project logo

Moby is an open-source project created by Docker to advance the software containerization movement. It provides a “Lego set” of dozens of components, the framework for assembling them into custom container-based systems, and a place for all container enthusiasts to experiment and exchange ideas.

Moby

Overview

At the core of Moby is a framework to assemble specialized container systems. It provides:

  • A library of containerized components for all vital aspects of a container system: OS, container runtime, orchestration, infrastructure management, networking, storage, security, build, image distribution, etc.
  • Tools to assemble the components into runnable artifacts for a variety of platforms and architectures: bare metal (both x86 and Arm); executables for Linux, Mac and Windows; VM images for popular cloud and virtualization providers.
  • A set of reference assemblies which can be used as-is, modified, or used as inspiration to create your own.

All Moby components are containers, so creating new components is as easy as building a new OCI-compatible container.

Principles

Moby is an open project guided by strong principles, but modular, flexible and without too strong an opinion on user experience, so it is open to the community to help set its direction. The guiding principles are:

  • Batteries included but swappable: Moby includes enough components to build fully featured container system, but its modular architecture ensures that most of the components can be swapped by different implementations.
  • Usable security: Moby will provide secure defaults without compromising usability.
  • Container centric: Moby is built with containers, for running containers.

With Moby, you should be able to describe all the components of your distributed application, from the high-level configuration files down to the kernel you would like to use and build and deploy it easily.

Moby uses containerd as the default container runtime.

Audience

Moby is recommended for anyone who wants to assemble a container-based system. This includes:

  • Hackers who want to customize or patch their Docker build
  • System engineers or integrators building a container system
  • Infrastructure providers looking to adapt existing container systems to their environment
  • Container enthusiasts who want to experiment with the latest container tech
  • Open-source developers looking to test their project in a variety of different systems
  • Anyone curious about Docker internals and how its built

Moby is NOT recommended for:

  • Application developers looking for an easy way to run their applications in containers. We recommend Docker CE instead.
  • Enterprise IT and development teams looking for a ready-to-use, commercially supported container platform. We recommend Docker EE instead.
  • Anyone curious about containers and looking for an easy way to learn. We recommend the docker.com website instead.

Transitioning to Moby

Docker is transitioning all of its open source collaborations to the Moby project going forward. During the transition, all open source activity should continue as usual.

We are proposing the following list of changes:

  • splitting up the engine into more open components
  • removing the docker UI, SDK etc to keep them in the Docker org
  • clarifying that the project is not limited to the engine, but to the assembly of all the individual components of the Docker platform
  • open-source new tools & components which we currently use to assemble the Docker product, but could benefit the community
  • defining an open, community-centric governance inspired by the Fedora project (a very successful example of balancing the needs of the community with the constraints of the primary corporate sponsor)

Legal

Brought to you courtesy of our legal counsel. For more context, please see the NOTICE document in this repo.

Use and transfer of Moby may be subject to certain restrictions by the United States and other governments.

It is your responsibility to ensure that your use and/or transfer does not violate applicable laws.

For more information, please see https://www.bis.doc.gov

Licensing

Moby is licensed under the Apache License, Version 2.0. See LICENSE for the full license text.