Commit graph

33244 commits

Author SHA1 Message Date
Kir Kolyshkin
9a60e1cc87 Test cases for new ipc modes
These test cases cover various arguments for docker create/run --ipc
option, as well as daemon's --default-ipc-mode cli option and
configuration file parameter.

For the description of container IPC modes, see previous commit.

To run these:

	TESTFLAGS='-check.f IpcMode' make test-integration-cli

[v2: simplify TestDaemonEvents(), add default-ipc-mode presense check]
[v3: add TestDaemonIpcModeVSRestart]
[v4: ipcmode test now uses client lib instead of CLI (except for exec)]
[v5: nitpicks in comments]
[v6: add test case for "none"; fix a typo; simplify TestDaemonEvents() more]

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2017-08-14 11:00:26 +03:00
Kir Kolyshkin
7120976d74 Implement none, private, and shareable ipc modes
Since the commit d88fe447df ("Add support for sharing /dev/shm/ and
/dev/mqueue between containers") container's /dev/shm is mounted on the
host first, then bind-mounted inside the container. This is done that
way in order to be able to share this container's IPC namespace
(and the /dev/shm mount point) with another container.

Unfortunately, this functionality breaks container checkpoint/restore
(even if IPC is not shared). Since /dev/shm is an external mount, its
contents is not saved by `criu checkpoint`, and so upon restore any
application that tries to access data under /dev/shm is severily
disappointed (which usually results in a fatal crash).

This commit solves the issue by introducing new IPC modes for containers
(in addition to 'host' and 'container:ID'). The new modes are:

 - 'shareable':	enables sharing this container's IPC with others
		(this used to be the implicit default);

 - 'private':	disables sharing this container's IPC.

In 'private' mode, container's /dev/shm is truly mounted inside the
container, without any bind-mounting from the host, which solves the
issue.

While at it, let's also implement 'none' mode. The motivation, as
eloquently put by Justin Cormack, is:

> I wondered a while back about having a none shm mode, as currently it is
> not possible to have a totally unwriteable container as there is always
> a /dev/shm writeable mount. It is a bit of a niche case (and clearly
> should never be allowed to be daemon default) but it would be trivial to
> add now so maybe we should...

...so here's yet yet another mode:

 - 'none':	no /dev/shm mount inside the container (though it still
		has its own private IPC namespace).

Now, to ultimately solve the abovementioned checkpoint/restore issue, we'd
need to make 'private' the default mode, but unfortunately it breaks the
backward compatibility. So, let's make the default container IPC mode
per-daemon configurable (with the built-in default set to 'shareable'
for now). The default can be changed either via a daemon CLI option
(--default-shm-mode) or a daemon.json configuration file parameter
of the same name.

Note one can only set either 'shareable' or 'private' IPC modes as a
daemon default (i.e. in this context 'host', 'container', or 'none'
do not make much sense).

Some other changes this patch introduces are:

1. A mount for /dev/shm is added to default OCI Linux spec.

2. IpcMode.Valid() is simplified to remove duplicated code that parsed
   'container:ID' form. Note the old version used to check that ID does
   not contain a semicolon -- this is no longer the case (tests are
   modified accordingly). The motivation is we should either do a
   proper check for container ID validity, or don't check it at all
   (since it is checked in other places anyway). I chose the latter.

3. IpcMode.Container() is modified to not return container ID if the
   mode value does not start with "container:", unifying the check to
   be the same as in IpcMode.IsContainer().

3. IPC mode unit tests (runconfig/hostconfig_test.go) are modified
   to add checks for newly added values.

[v2: addressed review at https://github.com/moby/moby/pull/34087#pullrequestreview-51345997]
[v3: addressed review at https://github.com/moby/moby/pull/34087#pullrequestreview-53902833]
[v4: addressed the case of upgrading from older daemon, in this case
     container.HostConfig.IpcMode is unset and this is valid]
[v5: document old and new IpcMode values in api/swagger.yaml]
[v6: add the 'none' mode, changelog entry to docs/api/version-history.md]

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2017-08-14 10:50:39 +03:00
Kir Kolyshkin
0fb1fb1ce0 Use container.HostConfig.ShmSize directly
It was noted[1] that container's HostConfig.ShmSize, if not set, should be
initialized to daemon default value during container creation.

In fact, it is already done in daemon.adaptContainerSettings, so we can use
value from container.HostConfig directly.

[1] https://github.com/moby/moby/pull/34087#discussion_r128656429

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2017-08-11 03:29:27 -07:00
Victor Vieux
aaee3ca6c1 Merge pull request #34378 from fcrisciani/ln-vendoring
Vendoring libnetwork
2017-08-10 17:07:14 -07:00
Yong Tang
45248ae668 Merge pull request #34452 from Microsoft/jjh/gowiniov0.4.5
Revendor Microsoft/go-winio @ v0.4.5
2017-08-10 17:05:36 -07:00
Yong Tang
2638729c7a Merge pull request #34470 from thaJeztah/fix-solaris-config
Fix solaris reference to config
2017-08-10 09:41:50 -07:00
Flavio Crisciani
709a433ec8
Vendoring libnetwork
Signed-off-by: Flavio Crisciani <flavio.crisciani@docker.com>
2017-08-10 08:51:30 -07:00
Yong Tang
7ccd8bda77 Merge pull request #33722 from TomSweeneyRedHat/tsweeney/privmessage
Add clarification to --privileged error message
2017-08-09 16:08:10 -07:00
Sebastiaan van Stijn
2e38c07814 Merge pull request #34430 from AkihiroSuda/promote-overlay2
graphdriver: promote overlay2 over aufs
2017-08-09 21:41:36 +02:00
Sebastiaan van Stijn
335033e25f
Fix solaris reference to config
Commit db63f9370e
extracted  daemon configuration to its own
package, but did not update the Solaris stubs.

This updates the Solaris daemon.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2017-08-09 20:58:31 +02:00
Yong Tang
bbb401de87 Merge pull request #34445 from pmoust/f-seccomp-quotacl
seccomp: whitelist quotactl with CAP_SYS_ADMIN
2017-08-09 11:53:13 -07:00
Panagiotis Moustafellos
cf6e1c5dfd
seccomp: whitelist quotactl with CAP_SYS_ADMIN
The quotactl syscall is being whitelisted in default seccomp profile,
gated by CAP_SYS_ADMIN.

Signed-off-by: Panagiotis Moustafellos <pmoust@elastic.co>
2017-08-09 18:52:15 +03:00
Sebastiaan van Stijn
ab29a85103 Merge pull request #34455 from vieux/hide_swarm_plugins_exp
hide swarm plugins behind experimental flag
2017-08-09 17:52:14 +02:00
John Howard
779469d9c9 Revendor Microsoft/go-winio @ v0.4.5
Signed-off-by: John Howard <jhoward@microsoft.com>
2017-08-09 08:31:43 -07:00
Yong Tang
187cd25517 Merge pull request #34276 from keloyang/testcase-docker-cp-hardlink
Add unit  test  for RebaseArchiveEntries
2017-08-09 07:29:09 -07:00
Yong Tang
5c49240c94 Merge pull request #34449 from Microsoft/jjh/hcsshim0.6.3
Revendor Microsoft/hcsshim @V0.6.3
2017-08-09 07:21:49 -07:00
Justin Cormack
ac87bb3f85 Merge pull request #34448 from Microsoft/jjh/donotblockexport
LCOW: Don't block export
2017-08-09 14:17:25 +01:00
Justin Cormack
31582d00ab Merge pull request #34450 from Microsoft/jjh/opengcsclient
Vendor change jhowardmsft/opengcs to Microsoft/opengcs
2017-08-09 14:16:31 +01:00
yangshukui
1f1e8e9c0e use testify to check error for test unit test of archive package
Signed-off-by: yangshukui <yangshukui@huawei.com>
2017-08-09 10:27:01 +08:00
Victor Vieux
4930020210 hide swarm plugins behind experimental flag
Signed-off-by: Victor Vieux <victorvieux@gmail.com>
2017-08-08 18:33:25 -07:00
Tibor Vass
c7911389e6 Merge pull request #34441 from vdemeester/move-pkg-templates-away
Move pkg/templates away
2017-08-08 17:54:17 -07:00
John Howard
5463c58217 Vendor Microsoft/opengcs @ v0.3.2
Signed-off-by: John Howard <jhoward@microsoft.com>
2017-08-08 15:43:43 -07:00
John Howard
ee0587a45d jhoward/opengcs --> Microsoft/opengcs
Signed-off-by: John Howard <jhoward@microsoft.com>
2017-08-08 14:43:43 -07:00
John Howard
3e71acf0fc Revendor Microsoft/hcsshim @V0.6.3
Signed-off-by: John Howard <jhoward@microsoft.com>
2017-08-08 14:08:03 -07:00
John Howard
5459ef0ecf LCOW: Don't block export
Signed-off-by: John Howard <jhoward@microsoft.com>
2017-08-08 12:54:06 -07:00
Yong Tang
cbbc28341c Merge pull request #34442 from vdemeester/move-pkg-listeners-away
Move pkg/listeners away
2017-08-08 12:04:18 -07:00
Vincent Demeester
9ef3b53597
Move pkg/templates away
- Remove unused function and variables from the package
- Remove usage of it from `profiles/apparmor` where it wasn't required
- Move the package to `daemon/logger/templates` where it's only used

Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2017-08-08 18:16:41 +02:00
Vincent Demeester
c204fce2ee
Move pkg/listeners away
It is only used in `daemon` and should really live there.

Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2017-08-08 18:14:04 +02:00
Sebastiaan van Stijn
526fc40145 Merge pull request #34408 from rcjsuen/ignore-case
Update tmLanguage file to not be case sensitive
2017-08-08 16:13:50 +02:00
Akihiro Suda
fd6ffc2337 graphdriver: promote overlay2 over aufs
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2017-08-08 10:17:37 +00:00
Yong Tang
202cf001dd Merge pull request #33852 from jstarks/win_named_pipes
Windows: named pipe mounts
2017-08-07 16:54:05 -07:00
John Starks
54354db850 Windows: Add named pipe mount support
Current insider builds of Windows have support for mounting individual
named pipe servers from the host to the guest. This allows, for example,
exposing the docker engine's named pipe to a container.

This change allows the user to request such a mount via the normal bind
mount syntax in the CLI:

  docker run -v \\.\pipe\docker_engine:\\.\pipe\docker_engine <args>

Signed-off-by: John Starks <jostarks@microsoft.com>
2017-08-07 11:34:36 -07:00
Kenfe-Mickaël Laventure
6f1907898f Merge pull request #34386 from tklauser/pkg-term-makeraw-fix
pkg/term: set termios VMIN and VTIME in MakeRaw on Linux
2017-08-07 16:34:40 +02:00
Kenfe-Mickaël Laventure
537f8c1e8d Merge pull request #34398 from Microsoft/jjh/libcontainerd
LCOW: Remove hard-coding
2017-08-07 16:31:44 +02:00
yangshukui
82eb9002e9 Add unit test for RebaseArchiveEntries
Signed-off-by: yangshukui <yangshukui@huawei.com>
2017-08-07 16:40:54 +08:00
Remy Suen
abd39744c6 Update tmLanguage file to not be case sensitive
While convention states that Dockerfile instructions should be
written in uppercase, the engine allows them to be mixed case or in
lowercase. The tmLanguage file should tolerate this and provide
highlighting support even if instructions are not written in
uppercase.

Signed-off-by: Remy Suen <remy.suen@gmail.com>
2017-08-05 19:45:54 +09:00
Sebastiaan van Stijn
d4f6db83c2 Merge pull request #34396 from Microsoft/jjh/movescratch
LCOW: Move toolsScratchPath to /tmp
2017-08-05 11:37:15 +01:00
Sebastiaan van Stijn
80f2c054ca Merge pull request #34399 from Microsoft/jjh/hcsshim0.6.2
Revendor Microsoft/hcsshim @ v0.6.2
2017-08-05 11:35:11 +01:00
Yong Tang
cb9128677f Merge pull request #34371 from Microsoft/jjh/logrustimestamp
logrus use full timestamp
2017-08-04 07:25:59 -07:00
Sebastiaan van Stijn
7e59fbe50e Merge pull request #34395 from aboch/npe
Fix api server null pointer def on inspect/ls null ipam-driver networks
2017-08-04 02:07:46 +02:00
Yong Tang
7843e40ad5 Merge pull request #34383 from wrfly/patch-1
fix client.Transport verify
2017-08-03 16:47:41 -07:00
John Howard
809fed5fa6 Revendor Microsoft/hcsshim @ v0.6.2
Signed-off-by: John Howard <jhoward@microsoft.com>
2017-08-03 16:29:25 -07:00
John Howard
ffdef6255e LCOW: Remove hard-coding
Signed-off-by: John Howard <jhoward@microsoft.com>
2017-08-03 16:10:25 -07:00
John Stephens
a3ffc42b13 Merge pull request #34170 from Microsoft/jjh/sandbox
LCOW: Dynamic sandbox management
2017-08-03 16:07:40 -07:00
John Howard
993f407287 LCOW: Move toolsScratchPath to /tmp
Signed-off-by: John Howard <jhoward@microsoft.com>
2017-08-03 15:48:53 -07:00
Alessandro Boch
beebfc0cf6 Fix api server null pointer def on inspect/ls null ipam-driver networks
- When a network is created with the null ipam driver, docker api server
  thread will deference a nil pointer on `docker network ls` and on
  `docker network inspect <nw>`. This because buildIpamResource()
  assumes a gateway address is always present, which is not correct.

Signed-off-by: Alessandro Boch <aboch@tetrationanalytics.com>
2017-08-03 13:35:58 -07:00
John Howard
8c279ef3ad LCOW: Graphdriver dynamic sandbox management
Signed-off-by: John Howard <jhoward@microsoft.com>

This changes the graphdriver to perform dynamic sandbox management.
Previously, as a temporary 'hack', the service VM had a prebuilt
sandbox in it. With this change, management is under the control
of the client (docker) and executes a mkfs.ext4 on it. This enables
sandboxes of non-default sizes too (a TODO previously in the code).

It also addresses https://github.com/moby/moby/pull/33969#discussion_r127287887

Requires:
- go-winio: v0.4.3
- opengcs:  v0.0.12
- hcsshim:  v0.6.x
2017-08-03 09:06:45 -07:00
John Howard
0bd1cf2517 Revendor jhowardmsft/opengcs @ v0.0.12
Signed-off-by: John Howard <jhoward@microsoft.com>
2017-08-03 09:06:45 -07:00
John Howard
45e2dd123b Revendor microsoft/go-winio @ v0.4.4
Signed-off-by: John Howard <jhoward@microsoft.com>
2017-08-03 09:06:26 -07:00
Tobias Klauser
8a3730d251 pkg/term: set termios VMIN and VTIME in MakeRaw on Linux
The BSD and Solaris versions of term.MakeRaw already set VMIN and VTIME
explicitly such that a read returns when one character is available.
cfmakeraw (which was previously used) in glibc also sets these values
explicitly, so it should be done in the Linux version of MakeRaw as well
to be consistent.

Signed-off-by: Tobias Klauser <tklauser@distanz.ch>
2017-08-03 13:21:42 +02:00