Commit graph

353 commits

Author SHA1 Message Date
Brian Goff
e4b6adc88e Extract volume interaction to a volumes service
This cleans up some of the package API's used for interacting with
volumes, and simplifies management.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2018-05-25 14:21:07 -04:00
Sebastiaan van Stijn
f23c00d870
Various code-cleanup
remove unnescessary import aliases, brackets, and so on.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-05-23 17:50:54 +02:00
Brian Goff
d6558ad6a4 Use mount package to unmount container stuff
Really these mounts shouldn't be in the container pacakge.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2018-05-10 17:16:02 -04:00
Brian Goff
cc8f358c23 Move network operations out of container package
These network operations really don't have anything to do with the
container but rather are setting up the networking.

Ideally these wouldn't get shoved into the daemon package, but doing
something else (e.g. extract a network service into a new package) but
there's a lot more work to do in that regard.
In reality, this probably simplifies some of that work as it moves all
the network operations to the same place.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2018-05-10 17:16:00 -04:00
Kir Kolyshkin
69b4fe4065 daemon.ContainerStop(): fix for a negative timeout
1. As daemon.ContainerStop() documentation says,

> If a negative number of seconds is given, ContainerStop
> will wait for a graceful termination.

but since commit cfdf84d5d0 (PR #32237) this is no longer the case.

This happens because `context.WithTimeout(ctx, timeout)` is implemented
as `WithDeadline(ctx, time.Now().Add(timeout))`, resulting in a deadline
which is in the past.

To fix, don't use WithDeadline() if the timeout is negative.

2. Add a test case to validate the correct behavior and
as a means to prevent a similar regression in the future.

3. Fix/improve daemon.ContainerStop() and client.ContainerStop()
description for clarity and completeness.

4. Fix/improve DefaultStopTimeout description.

Fixes: cfdf84d5d0 ("Update Container Wait")
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2018-05-03 10:04:33 -07:00
Kir Kolyshkin
7d62e40f7e Switch from x/net/context -> context
Since Go 1.7, context is a standard package. Since Go 1.9, everything
that is provided by "x/net/context" is a couple of type aliases to
types in "context".

Many vendored packages still use x/net/context, so vendor entry remains
for now.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2018-04-23 13:52:44 -07:00
Kir Kolyshkin
743f5afcd1 TestContainerStopTimeout: fix
The unit test is checking that setting of non-default StopTimeout
works, but it checked the value of StopSignal instead.

Amazingly, the test was working since the default StopSignal is SIGTERM,
which has the numeric value of 15.

Fixes: commit e66d21089 ("Add config parameter to change ...")
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2018-04-19 11:09:46 -07:00
Brian Goff
6a70fd222b Move mount parsing to separate package.
This moves the platform specific stuff in a separate package and keeps
the `volume` package and the defined interfaces light to import.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2018-04-19 06:35:54 -04:00
Daniel Nephin
6be0f70983 Automated migration using
gty-migrate-from-testify --ignore-build-tags

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2018-03-16 11:03:43 -04:00
Kir Kolyshkin
d6ea46ceda container.BaseFS: check for nil before deref
Commit 7a7357dae1 ("LCOW: Implemented support for docker cp + build")
changed `container.BaseFS` from being a string (that could be empty but
can't lead to nil pointer dereference) to containerfs.ContainerFS,
which could be be `nil` and so nil dereference is at least theoretically
possible, which leads to panic (i.e. engine crashes).

Such a panic can be avoided by carefully analysing the source code in all
the places that dereference a variable, to make the variable can't be nil.
Practically, this analisys are impossible as code is constantly
evolving.

Still, we need to avoid panics and crashes. A good way to do so is to
explicitly check that a variable is non-nil, returning an error
otherwise. Even in case such a check looks absolutely redundant,
further changes to the code might make it useful, and having an
extra check is not a big price to pay to avoid a panic.

This commit adds such checks for all the places where it is not obvious
that container.BaseFS is not nil (which in this case means we do not
call daemon.Mount() a few lines earlier).

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2018-03-13 21:24:48 -07:00
mnussbaum
07d5446fe2 Clean-up after container unit test
Remove temp directories and close file loggers in container unit tests.

Signed-off-by: mnussbaum <michael.nussbaum@getbraintree.com>
2018-02-28 01:15:15 +00:00
Brian Goff
a1afe38e52
Merge pull request #36272 from mnussbaum/36255-fix_log_path
Fix empty LogPath with non-blocking logging mode
2018-02-27 11:25:39 -05:00
Yong Tang
66e6beeb24
Merge pull request #35967 from Microsoft/jjh/32838-pass-container-shutdown-error-back
Windows: Pass back system errors on container exit
2018-02-22 19:12:10 -08:00
John Howard
8c52560ea4 Windows: Pass back system errors on container exit
Signed-off-by: John Howard <jhoward@microsoft.com>

While debugging #32838, it was found (https://github.com/moby/moby/issues/32838#issuecomment-356005845) that the utility VM in some circumstances was crashing. Unfortunately, this was silently thrown away, and as far as the build step (also applies to docker run) was concerned, the exit code was zero and the error was thrown away. Windows containers operate differently to containers on Linux, and there can be legitimate system errors during container shutdown after the init process exits. This PR handles this and passes the error all the way back to the client, and correctly causes a build step running a container which hits a system error to fail, rather than blindly trying to keep going, assuming all is good, and get a subsequent failure on a commit.

With this change, assuming an error occurs, here's an example of a failure which previous was reported as a commit error:

```
The command 'powershell -Command $ErrorActionPreference = 'Stop'; $ProgressPreference = 'SilentlyContinue'; Install-WindowsFeature -Name Web-App-Dev ;   Install-WindowsFeature -Name ADLDS;   Install-WindowsFeature -Name Web-Mgmt-Compat;   Install-WindowsFeature -Name Web-Mgmt-Service;   Install-WindowsFeature -Name Web-Metabase;   Install-WindowsFeature -Name Web-Lgcy-Scripting;   Install-WindowsFeature -Name Web-WMI;   Install-WindowsFeature -Name Web-WHC;   Install-WindowsFeature -Name Web-Scripting-Tools;   Install-WindowsFeature -Name Web-Net-Ext45;   Install-WindowsFeature -Name Web-ASP;   Install-WindowsFeature -Name Web-ISAPI-Ext;   Install-WindowsFeature -Name Web-ISAPI-Filter;   Install-WindowsFeature -Name Web-Default-Doc;   Install-WindowsFeature -Name Web-Dir-Browsing;   Install-WindowsFeature -Name Web-Http-Errors;   Install-WindowsFeature -Name Web-Static-Content;   Install-WindowsFeature -Name Web-Http-Redirect;   Install-WindowsFeature -Name Web-DAV-Publishing;   Install-WindowsFeature -Name Web-Health;   Install-WindowsFeature -Name Web-Http-Logging;   Install-WindowsFeature -Name Web-Custom-Logging;   Install-WindowsFeature -Name Web-Log-Libraries;   Install-WindowsFeature -Name Web-Request-Monitor;   Install-WindowsFeature -Name Web-Http-Tracing;   Install-WindowsFeature -Name Web-Stat-Compression;   Install-WindowsFeature -Name Web-Dyn-Compression;   Install-WindowsFeature -Name Web-Security;   Install-WindowsFeature -Name Web-Windows-Auth;   Install-WindowsFeature -Name Web-Basic-Auth;   Install-WindowsFeature -Name Web-Url-Auth;   Install-WindowsFeature -Name Web-WebSockets;   Install-WindowsFeature -Name Web-AppInit;   Install-WindowsFeature -Name NET-WCF-HTTP-Activation45;   Install-WindowsFeature -Name NET-WCF-Pipe-Activation45;   Install-WindowsFeature -Name NET-WCF-TCP-Activation45;' returned a non-zero code: 4294967295: container shutdown failed: container ba9c65054d42d4830fb25ef55e4ab3287550345aa1a2bb265df4e5bfcd79c78a encountered an error during WaitTimeout: failure in a Windows system call: The compute system exited unexpectedly. (0xc0370106)
```

Without this change, it would be incorrectly reported such as in this comment: https://github.com/moby/moby/issues/32838#issuecomment-309621097

```
Step 3/8 : ADD buildtools C:/buildtools
re-exec error: exit status 1: output: time="2017-06-20T11:37:38+10:00" level=error msg="hcsshim::ImportLayer failed in Win32: The system cannot find the path specified. (0x3) layerId=\\\\?\\C:\\ProgramData\\docker\\windowsfilter\\b41d28c95f98368b73fc192cb9205700e21
6691495c1f9ac79b9b04ec4923ea2 flavour=1 folder=C:\\Windows\\TEMP\\hcs232661915"
hcsshim::ImportLayer failed in Win32: The system cannot find the path specified. (0x3) layerId=\\?\C:\ProgramData\docker\windowsfilter\b41d28c95f98368b73fc192cb9205700e216691495c1f9ac79b9b04ec4923ea2 flavour=1 folder=C:\Windows\TEMP\hcs232661915
```
2018-02-22 08:53:43 -08:00
junzhe and mnussbaum
20ca612a59 Fix empty LogPath with non-blocking logging mode
This fixes an issue where the container LogPath was empty when the
non-blocking logging mode was enabled. This change sets the LogPath on
the container as soon as the path is generated, instead of setting the
LogPath on a logger struct and then attempting to pull it off that
logger at a later point. That attempt to pull the LogPath off the logger
was error prone since it assumed that the logger would only ever be a
single type.

Prior to this change docker inspect returned an empty string for
LogPath. This caused issues with tools that rely on docker inspect
output to discover container logs, e.g. Kubernetes.

This commit also removes some LogPath methods that are now unnecessary
and are never invoked.

Signed-off-by: junzhe and mnussbaum <code@getbraintree.com>
2018-02-20 23:12:34 -08:00
Brian Goff
c02171802b Merge configs/secrets in unix implementation
On unix, merge secrets/configs handling. This is important because
configs can contain secrets (via templating) and potentially a config
could just simply have secret information "by accident" from the user.
This just make sure that configs are as secure as secrets and de-dups a
lot of code.
Generally this makes everything simpler and configs more secure.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2018-02-16 11:25:14 -05:00
Aaron Lehmann
cd3d0486a6 Store configs that contain secrets on tmpfs
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2018-02-16 11:25:14 -05:00
Brian Goff
b3aab5e31f Use continuity fs package for volume copy
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2018-02-12 16:43:51 -05:00
Daniel Nephin
4f0d95fa6e Add canonical import comment
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2018-02-05 16:51:57 -05:00
Brian Goff
eaa5192856 Make container resource mounts unbindable
It's a common scenario for admins and/or monitoring applications to
mount in the daemon root dir into a container. When doing so all mounts
get coppied into the container, often with private references.
This can prevent removal of a container due to the various mounts that
must be configured before a container is started (for example, for
shared /dev/shm, or secrets) being leaked into another namespace,
usually with private references.

This is particularly problematic on older kernels (e.g. RHEL < 7.4)
where a mount may be active in another namespace and attempting to
remove a mountpoint which is active in another namespace fails.

This change moves all container resource mounts into a common directory
so that the directory can be made unbindable.
What this does is prevents sub-mounts of this new directory from leaking
into other namespaces when mounted with `rbind`... which is how all
binds are handled for containers.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2018-01-16 15:09:05 -05:00
Sebastiaan van Stijn
b4a6313969
Golint: remove redundant ifs
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-01-15 00:42:25 +01:00
Vincent Demeester
7c7a7c7944
Merge pull request #35940 from yongtang/35920-filter-health-starting
Fix issue of filter in `docker ps` where `health=starting` returns nothing
2018-01-10 14:12:47 +01:00
Daniel Nephin
3fec7c0858 Remove libcontainerd.IOPipe
replaced with cio.DirectIO

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2018-01-09 12:00:28 -05:00
Yong Tang
f509a54bdd Add test case for docker ps -f health=starting
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
2018-01-07 05:10:36 +00:00
Yong Tang
97b16aecf9 Fix issue of filter in docker ps where health=starting returns nothing
This fix tries to address the issue raised in 35920 where the filter
of `docker ps` with  `health=starting` always returns nothing.

The issue was that in container view, the human readable string (`HealthString()` => `Health.String()`)
of health status was used. In case of starting it is `"health: starting"`.
However, the filter still uses `starting` so no match returned.

This fix fixes the issue by using `container.Health.Status()` instead so that it matches
the string (`starting`) passed by filter.

This fix fixes 35920.

Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
2018-01-05 05:16:09 +00:00
Sebastiaan van Stijn
6ed1163c98
Remove redundant build-tags
Files that are suffixed with `_linux.go` or `_windows.go` are
already only built on Linux / Windows, so these build-tags
were redundant.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2017-12-18 17:41:53 +01:00
Sebastiaan van Stijn
1589cc0a85
Remove Solaris files
Solaris is no longer being worked on, so these files
are now just dead code.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2017-12-18 17:22:25 +01:00
Michael Crosby
aa3ce07c41 Update daemon code for containerd API changes
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-11-30 09:55:03 -05:00
Antonio Murdaca
dca82b9e5c
container: update real-time resources
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2017-11-22 22:21:40 +01:00
Kir Kolyshkin
1861abdc4a Fix "duplicate mount point" when --tmpfs /dev/shm is used
This is a fix to the following issue:

$ docker run --tmpfs /dev/shm busybox sh
docker: Error response from daemon: linux mounts: Duplicate mount point '/dev/shm'.

In current code (daemon.createSpec()), tmpfs mount from --tmpfs is added
to list of mounts (`ms`), when the mount from IpcMounts() is added.
While IpcMounts() is checking for existing mounts first, it does that
by using container.HasMountFor() function which only checks container.Mounts
but not container.Tmpfs.

Ultimately, the solution is to get rid of container.Tmpfs (moving its
data to container.Mounts). Current workaround is to add checking
of container.Tmpfs into container.HasMountFor().

A unit test case is included.

Unfortunately we can't call daemon.createSpec() from a unit test,
as the code relies a lot on various daemon structures to be initialized
properly, and it is hard to achieve. Therefore, we minimally mimick
the code flow of daemon.createSpec() -- barely enough to reproduce
the issue.

https://github.com/moby/moby/issues/35455

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2017-11-20 18:48:27 -08:00
Kir Kolyshkin
eab3ac3e70 container.NetworkMounts(): don't lookup mounts twice
The code in question looks up mounts two times: first by using
HasMountFor(), and then directly by looking in container.MountPoints.
There is no need to do it twice.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2017-11-20 18:48:27 -08:00
Stephen J Day
7db30ab0cd
container: protect the health status with mutex
Adds a mutex to protect the status, as well. When running the race
detector with the unit test, we can see that the Status field is written
without holding this lock. Adding a mutex to read and set status
addresses the issue.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-11-16 15:04:01 -08:00
Stephen J Day
5b55747a52
container: protect health monitor channel
While this code was likely called from a single thread before, we have
now seen panics, indicating that it could be called in parallel. This
change adds a mutex to protect opening and closing of the channel. There
may be another root cause associated with this panic, such as something
that led to the calling of this in parallel, as this code is old and we
had seen this condition until recently.

This fix is by no means a permanent fix. Typically, bugs like this
indicate misplaced channel ownership. In idiomatic uses, the channel
should have a particular "owner" that coordinates sending and closure.
In this case, the owner of the channel is unclear, so it gets opened
lazily. Synchronizing this access is a decent solution, but a refactor
may yield better results.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-11-13 13:31:28 -08:00
Yong Tang
4785f1a7ab Remove solaris build tag and `contrib/mkimage/solaris
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
2017-11-02 00:01:46 +00:00
Yong Tang
220d6c4aff
Merge pull request #34999 from kolyshkin/wait-on-rm
ContainerWait on remove: don't stuck on rm fail
2017-10-29 11:04:41 -07:00
Kir Kolyshkin
f963500c54 ContainerWait on remove: don't stuck on rm fail
Currently, if a container removal has failed for some reason,
any client waiting for removal (e.g. `docker run --rm`) is
stuck, waiting for removal to succeed while it has failed already.
For more details and the reproducer, please check
https://github.com/moby/moby/issues/34945

This commit addresses that by allowing `ContainerWait()` with
`container.WaitCondition == "removed"` argument to return an
error in case of removal failure. The `ContainerWaitOKBody`
stucture returned to a client is amended with a pointer to `struct Error`,
containing an error message string, and the `Client.ContainerWait()`
is modified to return the error, if any, to the client.

Note that this feature is only available for API version >= 1.34.
In order for the old clients to be unstuck, we just close the connection
without writing anything -- this causes client's error.

Now, docker-cli would need a separate commit to bump the API to 1.34
and to show an error returned, if any.

[v2: recreate the waitRemove channel after closing]
[v3: document; keep legacy behavior for older clients]
[v4: convert Error from string to pointer to a struct]
[v5: don't emulate old behavior, send empty response in error case]
[v6: rename legacy* vars to include version suffix]

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2017-10-25 13:11:56 -07:00
Michael Crosby
05026b187b Merge pull request #35273 from chchliang/containerstate
add testcase IsValidStateString
2017-10-24 14:32:37 -04:00
chchliang
675ac37482 add testcase IsValidStateString
Signed-off-by: chchliang <chen.chuanliang@zte.com.cn>
2017-10-24 09:49:58 +08:00
Darren Stahl
ed74ee127f Increase container default shutdown timeout on Windows
The shutdown timeout for containers in insufficient on Windows. If the daemon is shutting down, and a container takes longer than expected to shut down, this can cause the container to remain in a bad state after restart, and never be able to start again. Increasing the timeout makes this less likely to occur.

Signed-off-by: Darren Stahl <darst@microsoft.com>
2017-10-23 10:31:31 -07:00
Kenfe-Mickael Laventure
ddae20c032
Update libcontainerd to use containerd 1.0
Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
2017-10-20 07:11:37 -07:00
John Howard
d98ecf2d6c LCOW: API change JSON header to string POST parameter
Signed-off-by: John Howard <jhoward@microsoft.com>
2017-10-06 15:26:48 -07:00
John Howard
0380fbff37 LCOW: API: Add platform to /images/create and /build
Signed-off-by: John Howard <jhoward@microsoft.com>

This PR has the API changes described in https://github.com/moby/moby/issues/34617.
Specifically, it adds an HTTP header "X-Requested-Platform" which is a JSON-encoded
OCI Image-spec `Platform` structure.

In addition, it renames (almost all) uses of a string variable platform (and associated)
methods/functions to os. This makes it much clearer to disambiguate with the swarm
"platform" which is really os/arch. This is a stepping stone to getting the daemon towards
fully multi-platform/arch-aware, and makes it clear when "operating system" is being
referred to rather than "platform" which is misleadingly used - sometimes in the swarm
meaning, but more often as just the operating system.
2017-10-06 11:44:18 -07:00
Yong Tang
c982ee805d Merge pull request #34929 from stevvooe/remove-promise-package
pkg/package: remove promise package
2017-09-22 11:52:23 -07:00
Yuanhong Peng
8c32659979 Fixes #29654: take reference to RWLayer while committing/exporting
Take an extra reference to rwlayer while the container is being
committed or exported to avoid the removal of that layer.

Also add some checks before commit/export.

Signed-off-by: Yuanhong Peng <pengyuanhong@huawei.com>
2017-09-22 09:47:42 +08:00
Stephen J Day
0cd4ab3f9a
pkg/package: remove promise package
The promise package represents a simple enough concurrency pattern that
replicating it in place is sufficient. To end the propagation of this
package, it has been removed and the uses have been inlined.

While this code could likely be refactored to be simpler without the
package, the changes have been minimized to reduce the possibility of
defects. Someone else may want to do further refactoring to remove
closures and reduce the number of goroutines in use.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-09-21 17:56:45 -07:00
Victor Vieux
a5f9783c93 Merge pull request #34252 from Microsoft/akagup/lcow-remotefs-sandbox
LCOW: Support for docker cp, ADD/COPY on build
2017-09-15 16:49:48 -07:00
Simon Ferquel
e89b6e8c2d Volume refactoring for LCOW
Signed-off-by: Simon Ferquel <simon.ferquel@docker.com>
2017-09-14 12:33:31 -07:00
Akash Gupta
7a7357dae1 LCOW: Implemented support for docker cp + build
This enables docker cp and ADD/COPY docker build support for LCOW.
Originally, the graphdriver.Get() interface returned a local path
to the container root filesystem. This does not work for LCOW, so
the Get() method now returns an interface that LCOW implements to
support copying to and from the container.

Signed-off-by: Akash Gupta <akagup@microsoft.com>
2017-09-14 12:07:52 -07:00
Daniel Nephin
f7f101d57e Add gosimple linter
Update gometalinter

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-09-12 12:09:59 -04:00
Daniel Nephin
709bf8b7bc Add interfacer linter
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-08-24 15:08:26 -04:00
John Howard
9fa449064c LCOW: WORKDIR correct handling
Signed-off-by: John Howard <jhoward@microsoft.com>
2017-08-17 15:29:17 -07:00
Brian Goff
ebcb7d6b40 Remove string checking in API error handling
Use strongly typed errors to set HTTP status codes.
Error interfaces are defined in the api/errors package and errors
returned from controllers are checked against these interfaces.

Errors can be wraeped in a pkg/errors.Causer, as long as somewhere in the
line of causes one of the interfaces is implemented. The special error
interfaces take precedence over Causer, meaning if both Causer and one
of the new error interfaces are implemented, the Causer is not
traversed.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2017-08-15 16:01:11 -04:00
Yong Tang
e4c19aa529 Merge pull request #34419 from keloyang/config.v2.json
security: Chmod config.v2.json to 0600
2017-08-14 08:39:58 -07:00
Kir Kolyshkin
7120976d74 Implement none, private, and shareable ipc modes
Since the commit d88fe447df ("Add support for sharing /dev/shm/ and
/dev/mqueue between containers") container's /dev/shm is mounted on the
host first, then bind-mounted inside the container. This is done that
way in order to be able to share this container's IPC namespace
(and the /dev/shm mount point) with another container.

Unfortunately, this functionality breaks container checkpoint/restore
(even if IPC is not shared). Since /dev/shm is an external mount, its
contents is not saved by `criu checkpoint`, and so upon restore any
application that tries to access data under /dev/shm is severily
disappointed (which usually results in a fatal crash).

This commit solves the issue by introducing new IPC modes for containers
(in addition to 'host' and 'container:ID'). The new modes are:

 - 'shareable':	enables sharing this container's IPC with others
		(this used to be the implicit default);

 - 'private':	disables sharing this container's IPC.

In 'private' mode, container's /dev/shm is truly mounted inside the
container, without any bind-mounting from the host, which solves the
issue.

While at it, let's also implement 'none' mode. The motivation, as
eloquently put by Justin Cormack, is:

> I wondered a while back about having a none shm mode, as currently it is
> not possible to have a totally unwriteable container as there is always
> a /dev/shm writeable mount. It is a bit of a niche case (and clearly
> should never be allowed to be daemon default) but it would be trivial to
> add now so maybe we should...

...so here's yet yet another mode:

 - 'none':	no /dev/shm mount inside the container (though it still
		has its own private IPC namespace).

Now, to ultimately solve the abovementioned checkpoint/restore issue, we'd
need to make 'private' the default mode, but unfortunately it breaks the
backward compatibility. So, let's make the default container IPC mode
per-daemon configurable (with the built-in default set to 'shareable'
for now). The default can be changed either via a daemon CLI option
(--default-shm-mode) or a daemon.json configuration file parameter
of the same name.

Note one can only set either 'shareable' or 'private' IPC modes as a
daemon default (i.e. in this context 'host', 'container', or 'none'
do not make much sense).

Some other changes this patch introduces are:

1. A mount for /dev/shm is added to default OCI Linux spec.

2. IpcMode.Valid() is simplified to remove duplicated code that parsed
   'container:ID' form. Note the old version used to check that ID does
   not contain a semicolon -- this is no longer the case (tests are
   modified accordingly). The motivation is we should either do a
   proper check for container ID validity, or don't check it at all
   (since it is checked in other places anyway). I chose the latter.

3. IpcMode.Container() is modified to not return container ID if the
   mode value does not start with "container:", unifying the check to
   be the same as in IpcMode.IsContainer().

3. IPC mode unit tests (runconfig/hostconfig_test.go) are modified
   to add checks for newly added values.

[v2: addressed review at https://github.com/moby/moby/pull/34087#pullrequestreview-51345997]
[v3: addressed review at https://github.com/moby/moby/pull/34087#pullrequestreview-53902833]
[v4: addressed the case of upgrading from older daemon, in this case
     container.HostConfig.IpcMode is unset and this is valid]
[v5: document old and new IpcMode values in api/swagger.yaml]
[v6: add the 'none' mode, changelog entry to docs/api/version-history.md]

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2017-08-14 10:50:39 +03:00
yangshukui
ae52cea3ab chmod config.v2.json to 0600
Signed-off-by: yangshukui <yangshukui@huawei.com>
2017-08-07 18:10:08 +08:00
Derek McGowan
1009e6a40b
Update logrus to v1.0.1
Fixes case sensitivity issue

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2017-07-31 13:16:46 -07:00
Aaron Lehmann
1d9546fc62 container: Fix Delete on nonexistent container
Delete needs to release names related to a container even if that
container isn't present in the db. However, slightly overzealous error
checking causes the transaction to get rolled back. Ignore the error
from Delete on the container itself, since it may not be present.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2017-07-26 16:45:54 -07:00
Sebastiaan van Stijn
8d703b98b5 Merge pull request #34194 from abhinandanpb/opt
Fixing issue with driver opt not passed to drivers
2017-07-20 20:02:05 +02:00
Abhinandan Prativadi
bcb55c6202 Fixing issue with driver opt not passed to drivers
Signed-off-by: Abhinandan Prativadi <abhi@docker.com>
2017-07-19 17:44:53 -07:00
Sebastiaan van Stijn
4309075610 Merge pull request #34071 from FengtuWang/pause
Keep pause state when restoring container's status
2017-07-19 17:49:38 +02:00
Aaron Lehmann
0e57eb95c5 container: Use wrapper to ensure commit/abort happens
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2017-07-13 12:35:03 -07:00
Aaron Lehmann
bc3209bc15 container: Abort transactions when memdb calls fail
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2017-07-13 12:35:02 -07:00
Aaron Lehmann
1128fc1add Store container names in memdb
Currently, names are maintained by a separate system called "registrar".
This means there is no way to atomically snapshot the state of
containers and the names associated with them.

We can add this atomicity and simplify the code by storing name
associations in the memdb. This removes the need for pkg/registrar, and
makes snapshots a lot less expensive because they no longer need to copy
all the names. This change also avoids some problematic behavior from
pkg/registrar where it returns slices which may be modified later on.

Note that while this change makes the *snapshotting* atomic, it doesn't
yet do anything to make sure containers are named at the same time that
they are added to the database. We can do that by adding a transactional
interface, either as a followup, or as part of this PR.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2017-07-13 12:35:00 -07:00
Fengtu Wang
977c4046fd Keep pause state when restoring container's status
Do not change pause state when restoring container's
status, or status in docker will be different with
status in runc.

Signed-off-by: Fengtu Wang <wangfengtu@huawei.com>
2017-07-12 16:25:17 +08:00
Madhan Raj Mookkandy
349913ce9f Include Endpoint List for Shared Endpoints
Do not allow sharing of container network with hyperv containers

Signed-off-by: Madhan Raj Mookkandy <madhanm@microsoft.com>
2017-07-06 12:19:17 -07:00
Josh Soref
39bcaee47b
Spelling fixes
* additional
* ambiguous
* anonymous
* anything
* application
* because
* before
* building
* capabilities
* circumstances
* commit
* committer
* compresses
* concatenated
* config
* container
* container's
* current
* definition
* delimiter
* disassociates
* discovery
* distributed
* doesnotexist
* downloads
* duplicates
* either
* enhancing
* enumerate
* escapable
* exactly
* expect
* expectations
* expected
* explicitly
* false
* filesystem
* following
* forbidden
* git with
* healthcheck
* ignore
* independent
* inheritance
* investigating
* irrelevant
* it
* logging
* looking
* membership
* mimic
* minimum
* modify
* mountpoint
* multiline
* notifier
* outputting
* outside
* overridden
* override
* parsable
* plugins
* precedence
* propagation
* provided
* provides
* registries
* repositories
* returning
* settings
* should
* signals
* someone
* something
* specifically
* successfully
* synchronize
* they've
* thinking
* uninitialized
* unintentionally
* unmarshaling
* unnamed
* unreferenced
* verify

Signed-off-by: Josh Soref <jsoref@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2017-07-03 13:13:09 -07:00
Wentao Zhang
fe1b4cfba6 Set unpasued state when receiving 'stateExit' event
Description:
 1. start a container with restart=always.
    `docker run -d --restart=always ubuntu sleep 3`
 2. container init process exits.
 3. use `docker pause <id>` to pause this container.

if the pause action is before cgroup data is removed and after the init process died.
`Pause` operation will success to write cgroup data, but actually do not freeze any process.

And then docker received pause event and stateExit event from
containerd, the docker state will be Running(paused), but the container
is free running.

Then we can not remove it, stop it , pause it  and unpause it.

Signed-off-by: Wentao Zhang <zhangwentao234@huawei.com>
2017-07-03 17:26:13 +08:00
Brian Goff
18d874a20f Merge pull request #33882 from aaronlehmann/memdb-no-container
container: Handle failed memdb lookups
2017-06-30 14:31:10 -04:00
Aaron Lehmann
c26b0cdfd1 container: Handle failed memdb lookups
If a container doesn't exist in the memdb, First will return nil, not an
error. This should be checked for before using the result.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2017-06-29 16:33:30 -07:00
John Howard
4ec9766a27 LCOW: Fix nits from 33241
Signed-off-by: John Howard <jhoward@microsoft.com>
2017-06-27 11:59:49 -07:00
Fabio Kung
04bd768a88 ensure heath monitor status updates are propagated
initHealthMonitor and updateHealthMonitor can cause container state to
be changed (State.Health).

Signed-off-by: Fabio Kung <fabio.kung@gmail.com>
2017-06-23 07:52:34 -07:00
Fabio Kung
a43be3431e avoid re-reading json files when copying containers
Signed-off-by: Fabio Kung <fabio.kung@gmail.com>
2017-06-23 07:52:34 -07:00
Fabio Kung
9134e87afc only Daemon.load needs to call label.ReserveLabel
Signed-off-by: Fabio Kung <fabio.kung@gmail.com>
2017-06-23 07:52:33 -07:00
Fabio Kung
edad52707c save deep copies of Container in the replica store
Reuse existing structures and rely on json serialization to deep copy
Container objects.

Also consolidate all "save" operations on container.CheckpointTo, which
now both saves a serialized json to disk, and replicates state to the
ACID in-memory store.

Signed-off-by: Fabio Kung <fabio.kung@gmail.com>
2017-06-23 07:52:33 -07:00
Fabio Kung
f668af4475 no need to save container state here
it is already being saved (with a lock held) on the subsequent
operations.

Signed-off-by: Fabio Kung <fabio.kung@gmail.com>
2017-06-23 07:52:33 -07:00
Fabio Kung
2ed6f9257a how to maintain the container snapshot struct
Signed-off-by: Fabio Kung <fabio.kung@gmail.com>
2017-06-23 07:52:32 -07:00
Fabio Kung
aacddda89d Move checkpointing to the Container object
Also hide ViewDB behind an inteface.

Signed-off-by: Fabio Kung <fabio.kung@gmail.com>
2017-06-23 07:52:32 -07:00
Fabio Kung
eed4c7b73f keep a consistent view of containers rendered
Replicate relevant mutations to the in-memory ACID store. Readers will
then be able to query container state without locking.

Signed-off-by: Fabio Kung <fabio.kung@gmail.com>
2017-06-23 07:52:31 -07:00
Fabio Kung
054728b1f5 in-memory ACID store for containers
This can be used by readers/queries so they don't need locks.

Signed-off-by: Fabio Kung <fabio.kung@gmail.com>
2017-06-23 07:24:11 -07:00
Fabio Kung
cfc404a375 Move platform specific mount data to Container
The Solaris version (previously daemon/inspect_solaris.go) was
apparently missing some fields that should be available on that
platform.

Signed-off-by: Fabio Kung <fabio.kung@gmail.com>
2017-06-23 07:22:47 -07:00
John Howard
ed10ac6ee9 LCOW: Create layer folders with correct ACL
Signed-off-by: John Howard <jhoward@microsoft.com>
2017-06-20 19:50:12 -07:00
John Howard
f154588226 LCOW: OCI Spec and Environment for container start
Signed-off-by: John Howard <jhoward@microsoft.com>
2017-06-20 19:50:11 -07:00
John Howard
f97fbba5ce LCOW: Add platform to container, init on FromDisk()
Signed-off-by: John Howard <jhoward@microsoft.com>
2017-06-20 19:49:51 -07:00
John Howard
55f8828eec LCOW: Remove CommonContainer - just Container
Signed-off-by: John Howard <jhoward@microsoft.com>
2017-06-20 08:55:46 -07:00
Brian Goff
f05a023760 Don't chown/chmod volumes if not needed.
Doing a chown/chmod automatically can cause `EPERM` in some cases (e.g.
with an NFS mount). Currently Docker will always call chown+chmod on a
volume path unless `:nocopy` is passed in, but we don't need to make
these calls if the perms and ownership already match and potentially
avoid an uneccessary `EPERM`.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2017-06-18 16:44:52 -07:00
unclejack
014095e6a0 container/stream/attach: use pools.Copy
The use of pools.Copy avoids io.Copy's internal buffer allocation.
This commit replaces io.Copy with pools.Copy to avoid the allocation of
buffers in io.Copy.

Signed-off-by: Cristian Staretu <cristian.staretu@gmail.com>
2017-06-10 21:13:33 +03:00
Daniel Nephin
967ef7e6d2 Remove unused functions from archive.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-06-07 11:44:33 -04:00
Daniel Nephin
09cd96c5ad Partial refactor of UID/GID usage to use a unified struct.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-06-07 11:44:33 -04:00
Chen Min
2b62eb434b Fix Typos: continer,contianer -> container
Signed-off-by: Chen Min <chenmin46@huawei.com>
2017-06-01 19:21:01 +08:00
Vincent Demeester
41c3c9cc86 Merge pull request #33329 from wenjianhn/EINVAL
Don't log EINVAL when unmount IPC
2017-05-31 11:17:19 -07:00
Jacob Wen
cfa2591d3f Don't warn if the shm or mquere is not mounted
Fix #33328

Signed-off-by: Jacob Wen <jian.w.wen@oracle.com>
2017-05-31 16:19:59 +08:00
Sebastiaan van Stijn
b654b6244d
Improve description of Running and Paused booleans
Commit abd72d4008 added
a "FIXME" comment to the container "State", mentioning
that a container cannot be both "Running" and "Paused".

This comment was incorrect, because containers on
Linux actually _must_ be running in order to be
paused.

This patch adds additional information both in a
comment, and in the API documentation to clarify
that these booleans are not mutually exclusive.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2017-05-19 12:29:54 +02:00
Abhinandan Prativadi
fe11de0177 Adding network specific options to service create/update
The commit adds capability to accept csv parameters
for network option in service create/update commands.The change
includes name,alias driver options specific to the network.
With this the following will be supported

docker service create --name web --network name=docknet,alias=web1,driver-opt=field1=value1 nginx
docker service create --name web --network docknet nginx
docker service update web --network-add name=docknet,alias=web1,driver-opt=field1=value1
docker service update web --network-rm docknet

Signed-off-by: Abhinandan Prativadi <abhi@docker.com>
2017-05-17 18:46:34 -07:00
Sebastiaan van Stijn
c053a2069e Merge pull request #32237 from jlhawn/update_container_wait
Update Container Wait
2017-05-17 02:39:52 +02:00
Josh Hawn
4921171587 Update ContainerWait API
This patch adds the untilRemoved option to the ContainerWait API which
allows the client to wait until the container is not only exited but
also removed.

This patch also adds some more CLI integration tests for waiting for a
created container and waiting with the new --until-removed flag.

Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)

Handle detach sequence in CLI

Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)

Update Container Wait Conditions

Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)

Apply container wait changes to API 1.30

The set of changes to the containerWait API missed the cut for the
Docker 17.05 release (API version 1.29). This patch bumps the version
checks to use 1.30 instead.

This patch also makes a minor update to a testfile which was added to
the builder/dockerfile package.

Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)

Remove wait changes from CLI

Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)

Address minor nits on wait changes

- Changed the name of the tty Proxy wrapper to `escapeProxy`
- Removed the unnecessary Error() method on container.State
- Fixes a typo in comment (repeated word)

Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)

Use router.WithCancel in the containerWait handler

This handler previously added this functionality manually but now uses
the existing wrapper which does it for us.

Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)

Add WaitCondition constants to api/types/container

Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)

Address more ContainerWait review comments

- Update ContainerWait backend interface to not return pointer values
  for container.StateStatus type.
- Updated container state's Wait() method comments to clarify that a
  context MUST be used for cancelling the request, setting timeouts,
  and to avoid goroutine leaks.
- Removed unnecessary buffering when making channels in the client's
  ContainerWait methods.
- Renamed result and error channels in client's ContainerWait methods
  to clarify that only a single result or error value would be sent
  on the channel.

Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)

Move container.WaitCondition type to separate file

... to avoid conflict with swagger-generated code for API response

Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)

Address more ContainerWait review comments

Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
2017-05-16 15:11:39 -07:00
Josh Hawn
cfdf84d5d0 Update Container Wait Backend
This patch consolidates the two WaitStop and WaitWithContext methods
on the container.State type. Now there is a single method, Wait, which
takes a context and a bool specifying whether to wait for not just a
container exit but also removal.

The behavior has been changed slightly so that a wait call during a
Created state will not return immediately but instead wait for the
container to be started and then exited.

The interface has been changed to no longer block, but instead returns
a channel on which the caller can receive a *StateStatus value which
indicates the ExitCode or an error if there was one (like a context
timeout or state transition error).

These changes have been propagated through the rest of the deamon to
preserve all other existing behavior.

Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
2017-05-16 15:09:14 -07:00
John Stephens
e0d533b1e8
Add Windows configs support
Signed-off-by: John Stephens <johnstep@docker.com>
2017-05-16 14:25:32 -07:00
John Stephens
bd4e8aa64e
Add Windows secrets support
Signed-off-by: John Stephens <johnstep@docker.com>
2017-05-16 11:30:06 -07:00
Aaron Lehmann
9e9fc7b57c Add config support to executor backend
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2017-05-11 10:08:21 -07:00
Aaron Lehmann
37ce91ddd6 Use "local" secret paths based on the secretID
This prevents targets with the same basename from colliding.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2017-05-10 10:23:09 -07:00