Commit graph

6126 commits

Author SHA1 Message Date
Sunny Gogoi
74eb258ffb Add pids-limit support in docker update
- Adds updating PidsLimit in UpdateContainer().
- Adds setting PidsLimit in toContainerResources().

Signed-off-by: Sunny Gogoi <indiasuny000@gmail.com>
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2019-02-21 14:17:38 -08:00
Sebastiaan van Stijn
5801c04345
Merge pull request #38380 from olljanat/capabilities-support
Add support for exact list of capabilities + capAdd / capDrop refactor
2019-01-28 16:36:03 +01:00
Olli Janatuinen
80d7bfd54d Capabilities refactor
- Add support for exact list of capabilities, support only OCI model
- Support OCI model on CapAdd and CapDrop but remain backward compatibility
- Create variable locally instead of declaring it at the top
- Use const for magic "ALL" value
- Rename `cap` variable as it overlaps with `cap()` built-in
- Normalize and validate capabilities before use
- Move validation for conflicting options to validateHostConfig()
- TweakCapabilities: simplify logic to calculate capabilities

Signed-off-by: Olli Janatuinen <olli.janatuinen@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-01-22 21:50:41 +02:00
Sebastiaan van Stijn
3449b12cc7
Use assert.NilError() instead of assert.Assert()
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-01-21 13:16:02 +01:00
Sebastiaan van Stijn
2137b8ccf2
Add containerd, runc, and docker-init versions to /version
This patch adds version information about the containerd,
runc, and docker-init components to the /version endpoint.

With this patch applied, running:

```
curl --unix-socket /var/run/docker.sock http://localhost/version | jq .
```

Will produce this response:

```json
{
  "Platform": {
    "Name": ""
  },
  "Components": [
    {
      "Name": "Engine",
      "Version": "dev",
      "Details": {
        "ApiVersion": "1.40",
        "Arch": "amd64",
        "BuildTime": "2018-11-08T10:23:42.000000000+00:00",
        "Experimental": "false",
        "GitCommit": "7d02782d2f",
        "GoVersion": "go1.11.2",
        "KernelVersion": "4.9.93-linuxkit-aufs",
        "MinAPIVersion": "1.12",
        "Os": "linux"
      }
    },
    {
      "Name": "containerd",
      "Version": "v1.1.4",
      "Details": {
        "GitCommit": "9f2e07b1fc1342d1c48fe4d7bbb94cb6d1bf278b"
      }
    },
    {
      "Name": "runc",
      "Version": "1.0.0-rc5+dev",
      "Details": {
        "GitCommit": "a00bf0190895aa465a5fbed0268888e2c8ddfe85"
      }
    },
    {
      "Name": "docker-init",
      "Version": "0.18.0",
      "Details": {
        "GitCommit": "fec3683"
      }
    }
  ],
  "Version": "dev",
  "ApiVersion": "1.40",
  "MinAPIVersion": "1.12",
  "GitCommit": "7d02782d2f",
  "GoVersion": "go1.11.2",
  "Os": "linux",
  "Arch": "amd64",
  "KernelVersion": "4.9.93-linuxkit-aufs",
  "BuildTime": "2018-11-08T10:23:42.000000000+00:00"
}
```

When using a recent version of the CLI, that information is included in the
output of `docker version`:

```
Client: Docker Engine - Community
 Version:           18.09.0
 API version:       1.39
 Go version:        go1.10.4
 Git commit:        4d60db4
 Built:             Wed Nov  7 00:46:51 2018
 OS/Arch:           linux/amd64
 Experimental:      false

Server:
 Engine:
  Version:          dev
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.11.2
  Git commit:       7d02782d2f
  Built:            Thu Nov  8 10:23:42 2018
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          v1.1.4
  GitCommit:        9f2e07b1fc1342d1c48fe4d7bbb94cb6d1bf278b
 runc:
  Version:          1.0.0-rc5+dev
  GitCommit:        a00bf0190895aa465a5fbed0268888e2c8ddfe85
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683
```

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-01-14 23:27:05 +01:00
Sascha Grunert
7f3910c92e
Fix possible segfault in config reload
This commit fixes two possible crashes in the `*Daemon` bound method
`reloadMaxConcurrentDownloadsAndUploads()`.

The first fixed issue is when `daemon.imageService` is `nil`. The second
panic can occur if the provided `*config.Config` is incomplete and the
fields `conf.MaxConcurrentDownloads` or `conf.MaxConcurrentUploads` are
`nil`.

Signed-off-by: Sascha Grunert <sgrunert@suse.com>
2019-01-10 15:34:02 +01:00
Yong Tang
7315a2bb11 Fix go vet issue in daemon/daemon.go
This fix fixes go vet issue:
```
daemon/daemon.go:273: loop variable id captured by func literal
daemon/daemon.go:280: loop variable id captured by func literal
```

Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
2019-01-06 00:18:29 +00:00
Yong Tang
0104abf0d6
Merge pull request #38409 from innovimax/patch-1
fix typo
2019-01-04 23:35:09 -08:00
Brian Goff
6825db8c94
Merge pull request #38450 from thaJeztah/remove_deprecated_grpc_functions
Replace deprecated grpc.ErrorDesc() and grpc.Code() calls
2019-01-04 16:46:49 -08:00
Yong Tang
0281db99a9 Follow up to PR 38407
This fix is a follow up to PR 38407 to use assert.Error
and assert.NilError when appropriate

Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
2019-01-03 01:23:24 +00:00
Yong Tang
626022d0f6
Merge pull request #38407 from maximilianomaccanti/master
Add two configurable options to awslogs driver
2019-01-03 08:48:25 +08:00
Sebastiaan van Stijn
72b0b0387d
Replace deprecated grpc.ErrorDesc() and grpc.Code() calls
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-12-30 12:34:28 +01:00
Innovimax
31348a2936 fix typo
Signed-off-by: innovimax <innovimax@gmail.com>
2018-12-28 01:30:31 +01:00
Joao Trindade
a7020454ca
Add options validation to syslog logger test
Adds the following validations to the syslog logger test:

 1. Only supported options are valid
 2. Log option syslog-address has to be a valid URI
 3. Log option syslog-address if is file has to exist
 4. Log option syslog-address if udp/tcp scheme, default to port 513
 5. Log-option syslog-facility has to be a valid facility
 6. Log-option syslog-format has to be a valid format

Signed-off-by: Joao Trindade <trindade.joao@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-12-24 20:43:41 +01:00
Sebastiaan van Stijn
8fbf2598f5
Merge pull request #37940 from olljanat/replicas-max-per-node
Added support for maximum replicas per node
2018-12-24 19:29:53 +01:00
Olli Janatuinen
153171e9dd Added support for maximum replicas per node to services
Signed-off-by: Olli Janatuinen <olli.janatuinen@gmail.com>
2018-12-24 02:04:15 +02:00
Maximiliano Maccanti
ad8a8e8a9e NewStreamConfig UTest fixes
Signed-off-by: Maximiliano Maccanti <maccanti@amazon.com>
2018-12-21 22:24:40 +00:00
Maximiliano Maccanti
687cbfa739 Split StreamConfig from New, Utest table driven
Signed-off-by: Maximiliano Maccanti <maccanti@amazon.com>
2018-12-21 20:45:11 +00:00
Maximiliano Maccanti
512ac778bf Add two configurable options to awslogs driver
Add awslogs-force-flush-interval-seconds and awslogs-max-buffered-events configurable options to aswlogs driver to replace hardcoded values of repsectively 5 seconds and 4K.

Signed-off-by: Maximiliano Maccanti <maccanti@amazon.com>
2018-12-21 20:45:11 +00:00
Akihiro Suda
2cb26cfe9c
Merge pull request #38301 from cyphar/waitgroup-limits
daemon: switch to semaphore-gated WaitGroup for startup tasks
2018-12-22 00:07:55 +09:00
Aleksa Sarai
5a52917e4d
daemon: switch to semaphore-gated WaitGroup for startup tasks
Many startup tasks have to run for each container, and thus using a
WaitGroup (which doesn't have a limit to the number of parallel tasks)
can result in Docker exceeding the NOFILE limit quite trivially. A more
optimal solution is to have a parallelism limit by using a semaphore.

In addition, several startup tasks were not parallelised previously
which resulted in very long startup times. According to my testing, 20K
dead containers resulted in ~6 minute startup times (during which time
Docker is completely unusable).

This patch fixes both issues, and the parallelStartupTimes factor chosen
(128 * NumCPU) is based on my own significant testing of the 20K
container case. This patch (on my machines) reduces the startup time
from 6 minutes to less than a minute (ideally this could be further
reduced by removing the need to scan all dead containers on startup --
but that's beyond the scope of this patchset).

In order to avoid the NOFILE limit problem, we also detect this
on-startup and if NOFILE < 2*128*NumCPU we will reduce the parallelism
factor to avoid hitting NOFILE limits (but also emit a warning since
this is almost certainly a mis-configuration).

Signed-off-by: Aleksa Sarai <asarai@suse.de>
2018-12-21 21:51:02 +11:00
Vincent Demeester
bcd817ee6b
Merge pull request #38393 from thaJeztah/refactor_container_validation
Refactor container validation
2018-12-20 14:20:01 +01:00
Vincent Demeester
b33dc72523
Merge pull request #38335 from yongtang/38258-syslog-rfc5424
Add zero padding for RFC5424 syslog format
2018-12-20 08:30:22 +01:00
Sebastiaan van Stijn
f6002117a4
Extract container-config and container-hostconfig validation
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-12-19 13:09:12 +01:00
Sebastiaan van Stijn
5fc0f03426
Extract workingdir validation/conversion to a function
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-12-19 10:24:39 +01:00
Sebastiaan van Stijn
c0697c27aa
Extract port-mapping validation to a function
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-12-19 10:24:33 +01:00
Sebastiaan van Stijn
e1809510ca
Extract restart-policy-validation to a function
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-12-19 10:24:28 +01:00
Sebastiaan van Stijn
6a7da0b31b
Extract healthcheck-validation to a function
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-12-19 10:24:23 +01:00
Sebastiaan van Stijn
b6e373c525
Rename verifyContainerResources to verifyPlatformContainerResources
This validation function is platform-specific; rename it to be
more explicit.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-12-19 10:24:09 +01:00
Sebastiaan van Stijn
e278678705
Remove unused argument from verifyPlatformContainerSettings
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-12-19 09:23:09 +01:00
Sebastiaan van Stijn
10c97b9357
Unify logging container validation warnings
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-12-19 09:15:21 +01:00
Sebastiaan van Stijn
2e23ef5350
Move port-publishing check to linux platform-check
Windows does not have host-mode networking, so on Windows, this
check was a no-op

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-12-18 22:46:05 +01:00
Sebastiaan van Stijn
57f1305e74
Move "OOM Kill disable" warning to the daemon
Disabling the oom-killer for a container without setting a memory limit
is dangerous, as it can result in the container consuming unlimited memory,
without the kernel being able to kill it. A check for this situation is curently
done in the CLI, but other consumers of the API won't receive this warning.

This patch adds a check for this situation to the daemon, so that all consumers
of the API will receive this warning.

This patch will have one side-effect; docker cli's that also perform this check
client-side will print the warning twice; this can be addressed by disabling
the cli-side check for newer API versions, but will generate a bit of extra
noise when using an older CLI.

With this patch applied (and a cli that does not take the new warning into account);

```
docker create --oom-kill-disable busybox
WARNING: OOM killer is disabled for the container, but no memory limit is set, this can result in the system running out of resources.
669933b9b237fa27da699483b5cf15355a9027050825146587a0e5be0d848adf

docker run --rm --oom-kill-disable busybox
WARNING: Disabling the OOM killer on containers without setting a '-m/--memory' limit may be dangerous.
WARNING: OOM killer is disabled for the container, but no memory limit is set, this can result in the system running out of resources.
```

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-12-18 22:30:56 +01:00
Vincent Demeester
56cc56b0fa
Merge pull request #38126 from mjameswh/fix-1715
Use idtools.LookupGroup instead of parsing /etc/group file for docker.sock ownership
2018-12-12 17:29:28 +01:00
Vincent Demeester
d4a6e1c44f
Merge pull request #38068 from kolyshkin/err
More context for errors
2018-12-12 09:02:37 +01:00
Akihiro Suda
62d80835ab
Merge pull request #38342 from crosbymichael/oci-refactor
Move caps and device spec utils to `oci` pkg
2018-12-11 13:48:38 -08:00
Vincent Demeester
510805655b
Merge pull request #38265 from AkihiroSuda/remove-migrate-v1
Remove v1.10 migrator
2018-12-11 16:21:09 +01:00
Michael Crosby
b940cc5cff Move caps and device spec utils to oci pkg
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-12-11 10:20:25 -05:00
Kir Kolyshkin
6533136961 pkg/mount: wrap mount/umount errors
The errors returned from Mount and Unmount functions are raw
syscall.Errno errors (like EPERM or EINVAL), which provides
no context about what has happened and why.

Similar to os.PathError type, introduce mount.Error type
with some context. The error messages will now look like this:

> mount /tmp/mount-tests/source:/tmp/mount-tests/target, flags: 0x1001: operation not permitted

or

> mount tmpfs:/tmp/mount-test-source-516297835: operation not permitted

Before this patch, it was just

> operation not permitted

[v2: add Cause()]
[v3: rename MountError to Error, document Cause()]
[v4: fixes; audited all users]
[v5: make Error type private; changes after @cpuguy83 reviews]

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2018-12-10 20:07:02 -08:00
Kir Kolyshkin
2f98b5f51f aufs: get rid of mount()
The function is not needed as it's just a shallow wrapper around
unix.Mount().

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2018-12-10 20:06:10 -08:00
Kir Kolyshkin
77bc327e24 UnmountIpcMount: simplify
As standard mount.Unmount does what we need, let's use it.

In addition, this adds ignoring "not mounted" condition, which
was previously implemented (see PR#33329, commit cfa2591d3f)
via a very expensive call to mount.Mounted().

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2018-12-10 20:06:10 -08:00
Tibor Vass
6e3113f700
Merge pull request #38327 from andrewhsu/ctrd
update containerd to v1.2.1
2018-12-10 17:28:50 +01:00
Yong Tang
1082d1edf2 go vet fix for TestfillLicense
This small fix renames `TestfillLicense` to `TestFillLicense`
as otherwise go vet reports:
```
$ go tool vet daemon/licensing_test.go
daemon/licensing_test.go:11: TestfillLicense has malformed name: first letter after 'Test' must not be lowercase
```

Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
2018-12-09 00:51:37 +00:00
Yong Tang
fa6dabf876 Add zero padding for RFC5424 syslog format
This fix tries to address the issue raised in 38258
where current RFC5424 sys log format does not zero pad
the time (trailing zeros are removed)

This fix apply the patch to fix the issue. This fix fixes 38258.

Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
2018-12-08 22:40:02 +00:00
Andrew Hsu
78045a5419
use empty string as cgroup path to grab first find
Signed-off-by: Andrew Hsu <andrewhsu@docker.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-12-07 18:44:00 +01:00
Yong Tang
65d9a5dde5
Merge pull request #38267 from thaJeztah/wrap_errors
Use errors.Wrap() in daemon/config
2018-12-03 08:30:40 -08:00
Aleksa Sarai
f38ac72bca
oci: add integration tests for kernel.domainname configuration
This also includes a few refactors of oci_linux_test.go.

Signed-off-by: Aleksa Sarai <asarai@suse.de>
2018-11-30 19:44:50 +11:00
Aleksa Sarai
7417f50575
oci: include the domainname in "kernel.domainname"
The OCI doesn't have a specific field for an NIS domainname[1] (mainly
because FreeBSD and Solaris appear to have a similar concept but it is
configured entirely differently).

However, on Linux, the NIS domainname can be configured through both the
setdomainname(2) syscall but also through the "kernel.domainname"
sysctl. Since the OCI has a way of injecting sysctls this means we don't
need to have any OCI changes to support NIS domainnames (and we can
always switch if the OCI picks up such support in the future).

It should be noted that because we have to generate this each spec
creation we also have to make sure that it's not clobbered by the
HostConfig. I'm pretty sure making this change generic (so that
HostConfig will not clobber any pre-set sysctls) will not cause other
issues to crop up.

[1]: https://github.com/opencontainers/runtime-spec/issues/592

Signed-off-by: Aleksa Sarai <asarai@suse.de>
2018-11-30 17:31:38 +11:00
Sebastiaan van Stijn
a8d2b29e8d
Use errors.Wrap() in daemon/config
using `errors.Wrap()` preserves the original error

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-11-30 01:27:47 +01:00
James Watkins-Harvey
a2e384682b Use idtools.LookupGroup instead of parsing /etc/group file for docker.sock ownership
Signed-off-by: James Watkins-Harvey <jwatkins@progi-media.com>
2018-11-29 16:24:42 -05:00