Many startup tasks have to run for each container, and thus using a
WaitGroup (which doesn't have a limit to the number of parallel tasks)
can result in Docker exceeding the NOFILE limit quite trivially. A more
optimal solution is to have a parallelism limit by using a semaphore.
In addition, several startup tasks were not parallelised previously
which resulted in very long startup times. According to my testing, 20K
dead containers resulted in ~6 minute startup times (during which time
Docker is completely unusable).
This patch fixes both issues, and the parallelStartupTimes factor chosen
(128 * NumCPU) is based on my own significant testing of the 20K
container case. This patch (on my machines) reduces the startup time
from 6 minutes to less than a minute (ideally this could be further
reduced by removing the need to scan all dead containers on startup --
but that's beyond the scope of this patchset).
In order to avoid the NOFILE limit problem, we also detect this
on-startup and if NOFILE < 2*128*NumCPU we will reduce the parallelism
factor to avoid hitting NOFILE limits (but also emit a warning since
this is almost certainly a mis-configuration).
Signed-off-by: Aleksa Sarai <asarai@suse.de>
A lack of check in the test code can lead to a panic due to
`len(ids)` being `0`.
Avoid the panic by adding appropriate checks. Note `Assert()` should be
used rather than `Check()` as if it fails we should not proceed with the
test.
Originally found in https://github.com/moby/moby/pull/38404.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
go1.11.4 (released 2018/12/14) includes fixes to cgo, the compiler, linker,
runtime, documentation, go command, and the net/http and go/types packages. It
includes a fix to a bug introduced in Go 1.11.3 that broke go get for import
path patterns containing "...".
See the Go 1.11.4 milestone for details:
https://github.com/golang/go/issues?q=milestone%3AGo1.11.4+label%3ACherryPickApproved
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Some messages are logged before the logrus format was set,
therefore resulting in inconsistent log-message formatting
during startup;
Before this patch;
```
dockerd --experimental
WARN[0000] Running experimental build
INFO[2018-11-24T11:24:05.615249610Z] libcontainerd: started new containerd process pid=132
INFO[2018-11-24T11:24:05.615348322Z] parsed scheme: "unix" module=grpc
...
```
With this patch applied;
```
dockerd --experimental
WARN[2018-11-24T13:41:51.199057259Z] Running experimental build
INFO[2018-11-24T13:41:51.200412645Z] libcontainerd: started new containerd process pid=293
INFO[2018-11-24T13:41:51.200523051Z] parsed scheme: "unix" module=grpc
```
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Disabling the oom-killer for a container without setting a memory limit
is dangerous, as it can result in the container consuming unlimited memory,
without the kernel being able to kill it. A check for this situation is curently
done in the CLI, but other consumers of the API won't receive this warning.
This patch adds a check for this situation to the daemon, so that all consumers
of the API will receive this warning.
This patch will have one side-effect; docker cli's that also perform this check
client-side will print the warning twice; this can be addressed by disabling
the cli-side check for newer API versions, but will generate a bit of extra
noise when using an older CLI.
With this patch applied (and a cli that does not take the new warning into account);
```
docker create --oom-kill-disable busybox
WARNING: OOM killer is disabled for the container, but no memory limit is set, this can result in the system running out of resources.
669933b9b237fa27da699483b5cf15355a9027050825146587a0e5be0d848adf
docker run --rm --oom-kill-disable busybox
WARNING: Disabling the OOM killer on containers without setting a '-m/--memory' limit may be dangerous.
WARNING: OOM killer is disabled for the container, but no memory limit is set, this can result in the system running out of resources.
```
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
go1.11.13 (released 2018/12/14)
- crypto/x509: CPU denial of service in chain validation golang/go#29233
- cmd/go: directory traversal in "go get" via curly braces in import paths golang/go#29231
- cmd/go: remote command execution during "go get -u" golang/go#29230
See the Go 1.11.3 milestone on the issue tracker for details:
https://github.com/golang/go/issues?q=milestone%3AGo1.11.3
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This import got lost after commit 56cc56b0fa
was merged, likely because the PR was built against an outdated
master.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
