Signal systemd when we start shutting down to complement the "READY" notify
that was originally implemented in 97088ebef7
From [sd_notify(3)](https://www.freedesktop.org/software/systemd/man/sd_notify.html#STOPPING=1)
> STOPPING=1
> Tells the service manager that the service is beginning its shutdown. This is useful
> to allow the service manager to track the service's internal state, and present it to
> the user.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
People keep doing this and getting pwned because they accidentally left
it exposed to the internet.
The warning about doing this has been there forever.
This introduces a sleep after warning.
To disable the extra sleep users must explicitly specify `--tls=false`
or `--tlsverify=false`
Warning also specifies this sleep will be removed in the next release
where the flag will be required if running unauthenticated.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
This patch adds a new "prune" event type to indicate that pruning of a resource
type completed.
This event-type can be used on systems that want to perform actions after
resources have been cleaned up. For example, Docker Desktop performs an fstrim
after resources are deleted (https://github.com/linuxkit/linuxkit/tree/v0.7/pkg/trim-after-delete).
While the current (remove, destroy) events can provide information on _most_
resources, there is currently no event triggered after the BuildKit build-cache
is cleaned.
Prune events have a `reclaimed` attribute, indicating the amount of space that
was reclaimed (in bytes). The attribute can be used, for example, to use as a
threshold for performing fstrim actions. Reclaimed space for `network` events
will always be 0, but the field is added to be consistent with prune events for
other resources.
To test this patch:
Create some resources:
for i in foo bar baz; do \
docker network create network_$i \
&& docker volume create volume_$i \
&& docker run -d --name container_$i -v volume_$i:/volume busybox sh -c 'truncate -s 5M somefile; truncate -s 5M /volume/file' \
&& docker tag busybox:latest image_$i; \
done;
docker pull alpine
docker pull nginx:alpine
echo -e "FROM busybox\nRUN truncate -s 50M bigfile" | DOCKER_BUILDKIT=1 docker build -
Start listening for "prune" events in another shell:
docker events --filter event=prune
Prune containers, networks, volumes, and build-cache:
docker system prune -af --volumes
See the events that are returned:
docker events --filter event=prune
2020-07-25T12:12:09.268491000Z container prune (reclaimed=15728640)
2020-07-25T12:12:09.447890400Z network prune (reclaimed=0)
2020-07-25T12:12:09.452323000Z volume prune (reclaimed=15728640)
2020-07-25T12:12:09.517236200Z image prune (reclaimed=21568540)
2020-07-25T12:12:09.566662600Z builder prune (reclaimed=52428841)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Close#40484
Note that the support for cgroup v2 isn't ready for production yet,
regardless to rootful or rootless.
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
The namespaces com.docker.*, io.docker.*, org.dockerproject.*
have been documented to be reserved for Docker's internal use.
Co-Authored-By: Sebastiaan van Stijn <thaJeztah@users.noreply.github.com>
Signed-off-by: Jintao Zhang <zhangjintao9020@gmail.com>
This feature was used by docker build --stream and it was kept experimental.
Users of this endpoint should enable BuildKit anyway by setting Version to BuilderBuildKit.
Signed-off-by: Tibor Vass <tibor@docker.com>
Fix TODO: move into startMetricsServer()
Fix errors.Wrap return nil when passed err is nil
Co-Authored-By: Sebastiaan van Stijn <thaJeztah@users.noreply.github.com>
Signed-off-by: HuanHuan Ye <logindaveye@gmail.com>
also renamed the non-windows variant of this file to be
consistent with other files in this package
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This hack was added to fix a compatibility with clients
that were built using Go 1.5 and older (added in 3d6f5984f5)
This hack causes some problems with current clients; with Go 1.5 and older
no longer being supported for some time, and being several years old, it
should now be ok to remove this hack altogether.
People using tools that are built with those versions of Go wouldn't have
updated those for years, and are probably out of date anyway; that's not
something we can continue taking into account.
This will affect docker clients (the docker cli) for docker 1.12 and older.
Those versions have reached EOL a long time ago (and have known unpatched
vulnerabilities), so should no longer be used anyway, but We should add
a nebtuib in the release notes, just in case someone, somewhere, still
has such old tools.
For those affected, using a more recent client (and if needed, setting
the DOCKER_API_VERSION environment variable to the needed API version)
should provide a way out.
This reverts the changes originally made in; #22000 and #22888,
which were to address #20865.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Before this change:
dockerd -H unix:///run/docker.sock -H unix:///run/docker.sock -H unix:///run/docker.sock
...
INFO[2019-07-13T00:02:36.195090937Z] Daemon has completed initialization
INFO[2019-07-13T00:02:36.215940441Z] API listen on /run/docker.sock
INFO[2019-07-13T00:02:36.215933172Z] API listen on /run/docker.sock
INFO[2019-07-13T00:02:36.215990566Z] API listen on /run/docker.sock
After this change:
dockerd -H unix:///run/docker.sock -H unix:///run/docker.sock -H unix:///run/docker.sock
...
INFO[2019-07-13T00:01:37.533579874Z] Daemon has completed initialization
INFO[2019-07-13T00:01:37.567045771Z] API listen on /run/docker.sock
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This reverts commit 98fc09128b in order to
keep registry v2 schema1 handling and libtrust-key-based engine ID.
Because registry v2 schema1 was not officially deprecated and
registries are still relying on it, this patch puts its logic back.
However, registry v1 relics are not added back since v1 logic has been
removed a while ago.
This also fixes an engine upgrade issue in a swarm cluster. It was relying
on the Engine ID to be the same upon upgrade, but the mentioned commit
modified the logic to use UUID and from a different file.
Since the libtrust key is always needed to support v2 schema1 pushes,
that the old engine ID is based on the libtrust key, and that the engine ID
needs to be conserved across upgrades, adding a UUID-based engine ID logic
seems to add more complexity than it solves the problems.
Hence reverting the engine ID changes as well.
Signed-off-by: Tibor Vass <tibor@docker.com>
The `--rootless` flag had a couple of issues:
* #38702: euid=0, $USER="root" but no access to cgroup ("rootful" Docker in rootless Docker)
* #39009: euid=0 but $USER="docker" (rootful boot2docker)
To fix#38702, XDG dirs are ignored as in rootful Docker, unless the
dockerd is directly running under RootlessKit namespaces.
RootlessKit detection is implemented by checking whether `$ROOTLESSKIT_STATE_DIR` is set.
To fix#39009, the non-robust `$USER` check is now completely removed.
The entire logic can be illustrated as follows:
```
withRootlessKit := getenv("ROOTLESSKIT_STATE_DIR")
rootlessMode := withRootlessKit || cliFlag("--rootless")
honorXDG := withRootlessKit
useRootlessKitDockerProxy := withRootlessKit
removeCgroupSpec := rootlessMode
adjustOOMScoreAdj := rootlessMode
```
Close#39024Fix#38702#39009
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
`defer r.WaitTimeout(10s)` was in a wrong place and had caused the
daemon to hang for 10 seconds.
Fix#39025
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
Signed-off-by: John Howard <jhoward@microsoft.com>
This is the first step in refactoring moby (dockerd) to use containerd on Windows.
Similar to the current model in Linux, this adds the option to enable it for runtime.
It does not switch the graphdriver to containerd snapshotters.
- Refactors libcontainerd to a series of subpackages so that either a
"local" containerd (1) or a "remote" (2) containerd can be loaded as opposed
to conditional compile as "local" for Windows and "remote" for Linux.
- Updates libcontainerd such that Windows has an option to allow the use of a
"remote" containerd. Here, it communicates over a named pipe using GRPC.
This is currently guarded behind the experimental flag, an environment variable,
and the providing of a pipename to connect to containerd.
- Infrastructure pieces such as under pkg/system to have helper functions for
determining whether containerd is being used.
(1) "local" containerd is what the daemon on Windows has used since inception.
It's not really containerd at all - it's simply local invocation of HCS APIs
directly in-process from the daemon through the Microsoft/hcsshim library.
(2) "remote" containerd is what docker on Linux uses for it's runtime. It means
that there is a separate containerd service running, and docker communicates over
GRPC to it.
To try this out, you will need to start with something like the following:
Window 1:
containerd --log-level debug
Window 2:
$env:DOCKER_WINDOWS_CONTAINERD=1
dockerd --experimental -D --containerd \\.\pipe\containerd-containerd
You will need the following binary from github.com/containerd/containerd in your path:
- containerd.exe
You will need the following binaries from github.com/Microsoft/hcsshim in your path:
- runhcs.exe
- containerd-shim-runhcs-v1.exe
For LCOW, it will require and initrd.img and kernel in `C:\Program Files\Linux Containers`.
This is no different to the current requirements. However, you may need updated binaries,
particularly initrd.img built from Microsoft/opengcs as (at the time of writing), Linuxkit
binaries are somewhat out of date.
Note that containerd and hcsshim for HCS v2 APIs do not yet support all the required
functionality needed for docker. This will come in time - this is a baby (although large)
step to migrating Docker on Windows to containerd.
Note that the HCS v2 APIs are only called on RS5+ builds. RS1..RS4 will still use
HCS v1 APIs as the v2 APIs were not fully developed enough on these builds to be usable.
This abstraction is done in HCSShim. (Referring specifically to runtime)
Note the LCOW graphdriver still uses HCS v1 APIs regardless.
Note also that this does not migrate docker to use containerd snapshotters
rather than graphdrivers. This needs to be done in conjunction with Linux also
doing the same switch.
As people are using the UUID in `docker info` that was based on the v1 manifest signing key, replace
with a UUID instead.
Remove deprecated `--disable-legacy-registry` option that was scheduled to be removed in 18.03.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
This fix fixes the following issue with `go vet`:
```
$ go tool vet cmd/dockerd/daemon.go
cmd/dockerd/daemon.go:163: the cancel function is not used on all paths (possible context leak)
cmd/dockerd/daemon.go:167: this return statement may be reached without using the cancel var defined on line 163
```
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Please refer to `docs/rootless.md`.
TLDR:
* Make sure `/etc/subuid` and `/etc/subgid` contain the entry for you
* `dockerd-rootless.sh --experimental`
* `docker -H unix://$XDG_RUNTIME_DIR/docker.sock run ...`
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
`time.After` keeps a timer running until the specified duration is
completed. It also allocates a new timer on each call. This can wind up
leaving lots of uneccessary timers running in the background that are
not needed and consume resources.
Instead of `time.After`, use `time.NewTimer` so the timer can actually
be stopped.
In some of these cases it's not a big deal since the duraiton is really
short, but in others it is much worse.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Some messages are logged before the logrus format was set,
therefore resulting in inconsistent log-message formatting
during startup;
Before this patch;
```
dockerd --experimental
WARN[0000] Running experimental build
INFO[2018-11-24T11:24:05.615249610Z] libcontainerd: started new containerd process pid=132
INFO[2018-11-24T11:24:05.615348322Z] parsed scheme: "unix" module=grpc
...
```
With this patch applied;
```
dockerd --experimental
WARN[2018-11-24T13:41:51.199057259Z] Running experimental build
INFO[2018-11-24T13:41:51.200412645Z] libcontainerd: started new containerd process pid=293
INFO[2018-11-24T13:41:51.200523051Z] parsed scheme: "unix" module=grpc
```
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
