Commit graph

341 commits

Author SHA1 Message Date
Brian Goff
e7cf711c02
Move proxy CLI to main cmd/
Since this command is part of the official distribution and even
required for tests, let's move this up to the main cmd's.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2021-06-25 18:01:55 +02:00
aiordache
f558056d2b Make validate flag description more generic
Signed-off-by: aiordache <anca.iordache@docker.com>
2021-06-24 15:43:28 +00:00
Sebastiaan van Stijn
314759dc2f
Merge pull request #42393 from aiordache/daemon_config
Daemon config validation
2021-06-23 19:32:07 +02:00
Rich Horwood
8f80e55111 Add configuration validation option and tests.
Fixes #36911

If config file is invalid we'll exit anyhow, so this just prevents
the daemon from starting if the configuration is fine.

Mainly useful for making config changes and restarting the daemon
iff the config is valid.

Signed-off-by: Rich Horwood <rjhorwood@apple.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Anca Iordache <anca.iordache@docker.com>
2021-06-23 09:54:55 +00:00
Brian Goff
a7ea29a5a6
Merge pull request #42451 from thaJeztah/remove_lcow_step1
Remove LCOW code (step 1)
2021-06-08 13:41:45 -07:00
Akihiro Suda
0ad2293d0e
Merge pull request #41656 from thaJeztah/unexport_things 2021-06-08 12:07:40 +09:00
Sebastiaan van Stijn
e047d984dc
Remove LCOW code (step 1)
The LCOW implementation in dockerd has been deprecated in favor of re-implementation
in containerd (in progress). Microsoft started removing the LCOW V1 code from the
build dependencies we use in Microsoft/opengcs (soon to be part of Microsoft/hcshhim),
which means that we need to start removing this code.

This first step removes the lcow graphdriver, the LCOW initialization code, and
some LCOW-related utilities.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-06-03 21:16:21 +02:00
Brian Goff
4b981436fe Fixup libnetwork lint errors
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2021-06-01 23:48:32 +00:00
Brian Goff
a0a473125b Fix libnetwork imports
After moving libnetwork to this repo, we need to update all the import
paths for libnetwork to point to docker/docker/libnetwork instead of
docker/libnetwork.
This change implements that.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2021-06-01 21:51:23 +00:00
Sebastiaan van Stijn
bf07c06c63
daemon: move DefaultShimBinary, DefaultRuntimeBinary to config package
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-05-31 19:06:16 +02:00
Sebastiaan van Stijn
97a5b797b6
Use designated test domains (RFC2606) in tests
Some tests were using domain names that were intended to be "fake", but are
actually registered domain names (such as domain.com, registry.com, mytest.com).

Even though we were not actually making connections to these domains, it's
better to use domains that are designated for testing/examples in RFC2606:
https://tools.ietf.org/html/rfc2606

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-04-02 14:06:27 +02:00
Brian Goff
94c07441c2
buildkit: Apply apparmor profile
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit 611eb6ffb3)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-02-02 13:32:24 +01:00
Sebastiaan van Stijn
f3d0f7054d
cmd/dockerd: sd_notify STOPPING=1 when shutting down
Signal systemd when we start shutting down to complement the "READY" notify
that was originally implemented in 97088ebef7

From [sd_notify(3)](https://www.freedesktop.org/software/systemd/man/sd_notify.html#STOPPING=1)

> STOPPING=1
> Tells the service manager that the service is beginning its shutdown. This is useful
> to allow the service manager to track the service's internal state, and present it to
> the user.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-12-22 10:51:17 +01:00
Sebastiaan van Stijn
cf31b9622a
Merge pull request #41622 from bboehmke/ipv6_nat
IPv6 iptables config option
2020-12-07 11:59:42 +01:00
Sebastiaan van Stijn
6458f750e1
use containerd/cgroups to detect cgroups v2
libcontainer does not guarantee a stable API, and is not intended
for external consumers.

this patch replaces some uses of libcontainer/cgroups with
containerd/cgroups.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-11-09 15:00:32 +01:00
Tianon Gravi
d99e8d2c3b Unhide containerd-namespace flags
The daemon expects to manage/handle everything going into the containerd namespace it uses, and things break when it shares that namespace with a second instance, so we should expose these flags and expect users that want two daemons sharing a containerd instance to use separate namespaces for them.

Signed-off-by: Tianon Gravi <admwiggin@gmail.com>
2020-11-05 12:14:01 -08:00
Benjamin Böhmke
66459cc623 Added ip6tables config option
Signed-off-by: Benjamin Böhmke <benjamin@boehmke.net>
2020-11-05 16:18:23 +01:00
Sebastiaan van Stijn
2b8e68ef06
dockerd: remove default -500 oom-score-adjust
dockerd currently sets the oom-score-adjust itself. This functionality
was added when we did not yet run dockerd as a systemd service.

Now that we do, it's better to instead have systemd handle this.

Keeping the option itself for situations where dockerd is started
manually or without using systemd.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-10-16 18:49:52 +02:00
Brian Goff
816fbcd306
Merge pull request #41072 from AkihiroSuda/fix-41071
cgroup2: unshare cgroupns by default regardless to API version
2020-10-01 11:56:00 -07:00
Brian Goff
5f5285a6e2 Sterner warnings for unathenticated tcp
People keep doing this and getting pwned because they accidentally left
it exposed to the internet.

The warning about doing this has been there forever.
This introduces a sleep after warning.
To disable the extra sleep users must explicitly specify `--tls=false`
or `--tlsverify=false`

Warning also specifies this sleep will be removed in the next release
where the flag will be required if running unauthenticated.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2020-09-25 00:21:54 +00:00
Sebastiaan van Stijn
51c7992928
API: add "prune" events
This patch adds a new "prune" event type to indicate that pruning of a resource
type completed.

This event-type can be used on systems that want to perform actions after
resources have been cleaned up. For example, Docker Desktop performs an fstrim
after resources are deleted (https://github.com/linuxkit/linuxkit/tree/v0.7/pkg/trim-after-delete).

While the current (remove, destroy) events can provide information on _most_
resources, there is currently no event triggered after the BuildKit build-cache
is cleaned.

Prune events have a `reclaimed` attribute, indicating the amount of space that
was reclaimed (in bytes). The attribute can be used, for example, to use as a
threshold for performing fstrim actions. Reclaimed space for `network` events
will always be 0, but the field is added to be consistent with prune events for
other resources.

To test this patch:

Create some resources:

    for i in foo bar baz; do \
        docker network create network_$i \
        && docker volume create volume_$i \
        && docker run -d --name container_$i -v volume_$i:/volume busybox sh -c 'truncate -s 5M somefile; truncate -s 5M /volume/file' \
        && docker tag busybox:latest image_$i; \
    done;

    docker pull alpine
    docker pull nginx:alpine

    echo -e "FROM busybox\nRUN truncate -s 50M bigfile" | DOCKER_BUILDKIT=1 docker build -

Start listening for "prune" events in another shell:

    docker events --filter event=prune

Prune containers, networks, volumes, and build-cache:

    docker system prune -af --volumes

See the events that are returned:

    docker events --filter event=prune
    2020-07-25T12:12:09.268491000Z container prune  (reclaimed=15728640)
    2020-07-25T12:12:09.447890400Z network prune  (reclaimed=0)
    2020-07-25T12:12:09.452323000Z volume prune  (reclaimed=15728640)
    2020-07-25T12:12:09.517236200Z image prune  (reclaimed=21568540)
    2020-07-25T12:12:09.566662600Z builder prune  (reclaimed=52428841)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-07-28 12:41:14 +02:00
Akihiro Suda
79cfcba76c
cgroup2: unshare cgroupns by default regardless to API version
Fix #41071

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-06-15 16:11:32 +09:00
Sebastiaan van Stijn
2f003f7774
Merge pull request #40930 from thaJeztah/bump_cobra
vendor: spf13/cobra v1.0.0, spf13/pflag v1.0.5
2020-05-14 21:28:51 +02:00
Tibor Vass
298ba5b131
Merge pull request #40427 from thaJeztah/prometheus_remove_experimental
Do not require "experimental" for metrics API
2020-05-08 11:10:53 -07:00
Sebastiaan van Stijn
dc0a400738
vendor: spf13/cobra v1.0.0
full diff: https://github.com/spf13/cobra/compare/v0.0.3...v1.0.0

Notable Changes

- Fish completion (including support for Go custom completion)
- API (urgent): Rename BashCompDirectives to ShellCompDirectives
- Remove/replace SetOutput on Command - deprecated
- Custom completions coded in Go (instead of Bash)
- Partial Revert of 922
- Correct documentation for InOrStdin
- Apply formatting to templates
- Revert change so help is printed on stdout again
- Update md2man to v2.0.0
- update viper to v1.4.0
- Update cmd/root.go example in README.md

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-05-08 10:44:36 +02:00
Sebastiaan van Stijn
c3b3aedfa4
Merge pull request #40662 from AkihiroSuda/cgroup2-dockerinfo
cgroup2: implement `docker info`
2020-04-29 22:57:00 +02:00
Akihiro Suda
90f19fc375 cmd/dockerd: remove outdated "experimental" annotation
Follow up to #40759

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-04-29 11:06:11 +09:00
Sebastiaan van Stijn
8312004f41
remove uses of deprecated pkg/term
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-04-21 16:29:27 +02:00
Sebastiaan van Stijn
f337a8d21d
Do not require "experimental" for metrics API
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-04-20 22:19:00 +02:00
Brian Goff
130b0bc603
Merge pull request #40823 from thaJeztah/remove_distrib_uuid
daemon: remove distribution/uuid package
2020-04-16 20:59:58 -07:00
Akihiro Suda
f350b53241 cgroup2: implement docker info
ref: https://www.kernel.org/doc/html/latest/admin-guide/cgroup-v2.html

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-04-17 07:20:01 +09:00
Sebastiaan van Stijn
2376ebd3f2
fix --cluster-store-opt deprecation annotation
Commit 1470697b67 marked this flag
as deprecated, but inadvertedtly used the plural (which is used
for `daemon.json`, but not for the flag name.

This fixes the inccorect name of the flag.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-04-16 14:42:19 +02:00
Sebastiaan van Stijn
7400375526
daemon: remove distribution/uuid package
This appeared to be unused because we no longer generate
a uuid using this package.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-04-16 09:16:38 +02:00
Tonis Tiigi
0cdf6ba9c8 vendor: update buildkit to ae7ff174
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-04-14 08:26:07 -07:00
Akihiro Suda
5ca47f5179 rootless: graduate from experimental
Close #40484

Note that the support for cgroup v2 isn't ready for production yet,
regardless to rootful or rootless.

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-04-07 00:59:15 +09:00
Akihiro Suda
9a82a9a8ea vendor containerd, BuildKit, protobuf, grpc, and golang.org/x
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-03-03 10:25:20 +09:00
Brian Goff
bc1c0c7a8a
Merge pull request #40510 from aiordache/moby_cluster_flags_deprecate
Deprecate '--cluster-xx' options and add warning
2020-02-27 11:25:31 -08:00
Anca Iordache
f7c4be86c6 fixed deprecation message
Signed-off-by: Anca Iordache <anca.iordache@docker.com>
2020-02-19 11:22:20 +01:00
Anca Iordache
1470697b67 Deprecate '--cluster-xx' options and add warning
Co-authored-by: Yves Brissaud <yves.brissaud@gmail.com>

Signed-off-by: Anca Iordache <anca.iordache@docker.com>
2020-02-12 18:33:23 +01:00
Jintao Zhang
35d6c1870f enforce reserve internal labels.
The namespaces com.docker.*, io.docker.*, org.dockerproject.*
have been documented to be reserved for Docker's internal use.

Co-Authored-By: Sebastiaan van Stijn <thaJeztah@users.noreply.github.com>
Signed-off-by: Jintao Zhang <zhangjintao9020@gmail.com>
2020-02-12 12:03:35 +08:00
Sebastiaan van Stijn
9f0b3f5609
bump gotest.tools v3.0.1 for compatibility with Go 1.14
full diff: https://github.com/gotestyourself/gotest.tools/compare/v2.3.0...v3.0.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-02-11 00:06:42 +01:00
Sebastiaan van Stijn
ca20bc4214
Merge pull request #40007 from arkodg/add-host-docker-internal
Support host.docker.internal in dockerd on Linux
2020-01-27 13:42:26 +01:00
Arko Dasgupta
92e809a680 Support host.docker.internal in dockerd on Linux
Docker Desktop (on MAC and Windows hosts) allows containers
running inside a Linux VM to connect to the host using
the host.docker.internal DNS name, which is implemented by
VPNkit (DNS proxy on the host)

This PR allows containers to connect to Linux hosts
by appending a special string "host-gateway" to --add-host
e.g. "--add-host=host.docker.internal:host-gateway" which adds
host.docker.internal DNS entry in /etc/hosts and maps it to host-gateway-ip

This PR also add a daemon flag call host-gateway-ip which defaults to
the default bridge IP
Docker Desktop will need to set this field to the Host Proxy IP
so DNS requests for host.docker.internal can be routed to VPNkit

Addresses: https://github.com/docker/for-linux/issues/264

Signed-off-by: Arko Dasgupta <arko.dasgupta@docker.com>
2020-01-22 13:30:00 -08:00
Akihiro Suda
19baeaca26 cgroup2: enable cgroup namespace by default
For cgroup v1, we were unable to change the default because of
compatibility issue.

For cgroup v2, we should change the default right now because switching
to cgroup v2 is already breaking change.

See also containers/libpod#4363 containers/libpod#4374

Privileged containers also use cgroupns=private by default.
https://github.com/containers/libpod/pull/4374#issuecomment-549776387

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-01-01 02:58:40 +09:00
Sebastiaan van Stijn
6b91ceff74
Use hcsshim osversion package for Windows versions
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-22 02:53:00 +02:00
Tibor Vass
6ca3ec88ae builder: remove legacy build's session handling
This feature was used by docker build --stream and it was kept experimental.

Users of this endpoint should enable BuildKit anyway by setting Version to BuilderBuildKit.

Signed-off-by: Tibor Vass <tibor@docker.com>
2019-10-02 20:29:15 +00:00
Lukas Heeren
ce61a1ed98
Adding ability to change max download attempts
Moby works perfectly when you are in a situation when one has a good and stable
internet connection. Operating in area's where internet connectivity is likely
to be lost in undetermined intervals, like a satellite connection or 4G/LTE in
rural area's, can become a problem when pulling a new image. When connection is
lost while image layers are being pulled, Moby will try to reconnect up to 5 times.
If this fails, the incompletely downloaded layers are lost will need to be completely
downloaded again during the next pull request. This means that we are using more
data than we might have to.

Pulling a layer multiple times from the start can become costly over a satellite
or 4G/LTE connection. As these techniques (especially 4G) quite common in IoT and
Moby is used to run Azure IoT Edge devices, I would like to add a settable maximum
download attempts. The maximum download attempts is currently set at 5
(distribution/xfer/download.go). I would like to change this constant to a variable
that the user can set. The default will still be 5, so nothing will change from
the current version unless specified when starting the daemon with the added flag
or in the config file.

I added a default value of 5 for DefaultMaxDownloadAttempts and a settable
max-download-attempts in the daemon config file. It is also added to the config
of dockerd so it can be set with a flag when starting the daemon. This value gets
stored in the imageService of the daemon when it is initiated and can be passed
to the NewLayerDownloadManager as a parameter. It will be stored in the
LayerDownloadManager when initiated. This enables us to set the max amount of
retries in makeDownoadFunc equal to the max download attempts.

I also added some tests that are based on maxConcurrentDownloads/maxConcurrentUploads.

You can pull this version and test in a development container. Either create a config
`file /etc/docker/daemon.json` with `{"max-download-attempts"=3}``, or use
`dockerd --max-download-attempts=3 -D &` to start up the dockerd. Start downloading
a container and disconnect from the internet whilst downloading. The result would
be that it stops pulling after three attempts.

Signed-off-by: Lukas Heeren <lukas-heeren@hotmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-19 13:51:40 +02:00
Sebastiaan van Stijn
90c372bb2c
cmd/dockerd: explicitly ignore unchecked errors (errcheck)
```
cmd/dockerd/config.go:50:18: Error return value of `flags.MarkHidden` is not checked (errcheck)
cmd/dockerd/config.go:55:22: Error return value of `flags.MarkDeprecated` is not checked (errcheck)
cmd/dockerd/config.go:78:18: Error return value of `flags.MarkHidden` is not checked (errcheck)
```

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-18 12:57:53 +02:00
Sebastiaan van Stijn
07ff4f1de8
goimports: fix imports
Format the source according to latest goimports.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-18 12:56:54 +02:00
HuanHuan Ye
88c554f950 DaemonCli: Move check into startMetricsServer
Fix TODO: move into startMetricsServer()
Fix errors.Wrap return nil when passed err is nil

Co-Authored-By: Sebastiaan van Stijn <thaJeztah@users.noreply.github.com>
Signed-off-by: HuanHuan Ye <logindaveye@gmail.com>
2019-09-12 15:18:05 +08:00