Commit graph

1243 commits

Author SHA1 Message Date
Sebastiaan van Stijn
0fc914de5c
Merge pull request #40595 from sfzhu93/GL_2_25
[integration] add buffer to prevent goroutine leak
2020-05-08 16:00:52 +02:00
Akihiro Suda
33ee7941d4 support --privileged --cgroupns=private on cgroup v1
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-04-21 23:11:32 +09:00
Akihiro Suda
ba8129b28a
Merge pull request #39946 from cpuguy83/plugins_use_containerd_dist
Replace plugin distribution code w/ containerd
2020-04-21 10:03:16 +09:00
Sebastiaan van Stijn
c8e31dc2f2
Merge pull request #39882 from thaJeztah/swarm_pids_limit
Add API support for PidsLimit on services
2020-04-16 21:02:30 +02:00
Sebastiaan van Stijn
54d88a7cd3
Merge pull request #40478 from cpuguy83/dont-prime-the-stats
Add stats options to not prime the stats
2020-04-16 20:57:06 +02:00
Sebastiaan van Stijn
157c53c8e0
Add API support for PidsLimit on services
Support for PidsLimit was added to SwarmKit in docker/swarmkit/pull/2415,
but never exposed through the Docker remove API.

This patch exposes the feature in the repote API.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-04-15 22:37:42 +02:00
Brian Goff
eac618caaa
Merge pull request #39771 from thaJeztah/cleanup_TestTemplatedConfig
Integration: fix various issues in "config" and "secret" tests
2020-04-15 10:35:33 -07:00
Sebastiaan van Stijn
cdc39fa29c
integration/config: add check for removing non-existing config
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-04-14 18:31:38 +02:00
Sebastiaan van Stijn
1a7ffe4fe4
integration/secret: add check for empty list not producing an error
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-04-14 18:31:36 +02:00
Sebastiaan van Stijn
632cc7019a
integration/config: remove check that was already done in TestConfigInspect
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-04-14 18:31:34 +02:00
Sebastiaan van Stijn
26f2eddaa3
integration/config: combine TestConfigCreateWithLabels with TestConfigsCreateAndDelete
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-04-14 18:31:29 +02:00
Sebastiaan van Stijn
3c4b258f21
integration/secret: don't string-match errors
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-04-14 18:31:26 +02:00
Sebastiaan van Stijn
18cae89436
integration/config: don't string-match errors
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-04-14 18:31:24 +02:00
Sebastiaan van Stijn
348f412d85
integration: sync minor changes between config and secret tests
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-04-14 18:31:22 +02:00
Sebastiaan van Stijn
a45c89ecd1
integration/secret: fix TestSecretInspect not actually checking response
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-04-14 18:31:21 +02:00
Sebastiaan van Stijn
25424cf772
integration/config: move functions to be in line with "secret" tests
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-04-14 18:31:19 +02:00
Sebastiaan van Stijn
3c6f018f94
integration/secret: TestTemplatedSecret: simplify task code
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-04-14 18:31:17 +02:00
Sebastiaan van Stijn
de78663181
integration/secret: rename variables that collided with imported package name
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-04-14 18:31:15 +02:00
Sebastiaan van Stijn
dd67b4794e
integration/config: TestTemplatedConfig: simplify task code
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-04-14 18:31:13 +02:00
Sebastiaan van Stijn
ec78112edd
integration/config: rename variables that collided with imported package name
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-04-14 18:31:11 +02:00
Brian Goff
ced91bee4b On startup, actually shutdown the container.
When a container is left running after the daemon exits (e.g. the daemon
is SIGKILL'd or crashes), it should stop any running containers when the
daemon starts back up.

What actually happens is the daemon only sends the container's
configured stop signal and does not check if it has exited.
If the container does not actually exit then it is left running.

This fixes this unexpected behavior by calling the same function to shut
down the container that the daemon shutdown process does.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2020-04-13 14:20:12 -07:00
Brian Goff
d7ba1f85ef Use containerd dist libs for plugin pull/pull
This removes the use of the old distribution code in the plugin packages
and replaces it with containerd libraries for plugin pushes and pulls.

Additionally it uses a content store from containerd which seems like
it's compatible with the old "basicBlobStore" in the plugin package.
This is being used locally isntead of through the containerd client for
now.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2020-04-02 15:03:06 -07:00
Sebastiaan van Stijn
af0415257e
Merge pull request #40694 from kolyshkin/moby-sys-mount-part-II
switch to moby/sys/{mount,mountinfo} part II
2020-04-02 21:52:21 +02:00
Brian Goff
7a9cb29fb9 Accept platform spec on container create
This enables image lookup when creating a container to fail when the
reference exists but it is for the wrong platform. This prevents trying
to run an image for the wrong platform, as can be the case with, for
example binfmt_misc+qemu.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2020-03-20 16:10:36 -07:00
Kir Kolyshkin
39048cf656 Really switch to moby/sys/mount*
Switch to moby/sys/mount and mountinfo. Keep the pkg/mount for potential
outside users.

This commit was generated by the following bash script:

```
set -e -u -o pipefail

for file in $(git grep -l 'docker/docker/pkg/mount"' | grep -v ^pkg/mount); do
	sed -i -e 's#/docker/docker/pkg/mount"#/moby/sys/mount"#' \
		-e 's#mount\.\(GetMounts\|Mounted\|Info\|[A-Za-z]*Filter\)#mountinfo.\1#g' \
		$file
	goimports -w $file
done
```

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2020-03-20 09:46:25 -07:00
Akihiro Suda
5e1b246b9a test-integration: support more rootless tests
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-03-16 17:37:38 +09:00
Akihiro Suda
5bf33adba2 hack: support $DOCKER_ROOTLESS for testing rootless
```
$ DOCKER_EXPERIMENTAL=1 DOCKER_ROOTLESS=1 TEST_SKIP_INTEGRATION_CLI=1 \
 make test-integration
```

test-integration-cli is unsupported currently.
Also, tests that spawn custom daemon (testutil/daemon) are skipped.

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-03-10 05:49:45 +09:00
Arko Dasgupta
2e0762ae44
Add TC for com.docker.network.host_ipv4 label
This PR adds a testcase for the com.docker.network.host_ipv4
label commited via https://github.com/docker/libnetwork/pull/2454

Signed-off-by: Arko Dasgupta <arko.dasgupta@docker.com>
2020-03-09 18:51:19 +01:00
Sebastiaan van Stijn
e1710b42d3
vendor: bump libnetwork bf2bd42abc0a3734f12b5ec724e571434e42c669
full diff: 264bffcb88...bf2bd42abc

relevant changes:

- docker/libnetwork#2407 Macvlan internal network should not change default gateway
    - fixes docker/libnetwork#2406 Internal macvlan network overrides default gateway
- vendor godbus/dbus v5
- Fix InhibitIPv4 nil panic
- Cleanup VFP during overlay network removal
    - fixes VFP leak in windows overlay network deletion

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-03-07 00:35:47 +01:00
Akihiro Suda
3fbc41fe36 TestCgroupDriverSystemdMemoryLimit: fix failure
The test was failing:

  --- FAIL: TestCgroupDriverSystemdMemoryLimit (1.76s)
      cgroupdriver_systemd_test.go:54: assertion failed: 67108864 (s.HostConfig.Memory int64) != 67108864 (mem int)

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-03-03 11:30:55 +09:00
Brian Goff
ce1ceeb257 Add stats options to not prime the stats
Metrics collectors generally don't need the daemon to prime the stats
with something to compare since they already have something to compare
with.
Before this change, the API does 2 collection cycles (which takes
roughly 2s) in order to provide comparison for CPU usage over 1s. This
was primarily added so that `docker stats --no-stream` had something to
compare against.

Really the CLI should have just made a 2nd call and done the comparison
itself rather than forcing it on all API consumers.
That ship has long sailed, though.

With this change, clients can set an option to just pull a single stat,
which is *at least* a full second faster:

Old:
```
time curl --unix-socket
/go/src/github.com/docker/docker/bundles/test-integration-shell/docker.sock
http://./containers/test/stats?stream=false\&one-shot=false > /dev/null
2>&1

real0m1.864s
user0m0.005s
sys0m0.007s

time curl --unix-socket
/go/src/github.com/docker/docker/bundles/test-integration-shell/docker.sock
http://./containers/test/stats?stream=false\&one-shot=false > /dev/null
2>&1

real0m1.173s
user0m0.010s
sys0m0.006s
```

New:
```
time curl --unix-socket
/go/src/github.com/docker/docker/bundles/test-integration-shell/docker.sock
http://./containers/test/stats?stream=false\&one-shot=true > /dev/null
2>&1
real0m0.680s
user0m0.008s
sys0m0.004s

time curl --unix-socket
/go/src/github.com/docker/docker/bundles/test-integration-shell/docker.sock
http://./containers/test/stats?stream=false\&one-shot=true > /dev/null
2>&1

real0m0.156s
user0m0.007s
sys0m0.007s
```

This fixes issues with downstreams ability to use the stats API to
collect metrics.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2020-02-28 09:54:37 -08:00
Ziheng Liu
c322af8019 test: add buffer to prevent goroutine leak
Signed-off-by: Ziheng Liu <lzhfromustc@gmail.com>
2020-02-27 21:37:26 -05:00
Sebastiaan van Stijn
39679991f4
Merge pull request #40543 from SamWhited/upstream_logging
Upstream logging changes from Enterprise Edition
2020-02-27 13:54:14 +01:00
Akihiro Suda
26e6708ccf
Merge pull request #40475 from cpuguy83/fix_client_exec_type_id
Exec inspect field should be "ID" not "ExecID"
2020-02-26 19:11:33 +09:00
Sam Whited
a6f99232f5 Always execute tests and skip in tests themselves
See issue #40193

Signed-off-by: Sam Whited <sam@samwhited.com>
2020-02-21 14:31:01 -05:00
Brian Goff
750f0d1648 Support configuration of log cacher.
Configuration over the API per container is intentionally left out for
the time being, but is supported to configure the default from the
daemon config.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit cbecf48bc352e680a5390a7ca9cff53098cd16d7)
Signed-off-by: Madhu Venugopal <madhu@docker.com>
2020-02-19 17:02:34 -05:00
Brian Goff
e2ceb83a53 Support reads for all log drivers.
This supplements any log driver which does not support reads with a
custom read implementation that uses a local file cache.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit d675e2bf2b75865915c7a4552e00802feeb0847f)
Signed-off-by: Madhu Venugopal <madhu@docker.com>
2020-02-19 17:01:44 -05:00
Sebastiaan van Stijn
58c2615208
Merge pull request #40497 from arkodg/fix-bip-subnet-config
Set the bip network value as the subnet
2020-02-12 12:41:29 +01:00
Sebastiaan van Stijn
562880b276
Fix more goimports
```
daemon/logger/splunk/splunk_test.go:33: File is not `goimports`-ed (goimports)
        envKey:      "a",
        envRegexKey: "^foo",
        labelsKey:   "b",
        tagKey:      "c",
integration/build/build_test.go:41: File is not `goimports`-ed (goimports)
            rm:      false,
            forceRm: false,
integration/image/remove_unix_test.go:49: File is not `goimports`-ed (goimports)
        Root: d.Root,
```

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-02-11 18:56:25 +01:00
Arko Dasgupta
f800d5f786 Set the bip network value as the subnet
Dont assign the --bip value directly to the subnet
for the default bridge. Instead use the network value
from the ParseCIDR output

Addresses: https://github.com/moby/moby/issues/40392

Signed-off-by: Arko Dasgupta <arko.dasgupta@docker.com>
2020-02-10 17:38:54 -08:00
Sebastiaan van Stijn
9f0b3f5609
bump gotest.tools v3.0.1 for compatibility with Go 1.14
full diff: https://github.com/gotestyourself/gotest.tools/compare/v2.3.0...v3.0.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-02-11 00:06:42 +01:00
Brian Goff
cc993a9cbf Exec inspect field should be "ID" not "ExecID"
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2020-02-10 14:08:54 -08:00
Sebastiaan van Stijn
e55bff541e
TestLoginFailsWithBadCredentials: prepare for Go 1.14 changes
Go 1.14 adds quotes around the url in the error returned:

    === FAIL: arm64.integration.system TestLoginFailsWithBadCredentials (0.27s)
        TestLoginFailsWithBadCredentials: login_test.go:27: assertion failed: expected error "Error response from daemon: Get https://registry-1.docker.io/v2/: unauthorized: incorrect username or password", got "Error response from daemon: Get \"https://registry-1.docker.io/v2/\": unauthorized: incorrect username or password"
            Error response from daemon: Get "https://registry-1.docker.io/v2/": unauthorized: incorrect username or password

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-02-10 19:05:12 +01:00
Brian Goff
f464c31668 Check tmpfs mounts before create anon volume
This makes sure that things like `--tmpfs` mounts over an anonymous
volume don't create volumes uneccessarily.
One method only checks mountpoints, the other checks both mountpoints
and tmpfs... the usage of these should likely be consolidated.

Ideally, processing for `--tmpfs` mounts would get merged in with the
rest of the mount parsing. I opted not to do that for this change so the
fix is minimal and can potentially be backported with fewer changes of
breaking things.
Merging the mount processing for tmpfs can be handled in a followup.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2020-02-04 10:12:05 -08:00
Sebastiaan van Stijn
ca20bc4214
Merge pull request #40007 from arkodg/add-host-docker-internal
Support host.docker.internal in dockerd on Linux
2020-01-27 13:42:26 +01:00
Arko Dasgupta
92e809a680 Support host.docker.internal in dockerd on Linux
Docker Desktop (on MAC and Windows hosts) allows containers
running inside a Linux VM to connect to the host using
the host.docker.internal DNS name, which is implemented by
VPNkit (DNS proxy on the host)

This PR allows containers to connect to Linux hosts
by appending a special string "host-gateway" to --add-host
e.g. "--add-host=host.docker.internal:host-gateway" which adds
host.docker.internal DNS entry in /etc/hosts and maps it to host-gateway-ip

This PR also add a daemon flag call host-gateway-ip which defaults to
the default bridge IP
Docker Desktop will need to set this field to the Host Proxy IP
so DNS requests for host.docker.internal can be routed to VPNkit

Addresses: https://github.com/docker/for-linux/issues/264

Signed-off-by: Arko Dasgupta <arko.dasgupta@docker.com>
2020-01-22 13:30:00 -08:00
Drew Erny
30d9fe30b1 Add swarm jobs
Adds support for ReplicatedJob and GlobalJob service modes. These modes
allow running service which execute tasks that exit upon success,
instead of daemon-type tasks.

Signed-off-by: Drew Erny <drew.erny@docker.com>
2020-01-13 13:21:12 -06:00
Sebastiaan van Stijn
d64156970f
Merge pull request #40309 from dperny/bump-swarmkit
Bump swarmkit to 24fb4cfe8af56803640180c5592bf32da732ced2
2020-01-09 20:16:33 +01:00
Drew Erny
07efe6a0a7 Bump swarmkit to 24fb4cfe8af56803640180c5592bf32da732ced2
Bumps the vendoring of github.com/docker/swarmkit to the above commit,
which is the current master at commit time.

Most notably, this includes a change making the ingress network respect
the default address pool. Because of this change, a change to network
integration tests was needed.

Signed-off-by: Drew Erny <drew.erny@docker.com>
2020-01-07 09:43:22 -06:00
Akihiro Suda
19baeaca26 cgroup2: enable cgroup namespace by default
For cgroup v1, we were unable to change the default because of
compatibility issue.

For cgroup v2, we should change the default right now because switching
to cgroup v2 is already breaking change.

See also containers/libpod#4363 containers/libpod#4374

Privileged containers also use cgroupns=private by default.
https://github.com/containers/libpod/pull/4374#issuecomment-549776387

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-01-01 02:58:40 +09:00