This fixes a panic when an admin specifies a custom default runtime,
when a plugin is started the shim config is nil.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Commit f2f5106c92 added this test to verify loading
of images that were built with user-namespaces enabled.
However, because this test spins up a new daemon, not the daemon that's set up by
the test-suite's `TestMain()` (which loads the frozen images).
As a result, the `debian:bullseye` image was pulled from Docker Hub when running
the test;
Calling POST /v1.41/images/load?quiet=1
Applying tar in /go/src/github.com/docker/docker/bundles/test-integration/TestBuildUserNamespaceValidateCapabilitiesAreV2/d4d366b15997b/root/165536.165536/overlay2/3f7f9375197667acaf7bc810b34689c21f8fed9c52c6765c032497092ca023d6/diff" storage-driver=overlay
Applied tar sha256:845f0e5159140e9dbcad00c0326c2a506fbe375aa1c229c43f082867d283149c to 3f7f9375197667acaf7bc810b34689c21f8fed9c52c6765c032497092ca023d6, size: 5922359
Calling POST /v1.41/build?buildargs=null&cachefrom=null&cgroupparent=&cpuperiod=0&cpuquota=0&cpusetcpus=&cpusetmems=&cpushares=0&dockerfile=&labels=null&memory=0&memswap=0&networkmode=&rm=0&shmsize=0&t=capabilities%3A1.0&target=&ulimits=null&version=
Trying to pull debian from https://registry-1.docker.io v2
Fetching manifest from remote" digest="sha256:f169dbadc9021fc0b08e371d50a772809286a167f62a8b6ae86e4745878d283d" error="<nil>" remote="docker.io/library/debian:bullseye
Pulling ref from V2 registry: debian:bullseye
...
This patch updates `TestBuildUserNamespaceValidateCapabilitiesAreV2` to load the
frozen image. `StartWithBusybox` is also changed to `Start`, because the test
is not using the busybox image, so there's no need to load it.
In a followup, we should probably add some utilities to make this easier to set up
(and to allow passing the list frozen images that we want to load, without having
to "hard-code" the image name to load).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Using `d.Kill()` with rootless mode causes the restarted daemon to not
be able to start containerd (it times out).
Originally this was SIGKILLing the daemon because we were hoping to not
have to manipulate on disk state, but since we need to anyway we can
shut it down normally.
I also tested this to ensure the test fails correctly without the fix
that the test was added to check for.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Adds a test case for the case where dockerd gets stuck on startup due to
hanging `daemon.shutdownContainer`
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
When building images in a user-namespaced container, v3 capabilities are
stored including the root UID of the creator of the user-namespace.
This UID does not make sense outside the build environment however. If
the image is run in a non-user-namespaced runtime, or if a user-namespaced
runtime uses a different UID, the capabilities requested by the effective
bit will not be honoured by `execve(2)` due to this mismatch.
Instead, we convert v3 capabilities to v2, dropping the root UID on the
fly.
Signed-off-by: Eric Mountain <eric.mountain@datadoghq.com>
Capabilities are serialised in VFS_CAP_REVISION_3 when an image is
built in a user-namespaced daemon, instead of VFS_CAP_REVISION_2.
This adds a test for this, though it's currently wired to fail if
the capabilities are serialised in VFS_CAP_REVISION_2 instead in this
situation, since this is unexpected.
Signed-off-by: Eric Mountain <eric.mountain@datadoghq.com>
This adds a regression test for https://github.com/moby/moby/issues/24595
make DOCKER_GRAPHDRIVER=vfs TEST_FILTER='TestNetworkList' test-integration
INFO: Testing against a local daemon
=== RUN TestNetworkList
=== RUN TestNetworkList//networks
=== PAUSE TestNetworkList//networks
=== RUN TestNetworkList//networks/
=== PAUSE TestNetworkList//networks/
=== CONT TestNetworkList//networks
=== CONT TestNetworkList//networks/
--- PASS: TestNetworkList (0.05s)
--- PASS: TestNetworkList//networks/ (0.01s)
--- PASS: TestNetworkList//networks (0.01s)
PASS
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The free disk space on the Windows RS5 CI nodes appears to be just the
right size that the TestBuildWCOWSandboxSize test can generate 21GB of
layers, and then a 21GB sandbox inside a container, and then runs out of
space while committing the layer.
Helpfully, this failure is distinguishable in the logs from a failure
when the sandbox is too small, so we can do that.
TODO: Revert this if-and-when the Windows-RS5 CI nodes have more free
space.
Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
This test validates that `RUN` and `COPY` both target a read-write
sandbox on Windows that is configured according to the daemon's
`storage-opts` setting.
Sadly, this is a slow test, so we need to bump the timeout to 60 minutes
from the default of 10 minutes.
Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
This fixes a regression based on expectations of the runtime:
```
docker pull arm32v7/alpine
docker run arm32v7/alpine
```
Without this change, the `docker run` will fail due to platform
matching on non-arm32v7 systems, even though the image could run
(assuming the system is setup correctly).
This also emits a warning to make sure that the user is aware that a
platform that does not match the default platform of the system is being
run, for the cases like:
```
docker pull --platform armhf busybox
docker run busybox
```
Not typically an issue if the requests are done together like that, but
if the image was already there and someone did `docker run` without an
explicit `--platform`, they may very well be expecting to run a native
version of the image instead of the armhf one.
This warning does add some extra noise in the case of platform specific
images being run, such as `arm32v7/alpine`, but this can be supressed by
explicitly setting the platform.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
After dicussing with maintainers, it was decided putting the burden of
providing the full cap list on the client is not a good design.
Instead we decided to follow along with the container API and use cap
add/drop.
This brings in the changes already merged into swarmkit.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
In order to run tests at mips64el device.
Now official-images has supported the following images for mips64el.
buildpack-deps:stretch
buildpack-deps:buster
debian:stretch
debian:buster
But official-images does not support the following images for mips64el.
debian:jessie
buildpack-deps:jessie
Signed-off-by: wanghuaiqing <wanghuaiqing@loongson.cn>
If the container specified in `--volumes-from` did not exist, the
API returned a 404 status, which was interpreted by the CLI as the
specified _image_ to be missing (even if that was not the case).
This patch changes these error to return a 400 (bad request);
Before this change:
# make sure the image is present
docker pull busybox
docker create --volumes-from=nosuchcontainer busybox
# Unable to find image 'busybox:latest' locally
# latest: Pulling from library/busybox
# Digest: sha256:95cf004f559831017cdf4628aaf1bb30133677be8702a8c5f2994629f637a209
# Status: Image is up to date for busybox:latest
# Error response from daemon: No such container: nosuchcontainer
After this change:
# make sure the image is present
docker pull busybox
docker create --volumes-from=nosuchcontainer busybox
# Error response from daemon: No such container: nosuchcontainer
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Usage: DOCKER_BUILD_ARGS="--build-arg CONTAINERD_COMMIT=master --build-arg RUNC_COMMIT=master" DOCKER_EXPERIMENTAL=1 TEST_SKIP_INTEGRATION_CLI=1 make test-integration
Depends on containerd master (v1.4) and runc master (v1.0.0-rc91).
Currently `TEST_SKIP_INTEGRATION_CLI=1` must be specified.
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
The initial implementation followed the Swarm API, where
PidsLimit is located in ContainerSpec. This is not the
desired place for this property, so moving the field to
TaskTemplate.Resources in our API.
A similar change should be made in the SwarmKit API (likely
keeping the old field for backward compatibility, because
it was merged some releases back)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The test case panics like that:
> build_test.go:381: assertion failed: 3 (int) != 1 (int)
> panic: runtime error: index out of range [2] with length 1 [recovered]
> panic: runtime error: index out of range [2] with length 1
The fix is trivial.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Support for PidsLimit was added to SwarmKit in docker/swarmkit/pull/2415,
but never exposed through the Docker remove API.
This patch exposes the feature in the repote API.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
When a container is left running after the daemon exits (e.g. the daemon
is SIGKILL'd or crashes), it should stop any running containers when the
daemon starts back up.
What actually happens is the daemon only sends the container's
configured stop signal and does not check if it has exited.
If the container does not actually exit then it is left running.
This fixes this unexpected behavior by calling the same function to shut
down the container that the daemon shutdown process does.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
This removes the use of the old distribution code in the plugin packages
and replaces it with containerd libraries for plugin pushes and pulls.
Additionally it uses a content store from containerd which seems like
it's compatible with the old "basicBlobStore" in the plugin package.
This is being used locally isntead of through the containerd client for
now.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
This enables image lookup when creating a container to fail when the
reference exists but it is for the wrong platform. This prevents trying
to run an image for the wrong platform, as can be the case with, for
example binfmt_misc+qemu.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Switch to moby/sys/mount and mountinfo. Keep the pkg/mount for potential
outside users.
This commit was generated by the following bash script:
```
set -e -u -o pipefail
for file in $(git grep -l 'docker/docker/pkg/mount"' | grep -v ^pkg/mount); do
sed -i -e 's#/docker/docker/pkg/mount"#/moby/sys/mount"#' \
-e 's#mount\.\(GetMounts\|Mounted\|Info\|[A-Za-z]*Filter\)#mountinfo.\1#g' \
$file
goimports -w $file
done
```
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Metrics collectors generally don't need the daemon to prime the stats
with something to compare since they already have something to compare
with.
Before this change, the API does 2 collection cycles (which takes
roughly 2s) in order to provide comparison for CPU usage over 1s. This
was primarily added so that `docker stats --no-stream` had something to
compare against.
Really the CLI should have just made a 2nd call and done the comparison
itself rather than forcing it on all API consumers.
That ship has long sailed, though.
With this change, clients can set an option to just pull a single stat,
which is *at least* a full second faster:
Old:
```
time curl --unix-socket
/go/src/github.com/docker/docker/bundles/test-integration-shell/docker.sock
http://./containers/test/stats?stream=false\&one-shot=false > /dev/null
2>&1
real0m1.864s
user0m0.005s
sys0m0.007s
time curl --unix-socket
/go/src/github.com/docker/docker/bundles/test-integration-shell/docker.sock
http://./containers/test/stats?stream=false\&one-shot=false > /dev/null
2>&1
real0m1.173s
user0m0.010s
sys0m0.006s
```
New:
```
time curl --unix-socket
/go/src/github.com/docker/docker/bundles/test-integration-shell/docker.sock
http://./containers/test/stats?stream=false\&one-shot=true > /dev/null
2>&1
real0m0.680s
user0m0.008s
sys0m0.004s
time curl --unix-socket
/go/src/github.com/docker/docker/bundles/test-integration-shell/docker.sock
http://./containers/test/stats?stream=false\&one-shot=true > /dev/null
2>&1
real0m0.156s
user0m0.007s
sys0m0.007s
```
This fixes issues with downstreams ability to use the stats API to
collect metrics.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Configuration over the API per container is intentionally left out for
the time being, but is supported to configure the default from the
daemon config.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit cbecf48bc352e680a5390a7ca9cff53098cd16d7)
Signed-off-by: Madhu Venugopal <madhu@docker.com>
This supplements any log driver which does not support reads with a
custom read implementation that uses a local file cache.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit d675e2bf2b75865915c7a4552e00802feeb0847f)
Signed-off-by: Madhu Venugopal <madhu@docker.com>
Dont assign the --bip value directly to the subnet
for the default bridge. Instead use the network value
from the ParseCIDR output
Addresses: https://github.com/moby/moby/issues/40392
Signed-off-by: Arko Dasgupta <arko.dasgupta@docker.com>
Go 1.14 adds quotes around the url in the error returned:
=== FAIL: arm64.integration.system TestLoginFailsWithBadCredentials (0.27s)
TestLoginFailsWithBadCredentials: login_test.go:27: assertion failed: expected error "Error response from daemon: Get https://registry-1.docker.io/v2/: unauthorized: incorrect username or password", got "Error response from daemon: Get \"https://registry-1.docker.io/v2/\": unauthorized: incorrect username or password"
Error response from daemon: Get "https://registry-1.docker.io/v2/": unauthorized: incorrect username or password
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This makes sure that things like `--tmpfs` mounts over an anonymous
volume don't create volumes uneccessarily.
One method only checks mountpoints, the other checks both mountpoints
and tmpfs... the usage of these should likely be consolidated.
Ideally, processing for `--tmpfs` mounts would get merged in with the
rest of the mount parsing. I opted not to do that for this change so the
fix is minimal and can potentially be backported with fewer changes of
breaking things.
Merging the mount processing for tmpfs can be handled in a followup.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Docker Desktop (on MAC and Windows hosts) allows containers
running inside a Linux VM to connect to the host using
the host.docker.internal DNS name, which is implemented by
VPNkit (DNS proxy on the host)
This PR allows containers to connect to Linux hosts
by appending a special string "host-gateway" to --add-host
e.g. "--add-host=host.docker.internal:host-gateway" which adds
host.docker.internal DNS entry in /etc/hosts and maps it to host-gateway-ip
This PR also add a daemon flag call host-gateway-ip which defaults to
the default bridge IP
Docker Desktop will need to set this field to the Host Proxy IP
so DNS requests for host.docker.internal can be routed to VPNkit
Addresses: https://github.com/docker/for-linux/issues/264
Signed-off-by: Arko Dasgupta <arko.dasgupta@docker.com>
Adds support for ReplicatedJob and GlobalJob service modes. These modes
allow running service which execute tasks that exit upon success,
instead of daemon-type tasks.
Signed-off-by: Drew Erny <drew.erny@docker.com>
Bumps the vendoring of github.com/docker/swarmkit to the above commit,
which is the current master at commit time.
Most notably, this includes a change making the ingress network respect
the default address pool. Because of this change, a change to network
integration tests was needed.
Signed-off-by: Drew Erny <drew.erny@docker.com>
