When a container is left running after the daemon exits (e.g. the daemon
is SIGKILL'd or crashes), it should stop any running containers when the
daemon starts back up.
What actually happens is the daemon only sends the container's
configured stop signal and does not check if it has exited.
If the container does not actually exit then it is left running.
This fixes this unexpected behavior by calling the same function to shut
down the container that the daemon shutdown process does.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
This removes the use of the old distribution code in the plugin packages
and replaces it with containerd libraries for plugin pushes and pulls.
Additionally it uses a content store from containerd which seems like
it's compatible with the old "basicBlobStore" in the plugin package.
This is being used locally isntead of through the containerd client for
now.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
This enables image lookup when creating a container to fail when the
reference exists but it is for the wrong platform. This prevents trying
to run an image for the wrong platform, as can be the case with, for
example binfmt_misc+qemu.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Switch to moby/sys/mount and mountinfo. Keep the pkg/mount for potential
outside users.
This commit was generated by the following bash script:
```
set -e -u -o pipefail
for file in $(git grep -l 'docker/docker/pkg/mount"' | grep -v ^pkg/mount); do
sed -i -e 's#/docker/docker/pkg/mount"#/moby/sys/mount"#' \
-e 's#mount\.\(GetMounts\|Mounted\|Info\|[A-Za-z]*Filter\)#mountinfo.\1#g' \
$file
goimports -w $file
done
```
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Metrics collectors generally don't need the daemon to prime the stats
with something to compare since they already have something to compare
with.
Before this change, the API does 2 collection cycles (which takes
roughly 2s) in order to provide comparison for CPU usage over 1s. This
was primarily added so that `docker stats --no-stream` had something to
compare against.
Really the CLI should have just made a 2nd call and done the comparison
itself rather than forcing it on all API consumers.
That ship has long sailed, though.
With this change, clients can set an option to just pull a single stat,
which is *at least* a full second faster:
Old:
```
time curl --unix-socket
/go/src/github.com/docker/docker/bundles/test-integration-shell/docker.sock
http://./containers/test/stats?stream=false\&one-shot=false > /dev/null
2>&1
real0m1.864s
user0m0.005s
sys0m0.007s
time curl --unix-socket
/go/src/github.com/docker/docker/bundles/test-integration-shell/docker.sock
http://./containers/test/stats?stream=false\&one-shot=false > /dev/null
2>&1
real0m1.173s
user0m0.010s
sys0m0.006s
```
New:
```
time curl --unix-socket
/go/src/github.com/docker/docker/bundles/test-integration-shell/docker.sock
http://./containers/test/stats?stream=false\&one-shot=true > /dev/null
2>&1
real0m0.680s
user0m0.008s
sys0m0.004s
time curl --unix-socket
/go/src/github.com/docker/docker/bundles/test-integration-shell/docker.sock
http://./containers/test/stats?stream=false\&one-shot=true > /dev/null
2>&1
real0m0.156s
user0m0.007s
sys0m0.007s
```
This fixes issues with downstreams ability to use the stats API to
collect metrics.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Configuration over the API per container is intentionally left out for
the time being, but is supported to configure the default from the
daemon config.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit cbecf48bc352e680a5390a7ca9cff53098cd16d7)
Signed-off-by: Madhu Venugopal <madhu@docker.com>
This supplements any log driver which does not support reads with a
custom read implementation that uses a local file cache.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit d675e2bf2b75865915c7a4552e00802feeb0847f)
Signed-off-by: Madhu Venugopal <madhu@docker.com>
Dont assign the --bip value directly to the subnet
for the default bridge. Instead use the network value
from the ParseCIDR output
Addresses: https://github.com/moby/moby/issues/40392
Signed-off-by: Arko Dasgupta <arko.dasgupta@docker.com>
Go 1.14 adds quotes around the url in the error returned:
=== FAIL: arm64.integration.system TestLoginFailsWithBadCredentials (0.27s)
TestLoginFailsWithBadCredentials: login_test.go:27: assertion failed: expected error "Error response from daemon: Get https://registry-1.docker.io/v2/: unauthorized: incorrect username or password", got "Error response from daemon: Get \"https://registry-1.docker.io/v2/\": unauthorized: incorrect username or password"
Error response from daemon: Get "https://registry-1.docker.io/v2/": unauthorized: incorrect username or password
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This makes sure that things like `--tmpfs` mounts over an anonymous
volume don't create volumes uneccessarily.
One method only checks mountpoints, the other checks both mountpoints
and tmpfs... the usage of these should likely be consolidated.
Ideally, processing for `--tmpfs` mounts would get merged in with the
rest of the mount parsing. I opted not to do that for this change so the
fix is minimal and can potentially be backported with fewer changes of
breaking things.
Merging the mount processing for tmpfs can be handled in a followup.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Docker Desktop (on MAC and Windows hosts) allows containers
running inside a Linux VM to connect to the host using
the host.docker.internal DNS name, which is implemented by
VPNkit (DNS proxy on the host)
This PR allows containers to connect to Linux hosts
by appending a special string "host-gateway" to --add-host
e.g. "--add-host=host.docker.internal:host-gateway" which adds
host.docker.internal DNS entry in /etc/hosts and maps it to host-gateway-ip
This PR also add a daemon flag call host-gateway-ip which defaults to
the default bridge IP
Docker Desktop will need to set this field to the Host Proxy IP
so DNS requests for host.docker.internal can be routed to VPNkit
Addresses: https://github.com/docker/for-linux/issues/264
Signed-off-by: Arko Dasgupta <arko.dasgupta@docker.com>
Adds support for ReplicatedJob and GlobalJob service modes. These modes
allow running service which execute tasks that exit upon success,
instead of daemon-type tasks.
Signed-off-by: Drew Erny <drew.erny@docker.com>
Bumps the vendoring of github.com/docker/swarmkit to the above commit,
which is the current master at commit time.
Most notably, this includes a change making the ingress network respect
the default address pool. Because of this change, a change to network
integration tests was needed.
Signed-off-by: Drew Erny <drew.erny@docker.com>
The tests starts a new daemon, but attempts to run it with overlay2,
and using a unix:// socket, which doesn't really work on Windows.
40155 tried to disable such tests but missed two of them.
They are being disabled with this change.
Signed-off-by: vikrambirsingh <vikrambir.singh@docker.com>
Premature check for OS type means that the test
will never even get to run on other OS types. This
will cause it to be always flagged as a failure on
such OS types.
Signed-off-by: vikrambirsingh <vikrambir.singh@docker.com>
- Updated TestInfoSecurityOptions to not rely on CLI output. Note that this
test should be migrated to the integration suite, but that suite does not yet
have checks for "Seccomp" and "AppArmor"
- TestInfoAPIWarnings: don't start with busybox because we're not running containers in this test
- Migrate TestInfoDebug to integration suite
- Migrate TestInsecureRegistries to integration suite (renamed to TestInfoInsecureRegistries)
- Migrate TestRegistryMirrors to integration suite (renamed to TestInfoRegistryMirrors)
- Migrate TestInfoDiscoveryBackend to integration suite
- Migrate TestInfoDiscoveryInvalidAdvertise to integration suite
- Migrate TestInfoDiscoveryAdvertiseInterfaceName to integration suite
- Remove TestInfoFormat, which is testing the CLI functionality, and there is an
existing test in docker/cli (TestFormatInfo) covering this
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Adds a new ServiceStatus field to the Service object, which includes the
running and desired task counts. This new field is gated behind a
"status" query parameter.
Signed-off-by: Drew Erny <drew.erny@docker.com>
This feature was used by docker build --stream and it was kept experimental.
Users of this endpoint should enable BuildKit anyway by setting Version to BuilderBuildKit.
Signed-off-by: Tibor Vass <tibor@docker.com>
```
integration/plugin/graphdriver/external_test.go:427:2: SA5001: should check returned error before deferring responseReader.Close() (staticcheck)
```
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
```
internal/test/environment/environment.go:37:23: `useing` is a misspelling of `using`(misspell)
integration/container/wait_test.go:49:9: `waitres` is a misspelling of `waiters`(misspell)
integration/container/wait_test.go:95:9: `waitres` is a misspelling of `waiters`(misspell)
integration-cli/docker_api_containers_test.go:1042:7: `waitres` is a misspelling of `waiters`(misspell)
```
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The linter marked an issue because testMacvlanMultiSubnet was not used.
Re-enabling the test showed that there was a typo in the assert, causing the
test to fail:
```
--- FAIL: TestDockerNetworkMacvlan/MultiSubnet (4.74s)
macvlan_test.go:243: assertion failed: 2001:db8:abc4::254 (c3.NetworkSettings.Networks["dualstackbridge"].IPv6Gateway string) != 2001:db8.abc4::254 (string)
```
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Trying to link to a non-existing container is not valid, and should return an
"invalid parameter" (400) error. Returning a "not found" error in this situation
would make the client report the container's image could not be found.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
1. Modify comments added in 5858a99267
Windows Volume GUID path format is: \\?\Volume{<GUID Value>}\<path>
Rewrote the example given in comments to conform to the format..
2. Remove two redundant asserts[assert.NilError]. They are redundant
because the last statement will not change the value of err.
Signed-off-by: Vikram bir Singh <vikrambir.singh@docker.com>
1.This commit replaces serviceRunningCount with
swarm.RunningTasksCount to accurately check if the
service is running with the accurate number of instances
or not. serviceRunningCount was only checking the ServiceList
and was not checking if the tasks were running or not
This adds a safe barrier to execute docker network inspect
commands for overlay networks which get created
asynchronously via Swarm
2. Make sure client connections are closed
3. Make sure every service and network name is unique
4. Make sure services and networks are cleaned up
Signed-off-by: Arko Dasgupta <arko.dasgupta@docker.com>
This fixes a regression introduced in 6d87f19142,
causing `COPY --from` to fail if the target directory does not exist:
```
FROM mcr.microsoft.com/windows/servercore:ltsc2019 as s1
RUN echo "Hello World" > /hello
FROM mcr.microsoft.com/windows/servercore:ltsc2019
COPY --from=s1 /hello /hello/another/world
```
Would produce an error:
```
Step 4/4 : COPY --from=s1 /hello /hello/another/world
failed to copy files: mkdir \\?: The filename, directory name, or volume label syntax is incorrect.
```
The cause for this was that Go's `os.MkdirAll()` does not support/detect volume GUID paths
(`\\?\Volume{dae8d3ac-b9a1-11e9-88eb-e8554b2ba1db}\hello\another}`), and as a result
attempted to create the volume as a directory (`\\?`), causing it to fail.
This patch replaces `os.MkdirAll()` with our own `system.MkdirAll()` function, which
is capable of detecting GUID volumes.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
1. Use `go list` to get list of integration dirs to build. This means we
do not need to have a valid `.go` in every subdirectory and also
filters out other dirs like "bundles" which may have been created.
2. Add option to specify custom flags for integration and
integration-cli. This is needed so both suites can be run AND set
custom flags... since the cli suite does not support standard go
flags.
3. Add options to skip an entire integration suite.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
This test is failing on Windows currently:
```
11:59:47 --- FAIL: TestHealthKillContainer (8.12s)
11:59:47 health_test.go:57: assertion failed: error is not nil: Error response from daemon: Invalid signal: SIGUSR1
``
That test was added recently in https://github.com/moby/moby/pull/39454, but
rewritten in a commit in the same PR:
f8aef6a92f
In that rewrite, there were some changes:
- originally it was skipped on Windows, but the rewritten test doesn't have that skip:
```go
testRequires(c, DaemonIsLinux) // busybox doesn't work on Windows
```
- the original test used `SIGINT`, but the new one uses `SIGUSR1`
Analysis:
- The Error bubbles up from: 8e610b2b55/pkg/signal/signal.go (L29-L44)
- Interestingly; `ContainerKill` should validate if a signal is valid for the given platform, but somehow we don't hit that part; f1b5612f20/daemon/kill.go (L40-L48)
- Windows only looks to support 2 signals currently 8e610b2b55/pkg/signal/signal_windows.go (L17-L26)
- Upstream Golang looks to define `SIGINT` as well; 77f9b2728e/src/runtime/defs_windows.go (L44)
- This looks like the current list of Signals upstream in Go; 3b58ed4ad3/windows/types_windows.go (L52-L67)
```go
const (
// More invented values for signals
SIGHUP = Signal(0x1)
SIGINT = Signal(0x2)
SIGQUIT = Signal(0x3)
SIGILL = Signal(0x4)
SIGTRAP = Signal(0x5)
SIGABRT = Signal(0x6)
SIGBUS = Signal(0x7)
SIGFPE = Signal(0x8)
SIGKILL = Signal(0x9)
SIGSEGV = Signal(0xb)
SIGPIPE = Signal(0xd)
SIGALRM = Signal(0xe)
SIGTERM = Signal(0xf)
)
```
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The session endpoint is no longer experimental since
01c9e7082e, so we don't
need to start an experimental daemon.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Looks like TestServiceWithDefaultAddressPoolInit is failing
randomly in CI. I am not able to reproduce the issue locally
but this has been reported few times. So I tried to modify
code and see if I can fix the random failure.
Signed-off-by: selansen <elango.siva@docker.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The daemon requires kernel 3.10 or up to start, so there's no need
to check if the daemon is kernel 3.8 or up.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Ref: #39426
This is a common flaky test that I have seen on multiple PRs. It is not
consistent and should be skipped until it is fixed to be robust. A
simple fix for the swarm tests is not easy as they all poll and have 1
billion timeouts in all the tests so a skip is valid here.
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
