Commit graph

93 commits

Author SHA1 Message Date
Sebastiaan van Stijn
4f057d8bb6
Merge pull request #43887 from thaJeztah/22.06_backport_implicit_runtime_config
[22.06 backport] daemon: support other containerd runtimes (MVP)
2022-08-03 23:55:51 +02:00
Cory Snider
6de52a29a8
daemon: support other containerd runtimes (MVP)
Contrary to popular belief, the OCI Runtime specification does not
specify the command-line API for runtimes. Looking at containerd's
architecture from the lens of the OCI Runtime spec, the _shim_ is the
OCI Runtime and runC is "just" an implementation detail of the
io.containerd.runc.v2 runtime. When one configures a non-default runtime
in Docker, what they're really doing is instructing Docker to create
containers using the io.containerd.runc.v2 runtime with a configuration
option telling the runtime that the runC binary is at some non-default
path. Consequently, only OCI runtimes which are compatible with the
io.containerd.runc.v2 shim, such as crun, can be used in this manner.
Other OCI runtimes, including kata-containers v2, come with their own
containerd shim and are not compatible with io.containerd.runc.v2.
As Docker has not historically provided a way to select a non-default
runtime which requires its own shim, runtimes such as kata-containers v2
could not be used with Docker.

Allow other containerd shims to be used with Docker; no daemon
configuration required. If the daemon is instructed to create a
container with a runtime name which does not match any of the configured
or stock runtimes, it passes the name along to containerd verbatim. A
user can start a container with the kata-containers runtime, for
example, simply by calling

    docker run --runtime io.containerd.kata.v2

Runtime names which containerd would interpret as a path to an arbitrary
binary are disallowed. While handy for development and testing it is not
strictly necessary and would allow anyone with Engine API access to
trivially execute any binary on the host as root, so we have decided it
would be safest for our users if it was not allowed.

It is not yet possible to set an alternative containerd shim as the
default runtime; it can only be configured per-container.

Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit 547da0d575)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-29 20:36:50 +02:00
Paweł Gronowski
44fde1bdb7 integration: Add TestContainerWithAutoRemoveCanBeRestarted
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2022-07-29 16:49:56 +02:00
Sebastiaan van Stijn
cdbca4061b
gofmt GoDoc comments with go1.19
Older versions of Go don't format comments, so committing this as
a separate commit, so that we can already make these changes before
we upgrade to Go 1.19.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 52c1a2fae8)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-13 22:42:29 +02:00
Sebastiaan van Stijn
1cab8eda24
replace golint with revive, as it's deprecated
WARN [runner] The linter 'golint' is deprecated (since v1.41.0) due to: The repository of the linter has been archived by the owner.  Replaced by revive.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-04 10:15:54 +02:00
Paweł Gronowski
56a20dbc19 container/exec: Support ConsoleSize
Now client have the possibility to set the console size of the executed
process immediately at the creation. This makes a difference for example
when executing commands that output some kind of text user interface
which is bounded by the console dimensions.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2022-06-24 11:54:25 +02:00
Paweł Gronowski
2ec3e14c0f test: Add tests for logging
1. Add integration tests for the ContainerLogs API call
Each test handle a distinct case of ContainerLogs output.
- Muxed stream, when container is started without tty
- Single stream, when container is started with tty

2. Add unit test for LogReader suite that tests concurrent logging
It checks that there are no race conditions when logging concurrently
from multiple goroutines.

Co-authored-by: Cory Snider <csnider@mirantis.com>
Signed-off-by: Cory Snider <csnider@mirantis.com>
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2022-06-10 09:26:17 +02:00
Paweł Gronowski
85a7f5a09a daemon/linux: Set console size on creation
On Linux the daemon was not respecting the HostConfig.ConsoleSize
property and relied on cli initializing the tty size after the container
was created. This caused a delay between container creation and
the tty actually being resized.

This is also a small change to the api description, because
HostConfig.ConsoleSize is no longer Windows-only.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2022-05-19 07:57:27 +02:00
Sebastiaan van Stijn
41b96bff55
update uses of container.ContainerCreateCreatedBody to CreateResponse
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-28 22:39:20 +02:00
Paul "TBBle" Hampson
c60f70f112 Break out setupWindowsDevices and add tests
Since this function is about to get more complicated, and change
behaviour, this establishes tests for the existing implementation.

Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
2022-03-27 13:23:48 +11:00
Sebastiaan van Stijn
686be57d0a
Update to Go 1.17.0, and gofmt with Go 1.17
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-08-24 23:33:27 +02:00
Sebastiaan van Stijn
bb17074119
reformat "nolint" comments
Unlike regular comments, nolint comments should not have a leading space.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-06-10 13:03:42 +02:00
Akihiro Suda
7994443c15
integration: port TestRunModePIDHost from CLI test to API test
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-02-01 14:54:07 +09:00
Brian Goff
24f173a003 Replace service "Capabilities" w/ add/drop API
After dicussing with maintainers, it was decided putting the burden of
providing the full cap list on the client is not a good design.
Instead we decided to follow along with the container API and use cap
add/drop.

This brings in the changes already merged into swarmkit.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2020-07-27 10:09:42 -07:00
Sebastiaan van Stijn
687bdc7c71
API: swarm: move PidsLimit to TaskTemplate.Resources
The initial implementation followed the Swarm API, where
PidsLimit is located in ContainerSpec. This is not the
desired place for this property, so moving the field to
TaskTemplate.Resources in our API.

A similar change should be made in the SwarmKit API (likely
keeping the old field for backward compatibility, because
it was merged some releases back)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-06-05 12:50:38 +02:00
Tibor Vass
5c10ea6ae8
Merge pull request #40725 from cpuguy83/check_img_platform
Accept platform spec on container create
2020-05-21 11:33:27 -07:00
Sebastiaan van Stijn
157c53c8e0
Add API support for PidsLimit on services
Support for PidsLimit was added to SwarmKit in docker/swarmkit/pull/2415,
but never exposed through the Docker remove API.

This patch exposes the feature in the repote API.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-04-15 22:37:42 +02:00
Brian Goff
7a9cb29fb9 Accept platform spec on container create
This enables image lookup when creating a container to fail when the
reference exists but it is for the wrong platform. This prevents trying
to run an image for the wrong platform, as can be the case with, for
example binfmt_misc+qemu.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2020-03-20 16:10:36 -07:00
Akihiro Suda
5e1b246b9a test-integration: support more rootless tests
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-03-16 17:37:38 +09:00
Sebastiaan van Stijn
9f0b3f5609
bump gotest.tools v3.0.1 for compatibility with Go 1.14
full diff: https://github.com/gotestyourself/gotest.tools/compare/v2.3.0...v3.0.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-02-11 00:06:42 +01:00
Brian Goff
f464c31668 Check tmpfs mounts before create anon volume
This makes sure that things like `--tmpfs` mounts over an anonymous
volume don't create volumes uneccessarily.
One method only checks mountpoints, the other checks both mountpoints
and tmpfs... the usage of these should likely be consolidated.

Ideally, processing for `--tmpfs` mounts would get merged in with the
rest of the mount parsing. I opted not to do that for this change so the
fix is minimal and can potentially be backported with fewer changes of
breaking things.
Merging the mount processing for tmpfs can be handled in a followup.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2020-02-04 10:12:05 -08:00
Sebastiaan van Stijn
ca20bc4214
Merge pull request #40007 from arkodg/add-host-docker-internal
Support host.docker.internal in dockerd on Linux
2020-01-27 13:42:26 +01:00
Arko Dasgupta
92e809a680 Support host.docker.internal in dockerd on Linux
Docker Desktop (on MAC and Windows hosts) allows containers
running inside a Linux VM to connect to the host using
the host.docker.internal DNS name, which is implemented by
VPNkit (DNS proxy on the host)

This PR allows containers to connect to Linux hosts
by appending a special string "host-gateway" to --add-host
e.g. "--add-host=host.docker.internal:host-gateway" which adds
host.docker.internal DNS entry in /etc/hosts and maps it to host-gateway-ip

This PR also add a daemon flag call host-gateway-ip which defaults to
the default bridge IP
Docker Desktop will need to set this field to the Host Proxy IP
so DNS requests for host.docker.internal can be routed to VPNkit

Addresses: https://github.com/docker/for-linux/issues/264

Signed-off-by: Arko Dasgupta <arko.dasgupta@docker.com>
2020-01-22 13:30:00 -08:00
Drew Erny
30d9fe30b1 Add swarm jobs
Adds support for ReplicatedJob and GlobalJob service modes. These modes
allow running service which execute tasks that exit upon success,
instead of daemon-type tasks.

Signed-off-by: Drew Erny <drew.erny@docker.com>
2020-01-13 13:21:12 -06:00
Ziheng Liu
6233217a31 integration/internal/container: fix a goroutine leak bug by adding 1 buffer
Signed-off-by: Ziheng Liu <lzhfromustc@gmail.com>
2019-10-27 20:22:52 -04:00
Sebastiaan van Stijn
554d9cec25
testutil: update WithExperimental signature to be a daemon.Option
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-30 17:38:24 +02:00
Sam Whited
41adef29f5 testutil/daemon: group options under type
Signed-off-by: Sam Whited <sam@samwhited.com>
2019-09-18 09:14:50 -05:00
Sam Whited
b37c214e3c testutil: make testing packages public
This was done with something along the lines of:

```
mv internal/test testutil
pushd testutil/; grep -IRl "package test" | xargs -I '{}' sed -i -e 's|package test|package testutil|g' {}; popd
mv internal/testutil/*.go testutil/ && rm -rf internal/
grep -IRl "github.com\/docker\/docker\/internal\/test" | xargs -I '{}' sed -i -e 's|github.com/docker/docker/internal/test|github.com/docker/docker/test|g' {}
goimports .
```

I also modified the basic plugin path in testutil/fixtures/plugin.

Signed-off-by: Sam Whited <sam@samwhited.com>
2019-09-11 07:47:23 -05:00
Brian Goff
02dceeaa7b
Merge pull request #39240 from olljanat/enable-new-integration-tests-for-win
Enable integrations API tests for Windows CI
2019-06-11 14:24:03 -07:00
Olli Janatuinen
2f22247cad Enable integrations API tests for Windows CI
Signed-off-by: Olli Janatuinen <olli.janatuinen@gmail.com>
2019-06-10 21:32:36 +03:00
Sebastiaan van Stijn
9f9b4290b9
integration: change container.Run signature to fix linting
Line 59: warning: context.Context should be the first parameter of a function (golint)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-06-07 13:05:26 +02:00
Sebastiaan van Stijn
b4c46b0dac
integration: change container.Create signature to fix linting
```
Line 25: warning: context.Context should be the first parameter of a function (golint)
Line 44: warning: context.Context should be the first parameter of a function (golint)
Line 52: warning: context.Context should be the first parameter of a function (golint)
```

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-06-07 13:04:44 +02:00
Sebastiaan van Stijn
caec45a37f
integration: change network.CreateNoError signature to fix linting
Line 30: warning: context.Context should be the first parameter of a function (golint)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-06-07 13:03:24 +02:00
Olli Janatuinen
f787b235de Add support capabilities list on services
Signed-off-by: Olli Janatuinen <olli.janatuinen@gmail.com>
2019-05-28 19:52:36 +03:00
Rob Gulewich
072400fc4b Make cgroup namespaces configurable
This adds both a daemon-wide flag and a container creation property:
- Set the `CgroupnsMode: "host|private"` HostConfig property at
  container creation time to control what cgroup namespace the container
  is created in
- Set the `--default-cgroupns-mode=host|private` daemon flag to control
  what cgroup namespace containers are created in by default
- Set the default if the daemon flag is unset to "host", for backward
  compatibility
- Default to CgroupnsMode: "host" for client versions < 1.40

Signed-off-by: Rob Gulewich <rgulewich@netflix.com>
2019-05-07 10:22:16 -07:00
Yong Tang
5635c248cd
Merge pull request #38871 from crosbymichael/exec-spec
Use original process spec for execs
2019-03-25 11:58:33 +08:00
Michael Crosby
7603c22c73 Use original process spec for execs
Fixes #38865

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-03-21 15:41:53 -04:00
Sebastiaan van Stijn
1101568fa1
Update TestUpdatePidsLimit to be more atomic
Create a new container for each subtest, so that individual
subtests are self-contained, and there's no need to execute
them in the exact order, or resetting the container in between.

This makes the test slower (6.54s vs  3.43s), but reduced the
difference by using `network=host`, which made a substantial
difference (without `network=host`, the test took more than
twice as long: 13.96s).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-03-13 00:27:15 +01:00
Kir Kolyshkin
f664df01d1 integration: add/use WithRestartPolicy
NOTE TestUpdateRestartPolicy is left as is as otherwise
it will decrease its readability.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2019-03-08 10:03:55 -08:00
Kir Kolyshkin
17022b3ad2 integration/internal/container/ops: rm unused code
Since container.Create() already initializes HostConfig
to be non-nil, there is no need for this code. Remove it.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2019-03-08 10:00:14 -08:00
Sebastiaan van Stijn
67d6f174ae
Add more details to RunningTasksCount
Provide slightly better information about the current state
of tasks.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-02-27 17:08:44 +01:00
Yong Tang
e485a60e2b Move serviceRunningTasksCount to integration/internal/swarm
This fix moves multiple places of serviceRunningTasksCount
to one location in integration/internal/swarm, so that
code duplication could be removed.

Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
2019-01-19 18:54:32 +00:00
Vincent Demeester
38015177d8
Merge pull request #38557 from thaJeztah/remove_duplicated_code
Integration tests: remove some duplicated code, and preserve context
2019-01-15 09:03:45 +01:00
Sebastiaan van Stijn
56a68c15f8
Integration tests: remove some duplicated code, and preserve context
This introduces `NoTasksForService` and `NoTasks` poller checks, that
can be used to check if no tasks are left in general, or for a specific
service.

Some redundant checks were also removed from some tests.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-01-14 04:53:27 +01:00
Sebastiaan van Stijn
8172edf04c
GetRunningTasks: pass client instead of daemon
Tests generally already have a client instance, so it
probably makes more sense to just pass it, and make
this utility a bit more flexible to use.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-01-14 02:30:40 +01:00
Yong Tang
28b7824caa Remove code duplication and consolidate networkIsRemoved
This fix removes code duplication and consolidates networkIsRemoved
into one place.

Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
2019-01-12 23:01:21 +00:00
Olli Janatuinen
153171e9dd Added support for maximum replicas per node to services
Signed-off-by: Olli Janatuinen <olli.janatuinen@gmail.com>
2018-12-24 02:04:15 +02:00
Sebastiaan van Stijn
b0de11cf30
Add test for status code on conflicting service names
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-11-24 11:53:08 +01:00
Akihiro Suda
596cdffb9f mount: add BindOptions.NonRecursive (API v1.40)
This allows non-recursive bind-mount, i.e. mount(2) with "bind" rather than "rbind".

Swarm-mode will be supported in a separate PR because of mutual vendoring.

Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-11-06 17:51:58 +09:00
Salahuddin Khan
4c8b1fd5a2 Enabling Windows integration tests
Signed-off-by: Salahuddin Khan <salah@docker.com>
2018-09-26 15:57:40 -07:00