Commit graph

44358 commits

Author SHA1 Message Date
Sebastiaan van Stijn
3dae8e9fc2
api: swagger: add missing "platform" query-arg on create
Commit 7a9cb29fb9 added a new "platform" query-
parameter to the `POST /containers/create` endpoint, but did not update the
swagger file and documentation.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-29 23:19:51 +02:00
Brian Goff
6e21389917
Merge pull request #43800 from corhere/implicit-runtime-config
daemon: support alternative runtimes MVP
2022-07-29 09:35:22 -07:00
Sebastiaan van Stijn
f34567bf41
Merge pull request #43751 from vvoland/fix-exitcode-wait
state/Wait: Fix race when reading exit status
2022-07-29 16:01:39 +02:00
Sebastiaan van Stijn
71cb54cec4
Merge pull request #43872 from thaJeztah/containerd_config_v2
libcontainerd: switch generated containerd.toml to v2 (v1 is deprecated)
2022-07-28 18:21:18 +02:00
Brian Goff
c5d2c8fb80
Merge pull request #43877 from Snorch/integration-cli-make-service-process-live-forever
integration-cli: Make service process live forever
2022-07-28 07:47:33 -07:00
Pavel Tikhomirov
f4c0ec8ffc integration-cli: Make service process live forever
- TestServiceLogsCompleteness runs service with command to write 6 log
lines but as command exits immediately, service is restarted and 6 more
lines are printed in logs, which confuses the checker.Equals(6) check.

- TestServiceLogsSince runs service with command to write 3 log lines,
and service restart can also affect it's checks.

Let's change from `tail` which exits immediately to `tail -f` which
hangs forever, this way we would not confuse checks with more log lines
when expected.

Signed-off-by: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>
2022-07-28 16:27:27 +03:00
Brian Goff
9772c68e54
Merge pull request #42934 from abdulrahimiliasu/42655-vfs-storage-driver
Fix file capabilites droping in Dockerfile
2022-07-27 14:31:39 -07:00
Cory Snider
547da0d575 daemon: support other containerd runtimes (MVP)
Contrary to popular belief, the OCI Runtime specification does not
specify the command-line API for runtimes. Looking at containerd's
architecture from the lens of the OCI Runtime spec, the _shim_ is the
OCI Runtime and runC is "just" an implementation detail of the
io.containerd.runc.v2 runtime. When one configures a non-default runtime
in Docker, what they're really doing is instructing Docker to create
containers using the io.containerd.runc.v2 runtime with a configuration
option telling the runtime that the runC binary is at some non-default
path. Consequently, only OCI runtimes which are compatible with the
io.containerd.runc.v2 shim, such as crun, can be used in this manner.
Other OCI runtimes, including kata-containers v2, come with their own
containerd shim and are not compatible with io.containerd.runc.v2.
As Docker has not historically provided a way to select a non-default
runtime which requires its own shim, runtimes such as kata-containers v2
could not be used with Docker.

Allow other containerd shims to be used with Docker; no daemon
configuration required. If the daemon is instructed to create a
container with a runtime name which does not match any of the configured
or stock runtimes, it passes the name along to containerd verbatim. A
user can start a container with the kata-containers runtime, for
example, simply by calling

    docker run --runtime io.containerd.kata.v2

Runtime names which containerd would interpret as a path to an arbitrary
binary are disallowed. While handy for development and testing it is not
strictly necessary and would allow anyone with Engine API access to
trivially execute any binary on the host as root, so we have decided it
would be safest for our users if it was not allowed.

It is not yet possible to set an alternative containerd shim as the
default runtime; it can only be configured per-container.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-07-27 14:22:49 -04:00
Illo Abdulrahim
31f654a704
Fix file capabilities droping in Dockerfile
doCopyXattrs() never reached due to copyXattrs boolean being false, as
a result file capabilities not being copied.

moved copyXattr() out of doCopyXattrs()

Signed-off-by: Illo Abdulrahim <abdulrahim.illo@nokia.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-27 17:21:02 +02:00
Sebastiaan van Stijn
ba2ff69894
libcontainerd: switch generated containerd.toml to v2 (v1 is deprecated)
Before this patch:

    INFO[2022-07-27T14:30:06.188762628Z] Starting up
    INFO[2022-07-27T14:30:06.190750725Z] libcontainerd: started new containerd process  pid=2028
    ...
    WARN[0000] containerd config version `1` has been deprecated and will be removed in containerd v2.0, please switch to version `2`, see https://github.com/containerd/containerd/blob/main/docs/PLUGINS.md#version-header
    INFO[2022-07-27T14:30:06.220024286Z] starting containerd                           revision=10c12954828e7c7c9b6e0ea9b0c02b01407d3ae1 version=v1.6.6

With this patch:

    INFO[2022-07-27T14:28:04.025543517Z] Starting up
    INFO[2022-07-27T14:28:04.027447105Z] libcontainerd: started new containerd process  pid=1377
    ...
    INFO[2022-07-27T14:28:04.054483270Z] starting containerd                           revision=10c12954828e7c7c9b6e0ea9b0c02b01407d3ae1 version=v1.6.6

And the generated /var/run/docker/containerd/containerd.toml:

```toml
disabled_plugins = ["io.containerd.grpc.v1.cri"]
imports = []
oom_score = 0
plugin_dir = ""
required_plugins = []
root = "/var/lib/docker/containerd/daemon"
state = "/var/run/docker/containerd/daemon"
temp = ""
version = 2

[cgroup]
  path = ""

[debug]
  address = "/var/run/docker/containerd/containerd-debug.sock"
  format = ""
  gid = 0
  level = "debug"
  uid = 0

[grpc]
  address = "/var/run/docker/containerd/containerd.sock"
  gid = 0
  max_recv_message_size = 16777216
  max_send_message_size = 16777216
  tcp_address = ""
  tcp_tls_ca = ""
  tcp_tls_cert = ""
  tcp_tls_key = ""
  uid = 0

[metrics]
  address = ""
  grpc_histogram = false

[plugins]

[proxy_plugins]

[stream_processors]

[timeouts]

[ttrpc]
  address = ""
  gid = 0
  uid = 0
```

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-27 16:33:00 +02:00
Sebastiaan van Stijn
2bfc7aedab
Merge pull request #43866 from olljanat/win-enable-attach-websocket
integration-cli: Enable TestGetContainersAttachWebsocket for Windows
2022-07-26 14:20:05 +02:00
Sebastiaan van Stijn
45f845a260
Merge pull request #43863 from vvoland/nit-windowscode
integration-cli: Remove unnecessary Windows-only code
2022-07-26 14:16:25 +02:00
Olli Janatuinen
a61f7abf5a integration-cli: Enable TestGetContainersAttachWebsocket for Windows
Signed-off-by: Olli Janatuinen <olli.janatuinen@gmail.com>
2022-07-25 08:17:53 -07:00
Paweł Gronowski
6cc644abef integration-cli: Remove unnecessary Windows-only code
This test is skipped on Windows anyway.
Also add a short explanation why emptyfs image was chosen.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2022-07-25 15:08:55 +02:00
Sebastiaan van Stijn
98d8343aa2
Merge pull request #43838 from thaJeztah/remove_journald_compat
logger/journald: remove journald_compat (for systemd < 209)
2022-07-25 11:25:52 +02:00
Sebastiaan van Stijn
49f021ebf0
Merge pull request #42829 from zq-david-wang/testfix_TestNetworkDBNodeJoinLeaveIteration
Test: wait for network changes in TestNetworkDBNodeJoinLeaveIteration
2022-07-22 06:55:12 +02:00
Sebastiaan van Stijn
d515863abc
Merge pull request #43836 from olljanat/restore-custom-nat-networks
Windows: Re-create custom NAT networks after restart if missing from HNS
2022-07-21 22:37:27 +02:00
Sebastiaan van Stijn
e3a18e1990
Merge pull request #43816 from thaJeztah/image_filters
containerd integration: add support for since,before,labels images filters
2022-07-21 21:45:35 +02:00
David Wang
f499c6b9ec
Test: wait for network changes in TestNetworkDBNodeJoinLeaveIteration
In network node change test, the expected behavior is focused on how many nodes
left in networkDB, besides timing issues, things would also go tricky for a
leave-then-join sequence, if the check (counting the nodes) happened before the
first "leave" event, then the testcase actually miss its target and report PASS
without verifying its final result; if the check happened after the 'leave' event,
but before the 'join' event, the test would report FAIL unnecessary;

This code change would check both the db changes and the node count, it would
report PASS only when networkdb has indeed changed and the node count is expected.

Signed-off-by: David Wang <00107082@163.com>
2022-07-21 22:56:58 +08:00
Sebastiaan van Stijn
669fbed1ac
daemon/containerd: validate image filters
Not all filters are implemented yet, so make sure an error
is returned if a not-yet implemented filter is used.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-21 11:14:12 +02:00
Nicolas De Loof
4d3b32bf3d
add support for since, before, labels images filters
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-21 11:14:05 +02:00
Sebastiaan van Stijn
0e9a66d35a
logger/journald: remove journald_compat (for systemd < 209)
This was added in 6cdc4ba6cd in 2016, likely
because at the time we were still building for CentOS 6 and Ubuntu 14.04.

All currently supported distros appear to be on _at least_ 219 now, so it looks
safe to remove this;

```bash
docker run -it --rm centos:7

yum install -y systemd-devel

pkg-config 'libsystemd >= 209' && echo "OK" || echo "KO"
OK

pkg-config --print-provides 'libsystemd'
libsystemd = 219

pkg-config --print-provides 'libsystemd-journal'
libsystemd-journal = 219
```

And on a `debian:buster` (old stable)

```bash
docker run -it --rm debian:buster

apt-get update && apt-get install -y libsystemd-dev pkg-config

pkg-config 'libsystemd >= 209' && echo "OK" || echo "KO"
OK

pkg-config --print-provides 'libsystemd'
libsystemd = 241

pkg-config --print-provides 'libsystemd-journal'
Package libsystemd-journal was not found in the pkg-config search path.
Perhaps you should add the directory containing `libsystemd-journal.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libsystemd-journal' found
```

OpenSUSE leap (I think that's built for s390x)

```bash
docker run -it --rm docker.io/opensuse/leap:15

zypper install -y systemd-devel

pkg-config 'libsystemd >= 209' && echo "OK" || echo "KO"
OK

pkg-config --print-provides 'libsystemd'
libsystemd = 246

pkg-config --print-provides 'libsystemd-journal'
Package libsystemd-journal was not found in the pkg-config search path.
Perhaps you should add the directory containing `libsystemd-journal.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libsystemd-journal' found
```

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-20 18:48:00 +02:00
Sebastiaan van Stijn
7624f8aeb1
Merge pull request #43839 from thaJeztah/journald_fix_break
logger/journald: fix SA4011: ineffective break statement
2022-07-20 18:46:11 +02:00
Sebastiaan van Stijn
75577fe7a8
logger/journald: fix SA4011: ineffective break statement
This was introduced in 906b979b88, which changed
a `goto` to a `break`, but afaics, the intent was still to break out of the loop.
(linter didn't catch this before because it didn't have the right build-tag set)

    daemon/logger/journald/read.go:238:4: SA4011: ineffective break statement. Did you mean to break out of the outer loop? (staticcheck)
                break // won't be able to write anything anymore
                ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-20 15:14:41 +02:00
Paweł Gronowski
a290f5d04c state/Wait: Fix race when reading exit status
Before this change there was a race condition between State.Wait reading
the exit code from State and the State being changed instantly after the
change which ended the State.Wait.

Now, each State.Wait has its own channel which is used to transmit the
desired StateStatus at the time the state transitions to the awaited
one. Wait no longer reads the status by itself so there is no race.

The issue caused the `docker run --restart=always ...' to sometimes exit
with 0 exit code, because the process was already restarted by the time
State.Wait got the chance to read the exit code.

Test run
--------
Before:
```
$ go test -count 1 -run TestCorrectStateWaitResultAfterRestart .
--- FAIL: TestCorrectStateWaitResultAfterRestart (0.00s)
    state_test.go:198: expected exit code 10, got 0
FAIL
FAIL    github.com/docker/docker/container      0.011s
FAIL

```

After:
```
$ go test -count 1 -run TestCorrectStateWaitResultAfterRestart .
ok      github.com/docker/docker/container      0.011s
```

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2022-07-20 09:23:31 +02:00
Paweł Gronowski
5571d5139b integration: TestWaitRestartedContainer
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2022-07-20 09:23:31 +02:00
Paweł Gronowski
498803bec9 daemon/restart: Don't mutate AutoRemove when restarting
This caused a race condition where AutoRemove could be restored before
container was considered for restart and made autoremove containers
impossible to restart.

```
$ make DOCKER_GRAPHDRIVER=vfs BIND_DIR=. TEST_FILTER='TestContainerWithAutoRemoveCanBeRestarted' TESTFLAGS='-test.count 1' test-integration
...
=== RUN   TestContainerWithAutoRemoveCanBeRestarted
=== RUN   TestContainerWithAutoRemoveCanBeRestarted/kill
=== RUN   TestContainerWithAutoRemoveCanBeRestarted/stop
--- PASS: TestContainerWithAutoRemoveCanBeRestarted (1.61s)
    --- PASS: TestContainerWithAutoRemoveCanBeRestarted/kill (0.70s)
    --- PASS: TestContainerWithAutoRemoveCanBeRestarted/stop (0.86s)
PASS

DONE 3 tests in 3.062s
```

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2022-07-20 09:23:31 +02:00
Paweł Gronowski
c492105b8f integration: Add TestContainerWithAutoRemoveCanBeRestarted
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2022-07-20 09:23:28 +02:00
Olli Janatuinen
67c36d5d6e Windows: Re-create custom NAT networks after restart if missing from HNS
Signed-off-by: Olli Janatuinen <olli.janatuinen@gmail.com>
2022-07-19 14:16:31 -07:00
Brian Goff
2fbc30739b
Merge pull request #43815 from thaJeztah/compute_virtualsize
containerd integration: compute virtualsize
2022-07-18 15:49:59 -07:00
Nicolas De Loof
a46ce37d25
compute image VirtualSize
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-18 19:23:20 +02:00
Sebastiaan van Stijn
992961c56c
Merge pull request #43831 from thaJeztah/containerd_restructure
containerd-integration: restructure code to match existing implementation
2022-07-18 19:20:35 +02:00
Brian Goff
6acf0972ec
Merge pull request #43832 from thaJeztah/rename_colliding_vars
daemon/images: rename variables that collided with imports
2022-07-18 10:09:12 -07:00
Sebastiaan van Stijn
9c8b0b6050
daemon/containerd: split methods to separate files
This splits the ImageService methods to separate files, to closer
match the existing implementation, and to reduce the amount of code
per file, making it easier to read, and to reduce merge conflicts if
new functionality is added.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-18 14:46:31 +02:00
Sebastiaan van Stijn
e4cd1933c6
daemon/containerd: ImageService: rename receiver
This renames the receiver to match the existing implementation
in daemon/images.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-18 14:46:29 +02:00
Sebastiaan van Stijn
91bd9a6642
daemon/containerd: change alias for oci-specs to match existing code
We use "specs" as alias in most places; rename the alias here accordingly
to prevent confusiong and reduce the risk of introducing duplicate imports.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-18 14:46:27 +02:00
Sebastiaan van Stijn
b529fe0995
daemon/images: fix some linting issues
- fix GoDoc
- rename a variable that collided with an import

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-18 14:46:02 +02:00
Sebastiaan van Stijn
b94f7157ca
daemon/images: rename images.go to image_list.go for consistency
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-18 14:34:37 +02:00
Sebastiaan van Stijn
046b3fc5dd
daemon/images: move SquashImage to its own file
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-18 14:34:34 +02:00
Sebastiaan van Stijn
6b70975efc
Merge pull request #43829 from thaJeztah/image_service_deadcode
daemon: ImageService: remove unused Map(), LookupImage()
2022-07-18 14:25:50 +02:00
Sebastiaan van Stijn
6e0e5d7a31
daemon/images: rename variables that collided with imports
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-18 13:22:12 +02:00
Sebastiaan van Stijn
2527e6dd09
daemon/containerd: ImageService: remove unused LookupImage()
Looks like this method was not part of the interface, and is not
used anywhere.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-18 12:22:12 +02:00
Sebastiaan van Stijn
7b510fda0c
daemon: ImageService: remove unused Map() method
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-18 11:32:31 +02:00
Sebastiaan van Stijn
5de77049db
Merge pull request #43793 from AkihiroSuda/fix-43781
libnetwork: skip firewalld management for rootless
2022-07-15 13:44:11 +02:00
Sebastiaan van Stijn
00795d55df
Merge pull request #43784 from thaJeztah/containerd_image_service
Create the containerd image service
2022-07-15 13:38:27 +02:00
Samuel Karp
0136a7c1bb
Merge pull request #43804 from thaJeztah/gofmt_119_2 2022-07-14 21:29:07 -07:00
Tianon Gravi
d61c85f7df
Merge pull request #43811 from thaJeztah/moar_maintainers
Add Cory Snider (corhere) as maintainer
2022-07-14 11:00:46 -07:00
Sebastiaan van Stijn
ff3d2eb589
Add Cory Snider (corhere) as maintainer
Cory has actively participated in the project for many months, assisted in several
security advisories, code review, and triage, and (in short) already acted a
maintainer for some time (thank you!).

I nominated Cory as a maintainer per e-mail, and we reached quorum, so opening
this pull request to (should he choose to accept it) be added as a maintainer.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-14 19:38:48 +02:00
Akihiro Suda
9464898b47
libnetwork: skip firewalld management for rootless
Fix issue 43781

Co-authored-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2022-07-15 00:59:11 +09:00
Djordje Lukic
7d74269c0d
Create the containerd image service
Initial pull/ls works
Build is deactivated if the feature is active

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-14 16:46:11 +02:00