Commit graph

44393 commits

Author SHA1 Message Date
Sebastiaan van Stijn
ebf339628a
api: deprecate BuildCache.Parent in API >= v1.42
This field has been deprecated in BuildKit, so this follows the deprecation
in the Engine API.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-04 09:27:34 +02:00
Sebastiaan van Stijn
e0286d7f4e
api/types: add missing GoDoc for BuildCache fields.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-04 09:27:32 +02:00
Sebastiaan van Stijn
dc2b34af6a
api: swagger: document BuildCache fields.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-04 09:27:30 +02:00
Brian Goff
7e8df0e2c9
Merge pull request #43820 from thaJeztah/image_delete
containerd integration: Implement ImageDelete for containerd
2022-08-03 15:47:07 -07:00
Tianon Gravi
e60bddcc60
Merge pull request #43885 from thaJeztah/auth_header_refactor
Move AuthConfig to types/registry, and implement utilities for encoding/decoding
2022-08-03 11:31:26 -07:00
Sebastiaan van Stijn
9100e3c087
Merge pull request #43219 from corhere/sdjournal
Rewrite journald LogReader and unit test it
2022-08-02 23:47:35 +02:00
Sebastiaan van Stijn
510beeccf4
Merge pull request #43898 from vvoland/vendor-hcsshim
vendor: github.com/Microsoft/hcsshim v0.9.4
2022-08-02 17:19:35 +02:00
Sebastiaan van Stijn
074635002e
Merge pull request #43888 from thaJeztah/api_fix_missing_platform
api: swagger: add missing "platform" query-arg on create
2022-08-02 10:53:08 +02:00
Paweł Gronowski
69f077f1aa
vendor: github.com/Microsoft/hcsshim v0.9.4
full diff: https://github.com/microsoft/hcsshim/compare/v0.9.3...v0.9.4

Changes are mostly fixes of unsafe usage of `unsafe.Pointer`

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2022-08-02 10:03:11 +02:00
Sebastiaan van Stijn
982f09f837
docs: api: add missing "platform" query-arg on create (v1.42)
Commit 7a9cb29fb9 added a new "platform" query-
parameter to the `POST /containers/create` endpoint, but did not update the
swagger file and documentation.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-29 23:21:02 +02:00
Sebastiaan van Stijn
1000e4ee7d
docs: api: add missing "platform" query-arg on create (v1.41)
Commit 7a9cb29fb9 added a new "platform" query-
parameter to the `POST /containers/create` endpoint, but did not update the
swagger file and documentation.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-29 23:20:39 +02:00
Sebastiaan van Stijn
3dae8e9fc2
api: swagger: add missing "platform" query-arg on create
Commit 7a9cb29fb9 added a new "platform" query-
parameter to the `POST /containers/create` endpoint, but did not update the
swagger file and documentation.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-29 23:19:51 +02:00
Sebastiaan van Stijn
7ca66e3cfc
api: add registry.EncodeAuthConfig
Based on EncodeAuthToBase64 in docker/cli;
1f4111d2bf/cli/command/registry.go (L30-L37)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-29 23:11:19 +02:00
Sebastiaan van Stijn
7819811835
api: add registry.DecodeAuthConfig, registry.DecodeAuthConfigBody
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-29 23:11:13 +02:00
Sebastiaan van Stijn
e3a7a1c6ae
client: linting: fix "invalid auth header" error
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-29 23:05:22 +02:00
Sebastiaan van Stijn
360d38adb1
daemon/containerd: use types/registry.AuthConfig
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-29 23:05:21 +02:00
Sebastiaan van Stijn
2b7416ef34
testutil, integration: use types/registry.AuthConfig
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-29 23:05:21 +02:00
Sebastiaan van Stijn
d817f4dcee
registry: use types/registry.AuthConfig
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-29 23:05:20 +02:00
Sebastiaan van Stijn
55d1a56826
plugin: use types/registry.AuthConfig
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-29 23:05:19 +02:00
Sebastiaan van Stijn
04c339a12b
distribution: use types/registry.AuthConfig
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-29 23:05:18 +02:00
Sebastiaan van Stijn
a02abca49a
daemon: use types/registry.AuthConfig
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-29 23:05:18 +02:00
Sebastiaan van Stijn
39f3adc079
daemon/images: use types/registry.AuthConfig
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-29 23:05:17 +02:00
Sebastiaan van Stijn
05eacf92fe
daemon/cluster: use types/registry.AuthConfig
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-29 23:05:16 +02:00
Sebastiaan van Stijn
eaf1a604f2
client: use types/registry.AuthConfig
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-29 23:05:15 +02:00
Sebastiaan van Stijn
d8a43399a8
api/server/router: use types/registry.AuthConfig
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-29 23:05:14 +02:00
Sebastiaan van Stijn
818ee96219
api/types: move AuthConfig to registry types
Making the api types more focused per API type, and the general
api/types package somewhat smaller.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-29 23:05:12 +02:00
Sebastiaan van Stijn
857cb260c7
api: add const for 'X-Registry-Auth'
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-29 23:04:34 +02:00
Brian Goff
6e21389917
Merge pull request #43800 from corhere/implicit-runtime-config
daemon: support alternative runtimes MVP
2022-07-29 09:35:22 -07:00
Sebastiaan van Stijn
f34567bf41
Merge pull request #43751 from vvoland/fix-exitcode-wait
state/Wait: Fix race when reading exit status
2022-07-29 16:01:39 +02:00
Djordje Lukic
26c65447df
Implement ImageDelete for containerd
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-28 22:56:34 +02:00
Sebastiaan van Stijn
71cb54cec4
Merge pull request #43872 from thaJeztah/containerd_config_v2
libcontainerd: switch generated containerd.toml to v2 (v1 is deprecated)
2022-07-28 18:21:18 +02:00
Brian Goff
c5d2c8fb80
Merge pull request #43877 from Snorch/integration-cli-make-service-process-live-forever
integration-cli: Make service process live forever
2022-07-28 07:47:33 -07:00
Pavel Tikhomirov
f4c0ec8ffc integration-cli: Make service process live forever
- TestServiceLogsCompleteness runs service with command to write 6 log
lines but as command exits immediately, service is restarted and 6 more
lines are printed in logs, which confuses the checker.Equals(6) check.

- TestServiceLogsSince runs service with command to write 3 log lines,
and service restart can also affect it's checks.

Let's change from `tail` which exits immediately to `tail -f` which
hangs forever, this way we would not confuse checks with more log lines
when expected.

Signed-off-by: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>
2022-07-28 16:27:27 +03:00
Brian Goff
9772c68e54
Merge pull request #42934 from abdulrahimiliasu/42655-vfs-storage-driver
Fix file capabilites droping in Dockerfile
2022-07-27 14:31:39 -07:00
Cory Snider
547da0d575 daemon: support other containerd runtimes (MVP)
Contrary to popular belief, the OCI Runtime specification does not
specify the command-line API for runtimes. Looking at containerd's
architecture from the lens of the OCI Runtime spec, the _shim_ is the
OCI Runtime and runC is "just" an implementation detail of the
io.containerd.runc.v2 runtime. When one configures a non-default runtime
in Docker, what they're really doing is instructing Docker to create
containers using the io.containerd.runc.v2 runtime with a configuration
option telling the runtime that the runC binary is at some non-default
path. Consequently, only OCI runtimes which are compatible with the
io.containerd.runc.v2 shim, such as crun, can be used in this manner.
Other OCI runtimes, including kata-containers v2, come with their own
containerd shim and are not compatible with io.containerd.runc.v2.
As Docker has not historically provided a way to select a non-default
runtime which requires its own shim, runtimes such as kata-containers v2
could not be used with Docker.

Allow other containerd shims to be used with Docker; no daemon
configuration required. If the daemon is instructed to create a
container with a runtime name which does not match any of the configured
or stock runtimes, it passes the name along to containerd verbatim. A
user can start a container with the kata-containers runtime, for
example, simply by calling

    docker run --runtime io.containerd.kata.v2

Runtime names which containerd would interpret as a path to an arbitrary
binary are disallowed. While handy for development and testing it is not
strictly necessary and would allow anyone with Engine API access to
trivially execute any binary on the host as root, so we have decided it
would be safest for our users if it was not allowed.

It is not yet possible to set an alternative containerd shim as the
default runtime; it can only be configured per-container.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-07-27 14:22:49 -04:00
Illo Abdulrahim
31f654a704
Fix file capabilities droping in Dockerfile
doCopyXattrs() never reached due to copyXattrs boolean being false, as
a result file capabilities not being copied.

moved copyXattr() out of doCopyXattrs()

Signed-off-by: Illo Abdulrahim <abdulrahim.illo@nokia.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-27 17:21:02 +02:00
Sebastiaan van Stijn
ba2ff69894
libcontainerd: switch generated containerd.toml to v2 (v1 is deprecated)
Before this patch:

    INFO[2022-07-27T14:30:06.188762628Z] Starting up
    INFO[2022-07-27T14:30:06.190750725Z] libcontainerd: started new containerd process  pid=2028
    ...
    WARN[0000] containerd config version `1` has been deprecated and will be removed in containerd v2.0, please switch to version `2`, see https://github.com/containerd/containerd/blob/main/docs/PLUGINS.md#version-header
    INFO[2022-07-27T14:30:06.220024286Z] starting containerd                           revision=10c12954828e7c7c9b6e0ea9b0c02b01407d3ae1 version=v1.6.6

With this patch:

    INFO[2022-07-27T14:28:04.025543517Z] Starting up
    INFO[2022-07-27T14:28:04.027447105Z] libcontainerd: started new containerd process  pid=1377
    ...
    INFO[2022-07-27T14:28:04.054483270Z] starting containerd                           revision=10c12954828e7c7c9b6e0ea9b0c02b01407d3ae1 version=v1.6.6

And the generated /var/run/docker/containerd/containerd.toml:

```toml
disabled_plugins = ["io.containerd.grpc.v1.cri"]
imports = []
oom_score = 0
plugin_dir = ""
required_plugins = []
root = "/var/lib/docker/containerd/daemon"
state = "/var/run/docker/containerd/daemon"
temp = ""
version = 2

[cgroup]
  path = ""

[debug]
  address = "/var/run/docker/containerd/containerd-debug.sock"
  format = ""
  gid = 0
  level = "debug"
  uid = 0

[grpc]
  address = "/var/run/docker/containerd/containerd.sock"
  gid = 0
  max_recv_message_size = 16777216
  max_send_message_size = 16777216
  tcp_address = ""
  tcp_tls_ca = ""
  tcp_tls_cert = ""
  tcp_tls_key = ""
  uid = 0

[metrics]
  address = ""
  grpc_histogram = false

[plugins]

[proxy_plugins]

[stream_processors]

[timeouts]

[ttrpc]
  address = ""
  gid = 0
  uid = 0
```

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-27 16:33:00 +02:00
Sebastiaan van Stijn
2bfc7aedab
Merge pull request #43866 from olljanat/win-enable-attach-websocket
integration-cli: Enable TestGetContainersAttachWebsocket for Windows
2022-07-26 14:20:05 +02:00
Sebastiaan van Stijn
45f845a260
Merge pull request #43863 from vvoland/nit-windowscode
integration-cli: Remove unnecessary Windows-only code
2022-07-26 14:16:25 +02:00
Cory Snider
ef5b279887 logger/journald: implement --follow correctly
Implement --follow entirely correctly for the journald log reader, such
that it exits immediately upon reading back the last log message written
to the journal before the logger was closed. The impossibility of doing
so has been slightly exaggerated.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-07-25 16:41:38 -04:00
Cory Snider
e278d3f185 logger/*: fix reading w/ non-monotonic timestamps
Fix journald and logfile-powered (jsonfile, local) log readers
incorrectly filtering out messages with timestamps < Since which were
preceded by a message with a timestamp >= Since.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-07-25 16:41:38 -04:00
Cory Snider
342b44bf20 logger/journald: rewrite reader w/o cursors
Careful management of the journal read pointer is sufficient to ensure
that no entry is read more than once.

Unit test the journald logger without requiring a running journald by
using the systemd-journal-remote command to write arbitrary entries to
journal files.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-07-25 16:41:38 -04:00
Cory Snider
3e8405aa82 logger/journald: refactor cgo out of business logic
Wrap the libsystemd journal reading functionality in a more idiomatic Go
API and refactor the journald logging driver's ReadLogs implementation
to use the wrapper. Rewrite the parts of the ReadLogs implementation in
Go which were previously implemented in C as part of the cgo preamble.
Separating the business logic from the cgo minutiae should hopefully
make the code more accessible to a wider audience of developers for
reviewing the code and contributing improvements.

The structure of the ReadLogs implementation is retained with few
modifications. Any ignored errors were also ignored before the refactor;
the explicit error return values afforded by the sdjournal wrapper makes
this more obvious.

The package github.com/coreos/go-systemd/v22/sdjournal also provides a
more idiomatic Go wrapper around libsystemd. It is unsuitable for our
needs as it does not expose wrappers for the sd_journal_process and
sd_journal_get_fd functions.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-07-25 16:41:09 -04:00
Cory Snider
7ff4b64319 logger/journald: simplify control flow
Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-07-25 16:37:59 -04:00
Cory Snider
f7fe2c2290 logger/journald: follow the thread-safety rules
Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-07-25 16:37:59 -04:00
Cory Snider
12744335c0 logger/journald: simplify build constraints
Ensure the package can be imported, no matter the build constratints, by
adding an unconstrained doc.go containing a package statement.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-07-25 16:37:59 -04:00
Cory Snider
0e0b300a1c Fix make BIND_DIR=. DOCKER_SYSTEMD=1 shell
Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-07-25 16:37:59 -04:00
Olli Janatuinen
a61f7abf5a integration-cli: Enable TestGetContainersAttachWebsocket for Windows
Signed-off-by: Olli Janatuinen <olli.janatuinen@gmail.com>
2022-07-25 08:17:53 -07:00
Paweł Gronowski
6cc644abef integration-cli: Remove unnecessary Windows-only code
This test is skipped on Windows anyway.
Also add a short explanation why emptyfs image was chosen.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2022-07-25 15:08:55 +02:00
Sebastiaan van Stijn
98d8343aa2
Merge pull request #43838 from thaJeztah/remove_journald_compat
logger/journald: remove journald_compat (for systemd < 209)
2022-07-25 11:25:52 +02:00