Commit graph

7388 commits

Author SHA1 Message Date
Sebastiaan van Stijn
99a65dc93e
Merge pull request #45055 from vvoland/c8d-tag-fix-createderr
c8d/tag: Don't create a separate error variable
2023-02-22 13:13:46 +01:00
Paweł Gronowski
8657c87c8c
c8d/tag: Don't create a separate error variable
Checking if the image creation failed due to IsAlreadyExists didn't use
the error from ImageService.Create.
Error from ImageService.Create was stored in a separate variable and
later IsAlreadyExists checked the standard `err` variable instead of the
`createErr`.
As there's no need to store the error in a separate variable - just
assign it to err variable and fix the check.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-02-22 10:09:23 +01:00
Bjorn Neergaard
782a369f92
c8d/pull: Add CRI-compatible annotation of pulled content
Co-authored-by: Paweł Gronowski <pawel.gronowski@docker.com>
Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
2023-02-21 08:35:52 -07:00
Bjorn Neergaard
c4c54683a9
Revert "apparmor: Check if apparmor_parser is available"
This reverts commit ab3fa46502.

This fix was partial, and is not needed with the proper fix in
containerd.

Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
2023-02-16 08:26:25 -07:00
Cory Snider
efb6950299
Merge pull request #45007 from corhere/fix-ineffective-add
daemon/logger/awslogs: fix ineffective Add in test
2023-02-15 15:48:43 -05:00
Cory Snider
e66995d840 d/l/awslogs: fix ineffective Add in test
...flagged by golangci-lint v1.51.1 (staticcheck).

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-02-15 13:11:55 -05:00
Cory Snider
713e02e03e daemon: handle EISDIR error from runtime
Go 1.20 made a change to the behaviour of package "os/exec" which was
not mentioned in the release notes:
2b8f214094
Attempts to execute a directory now return syscall.EISDIR instead of
syscall.EACCESS. Check for EISDIR errors from the runtime and fudge the
returned error message to maintain compatibility with existing versions
of docker/cli when using a version of runc compiled with Go 1.20+.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-02-15 13:06:04 -05:00
Paweł Gronowski
62be425bcc
api: Extract parsing reference from repo and tag
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-02-07 15:43:58 +01:00
Paweł Gronowski
afc6e3fa46
c8d/import: Use danglingImageName instead of hardcoded
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-02-07 15:43:56 +01:00
Paweł Gronowski
eaa56afda9
daemon/c8d: Implement TagImageWithReference
Implements image tagging under containerd image store.

If an image with this tag already exists, and there's no other image
with the same target, change its name. The name will have a special
format `moby-dangling@<digest>` which isn't a valid canonical reference
and doesn't resolve to any repository.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-02-07 15:43:53 +01:00
Paweł Gronowski
1ca85e835c
daemon/imageService: Remove TagImageWithReference
TagImage is just a wrapper for TagImageWithReference which parses the
repo and tag into a reference. Change TagImageWithReference into
TagImage and move the responsibility of reference parsing to caller.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-02-07 15:43:52 +01:00
Nicolas De Loof
7b6f71dced
daemon: Pass ctx to image tagging operations
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-02-07 15:43:48 +01:00
Sebastiaan van Stijn
b5568723ce
Merge pull request #44902 from vvoland/apparmor-check-binary
apparmor: Check if apparmor_parser is available
2023-02-07 11:26:24 +01:00
Paweł Gronowski
ab3fa46502
apparmor: Check if apparmor_parser is available
`hostSupports` doesn't check if the apparmor_parser is available.
It's possible in some environments that the apparmor will be enabled but
the tool to load the profile is not available which will cause the
ensureDefaultAppArmorProfile to fail completely.

This patch checks if the apparmor_parser is available. Otherwise the
function returns early, but still logs a warning to the daemon log.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-02-07 09:39:08 +01:00
Djordje Lukic
0137446248 Implement run using the containerd snapshotter
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>

c8d/daemon: Mount root and fill BaseFS

This fixes things that were broken due to nil BaseFS like `docker cp`
and running a container with workdir override.

This is more of a temporary hack than a real solution.
The correct fix would be to refactor the code to make BaseFS and LayerRW
an implementation detail of the old image store implementation and use
the temporary mounts for the c8d implementation instead.
That requires more work though.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>

daemon/images: Don't unset BaseFS

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-02-06 18:21:50 +01:00
Bjorn Neergaard
3bcb350711
graphdriver/overlay2: usingMetacopy ENOTSUP is non-fatal
Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
2023-02-03 05:10:53 -07:00
Bjorn Neergaard
0de32693d0
Merge pull request #44888 from corhere/fix-kata-exec-exit
Fix exit-event handling for Kata runtime
2023-02-02 18:42:22 -07:00
Brian Goff
4c0d75bc8e
Merge pull request #44833 from vvoland/c8d-saveload-upstream
daemon/c8d: Implement save and load
2023-02-02 12:45:42 -08:00
Paweł Gronowski
af32603ae3
daemon/c8d: Implement save and load
This makes the `docker save` and `docker load` work with the containerd
image store. The archive is both OCI and Docker compatible.

Saved archive will only contain content which is available locally.  In
case the saved image is a multi-platform manifest list, the behavior
depends on the local availability of the content. This is to be
reconsidered when we have the `--platform` option in the CLI.

- If all manifests and their contents, referenced by the manifest list
  are present, then the manifest-list is exported directly and the ID
will be the same.
- If only one platform manifest is present, only that manifest is
  exported (the image id will change and will be the id of
platform-specific manifest, instead of the full manifest list).
- If multiple, but not all, platform manifests are available, a new
  manifest list will be created which will be a subset of the original
manifest list.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-02-02 17:42:03 +01:00
Tonis Tiigi
666334bd48
vendor: update buildkit to v0.11.2
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2023-02-01 23:29:10 -08:00
Cory Snider
0e4d3feb94 daemon: identify container exits by ProcessID
The Pid field of an exit event cannot be relied upon to differentiate
exits of the container's task from exits of other container processes,
i.e. execs. The Pid is reported by the runtime and is implementation-
defined so there is no guarantee that a task's pid is distinct from the
pids of any other process in the same container. In particular,
kata-containers reports the pid of the hypervisor for all exit events.
ContainerD guarantees that the process ID of a task is set to the
corresponding container ID, so use that invariant to distinguish task
exits from other process exits.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-01-31 17:02:43 -05:00
Cory Snider
d634ae9b60 Wait longer for exit events on Windows
The latest version of containerd-shim-runhcs-v1 (v0.10.0-rc.4) pulled in
with the bump to ContainerD v1.7.0-rc.3 had several changes to make it
more robust, which had the side effect of increasing the worst-case
amount of time it takes for a container to exit in the worst case.
Notably, the total timeout for shutting down a task increased from 30
seconds to 60! Increase the timeouts hardcoded in the daemon and
integration tests so that they don't give up too soon.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-01-31 15:19:08 -05:00
Cory Snider
b54af02b51
Merge pull request #44875 from corhere/libnet/local-scope-only
libnetwork: clean up vestigial datastore-related code
2023-01-27 13:26:44 -05:00
Tianon Gravi
6a69560847
Merge pull request #44837 from vvoland/c8d-fix-progress-notending-upstream
c8d/progress: Update progress when finished
2023-01-26 14:58:09 -08:00
Cory Snider
befff0e13f libnetwork: remove more datastore scope plumbing
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-01-26 17:56:40 -05:00
Cory Snider
cc19eba579 daemon: let libnetwork assign default bridge IPAM
The netutils.ElectInterfaceAddresses function is only used in one place
outside of tests: in the daemon, to configure the default bridge
network. The function is also messy to reason about as it references the
shared mutable state of ipamutils.PredefinedLocalScopeDefaultNetworks.
It uses the list of predefined default networks to always return an IPv4
address even if the named interface does not exist or does not have any
IPv4 addresses. This list happens to be the same as the one used to
initialize the address pool of the 'builtin' IPAM driver, though that is
far from obvious. (Start with "./libnetwork".initIPAMDrivers and trace
the dataflow of the addressPool value. Surprise! Global state is being
mutated using the value of other global mutable state.)

The daemon does not need the fallback behaviour of
ElectInterfaceAddresses. In fact, the daemon does not have to configure
an address pool for the network at all! libnetwork will acquire one of
the available address ranges from the network's IPAM driver when the
preferred-pool configuration is unset. It will do so using the same list
of address ranges and the exact same logic
(netutils.FindAvailableNetworks) as ElectInterfaceAddresses. So unless
the daemon needs to force the network to use a specific address range
because the bridge interface already exists, it can leave the details
up to libnetwork.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-01-26 14:54:57 -05:00
Tianon Gravi
2b9de2e24a
Merge pull request #44809 from rumpl/containerd-image-history
daemon/c8d: Implement image history
2023-01-24 14:35:05 -08:00
Paweł Gronowski
1776f72c96
c8d/progress: Update progress when finished
After the context is cancelled, update the progress for the last time.
This makes sure that the progress also includes finishing updates.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-01-24 10:46:58 +01:00
Djordje Lukic
54f0b0ec4b c8d: Implement image history
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-01-23 16:50:02 +01:00
Bjorn Neergaard
65c371ed6e
Merge pull request #44842 from vvoland/c8d-fix-truncatedid
c8d: Fix resolving truncated id to a descriptor
2023-01-21 19:43:29 -07:00
Sebastiaan van Stijn
17b2c89116
Merge pull request #44818 from vvoland/c8d-commit-ctx-upstream
images: Pass context to commit related operations
2023-01-18 15:17:52 +01:00
Nicolas De Loof
9b5c21309b
images: Pass context to commit related operations
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-01-18 12:23:32 +01:00
Paweł Gronowski
596c8960b2
c8d: Fix resolving truncated id to a descriptor
Regular expression beginning anchor was placed after `sha256:` digest
prefix.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-01-18 12:10:07 +01:00
Cory Snider
c71555f030 libnetwork: return concrete-typed *Endpoint
libnetwork.Endpoint is an interface with a single implementation.

https://github.com/golang/go/wiki/CodeReviewComments#interfaces

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-01-13 14:19:06 -05:00
Cory Snider
0e91d2e0e9 libnetwork: return concrete-typed *Sandbox
Basically every exported method which takes a libnetwork.Sandbox
argument asserts that the value's concrete type is *sandbox. Passing any
other implementation of the interface is a runtime error! This interface
is a footgun, and clearly not necessary. Export and use the concrete
type instead.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-01-13 14:19:06 -05:00
Cory Snider
f96b9bf761 libnetwork: return concrete-typed *Controller
libnetwork.NetworkController is an interface with a single
implementation.

https://github.com/golang/go/wiki/CodeReviewComments#interfaces

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-01-13 14:09:37 -05:00
Bjorn Neergaard
868c897ad7
Merge pull request #44790 from neersighted/wide_json
daemon/config: support alternate (common) unicode encodings using a BOM
2023-01-12 13:15:58 -07:00
Bjorn Neergaard
4cd84c4b13
Merge pull request #44773 from neersighted/daemon_config_test
daemon/config: clean up tests to use common helper
2023-01-12 13:13:35 -07:00
Bjorn Neergaard
228f82fcda
Merge pull request #44628 from vvoland/c8d-import-upstream
daemon/c8d: Implement import
2023-01-12 09:35:43 -07:00
Bjorn Neergaard
4c02882f8a
Merge pull request #43225 from aivus/docker-in-overlayfs
Work around missing rename support when the backing filesystem is overlayfs
2023-01-11 17:54:32 -07:00
Bjorn Neergaard
6d212fa045
Merge pull request #44756 from rumpl/containerd-image-pull
containerd integration: image pull
2023-01-11 16:22:48 -07:00
Paweł Gronowski
9032e6779d c8d/resolver: Fallback to http for insecure registries
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-01-11 17:00:27 +01:00
Nicolas De Loof
c83fce86d4 c8d/resolver: Use hosts from daemon configuration
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2023-01-11 17:00:27 +01:00
Paweł Gronowski
3a3f98b32b c8d/pull: Don't unpack manually
We pass WithPullUnpack anyway

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-01-11 17:00:27 +01:00
Nicolas De Loof
4a8c4110e3 produce progress events polling ctrd's content.Store
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

containerd: Push progress

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-01-11 17:00:26 +01:00
Paweł Gronowski
d7deec1993
daemon/c8d: Implement import
If the imported layer archive is uncompressed, it gets compressed with
gzip before writing to the content store.
Archives compressed with gzip and zstd are imported as-is.
Xz and bzip2 are recompressed into gzip.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-01-11 13:53:44 +01:00
Paweł Gronowski
28327f10a2
daemon/import: Extract common logic to api
Extract logic that would need to be duplicated in both implementations
of ImageService.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-01-11 13:53:40 +01:00
Bjorn Neergaard
8dbc5df952
daemon/config: inform the user when the input JSON contains invalid UTF-8
This helps ensure that users are not surprised by unexpected tokens in
the JSON parser, or fallout later in the daemon.

Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
2023-01-10 17:27:21 -07:00
Bjorn Neergaard
d42495033e
daemon/config: support alternate (common) unicode encodings using a BOM
This is a pragmatic but impure choice, in order to better support the
default tools available on Windows Server, and reduce user confusion due
to otherwise inscrutable-to-the-uninitiated errors like the following:

> invalid character 'þ' looking for beginning of value
> invalid character 'ÿ' looking for beginning of value

While meaningful to those who are familiar with and are equipped to
diagnose encoding issues, these characters will be hidden when the file
is edited with a BOM-aware text editor, and further confuse the user.

Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
2023-01-10 15:49:15 -07:00
Bjorn Neergaard
1dcf7d5b03
daemon/config: clean up tests to use common helper
Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
2023-01-10 13:54:17 -07:00