Commit graph

46085 commits

Author SHA1 Message Date
Sebastiaan van Stijn
e8a79114b8
Merge pull request #45274 from thaJeztah/container_as_source_of_truth
c8d: use the container's configured snapshotter where possible
2023-04-05 17:03:18 +02:00
Sebastiaan van Stijn
f6cc8e3512
update go to go1.20.3
go1.20.3 (released 2023-04-04) includes security fixes to the go/parser,
html/template, mime/multipart, net/http, and net/textproto packages, as well
as bug fixes to the compiler, the linker, the runtime, and the time package.
See the Go 1.20.3 milestone on our issue tracker for details:

https://github.com/golang/go/issues?q=milestone%3AGo1.20.3+label%3ACherryPickApproved

full diff: https://github.com/golang/go/compare/go1.20.2...go1.20.3

Further details from the announcement on the mailing list:

We have just released Go versions 1.20.3 and 1.19.8, minor point releases.
These minor releases include 4 security fixes following the security policy:

- go/parser: infinite loop in parsing

  Calling any of the Parse functions on Go source code which contains `//line`
  directives with very large line numbers can cause an infinite loop due to
  integer overflow.
  Thanks to Philippe Antoine (Catena cyber) for reporting this issue.
  This is CVE-2023-24537 and Go issue https://go.dev/issue/59180.

- html/template: backticks not treated as string delimiters

  Templates did not properly consider backticks (`) as Javascript string
  delimiters, and as such did not escape them as expected. Backticks are
  used, since ES6, for JS template literals. If a template contained a Go
  template action within a Javascript template literal, the contents of the
  action could be used to terminate the literal, injecting arbitrary Javascript
  code into the Go template.

  As ES6 template literals are rather complex, and themselves can do string
  interpolation, we've decided to simply disallow Go template actions from being
  used inside of them (e.g. "var a = {{.}}"), since there is no obviously safe
  way to allow this behavior. This takes the same approach as
  github.com/google/safehtml. Template.Parse will now return an Error when it
  encounters templates like this, with a currently unexported ErrorCode with a
  value of 12. This ErrorCode will be exported in the next major release.

  Users who rely on this behavior can re-enable it using the GODEBUG flag
  jstmpllitinterp=1, with the caveat that backticks will now be escaped. This
  should be used with caution.

  Thanks to Sohom Datta, Manipal Institute of Technology, for reporting this issue.

  This is CVE-2023-24538 and Go issue https://go.dev/issue/59234.

- net/http, net/textproto: denial of service from excessive memory allocation

  HTTP and MIME header parsing could allocate large amounts of memory, even when
  parsing small inputs.

  Certain unusual patterns of input data could cause the common function used to
  parse HTTP and MIME headers to allocate substantially more memory than
  required to hold the parsed headers. An attacker can exploit this behavior to
  cause an HTTP server to allocate large amounts of memory from a small request,
  potentially leading to memory exhaustion and a denial of service.
  Header parsing now correctly allocates only the memory required to hold parsed
  headers.

  Thanks to Jakob Ackermann (@das7pad) for discovering this issue.

  This is CVE-2023-24534 and Go issue https://go.dev/issue/58975.

- net/http, net/textproto, mime/multipart: denial of service from excessive resource consumption

  Multipart form parsing can consume large amounts of CPU and memory when
  processing form inputs containing very large numbers of parts. This stems from
  several causes:

  mime/multipart.Reader.ReadForm limits the total memory a parsed multipart form
  can consume. ReadForm could undercount the amount of memory consumed, leading
  it to accept larger inputs than intended. Limiting total memory does not
  account for increased pressure on the garbage collector from large numbers of
  small allocations in forms with many parts. ReadForm could allocate a large
  number of short-lived buffers, further increasing pressure on the garbage
  collector. The combination of these factors can permit an attacker to cause an
  program that parses multipart forms to consume large amounts of CPU and
  memory, potentially resulting in a denial of service. This affects programs
  that use mime/multipart.Reader.ReadForm, as well as form parsing in the
  net/http package with the Request methods FormFile, FormValue,
  ParseMultipartForm, and PostFormValue.

  ReadForm now does a better job of estimating the memory consumption of parsed
  forms, and performs many fewer short-lived allocations.

  In addition, mime/multipart.Reader now imposes the following limits on the
  size of parsed forms:

  Forms parsed with ReadForm may contain no more than 1000 parts. This limit may
  be adjusted with the environment variable GODEBUG=multipartmaxparts=. Form
  parts parsed with NextPart and NextRawPart may contain no more than 10,000
  header fields. In addition, forms parsed with ReadForm may contain no more
  than 10,000 header fields across all parts. This limit may be adjusted with
  the environment variable GODEBUG=multipartmaxheaders=.

  Thanks to Jakob Ackermann for discovering this issue.

  This is CVE-2023-24536 and Go issue https://go.dev/issue/59153.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-05 14:24:07 +02:00
Sebastiaan van Stijn
465cbccdaf
c8d: use the container's configured snapshotter where possible
While we currently do not provide an option to specify the snapshotter to use
for individual containers (we may want to add this option in future), currently
it already is possible to configure the snapshotter in the daemon configuration,
which could (likely) cause issues when changing and restarting the daemon.

This patch updates some code-paths that have the container available to use
the snapshotter that's configured for the container (instead of the default
snapshotter configured).

There are still code-paths to be looked into, and a tracking ticket as well as
some TODO's were added for those.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-05 14:09:21 +02:00
Sebastiaan van Stijn
be541997af
Merge pull request #45256 from thaJeztah/integration_cli_cleanups
integration-cli: rename vars that collided with imports
2023-04-05 09:11:51 +02:00
Sebastiaan van Stijn
c2c83e16a2
Merge pull request #45266 from rumpl/c8d-tag-delete-events
c8d: Send an event when an image is tagged or deleted
2023-04-05 09:10:43 +02:00
Wesley Pettit
c8f8d11ac4
awslogs: fix non-blocking log drop bug
Previously, the AWSLogs driver attempted to implement
non-blocking itself. Non-blocking is supposed to
implemented solely by the Docker RingBuffer that
wraps the log driver.

Please see issue and explanation here:
https://github.com/moby/moby/issues/45217

Signed-off-by: Wesley Pettit <wppttt@amazon.com>
2023-04-04 17:03:06 -07:00
Sebastiaan van Stijn
878ee341d6
Merge pull request from GHSA-232p-vwff-86mp
libnetwork: ensure encryption is mandatory on encrypted overlay networks
2023-04-04 20:03:51 +02:00
Djordje Lukic
1b178dffa3
Send an event when an image is deleted
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-04-04 15:54:11 +02:00
Sebastiaan van Stijn
2fa8f54d08
Merge pull request #45264 from corhere/fix-default-network-opts-config
daemon/config: fix default-network-opts in daemon.json
2023-04-04 13:04:42 +02:00
Sebastiaan van Stijn
705cd2cabe
Merge pull request #45258 from thaJeztah/remove_fsutils_aliases
pkg/fileutils: remove aliases for deprecated functions and types
2023-04-04 13:04:15 +02:00
Djordje Lukic
36d28b73fc
Send an event when an image is tagged
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-04-04 11:32:02 +02:00
Sebastiaan van Stijn
5530226366
Merge pull request #45263 from corhere/libc8d/fix-stdin-close-err
libcontainerd: close stdin sync if possible
2023-04-04 10:44:10 +02:00
Cory Snider
750218c5e5 d/config: fix default-network-opts in daemon.json
Prevent the daemon from erroring out if daemon.json contains default
network options for network drivers aside from bridge. Configuring
defaults for the bridge driver previously worked by coincidence because
the unrelated CLI flag '--bridge' exists.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-04-03 15:59:53 -04:00
Cory Snider
36935bd869 libcontainerd: close stdin sync if possible
Closing stdin of a container or exec (a.k.a.: task or process) has been
somewhat broken ever since support for ContainerD 1.0 was introduced
back in Docker v17.11: the error returned from the CloseIO() call was
effectively ignored due to it being assigned to a local variable which
shadowed the intended variable. Serendipitously, that oversight
prevented a data race. In my recent refactor of libcontainerd, I
corrected the variable shadowing issue and introduced the aforementioned
data race in the process.

Avoid deadlocking when closing stdin without swallowing errors or
introducing data races by calling CloseIO() synchronously if the process
handle is available, falling back to an asynchronous close-and-log
strategy otherwise. This solution is inelegant and complex, but looks to
be the best that could be done without changing the libcontainerd API.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-04-03 15:25:16 -04:00
Sebastiaan van Stijn
816cd9802f
Merge pull request #45259 from rumpl/c8d-load-save-events
c8d: Send event when an image is imported
2023-04-03 20:05:45 +02:00
Sebastiaan van Stijn
3beda17773
integration-cli: rename vars that collided with imports
- use apiClient for api-clients to reduce shadowing (also more "accurate")
- use "ctr" instead of "container"

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-03 20:01:05 +02:00
Sebastiaan van Stijn
781740c19d
Merge pull request #45124 from vvoland/c8d-list-ignore-attestations
c8d/list: Ignore attestation manifests
2023-04-03 19:56:29 +02:00
Paweł Gronowski
92e38b6e27
c8d/list: Ignore attestation manifests
Attestation manifests have an OCI image media type, which makes them
being listed like they were a separate platform supported by the image.

Don't use `images.Platforms` and walk the manifest list ourselves
looking for all manifests that are an actual image manifest.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-04-03 17:50:56 +02:00
Djordje Lukic
773a9d85a1
c8d: Send event when an image is exported
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-04-03 15:30:04 +02:00
Djordje Lukic
00545ef4d6
c8d: Send event when an image is imported
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-04-03 15:21:18 +02:00
Sebastiaan van Stijn
e59f7fba6a
pkg/fileutils: remove aliases for deprecated functions and types
commit 3c69b9f2c5 replaced these functions
and types with github.com/moby/patternmatcher. That commit has shipped with
docker 23.0, and BuildKit v0.11 no longer uses the old functions, so we can
remove these.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-03 15:00:51 +02:00
Sebastiaan van Stijn
d2a5948ae8
Merge pull request #45250 from corhere/really-deprecate-newclient
client: deprecate NewClient properly
2023-04-01 01:50:02 +02:00
Sebastiaan van Stijn
348f83670a
Merge pull request #45249 from vvoland/c8d-push-upstream-2
c8d/push: Follow up fixes
2023-03-31 22:37:59 +02:00
Cory Snider
6b9968e8b1 client: deprecate NewClient properly
The 'Deprecated:' line in NewClient's doc comment was not in a new
paragraph, so GoDoc, linters, and IDEs were unaware that it was
deprecated. The package documentation also continued to reference
NewClient. Update the doc comments to finish documenting that NewClient
is deprecated.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-03-31 16:15:01 -04:00
Sebastiaan van Stijn
d94d51d00b
Merge pull request #45244 from thaJeztah/bump_containerd_1.6.20
vendor: github.com/containerd/containerd v1.6.20
2023-03-31 19:29:35 +02:00
Sebastiaan van Stijn
126e212fb0
Merge pull request #45243 from thaJeztah/c8d_push_nits
c8d: push: addressing some nits
2023-03-31 19:28:43 +02:00
Sebastiaan van Stijn
ae64fd8d6f
Merge pull request #45247 from akerouanton/drop-ElectInterfaceAddress
libnetwork/netutils: drop ElectInterfaceAddresses
2023-03-31 19:27:40 +02:00
Paweł Gronowski
027f70562e
c8d/push: Rename jobs to avoid shadowing type
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-03-31 16:30:14 +02:00
Paweł Gronowski
e1dd9c0396
c8d/push: Ignore :port when comparing source repository
Distribution source labels don't store port of the repository. If the
content was obtained from repository 172.17.0.2:5000 then its
corresponding label will have a key "containerd.io/distribution.source.172.17.0.2".

Fix the check in canBeMounted to ignore the :port part of the domain.

This also removes the check which prevented insecure repositories to use
cross-repo mount - the real cause was the mismatch in domain comparison.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-03-31 16:30:11 +02:00
Paweł Gronowski
4a34c501c1
c8d/push: Parse source labels with multiple sources
Distribution source label can specify multiple repositories - in this
case value is a comma separated list of source repositories.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-03-31 16:30:09 +02:00
Paweł Gronowski
cb788bea9c
c8d/push: Add distribution source labels only on success
Previously the labels would be appended for content that was pushed
even if subsequent pushes of other content failed.
Change the behavior to only append the labels if the whole push
operation succeeded.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-03-31 16:30:08 +02:00
Paweł Gronowski
013e44ec5d
c8d/push: Log error if releasing the lease failed
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-03-31 16:30:06 +02:00
Paweł Gronowski
c9012c798e
c8d/push: Sync mountableBlobs access
Handler is called in parallel and modifying a map without
synchronization is a race condition.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-03-31 16:30:03 +02:00
Sebastiaan van Stijn
0fa65ad6d5
Merge pull request #44242 from thaJeztah/libnetwork_migrate_extdns
libnetwork: sbState: rename ExtDNS2 back to ExtDNS
2023-03-31 16:05:57 +02:00
Albin Kerouanton
f6b50d52d4
libnetwork/netutils: drop ElectInterfaceAddresses
This is a follow-up of 48ad9e1. This commit removed the function
ElectInterfaceAddresses from utils_linux.go but not their FreeBSD &
Windows counterpart. As these functions are never called, they can be
safely removed.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-03-31 09:37:03 +02:00
Sebastiaan van Stijn
61a0e79ad6
vendor: github.com/containerd/containerd v1.6.20
full diff: https://github.com/containerd/containerd/compare/de33abf0547c...v1.6.20

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-03-31 00:44:31 +02:00
Sebastiaan van Stijn
b8e963595e libnetwork: sbState: rename ExtDNS2 back to ExtDNS
The ExtDNS2 field was added in
aad1632c15
to migrate existing state from < 1.14 to a new type. As it's unlikely
that installations still have state from before 1.14, rename ExtDNS2
back to ExtDNS and drop the migration code.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Co-authored-by: Cory Snider <csnider@mirantis.com>
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-03-30 18:44:24 -04:00
Sebastiaan van Stijn
0233029d5a
vendor: opencontainers/image-spec v1.1.0-rc2.0.20221005185240-3a7f492d3f1b
full diff: 02efb9a75e...3a7f492d3f

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-03-31 00:42:36 +02:00
Sebastiaan van Stijn
c733cf223d
vendor: github.com/containerd/ttrpc v1.1.1
- server: Fix connection leak when receiving ECONNRESET

full diff: https://github.com/containerd/ttrpc/compare/v1.1.0...v1.1.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-03-31 00:39:50 +02:00
Sebastiaan van Stijn
7c93e4a09b
Merge pull request #45241 from rumpl/c8d-image-events
c8d: Add the events services to the containerd image service
2023-03-31 00:31:04 +02:00
Sebastiaan van Stijn
a0ef2ac59c
c8d: push: addressing some nits
- make jobs.Add accept a list of jobs, so that we don't have to
  repeatedly lock/unlock the mutex
- rename some variables that collided with imports or types

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-03-31 00:23:09 +02:00
Tianon Gravi
f537ef5746
Merge pull request #44963 from vvoland/c8d-push-upstream
c8d: Implement push
2023-03-30 14:43:41 -07:00
Sebastiaan van Stijn
0656059ae7
Merge pull request #45122 from vvoland/c8d-upstream-inspect-digest
c8d/inspect: Add digested reference to details
2023-03-30 22:53:12 +02:00
Sebastiaan van Stijn
49d1e1c9b1
Merge pull request #45231 from vvoland/c8d-inspect-dangling
c8d/inspect: Handle dangling images
2023-03-30 22:26:52 +02:00
Sebastiaan van Stijn
328de8442d
Merge pull request #45224 from cpuguy83/move_bk_client
Move buildkit client opts to new package
2023-03-30 21:13:00 +02:00
Sebastiaan van Stijn
ea13744e53
Merge pull request #45239 from thaJeztah/vendor_runc_1.1.5
vendor: github.com/opencontainers/runc v1.1.5
2023-03-30 21:01:19 +02:00
Sebastiaan van Stijn
f7b6aefe0e
Merge pull request #44964 from laurazard/c8d-docker-diff
containerd integration: `docker diff`
2023-03-30 20:58:54 +02:00
Sebastiaan van Stijn
9c63abbfe7
Merge pull request #45236 from thaJeztah/update_runc_binary_1.1.5
update runc binary to v1.1.5
2023-03-30 20:47:11 +02:00
Paweł Gronowski
a75354c045
c8d: Implement push
This implements `docker push` under containerd image store.  When
pushing manifest lists that reference a content which is not present in
the local content store, it will attempt to perform the cross-repo mount
the content if possible.

Considering this scenario:

```bash
$ docker pull docker.io/library/busybox
```
This will download manifest list and only host platform-specific
manifest and blobs.

Note, tagging to a different repository (but still the same registry) and pushing:
```bash
$ docker tag docker.io/library/busybox docker.io/private-repo/mybusybox
$ docker push docker.io/private-repo/mybusybox
```

will result in error, because the neither we nor the target repository
doesn't have the manifests that the busybox manifest list references
(because manifests can't be cross-repo mounted).

If for some reason the manifests and configs for all other platforms
would be present in the content store, but only layer blobs were
missing, then the push would work, because the blobs can be cross-repo
mounted (only if we push to the same registry).

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-03-30 19:53:02 +02:00
Paweł Gronowski
e7393d0fc5
images/push: Accept reference
Push the reference parsing from repo and tag names into the api and pass
a reference object to the ImageService.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-03-30 18:08:40 +02:00