Commit graph

45119 commits

Author SHA1 Message Date
Sebastiaan van Stijn
6a90113e68
cli: move cobra customizations into cmd/dockerd
The cli package is a leftover from when the "docker" and "dockerd" cli
were both maintained in this repository; the only consumer of this is
now the dockerd CLI, so we can move this code (it should not be imported
by anyone).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-14 11:37:51 +01:00
Sebastiaan van Stijn
3d58b43efe
cli: remove unused Errors type
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-14 11:30:54 +01:00
Brian Goff
44a4ffd96f
Merge pull request #44629 from corhere/fix-44512
Prevent containers from being included in List API before they are registered
2022-12-13 14:57:30 -08:00
Bjorn Neergaard
186d3ba70e
Merge pull request #44503 from corhere/image-ls-before-after-filters
List images with multiple since/before filters
2022-12-13 14:55:07 -07:00
Sebastiaan van Stijn
a4cbaa958d
Merge pull request #44497 from neersighted/no_uncommitted_go_mod
hack: replace go-mod-prepare.sh with wrapper script
2022-12-13 17:09:50 +01:00
Bjorn Neergaard
25c3421802 hack: introduce validate/no-module
Moby is not a Go module; to prevent anyone from mistakenly trying to
convert it to one before we are ready, introduce a check (usable in CI
and locally) for a go.mod file.

This is preferable to trying to .gitignore the file as we can ensure
that a mistakenly created go.mod is surfaced by Git-based tooling and is
less likely to surprise a contributor.

Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
2022-12-12 18:39:06 -07:00
Bjorn Neergaard
a449f77774 hack: replace go-mod-prepare.sh with wrapper script
To make the local build environment more correct and consistent, we
should never leave an uncommitted go.mod in the tree; however, we need a
go.mod for certain commands to work properly. Use a wrapper script to
create and destroy the go.mod as needed instead of potentially changing
tooling behavior by leaving it.

If a go.mod already exists, this script will warn and call the wrapped
command with GO111MODULE=on.

Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
2022-12-12 18:39:06 -07:00
Cory Snider
388fe4aea8 daemon: drop side effect from registerLinks()
(*Daemon).registerLinks() calling the WriteHostConfig() method of its
container argument is a vestigial behaviour. In the distant past,
registerLinks() would persist the container links in an SQLite database
and drop the link config from the container's persisted HostConfig. This
changed in Docker v1.10 (#16032) which migrated away from SQLite and
began using the link config in the container's HostConfig as the
persistent source of truth. registerLinks() no longer mutates the
HostConfig at all so persisting the HostConfig to disk falls outside of
its scope of responsibilities.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-12-12 16:04:09 -05:00
Cory Snider
0141c6db81 daemon: don't checkpoint container until registered
(*Container).CheckpointTo() upserts a snapshot of the container to the
daemon's in-memory ViewDB and also persists the snapshot to disk. It
does not register the live container object with the daemon's container
store, however. The ViewDB and container store are used as the source of
truth for different operations, so having a container registered in one
but not the other can result in inconsistencies. In particular, the List
Containers API uses the ViewDB as its source of truth and the Container
Inspect API uses the container store.

The (*Daemon).setHostConfig() method is called fairly early in the
process of creating a container, long before the container is registered
in the daemon's container store. Due to a rogue CheckpointTo() call
inside setHostConfig(), there is a window of time where a container can
be included in a List Containers API response but "not exist" according
to the Container Inspect API and similar endpoints which operate on a
particular container. Remove the rogue call so that the caller has full
control over when the container is checkpointed and update callers to
checkpoint explicitly. No changes to (*Daemon).create() are needed as it
checkpoints the fully-created container via (*Daemon).Register().

Fixes #44512.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-12-12 15:53:49 -05:00
Cory Snider
00157a42d3 daemon: fix GetContainer() returning (nil, nil)
GetContainer() would return (nil, nil) when looking up a container
if the container was inserted into the containersReplica ViewDB but not
the containers Store at the time of the lookup. Callers which reasonably
assume that the returned err == nil implies returned container != nil
would dereference a nil pointer and panic. Change GetContainer() so that
it always returns a container or an error.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-12-12 14:39:10 -05:00
Sebastiaan van Stijn
487fa4a469
Merge pull request #44622 from thaJeztah/image_inspect_step1a
manage image inspect data in backend
2022-12-12 20:05:46 +01:00
Sebastiaan van Stijn
7b0d2464ff
Merge pull request #44459 from neersighted/validate_always_tidy
validate/vendor: always tidy
2022-12-12 18:49:42 +01:00
Nicolas De Loof
8fb71ce208
manage image inspect data in backend
This allows differentiating how the detailed data is collected between
the containerd-integration code and the existing implementation.

Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-09 19:02:12 +01:00
Sebastiaan van Stijn
a8eb15eafb
ImageService.ImageHistory(): pass context
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-09 19:00:49 +01:00
Sebastiaan van Stijn
77f9c550bd
Merge pull request #44605 from thaJeztah/bump_golang_net
vendor: golang.org/x/net v0.4.0
2022-12-09 00:03:14 +01:00
Sebastiaan van Stijn
3cfd3e626c
Merge pull request #44611 from thaJeztah/remove_stray_import
libcontainerd/remote: remove stray import comment
2022-12-08 23:53:24 +01:00
Cory Snider
0426c76142 List images with multiple since/before filters
The List Images API endpoint has accepted multiple values for the
`since` and `before` filter predicates, but thanks to Go's randomizing
of map iteration order, it would pick an arbitrary image to compare
created timestamps against. In other words, the behaviour was undefined.
Change these filter predicates to have well-defined semantics: the
logical AND of all values for each of the respective predicates. As
timestamps are a totally-ordered relation, this is exactly equivalent to
applying the newest and oldest creation timestamps for the `since` and
`before` predicates, respectively.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-12-08 16:36:57 -05:00
Sebastiaan van Stijn
18ef26761d
Merge pull request #44603 from vvoland/images-errdefs
image/store: Use errdefs for errors
2022-12-08 22:06:13 +01:00
Paweł Gronowski
e214503789
image/store: Use errdefs for errors
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2022-12-08 17:17:07 +01:00
Sebastiaan van Stijn
d9694c27c2
Merge pull request #44606 from thaJeztah/bump_containerd_1.6.12
vendor: github.com/containerd/containerd v1.6.12
2022-12-08 14:07:52 +01:00
Sebastiaan van Stijn
200edf8030
libcontainerd/remote: remove stray import comment
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-08 13:27:50 +01:00
Sebastiaan van Stijn
12c76d4a15
vendor: github.com/containerd/containerd v1.6.12
no changes in vendored code, but containerd v1.6.12 is a security release,
so updating, to prevent scanners marking the dependency to have a vulnerability.

full diff: https://github.com/containerd/containerd/compare/v1.6.11...v1.6.12

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-08 00:42:46 +01:00
Sebastiaan van Stijn
4bbc37687e
vendor: golang.org/x/net v0.4.0
golang.org/x/net contains a fix for CVE-2022-41717, which was addressed
in stdlib in go1.19.4 and go1.18.9;

> net/http: limit canonical header cache by bytes, not entries
>
> An attacker can cause excessive memory growth in a Go server accepting
> HTTP/2 requests.
>
> HTTP/2 server connections contain a cache of HTTP header keys sent by
> the client. While the total number of entries in this cache is capped,
> an attacker sending very large keys can cause the server to allocate
> approximately 64 MiB per open connection.
>
> This issue is also fixed in golang.org/x/net/http2 v0.4.0,
> for users manually configuring HTTP/2.

full diff: https://github.com/golang/net/compare/v0.2.0...v0.4.0

other dependency updates (due to circular dependencies):

- golang.org/x/sys v0.3.0: https://github.com/golang/sys/compare/v0.2.0...v0.3.0
- golang.org/x/text v0.5.0: https://github.com/golang/text/compare/v0.4.0...v0.5.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-07 22:25:49 +01:00
Sebastiaan van Stijn
0ff5ce825c
Merge pull request #44594 from thaJeztah/vendor_containerd_1.6.11
vendor: github.com/containerd/containerd v1.6.11
2022-12-07 13:25:18 +01:00
Sebastiaan van Stijn
84f64925bc
Merge pull request #44591 from thaJeztah/update_go_1.19.4
update to go1.19.4
2022-12-07 13:20:41 +01:00
Sebastiaan van Stijn
d331bc3b03
vendor: github.com/containerd/containerd v1.6.11
- Fix nil pointer deference for Windows containers in CRI plugin
- Fix lease labels unexpectedly overwriting expiration
- Fix for simultaneous diff creation using the same parent snapshot

full diff: https://github.com/containerd/containerd/v1.6.10...v1.6.11

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-06 23:26:25 +01:00
Sebastiaan van Stijn
52bc1ad744
update to go1.19.4
Includes security fixes for net/http (CVE-2022-41717, CVE-2022-41720),
and os (CVE-2022-41720).

These minor releases include 2 security fixes following the security policy:

- os, net/http: avoid escapes from os.DirFS and http.Dir on Windows

  The os.DirFS function and http.Dir type provide access to a tree of files
  rooted at a given directory. These functions permitted access to Windows
  device files under that root. For example, os.DirFS("C:/tmp").Open("COM1")
  would open the COM1 device.
  Both os.DirFS and http.Dir only provide read-only filesystem access.

  In addition, on Windows, an os.DirFS for the directory \(the root of the
  current drive) can permit a maliciously crafted path to escape from the
  drive and access any path on the system.

  The behavior of os.DirFS("") has changed. Previously, an empty root was
  treated equivalently to "/", so os.DirFS("").Open("tmp") would open the
  path "/tmp". This now returns an error.

  This is CVE-2022-41720 and Go issue https://go.dev/issue/56694.

- net/http: limit canonical header cache by bytes, not entries

  An attacker can cause excessive memory growth in a Go server accepting
  HTTP/2 requests.

  HTTP/2 server connections contain a cache of HTTP header keys sent by
  the client. While the total number of entries in this cache is capped,
  an attacker sending very large keys can cause the server to allocate
  approximately 64 MiB per open connection.

  This issue is also fixed in golang.org/x/net/http2 vX.Y.Z, for users
  manually configuring HTTP/2.

  Thanks to Josselin Costanzi for reporting this issue.

  This is CVE-2022-41717 and Go issue https://go.dev/issue/56350.

View the release notes for more information:
https://go.dev/doc/devel/release#go1.19.4

And the milestone on the issue tracker:
https://github.com/golang/go/issues?q=milestone%3AGo1.19.4+label%3ACherryPickApproved

Full diff: https://github.com/golang/go/compare/go1.19.3...go1.19.4

The golang.org/x/net fix is in 1e63c2f08a

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-06 22:57:25 +01:00
Sebastiaan van Stijn
0200623ef7
Merge pull request #44587 from thaJeztah/bjorn_maintainer
Add Bjorn Neergaard to maintainers
2022-12-06 21:26:32 +01:00
Sebastiaan van Stijn
f577f31f50
Add Bjorn Neergaard to maintainers
I nominated Bjorn as maintainer, and votes passed; welcome Bjorn!

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-06 21:26:03 +01:00
Sebastiaan van Stijn
a7b2b29b79
Merge pull request #44590 from ndeloof/master
Better not pretend to be an active currator
2022-12-06 19:53:24 +01:00
Nicolas De Loof
9da935629e
Better not pretend to be an active currator
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2022-12-06 17:18:40 +01:00
Sebastiaan van Stijn
42053a0fc3
Merge pull request #44588 from thaJeztah/kevin_curator
Add Kevin Alvarez ("crazy-max") to curators
2022-12-06 17:14:57 +01:00
Sebastiaan van Stijn
383e23fd61
Add Kevin Alvarez ("crazy-max") to curators
Kevin has been doing a lot of work helping modernize our CI, and helping
with BuildKit-related changes. Being our "resident GitHub actions expert",
I think it make sense to add him to the maintainers list as a curator (for
starters), to grant him permissions on the repository.

Perhaps we should nominate him to become a maintainer (with a focus on his
area of expertise) :)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-06 15:28:03 +01:00
Sebastiaan van Stijn
cc1884dc04
Merge pull request #44501 from tiborvass/immutable_radix
libnetwork: use go-immutable-radix instead of radix
2022-12-06 12:46:53 +01:00
Brian Goff
1523c78c34
Merge pull request #44458 from AdamKorcz/fuzzmigration
testing: move fuzzers over from OSS-Fuzz
2022-12-05 09:41:46 -08:00
Sebastiaan van Stijn
683b076a6e
Merge pull request #44537 from crazy-max/builder-host-gateway
builder: handle host-gateway with extra hosts
2022-12-03 14:38:31 +01:00
Brian Goff
676250adf9
Merge pull request #44567 from thaJeztah/relax_checkSupportedMediaType
distribution: checkSupportedMediaType: allow additional media-types
2022-12-02 15:22:54 -08:00
CrazyMax
521b8c02cc
builder: handle host-gateway with extra hosts
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-12-02 23:17:47 +01:00
Sebastiaan van Stijn
92e954a2f0
Merge pull request #44571 from crazy-max/ci-buildkit-temp-fix
ci(buildkit): update buildkit ref to fix issue with alpine image
2022-12-02 21:24:23 +01:00
Kevin Alvarez
381fa4afca
ci(buildkit): update buildkit ref to fix issue with alpine image
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-12-02 18:45:53 +01:00
Sebastiaan van Stijn
a6a539497a
distribution: checkSupportedMediaType: allow additional media-types
This addresses a regression introduced in 407e3a4552,
which turned out to be "too strict", as there's old images that use, for example;

    docker pull python:3.5.1-alpine
    3.5.1-alpine: Pulling from library/python
    unsupported media type application/octet-stream

Before 407e3a4552, such mediatypes were accepted;

    docker pull python:3.5.1-alpine
    3.5.1-alpine: Pulling from library/python
    e110a4a17941: Pull complete
    30dac23631f0: Pull complete
    202fc3980a36: Pull complete
    Digest: sha256:f88925c97b9709dd6da0cb2f811726da9d724464e9be17a964c70f067d2aa64a
    Status: Downloaded newer image for python:3.5.1-alpine
    docker.io/library/python:3.5.1-alpine

This patch copies the additional media-types, using the list of types that
were added in a215e15cb1, which fixed a
similar issue.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-02 01:26:06 +01:00
Tianon Gravi
cd8a090e67
Merge pull request #44329 from thaJeztah/remove_trustkey_id_migration
Remove trustkey id migration and config.TrustKeyPath
2022-12-01 12:49:54 -08:00
Sebastiaan van Stijn
40408d1640
Merge pull request #44562 from thaJeztah/seccomp_block_af_vsock
seccomp: block socket calls to AF_VSOCK in default profile
2022-12-01 21:39:30 +01:00
Sebastiaan van Stijn
5d88037e37
Merge pull request #44531 from thaJeztah/containerd_1.7_binary
update containerd binary to v1.7.0-beta.0
2022-12-01 20:49:32 +01:00
Sebastiaan van Stijn
57b229012a
seccomp: block socket calls to AF_VSOCK in default profile
This syncs the seccomp-profile with the latest changes in containerd's
profile, applying the same changes as 17a9324035

Some background from the associated ticket:

> We want to use vsock for guest-host communication on KubeVirt
> (https://github.com/kubevirt/kubevirt). In KubeVirt we run VMs in pods.
>
> However since anyone can just connect from any pod to any VM with the
> default seccomp settings, we cannot limit connection attempts to our
> privileged node-agent.
>
> ### Describe the solution you'd like
> We want to deny the `socket` syscall for the `AF_VSOCK` family by default.
>
> I see in [1] and [2] that AF_VSOCK was actually already blocked for some
> time, but that got reverted since some architectures support the `socketcall`
> syscall which can't be restricted properly. However we are mostly interested
> in `arm64` and `amd64` where limiting `socket` would probably be enough.
>
> ### Additional context
> I know that in theory we could use our own seccomp profiles, but we would want
> to provide security for as many users as possible which use KubeVirt, and there
> it would be very helpful if this protection could be added by being part of the
> DefaultRuntime profile to easily ensure that it is active for all pods [3].
>
> Impact on existing workloads: It is unlikely that this will disturb any existing
> workload, becuase VSOCK is almost exclusively used for host-guest commmunication.
> However if someone would still use it: Privileged pods would still be able to
> use `socket` for `AF_VSOCK`, custom seccomp policies could be applied too.
> Further it was already blocked for quite some time and the blockade got lifted
> due to reasons not related to AF_VSOCK.
>
> The PR in KubeVirt which adds VSOCK support for additional context: [4]
>
> [1]: https://github.com/moby/moby/pull/29076#commitcomment-21831387
> [2]: dcf2632945
> [3]: https://kubernetes.io/docs/tutorials/security/seccomp/#enable-the-use-of-runtimedefault-as-the-default-seccomp-profile-for-all-workloads
> [4]: https://github.com/kubevirt/kubevirt/pull/8546

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-01 14:06:37 +01:00
Sebastiaan van Stijn
5fd603ce60
Merge pull request #44514 from vvoland/client-sharedsize
client/list: Handle SharedSize
2022-12-01 09:15:59 +01:00
Tibor Vass
eaa74497b8
libnetwork: use go-immutable-radix instead of radix
This commit allows to remove dependency on the mutable version armon/go-radix.

The go-immutable-radix package is better maintained.

It is likely that a bit more memory will be used when using the
immutable version, though discarded nodes are being reused in a pool.
These changes happen when networks are added/removed or nodes come and
go in a cluster, so we are still talking about a relatively low
frequency event.

The major changes compared to the old radix are when modifying (insert
or delete) a tree, and those are pretty self-contained: we replace the
entire immutable tree under a lock.

Signed-off-by: Tibor Vass <teabee89@gmail.com>
2022-11-30 17:03:46 -08:00
AdamKorcz
93fa093122
testing: move fuzzers over from OSS-Fuzz
Signed-off-by: AdamKorcz <adam@adalogics.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-30 17:31:03 +01:00
Sebastiaan van Stijn
36a2dd36a0
Merge pull request #44554 from thaJeztah/remove_outdated_comment
vendor.mod: remove outdated comment about replaced module
2022-11-30 17:26:51 +01:00
Sebastiaan van Stijn
3760b589f6
Merge pull request #44462 from neersighted/generate_authors
AUTHORS: regenerate, cleanup, refactor
2022-11-30 17:25:40 +01:00