Commit graph

48329 commits

Author SHA1 Message Date
Sebastiaan van Stijn
69b7952b8b
Merge pull request #46963 from corhere/libn/resolver-pack-servfail
libnetwork: write ServFail if DNS reply msg is bad
2023-12-19 18:33:27 +01:00
Sebastiaan van Stijn
43a82dab90
Merge pull request #46840 from dmcgowan/c8d-rmi-cleanup
containerd: Image delete fixes and cleanup
2023-12-19 18:07:16 +01:00
Sebastiaan van Stijn
c378b1bf4c
Merge pull request #46966 from vvoland/c8d-dockerpy-skip-commit
c8d/docker-py: Temporarily skip test_commit_*
2023-12-19 17:52:05 +01:00
Cory Snider
5eaf898fcb libnetwork: write ServFail if DNS reply msg is bad
If the resolver's DNSBackend returns a name that cannot be marshaled
into a well-formed DNS message, the resolver will only discover this
when it attempts to write the reply message and it fails with an error.
No reply message is sent, leaving the client to wait out its timeout and
the user in the dark about what went wrong.

When writing the intended reply message fails, retry once with a
ServFail response to inform the client and user that the DNS query was
not resolved due to a problem with to the resolver, not the network.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-12-19 11:24:33 -05:00
Cory Snider
1da85f7bdc libnetwork: assert DNS replies are well-formed
The well-formedness of a DNS message is only checked when it is
serialized, through the (*dns.Msg).Pack() method. Add a call to Pack()
to our tstwriter mock to mirror the behaviour of the real
dns.ResponseWriter implementation. And fix tests which generated
ill-formed DNS query messages.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-12-19 11:13:35 -05:00
Paweł Gronowski
f6a160d34e
c8d/docker-py: Temporarily skip test_commit_*
These tests check deprecated fields. Reenable them once a new docker-py
is released.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-12-19 16:28:20 +01:00
Sebastiaan van Stijn
ae726dcd2b
Merge pull request #46965 from vvoland/c8d-dockerpy-skip-imagesaveload
c8d/docker-py: Temporarily skip test_save_and_load*
2023-12-19 14:58:48 +01:00
Derek McGowan
cf1ea9237c
Add dangling image reference on delete when last image has children
Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-12-19 05:56:43 -08:00
Paweł Gronowski
9603adb5b4
c8d/docker-py: Temporarily skip test_save_and_load*
They fail because exporting an image which targets a manifest list when
only one platform is available exports only the platform-specific
manifest so the ID of the loaded image is different (ID of the platform
manifest, not manifest list).

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-12-19 11:07:24 +01:00
Albin Kerouanton
7a9b680a9c
libnet: remove Endpoint.myAliases
This property is now unused, let's get rid of it.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-12-19 10:20:38 +01:00
Albin Kerouanton
523b907359
daemon: no more IsAnonymousEndpoint
The semantics of an "anonymous" endpoint has always been weird: it was
set on endpoints which name shouldn't be taken into account when
inserting DNS records into libnetwork's `Controller.svcRecords` (and
into the NetworkDB). However, in that case the endpoint's aliases would
still be used to create DNS records; thus, making those "anonymous
endpoints" not so anonymous.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-12-19 10:20:38 +01:00
Albin Kerouanton
8b7af1d0fc
libnet: update dnsNames on ContainerRename
The `(*Endpoint).rename()` method is changed to only mutate `ep.name`
and let a new method `(*Endpoint).UpdateDNSNames()` handle DNS updates.

As a consequence, the rollback code that was part of
`(*Endpoint).rename()` is now removed, and DNS updates are now
rolled back by `ContainerRename`.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-12-19 10:20:38 +01:00
Albin Kerouanton
3bb13c7eb4
libnet: Use Endpoint.dnsNames to create DNS records
Instead of special-casing anonymous endpoints, use the list of DNS names
associated to the endpoint.

`(*Endpoint).isAnonymous()` has no more uses, so let's delete it.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-12-19 10:20:37 +01:00
Sebastiaan van Stijn
581cba2a56
Merge pull request #46964 from thaJeztah/bump_crypto_0.17
vendor: golang.org/x/crypto v0.17.0
2023-12-19 10:17:11 +01:00
Albin Kerouanton
f5cc497eac
libnet: populate Endpoint.dnsNames on UnmarshalJSON
This new property will be empty if the daemon was upgraded with
live-restore enabled. To not break DNS resolutions for restored
containers, we need to populate dnsNames based on endpoint's myAliases &
anonymous properties.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-12-19 10:16:05 +01:00
Albin Kerouanton
ab8968437b
daemon: build the list of endpoint's DNS names
Instead of special-casing anonymous endpoints in libnetwork, let the
daemon specify what (non fully qualified) DNS names should be associated
to container's endpoints.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-12-19 10:16:04 +01:00
Derek McGowan
0c6e9121b2
Add support for removing repo and digest
When repo and digest is provided, remove all references within a
repository for the given digest.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-12-18 16:33:53 -08:00
Sebastiaan van Stijn
c22fc41ddf
vendor: golang.org/x/crypto v0.17.0
update the package, which contains a fix in the ssh package.

full diff: https://github.com/golang/crypto/compare/v0.16.0...v0.17.0

from the security mailing:

> Hello gophers,
>
> Version v0.17.0 of golang.org/x/crypto fixes a protocol weakness in the
> golang.org/x/crypto/ssh package that allowed a MITM attacker to compromise
> the integrity of the secure channel before it was established, allowing
> them to prevent transmission of a number of messages immediately after
> the secure channel was established without either side being aware.
>
> The impact of this attack is relatively limited, as it does not compromise
> confidentiality of the channel. Notably this attack would allow an attacker
> to prevent the transmission of the SSH2_MSG_EXT_INFO message, disabling a
> handful of newer security features.
>
> This protocol weakness was also fixed in OpenSSH 9.6.
>
> Thanks to Fabian Bäumer, Marcus Brinkmann, and Jörg Schwenk from Ruhr
> University Bochum for reporting this issue.
>
> This is CVE-2023-48795 and Go issue https://go.dev/issue/64784.
>
> Cheers,
> Roland on behalf of the Go team

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-19 00:30:17 +01:00
Sebastiaan van Stijn
0751141003
Merge pull request #46920 from dmcgowan/client-hijack-cleanup
Replace use of httputil in client hijack
2023-12-18 23:21:26 +01:00
Derek McGowan
87c87bccb5
Update cleanup logic to use resolve all images
Ensure that when removing an image, an image is checked consistently
against the images with the same target digest. Add unit testing around
delete.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-12-18 14:04:48 -08:00
Derek McGowan
529d19bad8
Add resolve all images and unit test
Add single resolve function to get a consistent list of images matching
the same digest.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-12-18 14:04:46 -08:00
Derek McGowan
eb9ce779f6
Update OTEL in client hijack to use transport
Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-12-18 13:16:37 -08:00
Derek McGowan
c84e889a69
Remove httputil lint exceptions
Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-12-18 13:16:37 -08:00
Derek McGowan
35a51fd697
Update authz plugin test to not use httputil
Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-12-18 13:16:36 -08:00
Derek McGowan
bb4e9c72d7
Replace use of httputil in client hijack
Simplify the hijack process by just performing the http request/response
on the connection and returning the raw conn after success. The client
conn from httputil is deprecated and easily replaced.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-12-18 13:16:36 -08:00
Albin Kerouanton
dc1e73cbbf
libnet: add a new dnsNames property to Endpoint
This new property is meant to replace myAliases and anonymous
properties.

The end goal is to get rid of both properties by letting the daemon
determine what (non fully qualified) DNS names should be associated to
them.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-12-18 18:38:25 +01:00
Albin Kerouanton
1eb0751803
daemon: endpoints on default nw aren't anonymous
They just happen to exist on a network that doesn't support DNS-based
service discovery (ie. no embedded DNS servers are started for them).

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-12-18 18:38:25 +01:00
Paweł Gronowski
74cf9a0f60
Merge pull request #46912 from vvoland/c8d-inspect-parent
c8d/inspect: Fill `Parent` property
2023-12-18 17:47:01 +01:00
Rob Murray
27f3abd893 Allow overlapping change in bridge's IPv6 network.
Calculate the IPv6 addreesses needed on a bridge, then reconcile them
with the addresses on an existing bridge by deleting then adding as
required.

(Previously, required addresses were added one-by-one, then unwanted
addresses were removed. This meant the daemon failed to start if, for
example, an existing bridge had address '2000:db8::/64' and the config
was changed to '2000:db8::/80'.)

IPv6 addresses are now calculated and applied in one go, so there's no
need for setupVerifyAndReconcile() to check the set of IPv6 addresses on
the bridge. And, it was guarded by !config.InhibitIPv4, which can't have
been right. So, removed its IPv6 parts, and added IPv4 to its name.

Link local addresses, the example given in the original ticket, are now
released when containers are stopped. Not releasing them meant that
when using an LL subnet on the default bridge, no container could be
started after a container was stopped (because the calculated address
could not be re-allocated). In non-default bridge networks using an
LL subnet, addresses leaked.

Linux always uses the standard 'fe80::/64' LL network. So, if a bridge
is configured with an LL subnet prefix that overlaps with it, a config
error is reported. Non-overlapping LL subnet prefixes are allowed.

Signed-off-by: Rob Murray <rob.murray@docker.com>
2023-12-18 16:10:41 +00:00
Sebastiaan van Stijn
ba321d318b
Merge pull request #46879 from martinjirku/add_taskstatus_to_api_yml
add TaskStatus to swagger docs
2023-12-18 16:33:07 +01:00
Paweł Gronowski
18b1aec0ee
c8d/inspect: Fill Parent property
Before this change `ParentId` was filled for images when calling the
`/images/json` (image list) endpoint but was not for the
`/images/<image>/json` (image inspect).

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-12-18 16:23:19 +01:00
Sebastiaan van Stijn
ec259ddfee
Merge pull request #46959 from thaJeztah/update_crypto
vendor: golang.org/x/crypto v0.16.0, golang.org/x/sys v0.15.0, golang.org/x/text v0.14.0
2023-12-18 14:36:14 +01:00
Sebastiaan van Stijn
2b0d667b14
Merge pull request #46955 from akerouanton/libnet-remove-unused-cmd
libnet: Remove unused cmd/readme_test
2023-12-18 14:17:07 +01:00
Sebastiaan van Stijn
e48b58e1be
vendor: golang.org/x/crypto v0.16.0
full diff: https://github.com/golang/crypto/compare/v0.14.0...v0.16.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-18 13:39:50 +01:00
Sebastiaan van Stijn
44511f9e43
vendor: golang.org/x/text v0.14.0
full diff: https://github.com/golang/text/compare/v0.13.0...v0.14.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-18 13:38:46 +01:00
Sebastiaan van Stijn
2cdd4906ac
vendor: golang.org/x/sys v0.15.0
full diff: https://github.com/golang/sys/compare/v0.13.0...v0.15.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-18 13:37:57 +01:00
Albin Kerouanton
d6a656cf7f
libnet: Remove unused cmd/readme_test
This command was originally added by ea7f555446
to test the code snippet put into libnet's README.md. Nothing compiles
this file and it doesn't add any value to the project. So better remove
it than maintaining it.

This commit also removes the code snippet from libnet's README.md for
the same reasons.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-12-16 13:06:15 +01:00
Sebastiaan van Stijn
1997933508
Merge pull request #46933 from vvoland/c8d-tag-label-duplicate
c8d/tag: Remove duplicate label copy
2023-12-15 19:10:25 +01:00
Sebastiaan van Stijn
7e1c8a5be3
Merge pull request #46943 from thaJeztah/typo_fixes
typo fixes
2023-12-15 18:39:40 +01:00
Sebastiaan van Stijn
2f80a91b3e
Merge pull request #46936 from crazy-max/fix-dco-on-tag
ci(bin-image): fix conditional run for skipped job
2023-12-15 18:38:53 +01:00
Sebastiaan van Stijn
21334593d1
Merge pull request #46942 from thaJeztah/go_compat
add //go:build directives to prevent downgrading to go1.16 language
2023-12-15 18:37:15 +01:00
Sebastiaan van Stijn
d90e5f283a
Merge pull request #46935 from vvoland/c8d-dockerpy-skip-swuash
c8d/docker-py: Skip test_build_squash
2023-12-15 17:24:32 +01:00
Akihiro Suda
aa5d1a8897
typo fixes
follow-up to 9e6d012e9c

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-15 17:21:34 +01:00
Sebastiaan van Stijn
79c759393f
Merge pull request #46647 from AkihiroSuda/runtime-status
docker info: expose runtime features ("rro" mount mode, etc.), in machine-parsable format
2023-12-15 17:19:37 +01:00
Sebastiaan van Stijn
f58f02f3e6
Merge pull request #46930 from rumpl/devcontainer
Add devcontainer configuration
2023-12-15 17:09:11 +01:00
Sebastiaan van Stijn
2cf230951f
add //go:build directives to prevent downgrading to go1.16 language
This repository is not yet a module (i.e., does not have a `go.mod`). This
is not problematic when building the code in GOPATH or "vendor" mode, but
when using the code as a module-dependency (in module-mode), different semantics
are applied since Go1.21, which switches Go _language versions_ on a per-module,
per-package, or even per-file base.

A condensed summary of that logic [is as follows][1]:

- For modules that have a go.mod containing a go version directive; that
  version is considered a minimum _required_ version (starting with the
  go1.19.13 and go1.20.8 patch releases: before those, it was only a
  recommendation).
- For dependencies that don't have a go.mod (not a module), go language
  version go1.16 is assumed.
- Likewise, for modules that have a go.mod, but the file does not have a
  go version directive, go language version go1.16 is assumed.
- If a go.work file is present, but does not have a go version directive,
  language version go1.17 is assumed.

When switching language versions, Go _downgrades_ the language version,
which means that language features (such as generics, and `any`) are not
available, and compilation fails. For example:

    # github.com/docker/cli/cli/context/store
    /go/pkg/mod/github.com/docker/cli@v25.0.0-beta.2+incompatible/cli/context/store/storeconfig.go:6:24: predeclared any requires go1.18 or later (-lang was set to go1.16; check go.mod)
    /go/pkg/mod/github.com/docker/cli@v25.0.0-beta.2+incompatible/cli/context/store/store.go:74:12: predeclared any requires go1.18 or later (-lang was set to go1.16; check go.mod)

Note that these fallbacks are per-module, per-package, and can even be
per-file, so _(indirect) dependencies_ can still use modern language
features, as long as their respective go.mod has a version specified.

Unfortunately, these failures do not occur when building locally (using
vendor / GOPATH mode), but will affect consumers of the module.

Obviously, this situation is not ideal, and the ultimate solution is to
move to go modules (add a go.mod), but this comes with a non-insignificant
risk in other areas (due to our complex dependency tree).

We can revert to using go1.16 language features only, but this may be
limiting, and may still be problematic when (e.g.) matching signatures
of dependencies.

There is an escape hatch: adding a `//go:build` directive to files that
make use of go language features. From the [go toolchain docs][2]:

> The go line for each module sets the language version the compiler enforces
> when compiling packages in that module. The language version can be changed
> on a per-file basis by using a build constraint.
>
> For example, a module containing code that uses the Go 1.21 language version
> should have a `go.mod` file with a go line such as `go 1.21` or `go 1.21.3`.
> If a specific source file should be compiled only when using a newer Go
> toolchain, adding `//go:build go1.22` to that source file both ensures that
> only Go 1.22 and newer toolchains will compile the file and also changes
> the language version in that file to Go 1.22.

This patch adds `//go:build` directives to those files using recent additions
to the language. It's currently using go1.19 as version to match the version
in our "vendor.mod", but we can consider being more permissive ("any" requires
go1.18 or up), or more "optimistic" (force go1.21, which is the version we
currently use to build).

For completeness sake, note that any file _without_ a `//go:build` directive
will continue to use go1.16 language version when used as a module.

[1]: 58c28ba286/src/cmd/go/internal/gover/version.go (L9-L56)
[2]: https://go.dev/doc/toolchain

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-15 15:24:15 +01:00
Akihiro Suda
9e6d012e9c
docker info: expose runtime features ("rro" mount mode, etc.)
Fix issue 46580

```console
$ curl -s --unix-socket /var/run/docker.sock http://docker/v1.44/info | jq .Runtimes
{
  "crun": {
    "path": "/usr/local/bin/crun",
    "status": {
      "org.opencontainers.runtime-spec.features": "{\"ociVersionMin\":\"1.0.0\",...}"
    }
  },
  "io.containerd.runc.v2": {
    "path": "runc",
    "status": {
      "org.opencontainers.runtime-spec.features": "{\"ociVersionMin\":\"1.0.0\",...}"
    }
  },
  "runc": {
    "path": "runc",
    "status": {
      "org.opencontainers.runtime-spec.features": "{\"ociVersionMin\":\"1.0.0\",...}"
    }
  },
  "runsc": {
    "path": "/usr/local/bin/runsc"
  }
}
```

Co-authored-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2023-12-15 11:48:46 +09:00
Sebastiaan van Stijn
bd70d66a62
Merge pull request #46939 from vvoland/api-deprecate-image-container
api: Deprecate `Container` and `ContainerConfig` for `/images/{id}/json`
2023-12-14 20:54:59 +01:00
Paweł Gronowski
1602e2f4f1
api: Deprecate Container and ContainerConfig for /images/{id}/json
These fields were an implementation detail of the classic image builder
and are empty when using buildkit.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-12-14 17:27:41 +01:00
CrazyMax
d91bf690ef
ci(bin-image): fix conditional run for skipped job
When the doc job is skipped, the dependent ones will be skipped
as well. To fix this issue we need to apply special conditions
to always run dependent jobs but not if canceled or failed.

Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2023-12-14 15:46:52 +01:00