Commit graph

48249 commits

Author SHA1 Message Date
Sebastiaan van Stijn
6fae583dba
pkg/aaparser: remove, and integrate into profiles/apparmor
This package provided utilities to obtain the apparmor_parser version, as well
as loading a profile.

Commit e3e715666f (included in v24.0.0 through
bfffb0974e) deprecated GetVersion, as it was no
longer used, which made LoadProfile the only utility remaining in this package.

LoadProfile appears to have no external consumers, and the only use in our code
is "profiles/apparmor".

This patch moves the remaining code (LoadProfile) to profiles/apparmor as a
non-exported function, and deletes the package.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-02 15:15:49 +01:00
Sebastiaan van Stijn
53d405c37f
pkg/aaparser: remove deprecated GetVersion utility
Commit e3e715666f (included in v24.0.0 through
bfffb0974e) deprecated GetVersion, as it was no
longer used.

This patch removes the deprecated utility, and inlines the remaining code into
the LoadProfile function.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-02 14:10:37 +01:00
Sebastiaan van Stijn
a5b4670c79
Merge pull request #47001 from thaJeztah/portmapper_fix_release
portmapper: fix defers to prevent potentially unreleased ports
2024-01-02 10:58:24 +01:00
Sebastiaan van Stijn
abdaf7c2b5
Merge pull request #47000 from thaJeztah/userland_proxy_validation
portmapper: move userland-proxy lookup to daemon config
2024-01-02 10:55:13 +01:00
Sebastiaan van Stijn
4f9db655ed
portmapper: move userland-proxy lookup to daemon config
When mapping a port with the userland-proxy enabled, the daemon would
perform an "exec.LookPath" for every mapped port (which, in case of
a range of ports, would be for every port in the range).

This was both inefficient (looking up the binary for each port), inconsistent
(when running in rootless-mode, the binary was looked-up once), as well as
inconvenient, because a missing binary, or a mis-configureed userland-proxy-path
would not be detected daeemon startup, and not produce an error until starting
the container;

    docker run -d -P nginx:alpine
    4f7b6589a1680f883d98d03db12203973387f9061e7a963331776170e4414194
    docker: Error response from daemon: driver failed programming external connectivity on endpoint romantic_wiles (7cfdc361821f75cbc665564cf49856cf216a5b09046d3c22d5b9988836ee088d): fork/exec docker-proxy: no such file or directory.

However, the container would still be created (but invalid);

    docker ps -a
    CONTAINER ID   IMAGE          COMMAND                  CREATED          STATUS    PORTS     NAMES
    869f41d7e94f   nginx:alpine   "/docker-entrypoint.…"   10 seconds ago   Created             romantic_wiles

This patch changes how the userland-proxy is configured;

- The path of the userland-proxy is now looked up / configured at daemon
  startup; this is similar to how the proxy is configured in rootless-mode.
- A warning is logged when failing to lookup the binary.
- If the daemon is configured with "userland-proxy" enabled, an error is
  produced, and the daemon will refuse to start.
- The "proxyPath" argument for newProxyCommand() (in libnetwork/portmapper)
  is now required to be set. It no longer looks up the executable, and
  produces an error if no path was provided. While this change was not
  required, it makes the daemon config the canonical source of truth, instead
  of logic spread accross multiplee locations.

Some of this logic is a change of behavior, but these changes were made with
the assumption that we don't want to support;

- installing the userland proxy _after_ the daemon was started
- moving the userland proxy (or installing a proxy with a higher
  preference in PATH)

With this patch:

Validating the config produces an error if the binary is not found:

    dockerd --validate
    WARN[2023-12-29T11:36:39.748699591Z] failed to lookup default userland-proxy binary       error="exec: \"docker-proxy\": executable file not found in $PATH"
    userland-proxy is enabled, but userland-proxy-path is not set

Disabling userland-proxy prints a warning, but validates as "OK":

    dockerd --userland-proxy=false --validate
    WARN[2023-12-29T11:38:30.752523879Z] ffailed to lookup default userland-proxy binary       error="exec: \"docker-proxy\": executable file not found in $PATH"
    configuration OK

Speficying a non-absolute path produces an error:

    dockerd --userland-proxy-path=docker-proxy --validate
    invalid userland-proxy-path: must be an absolute path: docker-proxy

Befor this patch, we would not validate this path, which would allow the daemon
to start, but fail to map a port;

    docker run -d -P nginx:alpine
    4f7b6589a1680f883d98d03db12203973387f9061e7a963331776170e4414194
    docker: Error response from daemon: driver failed programming external connectivity on endpoint romantic_wiles (7cfdc361821f75cbc665564cf49856cf216a5b09046d3c22d5b9988836ee088d): fork/exec docker-proxy: no such file or directory.

Specifying an invalid userland-proxy-path produces an error as well:

    dockerd --userland-proxy-path=/usr/local/bin/no-such-binary --validate
    userland-proxy-path is invalid: stat /usr/local/bin/no-such-binary: no such file or directory

    mkdir -p /usr/local/bin/not-a-file
    dockerd --userland-proxy-path=/usr/local/bin/not-a-file --validate
    userland-proxy-path is invalid: exec: "/usr/local/bin/not-a-file": is a directory

    touch /usr/local/bin/not-an-executable
    dockerd --userland-proxy-path=/usr/local/bin/not-an-executable --validate
    userland-proxy-path is invalid: exec: "/usr/local/bin/not-an-executable": permission denied

Same when using the daemon.json config-file;

    echo '{"userland-proxy-path":"no-such-binary"}' > /etc/docker/daemon.json
    dockerd --validate
    unable to configure the Docker daemon with file /etc/docker/daemon.json: merged configuration validation from file and command line flags failed: invalid userland-proxy-path: must be an absolute path: no-such-binary

    dockerd --userland-proxy-path=hello --validate
    unable to configure the Docker daemon with file /etc/docker/daemon.json: the following directives are specified both as a flag and in the configuration file: userland-proxy-path: (from flag: hello, from file: /usr/local/bin/docker-proxy)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-29 16:23:18 +01:00
Sebastiaan van Stijn
488f2bd141
Merge pull request #46998 from deciduously/update-manifest-v2-urls
Updated manifest v2 URLs in image spec README
2023-12-29 15:23:11 +01:00
Sebastiaan van Stijn
214ab2caef
libnetwork/portmapper: PortMapper.MapRange: inline "cleanup" closure
The cleanup function never returns an error, so didn't add much value. This
patch removes the closure, and calls it inline to remove the extra
indirection, and removes the error which would never be returned.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-29 14:30:42 +01:00
Sebastiaan van Stijn
6ae6dcfc53
libnetwork/portmapper: PortMapper.MapRange: fix defer
The defer was set after the switch, but various code-paths inside the switch
could return with an error after the port was allocated / reserved, which
could result in those ports not being released.

This patch moves the defer into each individual branch of the switch to set
it immediately after succesfully reserving the port.

We can also remove a redundant ReleasePort from the cleanup function, as
it's only called if an error occurs, and the defers already take care of
that.

Note that the cleanup function was handling errors returned by ReleasePort,
but this function never returns an error, so it was fully redundant.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-29 14:26:56 +01:00
Sebastiaan van Stijn
8712c6df22
libnetwork/portmapper: PortMapper.MapRange: rename err-return
Prevent accidentally shadowing the error, which is used in a defer.
Also re-format the code to make it more clear we're not acting on
a locally-scoped "allocatedHostPort" variable.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-29 14:26:06 +01:00
Ben Lovy
341e7b35a6 Updated manifest v2 URLs in image spec README
Signed-off-by: Ben Lovy <ben@deciduously.com>
2023-12-28 16:13:26 -05:00
Sebastiaan van Stijn
60b7e2bbbe
Merge pull request #46997 from thaJeztah/bump_crun
Dockerfile: update crun to 1.12
2023-12-28 21:06:51 +01:00
Sebastiaan van Stijn
882b79b254
Dockerfile: update crun to 1.12
full diff: https://github.com/containers/crun/compare/1.8.7...1.12

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-28 10:50:28 +01:00
Sebastiaan van Stijn
1ffc120558
Merge pull request #46991 from thaJeztah/bump_compress
vendor: github.com/klauspost/compress v1.17.4
2023-12-28 10:14:35 +01:00
Sebastiaan van Stijn
dc23efd3a2
vendor: github.com/klauspost/compress v1.17.4
full diff: https://github.com/klauspost/compress/compare/v1.17.2...v1.17.4

v1.17.4:

- huff0: Speed up symbol counting
- huff0: Remove byteReader
- gzhttp: Allow overriding decompression on transport
- gzhttp: Clamp compression level
- gzip: Error out if reserved bits are set

v1.17.3:

- fse: Fix max header size
- zstd: Improve better/best compression
- gzhttp: Fix missing content type on Close

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-27 10:23:44 +01:00
Sebastiaan van Stijn
46f7ab808b
Merge pull request #46977 from crazy-max/fix-push-tag
ci(bin-image): fix merge job run condition
2023-12-21 22:35:40 +01:00
CrazyMax
0252a6f475
ci(bin-image): fix merge job run condition
All underlying jobs inherit from the status of all parent jobs
in the tree, not just the very parent. We need to apply the same
kind of special condition.

Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2023-12-21 16:58:36 +01:00
CrazyMax
1ea1d561c7
ci: do not run ci workflow on push tag events
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2023-12-21 16:53:46 +01:00
Sebastiaan van Stijn
7bc56c5365
Merge pull request #46853 from akerouanton/libnet-ep-dns-names
libnet: Endpoint: remove isAnonymous & myAliases
2023-12-20 19:53:16 +01:00
Albin Kerouanton
13915f6521
libnet: document what Network.networkType represents
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-12-20 19:04:37 +01:00
Albin Kerouanton
6a2542dacf
libnet: remove Endpoint.anonymous
No more concept of "anonymous endpoints". The equivalent is now an
endpoint with no DNSNames set.

Some of the code removed by this commit was mutating user-supplied
endpoint's Aliases to add container's short ID to that list. In order to
preserve backward compatibility for the ContainerInspect endpoint, this
commit also takes care of adding that short ID (and the container
hostname) to `EndpointSettings.Aliases` before returning the response.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-12-20 19:04:37 +01:00
Sebastiaan van Stijn
388216fc45
Merge pull request #46850 from robmry/46829-allow_ipv6_subnet_change
Allow overlapping change in bridge's IPv6 network.
2023-12-19 18:35:13 +01:00
Sebastiaan van Stijn
69b7952b8b
Merge pull request #46963 from corhere/libn/resolver-pack-servfail
libnetwork: write ServFail if DNS reply msg is bad
2023-12-19 18:33:27 +01:00
Sebastiaan van Stijn
43a82dab90
Merge pull request #46840 from dmcgowan/c8d-rmi-cleanup
containerd: Image delete fixes and cleanup
2023-12-19 18:07:16 +01:00
Sebastiaan van Stijn
c378b1bf4c
Merge pull request #46966 from vvoland/c8d-dockerpy-skip-commit
c8d/docker-py: Temporarily skip test_commit_*
2023-12-19 17:52:05 +01:00
Cory Snider
5eaf898fcb libnetwork: write ServFail if DNS reply msg is bad
If the resolver's DNSBackend returns a name that cannot be marshaled
into a well-formed DNS message, the resolver will only discover this
when it attempts to write the reply message and it fails with an error.
No reply message is sent, leaving the client to wait out its timeout and
the user in the dark about what went wrong.

When writing the intended reply message fails, retry once with a
ServFail response to inform the client and user that the DNS query was
not resolved due to a problem with to the resolver, not the network.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-12-19 11:24:33 -05:00
Cory Snider
1da85f7bdc libnetwork: assert DNS replies are well-formed
The well-formedness of a DNS message is only checked when it is
serialized, through the (*dns.Msg).Pack() method. Add a call to Pack()
to our tstwriter mock to mirror the behaviour of the real
dns.ResponseWriter implementation. And fix tests which generated
ill-formed DNS query messages.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-12-19 11:13:35 -05:00
Paweł Gronowski
f6a160d34e
c8d/docker-py: Temporarily skip test_commit_*
These tests check deprecated fields. Reenable them once a new docker-py
is released.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-12-19 16:28:20 +01:00
Sebastiaan van Stijn
ae726dcd2b
Merge pull request #46965 from vvoland/c8d-dockerpy-skip-imagesaveload
c8d/docker-py: Temporarily skip test_save_and_load*
2023-12-19 14:58:48 +01:00
Derek McGowan
cf1ea9237c
Add dangling image reference on delete when last image has children
Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-12-19 05:56:43 -08:00
Paweł Gronowski
9603adb5b4
c8d/docker-py: Temporarily skip test_save_and_load*
They fail because exporting an image which targets a manifest list when
only one platform is available exports only the platform-specific
manifest so the ID of the loaded image is different (ID of the platform
manifest, not manifest list).

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-12-19 11:07:24 +01:00
Albin Kerouanton
7a9b680a9c
libnet: remove Endpoint.myAliases
This property is now unused, let's get rid of it.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-12-19 10:20:38 +01:00
Albin Kerouanton
523b907359
daemon: no more IsAnonymousEndpoint
The semantics of an "anonymous" endpoint has always been weird: it was
set on endpoints which name shouldn't be taken into account when
inserting DNS records into libnetwork's `Controller.svcRecords` (and
into the NetworkDB). However, in that case the endpoint's aliases would
still be used to create DNS records; thus, making those "anonymous
endpoints" not so anonymous.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-12-19 10:20:38 +01:00
Albin Kerouanton
8b7af1d0fc
libnet: update dnsNames on ContainerRename
The `(*Endpoint).rename()` method is changed to only mutate `ep.name`
and let a new method `(*Endpoint).UpdateDNSNames()` handle DNS updates.

As a consequence, the rollback code that was part of
`(*Endpoint).rename()` is now removed, and DNS updates are now
rolled back by `ContainerRename`.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-12-19 10:20:38 +01:00
Albin Kerouanton
3bb13c7eb4
libnet: Use Endpoint.dnsNames to create DNS records
Instead of special-casing anonymous endpoints, use the list of DNS names
associated to the endpoint.

`(*Endpoint).isAnonymous()` has no more uses, so let's delete it.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-12-19 10:20:37 +01:00
Sebastiaan van Stijn
581cba2a56
Merge pull request #46964 from thaJeztah/bump_crypto_0.17
vendor: golang.org/x/crypto v0.17.0
2023-12-19 10:17:11 +01:00
Albin Kerouanton
f5cc497eac
libnet: populate Endpoint.dnsNames on UnmarshalJSON
This new property will be empty if the daemon was upgraded with
live-restore enabled. To not break DNS resolutions for restored
containers, we need to populate dnsNames based on endpoint's myAliases &
anonymous properties.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-12-19 10:16:05 +01:00
Albin Kerouanton
ab8968437b
daemon: build the list of endpoint's DNS names
Instead of special-casing anonymous endpoints in libnetwork, let the
daemon specify what (non fully qualified) DNS names should be associated
to container's endpoints.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-12-19 10:16:04 +01:00
Derek McGowan
0c6e9121b2
Add support for removing repo and digest
When repo and digest is provided, remove all references within a
repository for the given digest.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-12-18 16:33:53 -08:00
Sebastiaan van Stijn
c22fc41ddf
vendor: golang.org/x/crypto v0.17.0
update the package, which contains a fix in the ssh package.

full diff: https://github.com/golang/crypto/compare/v0.16.0...v0.17.0

from the security mailing:

> Hello gophers,
>
> Version v0.17.0 of golang.org/x/crypto fixes a protocol weakness in the
> golang.org/x/crypto/ssh package that allowed a MITM attacker to compromise
> the integrity of the secure channel before it was established, allowing
> them to prevent transmission of a number of messages immediately after
> the secure channel was established without either side being aware.
>
> The impact of this attack is relatively limited, as it does not compromise
> confidentiality of the channel. Notably this attack would allow an attacker
> to prevent the transmission of the SSH2_MSG_EXT_INFO message, disabling a
> handful of newer security features.
>
> This protocol weakness was also fixed in OpenSSH 9.6.
>
> Thanks to Fabian Bäumer, Marcus Brinkmann, and Jörg Schwenk from Ruhr
> University Bochum for reporting this issue.
>
> This is CVE-2023-48795 and Go issue https://go.dev/issue/64784.
>
> Cheers,
> Roland on behalf of the Go team

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-19 00:30:17 +01:00
Sebastiaan van Stijn
0751141003
Merge pull request #46920 from dmcgowan/client-hijack-cleanup
Replace use of httputil in client hijack
2023-12-18 23:21:26 +01:00
Derek McGowan
87c87bccb5
Update cleanup logic to use resolve all images
Ensure that when removing an image, an image is checked consistently
against the images with the same target digest. Add unit testing around
delete.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-12-18 14:04:48 -08:00
Derek McGowan
529d19bad8
Add resolve all images and unit test
Add single resolve function to get a consistent list of images matching
the same digest.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-12-18 14:04:46 -08:00
Derek McGowan
eb9ce779f6
Update OTEL in client hijack to use transport
Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-12-18 13:16:37 -08:00
Derek McGowan
c84e889a69
Remove httputil lint exceptions
Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-12-18 13:16:37 -08:00
Derek McGowan
35a51fd697
Update authz plugin test to not use httputil
Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-12-18 13:16:36 -08:00
Derek McGowan
bb4e9c72d7
Replace use of httputil in client hijack
Simplify the hijack process by just performing the http request/response
on the connection and returning the raw conn after success. The client
conn from httputil is deprecated and easily replaced.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-12-18 13:16:36 -08:00
Albin Kerouanton
dc1e73cbbf
libnet: add a new dnsNames property to Endpoint
This new property is meant to replace myAliases and anonymous
properties.

The end goal is to get rid of both properties by letting the daemon
determine what (non fully qualified) DNS names should be associated to
them.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-12-18 18:38:25 +01:00
Albin Kerouanton
1eb0751803
daemon: endpoints on default nw aren't anonymous
They just happen to exist on a network that doesn't support DNS-based
service discovery (ie. no embedded DNS servers are started for them).

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-12-18 18:38:25 +01:00
Paweł Gronowski
74cf9a0f60
Merge pull request #46912 from vvoland/c8d-inspect-parent
c8d/inspect: Fill `Parent` property
2023-12-18 17:47:01 +01:00
Rob Murray
27f3abd893 Allow overlapping change in bridge's IPv6 network.
Calculate the IPv6 addreesses needed on a bridge, then reconcile them
with the addresses on an existing bridge by deleting then adding as
required.

(Previously, required addresses were added one-by-one, then unwanted
addresses were removed. This meant the daemon failed to start if, for
example, an existing bridge had address '2000:db8::/64' and the config
was changed to '2000:db8::/80'.)

IPv6 addresses are now calculated and applied in one go, so there's no
need for setupVerifyAndReconcile() to check the set of IPv6 addresses on
the bridge. And, it was guarded by !config.InhibitIPv4, which can't have
been right. So, removed its IPv6 parts, and added IPv4 to its name.

Link local addresses, the example given in the original ticket, are now
released when containers are stopped. Not releasing them meant that
when using an LL subnet on the default bridge, no container could be
started after a container was stopped (because the calculated address
could not be re-allocated). In non-default bridge networks using an
LL subnet, addresses leaked.

Linux always uses the standard 'fe80::/64' LL network. So, if a bridge
is configured with an LL subnet prefix that overlaps with it, a config
error is reported. Non-overlapping LL subnet prefixes are allowed.

Signed-off-by: Rob Murray <rob.murray@docker.com>
2023-12-18 16:10:41 +00:00