Commit graph

47484 commits

Author SHA1 Message Date
Paweł Gronowski
b805599ef6
libcontainer/windows: Remove unneeded var declaration
The cleanup defer uses an `outErr` now, so we don't need to worry about
shadowing.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-14 11:10:40 +02:00
Paweł Gronowski
55b664046c
libcontainer/windows: Fix process not being killed after stdio attach failure
Error check in defer block used wrong error variable which is always nil
if the flow reaches the defer. This caused the `newProcess.Kill` to be
never called if the subsequent attemp to attach to the stdio failed.
Although this only happens in Exec (as Start does overwrite the error),
this also adjusts the Start to also use the returned error to avoid this
kind of mistake in future changes.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-14 11:10:11 +02:00
Sebastiaan van Stijn
39b2bf51ca
Merge pull request #46406 from akerouanton/issue-46404
daemon: fix under what conditions container's mac-address is applied
2023-09-13 23:35:07 +02:00
Sebastiaan van Stijn
a232f9463c
Merge pull request #46474 from thaJeztah/remove_unused_arg
libnetwork/portmapper: Remove unused arg from New(), and un-export PortMapper.Allocator
2023-09-13 23:32:43 +02:00
Sebastiaan van Stijn
982e5afd1c
Merge pull request #46472 from thaJeztah/dmcg_triage
Add dmcgowan as curator
2023-09-13 20:54:11 +02:00
Sebastiaan van Stijn
9c84994830
libnetwork/portmapper: remove unused PortMapper.checkIP
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-13 18:54:11 +02:00
Sebastiaan van Stijn
f5d6af13d0
libnetwork/portmapper: un-export PortMapper.Allocator
It was only accessed through methods on PortMapper, and in tests.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-13 18:38:53 +02:00
Sebastiaan van Stijn
863909a749
libnetwork/portmapper: New(): remove unused argument
None of the code using this function was setting the value, so let's
simplify and remove the argument.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-13 18:12:53 +02:00
Sebastiaan van Stijn
0a8bd82a37
Merge pull request #46446 from rhansen/host_ipv4
Fix host_ipv4 bridge option when IPv6 and ip6tables are enabled
2023-09-13 18:08:03 +02:00
Sebastiaan van Stijn
5ab8d41d9f
Add dmcgowan as curator
Adding Derek as curator, so that he's able to perform triage tasks
in this repository :)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-13 17:22:24 +02:00
Sebastiaan van Stijn
76915b16e7
Merge pull request #46347 from thaJeztah/libnetwork_early_returns
libnetwork: Network: add some early returns
2023-09-13 12:36:34 +02:00
Sebastiaan van Stijn
3b04fd10e8
Merge pull request #46251 from akerouanton/libnet-forbid-duplicated-network-names
libnet: Make sure network names are unique
2023-09-12 16:53:14 +02:00
Sebastiaan van Stijn
9641c90eaf
Merge pull request #46458 from vvoland/vendor-resenje-singleflight-4.0.0
vendor: resenje.org/singleflight v0.4.0
2023-09-12 14:40:05 +02:00
Sebastiaan van Stijn
7fbc3a9bc0
Merge pull request #46453 from thaJeztah/update_golang_1.20.8
update to go1.20.8
2023-09-12 10:43:25 +02:00
Sebastiaan van Stijn
ff85c363a8
Merge pull request #46440 from thaJeztah/update_cdi
vendor: github.com/cncf-tags/container-device-interface v0.6.1
2023-09-12 10:43:02 +02:00
Sebastiaan van Stijn
0b69c2c42f
Merge pull request #46443 from thaJeztah/mod_tidy
go mod tidy
2023-09-12 10:42:40 +02:00
Albin Kerouanton
78479b1915
libnet: Make sure network names are unique
Fixes #18864, #20648, #33561, #40901.

[This GH comment][1] makes clear network name uniqueness has never been
enforced due to the eventually consistent nature of Classic Swarm
datastores:

> there is no guaranteed way to check for duplicates across a cluster of
> docker hosts.

And this is further confirmed by other comments made by @mrjana in that
same issue, eg. [this one][2]:

> we want to adopt a schema which can pave the way in the future for a
> completely decentralized cluster of docker hosts (if scalability is
> needed).

This decentralized model is what Classic Swarm was trying to be. It's
been superseded since then by Docker Swarm, which has a centralized
control plane.

To circumvent this drawback, the `NetworkCreate` endpoint accepts a
`CheckDuplicate` flag. However it's not perfectly reliable as it won't
catch concurrent requests.

Due to this design decision, API clients like Compose have to implement
workarounds to make sure names are really unique (eg.
docker/compose#9585). And the daemon itself has seen a string of issues
due to that decision, including some that aren't fixed to this day (for
instance moby/moby#40901):

> The problem is, that if you specify a network for a container using
> the ID, it will add that network to the container but it will then
> change it to reference the network by using the name.

To summarize, this "feature" is broken, has no practical use and is a
source of pain for Docker users and API consumers. So let's just remove
it for _all_ API versions.

[1]: https://github.com/moby/moby/issues/18864#issuecomment-167201414
[2]: https://github.com/moby/moby/issues/18864#issuecomment-167202589

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-09-12 10:40:13 +02:00
Sebastiaan van Stijn
ff328e748d
Merge pull request #46397 from thaJeztah/fix_neighbor_delete
libnetwork/osl: Namespace.DeleteNeighbor: assorted cleanups, and ignore "non-exist" warnings
2023-09-12 10:38:43 +02:00
Sebastiaan van Stijn
4f28802f09
Merge pull request #46413 from thaJeztah/builder_diffid_type
daemon/containerd: use "DiffID" type for diff fields
2023-09-12 10:08:46 +02:00
Sebastiaan van Stijn
75308e471c
Merge pull request #46411 from thaJeztah/simplify_scratch
daemon: GetImageAndReleasableLayer: simplify "FROM scratch" case
2023-09-12 09:36:38 +02:00
Paweł Gronowski
5fa011dc0c
vendor: resenje.org/singleflight v0.4.0
Fixes the context aware singleflight not preserving context values.

full diff: https://github.com/janos/singleflight/compare/v0.3.0...v0.4.0

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-12 09:34:03 +02:00
Sebastiaan van Stijn
fd15ddbc30
daemon: GetImageAndReleasableLayer: simplify "FROM scratch" case
Windows doesn't support "FROM scratch", and the platform was only used
for validation on other platforms if a platform was provided, so no need
to set defaults.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-11 23:18:13 +02:00
Sebastiaan van Stijn
4229032676
daemon/containerd: use "DiffID" type for diff fields
strong-type the fields with the expected type, to make it more explicit
what we're expecting here.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-11 23:15:04 +02:00
Sebastiaan van Stijn
cdb5947316
Merge pull request #46455 from akerouanton/enable-ipv6-automatically
daemon: automatically set network EnableIPv6 if needed
2023-09-11 22:18:22 +02:00
Albin Kerouanton
5d5eeac310
daemon: automatically set network EnableIPv6 if needed
PR 4f47013feb added a validation step to `NetworkCreate` to ensure
no IPv6 subnet could be set on a network if its `EnableIPv6` parameter
is false.

Before that, the daemon was accepting such request but was doing nothing
with the IPv6 subnet.

This validation step is now deleted, and we automatically set
`EnableIPv6` if an IPv6 subnet was specified.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-09-11 20:53:29 +02:00
Sebastiaan van Stijn
51d647122a
Merge pull request #46383 from vvoland/c8d-legacybuilder-fix-layer-parent-snapshot
c8d/legacybuilder: Assorted fixes
2023-09-11 20:41:59 +02:00
Djordje Lukic
29dcf646a3
Merge pull request #46454 from rumpl/c8d-disable-schema1 2023-09-11 19:55:43 +02:00
Djordje Lukic
d0d3ddd045
Merge pull request #46375 from rumpl/c8d-userns-remap
c8d: Handle userns properly
2023-09-11 19:04:35 +02:00
Djordje Lukic
0313544f4a
c8d: Handle userns properly
If the daemon is run with --userns-remap we need to chown the prepared
snapshot

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-09-11 16:39:29 +02:00
Paweł Gronowski
c63a952dc9
c8d/builder: Don't append empty tar layer to manifest
To match the number of layers in config created in
`images.CreateChildImage`.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-11 16:22:44 +02:00
Paweł Gronowski
9127285985
c8d/builder: Lease layer snapshots
Create a lease for the snapshot and hold it until the layer is released.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-11 16:22:39 +02:00
Paweł Gronowski
f22b112005
c8d/commit: Unpack committed image
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-11 16:20:14 +02:00
Paweł Gronowski
8832cdf6e1
c8d: Common unpack code for specific manifest
Use `ImageService.unpackImage` when we want to unpack an image and we
know the exact platform-manifest to be unpacked beforehand.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-11 16:20:10 +02:00
Paweł Gronowski
c6bded3475
c8d/builder: Don't use diffID as snapshot parent
DiffID is only a digest of the one tar layer and matches the snapshot ID
only for the first layer (DiffID = ChainID).

Instead of generating random ID as a key for rolayer, just use the
snapshot ID of the unpacked image content and use it later as a parent
for creating a new RWLayer.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-11 16:20:04 +02:00
Paweł Gronowski
8c7e19c5ff
c8d/builder: Set empty diffID for rolayer
diffID is the digest of a tar archive containing changes to the parent
layer - rolayer doesn't have any changes to the parent.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-11 16:20:02 +02:00
Paweł Gronowski
6ebfa57364
c8d/builder: Don't mount the rolayer snapshot
The view snapshot and its mounts are not used.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-11 16:19:59 +02:00
Sebastiaan van Stijn
c41121cc48
update to go1.20.8
go1.20.8 (released 2023-09-06) includes two security fixes to the html/template
package, as well as bug fixes to the compiler, the go command, the runtime,
and the crypto/tls, go/types, net/http, and path/filepath packages. See the
Go 1.20.8 milestone on our issue tracker for details:

https://github.com/golang/go/issues?q=milestone%3AGo1.20.8+label%3ACherryPickApproved

full diff: https://github.com/golang/go/compare/go1.20.7...go1.20.8

From the security mailing:

[security] Go 1.21.1 and Go 1.20.8 are released

Hello gophers,

We have just released Go versions 1.21.1 and 1.20.8, minor point releases.

These minor releases include 4 security fixes following the security policy:

- cmd/go: go.mod toolchain directive allows arbitrary execution
  The go.mod toolchain directive, introduced in Go 1.21, could be leveraged to
  execute scripts and binaries relative to the root of the module when the "go"
  command was executed within the module. This applies to modules downloaded using
  the "go" command from the module proxy, as well as modules downloaded directly
  using VCS software.

  Thanks to Juho Nurminen of Mattermost for reporting this issue.

  This is CVE-2023-39320 and Go issue https://go.dev/issue/62198.

- html/template: improper handling of HTML-like comments within script contexts
  The html/template package did not properly handle HMTL-like "<!--" and "-->"
  comment tokens, nor hashbang "#!" comment tokens, in <script> contexts. This may
  cause the template parser to improperly interpret the contents of <script>
  contexts, causing actions to be improperly escaped. This could be leveraged to
  perform an XSS attack.

  Thanks to Takeshi Kaneko (GMO Cybersecurity by Ierae, Inc.) for reporting this
  issue.

  This is CVE-2023-39318 and Go issue https://go.dev/issue/62196.

- html/template: improper handling of special tags within script contexts
  The html/template package did not apply the proper rules for handling occurrences
  of "<script", "<!--", and "</script" within JS literals in <script> contexts.
  This may cause the template parser to improperly consider script contexts to be
  terminated early, causing actions to be improperly escaped. This could be
  leveraged to perform an XSS attack.

  Thanks to Takeshi Kaneko (GMO Cybersecurity by Ierae, Inc.) for reporting this
  issue.

  This is CVE-2023-39319 and Go issue https://go.dev/issue/62197.

- crypto/tls: panic when processing post-handshake message on QUIC connections
  Processing an incomplete post-handshake message for a QUIC connection caused a panic.

  Thanks to Marten Seemann for reporting this issue.

  This is CVE-2023-39321 and CVE-2023-39322 and Go issue https://go.dev/issue/62266.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-11 15:47:00 +02:00
Djordje Lukic
4acbf7d8c3
c8d: disable schema1 registry integration tests
schema1 was deprecated a while ago, containerd fails to push to a
schema1 registry, let's just skip these tests for the containerd
integration

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-09-11 15:30:44 +02:00
Sebastiaan van Stijn
ccae064209
Merge pull request #46423 from vvoland/c8d-images-all
c8d/builder: Store parent in c8d image label
2023-09-11 14:51:20 +02:00
Djordje Lukic
833b514d98
c8d/mount: Create same directory owners as graphdrivers
Graph drivers create the parent directory with
rootPair().GID:CurrentIdentity().UID owner. This change brings these in
line

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-09-11 13:24:42 +02:00
Sebastiaan van Stijn
f8a4b6755d
Merge pull request #46442 from thaJeztah/less_polling
integration: don't poll for containers to be running
2023-09-11 13:15:38 +02:00
Richard Hansen
12e27dfd8f Fix host_ipv4 bridge option when IPv6 and ip6tables are enabled
Before this commit, setting the `com.docker.network.host_ipv4` bridge
option when `enable_ipv6` is true and the experimental `ip6tables`
option is enabled would cause Docker to fail to create the network:

> failed to create network `test-network`: Error response from daemon:
> Failed to Setup IP tables: Unable to enable NAT rule: (iptables
> failed: `ip6tables --wait -t nat -I POSTROUTING -s fd01::/64 ! -o
> br-test -j SNAT --to-source 192.168.0.2`: ip6tables
> v1.8.7 (nf_tables): Bad IP address "192.168.0.2"
>
> Try `ip6tables -h` or `ip6tables --help` for more information.
>  (exit status 2))

Fix this error by passing nil -- not the `host_ipv4` address -- when
creating the IPv6 rules.

Signed-off-by: Richard Hansen <rhansen@rhansen.org>
2023-09-10 04:03:07 -04:00
Sebastiaan van Stijn
7c0f6741eb
go mod tidy
Looks like some dependencies ended up on in the wrong list.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-09 00:53:46 +02:00
Sebastiaan van Stijn
cd204f1118
libnetwork/osl: Namespace.DeleteNeighbor: remove intermediate vars
- store linkIndex in a local variable so that it can be reused
- remove / rename some intermediate vars that shadowed existing declaration

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-08 23:46:14 +02:00
Sebastiaan van Stijn
4215a1542b
libnetwork/osl: Namespace.DeleteNeighbor: remove osDelete argument
This argument was originally added in libnetwork:
03f440667f

At the time, this argument was conditional, but currently it's always set
to "true", so let's remove it.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-08 23:46:14 +02:00
Sebastiaan van Stijn
bc77104fed
libnetwork/osl: Namespace.DeleteNeighbor: don't warn on non-existing neighbor
The code ignores these errors, but will unconditionally print a warning;

> If the kernel deletion fails for the neighbor entry still remote it
> from the namespace cache. Otherwise if the neighbor moves back to the
> same host again, kernel update can fail.

Let's reduce noise if the neighbor wasn't found, to prevent logs like:

    Aug 16 13:26:35 master1.local dockerd[4019880]: time="2023-08-16T13:26:35.186662370+02:00" level=warning msg="error while deleting neighbor entry" error="no such file or directory"
    Aug 16 13:26:35 master1.local dockerd[4019880]: time="2023-08-16T13:26:35.366585939+02:00" level=warning msg="error while deleting neighbor entry" error="no such file or directory"
    Aug 16 13:26:42 master1.local dockerd[4019880]: time="2023-08-16T13:26:42.366658513+02:00" level=warning msg="error while deleting neighbor entry" error="no such file or directory"

While changing this code, also slightly rephrase the code-comment, and
fix a typo ("remote -> remove").

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

libnetwork/osl: Namespace.DeleteNeighbor: rephrase code-comment

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-08 23:46:11 +02:00
Sebastiaan van Stijn
6ce5aa1cd5
Merge pull request #46434 from vvoland/test-ignore-cgroup-check
hack: Add TEST_IGNORE_CGROUP_CHECK
2023-09-08 23:23:18 +02:00
Sebastiaan van Stijn
c3ab3f1dde
Merge pull request #46412 from rumpl/c8d-pull-progress
c8d: Better pull progress
2023-09-08 23:10:23 +02:00
Sebastiaan van Stijn
d582a26107
Merge pull request #46439 from akerouanton/no-optimizations
hack: compile w/o optimizations & inlining when debugging
2023-09-08 23:09:33 +02:00
Sebastiaan van Stijn
5e7eade1f7
integration: don't poll for containers to be running
container.Run() should be a synchronous operation in normal circumstances;
the container is created and started, so polling after that for the
container to be in the "running" state should not be needed.

This should also prevent issues when a container (for whatever reason)
exited immediately after starting; in that case we would continue
polling for it to be running (which likely would never happen).

Let's skip the polling; if the container is not in the expected state
(i.e. exited), tests should fail as well.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-08 23:05:43 +02:00