Commit graph

47557 commits

Author SHA1 Message Date
Paweł Gronowski
7f281907dd
c8d/push: Add missing message about repository
Add "The push referers to repository X" message which is present in the
push output when using the graphdrivers.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-15 16:27:45 +02:00
Paweł Gronowski
babf907bfd
c8d/push: Support pushing all tags
Implement missing feature that pushes all tags from the provided local
repository.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-15 16:27:43 +02:00
Albin Kerouanton
7ec9f304e9
daemon/cluster: create managed ctr with multiple EndpointsConfig
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-09-15 14:30:03 +02:00
Albin Kerouanton
5e15ed314b
api: Improve error on ContainerCreate with multiple endpoints
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-09-15 14:30:03 +02:00
Sebastiaan van Stijn
cc9c930e29
vendor: github.com/Microsoft/hcsshim v0.9.10
Add support for platform compatibility check for windows + add windows builds

full diff: https://github.com/Microsoft/hcsshim/compare/v0.9.8...v0.9.10

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-15 12:52:02 +02:00
Sebastiaan van Stijn
24102aa8ca
update containerd binary to v1.7.6
Update the version used in testing;

full diff: https://github.com/containerd/containerd/compare/v1.7.3...v1.7.6

v1.7.6 release notes:

full diff: https://github.com/containerd/containerd/compare/v1.7.5...v1.7.6

The sixth patch release for containerd 1.7 contains various fixes and updates.

- Fix log package for clients overwriting the global logger
- Fix blockfile snapshotter copy on Darwin
- Add support for Linux usernames on non-Linux platforms
- Update Windows platform matcher to invoke stable ABI compability function
- Update Golang to 1.20.8
- Update push to inherit distribution sources from parent

v1.7.5 release notes:

full diff: https://github.com/containerd/containerd/compare/v1.7.4...v1.7.5

The fifth patch release for containerd 1.7 fixes a versioning issue from
the previous release and includes some internal logging API changes.

v1.7.4 release notes:

full diff: https://github.com/containerd/containerd/compare/v1.7.3...v1.7.4

The fourth patch release for containerd 1.7 contains remote differ plugin support,
a new block file based snapshotter, and various fixes and updates.

Notable Updates

- Add blockfile snapshotter
- Add remote/proxy differ
- Update runc binary to v1.1.9
- Cri: Don't use rel path for image volumes
- Allow attaching to any combination of stdin/out/err
- Fix ro mount option being passed
- Fix leaked shim caused by high IO pressure
- Add configurable mount options to overlay snapshotter

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-15 12:21:07 +02:00
Albin Kerouanton
bbcd662532
api: Allow ContainerCreate to take several EndpointsConfig for >= 1.44
The API endpoint `/containers/create` accepts several EndpointsConfig
since v1.22 but the daemon would error out in such case. This check is
moved from the daemon to the api and is now applied only for API < 1.44,
effectively allowing the daemon to create containers connected to
several networks.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-09-15 10:07:29 +02:00
Sebastiaan van Stijn
20f9635469
Merge pull request #46475 from dmcgowan/c8d-registry-token-support
Add support for registry token in containerd pull logic
2023-09-14 17:18:07 +02:00
Derek McGowan
62e55fd58a
Add support for registry token in containerd pull logic
When registry token is provided, the authorization header can be
directly applied to the registry request. No other type of
authorization will be attempted when the registry token is provided.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-09-14 06:35:23 -07:00
Paweł Gronowski
00e7ef5c3c
Merge pull request #46477 from thaJeztah/freeze_frozen
contrib: outline purpose of download-frozen-images-v2.sh
2023-09-14 14:38:26 +02:00
Sebastiaan van Stijn
01cc1cc923
Merge pull request #46471 from foundriesio/atomic-layer-data-write
daemon: overlay2: Write layer metadata atomically
2023-09-14 12:10:17 +02:00
Sebastiaan van Stijn
14c5f7bf1d
contrib: outline purpose of download-frozen-images-v2.sh
We occassionally receive contributions to this script that are outside
its intended scope. Let's add a comment to the script that outlines
what it's meant for, and a link to a GitHub ticket with alternatives.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-14 11:36:22 +02:00
Paweł Gronowski
0937aef261
libcontainerd/windows: Don't reap on failure
Synchronize the code to do the same thing as Exec.
reap doesn't need to be called before the start event was sent.
There's already a defer block which cleans up the process in case where
an error occurs.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-14 11:11:33 +02:00
Paweł Gronowski
b805599ef6
libcontainer/windows: Remove unneeded var declaration
The cleanup defer uses an `outErr` now, so we don't need to worry about
shadowing.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-14 11:10:40 +02:00
Paweł Gronowski
55b664046c
libcontainer/windows: Fix process not being killed after stdio attach failure
Error check in defer block used wrong error variable which is always nil
if the flow reaches the defer. This caused the `newProcess.Kill` to be
never called if the subsequent attemp to attach to the stdio failed.
Although this only happens in Exec (as Start does overwrite the error),
this also adjusts the Start to also use the returned error to avoid this
kind of mistake in future changes.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-14 11:10:11 +02:00
Sebastiaan van Stijn
39b2bf51ca
Merge pull request #46406 from akerouanton/issue-46404
daemon: fix under what conditions container's mac-address is applied
2023-09-13 23:35:07 +02:00
Sebastiaan van Stijn
a232f9463c
Merge pull request #46474 from thaJeztah/remove_unused_arg
libnetwork/portmapper: Remove unused arg from New(), and un-export PortMapper.Allocator
2023-09-13 23:32:43 +02:00
Sebastiaan van Stijn
982e5afd1c
Merge pull request #46472 from thaJeztah/dmcg_triage
Add dmcgowan as curator
2023-09-13 20:54:11 +02:00
Sebastiaan van Stijn
9c84994830
libnetwork/portmapper: remove unused PortMapper.checkIP
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-13 18:54:11 +02:00
Sebastiaan van Stijn
f5d6af13d0
libnetwork/portmapper: un-export PortMapper.Allocator
It was only accessed through methods on PortMapper, and in tests.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-13 18:38:53 +02:00
Sebastiaan van Stijn
863909a749
libnetwork/portmapper: New(): remove unused argument
None of the code using this function was setting the value, so let's
simplify and remove the argument.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-13 18:12:53 +02:00
Sebastiaan van Stijn
0a8bd82a37
Merge pull request #46446 from rhansen/host_ipv4
Fix host_ipv4 bridge option when IPv6 and ip6tables are enabled
2023-09-13 18:08:03 +02:00
Sebastiaan van Stijn
5ab8d41d9f
Add dmcgowan as curator
Adding Derek as curator, so that he's able to perform triage tasks
in this repository :)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-13 17:22:24 +02:00
Mike Sul
de2447c2ab
daemon: overlay2: Write layer metadata atomically
When the daemon process or the host running it is abruptly terminated,
the layer metadata file can become inconsistent on the file system.
Specifically, `link` and `lower` files may exist but be empty, leading
to overlay mounting errors during layer extraction, such as:
"failed to register layer: error creating overlay mount to <path>:
too many levels of symbolic links."

This commit introduces the use of `AtomicWriteFile` to ensure that the
layer metadata files contain correct data when they exist on the file system.

Signed-off-by: Mike <mike.sul@foundries.io>
2023-09-13 15:07:32 +02:00
Sebastiaan van Stijn
76915b16e7
Merge pull request #46347 from thaJeztah/libnetwork_early_returns
libnetwork: Network: add some early returns
2023-09-13 12:36:34 +02:00
Sebastiaan van Stijn
3b04fd10e8
Merge pull request #46251 from akerouanton/libnet-forbid-duplicated-network-names
libnet: Make sure network names are unique
2023-09-12 16:53:14 +02:00
Sebastiaan van Stijn
9641c90eaf
Merge pull request #46458 from vvoland/vendor-resenje-singleflight-4.0.0
vendor: resenje.org/singleflight v0.4.0
2023-09-12 14:40:05 +02:00
Sebastiaan van Stijn
7fbc3a9bc0
Merge pull request #46453 from thaJeztah/update_golang_1.20.8
update to go1.20.8
2023-09-12 10:43:25 +02:00
Sebastiaan van Stijn
ff85c363a8
Merge pull request #46440 from thaJeztah/update_cdi
vendor: github.com/cncf-tags/container-device-interface v0.6.1
2023-09-12 10:43:02 +02:00
Sebastiaan van Stijn
0b69c2c42f
Merge pull request #46443 from thaJeztah/mod_tidy
go mod tidy
2023-09-12 10:42:40 +02:00
Albin Kerouanton
78479b1915
libnet: Make sure network names are unique
Fixes #18864, #20648, #33561, #40901.

[This GH comment][1] makes clear network name uniqueness has never been
enforced due to the eventually consistent nature of Classic Swarm
datastores:

> there is no guaranteed way to check for duplicates across a cluster of
> docker hosts.

And this is further confirmed by other comments made by @mrjana in that
same issue, eg. [this one][2]:

> we want to adopt a schema which can pave the way in the future for a
> completely decentralized cluster of docker hosts (if scalability is
> needed).

This decentralized model is what Classic Swarm was trying to be. It's
been superseded since then by Docker Swarm, which has a centralized
control plane.

To circumvent this drawback, the `NetworkCreate` endpoint accepts a
`CheckDuplicate` flag. However it's not perfectly reliable as it won't
catch concurrent requests.

Due to this design decision, API clients like Compose have to implement
workarounds to make sure names are really unique (eg.
docker/compose#9585). And the daemon itself has seen a string of issues
due to that decision, including some that aren't fixed to this day (for
instance moby/moby#40901):

> The problem is, that if you specify a network for a container using
> the ID, it will add that network to the container but it will then
> change it to reference the network by using the name.

To summarize, this "feature" is broken, has no practical use and is a
source of pain for Docker users and API consumers. So let's just remove
it for _all_ API versions.

[1]: https://github.com/moby/moby/issues/18864#issuecomment-167201414
[2]: https://github.com/moby/moby/issues/18864#issuecomment-167202589

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-09-12 10:40:13 +02:00
Sebastiaan van Stijn
ff328e748d
Merge pull request #46397 from thaJeztah/fix_neighbor_delete
libnetwork/osl: Namespace.DeleteNeighbor: assorted cleanups, and ignore "non-exist" warnings
2023-09-12 10:38:43 +02:00
Sebastiaan van Stijn
4f28802f09
Merge pull request #46413 from thaJeztah/builder_diffid_type
daemon/containerd: use "DiffID" type for diff fields
2023-09-12 10:08:46 +02:00
Sebastiaan van Stijn
75308e471c
Merge pull request #46411 from thaJeztah/simplify_scratch
daemon: GetImageAndReleasableLayer: simplify "FROM scratch" case
2023-09-12 09:36:38 +02:00
Paweł Gronowski
5fa011dc0c
vendor: resenje.org/singleflight v0.4.0
Fixes the context aware singleflight not preserving context values.

full diff: https://github.com/janos/singleflight/compare/v0.3.0...v0.4.0

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-12 09:34:03 +02:00
Sebastiaan van Stijn
fd15ddbc30
daemon: GetImageAndReleasableLayer: simplify "FROM scratch" case
Windows doesn't support "FROM scratch", and the platform was only used
for validation on other platforms if a platform was provided, so no need
to set defaults.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-11 23:18:13 +02:00
Sebastiaan van Stijn
4229032676
daemon/containerd: use "DiffID" type for diff fields
strong-type the fields with the expected type, to make it more explicit
what we're expecting here.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-11 23:15:04 +02:00
Sebastiaan van Stijn
cdb5947316
Merge pull request #46455 from akerouanton/enable-ipv6-automatically
daemon: automatically set network EnableIPv6 if needed
2023-09-11 22:18:22 +02:00
Albin Kerouanton
5d5eeac310
daemon: automatically set network EnableIPv6 if needed
PR 4f47013feb added a validation step to `NetworkCreate` to ensure
no IPv6 subnet could be set on a network if its `EnableIPv6` parameter
is false.

Before that, the daemon was accepting such request but was doing nothing
with the IPv6 subnet.

This validation step is now deleted, and we automatically set
`EnableIPv6` if an IPv6 subnet was specified.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-09-11 20:53:29 +02:00
Sebastiaan van Stijn
51d647122a
Merge pull request #46383 from vvoland/c8d-legacybuilder-fix-layer-parent-snapshot
c8d/legacybuilder: Assorted fixes
2023-09-11 20:41:59 +02:00
Djordje Lukic
29dcf646a3
Merge pull request #46454 from rumpl/c8d-disable-schema1 2023-09-11 19:55:43 +02:00
Djordje Lukic
d0d3ddd045
Merge pull request #46375 from rumpl/c8d-userns-remap
c8d: Handle userns properly
2023-09-11 19:04:35 +02:00
Djordje Lukic
0313544f4a
c8d: Handle userns properly
If the daemon is run with --userns-remap we need to chown the prepared
snapshot

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-09-11 16:39:29 +02:00
Paweł Gronowski
c63a952dc9
c8d/builder: Don't append empty tar layer to manifest
To match the number of layers in config created in
`images.CreateChildImage`.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-11 16:22:44 +02:00
Paweł Gronowski
9127285985
c8d/builder: Lease layer snapshots
Create a lease for the snapshot and hold it until the layer is released.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-11 16:22:39 +02:00
Paweł Gronowski
f22b112005
c8d/commit: Unpack committed image
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-11 16:20:14 +02:00
Paweł Gronowski
8832cdf6e1
c8d: Common unpack code for specific manifest
Use `ImageService.unpackImage` when we want to unpack an image and we
know the exact platform-manifest to be unpacked beforehand.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-11 16:20:10 +02:00
Paweł Gronowski
c6bded3475
c8d/builder: Don't use diffID as snapshot parent
DiffID is only a digest of the one tar layer and matches the snapshot ID
only for the first layer (DiffID = ChainID).

Instead of generating random ID as a key for rolayer, just use the
snapshot ID of the unpacked image content and use it later as a parent
for creating a new RWLayer.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-11 16:20:04 +02:00
Paweł Gronowski
8c7e19c5ff
c8d/builder: Set empty diffID for rolayer
diffID is the digest of a tar archive containing changes to the parent
layer - rolayer doesn't have any changes to the parent.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-11 16:20:02 +02:00
Paweł Gronowski
6ebfa57364
c8d/builder: Don't mount the rolayer snapshot
The view snapshot and its mounts are not used.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-11 16:19:59 +02:00