Commit graph

46212 commits

Author SHA1 Message Date
Sebastiaan van Stijn
637ca59375
Dockerfile: use COPY --link to copy artifacts from build-stages
Build-cache for the build-stages themselves are already invalidated if the
base-images they're using is updated, and the COPY operations don't depend
on previous steps (as there's no overlap between artifacts copied).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-31 11:52:18 +02:00
Sebastiaan van Stijn
8d67d0c1a8
Merge pull request #45437 from thaJeztah/vendor_image_spec
vendor: github.com/opencontainers/image-spec v1.1.0-rc3
2023-05-31 11:12:51 +02:00
Bjorn Neergaard
abc05cf335
Merge pull request #45645 from sebthom/patch-1
Update blogpost URL
2023-05-30 15:36:37 -06:00
Bjorn Neergaard
988f5ac342
Merge pull request #45647 from rumpl/fix-snapshotter-change
c8d: Fix re-pull of an image when the snapshotter is changed
2023-05-30 15:32:55 -06:00
Cory Snider
d43b398746
Merge pull request #45657 from corhere/libn/setup-resolver-with-verbose-iptables
libnetwork: fix resolver restore w/ chatty 'iptables -C'
2023-05-30 21:44:14 +02:00
Cory Snider
a25434654e
Merge pull request #45654 from corhere/libn/fix-embedded-resolver-live-reload
libnetwork: fix sandbox restore
2023-05-30 21:43:46 +02:00
Cory Snider
1178319313 libn: fix resolver restore w/ chatty 'iptables -C'
Resolver.setupIPTable() checks whether it needs to flush or create the
user chains used for NATing container DNS requests by testing for the
existence of the rules which jump to said user chains. Unfortunately it
does so using the IPTable.RawCombinedOutputNative() method, which
returns a non-nil error if the iptables command returns any output even
if the command exits with a zero status code. While that is fine with
iptables-legacy as it prints no output if the rule exists, iptables-nft
v1.8.7 prints some information about the rule. Consequently,
Resolver.setupIPTable() would incorrectly think that the rule does not
exist during container restore and attempt to create it. This happened
work work by coincidence before 8f5a9a741b
because the failure to create the already-existing table would be
ignored and the new NAT rules would be inserted before the stale rules
left in the table from when the container was last started/restored. Now
that failing to create the table is treated as a fatal error, the
incompatibility with iptables-nft is no longer hidden.

Switch to using IPTable.ExistsNative() to test for the existence of the
jump rules as it correctly only checks the iptables command's exit
status without regard for whether it outputs anything.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-05-30 14:32:27 -04:00
Cory Snider
50eb2d2782 libnetwork: fix sandbox restore
The method to restore a network namespace takes a collection of
interfaces to restore with the options to apply. The interface names are
structured data, tuples of (SrcName, DstPrefix) but for whatever reason
are being passed into Restore() serialized to strings. A refactor,
f0be4d126d, accidentally broke the
serialization by dropping the delimiter. Rather than fix the
serialization and leave the time-bomb for someone else to trip over,
pass the interface names as structured data.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-05-30 12:27:59 -04:00
Cory Snider
18bf3aa442 libnetwork: log why osl sandbox restore failed
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-05-30 12:17:44 -04:00
Djordje Lukic
ed32f5e241 Make sure the image is unpacked for the current snapshotter
Switching snapshotter implementations would result in an error when
preparing a snapshot, check that the image is indeed unpacked for the
current snapshot before trying to prepare a snapshot.

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-05-30 14:45:30 +02:00
Sebastiaan van Stijn
2cd23ffeec
Merge pull request #45628 from thaJeztah/context_simplify
builder/remotecontext: remove mimeTypes struct, use consts
2023-05-30 13:40:42 +02:00
sebthom
d58df1fc6c Update blogpost URL
Signed-off-by: sebthom <sebthom@users.noreply.github.com>
2023-05-29 22:37:09 +02:00
Sebastiaan van Stijn
098b0fd1a0
Merge pull request #45627 from thaJeztah/remove_builder_streaming
builder/remotecontext: remove CachableSource, NewCachableSource
2023-05-29 19:04:32 +02:00
Sebastiaan van Stijn
44124ab6b0
builder/remotecontext: remove CachableSource, NewCachableSource
This type (as well as TarsumBackup), was used for the experimental --stream
support for the classic builder. This feature was removed in commit
6ca3ec88ae, which also removed uses of
the CachableSource type.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-29 16:35:42 +02:00
Sebastiaan van Stijn
3a643154be
Merge pull request #44697 from crazy-max/generate-files
Update and validation of generated files
2023-05-29 16:34:54 +02:00
CrazyMax
fd72b134d5
update generated files
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-29 03:28:35 +02:00
CrazyMax
735537d6b1
replace gogofast with gogofaster extension
gogofaster is identical as gogofast but removes XXX_unrecognized

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-29 03:28:35 +02:00
CrazyMax
1eaea43581
fix protos and "go generate" commands
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-29 03:28:35 +02:00
Kevin Alvarez
7daaa00120
hack: generated files update and validation
Adds a Dockerfile and make targets to update and validate
generated files (proto, seccomp default profile)

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-29 03:28:35 +02:00
CrazyMax
f1ca793980
use tools build constraint for proto dependencies
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-29 03:13:15 +02:00
Akihiro Suda
2ebd97dec1
Merge pull request #45641 from cpuguy83/exec_npe
Fix npe in exec resize when exec errored
2023-05-28 19:44:23 +09:00
Brian Goff
487ea81316 Fix npe in exec resize when exec errored
In cases where an exec start failed the exec process will be nil even
though the channel to signal that the exec started was closed.

Ideally ExecConfig would get a nice refactor to handle this case better
(ie. it's not started so don't close that channel).
This is a minimal fix to prevent NPE. Luckilly this would only get
called by a client and only the http request goroutine gets the panic
(http lib recovers the panic).

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2023-05-28 00:14:47 +00:00
Cory Snider
8f7bbc39a4
Merge pull request #45636 from corhere/libn/fix-encrypted-overlay-nonstandard-port
libnetwork/d/overlay: support encryption on any port
2023-05-26 22:40:56 +02:00
Cory Snider
9a692a3802 libn/d/overlay: support encryption on any port
While the VXLAN interface and the iptables rules to mark outgoing VXLAN
packets for encryption are configured to use the Swarm data path port,
the XFRM policies for actually applying the encryption are hardcoded to
match packets with destination port 4789/udp. Consequently, encrypted
overlay networks do not pass traffic when the Swarm is configured with
any other data path port: encryption is not applied to the outgoing
VXLAN packets and the destination host drops the received cleartext
packets. Use the configured data path port instead of hardcoding port
4789 in the XFRM policies.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-05-26 14:36:34 -04:00
Sebastiaan van Stijn
e410e27547
builder/remotecontext: remove mimeTypes struct, use consts
This struct was never modified; let's just use consts for these.

Also remove the args return from detectContentType(), as it was
not used anywhere.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-26 15:21:15 +02:00
Sebastiaan van Stijn
13fb24458c
Merge pull request #45626 from thaJeztah/deprecate_builder_streaming
builder/remotecontext: deprecate CachableSource, NewCachableSource
2023-05-26 15:12:49 +02:00
Sebastiaan van Stijn
b42e367045
vendor: github.com/opencontainers/image-spec v1.1.0-rc3
full diff: https://github.com/opencontainers/image-spec/compare/3a7f492d3f1b...v1.1.0-rc3

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-26 02:34:50 +02:00
Sebastiaan van Stijn
0db4174513
Merge pull request #45278 from AkihiroSuda/rro
Support recursively read-only (RRO) mounts
2023-05-26 02:24:43 +02:00
Sebastiaan van Stijn
37d4b0bee9
builder/remotecontext: deprecate CachableSource, NewCachableSource
This type (as well as TarsumBackup), was used for the experimental --stream
support for the classic builder. This feature was removed in commit
6ca3ec88ae, which also removed uses of
the CachableSource type.

As far as I could find, there's no external consumers of these types,
but let's deprecated it, to give potential users a heads-up that it
will be removed.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-26 00:05:08 +02:00
Sebastiaan van Stijn
88f6a92d22
Merge pull request #45624 from corhere/libc8d/serialize-exec-starts-workaround
libcontainerd: work around exec start bug in c8d
2023-05-25 23:02:34 +02:00
Sebastiaan van Stijn
a4c54362c3
Merge pull request #45581 from thaJeztah/vendor_buildkit_0.11.7_dev
vendor: github.com/moby/buildkit v0.11.7-0.20230525183624-798ad6b0ce9f
2023-05-25 22:27:06 +02:00
Cory Snider
fb7ec1555c libcontainerd: work around exec start bug in c8d
It turns out that the unnecessary serialization removed in
b75246202a happened to work around a bug
in containerd. When many exec processes are started concurrently in the
same containerd task, it takes seconds to minutes for them all to start.
Add the workaround back in, only deliberately this time.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-05-25 16:00:29 -04:00
Sebastiaan van Stijn
79ca6630d4
vendor: github.com/moby/buildkit v0.11.7-0.20230525183624-798ad6b0ce9f
full diff: https://github.com/moby/buildkit/compare/v0.11.6...798ad6b0ce9f2fe86dfb2b0277e6770d0b545871

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-25 21:35:53 +02:00
Sebastiaan van Stijn
d5dc675d37
Merge pull request #45280 from corhere/libnet/no-overlay-accept-rule
libnetwork/drivers/overlay: stop programming INPUT ACCEPT rule
2023-05-25 21:03:32 +02:00
Sebastiaan van Stijn
10505cac52
Merge pull request #45619 from thaJeztah/update_go_runc_v1.1.0
vendor: github.com/containerd/go-runc v1.1.0
2023-05-25 20:19:05 +02:00
Akihiro Suda
5045a2de24
Support recursively read-only (RRO) mounts
`docker run -v /foo:/foo:ro` is now recursively read-only on kernel >= 5.12.

Automatically falls back to the legacy non-recursively read-only mount mode on kernel < 5.12.

Use `ro-non-recursive` to disable RRO.
Use `ro-force-recursive` or `rro` to explicitly enable RRO. (Fails on kernel < 5.12)

Fix issue 44978
Fix docker/for-linux issue 788

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2023-05-26 01:58:24 +09:00
Sebastiaan van Stijn
3512b04093
vendor: github.com/containerd/go-runc v1.1.0
full diff: https://github.com/containerd/go-runc/compare/v1.0.0...v1.1.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-25 18:56:52 +02:00
Cory Snider
1b28b0ed5a
Merge pull request #45134 from elezar/add-cdi-support
Add support for CDI devices under Linux
2023-05-25 18:06:31 +02:00
Sebastiaan van Stijn
02c9f038b3
Merge pull request #45618 from vvoland/c8d-inspect-created-time
c8d/inspect: Fill `Created` time if available
2023-05-25 17:16:56 +02:00
Paweł Gronowski
b9b8b6597a
c8d/inspect: Fill Created time if available
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-05-25 14:59:49 +02:00
Sebastiaan van Stijn
f1d5385515
Merge pull request #45609 from thaJeztah/constantly_numb
c8d: ImageService.softImageDelete: use OCI and containerd constants
2023-05-25 09:52:31 +02:00
Sebastiaan van Stijn
91b976ba46
Merge pull request #45607 from corhere/lock-in-snapshotter-setting
daemon: lock in snapshotter setting at daemon init
2023-05-25 01:47:19 +02:00
Sebastiaan van Stijn
f17c9e4aeb
c8d: ImageService.softImageDelete: rename var that collided with import
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-25 01:44:36 +02:00
Sebastiaan van Stijn
df5deab20b
c8d: ImageService.softImageDelete: use OCI and containerd constants
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-25 01:44:31 +02:00
Cory Snider
9b9c5242eb daemon: lock in snapshotter setting at daemon init
Feature flags are one of the configuration items which can be reloaded
without restarting the daemon. Whether the daemon uses the containerd
snapshotter service or the legacy graph drivers is controlled by a
feature flag. However, much of the code which checks the snapshotter
feature flag assumes that the flag cannot change at runtime. Make it so
that the snapshotter setting can only be changed by restarting the
daemon, even if the flag state changes after a live configuration
reload.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-05-24 16:56:17 -04:00
Sebastiaan van Stijn
7e01865fb6
Merge pull request #45592 from vvoland/dangling-image-repotagsdigests-test
integration: Add TestImageInspectEmptyTagsAndDigests
2023-05-24 19:18:56 +02:00
Sebastiaan van Stijn
1e3cdd3512
Merge pull request #45603 from thaJeztah/windows_notgo
hack/make/.binary: don't use "netgo" when building Windows binaries
2023-05-24 18:41:38 +02:00
Sebastiaan van Stijn
53d1b12bc0
hack/make/.binary: don't use "netgo" when building Windows binaries
Starting with go1.19, the Go runtime on Windows now supports the `netgo` build-
flag to use a native Go DNS resolver. Prior to that version, the build-flag
only had an effect on non-Windows platforms. When using the `netgo` build-flag,
the Windows's host resolver is not used, and as a result, custom entries in
`etc/hosts` are ignored, which is a change in behavior from binaries compiled
with older versions of the Go runtime.

From the go1.19 release notes: https://go.dev/doc/go1.19#net

> Resolver.PreferGo is now implemented on Windows and Plan 9. It previously
> only worked on Unix platforms. Combined with Dialer.Resolver and Resolver.Dial,
> it's now possible to write portable programs and be in control of all DNS name
> lookups when dialing.
>
> The net package now has initial support for the netgo build tag on Windows.
> When used, the package uses the Go DNS client (as used by Resolver.PreferGo)
> instead of asking Windows for DNS results. The upstream DNS server it discovers
> from Windows may not yet be correct with complex system network configurations,
> however.

Our Windows binaries are compiled with the "static" (`make/binary-daemon`)
script, which has the `netgo` option set by default. This patch unsets the
`netgo` option when cross-compiling for Windows.

Co-authored-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-05-24 09:16:41 -06:00
Sebastiaan van Stijn
c5126d1435
Merge pull request #45601 from vvoland/c8d-exists
c8d/pull: Use same progress action as distribution
2023-05-24 12:48:12 +02:00
Paweł Gronowski
a7bc65fbd8
c8d/pull: Use same progress action as distribution
Docker with containerd integration emits "Exists" progress action when a
layer of the currently pulled image already exists. This is different
from the non-c8d Docker which emits "Already exists".

This makes both implementations consistent by emitting backwards
compatible "Already exists" action.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-05-24 11:16:57 +02:00