Commit graph

46305 commits

Author SHA1 Message Date
Sebastiaan van Stijn
2cd23ffeec
Merge pull request #45628 from thaJeztah/context_simplify
builder/remotecontext: remove mimeTypes struct, use consts
2023-05-30 13:40:42 +02:00
sebthom
d58df1fc6c Update blogpost URL
Signed-off-by: sebthom <sebthom@users.noreply.github.com>
2023-05-29 22:37:09 +02:00
Sebastiaan van Stijn
098b0fd1a0
Merge pull request #45627 from thaJeztah/remove_builder_streaming
builder/remotecontext: remove CachableSource, NewCachableSource
2023-05-29 19:04:32 +02:00
Sebastiaan van Stijn
44124ab6b0
builder/remotecontext: remove CachableSource, NewCachableSource
This type (as well as TarsumBackup), was used for the experimental --stream
support for the classic builder. This feature was removed in commit
6ca3ec88ae, which also removed uses of
the CachableSource type.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-29 16:35:42 +02:00
Sebastiaan van Stijn
3a643154be
Merge pull request #44697 from crazy-max/generate-files
Update and validation of generated files
2023-05-29 16:34:54 +02:00
CrazyMax
5efe72415d
remove Dockerfile.e2e
Dockerfile.e2e is not used anymore. Integration tests run
through the main Dockerfile.

Also removes the daemon OS/Arch detection script that is not
necessary anymore. It was used to select the Dockerfile based
on the arch like Dockerfile.arm64 but we don't have those
anymore. Was also used to check referenced frozen images
in the Dockerfile.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-29 11:03:48 +02:00
CrazyMax
fd72b134d5
update generated files
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-29 03:28:35 +02:00
CrazyMax
735537d6b1
replace gogofast with gogofaster extension
gogofaster is identical as gogofast but removes XXX_unrecognized

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-29 03:28:35 +02:00
CrazyMax
1eaea43581
fix protos and "go generate" commands
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-29 03:28:35 +02:00
Kevin Alvarez
7daaa00120
hack: generated files update and validation
Adds a Dockerfile and make targets to update and validate
generated files (proto, seccomp default profile)

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-29 03:28:35 +02:00
CrazyMax
f1ca793980
use tools build constraint for proto dependencies
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-29 03:13:15 +02:00
Brian Goff
1e55ace875 Add oci index and layout files to exported tars
This makes the output of `docker save` fully OCI compliant.

When using the containerd image store, this code is not used. That
exporter will just use containerd's export method and should give us the
output we want for multi-arch images.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2023-05-28 20:35:14 +00:00
Akihiro Suda
2ebd97dec1
Merge pull request #45641 from cpuguy83/exec_npe
Fix npe in exec resize when exec errored
2023-05-28 19:44:23 +09:00
Brian Goff
487ea81316 Fix npe in exec resize when exec errored
In cases where an exec start failed the exec process will be nil even
though the channel to signal that the exec started was closed.

Ideally ExecConfig would get a nice refactor to handle this case better
(ie. it's not started so don't close that channel).
This is a minimal fix to prevent NPE. Luckilly this would only get
called by a client and only the http request goroutine gets the panic
(http lib recovers the panic).

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2023-05-28 00:14:47 +00:00
Cory Snider
8f7bbc39a4
Merge pull request #45636 from corhere/libn/fix-encrypted-overlay-nonstandard-port
libnetwork/d/overlay: support encryption on any port
2023-05-26 22:40:56 +02:00
Cory Snider
9a692a3802 libn/d/overlay: support encryption on any port
While the VXLAN interface and the iptables rules to mark outgoing VXLAN
packets for encryption are configured to use the Swarm data path port,
the XFRM policies for actually applying the encryption are hardcoded to
match packets with destination port 4789/udp. Consequently, encrypted
overlay networks do not pass traffic when the Swarm is configured with
any other data path port: encryption is not applied to the outgoing
VXLAN packets and the destination host drops the received cleartext
packets. Use the configured data path port instead of hardcoding port
4789 in the XFRM policies.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-05-26 14:36:34 -04:00
Sebastiaan van Stijn
e410e27547
builder/remotecontext: remove mimeTypes struct, use consts
This struct was never modified; let's just use consts for these.

Also remove the args return from detectContentType(), as it was
not used anywhere.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-26 15:21:15 +02:00
Sebastiaan van Stijn
13fb24458c
Merge pull request #45626 from thaJeztah/deprecate_builder_streaming
builder/remotecontext: deprecate CachableSource, NewCachableSource
2023-05-26 15:12:49 +02:00
Sebastiaan van Stijn
b42e367045
vendor: github.com/opencontainers/image-spec v1.1.0-rc3
full diff: https://github.com/opencontainers/image-spec/compare/3a7f492d3f1b...v1.1.0-rc3

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-26 02:34:50 +02:00
Sebastiaan van Stijn
0db4174513
Merge pull request #45278 from AkihiroSuda/rro
Support recursively read-only (RRO) mounts
2023-05-26 02:24:43 +02:00
Sebastiaan van Stijn
37d4b0bee9
builder/remotecontext: deprecate CachableSource, NewCachableSource
This type (as well as TarsumBackup), was used for the experimental --stream
support for the classic builder. This feature was removed in commit
6ca3ec88ae, which also removed uses of
the CachableSource type.

As far as I could find, there's no external consumers of these types,
but let's deprecated it, to give potential users a heads-up that it
will be removed.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-26 00:05:08 +02:00
Sebastiaan van Stijn
88f6a92d22
Merge pull request #45624 from corhere/libc8d/serialize-exec-starts-workaround
libcontainerd: work around exec start bug in c8d
2023-05-25 23:02:34 +02:00
Sebastiaan van Stijn
a4c54362c3
Merge pull request #45581 from thaJeztah/vendor_buildkit_0.11.7_dev
vendor: github.com/moby/buildkit v0.11.7-0.20230525183624-798ad6b0ce9f
2023-05-25 22:27:06 +02:00
Cory Snider
fb7ec1555c libcontainerd: work around exec start bug in c8d
It turns out that the unnecessary serialization removed in
b75246202a happened to work around a bug
in containerd. When many exec processes are started concurrently in the
same containerd task, it takes seconds to minutes for them all to start.
Add the workaround back in, only deliberately this time.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-05-25 16:00:29 -04:00
Sebastiaan van Stijn
79ca6630d4
vendor: github.com/moby/buildkit v0.11.7-0.20230525183624-798ad6b0ce9f
full diff: https://github.com/moby/buildkit/compare/v0.11.6...798ad6b0ce9f2fe86dfb2b0277e6770d0b545871

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-25 21:35:53 +02:00
Sebastiaan van Stijn
d5dc675d37
Merge pull request #45280 from corhere/libnet/no-overlay-accept-rule
libnetwork/drivers/overlay: stop programming INPUT ACCEPT rule
2023-05-25 21:03:32 +02:00
Sebastiaan van Stijn
10505cac52
Merge pull request #45619 from thaJeztah/update_go_runc_v1.1.0
vendor: github.com/containerd/go-runc v1.1.0
2023-05-25 20:19:05 +02:00
Akihiro Suda
5045a2de24
Support recursively read-only (RRO) mounts
`docker run -v /foo:/foo:ro` is now recursively read-only on kernel >= 5.12.

Automatically falls back to the legacy non-recursively read-only mount mode on kernel < 5.12.

Use `ro-non-recursive` to disable RRO.
Use `ro-force-recursive` or `rro` to explicitly enable RRO. (Fails on kernel < 5.12)

Fix issue 44978
Fix docker/for-linux issue 788

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2023-05-26 01:58:24 +09:00
Sebastiaan van Stijn
3512b04093
vendor: github.com/containerd/go-runc v1.1.0
full diff: https://github.com/containerd/go-runc/compare/v1.0.0...v1.1.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-25 18:56:52 +02:00
Cory Snider
1b28b0ed5a
Merge pull request #45134 from elezar/add-cdi-support
Add support for CDI devices under Linux
2023-05-25 18:06:31 +02:00
Sebastiaan van Stijn
02c9f038b3
Merge pull request #45618 from vvoland/c8d-inspect-created-time
c8d/inspect: Fill `Created` time if available
2023-05-25 17:16:56 +02:00
Paweł Gronowski
b9b8b6597a
c8d/inspect: Fill Created time if available
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-05-25 14:59:49 +02:00
Sebastiaan van Stijn
f1d5385515
Merge pull request #45609 from thaJeztah/constantly_numb
c8d: ImageService.softImageDelete: use OCI and containerd constants
2023-05-25 09:52:31 +02:00
Sebastiaan van Stijn
91b976ba46
Merge pull request #45607 from corhere/lock-in-snapshotter-setting
daemon: lock in snapshotter setting at daemon init
2023-05-25 01:47:19 +02:00
Sebastiaan van Stijn
f17c9e4aeb
c8d: ImageService.softImageDelete: rename var that collided with import
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-25 01:44:36 +02:00
Sebastiaan van Stijn
df5deab20b
c8d: ImageService.softImageDelete: use OCI and containerd constants
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-25 01:44:31 +02:00
Cory Snider
9b9c5242eb daemon: lock in snapshotter setting at daemon init
Feature flags are one of the configuration items which can be reloaded
without restarting the daemon. Whether the daemon uses the containerd
snapshotter service or the legacy graph drivers is controlled by a
feature flag. However, much of the code which checks the snapshotter
feature flag assumes that the flag cannot change at runtime. Make it so
that the snapshotter setting can only be changed by restarting the
daemon, even if the flag state changes after a live configuration
reload.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-05-24 16:56:17 -04:00
Sebastiaan van Stijn
7e01865fb6
Merge pull request #45592 from vvoland/dangling-image-repotagsdigests-test
integration: Add TestImageInspectEmptyTagsAndDigests
2023-05-24 19:18:56 +02:00
Sebastiaan van Stijn
1e3cdd3512
Merge pull request #45603 from thaJeztah/windows_notgo
hack/make/.binary: don't use "netgo" when building Windows binaries
2023-05-24 18:41:38 +02:00
Sebastiaan van Stijn
53d1b12bc0
hack/make/.binary: don't use "netgo" when building Windows binaries
Starting with go1.19, the Go runtime on Windows now supports the `netgo` build-
flag to use a native Go DNS resolver. Prior to that version, the build-flag
only had an effect on non-Windows platforms. When using the `netgo` build-flag,
the Windows's host resolver is not used, and as a result, custom entries in
`etc/hosts` are ignored, which is a change in behavior from binaries compiled
with older versions of the Go runtime.

From the go1.19 release notes: https://go.dev/doc/go1.19#net

> Resolver.PreferGo is now implemented on Windows and Plan 9. It previously
> only worked on Unix platforms. Combined with Dialer.Resolver and Resolver.Dial,
> it's now possible to write portable programs and be in control of all DNS name
> lookups when dialing.
>
> The net package now has initial support for the netgo build tag on Windows.
> When used, the package uses the Go DNS client (as used by Resolver.PreferGo)
> instead of asking Windows for DNS results. The upstream DNS server it discovers
> from Windows may not yet be correct with complex system network configurations,
> however.

Our Windows binaries are compiled with the "static" (`make/binary-daemon`)
script, which has the `netgo` option set by default. This patch unsets the
`netgo` option when cross-compiling for Windows.

Co-authored-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-05-24 09:16:41 -06:00
Sebastiaan van Stijn
c5126d1435
Merge pull request #45601 from vvoland/c8d-exists
c8d/pull: Use same progress action as distribution
2023-05-24 12:48:12 +02:00
Paweł Gronowski
a7bc65fbd8
c8d/pull: Use same progress action as distribution
Docker with containerd integration emits "Exists" progress action when a
layer of the currently pulled image already exists. This is different
from the non-c8d Docker which emits "Already exists".

This makes both implementations consistent by emitting backwards
compatible "Already exists" action.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-05-24 11:16:57 +02:00
Akihiro Suda
1d68544fbf
Merge pull request #45596 from thaJeztah/bump_zfs
vendor: github.com/mistifyio/go-zfs/v3 v3.0.1
2023-05-24 16:10:12 +09:00
Sebastiaan van Stijn
cf4df9d8ae
Merge pull request #45597 from thaJeztah/skip_criu
Dockerfile: temporarily skip CRIU stage
2023-05-23 17:53:06 +02:00
Paweł Gronowski
6506579e18
integration: Add TestImageInspectEmptyTagsAndDigests
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-05-23 15:50:36 +02:00
Paweł Gronowski
a93aadc2e6
hack: Rename .ensure-emptyfs to .build-empty-images
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-05-23 15:50:35 +02:00
Paweł Gronowski
3a31f81838
hack/ensure-emptyfs: Create dangling image
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-05-23 15:50:33 +02:00
Sebastiaan van Stijn
4373547857
Add testenv.UsingSnapshotter utility
To allow skipping integration tests that don't apply to the
containerd snapshotter.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-05-23 15:50:30 +02:00
Sebastiaan van Stijn
d3d2823edf
Dockerfile: temporarily skip CRIU stage
The package repository currently has issues;

    => ERROR https://download.opensuse.org/repositories/devel:/tools:/criu/Debian_11/Release.key

The only test currently using this binary is currently skipped, as the test
was broken;
6e98a7f2c9/integration/container/checkpoint_test.go (L32-L33)

So let's disable this stage for the time being.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-23 12:58:22 +02:00
Sebastiaan van Stijn
86108812b8
vendor: github.com/mistifyio/go-zfs/v3 v3.0.1
Switching to the v3 version, which was renamed to be compatible with
go modules.

Full diff: https://github.com/mistifyio/go-zfs/compare/f784269be439...v3.0.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-23 12:52:16 +02:00