Commit graph

46867 commits

Author SHA1 Message Date
Djordje Lukic
f3a6b0fd08
c8d: Make sure the content isn't removed while we export
This change add leases for all the content that will be exported, once
the image(s) are exported the lease is removed, thus letting
containerd's GC to do its job if needed. This fixes the case where
someone would remove an image that is still being exported.

This fixes the TestAPIImagesSaveAndLoad cli integration test.

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-07-19 09:59:11 +02:00
Sebastiaan van Stijn
f061f39145
Merge pull request #46009 from vvoland/c8d-resolve-fix-digested-and-named
c8d/resolveImage: Fix Digested and Named reference
2023-07-19 00:12:03 +02:00
Bjorn Neergaard
124b1141d7
Merge pull request #46016 from neersighted/AUTHORS_.mailmap
AUTHORS,.mailmap: update with recent contributors
2023-07-18 15:29:58 -06:00
Bjorn Neergaard
6cba44277f
AUTHORS,.mailmap: update with recent contributors
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-07-18 13:55:16 -06:00
Sebastiaan van Stijn
f329397077
Merge pull request #46007 from vvoland/c8d-inspect-fix-duplicate-digested
c8d/inspect: Don't duplicate digested ref
2023-07-18 20:19:40 +02:00
Bjorn Neergaard
db7fef6e50
Merge pull request #46012 from neersighted/buildkit-ref_nits
ci(buildkit): remove misleading code from buildkit-ref
2023-07-18 11:33:38 -06:00
Artem Khramov
8b843732b3 pkg/chrootarchive: fix FreeBSD build
For unix targets, `goInChroot()` is only implemented for `Linux`,
hence FreeBSD build fails.

This change

- Adds FreeBSD-specific chrooted tar/untar implementation
- Fixes statUnix() to accomodate to FreeBSD devminor/devmajor
- quirk. See also: https://github.com/containerd/containerd/pull/5991

Signed-off-by: Artem Khramov <akhramov@pm.me>
Co-authored-by: Cory Snider <corhere@gmail.com>
2023-07-18 18:42:08 +02:00
Paweł Gronowski
eaf141c86a
c8d/resolveImage: Fix Digested and Named reference
When resolving a reference that is both a Named and Digested, it could
be resolved to an image that has the same digest, but completely
different repository name.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-07-18 17:55:46 +02:00
Paweł Gronowski
028eab9ebb
c8d/inspect: Don't duplicate digested ref
If image name is already an untagged digested reference, don't produce
additional digested ref.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-07-18 17:47:53 +02:00
Bjorn Neergaard
7310a7cd0c
ci(buildkit): remove early-return from buildkit-ref
This doesn't really make sense now that this script returns a
$GITHUB_ENV snippet.

Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-07-18 09:30:42 -06:00
Bjorn Neergaard
4ecc01f3ad
ci(buildkit): remove misleading code from buildkit-ref
Post-f8c0d92a22bad004cb9cbb4db704495527521c42, BUILDKIT_REPO doesn't
really do what it claims to. Instead, don't allow overloading since the
import path for BuildKit is always the same, and make clear the
provenance of values when generating the final variable definitions.

We also better document the script, and follow some best practices for
both POSIX sh and Bash.

Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-07-18 09:30:40 -06:00
Bjorn Neergaard
48ff8a95cc
hack/with-go-mod.sh: warn on stderr
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-07-18 09:30:39 -06:00
Sebastiaan van Stijn
8e51b8b59c
Merge pull request #46008 from thaJeztah/c8d_nits
c8d: some minor cleanups
2023-07-18 15:52:04 +02:00
Sebastiaan van Stijn
058a6e94a1
Merge pull request #45994 from jedevc/buildkit-ref-replace
ci: extract buildkit version correctly with replace-d modules
2023-07-18 15:22:38 +02:00
Sebastiaan van Stijn
750c441dfd
daemon: remove intermediate var
The imgSvcConfig is defined locally, and discarded if an error occurs,
so no need to use the intermediate vars here.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-18 14:25:41 +02:00
Sebastiaan van Stijn
468d6616bf
daemon: configureLocalContentStore: return concrete types
The interface is defined on the receiver-side, and returning concrete
types makes it more transparent what we're creating.

As these namespaced wrappers were not exported, let's inline them, so
that it's clear at a glance what it's doing.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-18 14:18:42 +02:00
Sebastiaan van Stijn
375c4eb31c
daemon: rename vars that shadowed package-level types
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-18 14:06:33 +02:00
Sebastiaan van Stijn
034e7e153f
daemon: rename containerdCli to containerdClient
The containerdCli was somewhat confusing (is it the CLI?); let's rename
to make it match what it is :)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-18 13:57:27 +02:00
Bjorn Neergaard
0c6b616656
Merge pull request #45946 from thaJeztah/plugin_refactor_setupRemotePluginServer
pkg/plugins: don't share plugin server between tests and t.Parallel()
2023-07-17 17:51:04 -06:00
Sebastiaan van Stijn
79a198bc08
Merge pull request #46000 from thaJeztah/notestyourself
quota: remove gotest.tools from testhelpers
2023-07-18 00:29:48 +02:00
Sebastiaan van Stijn
7799f23627
Merge pull request #45979 from thaJeztah/hijack_share_request_builder
client: Client.postHijacked: use Client.buildRequest
2023-07-17 23:50:34 +02:00
Sebastiaan van Stijn
1aa17222e7
quota: remove gotest.tools from testhelpers
gotest.tools has an init() which registers a '-update' flag;
a80f057529/internal/source/update.go (L21-L23)

The quota helper contains a testhelpers file, which is meant for usage
in (integration) tests, but as it's in the same pacakge as production
code, would also trigger the gotest.tools init.

This patch removes the gotest.tools code from this file.

Before this patch:

    $ (exec -a libnetwork-setkey "$(which dockerd)" -help)
    Usage of libnetwork-setkey:
      -exec-root string
            docker exec root (default "/run/docker")
      -update
            update golden values

With this patch applied:

    $ (exec -a libnetwork-setkey "$(which dockerd)" -help)
    Usage of libnetwork-setkey:
      -exec-root string
            docker exec root (default "/run/docker")

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-17 23:05:09 +02:00
Akihiro Suda
29eb0be134
Merge pull request #45993 from thaJeztah/update_fsutils
vendor: github.com/containerd/continuity v0.4.1, tonistiigi/fsutil v0.0.0-20230629203738-36ef4d8c0dbb
2023-07-18 04:12:07 +09:00
Bjorn Neergaard
5491f9a9a9
Merge pull request #45997 from neersighted/build_tweaks
Build system grab-bag
2023-07-17 11:16:59 -06:00
Bjorn Neergaard
a972dbd682
hack/make/.binary: use with-go-mod.sh
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-07-17 10:38:12 -06:00
Bjorn Neergaard
5563b09ac2
hack: use Git-free ROOTDIR convention
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-07-17 10:38:10 -06:00
Bjorn Neergaard
780e8b2332
hack/d/cli.sh: properly handle errors in curl
Add `-f` to output nothing to tar if the curl fails, and `-S` to report
errors if they happen.

Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-07-17 10:18:19 -06:00
Bjorn Neergaard
12a19dcd84
Dockerfile: improve CLI/rootlesskit caching
Use bind-mounts instead of a `COPY` for cli.sh, and use `COPY --link`
for rootlesskit's build stage.

Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-07-17 10:18:19 -06:00
Bjorn Neergaard
235cd6c6b2
Dockerfile(.simple): align APT_MIRROR support
Use a non-slash escape sequence to support mirrors with a path
component, and do not unconditionally replace the mirror in
Dockerfile.simple.

Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-07-17 10:18:18 -06:00
Bjorn Neergaard
bcea83ab9b
Makefile: pass through APT_MIRROR
This aligns `docker build` as invoked by the Makefile with both `docker
buildx bake` as invoked by the Makefile and directly by the user.

Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-07-17 10:18:14 -06:00
Bjorn Neergaard
37dcdac218
Merge pull request #40751 from cpuguy83/tmpfs_for_main_test_daemon
Put integration daemon socket in /run
2023-07-17 10:17:54 -06:00
Sebastiaan van Stijn
ba513805d0
Merge pull request #45983 from thaJeztah/libnetwork_bridge_error
libnetwork/drivers/bridge: setupBridgeNetFiltering: improve error handling
2023-07-17 16:23:49 +02:00
Justin Chadwell
f8c0d92a22 ci: extract buildkit version correctly with replace-d modules
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-07-17 14:17:49 +01:00
Bjorn Neergaard
75d7e053dc
Merge pull request #45984 from thaJeztah/libnetwork_cleanup_options
libnetwork/options: remove unused NewGeneric, and use map[string]any
2023-07-17 07:05:11 -06:00
Sebastiaan van Stijn
010c3718e3
vendor: github.com/tonistiigi/fsutil v0.0.0-20230629203738-36ef4d8c0dbb
- Fix copy on windows plus tests
- Fix follow symlinkResolver on Windows
- Implement proper renameFile on Windows
- Fix potential nil pointer dereference
- Use RunWithPrivileges
- Fix leaking file handle
- handle mkdir race for diskwriter
- walk: avoid stat()'ing files unnecessarily
- ci: fix freebsd workflow
- update to Go 1.20

full diff: fb433841cb...36ef4d8c0d

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-17 14:38:11 +02:00
Sebastiaan van Stijn
531dbd7af5
vendor: github.com/containerd/continuity v0.4.1
full diff: https://github.com/containerd/continuity/compare/v0.3.0...v0.4.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-17 14:37:20 +02:00
Bjorn Neergaard
3f8ca3553d
Merge pull request #45986 from thaJeztah/libnetwork_cleanup_config
libnetwork/config: add Config.DriverConfig() and un-export DriverCfg
2023-07-17 06:22:24 -06:00
Justin Chadwell
eb68c5e747 daemon: support using CIDR notation for getting registry hosts
insecure-registries supports using CIDR notation, however, buildkit in
moby was not respecting these. We can update the RegistryHosts function
to support this by inserting the correct host into the lookup map if
it's explicitly marked as insecure.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-07-17 12:34:54 +01:00
Justin Chadwell
e0b065cc33 daemon: use the registry service config for getting registry hosts
The RegistryHosts lookup function is used by both BuildKit and by the
containerd snapshotter. However, this function differs in behaviour from
the config parser for the RegistryConfig:

- The protocol for insecure registries is treated as significant by
  RegistryHosts, while the RegistryConfig strips this information.
- RegistryConfig validates and deduplicates mirrors.
- RegistryConfig does not parse the insecure-registries as URLs, which
  can lead to parsing opaque URLs as was possible by the RegistryHosts
  function.

This patch updates the lookup function to ensure consistency.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-07-17 12:30:34 +01:00
Sebastiaan van Stijn
2da589d454
client: return explicit "nil" for no errors, and remove nil check
- remove some intermediate variables
- explicitly return "nil" if there's no error
- remove redundant check for response-headers being nil

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-17 12:24:28 +02:00
Sebastiaan van Stijn
738b16d873
libnetwork/config: add Config.DriverConfig() and un-export DriverCfg
The driver-configurations are only set when creating a new controller,
using the `config.OptionDriverConfig()` option that can be passed to
`New()`, and used as "read-only" after that.

Taking away any other paths that set these options, the only type used
for per-driver options are a `map[string]interface{}`, so we can change
the type from `map[string]interface{}` to a `map[string]map[string]interface{}`,
(or its "modern" variant: `map[string]map[string]any`), so that it's
no longer needed to cast the type before use.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-17 09:57:14 +02:00
Sebastiaan van Stijn
82bb3d8d2b
libnetwork: TestUserChain: use assert.Check and is.ErrorContains
Don't fail early if we can still test more, and be slightly more strict
in what error we're looking for.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-17 09:53:13 +02:00
Sebastiaan van Stijn
4e709e75da
libnetwork: TestUserChain: re-use IPTables instances
The test already creates instances for each ip-version, so let's
re-use them. While changing, also use assert.Check to not fail early.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-17 09:53:13 +02:00
Sebastiaan van Stijn
c471255153
libnetwork: TestUserChain: don't manually manipulate Controller.cfg.DriverCfg
New() allows for driver-options to be passed using the config.OptionDriverConfig.
Update the test to not manually mutate the controller's configuration after
creating.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-17 09:53:10 +02:00
Sebastiaan van Stijn
ee79423124
Merge pull request #45985 from thaJeztah/libnetwork_remove_IsValidName
libnetwork/config: remove IsValidName utility
2023-07-17 09:33:45 +02:00
Sebastiaan van Stijn
0761240c43
Merge pull request #45988 from thaJeztah/libnetwork_drivers_clean
libnetwork, libnetwork/drivers: some minor cleanups
2023-07-17 09:32:33 +02:00
Sebastiaan van Stijn
ddd33c6bbd
libnetwork/iptables: move IPTable.LoopbackByVersion() to a utility
Not critical, but when used from ChainInfo, we had to construct an IPTable
based on the version of the ChainInfo, which then only used the version
we passed to get the right loopback.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-16 21:53:36 +02:00
Sebastiaan van Stijn
aad2dbb93d
libnetwork/iptables: GetIptable: validate provided IPversion
For backward-compatibility, continue to accept empty strings
as default (IPv4).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-16 21:53:29 +02:00
Sebastiaan van Stijn
47063ca3ae
libnetwork/iptables: un-export IPTable.Version
We have the GetIptable "constructor". Let's make that the canonical way
to initialize.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-16 20:47:00 +02:00
Sebastiaan van Stijn
9c2cd65c0d
libnetwork/iptables: make some vars local, and move bestEffortLock lock
Make some variables local to the if-branches to be slightly more iodiomatic,
and to make clear it's only used in that branch.

Move the bestEffortLock locking later in IPtable.raw(), because that function'
could return before the lock was even needed.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-16 20:46:59 +02:00