Commit graph

47675 commits

Author SHA1 Message Date
Paweł Gronowski
1337a66130
integration-cli: Skip Parent tests with c8d
Parent property is not available in the OCI config.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-22 15:33:47 +02:00
Sebastiaan van Stijn
3614749b55
Merge pull request #45966 from neersighted/buildkit_0.12
Update to BuildKit 0.12
2023-09-22 02:13:15 +02:00
Sebastiaan van Stijn
90b756b2a1
Merge pull request #46525 from thaJeztah/daemon_typed_logformat
daemon: strongly type containerd log.OutputFormat
2023-09-21 22:27:02 +02:00
CrazyMax
d5b067e04a
ci(buildkit): remove regex skipping tests with digest inconsistency
Skipping digest-related tests is no longer necessary after 4065831652

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-09-21 14:25:28 -06:00
CrazyMax
286704bf6f
ci(buildkit): expose github runtime for gha tests
This exposes `ACTIONS_RUNTIME_TOKEN` and `ACTIONS_CACHE_URL`, which are
used to skip cache exporter tests, when combined with
a8789cbd4a

Co-authored-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-09-21 14:25:10 -06:00
Bjorn Neergaard
34e71b374b
builder-next/a/ci/pull: strongly type flightcontrol.Group
Now that this is a generic, we can define a struct type at the package
level, and remove the casting logic necessary when we had to use
interface{}.

Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-09-21 14:18:57 -06:00
Bjorn Neergaard
c8773e10b1
builder-next/wrapper: use new exptypes constants
Introduced in 4fc2d7b5e7

Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-09-21 14:18:55 -06:00
Bjorn Neergaard
a728bd0ee9
builder-next/pull: use ResolveMode.String() helper
Introduced years ago in 6644e1b0da

Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-09-21 14:18:52 -06:00
Bjorn Neergaard
221b73e23d
builder-next: apply source policies on config resolve
SourcePolicy was accounted for in 330cf7ae7d

TODO: replace applySourcePolicies with BuildKit's implementation, which
is currently unexported.

Co-authored-by: Tonis Tiigi <tonistiigi@gmail.com>
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-09-21 14:18:50 -06:00
Bjorn Neergaard
2246297ae6
builder-next: adopt new wrapped types
With BuildKit 0.12, some existing types are now required to be wrapped
by new types:

* containerd's LeaseManager and ContentStore have to be a
  (namespace-aware) BuildKit type since f044e0a946
* BuildKit's solver.CacheManager is used instead of
  bboltstorage.CacheKeyStorage since 2b30693409
* The MaxAge config field is a bkconfig.Duration since e06c96274f

Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-09-21 14:18:48 -06:00
Bjorn Neergaard
c217e3c87a
vendor: github.com/moby/buildkit v0.12.2
The following changes were required:
* integration/build: progressui's signature changed in 6b8fbed01e
* builder-next: flightcontrol.Group has become a generic type in 8ffc03b8f0
* builder-next/executor: add github.com/moby/buildkit/executor/resources types, necessitated by 6e87e4b455
* builder-next: stub util/network/Namespace.Sample(), necessitated by 963f16179f

Co-authored-by: CrazyMax <crazy-max@users.noreply.github.com>
Co-authored-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-09-21 14:18:45 -06:00
Bjorn Neergaard
8cd5f04ea3
daemon/c8d: use new containerd LabelDistributionSource constant
Introduced in dd3eedf3c3

Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-09-21 14:18:42 -06:00
Bjorn Neergaard
fd6dd6935b
vendor: github.com/containerd/containerd v1.7.6
The DeepEqual ignore required in the daemon tests is a bit ugly, but it
works given the new protoc output.

We also have to ignore lints related to schema1 deprecations; these do
not apply as we must continue to support this schema version.

Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-09-21 14:18:40 -06:00
Bjorn Neergaard
79a4cbbec9
builder-next: make stub executor generic
The current executor is only tested on Linux, so let's be honest about
that. Stubbing this correctly helps avoid incorrectly trying to call
into Linux-only code in e.g. libnetwork.

Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-09-21 14:18:27 -06:00
Sebastiaan van Stijn
d7a31cfb2d
libnetwork: Sandbox.resolveName: slightly simplify locking
Simplify the lock/unlock cycle, and make the "lookupAlias" branch
more similar to the non-lookupAlias variant.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-21 16:23:36 +02:00
Sebastiaan van Stijn
f549aaa205
libnetwork: Sandbox.resolveName: add fast-path for alias lookups
Skip faster when we're looking for aliases. Also check for the list
of aliases to be empty, not just `nil` (although in practice it should
be equivalent).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-21 16:23:35 +02:00
Sebastiaan van Stijn
9249b34be8
libnetwork: Sandbox.resolveName: rename vars for clarity
- use `nameOrAlias` for the name (or alias) to resolve
- use `lookupAlias` to indicate what the intent is; this function
  is either looking up aliases or "regular" names. Ideally we would
  split the function, but let's keep that for a future exercise.
- name the `ipv6Miss` output variable. The "ipv6 miss" logic is rather
  confusing, and should probably be revisited, but let's start with
  giving the variable a name to make it more apparent what it is.
- use `nw` for networks, which is the more common local name

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-21 16:23:35 +02:00
Sebastiaan van Stijn
4401ccac22
libnetwork: Sandbox: remove some intermediate vars
- remove some intermediate vars, or move them closer to where they're used.
- ResolveService: use strings.SplitN to limit number of elements. This
  code is only used to validate the input, results are not used.
- ResolveService: return early instead of breaking the loop. This makes
  it clearer from the code that were not returning anything (nil, nil).
- Controller.sandboxCleanup(): rename a var, and slight refactor of
  error-handling.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-21 16:23:35 +02:00
Sebastiaan van Stijn
d3afa80b96
Merge pull request #46348 from thaJeztah/refactor_isClusterEligible
libnetwork: rewrite Network.isClusterEligible to return agent
2023-09-21 14:31:31 +02:00
Bjorn Neergaard
0e80073e01
daemon: strongly type containerd log.OutputFormat
This type was introduced in
0a79e67e4f

Make use of it throughout our log-format handling code, and convert back
to a string before we pass it to the containerd client.

Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-09-21 05:40:17 -06:00
Sebastiaan van Stijn
0d9da7367d
Merge pull request #46522 from thaJeztah/update_hcsshim
vendor: github.com/Microsoft/hcsshim v0.11.1
2023-09-21 10:25:18 +02:00
Sebastiaan van Stijn
4ff252456b
libnetwork: rewrite Network.isClusterEligible to return agent
This function was used to check if the network is a multi-host, swarm-scoped
network. Part of this check involved a check whether the cluster-agent was
present.

In all places where this function was used, the next step after checking if
the network was "cluster eligible", was to get the agent, and (again) check
if it was not nil.

This patch rewrites the isClusterEligible utility into a clusterAgent utility,
which both checks if the network is cluster-eligible, and returns the agent
(if set). For convenience, an "ok" bool is added, which callers can use to
return early (although just checking for nilness would likely have been
sufficient).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-21 10:19:21 +02:00
Sebastiaan van Stijn
6203e3660d
libnetwork: Endpoint: return early if no agent was found
This removes redundant nil-checks in Endpoint.deleteServiceInfoFromCluster
and Endpoint.addServiceInfoToCluster.

These functions return early if the network is not ["cluster eligible"][1],
and the function used for that (`Network.isClusterEligible`) requires the
[agent to not be `nil`][2].

This check moved around a few times ([3][3], [4][4]), but was originally
added in [libnetwork 1570][5] which, among others, tried to avoid a nil-pointer
exception reported in [moby 28712][6], which accessed the `Controller.agent`
[without locking][7]. That issue was addressed by adding locks, adding a
`Controller.getAgent` accessor, and updating deleteServiceInfoFromCluster
to use a local var. It also sprinkled this `nil` check to be on the safe
side, but as `Network.isClusterEligible` already checks for the agent
to not be `nil`, this should not be redundant.

[1]: 5b53ddfcdd/libnetwork/agent.go (L529-L534)
[2]: 5b53ddfcdd/libnetwork/agent.go (L688-L696)
[3]: f2307265c7
[4]: 6426d1e66f
[5]: 8dcf9960aa
[6]: https://github.com/moby/moby/issues/28712
[7]: 75fd88ba89/vendor/github.com/docker/libnetwork/agent.go (L452)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-21 10:19:21 +02:00
Sebastiaan van Stijn
6eeef51c6a
libnetwork: Controller.agentSetup: use structured logs
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-21 10:19:15 +02:00
Sebastiaan van Stijn
8b95ea4a35
libnetwork: Controller.agentSetup: remove redundant condition
The function returns at the start if there agent is non-nil.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-21 10:15:06 +02:00
Sebastiaan van Stijn
3a520c1c0f
Merge pull request #46521 from dmcgowan/fix-makefile-graphdriver
Fix graphdriver lookup in makefile
2023-09-21 08:43:43 +02:00
Sebastiaan van Stijn
e852ba55a5
vendor: github.com/Microsoft/hcsshim v0.11.1
full diff: https://github.com/Microsoft/hcsshim/compare/v0.9.10...v0.11.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-21 08:37:39 +02:00
Derek McGowan
06a640084f
Fix graphdriver lookup in makefile
When graphdriver is not provided the graphdriver is looked up
from docker info, but without quotes it may fail and set the
graphdriver to an incorrect value.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-09-20 14:46:43 -07:00
Sebastiaan van Stijn
1ed5d91555
Merge pull request #46365 from thaJeztah/libnetwork_endpoint_nits
libnetwork: Endpoint: fixing some nits
2023-09-20 22:01:28 +02:00
Sebastiaan van Stijn
4dbfe7e17e
Merge pull request #46502 from rumpl/c8d-fix-diff
c8d: Fix `docker diff`
2023-09-20 21:16:08 +02:00
Sebastiaan van Stijn
d8a51d2887
Merge pull request #46463 from thaJeztah/fix_version_checks
client: negotiate api version before handling version-specific code
2023-09-20 19:48:32 +02:00
Djordje Lukic
093481618d
c8d: Skip the test that checks Config.Image
It's not set when containerd is used as an image store and buildkit
never sets it either, so let's skip this test if snapshotters are used

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-09-20 17:01:42 +02:00
Djordje Lukic
7444b88f5d
Merge pull request #46495 from rumpl/c8d-fix-create-platform 2023-09-20 16:39:38 +02:00
Paweł Gronowski
7c865c156f
Merge pull request #46515 from vvoland/c8d-pull-fix-progress-orderd
c8d/pull: Fix stale Downloading progress
2023-09-20 16:38:39 +02:00
Sebastiaan van Stijn
e061793212
Merge pull request #46315 from thaJeztah/libnetwork_remove_sandbox_interface_step2
libnetwork/osl: Namespace: assorted cleanups
2023-09-20 16:13:15 +02:00
Sebastiaan van Stijn
e6907243af
client: negotiate api version before handling version-specific code
We try to perform API-version negotiation as lazy as possible (and only execute
when we are about to make an API request). However, some code requires API-version
dependent handling (to set options, or remove options based on the version of the
API we're using).

Currently this code depended on the caller code to perform API negotiation (or
to configure the API version) first, which may not happen, and because of that
we may be missing options (or set options that are not supported on older API
versions).

This patch:

- splits the code that triggered API-version negotiation to a separate
  Client.checkVersion() function.
- updates NewVersionError to accept a context
- updates NewVersionError to perform API-version negotiation (if enabled)
- updates various Client functions to manually trigger API-version negotiation

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-20 15:23:27 +02:00
Djordje Lukic
207c4d537c c8d: Fix docker diff
Diffing a container yielded some extra changes that come from the
files/directories that we mount inside the container (/etc/resolv.conf
for example). To avoid that we create an intermediate snapshot that has
these files, with this we can now diff the container fs with its parent
and only get the differences that were made inside the container.

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-09-20 14:16:22 +02:00
Paweł Gronowski
601c9b29e6
c8d/pull: Fix stale Downloading progress
Final progress messages were sent after the progress updater finished
which made the "Downloading" progress not being updated into "Download
complete".

Fix by sending the final messages after the progress has finished.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-20 14:10:58 +02:00
Sebastiaan van Stijn
313a090c0e
libnetwork/osl: add some TODOs
These came up during review of a refactor, and need further investigating.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-20 12:45:45 +02:00
Sebastiaan van Stijn
9d3b1f9419
libnetwork/osl: make constructing Interfaces more atomic
It's still not "great", but implement a `newInterface()` constructor
to create a new Interface instance, instead of creating a partial
instance and applying "options" after the fact.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-20 12:45:40 +02:00
Sebastiaan van Stijn
47f9e70385
libnetwork/osl: Namespace.Restore: conditionally fetch IPs
We're only using the results if the interface doesn't have an address
yet, so skip this step if we don't use it.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-20 12:38:27 +02:00
Sebastiaan van Stijn
ee5a91e663
libnetwork/osl: Namespace.Restore: flatten nested conditions
Flatten some nested "if"-statements, and improve error.

Errors returned by this function are not handled, and only logged, so
make them more informative if debugging is needed.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-20 12:38:27 +02:00
Sebastiaan van Stijn
299bd58c5a
libnetwork/osl: Namespace.Restore: rename vars for readability
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-20 12:38:27 +02:00
Sebastiaan van Stijn
7b96663082
libnetwork/osl: Namespace: inline setGateway and setGatewayIPv6
They were not consistently used, and the locations where they were
used were already "setters", so we may as well inline the code.

Also updating Namespace.Restore to keep the lock slightly longer,
instead of locking/unlocking for each property individually, although
we should consider to keep the long for the duration of the whole
function to make it more atomic.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-20 12:38:26 +02:00
Sebastiaan van Stijn
bd17d27658
libnetwork/osl: Namespace: make error-handling more idiomatic
Check for non-nil errors (and return early) instead of the reverse.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-20 12:38:26 +02:00
Sebastiaan van Stijn
0b4a70ca2c
libnetwork/osl: Namespace: programRoute, removeRoute rm path arg
Remove the argument, because it was not used.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-20 12:38:26 +02:00
Sebastiaan van Stijn
542fe0da40
libnetwork/osl: Namespace: make mutex private
Make the mutex internal to the Namespace; locking/unlocking should not
be done externally, and this makes it easier to see where it's used.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-20 12:38:26 +02:00
Sebastiaan van Stijn
338fc49060
libnetwork/osl: implement Namespace.RemoveInterface
Interface.Remove() was directly accessing Namespace "internals", such
as locking/unlocking. Move the code from Interface.Remove() into the
Namespace instead.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-20 12:34:47 +02:00
Sebastiaan van Stijn
7cfb81ba04
Merge pull request #46342 from thaJeztah/libnetwork_nwAgent_ip
libnetwork: nwAgent.bindAddr: change to net.IP
2023-09-20 10:27:06 +02:00
Sebastiaan van Stijn
34dc2f10b4
Merge pull request #46479 from vvoland/pull-check-reserved-name-in-api
api/pull: Move reference parsing from imageService, validate repo
2023-09-19 17:17:31 +02:00