Commit graph

48607 commits

Author SHA1 Message Date
Sebastiaan van Stijn
605c8fb75d
Merge pull request #46546 from thaJeztah/libnetwork_return_errs
libnetwork: Controller.cleanupLocalEndpoints, sandboxCleanup: return errors
2023-09-27 10:31:56 +02:00
Sebastiaan van Stijn
324cb3d08f
Merge pull request #46545 from thaJeztah/libnetwork_NetworkByID_simplify
libnetwork: Controller.NetworkByID: remove redundant error-handling
2023-09-27 10:30:47 +02:00
Sebastiaan van Stijn
f3143745b2
Merge pull request #46547 from thaJeztah/libnetwork_store_nolock
libnetwork: Controller: remove mutex for "store"
2023-09-27 10:23:32 +02:00
Sebastiaan van Stijn
b1855bb4af
Merge pull request #46548 from thaJeztah/libnetwork_inline_populateSpecial
libnetwork: inline populateSpecial NetworkWalker
2023-09-27 10:13:15 +02:00
Sebastiaan van Stijn
479079c6b1
Merge pull request #46549 from thaJeztah/nwAgent_unexport_mutex
libnetwork: nwAgent: un-export mutex
2023-09-27 10:12:23 +02:00
Xinfeng Liu
95aea39348 Make one-shot stats faster
This commit moves one-shot stats processing out of the publishing
channels, i.e. collect stats directly.

Also changes the method of getSystemCPUUsage() on Linux to return
number of online CPUs also.

Signed-off-by: Xinfeng Liu <XinfengLiu@icloud.com>
2023-09-27 14:10:23 +08:00
Bjorn Neergaard
97e28de7e2
Merge pull request #46544 from akerouanton/api-otel-operation
api: Add method and path to trace operation string
2023-09-26 13:07:03 -06:00
Sebastiaan van Stijn
618d9b5d54
libnetwork: nwAgent: un-export mutex
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-26 19:46:27 +02:00
Sebastiaan van Stijn
7cda3fb7b5
libnetwork: inline populateSpecial NetworkWalker
It was only used in a single place, and it was defined far away from
where it was used.

Move the code inline, so that it's clear at a glance what it's doing.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-26 19:41:50 +02:00
Sebastiaan van Stijn
ca1307c56e
libnetwork: Controller: remove mutex for "store"
The store field is only mutated by Controller.initStores(), which is
only called inside the cosntructor (libnetwork.New), so there should be
no need to protect the field with a mutex in non-exported functions.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-26 19:34:12 +02:00
Sebastiaan van Stijn
a8ea752a93
libnetwork: Controller.cleanupLocalEndpoints: return errors
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-26 19:28:18 +02:00
Sebastiaan van Stijn
2e60051c92
libnetwork: Controller.sandboxCleanup: return errors
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-26 19:28:18 +02:00
Sebastiaan van Stijn
642cf261a8
libnetwork: Controller.NetworkByID: remove redundant error-handling
Controller.getNetworkFromStore() already returns a ErrNoSuchNetwork if
no network was found, so we don't need to convert the existing error.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-26 19:22:52 +02:00
Sebastiaan van Stijn
e3975fba84
Merge pull request #46530 from thaJeztah/libnetwork_walkless_step1
libnetwork: assorted cleanups in Sandbox
2023-09-26 19:19:31 +02:00
Sebastiaan van Stijn
7f9cce2b50
Merge pull request #46541 from akerouanton/delve-integration-tests
hack: Run integration tests through Delve
2023-09-26 19:05:48 +02:00
Albin Kerouanton
430a5f8581
api: Add method and path to trace operation string
Currently, all traces coming from the API have an empty operation
string, which make them indistinguishable from each other without looking
at the logs of the root span, and prevent proper filtering on Jaeger UI.

With this change, traces get the route pattern as the operation string.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-09-26 18:56:25 +02:00
Albin Kerouanton
41c186c344
hack: Run integration tests through Delve
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-09-26 15:42:11 +02:00
Sebastiaan van Stijn
b94e88c1e2
Merge pull request #46386 from dperny/add-swarm-seccomp-apparmor
Add support for swarm seccomp and apparmor
2023-09-26 09:45:16 +02:00
Drew Erny
42a51cb285 Add support for swarm seccomp and apparmor
And also no-new-privileges

Signed-off-by: Drew Erny <derny@mirantis.com>
2023-09-25 12:38:26 -05:00
Sebastiaan van Stijn
bd7b27b5c8
Merge pull request #46535 from vvoland/integration-cli-saveload-dont-use-repositories
integration-cli/SaveLoad: Don't check `repositories` file
2023-09-25 17:22:28 +02:00
Paweł Gronowski
3d62ab9585
Merge pull request #46536 from vvoland/integration-cli-saveload-repoflags
integration-cli/TestSaveAndLoadRepoFlags: Ignore LastTagTime difference
2023-09-25 11:59:37 +02:00
Paweł Gronowski
7f6bd671fe
Merge pull request #46484 from vchiranjeeviak/vchiranjeeviak/container-list-code-refactor
daemon: containers list code refactor
2023-09-25 11:34:51 +02:00
Chiranjeevi Tirunagari
0c66111c40
daemon: containers list code refactor
Signed-off-by: Chiranjeevi Tirunagari <vchiranjeeviak.tirunagari@gmail.com>
2023-09-25 10:28:31 +05:30
Djordje Lukic
51f0e7b0e4
Merge pull request #46534 from vvoland/c8d-integration-cli-saveload-skip-parent 2023-09-22 16:49:23 +02:00
Paweł Gronowski
13b775a96e
integration-cli/TestSaveAndLoadRepoFlags: Ignore LastTagTime difference
The reason it doesn't change with the graphdrivers is caused by an
implementation detail and the fact that the image is loaded into the
same daemon it was saved from.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-22 16:35:07 +02:00
Paweł Gronowski
7c202d2fa4
integration-cli/SaveLoad: Don't check repositories file
Rewrite TestSaveMultipleNames and TestSaveSingleTag  so that they don't
use legacy `repositories` file (which isn't present in the OCI
archives).
`docker save` output is now OCI compatible, so we don't need
to use the legacy file.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-22 15:42:50 +02:00
Paweł Gronowski
724db8650c
integration-cli: Skip TestLoadZeroSizeLayer with c8d
The input archive is in the old Docker format that's not OCI compatible
and is not supported by the containerd archive import:
```
17d1436ef796af2fc2210cc37c4672e5aa1b62cb08ac4b95dd15372321105a66/
17d1436ef796af2fc2210cc37c4672e5aa1b62cb08ac4b95dd15372321105a66/VERSION
17d1436ef796af2fc2210cc37c4672e5aa1b62cb08ac4b95dd15372321105a66/json
17d1436ef796af2fc2210cc37c4672e5aa1b62cb08ac4b95dd15372321105a66/layer.tar
25445a0fc5025c3917a0cd6e307d92322540e0da691614312ddea22511b71513/
25445a0fc5025c3917a0cd6e307d92322540e0da691614312ddea22511b71513/VERSION
25445a0fc5025c3917a0cd6e307d92322540e0da691614312ddea22511b71513/json
25445a0fc5025c3917a0cd6e307d92322540e0da691614312ddea22511b71513/layer.tar
9c7cb910d84346a3fbf3cc2be046f44bf0af7f11eb8db2ef1f45e93c1202faac/
9c7cb910d84346a3fbf3cc2be046f44bf0af7f11eb8db2ef1f45e93c1202faac/VERSION
9c7cb910d84346a3fbf3cc2be046f44bf0af7f11eb8db2ef1f45e93c1202faac/json
9c7cb910d84346a3fbf3cc2be046f44bf0af7f11eb8db2ef1f45e93c1202faac/layer.tar
repositories
```

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-22 15:34:02 +02:00
Paweł Gronowski
1337a66130
integration-cli: Skip Parent tests with c8d
Parent property is not available in the OCI config.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-22 15:33:47 +02:00
Sebastiaan van Stijn
3614749b55
Merge pull request #45966 from neersighted/buildkit_0.12
Update to BuildKit 0.12
2023-09-22 02:13:15 +02:00
Sebastiaan van Stijn
90b756b2a1
Merge pull request #46525 from thaJeztah/daemon_typed_logformat
daemon: strongly type containerd log.OutputFormat
2023-09-21 22:27:02 +02:00
CrazyMax
d5b067e04a
ci(buildkit): remove regex skipping tests with digest inconsistency
Skipping digest-related tests is no longer necessary after 4065831652

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-09-21 14:25:28 -06:00
CrazyMax
286704bf6f
ci(buildkit): expose github runtime for gha tests
This exposes `ACTIONS_RUNTIME_TOKEN` and `ACTIONS_CACHE_URL`, which are
used to skip cache exporter tests, when combined with
a8789cbd4a

Co-authored-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-09-21 14:25:10 -06:00
Bjorn Neergaard
34e71b374b
builder-next/a/ci/pull: strongly type flightcontrol.Group
Now that this is a generic, we can define a struct type at the package
level, and remove the casting logic necessary when we had to use
interface{}.

Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-09-21 14:18:57 -06:00
Bjorn Neergaard
c8773e10b1
builder-next/wrapper: use new exptypes constants
Introduced in 4fc2d7b5e7

Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-09-21 14:18:55 -06:00
Bjorn Neergaard
a728bd0ee9
builder-next/pull: use ResolveMode.String() helper
Introduced years ago in 6644e1b0da

Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-09-21 14:18:52 -06:00
Bjorn Neergaard
221b73e23d
builder-next: apply source policies on config resolve
SourcePolicy was accounted for in 330cf7ae7d

TODO: replace applySourcePolicies with BuildKit's implementation, which
is currently unexported.

Co-authored-by: Tonis Tiigi <tonistiigi@gmail.com>
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-09-21 14:18:50 -06:00
Bjorn Neergaard
2246297ae6
builder-next: adopt new wrapped types
With BuildKit 0.12, some existing types are now required to be wrapped
by new types:

* containerd's LeaseManager and ContentStore have to be a
  (namespace-aware) BuildKit type since f044e0a946
* BuildKit's solver.CacheManager is used instead of
  bboltstorage.CacheKeyStorage since 2b30693409
* The MaxAge config field is a bkconfig.Duration since e06c96274f

Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-09-21 14:18:48 -06:00
Bjorn Neergaard
c217e3c87a
vendor: github.com/moby/buildkit v0.12.2
The following changes were required:
* integration/build: progressui's signature changed in 6b8fbed01e
* builder-next: flightcontrol.Group has become a generic type in 8ffc03b8f0
* builder-next/executor: add github.com/moby/buildkit/executor/resources types, necessitated by 6e87e4b455
* builder-next: stub util/network/Namespace.Sample(), necessitated by 963f16179f

Co-authored-by: CrazyMax <crazy-max@users.noreply.github.com>
Co-authored-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-09-21 14:18:45 -06:00
Bjorn Neergaard
8cd5f04ea3
daemon/c8d: use new containerd LabelDistributionSource constant
Introduced in dd3eedf3c3

Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-09-21 14:18:42 -06:00
Bjorn Neergaard
fd6dd6935b
vendor: github.com/containerd/containerd v1.7.6
The DeepEqual ignore required in the daemon tests is a bit ugly, but it
works given the new protoc output.

We also have to ignore lints related to schema1 deprecations; these do
not apply as we must continue to support this schema version.

Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-09-21 14:18:40 -06:00
Bjorn Neergaard
79a4cbbec9
builder-next: make stub executor generic
The current executor is only tested on Linux, so let's be honest about
that. Stubbing this correctly helps avoid incorrectly trying to call
into Linux-only code in e.g. libnetwork.

Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-09-21 14:18:27 -06:00
Sebastiaan van Stijn
d7a31cfb2d
libnetwork: Sandbox.resolveName: slightly simplify locking
Simplify the lock/unlock cycle, and make the "lookupAlias" branch
more similar to the non-lookupAlias variant.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-21 16:23:36 +02:00
Sebastiaan van Stijn
f549aaa205
libnetwork: Sandbox.resolveName: add fast-path for alias lookups
Skip faster when we're looking for aliases. Also check for the list
of aliases to be empty, not just `nil` (although in practice it should
be equivalent).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-21 16:23:35 +02:00
Sebastiaan van Stijn
9249b34be8
libnetwork: Sandbox.resolveName: rename vars for clarity
- use `nameOrAlias` for the name (or alias) to resolve
- use `lookupAlias` to indicate what the intent is; this function
  is either looking up aliases or "regular" names. Ideally we would
  split the function, but let's keep that for a future exercise.
- name the `ipv6Miss` output variable. The "ipv6 miss" logic is rather
  confusing, and should probably be revisited, but let's start with
  giving the variable a name to make it more apparent what it is.
- use `nw` for networks, which is the more common local name

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-21 16:23:35 +02:00
Sebastiaan van Stijn
4401ccac22
libnetwork: Sandbox: remove some intermediate vars
- remove some intermediate vars, or move them closer to where they're used.
- ResolveService: use strings.SplitN to limit number of elements. This
  code is only used to validate the input, results are not used.
- ResolveService: return early instead of breaking the loop. This makes
  it clearer from the code that were not returning anything (nil, nil).
- Controller.sandboxCleanup(): rename a var, and slight refactor of
  error-handling.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-21 16:23:35 +02:00
Sebastiaan van Stijn
d3afa80b96
Merge pull request #46348 from thaJeztah/refactor_isClusterEligible
libnetwork: rewrite Network.isClusterEligible to return agent
2023-09-21 14:31:31 +02:00
Bjorn Neergaard
0e80073e01
daemon: strongly type containerd log.OutputFormat
This type was introduced in
0a79e67e4f

Make use of it throughout our log-format handling code, and convert back
to a string before we pass it to the containerd client.

Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-09-21 05:40:17 -06:00
Sebastiaan van Stijn
0d9da7367d
Merge pull request #46522 from thaJeztah/update_hcsshim
vendor: github.com/Microsoft/hcsshim v0.11.1
2023-09-21 10:25:18 +02:00
Sebastiaan van Stijn
4ff252456b
libnetwork: rewrite Network.isClusterEligible to return agent
This function was used to check if the network is a multi-host, swarm-scoped
network. Part of this check involved a check whether the cluster-agent was
present.

In all places where this function was used, the next step after checking if
the network was "cluster eligible", was to get the agent, and (again) check
if it was not nil.

This patch rewrites the isClusterEligible utility into a clusterAgent utility,
which both checks if the network is cluster-eligible, and returns the agent
(if set). For convenience, an "ok" bool is added, which callers can use to
return early (although just checking for nilness would likely have been
sufficient).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-21 10:19:21 +02:00
Sebastiaan van Stijn
6203e3660d
libnetwork: Endpoint: return early if no agent was found
This removes redundant nil-checks in Endpoint.deleteServiceInfoFromCluster
and Endpoint.addServiceInfoToCluster.

These functions return early if the network is not ["cluster eligible"][1],
and the function used for that (`Network.isClusterEligible`) requires the
[agent to not be `nil`][2].

This check moved around a few times ([3][3], [4][4]), but was originally
added in [libnetwork 1570][5] which, among others, tried to avoid a nil-pointer
exception reported in [moby 28712][6], which accessed the `Controller.agent`
[without locking][7]. That issue was addressed by adding locks, adding a
`Controller.getAgent` accessor, and updating deleteServiceInfoFromCluster
to use a local var. It also sprinkled this `nil` check to be on the safe
side, but as `Network.isClusterEligible` already checks for the agent
to not be `nil`, this should not be redundant.

[1]: 5b53ddfcdd/libnetwork/agent.go (L529-L534)
[2]: 5b53ddfcdd/libnetwork/agent.go (L688-L696)
[3]: f2307265c7
[4]: 6426d1e66f
[5]: 8dcf9960aa
[6]: https://github.com/moby/moby/issues/28712
[7]: 75fd88ba89/vendor/github.com/docker/libnetwork/agent.go (L452)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-21 10:19:21 +02:00