Commit graph

3739 commits

Author SHA1 Message Date
Sebastiaan van Stijn
cff4f20c44
migrate to github.com/containerd/log v0.1.0
The github.com/containerd/containerd/log package was moved to a separate
module, which will also be used by upcoming (patch) releases of containerd.

This patch moves our own uses of the package to use the new module.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-11 17:52:23 +02:00
Albin Kerouanton
430a5f8581
api: Add method and path to trace operation string
Currently, all traces coming from the API have an empty operation
string, which make them indistinguishable from each other without looking
at the logs of the root span, and prevent proper filtering on Jaeger UI.

With this change, traces get the route pattern as the operation string.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-09-26 18:56:25 +02:00
Drew Erny
42a51cb285 Add support for swarm seccomp and apparmor
And also no-new-privileges

Signed-off-by: Drew Erny <derny@mirantis.com>
2023-09-25 12:38:26 -05:00
Sebastiaan van Stijn
34dc2f10b4
Merge pull request #46479 from vvoland/pull-check-reserved-name-in-api
api/pull: Move reference parsing from imageService, validate repo
2023-09-19 17:17:31 +02:00
Sebastiaan van Stijn
f13065b1f7
Merge pull request #46183 from akerouanton/validate-NetworkingConfig
daemon: Improve NetworkingConfig & EndpointSettings validation
2023-09-18 22:09:29 +02:00
Albin Kerouanton
acf825def2
api/t/net: test EndpointIPAMConfig.Validate()
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-09-18 18:26:56 +02:00
Albin Kerouanton
3092b261e2
daemon: move most of validateEndpointSettings into api/t/net
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-09-18 18:26:56 +02:00
Albin Kerouanton
81ab8db1c3
api/t/net: add missing comment to ValidateIPAM
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-09-18 17:28:08 +02:00
Albin Kerouanton
04a47e88d2
api/t/net: move endpoint structs into endpoint.go
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-09-18 17:28:08 +02:00
Albin Kerouanton
ff503882f7
daemon: Improve NetworkingConfig & EndpointSettings validation
So far, only a subset of NetworkingConfig was validated when calling
ContainerCreate. Other parameters would be validated when the container
was started. And the same goes for EndpointSettings on NetworkConnect.

This commit adds two validation steps:

1. Check if the IP addresses set in endpoint's IPAMConfig are valid,
   when ContainerCreate and ConnectToNetwork is called ;
2. Check if the network allows static IP addresses, only on
   ConnectToNetwork as we need the libnetwork's Network for that and it
   might not exist until NetworkAttachment requests are sent to the
   Swarm leader (which happens only when starting the container) ;

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-09-18 17:21:06 +02:00
Paweł Gronowski
0c4397deaf
api/pull: Validate repo name
Copy the check for "scratch" image pull attempt from the distribution
to the API.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-18 15:23:13 +02:00
Paweł Gronowski
ccb37fa567
imageService/PullImage: Move reference parse to api
Make `PullImage` accept `reference.Named` directly instead of
duplicating the parsing code for both graphdriver and containerd image
service implementations.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-18 15:23:12 +02:00
Djordje Lukic
5a4962f80d
c8d: Refuse images with digest algo when tagging
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-09-16 00:13:44 +02:00
Albin Kerouanton
5e15ed314b
api: Improve error on ContainerCreate with multiple endpoints
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-09-15 14:30:03 +02:00
Albin Kerouanton
bbcd662532
api: Allow ContainerCreate to take several EndpointsConfig for >= 1.44
The API endpoint `/containers/create` accepts several EndpointsConfig
since v1.22 but the daemon would error out in such case. This check is
moved from the daemon to the api and is now applied only for API < 1.44,
effectively allowing the daemon to create containers connected to
several networks.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-09-15 10:07:29 +02:00
Albin Kerouanton
78479b1915
libnet: Make sure network names are unique
Fixes #18864, #20648, #33561, #40901.

[This GH comment][1] makes clear network name uniqueness has never been
enforced due to the eventually consistent nature of Classic Swarm
datastores:

> there is no guaranteed way to check for duplicates across a cluster of
> docker hosts.

And this is further confirmed by other comments made by @mrjana in that
same issue, eg. [this one][2]:

> we want to adopt a schema which can pave the way in the future for a
> completely decentralized cluster of docker hosts (if scalability is
> needed).

This decentralized model is what Classic Swarm was trying to be. It's
been superseded since then by Docker Swarm, which has a centralized
control plane.

To circumvent this drawback, the `NetworkCreate` endpoint accepts a
`CheckDuplicate` flag. However it's not perfectly reliable as it won't
catch concurrent requests.

Due to this design decision, API clients like Compose have to implement
workarounds to make sure names are really unique (eg.
docker/compose#9585). And the daemon itself has seen a string of issues
due to that decision, including some that aren't fixed to this day (for
instance moby/moby#40901):

> The problem is, that if you specify a network for a container using
> the ID, it will add that network to the container but it will then
> change it to reference the network by using the name.

To summarize, this "feature" is broken, has no practical use and is a
source of pain for Docker users and API consumers. So let's just remove
it for _all_ API versions.

[1]: https://github.com/moby/moby/issues/18864#issuecomment-167201414
[2]: https://github.com/moby/moby/issues/18864#issuecomment-167202589

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-09-12 10:40:13 +02:00
Albin Kerouanton
5d5eeac310
daemon: automatically set network EnableIPv6 if needed
PR 4f47013feb added a validation step to `NetworkCreate` to ensure
no IPv6 subnet could be set on a network if its `EnableIPv6` parameter
is false.

Before that, the daemon was accepting such request but was doing nothing
with the IPv6 subnet.

This validation step is now deleted, and we automatically set
`EnableIPv6` if an IPv6 subnet was specified.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-09-11 20:53:29 +02:00
Brian Goff
642e9917ff Add otel support
This uses otel standard environment variables to configure tracing in
the daemon.
It also adds support for propagating trace contexts in the client and
reading those from the API server.

See
https://opentelemetry.io/docs/specs/otel/configuration/sdk-environment-variables/
for details on otel environment variables.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2023-09-07 18:38:19 +00:00
Sebastiaan van Stijn
1148a24e64
migrate to new github.com/distribution/reference module
The "reference" package was moved to a separate module, which was extracted
from b9b19409cf

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-05 12:09:26 +02:00
Sebastiaan van Stijn
9c4e82435e
Merge pull request #46351 from thaJeztah/api_events_actions_enum
api/types/events: define "Action" type and consts
2023-09-05 11:11:42 +02:00
Bjorn Neergaard
74fafbda51
Merge pull request #46265 from dvdksn/docs-rephrase-images-create
docs(api): rephrase description for /images/create
2023-09-01 07:04:27 -06:00
David Karlsson
237eb3b5f1 docs(api): rephrase description for /images/create
Updated the description to clarify that this is the endpoint to use if
you want to pull an image.

Signed-off-by: David Karlsson <35727626+dvdksn@users.noreply.github.com>
2023-08-31 21:05:43 +02:00
Paweł Gronowski
366a5f1d74
image/spec: Add Go structs
Add Go structs describing the image spec which extend the OCI types.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-08-31 12:29:45 +02:00
Sebastiaan van Stijn
0f871f8cb7
api/types/events: define "Action" type and consts
Define consts for the Actions we use for events, instead of "ad-hoc" strings.
Having these consts makes it easier to find where specific events are triggered,
makes the events less error-prone, and allows documenting each Action (if needed).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-29 00:38:08 +02:00
Sebastiaan van Stijn
a65c948e7e
Merge pull request #46335 from thaJeztah/api_move_checkpoint_types
api/types: move checkpoint-types to api/types/checkpoint
2023-08-28 19:02:19 +02:00
Sebastiaan van Stijn
8309206160
Merge pull request #46350 from thaJeztah/strongtype_eventstype
api/types/events: make events.Type an actual type
2023-08-28 16:44:26 +02:00
Sebastiaan van Stijn
7f6dda359b
Merge pull request #46352 from thaJeztah/restart_policy_improve_error
api: ValidateRestartPolicy: improve errors for invalid policies
2023-08-28 15:14:11 +02:00
Sebastiaan van Stijn
328c6159f2
Merge pull request #46334 from thaJeztah/remove_empty_file
api/server/router/network: remove empty file
2023-08-28 14:40:52 +02:00
Sebastiaan van Stijn
f6f6c32138
api: ValidateRestartPolicy: improve errors for invalid policies
Make the error message slightly clearer on "what" part is not valid,
and provide suggestions on what are acceptable values.

Before this change:

    docker create --restart=always:3 busybox
    Error response from daemon: invalid restart policy: maximum retry count cannot be used with restart policy 'always'

    docker create --restart=always:-1 busybox
    Error response from daemon: invalid restart policy: maximum retry count cannot be used with restart policy 'always'

    docker create --restart=unknown busybox
    Error response from daemon: invalid restart policy 'unknown'

After this change:

    docker create --restart=always:3 busybox
    Error response from daemon: invalid restart policy: maximum retry count can only be used with 'on-failure'

    docker create --restart=always:-1 busybox
    Error response from daemon: invalid restart policy: maximum retry count can only be used with 'on-failure' and cannot be negative

    docker create --restart=unknown busybox
    Error response from daemon: invalid restart policy: unknown policy 'unknown'; use one of 'no', 'always', 'on-failure', or 'unless-stopped'

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-28 14:00:58 +02:00
Sebastiaan van Stijn
70ad5b818f
api/types/events: make events.Type an actual type
This type was added in 247f4796d2, and
at the time was added as an alias for string;

> api/types/events: add "Type" type for event-type enum
>
> Currently just an alias for string, but we can change it to be an
> actual type.

Now that all code uses the defined types, we should be able to make
this an actual type.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-28 13:12:38 +02:00
Sebastiaan van Stijn
318b3d4fe5
api/types/versions: rename max/min as it collides with go1.21 builtin
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-26 19:38:47 +02:00
Sebastiaan van Stijn
385f6429be
api/server/router/network: remove empty file
The content of this file was removed in c0bc14e8dd,
and all it container since was the package name.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-26 12:51:19 +02:00
Sebastiaan van Stijn
b688af2226
api/types: move checkpoint-types to api/types/checkpoint
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-26 12:37:41 +02:00
Sebastiaan van Stijn
136e86bb5c
api: swagger: update link to Go documentation
Go documentation moved to the `go.dev` domain;

    curl -sI https://golang.org/doc/install/source#environment | grep 'location'
    location: https://go.dev/doc/install/source

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-25 12:43:30 +02:00
Sebastiaan van Stijn
d52f20048d
api/server: format code with gofumpt
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-24 18:00:19 +02:00
Sebastiaan van Stijn
70ae5c13ea
Merge pull request #44704 from thaJeztah/api_hostconfig_RestartPolicyMode_enum
api/types/container: add RestartPolicyMode type and enum
2023-08-22 22:31:51 +02:00
Albin Kerouanton
4f47013feb
api: Validate IPAM config before creating a network
Currently, IPAM config is never validated by the API. Some checks
are done by the CLI, but they're not exhaustive. And some of these
misconfigurations might be caught early by libnetwork (ie. when the
network is created), and others only surface when connecting a container
to a misconfigured network. In both cases, the API would return a 500.

Although the `NetworkCreate` endpoint might already return warnings,
these are never displayed by the CLI. As such, it was decided during a
maintainer's call to return validation errors _for all API versions_.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-22 17:11:54 +02:00
Sebastiaan van Stijn
2be118379e
api/types/container: add RestartPolicyMode type and enum
Also move the validation function to live with the type definition,
which allows it to be used outside of the daemon as well.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-22 16:40:57 +02:00
Albin Kerouanton
d146e592d8
api/t/network: Move IPAM types to their own file
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-08-19 11:56:28 +02:00
Sebastiaan van Stijn
f9cae2acbe
Merge pull request #46165 from akerouanton/remove-api-CheckDuplicate-warning
api: Remove duplicated check on CheckDuplicate
2023-08-08 13:42:17 +02:00
Albin Kerouanton
40adb4317b
api: Remove duplicated check on CheckDuplicate
Partially revert commit 94b880f.

The CheckDuplicate field has been introduced in commit 2ab94e1. At that
time, this check was done in the network router. It was then moved to
the daemon package in commit 3ca2982. However, commit 94b880f duplicated
the logic into the network router for no apparent reason. Finally,
commit ab18718 made sure a 409 would be returned instead of a 500.

As this logic is first done by the daemon, the error -> warning
conversion can't happen because CheckDuplicate has to be true for the
daemon package to return an error. If it's false, the daemon proceed
with the network creation, set the Warning field of its return value and
return no error.

Thus, the CheckDuplicate logic in the api is removed and
libnetwork.NetworkNameError now implements the ErrConflict interface.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-08-08 10:00:02 +02:00
Evan Lezar
7a59913b1a Add CDISpecDirs to Info output
This change adds the configured CDI spec directories to the
system info output.

Signed-off-by: Evan Lezar <elezar@nvidia.com>
2023-08-04 11:46:34 +02:00
Sebastiaan van Stijn
69c9adb7d3
Merge pull request #45925 from thaJeztah/deprecate_is_automated
api: search: deprecate is_automated field, and is-automated filter
2023-08-01 16:27:00 +02:00
Sebastiaan van Stijn
971083d419
api: search: deprecate is_automated field, and is-automated filter
The is-automated field is being deprecated by Docker Hub's search API,
and will always be set to "false" in future.

This patch deprecates the field and related filter for the Engine's API.

In future, the `is-automated` filter will no longer yield any results
when searching for `is-automated=true`, and will be ignored when
searching for `is-automated=false`.

Given that this field is deprecated by an external API, the deprecation
will not be versioned, and will apply to any API version.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-01 13:45:44 +02:00
Sebastiaan van Stijn
5e2a1195d7
swap logrus types for their containerd/logs aliases
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-01 13:02:55 +02:00
Sebastiaan van Stijn
077208287b
Merge pull request #46106 from thaJeztah/fix_filter_errors
api/types/filters: fix errors not being matched by errors.Is()
2023-07-29 03:20:01 +02:00
Sebastiaan van Stijn
6ec03d6745
libnetwork: move datastore Scope consts to libnetwork/scope
The "Capability" type defines DataScope and ConnectivityScope fields,
but their value was set from consts in the datastore package, which
required importing that package and its dependencies for the consts
only.

This patch:

- Moves the consts to a separate "scope" package
- Adds aliases for the consts in the datastore package.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-28 21:56:48 +02:00
Sebastiaan van Stijn
490fee7d45
api/types/filters: fix errors not being matched by errors.Is()
I found that the errors returned weren't matched with `errors.Is()` when
wrapped.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-28 17:07:41 +02:00
Sebastiaan van Stijn
d713b7ed45
api/server/router/network: remove unused Backend.FindNetwork
This function was used by the postNetworkConnect() handler, but is handled
by the backend itself, starting with d63a5a1ff5.
Since that commit, this function was no longer used, so we can remove it.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-25 16:33:57 +02:00
Sebastiaan van Stijn
64c6f72988
libnetwork: remove Network interface
There's only one implementation; drop the interface and use the
concrete type instead.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-22 11:56:41 +02:00