Commit graph

733 commits

Author SHA1 Message Date
Drew Erny
42a51cb285 Add support for swarm seccomp and apparmor
And also no-new-privileges

Signed-off-by: Drew Erny <derny@mirantis.com>
2023-09-25 12:38:26 -05:00
Paweł Gronowski
0c4397deaf
api/pull: Validate repo name
Copy the check for "scratch" image pull attempt from the distribution
to the API.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-18 15:23:13 +02:00
Paweł Gronowski
ccb37fa567
imageService/PullImage: Move reference parse to api
Make `PullImage` accept `reference.Named` directly instead of
duplicating the parsing code for both graphdriver and containerd image
service implementations.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-18 15:23:12 +02:00
Djordje Lukic
5a4962f80d
c8d: Refuse images with digest algo when tagging
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-09-16 00:13:44 +02:00
Albin Kerouanton
5e15ed314b
api: Improve error on ContainerCreate with multiple endpoints
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-09-15 14:30:03 +02:00
Albin Kerouanton
bbcd662532
api: Allow ContainerCreate to take several EndpointsConfig for >= 1.44
The API endpoint `/containers/create` accepts several EndpointsConfig
since v1.22 but the daemon would error out in such case. This check is
moved from the daemon to the api and is now applied only for API < 1.44,
effectively allowing the daemon to create containers connected to
several networks.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-09-15 10:07:29 +02:00
Brian Goff
642e9917ff Add otel support
This uses otel standard environment variables to configure tracing in
the daemon.
It also adds support for propagating trace contexts in the client and
reading those from the API server.

See
https://opentelemetry.io/docs/specs/otel/configuration/sdk-environment-variables/
for details on otel environment variables.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2023-09-07 18:38:19 +00:00
Sebastiaan van Stijn
1148a24e64
migrate to new github.com/distribution/reference module
The "reference" package was moved to a separate module, which was extracted
from b9b19409cf

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-05 12:09:26 +02:00
Sebastiaan van Stijn
a65c948e7e
Merge pull request #46335 from thaJeztah/api_move_checkpoint_types
api/types: move checkpoint-types to api/types/checkpoint
2023-08-28 19:02:19 +02:00
Sebastiaan van Stijn
385f6429be
api/server/router/network: remove empty file
The content of this file was removed in c0bc14e8dd,
and all it container since was the package name.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-26 12:51:19 +02:00
Sebastiaan van Stijn
b688af2226
api/types: move checkpoint-types to api/types/checkpoint
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-26 12:37:41 +02:00
Sebastiaan van Stijn
d52f20048d
api/server: format code with gofumpt
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-24 18:00:19 +02:00
Albin Kerouanton
40adb4317b
api: Remove duplicated check on CheckDuplicate
Partially revert commit 94b880f.

The CheckDuplicate field has been introduced in commit 2ab94e1. At that
time, this check was done in the network router. It was then moved to
the daemon package in commit 3ca2982. However, commit 94b880f duplicated
the logic into the network router for no apparent reason. Finally,
commit ab18718 made sure a 409 would be returned instead of a 500.

As this logic is first done by the daemon, the error -> warning
conversion can't happen because CheckDuplicate has to be true for the
daemon package to return an error. If it's false, the daemon proceed
with the network creation, set the Warning field of its return value and
return no error.

Thus, the CheckDuplicate logic in the api is removed and
libnetwork.NetworkNameError now implements the ErrConflict interface.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-08-08 10:00:02 +02:00
Sebastiaan van Stijn
5e2a1195d7
swap logrus types for their containerd/logs aliases
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-01 13:02:55 +02:00
Sebastiaan van Stijn
6ec03d6745
libnetwork: move datastore Scope consts to libnetwork/scope
The "Capability" type defines DataScope and ConnectivityScope fields,
but their value was set from consts in the datastore package, which
required importing that package and its dependencies for the consts
only.

This patch:

- Moves the consts to a separate "scope" package
- Adds aliases for the consts in the datastore package.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-28 21:56:48 +02:00
Sebastiaan van Stijn
d713b7ed45
api/server/router/network: remove unused Backend.FindNetwork
This function was used by the postNetworkConnect() handler, but is handled
by the backend itself, starting with d63a5a1ff5.
Since that commit, this function was no longer used, so we can remove it.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-25 16:33:57 +02:00
Sebastiaan van Stijn
64c6f72988
libnetwork: remove Network interface
There's only one implementation; drop the interface and use the
concrete type instead.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-22 11:56:41 +02:00
Brian Goff
f93cfb2e31 Support for health start interval to swarm mode
Adds conversions for health start interval to/from grpc for swarmkit.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2023-07-14 16:47:10 +00:00
Sebastiaan van Stijn
c90229ed9a
api/types: move system info types to api/types/system
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-07 13:01:36 +02:00
Bjorn Neergaard
e4c866f387
Merge pull request #40894 from cpuguy83/health_start_interval
Add health start interval
2023-07-05 19:47:41 -06:00
Brian Goff
2216d3ca8d Add health start interval
This adds an additional interval to be used by healthchecks during the
start period.
Typically when a container is just starting you want to check if it is
ready more quickly than a typical healthcheck might run. Without this
users have to balance between running healthchecks to frequently vs
taking a very long time to mark a container as healthy for the first
time.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-05 23:44:17 +00:00
Bjorn Neergaard
e77f9d2327
Merge pull request #45877 from thaJeztah/imports_and_vars
fix some minor linting issues
2023-07-05 15:54:57 -06:00
Sebastiaan van Stijn
a1c9a686b0
api: use singleflight for /info endpoint
Prevent potential suggestion when many concurrent requests happen on
the /info endpoint. It's worth noting that with this change,
requests to the endpoint while another request is still in flight
will share the results, hence might be slightly incorrect (for example,
the output includes SystemTime, which may now be incorrect).

Assuming that under normal circumstances, requests will still
happen fast enough to not be shared, this may not be a problem,
but we could decide to update specific fields to not be shared.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-04 12:22:32 +02:00
Sebastiaan van Stijn
8ea78b34ab
rename some variables that shadowed imports or package types
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-03 23:49:57 +02:00
Sebastiaan van Stijn
9f4acceb6a
remove redundant alias for libnetwork/datastore imports
These aliases were not needed, and only used in a couple of places,
which made it inconsistent, so let's use the import without aliasing.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-03 23:47:40 +02:00
Sebastiaan van Stijn
94c975e25a
api: info: don't use ad-hoc type for compatibility with old api versions
- Add the field as a "deprecated" field in the API type.
- Don't error when failing to parse the options, but produce a warning
  instead, because the client won't be able to fix issues in the daemon
  configuration. This was unlikely to happen, as the daemon probably
  would fail to start with an invalid config, but just in case.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-03 13:45:12 +02:00
Sebastiaan van Stijn
6592646632
api/server: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:25:20 +02:00
Brian Goff
74da6a6363 Switch all logging to use containerd log pkg
This unifies our logging and allows us to propagate logging and trace
contexts together.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2023-06-24 00:23:44 +00:00
Sebastiaan van Stijn
404160ad87
Merge pull request #45579 from tonistiigi/otlp-traces-for-history
builder-next: enable OTLP tracing for history records
2023-06-21 01:05:49 +02:00
Tonis Tiigi
cfa08f8366
builder-next: enable OTLP tracing for history records
This enables picking up OTLP tracing context for the gRPC
requests.

Also sets up the in-memory recorder that BuildKit History API
can use to store the traces associated with specific build
in a database after build completes.

This doesn't enable Jaeger tracing endpoints from env
but this can be easily enabled by adding another import if
maintainers want it.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2023-06-15 18:40:24 -07:00
Tianon Gravi
2a6ff3c24f Use OCI "History" type instead of inventing our own copy
The most notable change here is that the OCI's type uses a pointer for `Created`, which we probably should've been too, so most of these changes are accounting for that (and embedding our `Equal` implementation in the one single place it was used).

Signed-off-by: Tianon Gravi <admwiggin@gmail.com>
2023-06-12 13:47:17 -07:00
Cory Snider
982e4fb448 api/server: get features from a callback fn
Passing around a bare pointer to the map of configured features in order
to propagate to consumers changes to the configuration across reloads is
dangerous. Map operations are not atomic, so concurrently reading from
the map while it is being updated is a data race as there is no
synchronization. Use a getter function to retrieve the current features
map so the features can be retrieved race-free.

Remove the unused features argument from the build router.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-06-01 14:43:27 -04:00
Akihiro Suda
5045a2de24
Support recursively read-only (RRO) mounts
`docker run -v /foo:/foo:ro` is now recursively read-only on kernel >= 5.12.

Automatically falls back to the legacy non-recursively read-only mount mode on kernel < 5.12.

Use `ro-non-recursive` to disable RRO.
Use `ro-force-recursive` or `rro` to explicitly enable RRO. (Fails on kernel < 5.12)

Fix issue 44978
Fix docker/for-linux issue 788

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2023-05-26 01:58:24 +09:00
Sebastiaan van Stijn
1b3c2743cc
Merge pull request #45550 from corhere/fix-empty-container-decode
Allow empty body in `POST /commit` again
2023-05-18 21:46:00 +02:00
Paweł Gronowski
1be26e9f0c
api/inspect: Fix nil RepoTags and RepoDigests
Make RepoTags and RepoDigests empty arrays instead of nil.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-05-18 15:57:18 +02:00
Cory Snider
967c7bc5d3 api/server: allow empty body for POST /commit again
The error returned by DecodeConfig was changed in
b6d58d749c and caused this to regress.
Allow empty request bodies for this endpoint once again.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-05-17 16:01:11 -04:00
Akihiro Suda
1371aee3cc
Merge pull request #45469 from thaJeztah/deprecate_virtualsize_STEP2
API: omit deprecated VirtualSize field for API v1.44 and up
2023-05-16 15:10:07 +09:00
Jeyanthinath Muthuram
307b09e7eb
fixing consistent aliases for OCI spec imports
Signed-off-by: Jeyanthinath Muthuram <jeyanthinath10@gmail.com>
2023-05-08 15:27:52 +05:30
Sebastiaan van Stijn
913b0f51ca
API: omit deprecated VirtualSize field for API v1.44 and up
This field is deprecated since 1261fe69a3,
and will now be omitted on API v1.44 and up for the `GET /images/json`,
`GET /images/{id}/json`, and `GET /system/df`  endpoints.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-06 16:35:00 +02:00
Sebastiaan van Stijn
bf5ae17c16
api/server/router/container: containerRouter.postCommit: inline struct
Remove intermediate variable and inline the struct-literal.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-29 02:59:12 +02:00
Sebastiaan van Stijn
1261fe69a3
API: deprecate VirtualSize field for /images/json and /images/{id}/json
In versions of Docker before v1.10, this field was calculated from
the image itself and all of its parent images. Images are now stored
self-contained, and no longer use a parent-chain, making this field
an equivalent of the Size field.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-18 14:46:11 +02:00
Sebastiaan van Stijn
9a9dfb1d0d
api/server: getImagesJSON(): don't check version in a loop
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-17 13:07:49 +02:00
Tianon Gravi
f537ef5746
Merge pull request #44963 from vvoland/c8d-push-upstream
c8d: Implement push
2023-03-30 14:43:41 -07:00
Paweł Gronowski
e7393d0fc5
images/push: Accept reference
Push the reference parsing from repo and tag names into the api and pass
a reference object to the ImageService.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-03-30 18:08:40 +02:00
Laura Brehm
4c459ddabd
c8d: add support for docker diff
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
2023-03-30 10:29:07 +01:00
Sebastiaan van Stijn
cb76638699
registry/search: pass User-Agent through headers
Commit 3991faf464 moved search into the registry
package, which also made the `dockerversion` package a dependency for registry,
which brings additional (indirect) dependencies, such as `pkg/parsers/kernel`,
and `golang.org/x/sys/windows/registry`.

Client code, such as used in docker/cli may depend on the `registry` package,
but should not depend on those additional dependencies.

This patch moves setting the userAgent to the API router, and instead of
passing it as a separate argument, includes it into the "headers".

As these headers now not only contain the `X-Meta-...` headers, the variables
were renamed accordingly.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-03-21 14:41:29 +01:00
Bjorn Neergaard
1c84f63a40
Merge pull request #45086 from corhere/search-in-registry-service
Move filtered registry search out of the image service
2023-03-15 07:52:42 -06:00
Sebastiaan van Stijn
058a31e479
volumes: fix error-handling when removing volumes with swarm enabled
Commit 3246db3755 added handling for removing
cluster volumes, but in some conditions, this resulted in errors not being
returned if the volume was in use;

    docker swarm init
    docker volume create foo
    docker create -v foo:/foo busybox top
    docker volume rm foo

This patch changes the logic for ignoring "local" volume errors if swarm
is enabled (and cluster volumes supported).

While working on this fix, I also discovered that Cluster.RemoveVolume()
did not handle the "force" option correctly; while swarm correctly handled
these, the cluster backend performs a lookup of the volume first (to obtain
its ID), which would fail if the volume didn't exist.

Before this patch:

    make TEST_FILTER=TestVolumesRemoveSwarmEnabled DOCKER_GRAPHDRIVER=vfs test-integration
    ...
    Running /go/src/github.com/docker/docker/integration/volume (arm64.integration.volume) flags=-test.v -test.timeout=10m  -test.run TestVolumesRemoveSwarmEnabled
    ...
    === RUN   TestVolumesRemoveSwarmEnabled
    === PAUSE TestVolumesRemoveSwarmEnabled
    === CONT  TestVolumesRemoveSwarmEnabled
    === RUN   TestVolumesRemoveSwarmEnabled/volume_in_use
        volume_test.go:122: assertion failed: error is nil, not errdefs.IsConflict
        volume_test.go:123: assertion failed: expected an error, got nil
    === RUN   TestVolumesRemoveSwarmEnabled/volume_not_in_use
    === RUN   TestVolumesRemoveSwarmEnabled/non-existing_volume
    === RUN   TestVolumesRemoveSwarmEnabled/non-existing_volume_force
        volume_test.go:143: assertion failed: error is not nil: Error response from daemon: volume no_such_volume not found
    --- FAIL: TestVolumesRemoveSwarmEnabled (1.57s)
        --- FAIL: TestVolumesRemoveSwarmEnabled/volume_in_use (0.00s)
        --- PASS: TestVolumesRemoveSwarmEnabled/volume_not_in_use (0.01s)
        --- PASS: TestVolumesRemoveSwarmEnabled/non-existing_volume (0.00s)
        --- FAIL: TestVolumesRemoveSwarmEnabled/non-existing_volume_force (0.00s)
    FAIL

With this patch:

    make TEST_FILTER=TestVolumesRemoveSwarmEnabled DOCKER_GRAPHDRIVER=vfs test-integration
    ...
    Running /go/src/github.com/docker/docker/integration/volume (arm64.integration.volume) flags=-test.v -test.timeout=10m  -test.run TestVolumesRemoveSwarmEnabled
    ...
    make TEST_FILTER=TestVolumesRemoveSwarmEnabled DOCKER_GRAPHDRIVER=vfs test-integration
    ...
    Running /go/src/github.com/docker/docker/integration/volume (arm64.integration.volume) flags=-test.v -test.timeout=10m  -test.run TestVolumesRemoveSwarmEnabled
    ...
    === RUN   TestVolumesRemoveSwarmEnabled
    === PAUSE TestVolumesRemoveSwarmEnabled
    === CONT  TestVolumesRemoveSwarmEnabled
    === RUN   TestVolumesRemoveSwarmEnabled/volume_in_use
    === RUN   TestVolumesRemoveSwarmEnabled/volume_not_in_use
    === RUN   TestVolumesRemoveSwarmEnabled/non-existing_volume
    === RUN   TestVolumesRemoveSwarmEnabled/non-existing_volume_force
    --- PASS: TestVolumesRemoveSwarmEnabled (1.53s)
        --- PASS: TestVolumesRemoveSwarmEnabled/volume_in_use (0.00s)
        --- PASS: TestVolumesRemoveSwarmEnabled/volume_not_in_use (0.01s)
        --- PASS: TestVolumesRemoveSwarmEnabled/non-existing_volume (0.00s)
        --- PASS: TestVolumesRemoveSwarmEnabled/non-existing_volume_force (0.00s)
    PASS

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-03-13 19:17:35 +01:00
Cory Snider
3991faf464 Move filtered registry search out of image service
SearchRegistryForImages does not make sense as part of the image
service interface. The implementation just wraps the search API of the
registry service to filter the results client-side. It has nothing to do
with local image storage, and the implementation of search does not need
to change when changing which backend (graph driver vs. containerd
snapshotter) is used for local image storage.

Filtering of the search results is an implementation detail: the
consumer of the results does not care which actor does the filtering so
long as the results are filtered as requested. Move filtering into the
exported API of the registry service to hide the implementation details.
Only one thing---the registry service implementation---would need to
change in order to support server-side filtering of search results if
Docker Hub or other registry servers were to add support for it to their
APIs.

Use a fake registry server in the search unit tests to avoid having to
mock out the registry API client.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-03-10 18:36:33 -05:00
Nicolas De Loof
06619763a2
remove GetLayerByID from ImageService interface
Co-authored-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Co-authored-by: Paweł Gronowski <pawel.gronowski@docker.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-03-10 17:54:55 +01:00