Commit graph

47601 commits

Author SHA1 Message Date
Sebastiaan van Stijn
6ce5aa1cd5
Merge pull request #46434 from vvoland/test-ignore-cgroup-check
hack: Add TEST_IGNORE_CGROUP_CHECK
2023-09-08 23:23:18 +02:00
Sebastiaan van Stijn
c3ab3f1dde
Merge pull request #46412 from rumpl/c8d-pull-progress
c8d: Better pull progress
2023-09-08 23:10:23 +02:00
Sebastiaan van Stijn
d582a26107
Merge pull request #46439 from akerouanton/no-optimizations
hack: compile w/o optimizations & inlining when debugging
2023-09-08 23:09:33 +02:00
Sebastiaan van Stijn
5e7eade1f7
integration: don't poll for containers to be running
container.Run() should be a synchronous operation in normal circumstances;
the container is created and started, so polling after that for the
container to be in the "running" state should not be needed.

This should also prevent issues when a container (for whatever reason)
exited immediately after starting; in that case we would continue
polling for it to be running (which likely would never happen).

Let's skip the polling; if the container is not in the expected state
(i.e. exited), tests should fail as well.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-08 23:05:43 +02:00
Sebastiaan van Stijn
b7d29c1e23
registry: un-skip some tests that don't require root
These tests seem to be running fine without being root, so let's not skip them.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-08 18:35:09 +02:00
Sebastiaan van Stijn
de27973538
registry: merge validateEndpoint into newV1Endpoint
validateEndpoint was doing more than just validating; it was also implicitly
mutating the endpoint that was passed to it (by reference).

Given that validation only happend when constructing a new v1Endpoint, let's
merge these functions.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-08 18:31:17 +02:00
Albin Kerouanton
6cc6682f5f
daemon: fix under what conditions container's mac-address is applied
The daemon would pass an EndpointCreateOption to set the interface MAC
address if the network name and the provided network mode were matching.
Obviously, if the network mode is a network ID, it won't work. To make
things worse, the network mode is never normalized if it's a partial ID.

To fix that: 1. the condition under what the container's mac-address is
applied is updated to also match the full ID; 2. the network mode is
normalized to a full ID when it's only a partial one.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-09-08 18:15:00 +02:00
Sebastiaan van Stijn
97e782ee2b
Merge pull request #46422 from thaJeztah/fix_resize_tests
integration/container: fix flaky resize tests, and some cleaning up
2023-09-08 18:14:18 +02:00
Sebastiaan van Stijn
e6d9d216dd
Merge pull request #46432 from vvoland/c8d-skip-image-rm-test
integration-cli: Skip TestRmiParentImageFail when using c8d snapshotters
2023-09-08 17:44:55 +02:00
Albin Kerouanton
820f37b1a6
hack: compile w/o optimizations & inlining when debugging
Without these compile flags, Delve is unable to report the value of some
variables and it's not possible to jump into inlined code.

As the contributing docs already mention that `DOCKER_DEBUG` should
disable "build optimizations", the env var is reused here instead of
introducing a new one.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-09-08 17:16:58 +02:00
Sebastiaan van Stijn
c44f8958f6
vendor: github.com/cncf-tags/container-device-interface v0.6.1
Removes uses of the github.com/opencontainers/runc/libcontainer/devices
package.

full diff: https://github.com/cncf-tags/container-device-interface/compare/v0.6.0...v0.6.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-08 16:01:56 +02:00
Sebastiaan van Stijn
0434b653c8
Merge pull request #46435 from vvoland/c8d-inspect-prefix
c8d/image: Allow truncated id to have sha256: prefix
2023-09-08 15:59:24 +02:00
Sebastiaan van Stijn
fa3ddc5116
distribution: newRepository: remove naked return and intermediate vars
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-08 15:44:58 +02:00
Sebastiaan van Stijn
6dcefa3739
Merge pull request #46431 from rumpl/fix-ci-timeout
Use process substitution to redirect to tee
2023-09-08 15:28:09 +02:00
Sebastiaan van Stijn
78ccd5d07c
Merge pull request #46433 from rumpl/c8d-remove-panic
c8d: Remove the panic from UpdateConfig
2023-09-08 15:17:53 +02:00
Sebastiaan van Stijn
43740ab642
registry: remove intermediate vars in mock
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-08 14:22:21 +02:00
Sebastiaan van Stijn
550e628f05
registry: newV1Endpoint: make it clear we skip validation for Docker Hub
validateEndpoint uses `v1Endpoint.ping` to verify if the search API can
use a secure connection, and to fall back to basic auth. For Docker Hub,
we don't allow insecure connections, and `v1Endpoint.ping` will not connect
to Docker Hub (Docker Hub also does not implement the `_ping` endpoint,
so doing so would always fail).

Let's make it more clear that we don't do any validation, and return
early.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-08 14:19:33 +02:00
Paweł Gronowski
5728516371
c8d/image: Allow truncated id to have sha256: prefix
Fixes TestInspectByPrefix when running with c8d snapshotters enabled.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-08 14:10:10 +02:00
Paweł Gronowski
c2004fb8c4
hack: Add TEST_IGNORE_CGROUP_CHECK
Add env variable that allows to run integration-cli tests with cgroup v2 enabled.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-08 13:41:03 +02:00
Sebastiaan van Stijn
7e506f71f3
registry: v1Endpoint.ping: include URL in debug log
Also remove log from `validateEndpoint`, because we don't actually
ping the default (Docker Hub).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-08 13:21:28 +02:00
Sebastiaan van Stijn
0c6f8f9290
registry: v1Endpoint.ping: don't io.Readall the response
We have the response available, which is an io.Reader, so we don't have
to read the entire response into memory before decoding.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-08 13:17:22 +02:00
Sebastiaan van Stijn
e9ad878df6
registry: v1Endpoint.ping: add fast-path for X-Docker-Registry-Standalone
This function was making a request to the `_ping` endpoint, which (if
implemented) would return a JSON response, which we unmarshal (the only
field we use from the response is the `Standalone` field).

However, if the response had a `X-Docker-Registry-Standalone`, that header
took precedence, and would overwrite the earlier `Standalone` value we
obtained from the JSON response.

This patch adds a fast-path for situations where the header is present,
in which case we can skip handling the JSON response altogether.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-08 13:17:16 +02:00
Djordje Lukic
9e797a6db9
c8d: Remove the panic from UpdateConfig
We don't really want the daemon to panic for this so let's log a warning
about max downloads and uploads

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-09-08 12:21:16 +02:00
Paweł Gronowski
4982c70db4
integration-cli: Skip TestRmiParentImageFail when using c8d snapshotters
With containerd image store the images don't depend on each other even
if they share the same content and it's totally fine to delete the
"parent" image.

The skip is necessary because deleting the "parent" image does not
produce an error with the c8d image store and deleting the `busybox`
image breaks other tests.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-08 12:08:35 +02:00
Sebastiaan van Stijn
0f7a65e59b
registry: remove use of v1 Version field and X-Docker-Registry-Version
- The `Version` field was not used for any purpose, other than a debug log
- The `X-Docker-Registry-Version` header was part of the registry v1 spec,
  however, as we're not using the `Version` field, we don't need the
  header for anything.
- The `X-Docker-Registry-Config` header was only set by the mock registry;
  there's no code consuming it, so we don't need to mock it (even if an
  actual v1 registry / search API would return it).

It's also worth noting that we never call the `_ping` endpoint when using
Docker Hub's search API, and Docker Hub does not even implement the `_ping`
endpoint;

    curl -fsSL https://index.docker.io/_ping | head -n 4
    <!DOCTYPE html>
    <html lang="en">
    <head>
    <title>Docker</title>

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-08 11:55:53 +02:00
Djordje Lukic
3d8b8dc09a
Use process substitution to redirect to tee
In some cases, when the daemon launched by a test panics and quits, the
cleanup code would end with an error when trying to kill it by its pid.
In those cases the whole suite will end up waiting for the daemon that
we start in .integration-daemon-start to finish and we end up waiting 2
hours for the CI to cancel after a timeout.

Using process substitution makes the integration tests quit.

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-09-08 10:56:21 +02:00
Paweł Gronowski
c0ece8d4f3
integration-cli: Use different label in TestBuildMultiStageGlobalArg
To avoid collision with TestBuildMultiStageArg.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-08 09:47:56 +02:00
Paweł Gronowski
081b74b97d
c8d/builder: Store parent in c8d image label
Images built by classic builder will have an additional label (in the
containerd image object, not image config) pointing to a parent of that
image.

This allows to differentiate intermediate images (dangling
images created as a result of a each Dockerfile instruction) from the
final images.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-08 09:47:53 +02:00
CrazyMax
152036f0aa
Merge pull request #46429 from cpuguy83/swap_to_otel_action
CI: Switch to use tracing action
2023-09-08 08:22:20 +02:00
Sebastiaan van Stijn
5f59f7bb49
integration/container: combine TestResize tests into subtests
Reduce some of the boiler-plating, and by combining the tests, we skip
the testenv.Clean() in between each of the tests. Performance gain isn't
really measurable, but every bit should help :)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-08 01:36:12 +02:00
Sebastiaan van Stijn
a4ceb0e4ac
integration/container: TestResize, TestResizeWithInvalidSize: rm poll.WaitOn
container.Run should be an synchronous operation; the container should
be running after the request was made (or produce an error). Simplify
these tests, and remove the redundant polling.

These were added as part of 8f800c9415,
but no such polls were in place before the refactor, and there's no
mention of these during review of the PR, so I assume these were just
added either as a "precaution", or a result of "copy/paste" from another
test.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-08 01:34:45 +02:00
Sebastiaan van Stijn
ee7ca6822a
integration/container: fix flaky TestResizeWhenContainerNotStarted
This test was failing frequently on Windows, where the test was waiting
for the container to exit before continuing;

    === FAIL: github.com/docker/docker/integration/container TestResizeWhenContainerNotStarted (18.69s)
    resize_test.go:58: timeout hit after 10s: waiting for container to be one of (exited), currently running

It looks like this test is merely validating that a container in any non-
running state should produce an error, so there's no need to run a container
(waiting for it to stop), and just "creating" a container (which would be
in `created` state) should work for this purpose.

Looking at 8f800c9415, I see `createSimpleContainer`
and `runSimpleContainer` utilities were added, so I'm even wondering if the
original intent was to use `createSimpleContainer` for  this test.

While updating, also check if we get the expected error-type, instead of
only checking for the error-message.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-08 01:33:22 +02:00
Sebastiaan van Stijn
06499c52e2
Merge pull request #46416 from thaJeztah/move_IsOSSupported
pkg/system: deprecate IsOSSupported() and ErrNotSupportedOperatingSystem, and implement image.CheckOS
2023-09-08 00:25:36 +02:00
Brian Goff
da5ed8b35b CI: Switch to use tracing action
This takes care of the TODO item now that the action is merged.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2023-09-07 22:23:05 +00:00
Bjorn Neergaard
ce4e325504
Merge pull request #45652 from cpuguy83/otel
Add otel tracing
2023-09-07 15:42:08 -06:00
Sebastiaan van Stijn
a3c97beee0
image: implement CheckOS, deprecate pkg/system IsOSSupported
Implement a function that returns an error to replace existing uses of
the IsOSSupported utility, where callers had to produce the error after
checking.

The IsOSSupported function was used in combination with images, so implementing
a utility in "image" to prevent having to import pkg/system (which contains many
unrelated functions)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-07 22:14:44 +02:00
Sebastiaan van Stijn
150b657bad
pkg/system: move ErrNotSupportedOperatingSystem to where it's used
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-07 22:04:33 +02:00
Brian Goff
9b7784781d Add test for buildkit history trace propagation
This test ensures that we are able to propagate traces into buildkit's
history API.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2023-09-07 18:38:22 +00:00
Brian Goff
3b4ccb2eca CI: Setup otel tracing for integration tests
This wires up the integration tests to export spans to a jager instance.
After tests are finished it exports the data out of jaeger and uploads
as an artifact to the action run.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2023-09-07 18:38:22 +00:00
Brian Goff
e8dc902781 Wire up tests to support otel tracing
Integration tests will now configure clients to propagate traces as well
as create spans for all tests.

Some extra changes were needed (or desired for trace propagation) in the
test helpers to pass through tracing spans via context.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2023-09-07 18:38:22 +00:00
Brian Goff
642e9917ff Add otel support
This uses otel standard environment variables to configure tracing in
the daemon.
It also adds support for propagating trace contexts in the client and
reading those from the API server.

See
https://opentelemetry.io/docs/specs/otel/configuration/sdk-environment-variables/
for details on otel environment variables.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2023-09-07 18:38:19 +00:00
Sebastiaan van Stijn
de391bc48b
Merge pull request #46418 from vvoland/c8d-import-fix-double-close
c8d/import: Don't close compressed stream twice
2023-09-07 16:30:01 +02:00
Djordje Lukic
776c376227
c8d: Better pull progress
- check if we have to download layers and print the approriate message
- show the digest of the pulled manifest(list)
- skip pulling if we already have the right manifest

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-09-07 15:34:20 +02:00
Sebastiaan van Stijn
efea287b44
Merge pull request #46415 from thaJeztah/distribution_simplify_filterManifests
distribution: filterManifests split complicated condition
2023-09-07 13:59:27 +02:00
Paweł Gronowski
1cb45e582c
c8d/import: Don't close compressed stream twice
The compressor is already closed a few lines below and there's no error
returns between so the defer is not needed.

Calling Close twice on a writerCloserWrapper is unsafe as it causes it
to put the same buffer to the pool multiple times.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-07 10:33:25 +02:00
Sebastiaan van Stijn
075a2d89b9
Merge pull request #46417 from thaJeztah/idtools_preserve_error
pkg/idtools: remove sync.Once, and include lookup error
2023-09-06 23:08:43 +02:00
Sebastiaan van Stijn
b5376c7cec
pkg/idtools: remove sync.Once, and include lookup error
When running a `docker cp` to copy files to/from a container, the
lookup of the `getent` executable happens within the container's
filesystem, so we cannot re-use the results.

Unfortunately, that also means we can't preserve the results for
any other uses of these functions, but probably the lookup should not
be "too" costly.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-06 17:34:26 +02:00
Sebastiaan van Stijn
62725ab277
Merge pull request #46414 from thaJeztah/less_logrus
daemon: daemon.createSpec: remove uses of logrus
2023-09-06 15:32:01 +02:00
Sebastiaan van Stijn
0be416dfdc
distribution: filterManifests split complicated condition
Break up the complicated "if" condition into multiple checks.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-06 13:34:10 +02:00
Sebastiaan van Stijn
150b1c8c73
daemon: daemon.createSpec: remove uses of logrus
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-06 13:30:33 +02:00