Commit graph

47432 commits

Author SHA1 Message Date
Djordje Lukic
0313544f4a
c8d: Handle userns properly
If the daemon is run with --userns-remap we need to chown the prepared
snapshot

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-09-11 16:39:29 +02:00
Djordje Lukic
833b514d98
c8d/mount: Create same directory owners as graphdrivers
Graph drivers create the parent directory with
rootPair().GID:CurrentIdentity().UID owner. This change brings these in
line

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-09-11 13:24:42 +02:00
Sebastiaan van Stijn
6ce5aa1cd5
Merge pull request #46434 from vvoland/test-ignore-cgroup-check
hack: Add TEST_IGNORE_CGROUP_CHECK
2023-09-08 23:23:18 +02:00
Sebastiaan van Stijn
c3ab3f1dde
Merge pull request #46412 from rumpl/c8d-pull-progress
c8d: Better pull progress
2023-09-08 23:10:23 +02:00
Sebastiaan van Stijn
d582a26107
Merge pull request #46439 from akerouanton/no-optimizations
hack: compile w/o optimizations & inlining when debugging
2023-09-08 23:09:33 +02:00
Sebastiaan van Stijn
97e782ee2b
Merge pull request #46422 from thaJeztah/fix_resize_tests
integration/container: fix flaky resize tests, and some cleaning up
2023-09-08 18:14:18 +02:00
Sebastiaan van Stijn
e6d9d216dd
Merge pull request #46432 from vvoland/c8d-skip-image-rm-test
integration-cli: Skip TestRmiParentImageFail when using c8d snapshotters
2023-09-08 17:44:55 +02:00
Albin Kerouanton
820f37b1a6
hack: compile w/o optimizations & inlining when debugging
Without these compile flags, Delve is unable to report the value of some
variables and it's not possible to jump into inlined code.

As the contributing docs already mention that `DOCKER_DEBUG` should
disable "build optimizations", the env var is reused here instead of
introducing a new one.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-09-08 17:16:58 +02:00
Sebastiaan van Stijn
0434b653c8
Merge pull request #46435 from vvoland/c8d-inspect-prefix
c8d/image: Allow truncated id to have sha256: prefix
2023-09-08 15:59:24 +02:00
Sebastiaan van Stijn
6dcefa3739
Merge pull request #46431 from rumpl/fix-ci-timeout
Use process substitution to redirect to tee
2023-09-08 15:28:09 +02:00
Sebastiaan van Stijn
78ccd5d07c
Merge pull request #46433 from rumpl/c8d-remove-panic
c8d: Remove the panic from UpdateConfig
2023-09-08 15:17:53 +02:00
Paweł Gronowski
5728516371
c8d/image: Allow truncated id to have sha256: prefix
Fixes TestInspectByPrefix when running with c8d snapshotters enabled.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-08 14:10:10 +02:00
Paweł Gronowski
c2004fb8c4
hack: Add TEST_IGNORE_CGROUP_CHECK
Add env variable that allows to run integration-cli tests with cgroup v2 enabled.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-08 13:41:03 +02:00
Djordje Lukic
9e797a6db9
c8d: Remove the panic from UpdateConfig
We don't really want the daemon to panic for this so let's log a warning
about max downloads and uploads

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-09-08 12:21:16 +02:00
Paweł Gronowski
4982c70db4
integration-cli: Skip TestRmiParentImageFail when using c8d snapshotters
With containerd image store the images don't depend on each other even
if they share the same content and it's totally fine to delete the
"parent" image.

The skip is necessary because deleting the "parent" image does not
produce an error with the c8d image store and deleting the `busybox`
image breaks other tests.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-08 12:08:35 +02:00
Djordje Lukic
3d8b8dc09a
Use process substitution to redirect to tee
In some cases, when the daemon launched by a test panics and quits, the
cleanup code would end with an error when trying to kill it by its pid.
In those cases the whole suite will end up waiting for the daemon that
we start in .integration-daemon-start to finish and we end up waiting 2
hours for the CI to cancel after a timeout.

Using process substitution makes the integration tests quit.

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-09-08 10:56:21 +02:00
CrazyMax
152036f0aa
Merge pull request #46429 from cpuguy83/swap_to_otel_action
CI: Switch to use tracing action
2023-09-08 08:22:20 +02:00
Sebastiaan van Stijn
5f59f7bb49
integration/container: combine TestResize tests into subtests
Reduce some of the boiler-plating, and by combining the tests, we skip
the testenv.Clean() in between each of the tests. Performance gain isn't
really measurable, but every bit should help :)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-08 01:36:12 +02:00
Sebastiaan van Stijn
a4ceb0e4ac
integration/container: TestResize, TestResizeWithInvalidSize: rm poll.WaitOn
container.Run should be an synchronous operation; the container should
be running after the request was made (or produce an error). Simplify
these tests, and remove the redundant polling.

These were added as part of 8f800c9415,
but no such polls were in place before the refactor, and there's no
mention of these during review of the PR, so I assume these were just
added either as a "precaution", or a result of "copy/paste" from another
test.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-08 01:34:45 +02:00
Sebastiaan van Stijn
ee7ca6822a
integration/container: fix flaky TestResizeWhenContainerNotStarted
This test was failing frequently on Windows, where the test was waiting
for the container to exit before continuing;

    === FAIL: github.com/docker/docker/integration/container TestResizeWhenContainerNotStarted (18.69s)
    resize_test.go:58: timeout hit after 10s: waiting for container to be one of (exited), currently running

It looks like this test is merely validating that a container in any non-
running state should produce an error, so there's no need to run a container
(waiting for it to stop), and just "creating" a container (which would be
in `created` state) should work for this purpose.

Looking at 8f800c9415, I see `createSimpleContainer`
and `runSimpleContainer` utilities were added, so I'm even wondering if the
original intent was to use `createSimpleContainer` for  this test.

While updating, also check if we get the expected error-type, instead of
only checking for the error-message.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-08 01:33:22 +02:00
Sebastiaan van Stijn
06499c52e2
Merge pull request #46416 from thaJeztah/move_IsOSSupported
pkg/system: deprecate IsOSSupported() and ErrNotSupportedOperatingSystem, and implement image.CheckOS
2023-09-08 00:25:36 +02:00
Brian Goff
da5ed8b35b CI: Switch to use tracing action
This takes care of the TODO item now that the action is merged.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2023-09-07 22:23:05 +00:00
Bjorn Neergaard
ce4e325504
Merge pull request #45652 from cpuguy83/otel
Add otel tracing
2023-09-07 15:42:08 -06:00
Sebastiaan van Stijn
a3c97beee0
image: implement CheckOS, deprecate pkg/system IsOSSupported
Implement a function that returns an error to replace existing uses of
the IsOSSupported utility, where callers had to produce the error after
checking.

The IsOSSupported function was used in combination with images, so implementing
a utility in "image" to prevent having to import pkg/system (which contains many
unrelated functions)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-07 22:14:44 +02:00
Sebastiaan van Stijn
150b657bad
pkg/system: move ErrNotSupportedOperatingSystem to where it's used
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-07 22:04:33 +02:00
Brian Goff
9b7784781d Add test for buildkit history trace propagation
This test ensures that we are able to propagate traces into buildkit's
history API.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2023-09-07 18:38:22 +00:00
Brian Goff
3b4ccb2eca CI: Setup otel tracing for integration tests
This wires up the integration tests to export spans to a jager instance.
After tests are finished it exports the data out of jaeger and uploads
as an artifact to the action run.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2023-09-07 18:38:22 +00:00
Brian Goff
e8dc902781 Wire up tests to support otel tracing
Integration tests will now configure clients to propagate traces as well
as create spans for all tests.

Some extra changes were needed (or desired for trace propagation) in the
test helpers to pass through tracing spans via context.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2023-09-07 18:38:22 +00:00
Brian Goff
642e9917ff Add otel support
This uses otel standard environment variables to configure tracing in
the daemon.
It also adds support for propagating trace contexts in the client and
reading those from the API server.

See
https://opentelemetry.io/docs/specs/otel/configuration/sdk-environment-variables/
for details on otel environment variables.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2023-09-07 18:38:19 +00:00
Sebastiaan van Stijn
de391bc48b
Merge pull request #46418 from vvoland/c8d-import-fix-double-close
c8d/import: Don't close compressed stream twice
2023-09-07 16:30:01 +02:00
Djordje Lukic
776c376227
c8d: Better pull progress
- check if we have to download layers and print the approriate message
- show the digest of the pulled manifest(list)
- skip pulling if we already have the right manifest

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-09-07 15:34:20 +02:00
Sebastiaan van Stijn
efea287b44
Merge pull request #46415 from thaJeztah/distribution_simplify_filterManifests
distribution: filterManifests split complicated condition
2023-09-07 13:59:27 +02:00
Paweł Gronowski
1cb45e582c
c8d/import: Don't close compressed stream twice
The compressor is already closed a few lines below and there's no error
returns between so the defer is not needed.

Calling Close twice on a writerCloserWrapper is unsafe as it causes it
to put the same buffer to the pool multiple times.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-07 10:33:25 +02:00
Sebastiaan van Stijn
075a2d89b9
Merge pull request #46417 from thaJeztah/idtools_preserve_error
pkg/idtools: remove sync.Once, and include lookup error
2023-09-06 23:08:43 +02:00
Sebastiaan van Stijn
b5376c7cec
pkg/idtools: remove sync.Once, and include lookup error
When running a `docker cp` to copy files to/from a container, the
lookup of the `getent` executable happens within the container's
filesystem, so we cannot re-use the results.

Unfortunately, that also means we can't preserve the results for
any other uses of these functions, but probably the lookup should not
be "too" costly.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-06 17:34:26 +02:00
Sebastiaan van Stijn
62725ab277
Merge pull request #46414 from thaJeztah/less_logrus
daemon: daemon.createSpec: remove uses of logrus
2023-09-06 15:32:01 +02:00
Sebastiaan van Stijn
0be416dfdc
distribution: filterManifests split complicated condition
Break up the complicated "if" condition into multiple checks.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-06 13:34:10 +02:00
Sebastiaan van Stijn
150b1c8c73
daemon: daemon.createSpec: remove uses of logrus
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-06 13:30:33 +02:00
Djordje Lukic
2c95ddf4f3
Merge pull request #46405 from rumpl/image-events 2023-09-06 12:31:35 +02:00
Sebastiaan van Stijn
791549508a
Merge pull request #46084 from rumpl/fix-test-arch
test: Remove DOCKER_ENGINE_GOARCH from the tests
2023-09-05 18:08:57 +02:00
Djordje Lukic
dbb4d54e01
c8d: Add image save events
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-09-05 17:29:15 +02:00
Djordje Lukic
09470db089
c8d: Add image pull events
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-09-05 17:29:13 +02:00
Djordje Lukic
1847dc4b7f
c8d: Add image push events
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-09-05 17:28:57 +02:00
Sebastiaan van Stijn
032797ea4b
Merge pull request #46376 from thaJeztah/migrate_distribution_reference
migrate to new github.com/distribution/reference module
2023-09-05 15:04:51 +02:00
Sebastiaan van Stijn
7abd7fa739
Merge pull request #46398 from vvoland/tests-integration-cli-build-multline-images
integration-cli/build: Parse multiline images -q output
2023-09-05 12:22:34 +02:00
Sebastiaan van Stijn
1148a24e64
migrate to new github.com/distribution/reference module
The "reference" package was moved to a separate module, which was extracted
from b9b19409cf

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-05 12:09:26 +02:00
Sebastiaan van Stijn
9c4e82435e
Merge pull request #46351 from thaJeztah/api_events_actions_enum
api/types/events: define "Action" type and consts
2023-09-05 11:11:42 +02:00
Sebastiaan van Stijn
2243046bf8
Merge pull request #46384 from rumpl/push-print-digest
c8d: Print the tag, digest and size after push
2023-09-04 16:47:02 +02:00
Paweł Gronowski
dffccfd0f1
integration-cli/build: Alias cmp as is
To make it consistent with other tests.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-04 16:40:08 +02:00
Paweł Gronowski
5dbd198b53
integration-cli/build: Parse multiline images -q output
This causes the test to have a saner error message when the `images
-q` returns multiple images separated by newline.

Before this the test would fail with `invalid reference format` when
parsing the multiline string as an image reference.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-04 16:40:05 +02:00