Commit graph

75 commits

Author SHA1 Message Date
Sebastiaan van Stijn
9b02a84a5a
Merge pull request #46785 from rumpl/c8d-userns-commit
c8d: handle user namespace remapping on commit
2023-11-13 16:31:32 +01:00
Djordje Lukic
a7acfffea6
test: Add a test for userns remapped daemon commit
The files in an image created by a commit should have the right uid and gid set

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-11-08 12:33:03 +01:00
Rachit Sharma
7995e3288f
Add until filter to docker image ls
Signed-off-by: Rachit Sharma <rachitsharma613@gmail.com>

add handling for multiple filters

Signed-off-by: Rachit Sharma <rachitsharma613@gmail.com>

Update integration/image/list_test.go

Co-authored-by: Cory Snider <corhere@gmail.com>
Signed-off-by: Rachit Sharma <rachitsharma613@gmail.com>

Add documentation of filter

Signed-off-by: Rachit Sharma <rachitsharma613@gmail.com>

Update integration/image/list_test.go

Co-authored-by: Cory Snider <corhere@gmail.com>
Signed-off-by: Rachit Sharma <rachitsharma613@gmail.com>

Fix bug with CommitOptions

Signed-off-by: Rachit Sharma <rachitsharma613@gmail.com>

add wrapping of text to 80 chars

Signed-off-by: Rachit Sharma <rachitsharma613@gmail.com>
2023-11-08 12:12:23 +01:00
Sebastiaan van Stijn
4e8ba395f2
integration: TestSaveRepoWithMultipleImages: minor cleanup
- use consts for fixed values
- remove redundant `cmp.Nil(err)`

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-31 10:54:57 +01:00
Sebastiaan van Stijn
0dae0f2b5e
integration: TestSaveRepoWithMultipleImages remove redundant remove
This delete was originally added in b37fdc5dd1
and migrated from `deleteImages(repoName)` in commit 1e55ace875,
however, deleting `foobar-save-multi-images-test` (`foobar-save-multi-images-test:latest`)
always resulted in an error;

    Error response from daemon: No such image: foobar-save-multi-images-test:latest

This patch removes the redundant image delete.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-31 10:52:51 +01:00
Sebastiaan van Stijn
30cd8b8fca
integration: make TestSaveRepoWithMultipleImages less flaky
Shutting down containers on Windows can take a long time (with hyper-v),
causing this test to be flaky; seen failing on windows 2022;

    === FAIL: github.com/docker/docker/integration/image TestSaveRepoWithMultipleImages (23.16s)
        save_test.go:104: timeout waiting for container to exit

Looking at the test, we run a container only to commit it, and the test
does not make changes to the container's filesystem; it only runs a container
with a custom command (`true`).

Instead of running the container, we can _create_ a container and commit it;
this simplifies the tests, and prevents having to wait for the container to
exit (before committing).

To verify:

    make BIND_DIR=. DOCKER_GRAPHDRIVER=vfs TEST_FILTER=TestSaveRepoWithMultipleImages test-integration

    INFO: Testing against a local daemon
    === RUN   TestSaveRepoWithMultipleImages
    --- PASS: TestSaveRepoWithMultipleImages (1.20s)
    PASS

    DONE 1 tests in 2.668s

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-31 10:32:02 +01:00
Laura Brehm
6f625ae65a
c8d integration: skip TestImportWithCustomPlatform
We support importing images for other platforms when
using the containerd image store, so we shouldn't validate
the image OS on import.

This commit also splits the test into two, so that we can
keep running the "success" import with a custom platform tests
running w/ c8d while skipping the "error/rejection" test cases.

Signed-off-by: Laura Brehm <laurabrehm@hey.com>
2023-10-16 18:08:36 +01:00
Paweł Gronowski
42af8795a3
c8d/save: Implement exporting all tags
Implement a behavior from the graphdriver's export where `docker save
something` (untagged reference) would export all images matching the
specified repository.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-10-13 11:52:07 +02:00
Sebastiaan van Stijn
0f77875220
api/types: move ContainerRemoveOptions to api/types/container
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-12 11:29:23 +02:00
Sebastiaan van Stijn
9498d897ab
api/types: move ContainerCommitOptions to api/types/container
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-12 11:29:23 +02:00
Paweł Gronowski
5f062032f7
TestPullNonExistingImage: Move to integration
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-10-10 13:36:29 +02:00
Djordje Lukic
28861e0c47
c8d: Skip TestRemoveImageGarbageCollector
This test checks how the layer store works, so we don't need it when we
use containerd as image store

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-09-18 14:35:57 +02:00
Brian Goff
e8dc902781 Wire up tests to support otel tracing
Integration tests will now configure clients to propagate traces as well
as create spans for all tests.

Some extra changes were needed (or desired for trace propagation) in the
test helpers to pass through tracing spans via context.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2023-09-07 18:38:22 +00:00
Brian Goff
642e9917ff Add otel support
This uses otel standard environment variables to configure tracing in
the daemon.
It also adds support for propagating trace contexts in the client and
reading those from the API server.

See
https://opentelemetry.io/docs/specs/otel/configuration/sdk-environment-variables/
for details on otel environment variables.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2023-09-07 18:38:19 +00:00
Djordje Lukic
84a4f37cf7
test: use info from the version endpoint for arch checks
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-08-31 09:36:48 +02:00
Sebastiaan van Stijn
3d3ce9812f
integration/tag: Move to client unit test
This test was testing the client-side validation, so might as well
move it there, and validate that the client invalidates before
trying to make an API call.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-08-10 10:37:04 +02:00
Paweł Gronowski
71da8c13e1
integration/tag: Use subtests and make parallel
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-08-09 14:11:18 +02:00
Djordje Lukic
e5cffc07b3
c8d: Fix TestImagesFilterMultiReference
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-07-26 14:34:42 +02:00
Djordje Lukic
ad2760ec80
integration: Move image filter tests to integration
- use assert.Check to continue the test even if a check fails
- assert the total number of images returned, not only their RepoTags
- use subtests

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-07-26 14:33:16 +02:00
Paweł Gronowski
eaf141c86a
c8d/resolveImage: Fix Digested and Named reference
When resolving a reference that is both a Named and Digested, it could
be resolved to an image that has the same digest, but completely
different repository name.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-07-18 17:55:46 +02:00
Cory Snider
cdcb7c28c5 integration: disable iptables in parallel tests
Multiple daemons starting/running concurrently can collide with each
other when editing iptables rules. Most integration tests which opt into
parallelism and start daemons work around this problem by starting the
daemon with the --iptables=false option. However, some of the tests
neglect to pass the option when starting or restarting the daemon,
resulting in those tests being flaky.

Audit the integration tests which call t.Parallel() and (*Daemon).Stop()
and add --iptables=false arguments where needed.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-07-05 13:49:53 -04:00
Paweł Gronowski
a93298d4db
c8d/prune: Exclude dangling tag of the images used by containers
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-06-30 18:18:56 +02:00
Paweł Gronowski
a96e6044cc
testutil/environment: Add GetTestDanglingImageId
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-06-30 18:18:45 +02:00
Sebastiaan van Stijn
76d8bfdff4
testutil/environment: remove Execution.OSType field
This field was added in f0e5b3d7d8 to
account for older versions of the engine (Docker EE LTS versions), which
did not yet provide the OSType field in Docker info, and had to be manually
set using the TEST_OSTYPE env-var.

This patch removes the field in favor of the equivalent in DaemonInfo. It's
more verbose, but also less ambiguous what information we're using (i.e.,
the platform the daemon is running on, not the local platform).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-26 11:18:09 +02:00
Sebastiaan van Stijn
f139017bd0
Merge pull request #44598 from cpuguy83/save_tar_oci
image save: make output tarball OCI compliant
2023-06-09 14:50:51 +02:00
Brian Goff
1e55ace875 Add oci index and layout files to exported tars
This makes the output of `docker save` fully OCI compliant.

When using the containerd image store, this code is not used. That
exporter will just use containerd's export method and should give us the
output we want for multi-arch images.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2023-05-28 20:35:14 +00:00
Sebastiaan van Stijn
b42e367045
vendor: github.com/opencontainers/image-spec v1.1.0-rc3
full diff: https://github.com/opencontainers/image-spec/compare/3a7f492d3f1b...v1.1.0-rc3

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-26 02:34:50 +02:00
Paweł Gronowski
6506579e18
integration: Add TestImageInspectEmptyTagsAndDigests
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-05-23 15:50:36 +02:00
Sebastiaan van Stijn
ab35df454d
remove pre-go1.17 build-tags
Removed pre-go1.17 build-tags with go fix;

    go mod init
    go fix -mod=readonly ./...
    rm go.mod

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-19 20:38:51 +02:00
Jeyanthinath Muthuram
307b09e7eb
fixing consistent aliases for OCI spec imports
Signed-off-by: Jeyanthinath Muthuram <jeyanthinath10@gmail.com>
2023-05-08 15:27:52 +05:30
Cory Snider
0426c76142 List images with multiple since/before filters
The List Images API endpoint has accepted multiple values for the
`since` and `before` filter predicates, but thanks to Go's randomizing
of map iteration order, it would pick an arbitrary image to compare
created timestamps against. In other words, the behaviour was undefined.
Change these filter predicates to have well-defined semantics: the
logical AND of all values for each of the respective predicates. As
timestamps are a totally-ordered relation, this is exactly equivalent to
applying the newest and oldest creation timestamps for the `since` and
`before` predicates, respectively.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-12-08 16:36:57 -05:00
Sebastiaan van Stijn
4f43cb660a
skip TestImagePullStoredfDigestForOtherRepo() on Windows and rootless
- On Windows, we don't build and run a local  test registry (we're not running
  docker-in-docker), so we need to skip this test.
- On rootless, networking doesn't support this (currently)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-21 01:48:59 +02:00
Brian Goff
27530efedb
Validate digest in repo for pull by digest
This is accomplished by storing the distribution source in the content
labels. If the distribution source is not found then we check to the
registry to see if the digest exists in the repo, if it does exist then
the puller will use it.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-21 01:48:59 +02:00
Cory Snider
098a44c07f Finish refactor of UID/GID usage to a new struct
Finish the refactor which was partially completed with commit
34536c498d, passing around IdentityMapping structs instead of pairs of
[]IDMap slices.

Existing code which uses []IDMap relies on zero-valued fields to be
valid, empty mappings. So in order to successfully finish the
refactoring without introducing bugs, their replacement therefore also
needs to have a useful zero value which represents an empty mapping.
Change IdentityMapping to be a pass-by-value type so that there are no
nil pointers to worry about.

The functionality provided by the deprecated NewIDMappingsFromMaps
function is required by unit tests to to construct arbitrary
IdentityMapping values. And the daemon will always need to access the
mappings to pass them to the Linux kernel. Accommodate these use cases
by exporting the struct fields instead. BuildKit currently depends on
the UIDs and GIDs methods so we cannot get rid of them yet.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-03-14 16:28:57 -04:00
Sebastiaan van Stijn
b36d896fce
layer: remove OS from layerstore
This was added in commits fc21bf280b and
0380fbff37 in support of LCOW, but was
now always set to runtime.GOOS.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-25 15:23:23 +01:00
Sebastiaan van Stijn
01ae9525dd
Add support for platform (os and architecture) on image import
Commit 0380fbff37 added the ability to pass a
--platform flag on `docker import` when importing an archive. The intent
of that commit was to allow importing a Linux rootfs on a Windows daemon
(as part of the experimental LCOW feature).

A later commit (337ba71fc1) changed some
of this code to take both OS and Architecture into account (for `docker build`
and `docker pull`), but did not yet update the `docker image import`.

This patch updates the import endpoitn to allow passing both OS and
Architecture. Note that currently only matching OSes are accepted,
and an error will be produced when (e.g.) specifying `linux` on Windows
and vice-versa.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-24 18:24:51 +01:00
Eng Zer Jun
c55a4ac779
refactor: move from io/ioutil to io and os package
The io/ioutil package has been deprecated in Go 1.16. This commit
replaces the existing io/ioutil functions with their new definitions in
io and os packages.

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2021-08-27 14:56:57 +08:00
Sebastiaan van Stijn
686be57d0a
Update to Go 1.17.0, and gofmt with Go 1.17
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-08-24 23:33:27 +02:00
Sebastiaan van Stijn
dc7cbb9b33
remove layerstore indexing by OS (used for LCOW)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-06-10 17:49:11 +02:00
Olli Janatuinen
66a37b460c Windows: Enable more integration tests
Signed-off-by: Olli Janatuinen <olli.janatuinen@gmail.com>
2020-09-20 22:09:27 +03:00
Cristina Yenyxe Gonzalez Garcia
0154dc7a23 Using test names instead of hardcoded ones in integration/image directory
Signed-off-by: Cristina Yenyxe Gonzalez Garcia <cristina.yenyxe@gmail.com>
2020-05-18 16:37:09 +01:00
Akihiro Suda
5e1b246b9a test-integration: support more rootless tests
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-03-16 17:37:38 +09:00
Sebastiaan van Stijn
562880b276
Fix more goimports
```
daemon/logger/splunk/splunk_test.go:33: File is not `goimports`-ed (goimports)
        envKey:      "a",
        envRegexKey: "^foo",
        labelsKey:   "b",
        tagKey:      "c",
integration/build/build_test.go:41: File is not `goimports`-ed (goimports)
            rm:      false,
            forceRm: false,
integration/image/remove_unix_test.go:49: File is not `goimports`-ed (goimports)
        Root: d.Root,
```

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-02-11 18:56:25 +01:00
Sebastiaan van Stijn
9f0b3f5609
bump gotest.tools v3.0.1 for compatibility with Go 1.14
full diff: https://github.com/gotestyourself/gotest.tools/compare/v2.3.0...v3.0.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-02-11 00:06:42 +01:00
Olli Janatuinen
446fe7f263 Make sure that layers are removed from layerdb after succefull layer removal
Signed-off-by: Olli Janatuinen <olli.janatuinen@gmail.com>
2019-11-28 20:06:09 +02:00
Sam Whited
ae0a878b86 testutil, integration: untangle image dependency
Signed-off-by: Sam Whited <sam@samwhited.com>
2019-09-24 14:50:59 -05:00
Sebastiaan van Stijn
d948306255
integration: remove unused constants and fields (unused)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-18 12:57:10 +02:00
Sam Whited
b37c214e3c testutil: make testing packages public
This was done with something along the lines of:

```
mv internal/test testutil
pushd testutil/; grep -IRl "package test" | xargs -I '{}' sed -i -e 's|package test|package testutil|g' {}; popd
mv internal/testutil/*.go testutil/ && rm -rf internal/
grep -IRl "github.com\/docker\/docker\/internal\/test" | xargs -I '{}' sed -i -e 's|github.com/docker/docker/internal/test|github.com/docker/docker/test|g' {}
goimports .
```

I also modified the basic plugin path in testutil/fixtures/plugin.

Signed-off-by: Sam Whited <sam@samwhited.com>
2019-09-11 07:47:23 -05:00
Michael Crosby
d6cbeee470 Add extra permission check in removal test
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-07-22 20:39:22 +00:00
Sebastiaan van Stijn
9d1b4f5fc3
Add regression tests for invalid platform status codes
Before we handled containerd errors, using an invalid platform produced a 500 status:

```bash
curl -v \
  -X POST \
  --unix-socket /var/run/docker.sock \
  "http://localhost:2375/v1.40/images/create?fromImage=hello-world&platform=foobar&tag=latest" \
  -H "Content-Type: application/json"
```

```
* Connected to localhost (docker.sock) port 80 (#0)
> POST /v1.40/images/create?fromImage=hello-world&platform=foobar&tag=latest HTTP/1.1
> Host: localhost:2375
> User-Agent: curl/7.54.0
> Accept: */*
> Content-Type: application/json
>
< HTTP/1.1 500 Internal Server Error
< Api-Version: 1.40
< Content-Length: 85
< Content-Type: application/json
< Date: Mon, 15 Jul 2019 15:25:44 GMT
< Docker-Experimental: true
< Ostype: linux
< Server: Docker/19.03.0-rc2 (linux)
<
{"message":"\"foobar\": unknown operating system or architecture: invalid argument"}
```

That problem is now fixed, and the API correctly returns a 4xx status:

```bash
curl -v \
  -X POST \
  --unix-socket /var/run/docker.sock \
  "http://localhost:2375/v1.40/images/create?fromImage=hello-world&platform=foobar&tag=latest" \
  -H "Content-Type: application/json"
```

```
* Connected to localhost (/var/run/docker.sock) port 80 (#0)
> POST /v1.40/images/create?fromImage=hello-world&platform=foobar&tag=latest HTTP/1.1
> Host: localhost:2375
> User-Agent: curl/7.52.1
> Accept: */*
> Content-Type: application/json
>
< HTTP/1.1 400 Bad Request
< Api-Version: 1.41
< Content-Type: application/json
< Docker-Experimental: true
< Ostype: linux
< Server: Docker/dev (linux)
< Date: Mon, 15 Jul 2019 15:13:42 GMT
< Content-Length: 85
<
{"message":"\"foobar\": unknown operating system or architecture: invalid argument"}
* Curl_http_done: called premature == 0
```

This patch adds tests to validate the behaviour

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-07-15 20:37:00 +02:00