Commit graph

47975 commits

Author SHA1 Message Date
Ken Bannister
6979503a85 Include port in URL for locally built Swagger docs
Signed-off-by: Ken Bannister <kb2ma@runbox.com>
2023-11-07 08:07:32 -05:00
Sebastiaan van Stijn
796da163f9
Merge pull request #46778 from corhere/libc8d/revert-serialize-exec-starts-workaround
Revert "libcontainerd: work around exec start bug in c8d"
2023-11-06 21:12:39 +01:00
Cory Snider
7d9d601e6d project: document supported containerd versions
We only support containerd versions which have fixes for
https://github.com/containerd/containerd/issues/8557.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-11-06 13:43:18 -05:00
Cory Snider
29ac09ee9d Revert "libcontainerd: work around exec start bug in c8d"
The workaround is no longer required. The bug has been fixed in stable
versions of all supported containerd branches.

This reverts commit fb7ec1555c.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-11-06 13:26:44 -05:00
Sebastiaan van Stijn
02011af7b0
Merge pull request #46774 from elezar/bump-cdi-dependency
Update container-device-interface to v0.6.2
2023-11-04 13:27:17 +01:00
Evan Lezar
49e04102c1 Update container-device-interface to v0.6.2
This includes migrating from the github.com/container-orchestrated-devices
repo to tags.cncf.io.

Signed-off-by: Evan Lezar <elezar@nvidia.com>
2023-11-04 01:00:19 +01:00
Paweł Gronowski
3cb8e9526e
Merge pull request #46769 from vvoland/c8d-shared-mounter-impl
daemon/snapshotter: Align mounter implementations
2023-11-03 18:45:36 +01:00
Paweł Gronowski
26f63600c1
daemon/snapshotter: Align mounter implementations
Change the non-refcounted implementation to perform the mount using the
same identity and access right. They should be the same regardless if
we're refcounting or not.

This also allows to refactor refCountMounter into a mounter decorator.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-11-03 18:13:14 +01:00
Albin Kerouanton
6fd893bd3a
Merge pull request #46768 from akerouanton/dedicated-default-nw-config-struct
daemon/config: Put params for the default network into a dedicated struct
2023-11-03 17:03:05 +01:00
Albin Kerouanton
d5d41c2849
daemon/config: Put params for the default network into a dedicated struct
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-11-03 14:10:41 +01:00
Sebastiaan van Stijn
ed1a61dcb7
Merge pull request #46663 from akerouanton/ci-otel-windows
ci: Setup otel tracing for windows integration tests
2023-11-03 13:51:39 +01:00
Sebastiaan van Stijn
5b19725de2
Merge pull request #46668 from corhere/libn/svc-record-update-without-store
libnetwork: svc record update without store
2023-11-03 13:47:12 +01:00
Sebastiaan van Stijn
0ac748a340
Merge pull request #46628 from sgehrig/46621-container_wait
#46621 ensure that errors reading the response body are returned to the caller
2023-11-03 13:20:59 +01:00
Sebastiaan van Stijn
26c054edbf
Merge pull request #46531 from akerouanton/networking-suite-bridge-tests
integration: Add a new networking integration test suite
2023-11-03 12:33:44 +01:00
Sebastiaan van Stijn
39393f651e
Merge pull request #46766 from thaJeztah/fix_TestInfoAPI
integration/system: update TestInfoAPI to not use string-matching
2023-11-03 11:43:28 +01:00
Sebastiaan van Stijn
587ad8845a
integration/system: update TestInfoAPI to not use string-matching
This test was rewritten from an integration-cli test in commit
68d9beedbe, and originally implemented in
f4942ed864, which rewrote it from a unit-
test to an integration test.

Originally, it would check for the raw JSON response from the daemon, and
check for individual fields to be present in the output, but after commit
0fd5a65428, `client.Info()` was used, and
now the response is unmarshalled into a `system.Info`.

The remainder of the test remained the same in that rewrite, and as a
result were were now effectively testing if a `system.Info` struct,
when marshalled as JSON would show all the fields (surprise: it does).

TL;DR; the test would even pass with an empty `system.Info{}` struct,
which didn't provide much coverage, as it passed without a daemon:

    func TestInfoAPI(t *testing.T) {
        // always shown fields
        stringsToCheck := []string{
            "ID",
            "Containers",
            "ContainersRunning",
            "ContainersPaused",
            "ContainersStopped",
            "Images",
            "LoggingDriver",
            "OperatingSystem",
            "NCPU",
            "OSType",
            "Architecture",
            "MemTotal",
            "KernelVersion",
            "Driver",
            "ServerVersion",
            "SecurityOptions",
        }

        out := fmt.Sprintf("%+v", system.Info{})
        for _, linePrefix := range stringsToCheck {
            assert.Check(t, is.Contains(out, linePrefix))
        }
    }

This patch makes the test _slightly_ better by checking if the fields
are non-empty. More work is needed on this test though; currently it
uses the (already running) daemon, so it's hard to check for specific
fields to be correct (withouth knowing state of the daemon), but it's
not unlikely that other tests (partially) cover some of that. A TODO
comment was added to look into that (we should probably combine some
tests to prevent overlap, and make it easier to spot "gaps" as well).

While working on this, also moving the `SystemTime` into this test,
because that field is (no longer) dependent on "debug" state

(It is was actually this change that led me down this rabbit-hole)

                         ()_()
                         (-.-)
                        '(")(")'

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-11-03 10:17:15 +01:00
Albin Kerouanton
c1ab6eda4b
integration/networking: Test bridge ICC and INC
Following tests are implemented in this specific commit:

- Inter-container communications for internal and non-internal
  bridge networks, over IPv4 and IPv6.
- Inter-container communications using IPv6 link-local addresses for
  internal and non-internal bridge networks.
- Inter-network communications for internal and non-internal bridge
  networks, over IPv4 and IPv6, are disallowed.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-11-03 09:58:50 +01:00
Albin Kerouanton
409ea700c7
integration: Add a new networking integration test suite
This commit introduces a new integration test suite aimed at testing
networking features like inter-container communication, network
isolation, port mapping, etc... and how they interact with daemon-level
and network-level parameters.

So far, there's pretty much no tests making sure our networks are well
configured: 1. there're a few tests for port mapping, but they don't
cover all use cases ; 2. there're a few tests that check if a specific
iptables rule exist, but that doesn't prevent that specific iptables
rule to be wrong in the first place.

As we're planning to refactor how iptables rules are written, and change
some of them to fix known security issues, we need a way to test all
combinations of parameters. So far, this was done by hand, which is
particularly painful and time consuming. As such, this new test suite is
foundational to upcoming work.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-11-03 09:58:50 +01:00
Sebastiaan van Stijn
050e6066af
Merge pull request #46517 from rumpl/c8d-windows-pull-linux
c8d: test a backend dependent error on pull
2023-11-03 09:28:16 +01:00
Sebastiaan van Stijn
8068038a60
Merge pull request #46758 from rumpl/c8d-image-list-digest
Skip the busybox digest check when containerd is enabled
2023-11-03 09:16:48 +01:00
Sebastiaan van Stijn
90cc178baa
Merge pull request #46759 from rumpl/c8d-index-pull-test
Skip TestPullManifestList when using containerd
2023-11-03 09:14:42 +01:00
Sebastiaan van Stijn
5baf486545
Merge pull request #46764 from cpuguy83/fix_probe_interval
Fix case where health start interval is 0 uses default
2023-11-03 08:32:55 +01:00
Sebastiaan van Stijn
ec32f0db82
Merge pull request #46762 from akerouanton/seccomp-io_uring
seccomp: block io_uring_* syscalls in default profile
2023-11-02 21:10:29 +01:00
Brian Goff
02a932d63f Fix case where health start interval is 0 uses default
When the start interval is 0 we should treat that as unset.
This is especially important for older API versions where we reset the
value to 0.

Instead of using the default probe value we should be using the
configured `interval` value (which may be a default as well) which gives
us back the old behavior before support for start interval was added.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2023-11-02 20:02:16 +00:00
Brian Goff
2f138d860e
Merge pull request #46685 from rumpl/c8d-tag-does-not-exist
c8d: Return the "tag does not exist error"
2023-11-02 12:33:36 -07:00
Brian Goff
d3f4876f41
Merge pull request #46751 from thaJeztah/fix_TestSaveRepoWithMultipleImages_flaky
integration: make TestSaveRepoWithMultipleImages less flaky
2023-11-02 12:32:07 -07:00
Albin Kerouanton
891241e7e7
seccomp: block io_uring_* syscalls in default profile
This syncs the seccomp profile with changes made to containerd's default
profile in [1].

The original containerd issue and PR mention:

> Security experts generally believe io_uring to be unsafe. In fact
> Google ChromeOS and Android have turned it off, plus all Google
> production servers turn it off. Based on the blog published by Google
> below it seems like a bunch of vulnerabilities related to io_uring can
> be exploited to breakout of the container.
>
> [2]
>
> Other security reaserchers also hold this opinion: see [3] for a
> blackhat presentation on io_uring exploits.

For the record, these syscalls were added to the allowlist in [4].

[1]: a48ddf4a20
[2]: https://security.googleblog.com/2023/06/learnings-from-kctf-vrps-42-linux.html
[3]: https://i.blackhat.com/BH-US-23/Presentations/US-23-Lin-bad_io_uring.pdf
[4]: https://github.com/moby/moby/pull/39415

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-11-02 19:05:47 +01:00
Djordje Lukic
3c2b06099d
Skip TestPullManifestList when using containerd
This test is very weird, the Size in the manifests that it creates is
wrong, graph drivers only print a warning in that case but containerd
fails because it verifies more things. The media types are also wrong in
the containerd case, the manifest list forces the media type to be
"schema2.MediaTypeManifest" but in the containerd case the media type is
an OCI one.

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-11-02 16:09:35 +01:00
Djordje Lukic
6ca1b9e4ce
Skip the busybox digest check when containerd is enabled
We always have a digest with containerd.

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-11-02 14:07:49 +01:00
Sebastiaan van Stijn
fb3cc5e716
Merge pull request #46755 from corhere/libn/netip-overlaps
libnetwork/ipam: refactor prefix-overlap checks
2023-11-02 13:33:40 +01:00
Sebastiaan van Stijn
217054ddea
Merge pull request #46754 from Frankkkkk/master
builder/dockerfile: errmsg: quote build target
2023-11-02 12:40:22 +01:00
Frank Villaro-Dixon
123ade3763 builder/dockerfile: errmsg: quote build target
The build target is not quoted and it makes it difficult for some
persons to see what the problem is.

By quoting it we emphasize that the target name is variable.

Signed-off-by: Frank Villaro-Dixon <frank.villarodixon@merkle.com>
2023-11-02 09:38:14 +01:00
Cory Snider
7257c77e19 libnetwork/ipam: refactor prefix-overlap checks
I am finally convinced that, given two netip.Prefix values a and b, the
expression

    a.Contains(b.Addr()) || b.Contains(a.Addr())

is functionally equivalent to

    a.Overlaps(b)

The (netip.Prefix).Contains method works by masking the address with the
prefix's mask and testing whether the remaining most-significant bits
are equal to the same bits in the prefix. The (netip.Prefix).Overlaps
method works by masking the longer prefix to the length of the shorter
prefix and testing whether the remaining most-significant bits are
equal. This is equivalent to
shorterPrefix.Contains(longerPrefix.Addr()), therefore applying Contains
symmetrically to two prefixes will always yield the same result as
applying Overlaps to the two prefixes in either order.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-11-01 11:44:24 -04:00
Sebastiaan van Stijn
60e88c873e
Merge pull request #46736 from corhere/robust-tarsum
pkg/tarsum: handle xattrs like archive/tar does
2023-10-31 19:52:47 +01:00
Sebastiaan van Stijn
d9dce8e0d0
Merge pull request #46618 from vvoland/c8d-pull-all-tags-2
c8d/pull: Handle pull all tags (2nd approach)
2023-10-31 16:45:18 +01:00
Sebastiaan van Stijn
4be5b4147d
Merge pull request #46748 from tonistiigi/containerd-platforms-normalize
builder-next: make sure worker platforms normalized for containerd
2023-10-31 12:38:09 +01:00
Sebastiaan van Stijn
4e8ba395f2
integration: TestSaveRepoWithMultipleImages: minor cleanup
- use consts for fixed values
- remove redundant `cmp.Nil(err)`

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-31 10:54:57 +01:00
Sebastiaan van Stijn
0dae0f2b5e
integration: TestSaveRepoWithMultipleImages remove redundant remove
This delete was originally added in b37fdc5dd1
and migrated from `deleteImages(repoName)` in commit 1e55ace875,
however, deleting `foobar-save-multi-images-test` (`foobar-save-multi-images-test:latest`)
always resulted in an error;

    Error response from daemon: No such image: foobar-save-multi-images-test:latest

This patch removes the redundant image delete.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-31 10:52:51 +01:00
Sebastiaan van Stijn
30cd8b8fca
integration: make TestSaveRepoWithMultipleImages less flaky
Shutting down containers on Windows can take a long time (with hyper-v),
causing this test to be flaky; seen failing on windows 2022;

    === FAIL: github.com/docker/docker/integration/image TestSaveRepoWithMultipleImages (23.16s)
        save_test.go:104: timeout waiting for container to exit

Looking at the test, we run a container only to commit it, and the test
does not make changes to the container's filesystem; it only runs a container
with a custom command (`true`).

Instead of running the container, we can _create_ a container and commit it;
this simplifies the tests, and prevents having to wait for the container to
exit (before committing).

To verify:

    make BIND_DIR=. DOCKER_GRAPHDRIVER=vfs TEST_FILTER=TestSaveRepoWithMultipleImages test-integration

    INFO: Testing against a local daemon
    === RUN   TestSaveRepoWithMultipleImages
    --- PASS: TestSaveRepoWithMultipleImages (1.20s)
    PASS

    DONE 1 tests in 2.668s

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-31 10:32:02 +01:00
Tonis Tiigi
a99bb24ea8
builder-next: make sure worker platforms normalized for containerd
These platforms are filled by default from containerd
introspection API and may not be normalized. Initializing
wrong platform in here results in incorrect platform
for BUILDPLATFORM and TARGETPLATFORM build-args for
Dockerfile frontend (and probably other side effects).

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2023-10-30 23:14:34 -07:00
Sebastiaan van Stijn
e9efc0a361
Merge pull request #46741 from vvoland/c8d-push-notags
c8d/push: Return error when repository has no tags
2023-10-30 20:08:31 +01:00
Sebastiaan van Stijn
171f153e2c
Merge pull request #46728 from rumpl/golangci-lint
Move the golangci-lint configuration file to the root dir
2023-10-30 18:55:17 +01:00
Djordje Lukic
d60de4cafd
Move the golangci-lint configuration file to the root dir
This will help other tools and IDEs find our lint configuration

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-10-30 13:48:58 +01:00
Sebastiaan van Stijn
dcf7287d64
Merge pull request #46662 from vvoland/c8d-pull-access-denied-msg-2
c8d/pull: Don't wrap `no basic auth` error
2023-10-30 12:43:41 +01:00
Paweł Gronowski
cd95cd0671
c8d/push: Return error when repository has no tags
In case of `docker push -a`, we need to return an error if there is no
image for the given repository.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-10-30 10:55:08 +01:00
Paweł Gronowski
df34db1158
c8d/pull: Don't wrap no basic auth error
Don't wrap the `no basic auth credentials` error from containerd and
return it as-is.

The error will look like:
```
failed to resolve reference "docker.io/library/aodkoakds:latest": pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed
```

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-10-30 09:40:08 +01:00
Albin Kerouanton
5a83bebf76
ci: Setup otel tracing for windows integration tests
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-10-28 14:45:30 +02:00
Sebastiaan van Stijn
f44b085da6
Merge pull request #46732 from laurazard/add-exec-group-add-user-test
exec/tests: add test for `--group-add` with `--user`
2023-10-27 15:14:52 +02:00
Sebastiaan van Stijn
53ebecea7c
Merge pull request #46734 from thaJeztah/vendor_containerd_1.7.8
vendor: github.com/containerd/containerd v1.7.8
2023-10-27 15:13:09 +02:00
Sebastiaan van Stijn
ae75dafe47
Merge pull request #46735 from thaJeztah/containerd_binary_1.7.8
update containerd binary to v1.7.8
2023-10-27 14:50:04 +02:00