Commit graph

3395 commits

Author SHA1 Message Date
Paweł Gronowski
c01bbbddeb
pkg/ioutils: Make subsequent Close attempts noop
Turn subsequent `Close` calls into a no-op and produce a warning with an
optional stack trace (if debug mode is enabled).

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 585d74bad1)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-25 15:15:01 +01:00
Sebastiaan van Stijn
f8eaa14a18
pkg/platforms: internalize in daemon/containerd
This matcher was only used internally in the containerd implementation of
the image store. Un-export it, and make it a local utility in that package
to prevent external use.

This package was introduced in 1616a09b61
(v24.0), and there are no known external consumers of this package, so there
should be no need to deprecate / alias the old location.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 94b4765363)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-23 15:05:50 +01:00
Cory Snider
c0573b133f
pkg/system: return even richer xattr errors
The names of extended attributes are not completely freeform. Attributes
are namespaced, and the kernel enforces (among other things) that only
attributes whose names are prefixed with a valid namespace are
permitted. The name of the attribute therefore needs to be known in
order to diagnose issues with lsetxattr. Include the name of the
extended attribute in the errors returned from the Lsetxattr and
Lgetxattr so users and us can more easily troubleshoot xattr-related
issues. Include the name in a separate rich-error field to provide code
handling the error enough information to determine whether or not the
failure can be ignored.

Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit 43bf65c174)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-23 09:27:49 +01:00
Sebastiaan van Stijn
1f6c42c678
Merge pull request #47006 from thaJeztah/deprecate_ResolveScopedPath
pkg/containerfs: unify CleanScopedPath implementation, and deprecate ResolveScopedPath
2024-01-03 15:38:52 +01:00
Sebastiaan van Stijn
b8f2caa80a
pkg/containerfs: deprecate ResolveScopedPath
If was a very shallow wrapper around symlink.FollowSymlinkInScope, so inline
that code instead.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-02 15:32:31 +01:00
Sebastiaan van Stijn
90f37f48e2
pkg/containerfs: unify CleanScopedPath implementation
Use stdlib's filepath.VolumeName to get the volume-name (if present) instead
of a self-crafted implementation, and unify the implementations for Windows
and Unix.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-02 15:26:29 +01:00
Sebastiaan van Stijn
6fae583dba
pkg/aaparser: remove, and integrate into profiles/apparmor
This package provided utilities to obtain the apparmor_parser version, as well
as loading a profile.

Commit e3e715666f (included in v24.0.0 through
bfffb0974e) deprecated GetVersion, as it was no
longer used, which made LoadProfile the only utility remaining in this package.

LoadProfile appears to have no external consumers, and the only use in our code
is "profiles/apparmor".

This patch moves the remaining code (LoadProfile) to profiles/apparmor as a
non-exported function, and deletes the package.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-02 15:15:49 +01:00
Sebastiaan van Stijn
53d405c37f
pkg/aaparser: remove deprecated GetVersion utility
Commit e3e715666f (included in v24.0.0 through
bfffb0974e) deprecated GetVersion, as it was no
longer used.

This patch removes the deprecated utility, and inlines the remaining code into
the LoadProfile function.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-02 14:10:37 +01:00
Brian Goff
677d41aa3b Plumb context through info endpoint
I was trying to find out why `docker info` was sometimes slow so
plumbing a context through to propagate trace data through.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2023-11-10 20:09:25 +00:00
Cory Snider
63a9d72ee8 pkg/tarsum: handle xattrs like archive/tar does
When writing a tar file with archive/tar, extended attributes in the
deprecated (tar.Header).Xattrs map take precedence over conflicting
'SCHILY.xattr' records in the (tar.Header).PAXRecords map. Update
package tarsum to follow the same precedence rules as archive/tar.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-10-26 19:23:10 -04:00
Sebastiaan van Stijn
7cabe08399
Merge pull request #46707 from corhere/replace-xattrs-with-paxrecords
pkg/archive, pkg/tarsum: replace use of Xattrs with PAXRecords
2023-10-26 21:37:44 +02:00
Sebastiaan van Stijn
df3a321164
migrate to github.com/moby/sys/user
The github.com/opencontainers/runc/libcontainer/user package was moved
to a separate module. While there's still uses of the old module in
our code-base, runc itself is migrating to the new module, and deprecated
the old package (for runc 1.2).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-24 15:45:02 +02:00
Cory Snider
c44c9dfa79 pkg/tarsum: migrate to (tar.Header).PAXRecords
Fix a silly bug in the implementation which had the effect of
len(h.Xattrs) blank entries being inserted in the middle of
orderedHeaders. Luckily this is not a load-bearing bug: empty headers
are ignored as the tarsum digest is computed by concatenating header
keys and values without any intervening delimiter.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-10-23 16:21:41 -04:00
Cory Snider
3cf409aa9e pkg/archive: migrate to (tar.Header).PAXRecords
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-10-23 16:21:41 -04:00
Cory Snider
6a8a792019 pkg/archive: test tar headers are interoperable
The existing pkg/archive unit tests are primarily round-trip tests which
assert that pkg/archive produces tarballs which pkg/archive can unpack.
While these tests are effective at catching regressions in archiving or
unarchiving, they have a blind spot for regressions in compatibility
with the rest of the ecosystem. For example, a typo in the capabilities
extended attribute constant would result in subtly broken image layer
tarballs, but the existing tests would not catch the bug if both the
archiving and unarchiving implementations have the same typo.

Extend the test for archiving an overlay filesystem layer to assert that
the overlayfs style whiteouts (extended attributes and device files) are
transformed into AUFS-style whiteouts (magic file names).

Extend the test for archiving files with extended attributes to assert
that the extended attribute is encoded into the file's tar header in the
standard, interoperable format compatible with the rest of the
ecosystem.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-10-23 16:21:41 -04:00
Bjorn Neergaard
796c05ca32
Merge pull request #46094 from slonopotamus/darwin-build-fix
pkg/chrootarchive: fix Darwin build
2023-10-12 15:15:18 -07:00
Sebastiaan van Stijn
056be8b7b0
Merge pull request #46473 from thaJeztah/c8d_logs
migrate to github.com/containerd/log v0.1.0
2023-10-12 10:35:02 +02:00
Sebastiaan van Stijn
cff4f20c44
migrate to github.com/containerd/log v0.1.0
The github.com/containerd/containerd/log package was moved to a separate
module, which will also be used by upcoming (patch) releases of containerd.

This patch moves our own uses of the package to use the new module.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-11 17:52:23 +02:00
Akihiro Suda
ad877271f3
Limit OOMScoreAdj when running in UserNS ("Rootful-in-Rootless")
Fix issue 46563 "Rootful-in-Rootless dind doesn't work since systemd v250 (due to oom score adj)"

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2023-09-29 21:50:08 +09:00
Marat Radchenko
9e3ed7b429 pkg/chrootarchive: fix Darwin build
Before this commit, `doPack`, `doUnpack` and `doUnpackLayer` were not implemented for Darwin, causing build failure.

This change allows all non-Linux Unixes to use FreeBSD reexec-based pack/unpack implementation

See also: moby/buildkit#4059
See also: 8b843732b3

Signed-off-by: Marat Radchenko <marat@slonopotamus.org>
2023-09-28 16:23:33 +03:00
Sebastiaan van Stijn
bf692d47fb
deprecate pkg/loopback (utility package for devicemapper)
This package was introduced in af59752712
as a utility package for devicemapper, which was removed in commit
dc11d2a2d8 (v25.0.0).

It looks like there's no external consumers of this package, so we should
consider removing it, but deprecating it first, just in case.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-18 13:44:44 +02:00
Sebastiaan van Stijn
a3c97beee0
image: implement CheckOS, deprecate pkg/system IsOSSupported
Implement a function that returns an error to replace existing uses of
the IsOSSupported utility, where callers had to produce the error after
checking.

The IsOSSupported function was used in combination with images, so implementing
a utility in "image" to prevent having to import pkg/system (which contains many
unrelated functions)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-07 22:14:44 +02:00
Sebastiaan van Stijn
150b657bad
pkg/system: move ErrNotSupportedOperatingSystem to where it's used
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-07 22:04:33 +02:00
Sebastiaan van Stijn
b5376c7cec
pkg/idtools: remove sync.Once, and include lookup error
When running a `docker cp` to copy files to/from a container, the
lookup of the `getent` executable happens within the container's
filesystem, so we cannot re-use the results.

Unfortunately, that also means we can't preserve the results for
any other uses of these functions, but probably the lookup should not
be "too" costly.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-06 17:34:26 +02:00
Sebastiaan van Stijn
79495c5b6a
pkg/tailfile: rename max/min as it collides with go1.21 builtin
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-26 22:02:25 +02:00
Sebastiaan van Stijn
df2f25a977
pkg/plugins: rename max/min as it collides with go1.21 builtin
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-26 22:02:25 +02:00
Sebastiaan van Stijn
6c036f267f
pkg/sysinfo: rename max/min as it collides with go1.21 builtin
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-26 22:02:25 +02:00
Sebastiaan van Stijn
55192de9e3
pkg/archive: rename max/min as it collides with go1.21 builtin
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-26 22:02:25 +02:00
Sebastiaan van Stijn
6d5204b63e
Merge pull request #46316 from thaJeztah/rm_more_buildtags
remove some remaining pre-go1.17 build-tags
2023-08-25 01:06:39 +02:00
Sebastiaan van Stijn
211d3a8c32
pkg/stack: format code with gofumpt
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-24 17:59:01 +02:00
Sebastiaan van Stijn
95e99e629c
pkg/fileutils: format code with gofumpt
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-24 17:58:46 +02:00
Sebastiaan van Stijn
6c6e1fb13d
pkg/containerfs: format code with gofumpt
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-24 17:58:32 +02:00
Sebastiaan van Stijn
1e579a7a2a
pkg/sysinfo: format code with gofumpt
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-24 17:58:12 +02:00
Sebastiaan van Stijn
976da91fb7
pkg/loopback: format code with gofumpt
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-24 17:56:50 +02:00
Sebastiaan van Stijn
b4d6eca9b8
pkg/chrootarchive: format code with gofumpt
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-24 17:56:24 +02:00
Sebastiaan van Stijn
498da44aab
remove some remaining pre-go1.17 build-tags
commit ab35df454d removed most of the pre-go1.17
build-tags, but for some reason, "go fix" doesn't remove these, so removing
the remaining ones manually

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-24 17:51:07 +02:00
Paweł Gronowski
eb56493f4e
c8d/commit: Don't produce an empty layer
If the diff is empty and don't produce an empty layer.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-08-22 17:32:18 +02:00
Paweł Gronowski
11a0c2779b
pkg/plugins: Rewrite with assert.Check
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-08-09 12:22:25 +02:00
Sebastiaan van Stijn
05ef5559c3
pkg/plugins: override timeouts during tests
Some tests were testing non-existing plugins, but therefore triggered
the retry-loop, which times out after 15-30 seconds. Add some options
to allow overriding this timeout during tests.

Before:

    go test -v -run '^(TestGet|TestNewClientWithTimeout)$'
    === RUN   TestGet
    === RUN   TestGet/success
    === RUN   TestGet/not_implemented
    === RUN   TestGet/not_exists
    WARN[0000] Unable to locate plugin: vegetable, retrying in 1s
    WARN[0001] Unable to locate plugin: vegetable, retrying in 2s
    WARN[0003] Unable to locate plugin: vegetable, retrying in 4s
    WARN[0007] Unable to locate plugin: vegetable, retrying in 8s
    --- PASS: TestGet (15.02s)
        --- PASS: TestGet/success (0.00s)
        --- PASS: TestGet/not_implemented (0.00s)
        --- PASS: TestGet/not_exists (15.02s)
    === RUN   TestNewClientWithTimeout
        client_test.go:166: started remote plugin server listening on: http://127.0.0.1:36275
    WARN[0015] Unable to connect to plugin: 127.0.0.1:36275/Test.Echo: Post "http://127.0.0.1:36275/Test.Echo": context deadline exceeded (Client.Timeout exceeded while awaiting headers), retrying in 1s
    WARN[0017] Unable to connect to plugin: 127.0.0.1:36275/Test.Echo: Post "http://127.0.0.1:36275/Test.Echo": context deadline exceeded (Client.Timeout exceeded while awaiting headers), retrying in 2s
    WARN[0019] Unable to connect to plugin: 127.0.0.1:36275/Test.Echo: Post "http://127.0.0.1:36275/Test.Echo": net/http: request canceled (Client.Timeout exceeded while awaiting headers), retrying in 4s
    WARN[0024] Unable to connect to plugin: 127.0.0.1:36275/Test.Echo: Post "http://127.0.0.1:36275/Test.Echo": net/http: request canceled (Client.Timeout exceeded while awaiting headers), retrying in 8s
    --- PASS: TestNewClientWithTimeout (17.64s)
    PASS
    ok  	github.com/docker/docker/pkg/plugins	32.664s

After:

    go test -v -run '^(TestGet|TestNewClientWithTimeout)$'
    === RUN   TestGet
    === RUN   TestGet/success
    === RUN   TestGet/not_implemented
    === RUN   TestGet/not_exists
    WARN[0000] Unable to locate plugin: this-plugin-does-not-exist, retrying in 1s
    --- PASS: TestGet (1.00s)
        --- PASS: TestGet/success (0.00s)
        --- PASS: TestGet/not_implemented (0.00s)
        --- PASS: TestGet/not_exists (1.00s)
    === RUN   TestNewClientWithTimeout
        client_test.go:167: started remote plugin server listening on: http://127.0.0.1:45973
    --- PASS: TestNewClientWithTimeout (0.04s)
    PASS
    ok  	github.com/docker/docker/pkg/plugins	1.050s

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-08 15:41:11 +02:00
Sebastiaan van Stijn
b663c7c5c3
pkg/plugins: add convenience alias for VersionMimeType
Add an alias in the pkg/plugins package.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-07 20:38:51 +02:00
Brian Goff
b1c112d35e
Merge pull request #46157 from vvoland/pkg-plugins-racy 2023-08-05 10:46:00 -07:00
Paweł Gronowski
0034a98eb1
pkg/plugins: Guard storage and unparallel racy tests
These tests were made parallel to speed up the execution, but this
turned out to be flaky, because they mutate some shared state.

The tests use shared `storage` variable without any synchronization.
However, adding synchronization is not enough in all cases, some tests
register the same plugin, so they can't be run in parallel to each
other.

This commit adds the synchronization around `storage` variable
modification and removes parallel from the tests where it's not enough.

Before:
```
$ go test -race -v . -count 1
...
--- FAIL: TestGet (15.02s)
    --- FAIL: TestGet/not_implemented (0.00s)
        testing.go:1446: race detected during execution of test
    testing.go:1446: race detected during execution of test
FAIL
FAIL    github.com/docker/docker/pkg/plugins    17.655s
FAIL
```

After:
```
$ go test -race -v . -count 1
ok      github.com/docker/docker/pkg/plugins    32.702s
```

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-08-04 11:22:41 +02:00
Sebastiaan van Stijn
5e2a1195d7
swap logrus types for their containerd/logs aliases
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-01 13:02:55 +02:00
Sebastiaan van Stijn
0f7bf67f83
pkg/plugins: TestGet(): use sub-tests
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-25 22:38:38 +02:00
Sebastiaan van Stijn
e1ad4aa002
pkg/plugins: make package-level socketsPath var a LocalRegistry field
This variable was only accessed from within LocalRegistry methods, but
due to being a package-level variable, tests had to deal with setting
and resetting it.

Move it to be a field scoped to the LocalRegistry. This simplifies the
tests, and to make this more transparent, also removing the "Setup()"
helper (which, wasn't marked as a t.Helper()).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-25 22:38:37 +02:00
Sebastiaan van Stijn
5bd44cf3c4
pkg/plugins: remove "load()" function
It was used in a single place and was abstracting "loadWithRetry";
let's just inline it.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-25 22:38:37 +02:00
Sebastiaan van Stijn
77c03221c9
pkg/plugins/transport: remove RequestFactory interface
The client's transport can only be set by newClientWithTransport, which
is not exported, and always uses a transport.HTTPTransport.

However, requestFactory is mocked in one of the tests, so keep the interface,
but make it a local, non-exported one.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-25 22:38:37 +02:00
Sebastiaan van Stijn
d12ec5f796
pkg/plugins/transport: remove unused Transport interface
The interface is not consumed anywhere, and only non-exported functions
produced one, so we can remove it.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-25 22:38:37 +02:00
Sebastiaan van Stijn
dfd331b2c8
pkg/plugins/transport: export httpTransport, and return concrete type
Make NewHTTPTransport return a concrete type.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-25 22:38:37 +02:00
Sebastiaan van Stijn
77103c7c03
pkg/plugins/transport: inline newHTTPRequest
It was only used in a single location; just inline the code

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-25 22:38:37 +02:00