Commit graph

126 commits

Author SHA1 Message Date
Sebastiaan van Stijn
dfd331b2c8
pkg/plugins/transport: export httpTransport, and return concrete type
Make NewHTTPTransport return a concrete type.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-25 22:38:37 +02:00
Sebastiaan van Stijn
77103c7c03
pkg/plugins/transport: inline newHTTPRequest
It was only used in a single location; just inline the code

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-25 22:38:37 +02:00
Sebastiaan van Stijn
4af4adc7ba
pkg/plugins: remove LocalRegistry.SpecsPaths()
This field was exported, but never mutated outside of the package, and
effectively a rather "creative" way to define a method on LocalRegistry.

While un-exporting also store these paths in a field, instead of constructing
them on every call, as the results won't change during the lifecycle of the
LocalRegistry.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-25 22:38:36 +02:00
Sebastiaan van Stijn
a44c25c2f1
pkg/plugins: split exported from implementation
Split the exported SpecsPaths from the platform-specific implementations,
so that documentation can be maintained in a single location.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-25 22:38:11 +02:00
Sebastiaan van Stijn
954d50b88a
plg/plugins: rename vars that collided, or poorly cased
Reduce some noise while reading the code in my IDE :)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-25 22:35:43 +02:00
Sebastiaan van Stijn
20889dbac6
pkg/plugins: move Plugin.ScopedPath to platform-agnostic file
Since 0e5eaf8ee3, these implementations
were fully identical, so removing the duplicate, and move it to a
platform-agnostic file.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-25 22:35:40 +02:00
Bjorn Neergaard
0c6b616656
Merge pull request #45946 from thaJeztah/plugin_refactor_setupRemotePluginServer
pkg/plugins: don't share plugin server between tests and t.Parallel()
2023-07-17 17:51:04 -06:00
Sebastiaan van Stijn
6b7705d5b2
pkg/plugins: use a dummy hostname for local connections
For local communications (npipe://, unix://), the hostname is not used,
but we need valid and meaningful hostname.

The current code used the socket path as hostname, which gets rejected by
go1.20.6 and go1.19.11 because of a security fix for [CVE-2023-29406 ][1],
which was implemented in  https://go.dev/issue/60374.

Prior versions go Go would clean the host header, and strip slashes in the
process, but go1.20.6 and go1.19.11 no longer do, and reject the host
header.

Before this patch, tests would fail on go1.20.6:

    === FAIL: pkg/authorization TestAuthZRequestPlugin (15.01s)
    time="2023-07-12T12:53:45Z" level=warning msg="Unable to connect to plugin: //tmp/authz2422457390/authz-test-plugin.sock/AuthZPlugin.AuthZReq: Post \"http://%2F%2Ftmp%2Fauthz2422457390%2Fauthz-test-plugin.sock/AuthZPlugin.AuthZReq\": http: invalid Host header, retrying in 1s"
    time="2023-07-12T12:53:46Z" level=warning msg="Unable to connect to plugin: //tmp/authz2422457390/authz-test-plugin.sock/AuthZPlugin.AuthZReq: Post \"http://%2F%2Ftmp%2Fauthz2422457390%2Fauthz-test-plugin.sock/AuthZPlugin.AuthZReq\": http: invalid Host header, retrying in 2s"
    time="2023-07-12T12:53:48Z" level=warning msg="Unable to connect to plugin: //tmp/authz2422457390/authz-test-plugin.sock/AuthZPlugin.AuthZReq: Post \"http://%2F%2Ftmp%2Fauthz2422457390%2Fauthz-test-plugin.sock/AuthZPlugin.AuthZReq\": http: invalid Host header, retrying in 4s"
    time="2023-07-12T12:53:52Z" level=warning msg="Unable to connect to plugin: //tmp/authz2422457390/authz-test-plugin.sock/AuthZPlugin.AuthZReq: Post \"http://%2F%2Ftmp%2Fauthz2422457390%2Fauthz-test-plugin.sock/AuthZPlugin.AuthZReq\": http: invalid Host header, retrying in 8s"
        authz_unix_test.go:82: Failed to authorize request Post "http://%2F%2Ftmp%2Fauthz2422457390%2Fauthz-test-plugin.sock/AuthZPlugin.AuthZReq": http: invalid Host header

[1]: https://github.com/advisories/GHSA-f8f7-69v5-w4vx

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-14 18:57:55 +02:00
Sebastiaan van Stijn
2cb982b506
pkg/plugins: run tests with t.Parallel()
Some tests are testing timeouts and take a long time to run. Run the tests
in parallel, so that the test-suite takes shorter to run.

Before:

    ok  github.com/docker/docker/pkg/plugins	34.013s

After:

    ok  github.com/docker/docker/pkg/plugins	17.945s

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-12 22:30:46 +02:00
Sebastiaan van Stijn
b39362295a
pkg/plugins: don't share plugin server between tests
Refactor setupRemotePluginServer() to be a helper, and to spin up a test-
server for each test instead of sharing the same instance between tests.

This allows the tests to be run in parallel without stepping on each-other's
toes (tearing down the server).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-12 22:30:43 +02:00
Cory Snider
9cee34bc94 pkg/plugins: make unit test less time sensitive
TestClientWithRequestTimeout has been observed to flake in CI. The
timing in the test is quite tight, only giving the client a 10ms window
to time out, which could potentially be missed if the host is under
load and the goroutine scheduling is unlucky. Give the client a full
five seconds of grace to time out before failing the test.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-07-05 12:09:37 -04:00
Sebastiaan van Stijn
1216328c2d
pkg/plugins: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:31:52 +02:00
Brian Goff
74da6a6363 Switch all logging to use containerd log pkg
This unifies our logging and allows us to propagate logging and trace
contexts together.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2023-06-24 00:23:44 +00:00
Sebastiaan van Stijn
ab35df454d
remove pre-go1.17 build-tags
Removed pre-go1.17 build-tags with go fix;

    go mod init
    go fix -mod=readonly ./...
    rm go.mod

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-19 20:38:51 +02:00
Jan Garcia
1e1d326b50 plugins: Move GlobalSpecPaths into LocalRegistry
Signed-off-by: Jan Garcia <github-public@n-garcia.com>
2023-01-10 21:33:49 +01:00
Jan Garcia
6ab12ec8f4 rootless: move ./rootless to ./pkg/rootless
Signed-off-by: Jan Garcia <github-public@n-garcia.com>
2023-01-09 16:26:06 +01:00
Jan Garcia
c1bd5e9144 Use user paths for plugin discovery in rootless mode
Signed-off-by: Jan Garcia <github-public@n-garcia.com>
2023-01-09 16:26:06 +01:00
Sebastiaan van Stijn
412c650e05
pkg/*: fix "empty-lines" (revive)
pkg/directory/directory.go:9:49: empty-lines: extra empty line at the start of a block (revive)
    pkg/pubsub/publisher.go:8:48: empty-lines: extra empty line at the start of a block (revive)
    pkg/loopback/attach_loopback.go:96:69: empty-lines: extra empty line at the start of a block (revive)
    pkg/devicemapper/devmapper_wrapper.go:136:48: empty-lines: extra empty line at the start of a block (revive)
    pkg/devicemapper/devmapper.go:391:35: empty-lines: extra empty line at the end of a block (revive)
    pkg/devicemapper/devmapper.go:676:35: empty-lines: extra empty line at the end of a block (revive)
    pkg/archive/changes_posix_test.go:15:38: empty-lines: extra empty line at the end of a block (revive)
    pkg/devicemapper/devmapper.go:241:51: empty-lines: extra empty line at the start of a block (revive)
    pkg/fileutils/fileutils_test.go:17:47: empty-lines: extra empty line at the end of a block (revive)
    pkg/fileutils/fileutils_test.go:34:48: empty-lines: extra empty line at the end of a block (revive)
    pkg/fileutils/fileutils_test.go:318:32: empty-lines: extra empty line at the end of a block (revive)
    pkg/tailfile/tailfile.go:171:6: empty-lines: extra empty line at the end of a block (revive)
    pkg/tarsum/fileinfosums_test.go:16:41: empty-lines: extra empty line at the end of a block (revive)
    pkg/tarsum/tarsum_test.go:198:42: empty-lines: extra empty line at the start of a block (revive)
    pkg/tarsum/tarsum_test.go:294:25: empty-lines: extra empty line at the start of a block (revive)
    pkg/tarsum/tarsum_test.go:407:34: empty-lines: extra empty line at the end of a block (revive)
    pkg/ioutils/fswriters_test.go:52:45: empty-lines: extra empty line at the end of a block (revive)
    pkg/ioutils/writers_test.go:24:39: empty-lines: extra empty line at the end of a block (revive)
    pkg/ioutils/bytespipe_test.go:78:26: empty-lines: extra empty line at the end of a block (revive)
    pkg/sysinfo/sysinfo_linux_test.go:13:37: empty-lines: extra empty line at the end of a block (revive)
    pkg/archive/archive_linux_test.go:57:64: empty-lines: extra empty line at the end of a block (revive)
    pkg/archive/changes.go:248:72: empty-lines: extra empty line at the start of a block (revive)
    pkg/archive/changes_posix_test.go:15:38: empty-lines: extra empty line at the end of a block (revive)
    pkg/archive/copy.go:248:124: empty-lines: extra empty line at the end of a block (revive)
    pkg/archive/diff_test.go:198:44: empty-lines: extra empty line at the end of a block (revive)
    pkg/archive/archive.go:304:12: empty-lines: extra empty line at the end of a block (revive)
    pkg/archive/archive.go:749:37: empty-lines: extra empty line at the end of a block (revive)
    pkg/archive/archive.go:812:81: empty-lines: extra empty line at the start of a block (revive)
    pkg/archive/copy_unix_test.go:347:34: empty-lines: extra empty line at the end of a block (revive)
    pkg/system/path.go:11:39: empty-lines: extra empty line at the end of a block (revive)
    pkg/system/meminfo_linux.go:29:21: empty-lines: extra empty line at the end of a block (revive)
    pkg/plugins/plugins.go:135:32: empty-lines: extra empty line at the end of a block (revive)
    pkg/authorization/response.go:71:48: empty-lines: extra empty line at the start of a block (revive)
    pkg/authorization/api_test.go:18:51: empty-lines: extra empty line at the end of a block (revive)
    pkg/authorization/middleware_test.go:23:44: empty-lines: extra empty line at the end of a block (revive)
    pkg/authorization/middleware_unix_test.go:17:46: empty-lines: extra empty line at the end of a block (revive)
    pkg/authorization/api_test.go:57:45: empty-lines: extra empty line at the end of a block (revive)
    pkg/authorization/response.go:83:50: empty-lines: extra empty line at the start of a block (revive)
    pkg/authorization/api_test.go:66:47: empty-lines: extra empty line at the end of a block (revive)
    pkg/authorization/middleware_unix_test.go:45:48: empty-lines: extra empty line at the end of a block (revive)
    pkg/authorization/response.go:145:75: empty-lines: extra empty line at the start of a block (revive)
    pkg/authorization/middleware_unix_test.go:56:51: empty-lines: extra empty line at the end of a block (revive)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-28 01:58:49 +02:00
Bjorn Neergaard
85fa72c599 chore: update supported go version to 1.18+
The 1.16 `io/fs` compatibility code was being built on 1.18 and 1.19.
Drop it completely as 1.16 is long EOL, and additionally drop 1.17 as it
has been EOL for a month and 1.18 is both the minimum Go supported by
the 20.10 branch, as well as a very easy jump from 1.17.

Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
2022-09-15 13:52:39 -06:00
Sebastiaan van Stijn
52c1a2fae8
gofmt GoDoc comments with go1.19
Older versions of Go don't format comments, so committing this as
a separate commit, so that we can already make these changes before
we upgrade to Go 1.19.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-08 19:56:23 +02:00
Sebastiaan van Stijn
4203a97aad
staticcheck: ignore "SA1019: strings.Title is deprecated"
This function is marked deprecated in Go 1.18; however, the suggested replacement
brings in a large amount of new code, and most strings we generate will be ASCII,
so this would only be in case it's used for some user-provided string. We also
don't have a language to use, so would be using the "default".

Adding a `//nolint` comment to suppress the linting failure instead.

    daemon/logger/templates/templates.go:23:14: SA1019: strings.Title is deprecated: The rule Title uses for word boundaries does not handle Unicode punctuation properly. Use golang.org/x/text/cases instead. (staticcheck)
        "title":    strings.Title,
                    ^
    pkg/plugins/pluginrpc-gen/template.go:67:9: SA1019: strings.Title is deprecated: The rule Title uses for word boundaries does not handle Unicode punctuation properly. Use golang.org/x/text/cases instead. (staticcheck)
        return strings.Title(s)
               ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-16 12:11:54 +01:00
Akihiro Suda
4283e93e64
Merge pull request #41215 from cpuguy83/better_plugin_errors
Add more error details on plugin get
2021-10-24 12:37:07 +09:00
Sebastiaan van Stijn
303ea8e820
pkg/plugins: fix compatibility with go1.16
commit c55a4ac779 changed the ioutil utilities
to use the new os variants, per recommendation from the go 1.16 release notes:
https://golang.org/doc/go1.16#ioutil

> we encourage new code to use the new definitions in the io and os packages.
> Here is a list of the new locations of the names exported by io/ioutil:

However, the devil is in the detail, and io.ReadDir() is not a direct
replacement for ioutil.ReadDir();

> ReadDir => os.ReadDir (note: returns a slice of os.DirEntry rather than a slice of fs.FileInfo)

go1.16 added a io.FileInfoToDirEntry() utility to concert a DirEntry to
a FileInfo, but it's not available in go1.16

This patch copies the FileInfoToDirEntry code, and uses it for go1.16.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-08-31 15:42:54 +02:00
Eng Zer Jun
c55a4ac779
refactor: move from io/ioutil to io and os package
The io/ioutil package has been deprecated in Go 1.16. This commit
replaces the existing io/ioutil functions with their new definitions in
io and os packages.

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2021-08-27 14:56:57 +08:00
Sebastiaan van Stijn
686be57d0a
Update to Go 1.17.0, and gofmt with Go 1.17
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-08-24 23:33:27 +02:00
Sebastiaan van Stijn
00a4f67ddf
Reduce TestClientWithRequestTimeout flakiness
The test sometimes failed because no error was returned:

    === Failed
    === FAIL: pkg/plugins TestClientWithRequestTimeout (0.00s)
         client_test.go:254: assertion failed: expected an error, got nil: expected error

Possibly caused by a race condition, as the sleep was just 1 ms longer than the timeout;
this patch is increasing the sleep in the response to try to reduce flakiness.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-06-29 23:53:23 +02:00
Brian Goff
b1a3fe4934 Add more error details on plugin get
Seeing this `ErrNotImplements` in some of our logs and it's not very
helpful because we don't know what plugin is causing it or even what the
requested interface is.

```
{"message":"legacy plugin: Plugin does not implement the requested driver"}
```

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2020-07-15 13:10:30 -07:00
Sebastiaan van Stijn
07d60bc257
Replace errors.Cause() with errors.Is() / errors.As()
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-04-29 00:28:41 +02:00
Sebastiaan van Stijn
9f0b3f5609
bump gotest.tools v3.0.1 for compatibility with Go 1.14
full diff: https://github.com/gotestyourself/gotest.tools/compare/v2.3.0...v3.0.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-02-11 00:06:42 +01:00
Sebastiaan van Stijn
aa655a4d73
pkg/plugins: use constants for http methods
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-13 19:11:30 +02:00
Brian Goff
e6091b28c5
Merge pull request #39422 from lemrouch/35876-workaround
Don't try to load plugin without name
2019-07-22 10:21:50 -07:00
Sebastiaan van Stijn
c7816c5323
Harden TestClientWithRequestTimeout
DeadlineExceeded now implements a TimeOut() function,
since dc4427f372

Check for this interface, to prevent possibly incorrect failures;

```
00:16:41 --- FAIL: TestClientWithRequestTimeout (0.00s)
00:16:41     client_test.go:259: assertion failed:
00:16:41         --- context.DeadlineExceeded
00:16:41         +++ err
00:16:41         :
00:16:41         	-: context.deadlineExceededError{}
00:16:41         	+: &net.OpError{Op: "dial", Net: "tcp", Addr: s"127.0.0.1:49294", Err: &poll.TimeoutError{}}
00:16:41
```

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-07-12 12:31:51 +02:00
Pavel Matěja
ee09f5a4af Don't try to load plugin without name
This can happen when you have --config-only network
Such attempt will fail anyway and it will create 15s delay in container
startup

Signed-off-by: Pavel Matěja <pavel@verotel.cz>
2019-07-08 14:52:45 +02:00
mooncake
35739fd290 Remove duplicated words in pkg files
Signed-off-by: mooncake <xcoder@tenxcloud.com>
2018-10-05 22:32:14 +08:00
Vincent Demeester
3845728524
Update tests to use gotest.tools 👼
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2018-06-13 09:04:30 +02:00
Brian Goff
f51a96c016 Move plugin client creation to the extension point
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2018-05-25 15:18:53 -04:00
Daniel Nephin
c9e52bd0da Post migration assertion fixes
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2018-03-16 11:03:46 -04:00
Daniel Nephin
6be0f70983 Automated migration using
gty-migrate-from-testify --ignore-build-tags

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2018-03-16 11:03:43 -04:00
Brian Goff
7ca971fb49 Adds a unit test for plugin request timeout
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2018-03-05 15:38:56 -05:00
Sebastiaan van Stijn
20028325da
Merge pull request #35829 from cpuguy83/no_private_mount_for_plugins
Perform plugin mounts in the runtime
2018-02-21 12:28:13 +01:00
Brian Goff
0e5eaf8ee3 Ensure plugin returns correctly scoped paths
Before this change, volume management was relying on the fact that
everything the plugin mounts is visible on the host within the plugin's
rootfs. In practice this caused some issues with mount leaks, so we
changed the behavior such that mounts are not visible on the plugin's
rootfs, but available outside of it, which breaks volume management.

To fix the issue, allow the plugin to scope the path correctly rather
than assuming that everything is visible in `p.Rootfs`.
In practice this is just scoping the `PropagatedMount` paths to the
correct host path.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2018-02-07 15:48:27 -05:00
Daniel Nephin
4f0d95fa6e Add canonical import comment
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2018-02-05 16:51:57 -05:00
Brian Goff
b27f70d45a Fix issue with plugin scanner going to deep
The plugin spec says that plugins can live in one of:

- /var/run/docker/plugins/<name>.sock
- /var/run/docker/plugins/<name>/<name>.sock
- /etc/docker/plugins/<name>.[json,spec]
- /etc/docker/plugins/<name>/<name>.<json,spec>
- /usr/lib/docker/plugins/<name>.<json,spec>
- /usr/lib/docker/plugins/<name>/<name>.<json,spec>

However, the plugin scanner which is used by the volume list API was
doing `filepath.Walk`, which will walk the entire tree for each of the
supported paths.
This means that even v2 plugins in
`/var/run/docker/plugins/<id>/<name>.sock` were being detected as a v1
plugin.
When the v1 plugin loader tried to load such a plugin it would log an
error that it couldn't find it because it doesn't match one of the
supported patterns... e.g. when in a subdir, the subdir name must match
the plugin name for the socket.

There is no behavior change as the error is only on the `Scan()` call,
which is passing names to the plugin registry when someone calls the
volume list API.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2018-02-02 16:49:14 -05:00
Brian Goff
b15f8d2d4f Add timeouts for volume plugin ops
This protects the daemon from volume plugins that are slow or
deadlocked.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2018-01-16 20:30:49 -05:00
Daniel Nephin
09652bf878 Add ineffassign linter
Also enable GC in linting to reduce memory usage.

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-09-08 18:23:21 -04:00
Daniel Nephin
62c1f0ef41 Add deadcode linter
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-08-21 18:18:50 -04:00
Derek McGowan
1009e6a40b
Update logrus to v1.0.1
Fixes case sensitivity issue

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2017-07-31 13:16:46 -07:00
Josh Soref
39bcaee47b
Spelling fixes
* additional
* ambiguous
* anonymous
* anything
* application
* because
* before
* building
* capabilities
* circumstances
* commit
* committer
* compresses
* concatenated
* config
* container
* container's
* current
* definition
* delimiter
* disassociates
* discovery
* distributed
* doesnotexist
* downloads
* duplicates
* either
* enhancing
* enumerate
* escapable
* exactly
* expect
* expectations
* expected
* explicitly
* false
* filesystem
* following
* forbidden
* git with
* healthcheck
* ignore
* independent
* inheritance
* investigating
* irrelevant
* it
* logging
* looking
* membership
* mimic
* minimum
* modify
* mountpoint
* multiline
* notifier
* outputting
* outside
* overridden
* override
* parsable
* plugins
* precedence
* propagation
* provided
* provides
* registries
* repositories
* returning
* settings
* should
* signals
* someone
* something
* specifically
* successfully
* synchronize
* they've
* thinking
* uninitialized
* unintentionally
* unmarshaling
* unnamed
* unreferenced
* verify

Signed-off-by: Josh Soref <jsoref@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2017-07-03 13:13:09 -07:00
Raja Sami
8dd100a229 Increase the Coverage of pkg/plugins
Increases the test coverage of pkg/plugins.
Changed signature of function NewClientWithTimeout in pkg/plugin/client, to
take time.Duration instead of integers.

Signed-off-by: Raja Sami <raja.sami@tenpearl.com>
2017-06-12 12:23:10 +05:00
Felix Abecassis
62871ef2fa Do not reuse a http.Request after a failure in callWithRetry
Closes: #33412

Signed-off-by: Felix Abecassis <fabecassis@nvidia.com>
2017-05-26 18:02:31 -07:00