Commit graph

44018 commits

Author SHA1 Message Date
Sebastiaan van Stijn
90de570cfa
backend: add StopOptions to ContainerRestart and ContainerStop
While we're modifying the interface, also add a context to both.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-20 21:29:30 +02:00
Sebastiaan van Stijn
952902efbc
daemon: containerStop(): use a regular "defer" to log container event
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-20 21:29:27 +02:00
Sebastiaan van Stijn
5edf9acf9c
daemon: move default stop-timeout to containerStop()
This avoids having to determine what the default is in various
parts of the code. If no custom timeout is passed (nil), the
default will be used.

Also remove the named return variable from cleanupContainer(),
as it wasn't used.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-20 21:29:15 +02:00
Sebastiaan van Stijn
f3bce92a24
daemon: cleanupContainer(): pass ContainerRmConfig as parameter
We already have this config, so might as well pass it, instead of passing
each option as a separate argument.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-20 21:27:24 +02:00
Sebastiaan van Stijn
4430992af8
daemon: rename some variables, import-aliases and receivers
- daemon/delete: rename var that collided with import, remove output var
- daemon: fix inconsistent receiver name and package aliases
- daemon/stop: rename imports and variables to standard naming
  This is in preparation of some changes, but keeping it in  a
  separate commit to make review of other changes easier.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-20 21:22:28 +02:00
Brian Goff
c4e3a16373
Merge pull request #43494 from thaJeztah/client_cp_error_handling
client: CopyToContainer(), CopyFromContainer(): remove status-code handling
2022-04-20 10:36:19 -07:00
Brian Goff
9b4c516864
Merge pull request #43471 from thaJeztah/grpc_nit
api/server/router/grpc: fix some nits in NewRouter()
2022-04-20 09:59:58 -07:00
Sebastiaan van Stijn
79becebd3d
Merge pull request #43449 from thaJeztah/daemon_config_cleanup
daemon: move more consts for default configuration values to daemon/config
2022-04-20 17:01:19 +02:00
Sebastiaan van Stijn
86ce946945
Merge pull request #43431 from crazy-max/goversioninfo
Switch to go-winres to create Windows resources
2022-04-20 16:34:48 +02:00
Tianon Gravi
efd49bb6c0
Merge pull request #43501 from thaJeztah/document_imagesummary
api: swagger: document ImageSummary fields
2022-04-20 06:41:19 -07:00
Sebastiaan van Stijn
301b252b58
libnetwork: don't use strings.Fields() to improve performance
While looking at this code, I noticed that we were wasting quite some resources
by first constructing a string, only to split it again (with `strings.Fields()`)
into a string slice.

Some conversions were also happening multiple times (int to string, IP-address to
string, etc.)

Setting up networking is known to be costing a considerable amount of time when
starting containers, and while this may only be a small part of that, it doesn't
hurt to save some resources (and readability of the code isn't significantly
impacted).

For example, benchmarking the `redirector()` code before/after:

    BenchmarkParseOld-4   	  137646	      8398 ns/op	    4192 B/op	      75 allocs/op
    BenchmarkParseNew-4   	  629395	      1762 ns/op	    2362 B/op	      24 allocs/op

Average over 10 runs:

    benchstat old.txt new.txt

    name     old time/op    new time/op    delta
    Parse-4    8.43µs ± 2%    1.79µs ± 3%  -78.76%  (p=0.000 n=9+8)

    name     old alloc/op   new alloc/op   delta
    Parse-4    4.19kB ± 0%    2.36kB ± 0%  -43.65%  (p=0.000 n=10+10)

    name     old allocs/op  new allocs/op  delta
    Parse-4      75.0 ± 0%      24.0 ± 0%  -68.00%  (p=0.000 n=10+10)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-20 14:43:07 +02:00
Sebastiaan van Stijn
b9de761c26
docs: api: document ImageSummary fields (api v1.39-v1.41)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-20 12:10:17 +02:00
Sebastiaan van Stijn
f8a1d6b50a
api: swagger: document ImageSummary fields
Also fixes a typo in ImageInspect :)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-20 12:09:08 +02:00
Samuel Karp
c7395a4f01
Merge pull request #43503 from thaJeztah/optional_kill_signal 2022-04-20 02:35:21 -07:00
Akihiro Suda
db2b0feda2
Merge pull request #43498 from thaJeztah/bump_golang_1.18.1
update golang to 1.18.1
2022-04-20 09:37:46 +09:00
Sebastiaan van Stijn
b365924ec3
client: ContainerKill(): don't send signal query-param if none was set
Just a small clean-up (there's more endpoints to do this for, but
I was working on changes in this area on the CLI when I noticed we
were setting this query-parameter unconditionally.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-19 17:40:33 +02:00
Sebastiaan van Stijn
07dba5d9fe
api: docs: improve documentation of ContainerConfig type (API v1.30-v1.41)
ContainerConfig is used in multiple locations (for example, both for
Image.Config and Image.ContainerConfig). Unfortunately, swagger does
not allow documenting individual uses if a type is used; for this type,
the content is _optional_ when used as Image.ContainerConfig (which is
set by the classic builder, which does a "commit" of a container, but
not used when building an image with BuildKit).

This patch attempts to address this confusion by documenting that
"it may be empty (or fields not propagated) if it's used for the
Image.ContainerConfig field".

Perhaps alternatives are possible (aliasing the type?) but we can
look at those in a follow-up.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-19 16:22:29 +02:00
Trapier Marshall
a46700dbdb Cleanup servicebindings only on Windows
Make the call to cleanupServiceBindings during network deletion
conditional on Windows (where it is required), thereby providing a
performance improvement to network cleanup on Linux.

Signed-off-by: Trapier Marshall <tmarshall@mirantis.com>
2022-04-19 14:22:16 +00:00
Trapier Marshall
6861aade58 Delay network deletion until after lb cleanup
Removal of PolicyLists from Windows VFP must be performed prior to
removing the HNS network. Otherwise PolicyList removal fails with
HNS error "network not found".

Signed-off-by: Trapier Marshall <tmarshall@mirantis.com>
2022-04-19 14:22:08 +00:00
Trapier Marshall
556cb3ae81 Log HNS policylist removal failures
Signed-off-by: Trapier Marshall <tmarshall@mirantis.com>
2022-04-19 14:21:29 +00:00
Sebastiaan van Stijn
cb62919d28
api: improve documentation of ContainerConfig type
ContainerConfig is used in multiple locations (for example, both for
Image.Config and Image.ContainerConfig). Unfortunately, swagger does
not allow documenting individual uses if a type is used; for this type,
the content is _optional_ when used as Image.ContainerConfig (which is
set by the classic builder, which does a "commit" of a container, but
not used when building an image with BuildKit).

This patch attempts to address this confusion by documenting that
"it may be empty (or fields not propagated) if it's used for the
Image.ContainerConfig field".

Perhaps alternatives are possible (aliasing the type?) but we can
look at those in a follow-up.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-19 16:19:57 +02:00
Sebastiaan van Stijn
ef64db5021
update golang to 1.18.1
go1.18.1 (released 2022-04-12) includes security fixes to the crypto/elliptic,
crypto/x509, and encoding/pem packages, as well as bug fixes to the compiler,
linker, runtime, the go command, vet, and the bytes, crypto/x509, and go/types
packages. See the Go 1.18.1 milestone on the issue tracker for details:

https://github.com/golang/go/issues?q=milestone%3AGo1.18.1+label%3ACherryPickApproved

Includes fixes for:

- CVE-2022-24675 (https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-24675)
- CVE-2022-27536 (https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-27536)
- CVE-2022-28327 (https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-28327)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-19 09:14:18 +02:00
Sebastiaan van Stijn
013d648888
client: CopyToContainer(), CopyFromContainer(): remove status-code handling
This was added in 93c3e6c91e, at which time only
some basic handling of non-succesful status codes was present;
93c3e6c91e/api/client/utils.go (L112-L121)

Given that since 38e6d474af non-successful status-
codes are already handled, and a 204 ("no content") status should not be an error,
this special case should no longer be needed.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-19 08:28:20 +02:00
Sebastiaan van Stijn
83969fa3dd
daemon: move DefaultShutdownTimeout to daemon/config
Unifying defaults to the daemon/config package

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-17 13:11:03 +02:00
Sebastiaan van Stijn
690a6fddf9
daemon: move default namespaces to daemon/config
Keeping the defaults in a single location, which also reduces
the list of imports needed.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-17 13:10:57 +02:00
Sebastiaan van Stijn
881e326f7a
daemon/config: remove unneeded alias
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-17 13:08:34 +02:00
CrazyMax
4039850bef
add instructions to generate events message table bin
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-04-14 19:52:36 +02:00
CrazyMax
1efda78f2b
use go-winres for windows build and cleanup autogen and winresources
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-04-14 19:52:36 +02:00
CrazyMax
fd2143e0b0
use go-winres for cross to create Windows resources
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-04-14 19:52:35 +02:00
Sebastiaan van Stijn
61404de7df
Merge pull request #43474 from crazy-max/fix-cross
fix cross compilation for arm platforms
2022-04-14 18:40:44 +02:00
Cory Snider
1d6e0fb103 metrics: DRY metric definitions
Having to declare a package-scope variable and separately initialize it
is repetitive and error-prone. Refactor so that each metric is defined
and initialized in the same statement.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-04-14 11:21:56 -04:00
Tianon Gravi
c110770e08
Merge pull request #43485 from thaJeztah/fix_dco_check
Jenkinsfile: add workaround for CVE-2022-24765
2022-04-13 11:38:31 -07:00
Sebastiaan van Stijn
538f5338e0
Merge pull request #43477 from thaJeztah/deprecate_urlutil
pkg/urlutil: deprecate, and move to builder/remotecontext/urlutil
2022-04-13 17:16:15 +02:00
Sebastiaan van Stijn
efe03aa2d8
Jenkinsfile: add workaround for CVE-2022-24765
git published an advisory Yesterday, which (as a counter-measure)
requires the git repository's directory to be owned by the current
user, and otherwise produce an error:

    fatal: unsafe repository ('/workspace' is owned by someone else)
    To add an exception for this directory, call:

        git config --global --add safe.directory /workspace

The DCO check is run within a container, which is running as `root`
(to allow packages to be installed), but because of this, the user
does not match the files that are bind-mounted from the host (as they
are checked out by Jenkins, using a different user).

To work around this issue, this patch configures git to consider the
`/workspace` directory as "safe". We configure it in the `--system`
configuration so that it takes effect for "all users" inside the
container.

More details on the advisory can be found on GitHub's blog:
https://github.blog/2022-04-12-git-security-vulnerability-announced/

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-13 15:58:55 +02:00
Sebastiaan van Stijn
449250994f
builder/remotecontext/urlutil: simplify and improve documentation
Simplify some of the logic, and add documentation about the package,
as well as warnings that this package should not be used as a general-
purpose utility.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-12 19:58:09 +02:00
Sebastiaan van Stijn
5f89a6a78e
pkg/urlutil: deprecate, and move to builder/remotecontext/urlutil
pkg/urlutil (despite its poorly chosen name) is not really intended as a generic
utility to handle URLs, and should only be used by the builder to handle (remote)
build contexts.

- IsURL() only does a very rudimentary check for http(s):// prefixes, without any
  other validation, but due to its name may give incorrect expectations.
- IsGitURL() is written specifically with docker build remote git contexts in
  mind, and has handling for backward-compatibility, where strings that are
  not URLs, but start with "github.com/" are accepted.

Because of the above, this patch:

- moves the package inside builder/remotecontext, close to where it's intended
  to be used (ideally this would be part of build/remotecontext itself, but this
  package imports many other dependencies, which would introduce those as extra
  dependencies in the CLI).
- deprecates pkg/urlutil, but adds aliases as there are some external consumers.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-12 19:58:05 +02:00
Sebastiaan van Stijn
074bc1c3ab
pkg/urlutil: remove unused IsTransportURL()
This function is no longer used (either internally, or externally), so
can be removed.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-12 19:57:32 +02:00
Sebastiaan van Stijn
bca8d9f2ce
Merge pull request #43463 from thaJeztah/httputils_readjson
api/server/httputils: add ReadJSON() utility and fix handling of invalid JSON
2022-04-11 23:23:15 +02:00
Sebastiaan van Stijn
6f8bc7e553
Merge pull request #43476 from thaJeztah/split_urlutil
daemon/logger: remove uses of pkg/urlutil, fix fluentd validation and parsing
2022-04-11 21:51:25 +02:00
Sebastiaan van Stijn
b6d58d749c
runconfig: ContainerDecoder(): fix handling of invalid JSON
Implement similar logic as is used in httputils.ReadJSON(). Before
this patch, endpoints using the ContainerDecoder would incorrectly
return a 500 (internal server error) status.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-11 21:44:45 +02:00
Sebastiaan van Stijn
ff5f70e55f
api/server/httputils: move WriteJSON() together with ReadJSON()
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-11 21:37:57 +02:00
Sebastiaan van Stijn
0c9ff0b45a
api/server/httputils: add ReadJSON() utility
Implement a ReadJSON() utility to help reduce some code-duplication,
and to make sure we handle JSON requests consistently (e.g. always
check for the content-type).

Differences compared to current handling:

- prevent possible panic if request.Body is nil ("should never happen")
- always require Content-Type to be "application/json"
- be stricter about additional content after JSON (previously ignored)
- but, allow the body to be empty (an empty body is not invalid);
  update TestContainerInvalidJSON accordingly, which was testing the
  wrong expectation.
- close body after reading (some code did this)

We should consider to add a "max body size" on this function, similar to
7b9275c0da/api/server/middleware/debug.go (L27-L40)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-11 21:37:51 +02:00
Sebastiaan van Stijn
ef490cae45
api/server/httputils: matchesContentType(): return error instead of logging
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-11 21:37:00 +02:00
Sebastiaan van Stijn
3e47a7505e
daemon/logger/fluentd: remove udp, tcp+tls, unixgram, add tls scheme
unix and unixgram were added in cb176c848e, but at
the time, the driver only supported "tcp" and "unix":
cb176c848e/vendor/src/github.com/fluent/fluent-logger-golang/fluent/fluent.go (L243-L261)

support for tls was added in github.com/fluent/fluent-logger-golang v1.8.0, which
was vendored in e24d61b7ef.

the list of currently supported schemes by the driver is: tcp, tls and unix:
5179299b98/vendor/github.com/fluent/fluent-logger-golang/fluent/fluent.go (L435-L463)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-11 18:02:13 +02:00
Sebastiaan van Stijn
12424cfa6f
daemon/logger/fluentd: fix missing host, remove urlutil.IsTransportURL()
pkg/urlutil (despite its poorly chosen name) is not really intended as a generic
utility to handle URLs, and should only be used by the builder to handle (remote)
build contexts.

This patch:

- fix some cases where the host was ignored for valid addresses.
- removes a redundant use of urlutil.IsTransportURL(); instead adding code to
  check if the given scheme (protocol) is supported.
- improve port validation for out of range ports.
- fix some missing validation: the driver was silently ignoring path elements,
  but expected a host (not an URL)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-11 18:02:05 +02:00
Sebastiaan van Stijn
0f40aefccd
daemon/logger/fluentd: validate path element
fix some missing validation: the driver was silently ignoring path elements
in some cases, and expecting a host (not an URL), and for unix sockets did
not validate if a path was specified.

For the latter case, we should have a fix in the upstream driver, as it
uses an empty path as default path for the socket (`defaultSocketPath`),
and performs no validation.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-11 17:58:51 +02:00
Sebastiaan van Stijn
b161616202
daemon/logger/fluentd: make error-handling less DRY
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-11 17:55:48 +02:00
Sebastiaan van Stijn
0dd2b4d577
daemon/logger/fluentd: rename var that collided with import
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-11 17:55:46 +02:00
Sebastiaan van Stijn
40182954fa
daemon/logger/fluentd: add coverage for ValidateLogOpt(), parseAddress()
This exposed a bug where host is ignored on some valid cases (to be fixed).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-11 17:55:31 +02:00
Sebastiaan van Stijn
c2ca3e1118
daemon/logger/syslog: remove uses of pkg/urlutil.IsTransportURL()
pkg/urlutil (despite its poorly chosen name) is not really intended as a generic
utility to handle URLs, and should only be used by the builder to handle (remote)
build contexts.

This patch:

- removes a redundant use of urlutil.IsTransportURL(); instead adding some code
  to check if the given scheme (protocol) is supported.
- define a `defaultPort` const for the default port.
- use `net.JoinHostPort()` instead of string concatenating, to account for possible
  issues with IPv6 addresses.
- renames a variable that collided with an imported package.
- improves test coverage, and moves an integration test.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-11 17:48:40 +02:00