Commit graph

7184 commits

Author SHA1 Message Date
Roman Volosatovs
b308097ec3
daemon/images: refactor image listing
- Rename image summary constructor
    - Rename `newImage` into `newImageSummary`, since the returned type is
      `*types.ImageSummary`
- Rename variables for clarity
    - Rename `newImage` into `summary`, since the variable type is
      `*types.ImageSummary`
    - Rename `imagesMap` into `summaryMap`, since the value type
      contained is `*types.ImageSummary`
- Only compute `DiffSize` when more than 1 reference to the layer
  exists, since it is not used otherwise
- Move variable declarations closer to where they are used

Signed-off-by: Roman Volosatovs <roman.volosatovs@docker.com>
Co-authored-by: Sebastiaan van Stijn <github@gone.nl>
2021-06-30 11:32:32 +02:00
Sebastiaan van Stijn
314759dc2f
Merge pull request #42393 from aiordache/daemon_config
Daemon config validation
2021-06-23 19:32:07 +02:00
Rich Horwood
8f80e55111 Add configuration validation option and tests.
Fixes #36911

If config file is invalid we'll exit anyhow, so this just prevents
the daemon from starting if the configuration is fine.

Mainly useful for making config changes and restarting the daemon
iff the config is valid.

Signed-off-by: Rich Horwood <rjhorwood@apple.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Anca Iordache <anca.iordache@docker.com>
2021-06-23 09:54:55 +00:00
Brian Goff
bf11970fd5
Merge pull request #42536 from thaJeztah/replace_deprecated_userns
replace uses of deprecated containerd/sys.RunningInUserNS()
2021-06-18 13:20:19 -07:00
Akihiro Suda
7729ebfa1b
Merge pull request #42432 from dperny/fix-ip-overlap 2021-06-19 01:07:27 +09:00
Sebastiaan van Stijn
472f21b923
replace uses of deprecated containerd/sys.RunningInUserNS()
This utility was moved to a separate package, which has no
dependencies.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-06-18 11:01:24 +02:00
Akihiro Suda
9e8cf1016e
Merge pull request #42473 from thaJeztah/unfork_buildkit
revendor BuildKit (master branch)
2021-06-17 10:56:25 +09:00
Sebastiaan van Stijn
2773f81aa5
Merge pull request #42445 from thaJeztah/bump_golang_ci
[testing] ~update~ fix linting issues found by golangci-lint v1.40.1
2021-06-16 22:15:01 +02:00
Sebastiaan van Stijn
3eb1257698
revendor BuildKit (master branch)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-06-16 01:17:48 +02:00
Tianon Gravi
a060328874
Merge pull request #42472 from thaJeztah/improve_rootless_option
daemon: improve handling of ROOTLESSKIT_PARENT_EUID
2021-06-11 13:03:31 -07:00
Justin Cormack
1ba54a5fd0
Merge pull request #42511 from thaJeztah/remove_lcow_step4
Remove LCOW (step 4): remove layerstore indexing by OS (used for LCOW)
2021-06-11 18:31:59 +01:00
Samuel Karp
17bf6211af
Merge pull request #42325 from thaJeztah/warn_on_non_matching_platform
docker pull: warn when pulled single-arch image does not match --platform
2021-06-10 16:53:50 -07:00
Sebastiaan van Stijn
dc7cbb9b33
remove layerstore indexing by OS (used for LCOW)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-06-10 17:49:11 +02:00
Sebastiaan van Stijn
e6dabfa977
graphdriver: temporarily ignore unsafeptr: possible misuse of reflect.SliceHeader
Probably needs a similar change as c208f03fbd,
but this code makes my head spin, so for now suppressing, and created a
tracking issue:

    daemon/graphdriver/graphtest/graphtest_unix.go:305:12: unsafeptr: possible misuse of reflect.SliceHeader (govet)
        header := *(*reflect.SliceHeader)(unsafe.Pointer(&buf))
                  ^
    daemon/graphdriver/graphtest/graphtest_unix.go:308:36: unsafeptr: possible misuse of reflect.SliceHeader (govet)
        data := *(*[]byte)(unsafe.Pointer(&header))
                                          ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-06-10 13:03:47 +02:00
Sebastiaan van Stijn
d61b7c1211
daemon: var-declaration: should omit type bool (revive)
daemon/list.go:556:18: var-declaration: should omit type bool from declaration of var shouldSkip; it will be inferred from the right-hand side (revive)
                shouldSkip    bool = true
                              ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-06-10 13:03:45 +02:00
Sebastiaan van Stijn
16ced7622b
daemon/config: error strings should not be capitalized
daemon/config/config_unix.go:92:21: error-strings: error strings should not be capitalized or end with punctuation or a newline (revive)
            return fmt.Errorf("Default cgroup namespace mode (%v) is invalid. Use \"host\" or \"private\".", cm) // nolint: golint
                              ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-06-10 13:03:43 +02:00
Sebastiaan van Stijn
bb17074119
reformat "nolint" comments
Unlike regular comments, nolint comments should not have a leading space.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-06-10 13:03:42 +02:00
Sebastiaan van Stijn
4004a39d53
daemon/splunk: ignore G402: TLS MinVersion too low for now
daemon/logger/splunk/splunk.go:173:16: G402: TLS MinVersion too low. (gosec)
    	tlsConfig := &tls.Config{}
    	              ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-06-10 13:03:38 +02:00
Sebastiaan van Stijn
09191c0936
daemon/stats: fix notRunningErr / notFoundErr detected as unused (false positive)
Also looks like a false positive, but given that these were basically
testing for the `errdefs.Conflict` and `errdefs.NotFound` interfaces, I
replaced these with those;

    daemon/stats/collector.go:154:6: type `notRunningErr` is unused (unused)
    type notRunningErr interface {
         ^
    daemon/stats/collector.go:159:6: type `notFoundErr` is unused (unused)
    type notFoundErr interface {
         ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-06-10 13:03:34 +02:00
Sebastiaan van Stijn
b4c0c7c076
G601: Implicit memory aliasing in for loop
daemon/cluster/executor/container/adapter.go:446:42: G601: Implicit memory aliasing in for loop. (gosec)
            req := c.container.volumeCreateRequest(&mount)
                                                   ^
    daemon/network.go:577:10: G601: Implicit memory aliasing in for loop. (gosec)
                np := &n
                      ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-06-10 13:03:31 +02:00
Sebastiaan van Stijn
d13997b4ba
gosec: G601: Implicit memory aliasing in for loop
plugin/v2/plugin.go:141:50: G601: Implicit memory aliasing in for loop. (gosec)
                    updateSettingsEnv(&p.PluginObj.Settings.Env, &s)
                                                                 ^
    libcontainerd/remote/client.go:572:13: G601: Implicit memory aliasing in for loop. (gosec)
                cpDesc = &m
                         ^
    distribution/push_v2.go:400:34: G601: Implicit memory aliasing in for loop. (gosec)
                (metadata.CheckV2MetadataHMAC(&mountCandidate, pd.hmacKey) ||
                                              ^
    builder/dockerfile/builder.go:261:84: G601: Implicit memory aliasing in for loop. (gosec)
            currentCommandIndex = printCommand(b.Stdout, currentCommandIndex, totalCommands, &meta)
                                                                                             ^
    builder/dockerfile/builder.go:278:46: G601: Implicit memory aliasing in for loop. (gosec)
            if err := initializeStage(dispatchRequest, &stage); err != nil {
                                                       ^
    daemon/container.go:283:40: G601: Implicit memory aliasing in for loop. (gosec)
            if err := parser.ValidateMountConfig(&cfg); err != nil {
                                                 ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-06-10 13:03:29 +02:00
Sebastiaan van Stijn
f7433d6190
staticcheck: SA4001: &*x will be simplified to x. It will not copy x
daemon/volumes_unix_test.go:228:13: SA4001: &*x will be simplified to x. It will not copy x. (staticcheck)
                mp:      &(*c.MountPoints["/jambolan"]), // copy the mountpoint, expect no changes
                         ^
    daemon/logger/local/local_test.go:214:22: SA4001: &*x will be simplified to x. It will not copy x. (staticcheck)
            dst.PLogMetaData = &(*src.PLogMetaData)
                               ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-06-10 13:03:25 +02:00
Sebastiaan van Stijn
d43bcc8974
daemon/logger/journald: fix linting errors
daemon/logger/journald/read.go:128:3 comment on exported function `CErr` should be of the form `CErr ...`

    daemon/logger/journald/read.go:131:36: unnecessary conversion (unconvert)
            return C.GoString(C.strerror(C.int(-ret)))
	                                  ^
    daemon/logger/journald/read.go:380:2: S1023: redundant `return` statement (gosimple)
        return
        ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-06-10 13:03:21 +02:00
Sebastiaan van Stijn
076d9c6037
daemon: remove graphdriver indexing by OS (used for LCOW)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-06-10 10:17:26 +02:00
Sebastiaan van Stijn
08ddbfbdac
libcontainerd: remove LCOW bits
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-06-09 22:05:10 +02:00
Drew Erny
0d9b0ed678 Fix possible overlapping IPs
A node is no longer using its load balancer IP address when it no longer
has tasks that use the network that requires that load balancer. When
this occurs, the swarmkit manager will free that IP in IPAM, and may
reaassign it.

When a task shuts down cleanly, it attempts removal of the networks it
uses, and if it is the last task using those networks, this removal
succeeds, and the load balancer IP is freed.

However, this behavior is absent if the container fails. Removal of the
networks is never attempted.

To address this issue, I amend the executor. Whenever a node load
balancer IP is removed or changed, that information is passedd to the
executor by way of the Configure method. By keeping track of the set of
node NetworkAttachments from the previous call to Configure, we can
determine which, if any, have been removed or changed.

At first, this seems to create a race, by which a task can be attempting
to start and the network is removed right out from under it. However,
this is already addressed in the controller. The controller will attempt
to recreate missing networks before starting a task.

Signed-off-by: Drew Erny <derny@mirantis.com>
2021-06-09 11:21:41 -06:00
Tianon Gravi
adb26d3fbe
Merge pull request #42488 from sparrc/log-fix-wait-timeout
Fix log statement 'failed to exit' timeout accuracy
2021-06-08 15:14:07 -07:00
Brian Goff
a7ea29a5a6
Merge pull request #42451 from thaJeztah/remove_lcow_step1
Remove LCOW code (step 1)
2021-06-08 13:41:45 -07:00
Cam
d15ce134ef
Fix log statement 'failed to exit' timeout accuracy
log statement should reflect how long it actually waited, not how long
it theoretically could wait based on the 'seconds' integer passed in.

Signed-off-by: Cam <gh@sparr.email>
2021-06-08 13:37:58 -07:00
Akihiro Suda
0ad2293d0e
Merge pull request #41656 from thaJeztah/unexport_things 2021-06-08 12:07:40 +09:00
Sebastiaan van Stijn
424c0eb3c0
docker pull: warn when pulled single-arch image does not match --platform
This takes the same approach as was implemented on `docker build`, where a warning
is printed if `FROM --platform=...` is used (added in 399695305c)

Before:

    docker rmi armhf/busybox
    docker pull --platform=linux/s390x armhf/busybox

    Using default tag: latest
    latest: Pulling from armhf/busybox
    d34a655120f5: Pull complete
    Digest: sha256:8e51389cdda2158935f2b231cd158790c33ae13288c3106909324b061d24d6d1
    Status: Downloaded newer image for armhf/busybox:latest
    docker.io/armhf/busybox:latest

With this change:

    docker rmi armhf/busybox
    docker pull --platform=linux/s390x armhf/busybox

    Using default tag: latest
    latest: Pulling from armhf/busybox
    d34a655120f5: Pull complete
    Digest: sha256:8e51389cdda2158935f2b231cd158790c33ae13288c3106909324b061d24d6d1
    Status: Downloaded newer image for armhf/busybox:latest
    WARNING: image with reference armhf/busybox was found but does not match the specified platform: wanted linux/s390x, actual: linux/arm64
    docker.io/armhf/busybox:latest

And daemon logs print:

   WARN[2021-04-26T11:19:37.153572667Z] ignoring platform mismatch on single-arch image  error="image with reference armhf/busybox was found but does not match the specified platform: wanted linux/s390x, actual: linux/arm64" image=armhf/busybox

When pulling without specifying `--platform, no warning is currently printed (but we can add a warning in future);

    docker rmi armhf/busybox
    docker pull armhf/busybox

    Using default tag: latest
    latest: Pulling from armhf/busybox
    d34a655120f5: Pull complete
    Digest: sha256:8e51389cdda2158935f2b231cd158790c33ae13288c3106909324b061d24d6d1
    Status: Downloaded newer image for armhf/busybox:latest
    docker.io/armhf/busybox:latest

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-06-07 23:08:45 +02:00
Sebastiaan van Stijn
aa4dce742f
daemon: improve handling of ROOTLESSKIT_PARENT_EUID
- daemon.WithRootless():  make sure ROOTLESSKIT_PARENT_EUID is valid int
- daemon.RawSysInfo(): minor simplification, and rename variable that
  clashed with imported package.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-06-05 21:12:32 +02:00
Sebastiaan van Stijn
e047d984dc
Remove LCOW code (step 1)
The LCOW implementation in dockerd has been deprecated in favor of re-implementation
in containerd (in progress). Microsoft started removing the LCOW V1 code from the
build dependencies we use in Microsoft/opengcs (soon to be part of Microsoft/hcshhim),
which means that we need to start removing this code.

This first step removes the lcow graphdriver, the LCOW initialization code, and
some LCOW-related utilities.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-06-03 21:16:21 +02:00
Brian Goff
a77317882d
Merge pull request #42262 from cpuguy83/move_libnetwork
Move libnetwork
2021-06-03 12:06:31 -07:00
Brian Goff
20eb137e0a
Merge pull request #42334 from AkihiroSuda/rootless-overlay2-k511-selinux
rootless: disable overlay2 if running with SELinux
2021-06-03 10:33:27 -07:00
Brian Goff
4b981436fe Fixup libnetwork lint errors
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2021-06-01 23:48:32 +00:00
Brian Goff
a0a473125b Fix libnetwork imports
After moving libnetwork to this repo, we need to update all the import
paths for libnetwork to point to docker/docker/libnetwork instead of
docker/libnetwork.
This change implements that.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2021-06-01 21:51:23 +00:00
Sebastiaan van Stijn
bf07c06c63
daemon: move DefaultShimBinary, DefaultRuntimeBinary to config package
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-05-31 19:06:16 +02:00
Sebastiaan van Stijn
34b854f965
daemon: un-export ModifyRootKeyLimit()
it's only used internally, so no need to export

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-05-31 19:06:14 +02:00
Sebastiaan van Stijn
95d69658be
daemon: un-export VerifyCgroupDriver()
it's only used internally, so no need to export

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-05-31 19:06:12 +02:00
Sebastiaan van Stijn
a506630e57
daemon: use sync.Once for systemd detection
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-05-31 19:06:10 +02:00
Sebastiaan van Stijn
e7ba5cacc6
daemon: un-export IsRunningSystemd()
This utility was added after 19.03, and is only used in the daemon code
itself, so we can un-export it, until there's an external use for it.

Also updated the description, because the runc code already copied it
from coreos/go-systemd, so better to describe the actual source.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-05-31 19:06:07 +02:00
Brian Goff
a8a769f04f
Merge pull request #42291 from angelcar/awslogs-dont-log-messge-discarded-errors
Limit the rate at which logger errors are logged into daemon logs
2021-05-27 19:33:44 -07:00
Sebastiaan van Stijn
08096e3ee6
Merge pull request #42320 from anujva/fix_moby_ring_logger
#42316 Wait for `run` goroutine to exit before `Close`
2021-05-27 15:00:03 +02:00
Sebastiaan van Stijn
97303df921
Merge pull request #41588 from sparrc/kill-refactor
docker kill: fix bug where failed kills didnt fallback to unix kill
2021-05-25 14:47:58 +02:00
Angel Velazquez
fb5a9ec741
Limit the rate at which logger errors are logged into daemon logs
Logging to daemon logs every time there's an error with a log driver can be
problematic since daemon logs can grow rapidly, potentially exhausting disk
space.

Instead, it's preferable to limit the rate at which log driver errors are allowed
to be written. By default, this limit is 333 entries per second max.

Signed-off-by: Angel Velazquez <angelcar@amazon.com>
2021-05-24 16:41:38 -07:00
Brian Goff
e02bc91dcb
Merge pull request #42339 from awmirantis/allow-vhdx-as-data-root-windows
Add security privilege needed to write layers to VHDX
2021-05-20 14:19:52 -07:00
Muhammad Zohaib Aslam
56c88c94dd Added missing test cleanup for temporary directory
A temporary directory was created but not removed at the end of the test.
The missing remove directory call is added now.

Signed-off-by: Muhammad Zohaib Aslam <zohaibse011@gmail.com>
2021-05-01 15:39:50 +03:00
Anuj Varma
cf259eb8a0 Wait for run goroutine to exit before Close
The underlying Loggers Close() function can be called with the the
run() goroutine still writing to the driver. This is causing the
fluentd-golang-logger to panic cause it doesn't defensively check
for the closing of the channel before writing to it.
It relies on the docker daemon to keep the contract of not calling Log()
if Close() has already been called.

Contributions by: James Johnston <james.johnston@thumbtack.com>
                  Nathan Wong <nathanw@thumbtack.com>

Signed-off-by: Anuj Varma <anujvarma@thumbtack.com>
2021-04-30 17:23:32 -07:00
Sebastiaan van Stijn
dd3275c5f9
Merge pull request #42182 from thaJeztah/fix_exec_start_err_handling
Fix error-handling in `daemon.ContainerExecStart()` and `daemon.getExecConfig()`
2021-04-29 21:33:19 +02:00