Commit graph

43070 commits

Author SHA1 Message Date
Sebastiaan van Stijn
cff0276aaf
volume/mounts: rename windowsSplitRawSpec() to splitRawSpec()
It has the windowsParser/lcowParser as receiver, so no need to repeat
that it's for Windows.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-27 09:57:26 +02:00
Sebastiaan van Stijn
f77eb5349b
volume/mounts: minor formatting and linting fixes
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-27 09:57:24 +02:00
Sebastiaan van Stijn
efb87ad106
volume/mounts: pre-compile regular expressions
Compile the regular expression, instead of 'ad-hoc'. For this to work, I moved
the splitting was moved out of parseMountRaw() into ParseMountRaw(), and the
former was renamed to parseMount(). This function still receives the 'raw' string,
as it's used to include the "raw" spec for inclusion in error messages.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-27 09:57:21 +02:00
Akihiro Suda
3deac5dc85
btrfs: annotate error with human-readable hint string
Add hints for "Failed to destroy btrfs snapshot <DIR> for <ID>: operation not permitted" on rootless

Related to issue 41762

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-07-27 15:45:02 +09:00
Roman Volosatovs
a34d804572
integration: remove deprecated endpoint test
`/containers/<name>/copy` endpoint was deprecated in 1.8 and errors
since 1.12. See https://github.com/moby/moby/pull/22149 for more info.

Signed-off-by: Roman Volosatovs <roman.volosatovs@docker.com>
2021-07-26 22:42:21 +02:00
Aaron Lehmann
90f8d1b675 fileutils: Fix incorrect handling of "**/foo" pattern
(*PatternMatcher).Matches includes a special case for when the pattern
matches a parent dir, even though it doesn't match the current path.
However, it assumes that the parent dir which would match the pattern
must have the same number of separators as the pattern itself. This
doesn't hold true with a patern like "**/foo". A file foo/bar would have
len(parentPathDirs) == 1, which is less than the number of path
len(pattern.dirs) == 2... therefore this check would be skipped.

Given that "**/foo" matches "foo", I think it's a bug that the "parent
subdir matches" check is being skipped in this case.

It seems safer to loop over the parent subdirs and check each against
the pattern. It's possible there is a safe optimization to check only a
certain subset, but the existing logic seems unsafe.

Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
2021-07-26 11:28:10 -07:00
Brian Goff
12f1b3ce43
Merge pull request #42616 from thaJeztah/migrate_pkg_signal
replace pkg/signal with moby/sys/signal v0.5.0
2021-07-26 10:47:28 -07:00
Brian Goff
9674540ccf
Merge pull request #42520 from thaJeztah/remove_lcow_step5_alternative
Remove LCOW (step 5): volumes/mounts: remove LCOW code (alternative)
2021-07-26 10:24:52 -07:00
Samuel Karp
e9b07a730e
Merge pull request #42670 from yufeifly/fix-typo
fix a typo
2021-07-24 13:50:30 -07:00
yufeifly
17f39dcb4d fix a typo
Signed-off-by: yufeifly <yufei.xiong@qq.com>
2021-07-25 00:33:59 +08:00
Sebastiaan van Stijn
6317d7467a
Merge pull request #42661 from thaJeztah/promote_overlay2_btrfs_zfs_optin
storage-driver: promote overlay2, make Btrfs and ZFS opt-in
2021-07-23 11:37:01 +02:00
Sebastiaan van Stijn
28409ca6c7
replace pkg/signal with moby/sys/signal v0.5.0
This code was moved to the moby/sys repository

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-23 09:32:54 +02:00
Roman Volosatovs
dd01abf9bf
integration: copy loop variable into parallel test closures
Discovered a few instances, where loop variable is incorrectly used
within a test closure, which is marked as parallel.
Few of these were actually loops over singleton slices, therefore the issue
might not have surfaced there (yet), but it is good to fix there as
well, as this is an incorrect pattern used across different tests.

Signed-off-by: Roman Volosatovs <roman.volosatovs@docker.com>
2021-07-22 22:46:12 +02:00
Sebastiaan van Stijn
d5dbbb5369
storage-driver: promote overlay2, make Btrfs and ZFS opt-in
The daemon uses a priority list to automatically select the best-matching storage
driver for the backing filesystem that is used.

Historically, overlay2 was not supported on Btrfs and ZFS, and the daemon would
automatically pick the `btrfs` or `zfs` storage driver if that was the Backing
File System.

Commits 649e4c8889 and e226aea280
improved our detection to check if overlay2 was supported on the backing file-
system, allowing overlay2 to be used on top of Btrfs or ZFS,  but did not change
the priority list.

While both Btrfs and ZFS have advantages for certain use-cases, and provide
advanced features that are not available to overlay2, they also are known
to require more "handholding", and are generally considered to be mostly
useful for "advanced" users.

This patch changes the storage-driver priority list, to prefer overlay2 (if
supported by the backing filesystem), and effectively makes btrfs and zfs
opt-in storage drivers.

This change does not affect existing installations; the daemon will detect
the storage driver that was previously in use (based on the presence of
storage directories in `/var/lib/docker`).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-21 14:53:56 +02:00
Akihiro Suda
471fd27709
Merge pull request #42656 from thaJeztah/update_containerd_binary_1.5.4
Update containerd v1.5.4
2021-07-20 20:04:56 +09:00
Sebastiaan van Stijn
4fc2d4df03
vendor: github.com/containerd/containerd v1.5.4
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-20 08:57:20 +02:00
Sebastiaan van Stijn
22316ff1d8
Makefile: update buildx version to v0.6.0
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-20 00:58:13 +02:00
Brian Goff
9a6ff685a8
Merge pull request #42641 from thaJeztah/make_signal_selfcontained 2021-07-19 14:46:15 -07:00
Sebastiaan van Stijn
cf1328cd46
update containerd binary v1.4.8
Update to containerd 1.4.8 to address [CVE-2021-32760][1].

[1]: https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-32760

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-19 21:22:25 +02:00
Sebastiaan van Stijn
627bbd3fa4
Merge pull request #42132 from xia-wu/add-create-log-stream
Add an option to skip create log stream for awslogs driver
2021-07-19 16:42:36 +02:00
Justin Cormack
b337c70bdc
Merge pull request #42639 from thaJeztah/system_info_clean
pkg/sysinfo: assorted cleanup/refactoring for handling warnings and logging
2021-07-19 15:17:07 +01:00
Justin Cormack
b05d0604ea
Merge pull request #42648 from thaJeztah/seccomp_closer_to_oci
seccomp.Seccomp: embed oci-spec LinuxSeccomp, add support for seccomp flags
2021-07-19 15:15:43 +01:00
Justin Cormack
ab974f6b57
Merge pull request #42620 from thaJeztah/daemon_stats_literal
daemon: use object literal for stats
2021-07-19 15:14:41 +01:00
Justin Cormack
bde67dfc38
Merge pull request #42654 from AkihiroSuda/runc-v1.0.1
update runc binary and libcontainer to v1.0.1
2021-07-19 15:13:37 +01:00
Justin Cormack
fb21a1e474
Merge pull request #42580 from thaJeztah/reduce_TestClientWithRequestTimeout_flakiness
Reduce TestClientWithRequestTimeout flakiness
2021-07-19 15:11:43 +01:00
Justin Cormack
c485e901d9
Merge pull request #42531 from rvolosatovs/image_shared_size
Add support for `shared-size` parameter for images queries
2021-07-19 14:39:11 +01:00
Justin Cormack
3aa7d80e04
Merge pull request #42625 from rvolosatovs/fix_network_db_islands
Fix flaky libnetwork/networkdb tests
2021-07-19 14:35:07 +01:00
Justin Cormack
34058bc1d2
Merge pull request #42652 from thaJeztah/single_dev_image
Makefile: do not tag docker-dev image with GIT_BRANCH
2021-07-19 14:29:36 +01:00
Akihiro Suda
549060a1d3
vendor: github.com/sirupsen/logrus v1.8.1
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-07-19 15:59:45 +09:00
Akihiro Suda
82b264bd2d
vendor: github.com/coreos/go-systemd/v22 v22.3.2
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-07-19 15:58:54 +09:00
Akihiro Suda
9f9a0b872c
vendor: github.com/cilium/ebpf v0.6.2
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-07-19 15:57:52 +09:00
Akihiro Suda
1256aa0241
vendor: github.com/opencontainers/runc v1.0.1
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-07-19 15:55:28 +09:00
Akihiro Suda
f50c7644cf
update runc binary to v1.0.1
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-07-18 20:19:33 +09:00
Sebastiaan van Stijn
8de724390e
Makefile: do not tag docker-dev image with GIT_BRANCH
When building the dev image, the Makefile generates a tag-name for the image,
based on the current git branch. As a result of this naming, old images will
collect on a developer's machine (especially when building from different
branches, for example when reviewing pull requests):

    REPOSITORY   TAG                                                 IMAGE ID       CREATED        SIZE
    docker-dev   HEAD                                                9785a8fb82f5   30 hours ago   2.13GB
    docker-dev   master                                              9785a8fb82f5   30 hours ago   2.13GB
    docker-dev   seccomp-closer-to-oci                               9785a8fb82f5   30 hours ago   2.13GB
    docker-dev   move-stackdump                                      06882c142bfd   2 days ago     2.13GB
    docker-dev   add-dns-to-docker-info                              2961ed1b99bd   10 days ago    2.13GB
    docker-dev   add-platform-info                                   2961ed1b99bd   10 days ago    2.13GB
    docker-dev   rata-seccomp-new-fields                             2961ed1b99bd   10 days ago    2.13GB
    docker-dev   swagger-wip                                         2961ed1b99bd   10 days ago    2.13GB
    docker-dev   system-df-types                                     2961ed1b99bd   10 days ago    2.13GB
    docker-dev   use-oci-platform                                    2961ed1b99bd   10 days ago    2.13GB
    docker-dev   update-swagger-fork                                 3eeedecca85a   2 weeks ago    2.13GB
    docker-dev   remove-lcow-step5-alternative                       51f9720bbc19   2 weeks ago    2.13GB
    docker-dev   update-s390x-ubuntu-2004                            51f9720bbc19   2 weeks ago    2.13GB
    docker-dev   fix-image-shared-size                               09e9aa46694a   2 weeks ago    2.13GB
    docker-dev   remove-discovery                                    11823223ae83   3 weeks ago    2.13GB
    docker-dev   daemon-config                                       355643e371b0   4 weeks ago    2.12GB
    docker-dev   jenkins-windows-containerd                          68199214b860   4 weeks ago    2.11GB
    docker-dev   unfork-buildkit                                     68199214b860   4 weeks ago    2.11GB
    docker-dev   warn-on-non-matching-platform                       bc014b94017f   5 weeks ago    2.11GB
    docker-dev   remove-lcow                                         3a43c0900282   6 weeks ago    2.11GB
    docker-dev   remove-lcow-part5                                   3a43c0900282   6 weeks ago    2.11GB
    docker-dev   remove-lcow-step3                                   3a43c0900282   6 weeks ago    2.11GB
    docker-dev   remove-lcow-step4                                   3a43c0900282   6 weeks ago    2.11GB
    docker-dev   seccomp-unconfined-daemon                           3a43c0900282   6 weeks ago    2.11GB
    docker-dev   update-authors                                      3a43c0900282   6 weeks ago    2.11GB
    docker-dev   payall4u-fix-creating-sandbox-when-disable-bridge   114c0f2ceb17   6 weeks ago    2.12GB
    docker-dev   catch-almost-all                                    f437d2bc512b   8 weeks ago    2.12GB
    docker-dev   bin-criu                                            c72894ae66f3   2 months ago   2.12GB
    docker-dev   bump-golang-1-14                                    395932141809   2 months ago   2.14GB
    docker-dev   upstream-systemd-units                              d0cb07f9473c   2 months ago   2.12GB
    docker-dev   bump-criu                                           6ed9e8fcf59f   2 months ago   2.12GB

This images are a bit of a pain to clean up, and because they are tagged,
`docker image prune` or `docker system prune` doesn't help (unless `--all` is
used).

Looking at the background of this naming, a found that it was originally added
in a95712899e, after a discussion on PR 3471.
At the time, the image name was used to check if the image needed building, and
otherwise building was skipped in the makefile.

This is no longer the case; the image is built unconditionally, and the build-
cache helps (where possible) speed up rebuilding the image.

In _theory_ having unique names would allow for multiple dev containers (from
different branches) to be started in parallel, but in most situations, the
source-code will be mounted (`BIND_MOUNT=.`), so I'm not sure if that should
be a compelling reason to keep the current naming.

This patch removes the unique tag, and will always tag the image locally as
`docker-dev:latest`.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-17 17:52:45 +02:00
Sebastiaan van Stijn
0ef7e727d2
seccomp: Seccomp: embed oci-spec LinuxSeccomp, add support for seccomp flags
This patch, similar to d92739713c, embeds the
`LinuxSeccomp` type of the runtime-spec, so that we can support all options
provided by the spec, and decorates it with our own fields.

With this, profiles can make use of the recently added "Flags" field, to
specify flags that must be passed to seccomp(2) when installing the filter.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-17 15:57:54 +02:00
Sebastiaan van Stijn
bfd4b64600
seccomp: setupSeccomp(): update errors and remove redundant check
Make the error message slightly more informative, and remove the redundant
`len(config.ArchMap) != 0` check, as iterating over an empty, or 'nil' slice
is a no-op already. This allows to use a slightly more idiomatic "if ok := xx; ok"
condition.

Also move validation to the start of the loop (early return), and explicitly create
a new slice for "names" if the legacy "Name" field is used.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-17 15:57:41 +02:00
Sebastiaan van Stijn
c815b86f40
seccomp: add additional unit-tests
Add test to verify profile validation, and to verify that the legacy
format actually loads the profile as expected (instead of only verifying
it doesn't produce an error).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-16 18:01:25 +02:00
Sebastiaan van Stijn
c1ced23544
seccomp: use oci-spec consts in tests
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-16 18:01:23 +02:00
Sebastiaan van Stijn
b309e96b11
seccomp: improve GoDoc for Seccomp fields
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-16 18:01:12 +02:00
Brian Goff
b316cc059a
Merge pull request #42636 from thaJeztah/update_containerd
Update containerd binary and vendor to v1.5.3
2021-07-15 14:10:26 -07:00
Sebastiaan van Stijn
a2da507857
Merge pull request #42604 from kinvolk/rata/seccomp-new-fields
seccomp: Sync fields with runtime-spec fields
2021-07-15 23:02:45 +02:00
Sebastiaan van Stijn
6ff6913ac4
pkg/signal: remove gotest.tools dependency
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-15 18:24:23 +02:00
Sebastiaan van Stijn
0880df4644
pkg/signal: move Trap() to cmd/dockerd
It's the only location where this is used, and it's quite specific
to dockerd (not really a reusable function for external use), so
moving it into that package.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-15 18:11:00 +02:00
Sebastiaan van Stijn
ea5c94cdb9
pkg/signal: move signal.DumpStacks() to a separate package
It is not directly related to signal-handling, so can well live
in its own package.

Also added a variant that doesn't take a directory to write files
to, for easier consumption / better match to how it's used.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-15 18:09:43 +02:00
Tianon Gravi
b2e31eb416
Merge pull request #42614 from jk-vb/patch-1
updated names-generator.go for alphabetization
2021-07-14 22:50:48 +00:00
Tianon Gravi
40502f49f6
Merge pull request #42634 from kevpar/fix-vndr-tooling
Fix up vndr tooling
2021-07-14 22:43:21 +00:00
Sebastiaan van Stijn
7d63cbfd38
api/types: add GoDoc to Info.Warnings field
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-14 23:10:16 +02:00
Sebastiaan van Stijn
4ace1998e5
pkg/sysinfo: use correct name for AppArmor in t.Skip()
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-14 23:10:13 +02:00
Sebastiaan van Stijn
9b795c3e50
pkg/sysinfo.New(), daemon.RawSysInfo(): remove "quiet" argument
The "quiet" argument was only used in a single place (at daemon startup), and
every other use had to pass "false" to prevent this function from logging
warnings.

Now that SysInfo contains the warnings that occurred when collecting the
system information, we can make leave it up to the caller to use those
warnings (and log them if wanted).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-14 23:10:07 +02:00
Sebastiaan van Stijn
1fb62f455c
pkg/sysinfo: collect warnings in SysInfo struct
This allows the warnings to be consumed in other locations.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-14 17:28:25 +02:00