Commit graph

43151 commits

Author SHA1 Message Date
Sebastiaan van Stijn
627bbd3fa4
Merge pull request #42132 from xia-wu/add-create-log-stream
Add an option to skip create log stream for awslogs driver
2021-07-19 16:42:36 +02:00
Justin Cormack
b337c70bdc
Merge pull request #42639 from thaJeztah/system_info_clean
pkg/sysinfo: assorted cleanup/refactoring for handling warnings and logging
2021-07-19 15:17:07 +01:00
Justin Cormack
b05d0604ea
Merge pull request #42648 from thaJeztah/seccomp_closer_to_oci
seccomp.Seccomp: embed oci-spec LinuxSeccomp, add support for seccomp flags
2021-07-19 15:15:43 +01:00
Justin Cormack
ab974f6b57
Merge pull request #42620 from thaJeztah/daemon_stats_literal
daemon: use object literal for stats
2021-07-19 15:14:41 +01:00
Justin Cormack
bde67dfc38
Merge pull request #42654 from AkihiroSuda/runc-v1.0.1
update runc binary and libcontainer to v1.0.1
2021-07-19 15:13:37 +01:00
Justin Cormack
fb21a1e474
Merge pull request #42580 from thaJeztah/reduce_TestClientWithRequestTimeout_flakiness
Reduce TestClientWithRequestTimeout flakiness
2021-07-19 15:11:43 +01:00
Justin Cormack
c485e901d9
Merge pull request #42531 from rvolosatovs/image_shared_size
Add support for `shared-size` parameter for images queries
2021-07-19 14:39:11 +01:00
Justin Cormack
3aa7d80e04
Merge pull request #42625 from rvolosatovs/fix_network_db_islands
Fix flaky libnetwork/networkdb tests
2021-07-19 14:35:07 +01:00
Justin Cormack
34058bc1d2
Merge pull request #42652 from thaJeztah/single_dev_image
Makefile: do not tag docker-dev image with GIT_BRANCH
2021-07-19 14:29:36 +01:00
Akihiro Suda
549060a1d3
vendor: github.com/sirupsen/logrus v1.8.1
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-07-19 15:59:45 +09:00
Akihiro Suda
82b264bd2d
vendor: github.com/coreos/go-systemd/v22 v22.3.2
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-07-19 15:58:54 +09:00
Akihiro Suda
9f9a0b872c
vendor: github.com/cilium/ebpf v0.6.2
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-07-19 15:57:52 +09:00
Akihiro Suda
1256aa0241
vendor: github.com/opencontainers/runc v1.0.1
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-07-19 15:55:28 +09:00
Akihiro Suda
f50c7644cf
update runc binary to v1.0.1
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-07-18 20:19:33 +09:00
Sebastiaan van Stijn
8de724390e
Makefile: do not tag docker-dev image with GIT_BRANCH
When building the dev image, the Makefile generates a tag-name for the image,
based on the current git branch. As a result of this naming, old images will
collect on a developer's machine (especially when building from different
branches, for example when reviewing pull requests):

    REPOSITORY   TAG                                                 IMAGE ID       CREATED        SIZE
    docker-dev   HEAD                                                9785a8fb82f5   30 hours ago   2.13GB
    docker-dev   master                                              9785a8fb82f5   30 hours ago   2.13GB
    docker-dev   seccomp-closer-to-oci                               9785a8fb82f5   30 hours ago   2.13GB
    docker-dev   move-stackdump                                      06882c142bfd   2 days ago     2.13GB
    docker-dev   add-dns-to-docker-info                              2961ed1b99bd   10 days ago    2.13GB
    docker-dev   add-platform-info                                   2961ed1b99bd   10 days ago    2.13GB
    docker-dev   rata-seccomp-new-fields                             2961ed1b99bd   10 days ago    2.13GB
    docker-dev   swagger-wip                                         2961ed1b99bd   10 days ago    2.13GB
    docker-dev   system-df-types                                     2961ed1b99bd   10 days ago    2.13GB
    docker-dev   use-oci-platform                                    2961ed1b99bd   10 days ago    2.13GB
    docker-dev   update-swagger-fork                                 3eeedecca85a   2 weeks ago    2.13GB
    docker-dev   remove-lcow-step5-alternative                       51f9720bbc19   2 weeks ago    2.13GB
    docker-dev   update-s390x-ubuntu-2004                            51f9720bbc19   2 weeks ago    2.13GB
    docker-dev   fix-image-shared-size                               09e9aa46694a   2 weeks ago    2.13GB
    docker-dev   remove-discovery                                    11823223ae83   3 weeks ago    2.13GB
    docker-dev   daemon-config                                       355643e371b0   4 weeks ago    2.12GB
    docker-dev   jenkins-windows-containerd                          68199214b860   4 weeks ago    2.11GB
    docker-dev   unfork-buildkit                                     68199214b860   4 weeks ago    2.11GB
    docker-dev   warn-on-non-matching-platform                       bc014b94017f   5 weeks ago    2.11GB
    docker-dev   remove-lcow                                         3a43c0900282   6 weeks ago    2.11GB
    docker-dev   remove-lcow-part5                                   3a43c0900282   6 weeks ago    2.11GB
    docker-dev   remove-lcow-step3                                   3a43c0900282   6 weeks ago    2.11GB
    docker-dev   remove-lcow-step4                                   3a43c0900282   6 weeks ago    2.11GB
    docker-dev   seccomp-unconfined-daemon                           3a43c0900282   6 weeks ago    2.11GB
    docker-dev   update-authors                                      3a43c0900282   6 weeks ago    2.11GB
    docker-dev   payall4u-fix-creating-sandbox-when-disable-bridge   114c0f2ceb17   6 weeks ago    2.12GB
    docker-dev   catch-almost-all                                    f437d2bc512b   8 weeks ago    2.12GB
    docker-dev   bin-criu                                            c72894ae66f3   2 months ago   2.12GB
    docker-dev   bump-golang-1-14                                    395932141809   2 months ago   2.14GB
    docker-dev   upstream-systemd-units                              d0cb07f9473c   2 months ago   2.12GB
    docker-dev   bump-criu                                           6ed9e8fcf59f   2 months ago   2.12GB

This images are a bit of a pain to clean up, and because they are tagged,
`docker image prune` or `docker system prune` doesn't help (unless `--all` is
used).

Looking at the background of this naming, a found that it was originally added
in a95712899e, after a discussion on PR 3471.
At the time, the image name was used to check if the image needed building, and
otherwise building was skipped in the makefile.

This is no longer the case; the image is built unconditionally, and the build-
cache helps (where possible) speed up rebuilding the image.

In _theory_ having unique names would allow for multiple dev containers (from
different branches) to be started in parallel, but in most situations, the
source-code will be mounted (`BIND_MOUNT=.`), so I'm not sure if that should
be a compelling reason to keep the current naming.

This patch removes the unique tag, and will always tag the image locally as
`docker-dev:latest`.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-17 17:52:45 +02:00
Sebastiaan van Stijn
0ef7e727d2
seccomp: Seccomp: embed oci-spec LinuxSeccomp, add support for seccomp flags
This patch, similar to d92739713c, embeds the
`LinuxSeccomp` type of the runtime-spec, so that we can support all options
provided by the spec, and decorates it with our own fields.

With this, profiles can make use of the recently added "Flags" field, to
specify flags that must be passed to seccomp(2) when installing the filter.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-17 15:57:54 +02:00
Sebastiaan van Stijn
bfd4b64600
seccomp: setupSeccomp(): update errors and remove redundant check
Make the error message slightly more informative, and remove the redundant
`len(config.ArchMap) != 0` check, as iterating over an empty, or 'nil' slice
is a no-op already. This allows to use a slightly more idiomatic "if ok := xx; ok"
condition.

Also move validation to the start of the loop (early return), and explicitly create
a new slice for "names" if the legacy "Name" field is used.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-17 15:57:41 +02:00
Sebastiaan van Stijn
c815b86f40
seccomp: add additional unit-tests
Add test to verify profile validation, and to verify that the legacy
format actually loads the profile as expected (instead of only verifying
it doesn't produce an error).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-16 18:01:25 +02:00
Sebastiaan van Stijn
c1ced23544
seccomp: use oci-spec consts in tests
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-16 18:01:23 +02:00
Sebastiaan van Stijn
b309e96b11
seccomp: improve GoDoc for Seccomp fields
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-16 18:01:12 +02:00
Brian Goff
b316cc059a
Merge pull request #42636 from thaJeztah/update_containerd
Update containerd binary and vendor to v1.5.3
2021-07-15 14:10:26 -07:00
Sebastiaan van Stijn
a2da507857
Merge pull request #42604 from kinvolk/rata/seccomp-new-fields
seccomp: Sync fields with runtime-spec fields
2021-07-15 23:02:45 +02:00
Sebastiaan van Stijn
6ff6913ac4
pkg/signal: remove gotest.tools dependency
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-15 18:24:23 +02:00
Sebastiaan van Stijn
0880df4644
pkg/signal: move Trap() to cmd/dockerd
It's the only location where this is used, and it's quite specific
to dockerd (not really a reusable function for external use), so
moving it into that package.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-15 18:11:00 +02:00
Sebastiaan van Stijn
ea5c94cdb9
pkg/signal: move signal.DumpStacks() to a separate package
It is not directly related to signal-handling, so can well live
in its own package.

Also added a variant that doesn't take a directory to write files
to, for easier consumption / better match to how it's used.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-15 18:09:43 +02:00
Tianon Gravi
b2e31eb416
Merge pull request #42614 from jk-vb/patch-1
updated names-generator.go for alphabetization
2021-07-14 22:50:48 +00:00
Tianon Gravi
40502f49f6
Merge pull request #42634 from kevpar/fix-vndr-tooling
Fix up vndr tooling
2021-07-14 22:43:21 +00:00
Sebastiaan van Stijn
7d63cbfd38
api/types: add GoDoc to Info.Warnings field
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-14 23:10:16 +02:00
Sebastiaan van Stijn
4ace1998e5
pkg/sysinfo: use correct name for AppArmor in t.Skip()
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-14 23:10:13 +02:00
Sebastiaan van Stijn
9b795c3e50
pkg/sysinfo.New(), daemon.RawSysInfo(): remove "quiet" argument
The "quiet" argument was only used in a single place (at daemon startup), and
every other use had to pass "false" to prevent this function from logging
warnings.

Now that SysInfo contains the warnings that occurred when collecting the
system information, we can make leave it up to the caller to use those
warnings (and log them if wanted).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-14 23:10:07 +02:00
Sebastiaan van Stijn
1fb62f455c
pkg/sysinfo: collect warnings in SysInfo struct
This allows the warnings to be consumed in other locations.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-14 17:28:25 +02:00
Sebastiaan van Stijn
208d3c6efb
pkg/sysinfo: move cg2Controllers to be a field in SysInfo and unify v1/v2
We pass the SysInfo struct to all functions. Adding cg2Controllers as a
(non-exported) field makes passing around this information easier.

Now that infoCollector and infoCollectorV2 have the same signature, we can
simplify some bits and use a single slice for all "collectors".

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-14 16:39:44 +02:00
Sebastiaan van Stijn
5cc20ad9e5
pkg/sysinfo: adjust Opt to set new field
This removes the need to have the opts type.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-14 16:39:26 +02:00
Sebastiaan van Stijn
ca27b473cc
pkg/sysinfo: move cg2GroupPath to be a field in SysInfo
We pass the SysInfo struct to all functions. Adding cg2GroupPath as a
(non-exported) field makes passing around this information easier.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-14 16:37:03 +02:00
Sebastiaan van Stijn
e70c5ea1a9
pkg/sysinfo.newV2() remove redundant path.Clean()
path.Join() already does path.Clean(), and the opts.cg2GroupPath
field is already cleaned as part of WithCgroup2GroupPath()

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-14 16:37:02 +02:00
Sebastiaan van Stijn
6677ab6a63
pkg/sysinfo: move cgMounts to be a field in SysInfo
We pass the SysInfo struct to all functions. Adding cgMounts as a
(non-exported) field makes passing around this information easier.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-14 16:37:00 +02:00
Sebastiaan van Stijn
10ce0d84c2
pkg/sysinfo.New() move v1 code to a newV1() function
This makes it clearer that this code is the cgroups v1 equivalent of newV2().

Also moves the "options" handling to newV2() because it's currently only used
for cgroupsv2.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-14 16:36:56 +02:00
Akihiro Suda
22d6671db2
Merge pull request #42631 from thaJeztah/bump_go_1.16.6
Bump go 1.16.6 (addresses CVE-2021-34558)
2021-07-14 12:50:10 +09:00
jk-vb
1d055267a8
updated names-generator.go for alphabetization
re ordered some entries so they are in proper alphabetical order

Signed-off-by: Justin Keller <jkeller@vb-jkeller-mbp.local>
2021-07-13 23:52:51 +02:00
Sebastiaan van Stijn
a4ee738b31
Merge pull request #42611 from kevpar/update-hcsshim
Update hcsshim vendor to v0.8.20
2021-07-13 23:48:11 +02:00
Sebastiaan van Stijn
84df737f50
vendor: github.com/containerd/containerd v1.5.3
full diff: https://github.com/containerd/containerd/compare/v1.5.2...v1.5.3

- Fix User Agent sent to registry authentication server (changes default user-
  agent from "Go-http-client/1.1" to "containerd/v1.5.3")
- Fix missing Body.Close() calls on push to docker remote
- Change Wrapf of non-error to an actual error
    - fixes Failed to pull image (unexpected commit digest)
- fix invalid validation error checking
- Update hcsshim to 0.8.18
- Update Go to 1.16.6
- content/local: inline sys.StatATimeAsTime()
- windows: Use GetFinalPathNameByHandle for ResolveSymbolicLink
- Fix cleanup context of teardownPodNetwork
    - fixes CRI fails to invoke CNI plugin to teardown network when RunPodSandbox times out
- sandbox: send pod UID to CNI plugins as K8S_POD_UID

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-13 23:32:56 +02:00
Sebastiaan van Stijn
5ae2af41ee
Update containerd binary to v1.5.3
full diff: https://github.com/containerd/containerd/compare/v1.5.2...v1.5.3

Welcome to the v1.5.3 release of containerd!

The third patch release for containerd 1.5 updates runc to 1.0.0 and contains
various other fixes.

Notable Updates

- Update runc binary to 1.0.0
- Send pod UID to CNI plugins as K8S_POD_UID
- Fix invalid validation error checking
- Fix error on image pull resume
- Fix User Agent sent to registry authentication server
- Fix symlink resolution for disk mounts on Windows

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-13 23:17:29 +02:00
Kevin Parsons
3334fb9af1 Fix up vndr tooling
- Fix the error message in hack/validate/vendor to specify that
  hack/vendor.sh should be run instead of vndr.
- Fix hack/vendor.sh to also match on Windows paths for the whitelist.
  This allows the script to be run on Windows via Git Bash.

Signed-off-by: Kevin Parsons <kevpar@microsoft.com>
2021-07-13 09:45:07 -07:00
Justin Cormack
cf73a45a13
Merge pull request #42622 from aiordache/carry_wants_containerd
Move containerd.service from Requires= to Wants=
2021-07-13 13:55:11 +01:00
Roman Volosatovs
2af9bd3b26
API: add shared-size parameter to image queries
The reasoning for this change is to be able to query image shared size without having to rely on the more heavyweight `/system/df` endpoint.

Signed-off-by: Roman Volosatovs <roman.volosatovs@docker.com>
2021-07-13 13:46:07 +02:00
Roman Volosatovs
bf9c76f0a8
API, daemon/images: add ImageListOptions and pass context
This makes it easier to add more options to the backend without having to change
the signature.

While we're changing the signature, also adding a context.Context, which is not
currently used, but probably should be at some point.

Signed-off-by: Roman Volosatovs <roman.volosatovs@docker.com>
2021-07-13 13:45:24 +02:00
Sebastiaan van Stijn
fe6f1a4067
Bump go 1.16.6 (addresses CVE-2021-34558)
This addresses CVE-2021-34558: https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-34558

go1.16.6 (released 2021-07-12) includes a security fix to the crypto/tls package,
as well as bug fixes to the compiler, and the net and net/http packages. See the
Go 1.16.6 milestone on the issue tracker for details:

https://github.com/golang/go/issues?q=milestone%3AGo1.16.6+label%3ACherryPickApproved

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-13 12:42:59 +02:00
Sebastiaan van Stijn
bf78e25fe5
Merge pull request #42608 from rvolosatovs/fix_builder_size
Deprecate `BuilderSize` in API versions >= 1.42
2021-07-12 19:29:22 +02:00
Roman Volosatovs
2837fba75f
libnetwork: ensure all nodes are available in tests
`github.com/hashicorp/memberlist` update caused `TestNetworkDBCRUDTableEntries`
to occasionally fail, because the test would try to check whether an entry
write is propagated to all nodes, but it would not wait for all nodes to
be available before performing the write.
It could be that the failure is caused simply by improved performance of
the dependency - it could also be that some connectivity guarantee the
test depended on is not provided by the dependency anymore.
The same fix is applied to `TestNetworkDBNodeJoinLeaveIteration` due to
same issue.

Signed-off-by: Roman Volosatovs <roman.volosatovs@docker.com>
2021-07-12 19:25:50 +02:00
Roman Volosatovs
cdd04a94bc
vendor: hashicorp/memberlist, google/btree (dep) update
Upstream update fixes the issue where left node would be marked as
failed, which caused `TestNetworkDBIslands` to occasionally fail.

Signed-off-by: Roman Volosatovs <roman.volosatovs@docker.com>
2021-07-12 19:25:50 +02:00