Commit graph

44143 commits

Author SHA1 Message Date
Sebastiaan van Stijn
a747cd3702
update containerd binary to v1.6.5
Welcome to the v1.6.5 release of containerd!

The fifth patch release for containerd 1.6 includes a few fixes and updated
version of runc.

Notable Updates

- Fix for older CNI plugins not reporting version
- Fix mount path handling for CRI plugin on Windows

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-06-04 22:28:42 +02:00
Samuel Karp
a586d536be
Merge pull request #43688 from ferozsalam/fix-names-alphabetisation 2022-06-03 16:12:03 -07:00
Sebastiaan van Stijn
38633e7971
Merge pull request #43689 from thaJeztah/fix_incorrect_warnings
daemon.NewDaemon(): fix network feature detection on first start
2022-06-03 19:30:24 +02:00
Sebastiaan van Stijn
b241e2008e
daemon.NewDaemon(): fix network feature detection on first start
Commit 483aa6294b introduced a regression, causing
spurious warnings to be shown when starting a daemon for the first time after
a fresh install:

    docker info
    ...
    WARNING: IPv4 forwarding is disabled
    WARNING: bridge-nf-call-iptables is disabled
    WARNING: bridge-nf-call-ip6tables is disabled

The information shown is incorrect, as checking the corresponding options on
the system, shows that these options are available:

    cat /proc/sys/net/ipv4/ip_forward
    1
    cat /proc/sys/net/bridge/bridge-nf-call-iptables
    1
    cat /proc/sys/net/bridge/bridge-nf-call-ip6tables
    1

The reason this is failing is because the daemon itself reconfigures those
options during networking initialization in `configureIPForwarding()`;
cf4595265e/libnetwork/drivers/bridge/setup_ip_forwarding.go (L14-L25)

Network initialization happens in the `daemon.restore()` function within `daemon.NewDaemon()`:
cf4595265e/daemon/daemon.go (L475-L478)

However, 483aa6294b moved detection of features
earlier in the `daemon.NewDaemon()` function, and collects the system information
(`d.RawSysInfo()`) before we enter `daemon.restore()`;
cf4595265e/daemon/daemon.go (L1008-L1011)

For optimization (collecting the system information comes at a cost), those
results are cached on the daemon, and will only be performed once (using a
`sync.Once`).

This patch:

- introduces a `getSysInfo()` utility, which collects system information without
  caching the results
- uses `getSysInfo()` to collect the preliminary information needed at that
  point in the daemon's lifecycle.
- moves printing warnings to the end of `daemon.NewDaemon()`, after all information
  can be read correctly.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-06-03 17:54:43 +02:00
Feroz Salam
1db286c5e8 Fix alphabetisation of possible names
Signed-off-by: Feroz Salam <feroz.salam@sourcegraph.com>
2022-06-03 15:07:52 +01:00
Sebastiaan van Stijn
cf4595265e
Merge pull request #43687 from thaJeztah/update_authors_mailmap
update AUTHORS and mailmap
2022-06-03 13:15:27 +02:00
Sebastiaan van Stijn
ca6e94f994
update AUTHORS and mailmap
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-06-03 12:29:40 +02:00
Sebastiaan van Stijn
6e80c027c5
Merge pull request #43409 from vincentbernat/fix/udp-conntrack
bridge: also flush conntrack entries when setting up endpoints
2022-06-03 11:29:25 +02:00
Sebastiaan van Stijn
82e58a0011
Merge pull request #43066 from akerouanton/fix-43054
Revert "Added API to set ephemeral port allocator range."
2022-06-03 10:13:03 +02:00
Sebastiaan van Stijn
9cace7da9e
Merge pull request #43632 from thaJeztah/volumes_fixup_part2
volume/local: refactor validation to allow invalidating early
2022-06-03 09:37:57 +02:00
Sebastiaan van Stijn
cd58d11b2a
volume/local.New(): extract loading options to a function
Note that Windows does not support options, so strictly doesn't need
to have this code, but keeping it in case we're adding support.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-06-03 00:34:29 +02:00
Sebastiaan van Stijn
7e907e29a3
volume/local.New() always unmount existing mounts
Unmounting does not depend on wether or not loading options failed.

This code-path seemed to be used as a "hack" to prevent hitting the
unmount on Windows (which does not support unmounting).

Moving it outside of the "if" to make more clear that it's independent
of loading the options.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-06-03 00:34:27 +02:00
Sebastiaan van Stijn
c0f0cf6c19
volume/local: extract saving options to a separate method
Differentiate between Windows and Linux, as Windows doesn't support
options, so there's no need to save options to disk,

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-06-03 00:34:25 +02:00
Sebastiaan van Stijn
d3930330a7
volume/local: store both volume's "data" and "root" path
Instead of evaluating these paths each time (appending `_data`, or using
`filepath.Dir()` to find the root path from the `_data_` path).

This also removes the `root.DataPath()` utility, which is now no longer needed.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-06-03 00:34:24 +02:00
Sebastiaan van Stijn
e106e3f5c6
volume/local: make "validateOpts()" a method on Root
This way we can validate if Root supports quotaCtl, allowing us to
fail early, before creating any of the directories.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-06-03 00:34:22 +02:00
Sebastiaan van Stijn
29c6224fe9
volume/local.Create(): validate early
This moves validation of options to the start of the Create function
to prevent hitting the filesystem and having to remove the volume
from disk.

Also addressing some minor nits w.r.t. errors.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-06-03 00:34:21 +02:00
Sebastiaan van Stijn
a77b90c35e
volume/local: make setOpts() a method of localVolume
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-06-03 00:34:19 +02:00
Sebastiaan van Stijn
b56fc2d0f8
volume/local.New(): don't register volume before we're done
Loading options may fail, in which case we don't have to add
the volume to the list.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-06-03 00:34:17 +02:00
Sebastiaan van Stijn
eecf7a0840
volume/local: localVolume.mount() move errors.Wrap()
While the current code is correct (as errors.Wrap() returns nil if
err is nil), relying on this behavior has caused some confusion in
the past, resulting in regressions.

This patch makes the error-handling code slightly more idiomatic and
defensive against such regressions.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-06-03 00:34:16 +02:00
Sebastiaan van Stijn
c1671abf14
volume/local: add test for validation
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-06-03 00:34:08 +02:00
Sebastiaan van Stijn
b4b2a0323b
libnetwork/portallocator: un-export consts for defaults
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-06-02 22:59:32 +02:00
Albin Kerouanton
414dd017b3
Revert "Added API to set ephemeral port allocator range."
Since commit 2c4a868f64, Docker doesn't
use the value of net.ipv4.ip_local_port_range when choosing an ephemeral
port. This change reverts back to the previous behavior.

Fixes #43054.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2022-06-02 22:56:18 +02:00
Tianon Gravi
0e04b514fb
Merge pull request #43681 from thaJeztah/reduce_flakiness
fix flaky TestRunContainerWithRmFlag tests
2022-06-02 13:19:39 -07:00
Sebastiaan van Stijn
7bb335999e
Merge pull request #43675 from tonistiigi/make-pull-match-containerd
distribution: match manifest list resolution with containerd
2022-06-02 22:04:55 +02:00
Sebastiaan van Stijn
6f1cefe385
Merge pull request #43678 from crazy-max/ci-concurrency
ci: add concurrency check
2022-06-02 21:22:01 +02:00
Tonis Tiigi
9adad264d2 distribution: match manifest list resolution with containerd
Make finding the correct runtime image from image index
more compliant with OCI spec and match containerd implementation.

Changes:
- Manifest list is allowed to contain manifest lists
- Unknown mediatype inside manifest list is skipped instead of causing an error
- Platform in descriptor is optional 

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2022-06-02 11:21:07 -07:00
Tianon Gravi
1f00103722
Merge pull request #43676 from thaJeztah/update_golang_1.18.3
update golang to 1.18.3
2022-06-02 11:05:39 -07:00
Sebastiaan van Stijn
585c147b7a
fix flaky TestRunContainerWithRmFlag tests
This attempts to fix CI flakiness on the TestRunContainerWithRmFlagCannotStartContainer
and TestRunContainerWithRmFlagExitCodeNotEqualToZero tests.

These tests;

- get a list of all container ID's
- run a container with `--rm`
- wait for it to exit
- checks that the list of all container IDs is empty

The last step assumes that no other tests are running on the same daemon; if
another test is running, there may be other containers present (unrelated to
the test).

This patch updates the tests to use a `docker inspect` to verify the container
no longer exists afterwards.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-06-02 16:55:44 +02:00
Sebastiaan van Stijn
fbcdc6a980
Merge pull request #43604 from Ultimator14/master
Use separate openrc init script for containerd
2022-06-02 16:05:19 +02:00
CrazyMax
a0a6b82a0d
ci: add concurrency check
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-06-02 15:21:05 +02:00
Sebastiaan van Stijn
c1a9ffc97a
update golang to 1.18.3
go1.18.3 (released 2022-06-01) includes security fixes to the crypto/rand,
crypto/tls, os/exec, and path/filepath packages, as well as bug fixes to the
compiler, and the crypto/tls and text/template/parse packages. See the Go
1.18.3 milestone on our issue tracker for details:

https://github.com/golang/go/issues?q=milestone%3AGo1.18.3+label%3ACherryPickApproved

Hello gophers,

We have just released Go versions 1.18.3 and 1.17.11, minor point releases.

These minor releases include 4 security fixes following the security policy:

- crypto/rand: rand.Read hangs with extremely large buffers
  On Windows, rand.Read will hang indefinitely if passed a buffer larger than
  1 << 32 - 1 bytes.

  Thanks to Davis Goodin and Quim Muntal, working at Microsoft on the Go toolset,
  for reporting this issue.

  This is [CVE-2022-30634][CVE-2022-30634] and Go issue https://go.dev/issue/52561.
- crypto/tls: session tickets lack random ticket_age_add
  Session tickets generated by crypto/tls did not contain a randomly generated
  ticket_age_add. This allows an attacker that can observe TLS handshakes to
  correlate successive connections by comparing ticket ages during session
  resumption.

  Thanks to GitHub user nervuri for reporting this.

  This is [CVE-2022-30629][CVE-2022-30629] and Go issue https://go.dev/issue/52814.
- `os/exec`: empty `Cmd.Path` can result in running unintended binary on Windows

  If, on Windows, `Cmd.Run`, `cmd.Start`, `cmd.Output`, or `cmd.CombinedOutput`
  are executed when Cmd.Path is unset and, in the working directory, there are
  binaries named either "..com" or "..exe", they will be executed.

  Thanks to Chris Darroch, brian m. carlson, and Mikhail Shcherbakov for reporting
  this.

  This is [CVE-2022-30580][CVE-2022-30580] and Go issue https://go.dev/issue/52574.
- `path/filepath`: Clean(`.\c:`) returns `c:` on Windows

  On Windows, the `filepath.Clean` function could convert an invalid path to a
  valid, absolute path. For example, Clean(`.\c:`) returned `c:`.

  Thanks to Unrud for reporting this issue.

  This is [CVE-2022-29804][CVE-2022-29804] and Go issue https://go.dev/issue/52476.

[CVE-2022-30634]: https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-30634
[CVE-2022-30629]: https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-30629
[CVE-2022-30580]: https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-30580
[CVE-2022-29804]: https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-29804

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-06-02 09:16:46 +02:00
Jan Breig
bb69104381 Openrc: Depend on containerd init script
Signed-off-by: Jan Breig <git@pygos.space>
2022-06-01 15:09:52 +02:00
Sebastiaan van Stijn
3e32104e0e
Merge pull request #43672 from thaJeztah/remove_windows_jenkins
Jenkinsfile: remove Windows stages
2022-06-01 11:02:43 +02:00
Sebastiaan van Stijn
ea3d0f3b5e
Merge pull request #43665 from crazy-max/win-test-logs
ci(windows): upload test reports with logs
2022-06-01 11:01:54 +02:00
Sebastiaan van Stijn
caf9c35e5c
Merge pull request #43671 from thaJeztah/close_the_door_please
fix unclosed file-handles in tests
2022-05-31 23:25:01 +02:00
CrazyMax
6ecbd078c2
ci(windows): upload test reports with logs
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-05-31 22:19:23 +02:00
Sebastiaan van Stijn
e3c14acd33
Jenkinsfile: remove Windows stages
These are now running in GitHub actions, so we no longer need to
run them in Jenkins.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-05-31 22:14:15 +02:00
Sebastiaan van Stijn
5a8304c9c5
Merge pull request #43670 from evol262/ipvs-module-idempotency
Ensure performance tuning is always applied
2022-05-31 22:00:40 +02:00
Sebastiaan van Stijn
553b0edb4c
fix unclosed file-handles in tests
These seemed to prevent cleaning up directories;

On arm64:

    === RUN   TestSysctlOverride
        testing.go:1090: TempDir RemoveAll cleanup: unlinkat /tmp/TestSysctlOverride2860094781/001/mounts/shm: device or resource busy
    --- FAIL: TestSysctlOverride (0.00s)

On Windows:

    === Failed
    === FAIL: github.com/docker/docker/daemon TestLoadOrCreateTrustKeyInvalidKeyFile (0.00s)
        testing.go:1090: TempDir RemoveAll cleanup: remove C:\Users\CONTAI~1\AppData\Local\Temp\TestLoadOrCreateTrustKeyInvalidKeyFile2014634395\001\keyfile4156691647: The process cannot access the file because it is being used by another process.

    === FAIL: github.com/docker/docker/daemon/graphdriver TestIsEmptyDir (0.01s)
        testing.go:1090: TempDir RemoveAll cleanup: remove C:\Users\CONTAI~1\AppData\Local\Temp\TestIsEmptyDir1962964337\001\dir-with-empty-file\file2523853824: The process cannot access the file because it is being used by another process.

    === FAIL: github.com/docker/docker/pkg/directory TestSizeEmptyFile (0.00s)
        testing.go:1090: TempDir RemoveAll cleanup: remove C:\Users\CONTAI~1\AppData\Local\Temp\TestSizeEmptyFile1562416712\001\file16507846: The process cannot access the file because it is being used by another process.

    === FAIL: github.com/docker/docker/pkg/directory TestSizeNonemptyFile (0.00s)
        testing.go:1090: TempDir RemoveAll cleanup: remove C:\Users\CONTAI~1\AppData\Local\Temp\TestSizeNonemptyFile1240832785\001\file3265662846: The process cannot access the file because it is being used by another process.

    === FAIL: github.com/docker/docker/pkg/directory TestSizeFileAndNestedDirectoryEmpty (0.00s)
        testing.go:1090: TempDir RemoveAll cleanup: remove C:\Users\CONTAI~1\AppData\Local\Temp\TestSizeFileAndNestedDirectoryEmpty2163416550\001\file3715413181: The process cannot access the file because it is being used by another process.

    === FAIL: github.com/docker/docker/pkg/directory TestSizeFileAndNestedDirectoryNonempty (0.00s)
        testing.go:1090: TempDir RemoveAll cleanup: remove C:\Users\CONTAI~1\AppData\Local\Temp\TestSizeFileAndNestedDirectoryNonempty878205470\001\file3280422273: The process cannot access the file because it is being used by another process.

    === FAIL: github.com/docker/docker/volume/service TestSetGetMeta (0.01s)
        testing.go:1090: TempDir RemoveAll cleanup: remove C:\Users\CONTAI~1\AppData\Local\Temp\TestSetGetMeta3332268057\001\db: The process cannot access the file because it is being used by another process.

    === FAIL: github.com/docker/docker/volume/service TestList (0.03s)
        testing.go:1090: TempDir RemoveAll cleanup: remove C:\Users\CONTAI~1\AppData\Local\Temp\TestList2846947953\001\volumes\metadata.db: The process cannot access the file because it is being used by another process.

    === FAIL: github.com/docker/docker/volume/service TestRestore (0.02s)
        testing.go:1090: TempDir RemoveAll cleanup: remove C:\Users\CONTAI~1\AppData\Local\Temp\TestRestore3368254142\001\volumes\metadata.db: The process cannot access the file because it is being used by another process.

    === FAIL: github.com/docker/docker/daemon/graphdriver TestIsEmptyDir (0.00s)
        testing.go:1090: TempDir RemoveAll cleanup: remove C:\Users\CONTAI~1\AppData\Local\Temp\TestIsEmptyDir2823795693\001\dir-with-empty-file\file2625561089: The process cannot access the file because it is being used by another process.

    === FAIL: github.com/docker/docker/pkg/directory TestSizeFileAndNestedDirectoryNonempty (0.00s)
        testing.go:1090: TempDir RemoveAll cleanup: remove C:\Users\CONTAI~1\AppData\Local\Temp\TestSizeFileAndNestedDirectoryNonempty4246252950\001\nested3442260313\file21164327: The process cannot access the file because it is being used by another process.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-05-31 21:53:38 +02:00
Sebastiaan van Stijn
ceefa7c7ff
Merge pull request #43668 from thaJeztah/bump_ipvs
vendor: github.com/moby/ipvs v1.0.2
2022-05-31 20:36:39 +02:00
Sebastiaan van Stijn
bc541ba66f
Merge pull request #43666 from vvoland/fix-logs-regression3
daemon/logger: Fix TestConcurrentLogging race test
2022-05-31 18:05:12 +02:00
Ryan Barry
293cfd6c76 Ensure performance tuning is always applied
Previously, with the patch from #43146, it was possible for a
network configured with a single ingress or load balancer on a
distribution which does not have the `ip_vs` kernel module loaded
by default to try to apply sysctls which did not exist yet, and
subsequently dynamically load the module as part of ipvs/netlink.go.

This module is vendored, and not a great place to try to tie back
into core libnetwork functionality, so also ensure that the sysctls
(which are idempotent) are called after ingress/lb creation once
`ipvs` has been initialized.

Signed-off-by: Ryan Barry <rbarry@mirantis.com>
2022-05-31 11:47:30 -04:00
Sebastiaan van Stijn
a06bae9094
Merge pull request #43656 from fussybeaver/ND-optional-container-wait-error
api: swagger: Adjust ContainerWaitResponse error as optional
2022-05-31 17:44:36 +02:00
Sebastiaan van Stijn
c51d34dcfb
vendor: github.com/moby/ipvs v1.0.2
full diff: https://github.com/moby/ipvs/compare/v1.0.1...v1.0.2

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-05-31 15:40:51 +02:00
Paweł Gronowski
2463c40144 daemon/logger: Fix TestConcurrentLogging race test
The recent fix for log corruption changed the signature of the
NewLogFile and WriteLogEntry functions and the test wasn't adjusted to
this change.

Fix the test by adjusting to the new LogFile API.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2022-05-31 14:02:59 +02:00
Niel Drummond
152467d185 api: swagger: adjust ContainerWaitResponse error as optional
Signed-off-by: Niel Drummond <niel@drummond.lu>
2022-05-30 16:42:12 +01:00
Niel Drummond
2359da802b docs: api: adjust ContainerWaitResponse error as optional
Signed-off-by: Niel Drummond <niel@drummond.lu>
2022-05-30 16:40:34 +01:00
Sebastiaan van Stijn
4e09933aed
Merge pull request #43652 from thaJeztah/update_gotest_tools
vendor: gotest.tools v3.2.0
2022-05-30 13:00:29 +02:00
Sebastiaan van Stijn
8752ec9976
Merge pull request #43658 from thaJeztah/fix_TestPingSwarmHeader
integration: TestPingSwarmHeader(): fix incorrect ping, and cleanup
2022-05-30 12:59:58 +02:00
Sebastiaan van Stijn
a71463be98
Merge pull request #43659 from thaJeztah/update_TestCleanupMountsByID
daemon: fix daemon.Shutdown, daemon.Cleanup not cleaning up overlay2 mounts
2022-05-30 11:53:16 +02:00