Compare commits

..

789 commits

Author SHA1 Message Date
Sebastiaan van Stijn
b201b1b3d0
Merge pull request #46729 from thaJeztah/20.10_backport_remove-ibm-jenkins-jobs
[20.10 backport] Remove s390x and ppc64le Jenkins pipelines
2023-10-27 18:46:15 +02:00
Sam Thibault
79ae899640
remove s390x and ppc64ls pipelines
Signed-off-by: Sam Thibault <sam.thibault@docker.com>
(cherry picked from commit 59aa3dce8a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-26 10:39:04 +02:00
CrazyMax
f0e31a73fc
Makefile: remove unused BUILD_APT_MIRROR
BUILD_APT_MIRROR added in https://github.com/moby/moby/pull/26375
is not used anymore.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 7c697f58f2)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-26 10:36:12 +02:00
CrazyMax
e019f78e83
Dockerfile: use default apt mirrors
Use default apt mirrors and also check APT_MIRROR
is set before updating mirrors.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit a1d2132bf6)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-26 10:33:05 +02:00
Sebastiaan van Stijn
81ebe71275
Merge pull request from GHSA-jq35-85cj-fj4p
[20.10 backport] deny /sys/devices/virtual/powercap
2023-10-25 23:57:51 +02:00
Cory Snider
fb636657a7
Merge pull request #46705 from thaJeztah/20.10_backport_atomic-layer-data-write
[20.10 backport] daemon: overlay2: Write layer metadata atomically
2023-10-24 10:22:46 -04:00
Sebastiaan van Stijn
b967d89758
Merge pull request #46692 from corhere/backport-20.10/update-x-net-v0.17
[20.10] Update to go1.20.10, golang/org/x/net v0.17.0
2023-10-24 12:51:13 +02:00
Cory Snider
2c22bd5280 vendor: golang.org/x/net v0.17.0
full diff: https://github.com/golang/net/compare/ab34263943818b32f575efc978

This fixes the same CVE as go1.21.3 and go1.20.10;

- net/http: rapid stream resets can cause excessive work

  A malicious HTTP/2 client which rapidly creates requests and
  immediately resets them can cause excessive server resource consumption.
  While the total number of requests is bounded to the
  http2.Server.MaxConcurrentStreams setting, resetting an in-progress
  request allows the attacker to create a new request while the existing
  one is still executing.

  HTTP/2 servers now bound the number of simultaneously executing
  handler goroutines to the stream concurrency limit. New requests
  arriving when at the limit (which can only happen after the client
  has reset an existing, in-flight request) will be queued until a
  handler exits. If the request queue grows too large, the server
  will terminate the connection.

  This issue is also fixed in golang.org/x/net/http2 v0.17.0,
  for users manually configuring HTTP/2.

  The default stream concurrency limit is 250 streams (requests)
  per HTTP/2 connection. This value may be adjusted using the
  golang.org/x/net/http2 package; see the Server.MaxConcurrentStreams
  setting and the ConfigureServer function.

  This is CVE-2023-39325 and Go issue https://go.dev/issue/63417.
  This is also tracked by CVE-2023-44487.

Dependency full diffs:
a3d24e80b04bd7...v0.17.0
https://github.com/golang/sys/compare/33da011f77ade50ff5b6a6fb4a
9a1e6d6b285809...v0.13.0
https://github.com/golang/text/compare/v0.3.3...v0.13.0
https://github.com/golang/crypto/compare/c1f2f97bffc9c53fc40a1a28a5
b460094c0050d9...v0.14.0

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-10-23 16:37:52 -04:00
Cory Snider
d862c21eb2 Update to go1.20.10
go1.20.10 (released 2023-10-10) includes a security fix to the net/http
package. See the Go 1.20.10 milestone on our issue tracker for details.

- https://github.com/golang/go/issues?q=milestone%3AGo1.20.10+label%3ACherryPickApproved
- full diff: https://github.com/golang/go/compare/go1.19.12...go1.20.10

From the mailing list announcement:

[security] Go 1.21.3 and Go 1.20.10 are released

Hello gophers,

We have just released Go versions 1.21.3 and 1.20.10, minor point releases.

These minor releases include 1 security fixes following the security policy:

- net/http: rapid stream resets can cause excessive work

  A malicious HTTP/2 client which rapidly creates requests and
  immediately resets them can cause excessive server resource consumption.
  While the total number of requests is bounded to the
  http2.Server.MaxConcurrentStreams setting, resetting an in-progress
  request allows the attacker to create a new request while the existing
  one is still executing.

  HTTP/2 servers now bound the number of simultaneously executing
  handler goroutines to the stream concurrency limit. New requests
  arriving when at the limit (which can only happen after the client
  has reset an existing, in-flight request) will be queued until a
  handler exits. If the request queue grows too large, the server
  will terminate the connection.

  This issue is also fixed in golang.org/x/net/http2 v0.17.0,
  for users manually configuring HTTP/2.

  The default stream concurrency limit is 250 streams (requests)
  per HTTP/2 connection. This value may be adjusted using the
  golang.org/x/net/http2 package; see the Server.MaxConcurrentStreams
  setting and the ConfigureServer function.

  This is CVE-2023-39325 and Go issue https://go.dev/issue/63417.
  This is also tracked by CVE-2023-44487.

View the release notes for more information:
https://go.dev/doc/devel/release#go1.21.3

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-10-23 16:37:50 -04:00
Cory Snider
cb47414f41
Merge pull request #46696 from corhere/backport-20.10/go1.20-enablement
[20.10 backport] Go 1.20 Enablement
2023-10-23 16:36:49 -04:00
Cory Snider
ea4eb7398c
Merge pull request #46695 from corhere/backport-20.10/safer-fileinfo
[20.10 backport] Remove local fork of archive/tar package
2023-10-23 15:45:09 -04:00
Cory Snider
6c523aabae hack: fix suppressing Xattrs lint errors
Finish cherry-picking the remaining part of
65e1adc219 which was not included in
commit 432fbc8638.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-10-23 14:45:24 -04:00
Cory Snider
31b837499c pkg/archive: audit gosec file-traversal lints
The recently-upgraded gosec linter has a rule for archive extraction
code which may be vulnerable to directory traversal attacks, a.k.a. Zip
Slip. Gosec's detection is unfortunately prone to false positives,
however: it flags any filepath.Join call with an argument derived from a
tar.Header value, irrespective of whether the resultant path is used for
filesystem operations or if directory traversal attacks are guarded
against.

All of the lint errors reported by gosec appear to be false positives.

Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit 833139f390)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-10-23 14:45:24 -04:00
Cory Snider
8e4485536b Remove local fork of archive/tar package
A copy of Go's archive/tar packge was vendored with a patch applied to
mitigate CVE-2019-14271. Vendoring standard library packages is not
supported by Go in module-aware mode, which is getting in the way of
maintenance. A different approach to mitigate the vulnerability is
needed which does not involve vendoring parts of the standard library.

glibc implements name service lookups such as users, groups and DNS
using a scheme known as Name Service Switch. The services are
implemented as modules, shared libraries which glibc dynamically links
into the process the first time a function requiring the module is
called. This is the crux of the vulnerability: if a process linked
against glibc chroots, then calls one of the functions implemented with
NSS for the first time, glibc may load NSS modules out of the chrooted
filesystem.

The API underlying the `docker cp` command is implemented by forking a
new process which chroots into the container's rootfs and writes a tar
stream of files from the container over standard output. It utilizes the
Go standard library's archive/tar package to write the tar stream. It
makes use of the tar.FileInfoHeader function to construct a tar.Header
value from an fs.FileInfo value. In modern versions of Go on *nix
platforms, FileInfoHeader will attempt to resolve the file's UID and GID
to their respective user and group names by calling the os/user
functions LookupId and LookupGroupId. The cgo implementation of os/user
on *nix performs lookups by calling the corresponding libc functions. So
when linked against glibc, calls to tar.FileInfoHeader after the
process has chrooted into the container's rootfs can have the side
effect of loading NSS modules from the container! Without any
mitigations, a malicious container image author can trivially get
arbitrary code execution by leveraging this vulnerability and escape the
chroot (which is not a sandbox) into the host.

Mitigate the vulnerability without patching or forking archive/tar by
hiding the OS-dependent file info from tar.FileInfoHeader which it needs
to perform the lookups. Without that information available it falls back
to populating the tar.Header with only the information obtainable
directly from the FileInfo value without making any calls into os/user.

Fixes #42402

Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit e9bbc41dd1)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-10-23 14:45:24 -04:00
Mike Sul
2f98ada8b4
daemon: overlay2: Write layer metadata atomically
When the daemon process or the host running it is abruptly terminated,
the layer metadata file can become inconsistent on the file system.
Specifically, `link` and `lower` files may exist but be empty, leading
to overlay mounting errors during layer extraction, such as:
"failed to register layer: error creating overlay mount to <path>:
too many levels of symbolic links."

This commit introduces the use of `AtomicWriteFile` to ensure that the
layer metadata files contain correct data when they exist on the file system.

Signed-off-by: Mike <mike.sul@foundries.io>
(cherry picked from commit de2447c2ab)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-23 20:21:51 +02:00
Sebastiaan van Stijn
c31a50bf8c
Merge pull request #46694 from corhere/backport-20.10/runtime-eisdir
[20.10 backport] daemon: handle EISDIR error from runtime
2023-10-21 11:42:52 +02:00
Cory Snider
cf90912e90
Merge pull request #46693 from corhere/vendor-libnetwork-for-20.10.27
[20.10] vendor: github.com/docker/libnetwork 3797618f9a38372e8107d8c06f6ae199e1133ae8
2023-10-20 19:04:55 -04:00
Cory Snider
79d5066c80 Upgrade to golangci-lint v1.51.2
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-10-20 19:01:17 -04:00
Cory Snider
c322779dce distribution/xfer: make off-by-one error a feature
maxDownloadAttempts maps to the daemon configuration flag

    --max-download-attempts int
      Set the max download attempts for each pull (default 5)

and the daemon configuration machinery interprets a value of 0 as "apply
the default value" and not a valid user value (config validation/
normalization bugs notwithstanding). The intention is clearly that this
configuration value should be an upper limit on the number of times the
daemon should try to download a particular layer before giving up. So it
is surprising to have the configuration value interpreted as a _retry_
limit. The daemon will make up to N+1 attempts to download a layer! This
also means users cannot disable retries even if they wanted to.

As this is a longstanding bug, not a recent regression, it would not be
appropriate to backport the fix (97921915a8)
in a patch release. Update the test to assert on the buggy behaviour so
it passes again.

Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit 938ed9a1ed)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-10-20 19:01:17 -04:00
Cory Snider
2a69cc6e75 Fix loop-closure bugs in tests
...which were flagged by golangci-lint v1.51.

Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit 0c68b655f6)
Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit ca712d6947)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-10-20 19:01:17 -04:00
Cory Snider
a9485b75b1 d/l/awslogs: fix ineffective Add in test
...flagged by golangci-lint v1.51.1 (staticcheck).

Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit e66995d840)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-10-20 19:01:17 -04:00
Cory Snider
134008381b Migrate away from things deprecated in Go 1.20
"archive/tar".TypeRegA
  - The deprecated constant tar.TypeRegA is the same value as
    tar.TypeReg and so is not needed at all.

Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit dea3f2b417)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-10-20 19:01:17 -04:00
Sebastiaan van Stijn
f98e22354f daemon: fix TestVerifyPlatformContainerResources not capturing variable
This test runs with t.Parallel() _and_ uses subtests, but didn't capture
the `tc` variable, which potentialy (likely) makes it test the same testcase
multiple times.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 0c887404a8)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-10-20 19:01:17 -04:00
Roman Volosatovs
526d21449e integration: remove deprecated endpoint test
`/containers/<name>/copy` endpoint was deprecated in 1.8 and errors
since 1.12. See https://github.com/moby/moby/pull/22149 for more info.

Signed-off-by: Roman Volosatovs <roman.volosatovs@docker.com>
(cherry picked from commit a34d804572)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-10-20 19:01:17 -04:00
Roman Volosatovs
c643f0e48f integration: copy loop variable into parallel test closures
Discovered a few instances, where loop variable is incorrectly used
within a test closure, which is marked as parallel.
Few of these were actually loops over singleton slices, therefore the issue
might not have surfaced there (yet), but it is good to fix there as
well, as this is an incorrect pattern used across different tests.

Signed-off-by: Roman Volosatovs <roman.volosatovs@docker.com>
(cherry picked from commit dd01abf9bf)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-10-20 19:01:16 -04:00
Cory Snider
9794702db9 daemon: handle EISDIR error from runtime
Go 1.20 made a change to the behaviour of package "os/exec" which was
not mentioned in the release notes:
2b8f214094
Attempts to execute a directory now return syscall.EISDIR instead of
syscall.EACCESS. Check for EISDIR errors from the runtime and fudge the
returned error message to maintain compatibility with existing versions
of docker/cli when using a version of runc compiled with Go 1.20+.

Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit 713e02e03e)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-10-20 18:56:37 -04:00
Cory Snider
0b2be686ea vendor: github.com/docker/libnetwork 3797618f9a38372e8107d8c06f6ae199e1133ae8
full diff: 3f0048413d...3797618f9a

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-10-20 17:57:51 -04:00
Cory Snider
d69ed28a75
Merge pull request #46674 from neersighted/backport/46667/20.10
[20.10 backport] Add IP_NF_MANGLE to check-config.sh
2023-10-19 13:08:33 -04:00
Stephan Henningsen
c18c3e27ac
Update check-config.sh
Add IP_NF_MANGLE to "Generally Required" kernel features, since it appears to be necessary for Docker Swarm to work.

Closes https://github.com/moby/moby/issues/46636

Signed-off-by: Stephan Henningsen <stephan-henningsen@users.noreply.github.com>
(cherry picked from commit cf9073397c)
Conflicts: contrib/check-config.sh
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-10-18 16:00:34 -06:00
Bjorn Neergaard
f0a196bf22
profiles/apparmor: deny /sys/devices/virtual/powercap
While this is not strictly necessary as the default OCI config masks this
path, it is possible that the user disabled path masking, passed their
own list, or is using a forked (or future) daemon version that has a
modified default config/allows changing the default config.

Add some defense-in-depth by also masking out this problematic hardware
device with the AppArmor LSM.

Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
(cherry picked from commit bddd826d7a)
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-09-18 16:46:13 -06:00
Bjorn Neergaard
4b242784ca
oci/defaults: deny /sys/devices/virtual/powercap
The ability to read these files may offer a power-based sidechannel
attack against any workloads running on the same kernel.

This was originally [CVE-2020-8694][1], which was fixed in
[949dd0104c496fa7c14991a23c03c62e44637e71][2] by restricting read access
to root. However, since many containers run as root, this is not
sufficient for our use case.

While untrusted code should ideally never be run, we can add some
defense in depth here by masking out the device class by default.

[Other mechanisms][3] to access this hardware exist, but they should not
be accessible to a container due to other safeguards in the
kernel/container stack (e.g. capabilities, perf paranoia).

[1]: https://nvd.nist.gov/vuln/detail/CVE-2020-8694
[2]: 949dd0104c
[3]: https://web.eece.maine.edu/~vweaver/projects/rapl/

Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
(cherry picked from commit 83cac3c3e3)
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-09-18 16:46:09 -06:00
Bjorn Neergaard
29a0e76e64
Merge pull request #46424 from ameyag/20.10-uname-backslash
[20.10 backport] Fixing dockerd-rootless-setuptools.sh when user name contains a backslash
2023-09-18 08:08:11 -06:00
Jean-Michel Rouet
59e1e14929
more robust dockerd-rootless-setuptools.sh
Fixing case where username may contain a backslash.
This case can happen for winbind/samba active directory domain users.

Signed-off-by: Jean-Michel Rouet <jean-michel.rouet@philips.com>

Use more meaningful variable name

Signed-off-by: Jean-Michel Rouet <jean-michel.rouet@philips.com>

Update contrib/dockerd-rootless-setuptool.sh

Co-authored-by: Akihiro Suda <suda.kyoto@gmail.com>
Signed-off-by: Jean-Michel Rouet <jean-michel.rouet@philips.com>

Use more meaningful variable name

Signed-off-by: Jean-Michel Rouet <jean-michel.rouet@philips.com>

Update contrib/dockerd-rootless-setuptool.sh

Co-authored-by: Akihiro Suda <suda.kyoto@gmail.com>
Signed-off-by: Jean-Michel Rouet <jean-michel.rouet@philips.com>
(cherry picked from commit 2f0ba0a7e5)
Signed-off-by: Ameya Gawde <agawde@mirantis.com>
2023-09-06 09:46:02 -07:00
Cory Snider
fdb9c710a7
Merge pull request #46217 from thaJeztah/20.10_backport_windows_fix_service_register
[20.10 backport] windows: fix --register-service when executed from within binary directory
2023-08-30 17:43:39 -04:00
Sebastiaan van Stijn
f3e6a0600d
windows: fix --register-service when executed from within binary directory
Go 1.15.7 contained a security fix for CVE-2021-3115, which allowed arbitrary
code to be executed at build time when using cgo on Windows.

This issue was not limited to the go command itself, and could also affect binaries
that use `os.Command`, `os.LookPath`, etc.

From the related blogpost (https://blog.golang.org/path-security):

> Are your own programs affected?
>
> If you use exec.LookPath or exec.Command in your own programs, you only need to
> be concerned if you (or your users) run your program in a directory with untrusted
> contents. If so, then a subprocess could be started using an executable from dot
> instead of from a system directory. (Again, using an executable from dot happens
> always on Windows and only with uncommon PATH settings on Unix.)
>
> If you are concerned, then we’ve published the more restricted variant of os/exec
> as golang.org/x/sys/execabs. You can use it in your program by simply replacing

At time of the go1.15 release, the Go team considered changing the behavior of
`os.LookPath()` and `exec.LookPath()` to be a breaking change, and made the
behavior "opt-in" by providing the `golang.org/x/sys/execabs` package as a
replacement.

However, for the go1.19 release, this changed, and the default behavior of
`os.LookPath()` and `exec.LookPath()` was changed. From the release notes:
https://go.dev/doc/go1.19#os-exec-path

> Command and LookPath no longer allow results from a PATH search to be found
> relative to the current directory. This removes a common source of security
> problems but may also break existing programs that depend on using, say,
> exec.Command("prog") to run a binary named prog (or, on Windows, prog.exe)
> in the current directory. See the os/exec package documentation for information
> about how best to update such programs.
>
> On Windows, Command and LookPath now respect the NoDefaultCurrentDirectoryInExePath
> environment variable, making it possible to disable the default implicit search
> of “.” in PATH lookups on Windows systems.

A result of this change was that registering the daemon as a Windows service
no longer worked when done from within the directory of the binary itself:

    C:\> cd "Program Files\Docker\Docker\resources"
    C:\Program Files\Docker\Docker\resources> dockerd --register-service
    exec: "dockerd": cannot run executable found relative to current directory

Note that using an absolute path would work around the issue:

    C:\Program Files\Docker\Docker>resources\dockerd.exe --register-service

This patch changes `registerService()` to use `os.Executable()`, instead of
depending on `os.Args[0]` and `exec.LookPath()` for resolving the absolute
path of the binary.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 3e8fda0a70)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-13 21:59:26 +02:00
Akihiro Suda
791d8ab877
Merge pull request #46142 from thaJeztah/20.10_update_go1.19.12
[20.10] update to go1.19.12
2023-08-02 14:50:16 +09:00
Sebastiaan van Stijn
c42d7c7f6d
update to go1.19.12
Includes a fix for CVE-2023-29409

go1.19.12 (released 2023-08-01) includes a security fix to the crypto/tls
package, as well as bug fixes to the assembler and the compiler. See the
Go 1.19.12 milestone on our issue tracker for details.

- https://github.com/golang/go/issues?q=milestone%3AGo1.19.12+label%3ACherryPickApproved
- full diff: https://github.com/golang/go/compare/go1.19.11...go1.19.12

From the mailing list announcement:

[security] Go 1.20.7 and Go 1.19.12 are released

Hello gophers,

We have just released Go versions 1.20.7 and 1.19.12, minor point releases.

These minor releases include 1 security fixes following the security policy:

- crypto/tls: restrict RSA keys in certificates to <= 8192 bits

  Extremely large RSA keys in certificate chains can cause a client/server
  to expend significant CPU time verifying signatures. Limit this by
  restricting the size of RSA keys transmitted during handshakes to <=
  8192 bits.

  Based on a survey of publicly trusted RSA keys, there are currently only
  three certificates in circulation with keys larger than this, and all
  three appear to be test certificates that are not actively deployed. It
  is possible there are larger keys in use in private PKIs, but we target
  the web PKI, so causing breakage here in the interests of increasing the
  default safety of users of crypto/tls seems reasonable.

  Thanks to Mateusz Poliwczak for reporting this issue.

View the release notes for more information:
https://go.dev/doc/devel/release#go1.20.7

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-01 23:51:38 +02:00
Akihiro Suda
0ad2952dc7
Merge pull request #46105 from thaJeztah/20.10_update_containerd_binary_1.6.22
[20.10] update containerd binary to v1.6.22
2023-07-30 11:06:12 +09:00
Sebastiaan van Stijn
b883976e4c
update containerd binary to v1.6.22
- full diff: https://github.com/containerd/containerd/compare/v1.6.21...v1.6.22
- release notes: https://github.com/containerd/containerd/releases/tag/v1.6.22

---

Notable Updates

- RunC: Update runc binary to v1.1.8
- CRI: Fix `additionalGids`: it should fallback to `imageConfig.User`
  when `securityContext.RunAsUser`, `RunAsUsername` are empty
- CRI: Write generated CNI config atomically
- Fix concurrent writes for `UpdateContainerStats`
- Make `checkContainerTimestamps` less strict on Windows
- Port-Forward: Correctly handle known errors
- Resolve `docker.NewResolver` race condition
- SecComp: Always allow `name_to_handle_at`
- Adding support to run hcsshim from local clone
- Pinned image support
- Runtime/V2/RunC: Handle early exits w/o big locks
- CRITool: Move up to CRI-TOOLS v1.27.0
- Fix cpu architecture detection issue on emulated ARM platform
- Task: Don't `close()` io before `cancel()`
- Fix panic when remote differ returns empty result
- Plugins: Notify readiness when registered plugins are ready
- Unwrap io errors in server connection receive error handling

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-28 13:32:30 +02:00
Sebastiaan van Stijn
65849a573d
Merge pull request #46047 from neersighted/init_cleanup_20.10
[20.10 backport] Upstart & sysvinit cleanup
2023-07-21 22:08:59 +02:00
Tianon Gravi
da9f0732a4
Remove Upstart scripts
Upstart has been EOL for 8 years and isn't used by any distributions we support any more.

Signed-off-by: Tianon Gravi <admwiggin@gmail.com>
(cherry picked from commit 0d8087fbbc)
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-07-21 12:01:26 -06:00
Tianon Gravi
2ef669288a
Remove Upstart and cgroups bits from Debian sysvinit script
Upstart has been EOL for 8 years and isn't used by any distributions we support any more.

Additionally, this removes the "cgroups v1" setup code because it's more reasonable now for us to expect something _else_ to have set up cgroups appropriately (especially cgroups v2).

Signed-off-by: Tianon Gravi <admwiggin@gmail.com>
(cherry picked from commit ae737656f9)
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-07-21 12:01:19 -06:00
Sebastiaan van Stijn
b3d554724c
Merge pull request #46031 from thaJeztah/20.10_backport_bump_runc_binary_1.1.8
[20.10 backport] Dockerfile: update runc binary to v1.1.8
2023-07-20 20:16:34 +02:00
Sebastiaan van Stijn
0539294652
Dockerfile: update runc binary to v1.1.8
release notes: https://github.com/opencontainers/runc/releases/tag/v1.1.8
full diff: https://github.com/opencontainers/runc/compare/v1.1.7...v1.1.9

This is the eighth patch release of the 1.1.z release branch of runc.
The most notable change is the addition of RISC-V support, along with a
few bug fixes.

- Support riscv64.
- init: do not print environment variable value.
- libct: fix a race with systemd removal.
- tests/int: increase num retries for oom tests.
- man/runc: fixes.
- Fix tmpfs mode opts when dir already exists.
- docs/systemd: fix a broken link.
- ci/cirrus: enable some rootless tests on cs9.
- runc delete: call systemd's reset-failed.
- libct/cg/sd/v1: do not update non-frozen cgroup after frozen failed.
- CI: bump Fedora, Vagrant, bats.
- .codespellrc: update for 2.2.5.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit df86d855f5)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-19 18:27:28 +02:00
Sebastiaan van Stijn
334576ebbd
Merge pull request #46003 from thaJeztah/20.10_backport_notestyourself
[20.10 backport] quota: remove gotest.tools from testhelpers
2023-07-18 13:00:53 +02:00
Sebastiaan van Stijn
7ed8ff084f
quota: remove gotest.tools from testhelpers
gotest.tools has an init() which registers a '-update' flag;
a80f057529/internal/source/update.go (L21-L23)

The quota helper contains a testhelpers file, which is meant for usage
in (integration) tests, but as it's in the same pacakge as production
code, would also trigger the gotest.tools init.

This patch removes the gotest.tools code from this file.

Before this patch:

    $ (exec -a libnetwork-setkey "$(which dockerd)" -help)
    Usage of libnetwork-setkey:
      -exec-root string
            docker exec root (default "/run/docker")
      -update
            update golden values

With this patch applied:

    $ (exec -a libnetwork-setkey "$(which dockerd)" -help)
    Usage of libnetwork-setkey:
      -exec-root string
            docker exec root (default "/run/docker")

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 1aa17222e7)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-17 23:13:58 +02:00
Sebastiaan van Stijn
dc00abf07a
Merge pull request #45995 from thaJeztah/20.10_update_go_1.19.11
[20.10] update go to go1.19.11
2023-07-17 22:23:54 +02:00
Sebastiaan van Stijn
43fe787c23
[20.10] update go to go1.19.11
go1.19.11 (released 2023-07-11) includes a security fix to the net/http package,
as well as bug fixes to cgo, the cover tool, the go command, the runtime, and
the go/printer package. See the Go 1.19.11 milestone on our issue tracker for
details:

https://github.com/golang/go/issues?q=milestone%3AGo1.19.11+label%3ACherryPickApproved

Full diff: https://github.com/golang/go/compare/go1.19.10...go1.19.11

These minor releases include 1 security fixes following the security policy:

net/http: insufficient sanitization of Host header

The HTTP/1 client did not fully validate the contents of the Host header.
A maliciously crafted Host header could inject additional headers or entire
requests. The HTTP/1 client now refuses to send requests containing an
invalid Request.Host or Request.URL.Host value.

Thanks to Bartek Nowotarski for reporting this issue.

Includes security fixes for [CVE-2023-29406 ][1] and Go issue https://go.dev/issue/60374

[1]: https://github.com/advisories/GHSA-f8f7-69v5-w4vx

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-17 16:21:03 +02:00
Sebastiaan van Stijn
0d4b9ed98b
Merge pull request #45980 from thaJeztah/20.10_backport_vendor_distribution_v2.8.2
[20.10 backport] vendor: github.com/docker/distribution v2.8.2
2023-07-17 11:59:28 +02:00
Sebastiaan van Stijn
1bd0805099
vendor: github.com/docker/distribution v2.8.2
CI

- Dockerfile: fix filenames of artifacts

Bugfixes

-  Fix panic in inmemory driver
-  Add code to handle pagination of parts. Fixes max layer size of 10GB bug
-  Parse http forbidden as denied
-  Revert "registry/client: set Accept: identity header when getting layers

Runtime

- Update to go1.19.9
- Dockerfile: update xx to v1.2.1 ([#3907](https://github.com/distribution/distribution/pull/3907))

Security

- Fix [CVE-2022-28391](https://www.cve.org/CVERecord?id=CVE-2022-28391) by bumping alpine from 3.14 to 3.16
- Fix [CVE-2023-2253](https://www.cve.org/CVERecord?id=CVE-2023-2253) runaway allocation on /v2/_catalog [`521ea3d9`](521ea3d973)

full diff: https://github.com/docker/distribution/compare/v2.8.1...v2.8.2

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

bump to release/2.8

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 7821d2d788)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-15 19:20:47 +02:00
Sebastiaan van Stijn
f7a770ac61
Merge pull request #45972 from thaJeztah/20.10_backport_fix_host_header
[20.10 backport] client: define a "dummy" hostname to use for local connections
2023-07-15 02:57:34 +02:00
Sebastiaan van Stijn
e925a5d304
testutil: use dummyhost for non-tcp connections
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit e1db9e9848)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-15 01:27:29 +02:00
Sebastiaan van Stijn
1603196c5b
pkg/plugins: use a dummy hostname for local connections
For local communications (npipe://, unix://), the hostname is not used,
but we need valid and meaningful hostname.

The current code used the socket path as hostname, which gets rejected by
go1.20.6 and go1.19.11 because of a security fix for [CVE-2023-29406 ][1],
which was implemented in  https://go.dev/issue/60374.

Prior versions go Go would clean the host header, and strip slashes in the
process, but go1.20.6 and go1.19.11 no longer do, and reject the host
header.

Before this patch, tests would fail on go1.20.6:

    === FAIL: pkg/authorization TestAuthZRequestPlugin (15.01s)
    time="2023-07-12T12:53:45Z" level=warning msg="Unable to connect to plugin: //tmp/authz2422457390/authz-test-plugin.sock/AuthZPlugin.AuthZReq: Post \"http://%2F%2Ftmp%2Fauthz2422457390%2Fauthz-test-plugin.sock/AuthZPlugin.AuthZReq\": http: invalid Host header, retrying in 1s"
    time="2023-07-12T12:53:46Z" level=warning msg="Unable to connect to plugin: //tmp/authz2422457390/authz-test-plugin.sock/AuthZPlugin.AuthZReq: Post \"http://%2F%2Ftmp%2Fauthz2422457390%2Fauthz-test-plugin.sock/AuthZPlugin.AuthZReq\": http: invalid Host header, retrying in 2s"
    time="2023-07-12T12:53:48Z" level=warning msg="Unable to connect to plugin: //tmp/authz2422457390/authz-test-plugin.sock/AuthZPlugin.AuthZReq: Post \"http://%2F%2Ftmp%2Fauthz2422457390%2Fauthz-test-plugin.sock/AuthZPlugin.AuthZReq\": http: invalid Host header, retrying in 4s"
    time="2023-07-12T12:53:52Z" level=warning msg="Unable to connect to plugin: //tmp/authz2422457390/authz-test-plugin.sock/AuthZPlugin.AuthZReq: Post \"http://%2F%2Ftmp%2Fauthz2422457390%2Fauthz-test-plugin.sock/AuthZPlugin.AuthZReq\": http: invalid Host header, retrying in 8s"
        authz_unix_test.go:82: Failed to authorize request Post "http://%2F%2Ftmp%2Fauthz2422457390%2Fauthz-test-plugin.sock/AuthZPlugin.AuthZReq": http: invalid Host header

[1]: https://github.com/advisories/GHSA-f8f7-69v5-w4vx

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 6b7705d5b2)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-15 01:27:29 +02:00
Sebastiaan van Stijn
74b4974cb7
client: define a "dummy" hostname to use for local connections
For local communications (npipe://, unix://), the hostname is not used,
but we need valid and meaningful hostname.

The current code used the client's `addr` as hostname in some cases, which
could contain the path for the unix-socket (`/var/run/docker.sock`), which
gets rejected by go1.20.6 and go1.19.11 because of a security fix for
[CVE-2023-29406 ][1], which was implemented in  https://go.dev/issue/60374.

Prior versions go Go would clean the host header, and strip slashes in the
process, but go1.20.6 and go1.19.11 no longer do, and reject the host
header.

This patch introduces a `DummyHost` const, and uses this dummy host for
cases where we don't need an actual hostname.

Before this patch (using go1.20.6):

    make GO_VERSION=1.20.6 TEST_FILTER=TestAttach test-integration
    === RUN   TestAttachWithTTY
        attach_test.go:46: assertion failed: error is not nil: http: invalid Host header
    --- FAIL: TestAttachWithTTY (0.11s)
    === RUN   TestAttachWithoutTTy
        attach_test.go:46: assertion failed: error is not nil: http: invalid Host header
    --- FAIL: TestAttachWithoutTTy (0.02s)
    FAIL

With this patch applied:

    make GO_VERSION=1.20.6 TEST_FILTER=TestAttach test-integration
    INFO: Testing against a local daemon
    === RUN   TestAttachWithTTY
    --- PASS: TestAttachWithTTY (0.12s)
    === RUN   TestAttachWithoutTTy
    --- PASS: TestAttachWithoutTTy (0.02s)
    PASS

[1]: https://github.com/advisories/GHSA-f8f7-69v5-w4vx

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 92975f0c11)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-15 01:27:29 +02:00
Sebastiaan van Stijn
cccf90f1d8
client: TestSetHostHeader: don't use un-keyed literals
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 2a59188760)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-15 01:27:29 +02:00
Sebastiaan van Stijn
c0e3253214
Merge pull request #45973 from thaJeztah/20.10_fix_build
[20.10] integration/daemon: fix missing import
2023-07-15 01:26:54 +02:00
Sebastiaan van Stijn
763d117b4f
[20.10] integration/daemon: fix missing import
commit 44152f6fb6 backported a change
that added `os.TempDir()` to a test, but that import was not yet
in this file in the 20.10 branch.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-14 23:53:24 +02:00
Sebastiaan van Stijn
53e302dc7d
Merge pull request #45911 from corhere/backport-20.10/improve-test-flakiness
[20.10 backport] make tests less flaky
2023-07-08 17:59:23 +02:00
Brian Goff
b49bb5dcf8
Merge pull request #45913 from corhere/backport-20.10/fix-volume-npe 2023-07-07 16:30:18 -07:00
Sebastiaan van Stijn
44152f6fb6 daemon: daemon.prepareMountPoints(): fix panic if mount is not a volume
The daemon.lazyInitializeVolume() function only handles restoring Volumes
if a Driver is specified. The Container's MountPoints field may also
contain other kind of mounts (e.g., bind-mounts). Those were ignored, and
don't return an error; 1d9c8619cd/daemon/volumes.go (L243-L252C2)

However, the prepareMountPoints() assumed each MountPoint was a volume,
and logged an informational message about the volume being restored;
1d9c8619cd/daemon/mounts.go (L18-L25)

This would panic if the MountPoint was not a volume;

    github.com/docker/docker/daemon.(*Daemon).prepareMountPoints(0xc00054b7b8?, 0xc0007c2500)
            /root/rpmbuild/BUILD/src/engine/.gopath/src/github.com/docker/docker/daemon/mounts.go:24 +0x1c0
    github.com/docker/docker/daemon.(*Daemon).restore.func5(0xc0007c2500, 0x0?)
            /root/rpmbuild/BUILD/src/engine/.gopath/src/github.com/docker/docker/daemon/daemon.go:552 +0x271
    created by github.com/docker/docker/daemon.(*Daemon).restore
            /root/rpmbuild/BUILD/src/engine/.gopath/src/github.com/docker/docker/daemon/daemon.go:530 +0x8d8
    panic: runtime error: invalid memory address or nil pointer dereference
    [signal SIGSEGV: segmentation violation code=0x1 addr=0x30 pc=0x564e9be4c7c0]

This issue was introduced in 647c2a6cdd

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit a490248f4d)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-07-07 16:02:17 -04:00
Cory Snider
492802acec integration: disable iptables in parallel tests
Multiple daemons starting/running concurrently can collide with each
other when editing iptables rules. Most integration tests which opt into
parallelism and start daemons work around this problem by starting the
daemon with the --iptables=false option. However, some of the tests
neglect to pass the option when starting or restarting the daemon,
resulting in those tests being flaky.

Audit the integration tests which call t.Parallel() and (*Daemon).Stop()
and add --iptables=false arguments where needed.

Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit cdcb7c28c5)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-07-07 15:54:00 -04:00
Cory Snider
30e297f664 pkg/plugins: make unit test less time sensitive
TestClientWithRequestTimeout has been observed to flake in CI. The
timing in the test is quite tight, only giving the client a 10ms window
to time out, which could potentially be missed if the host is under
load and the goroutine scheduling is unlucky. Give the client a full
five seconds of grace to time out before failing the test.

Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit 9cee34bc94)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-07-07 15:53:17 -04:00
Sebastiaan van Stijn
bbae7f4319
Merge pull request #45840 from cpuguy83/20.10_fix_live_restore_local_vol_mounts
[20.10] Backport Restore active mount counts on live-restore
2023-06-29 01:04:06 +02:00
Brian Goff
b5fe60d3db [20.10] Backport Restore active mount counts on live-restore
Backporting commit 647c2a6cdd for 20.10

When live-restoring a container the volume driver needs be notified that
there is an active mount for the volume.
Before this change the count is zero until the container stops and the
uint64 overflows pretty much making it so the volume can never be
removed until another daemon restart.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2023-06-28 18:48:30 +00:00
Bjorn Neergaard
a3debeefd7
Merge pull request #45835 from neersighted/backport/45766/20.10
[20.10 backport] seccomp: always allow name_to_handle_at(2)
2023-06-28 10:36:49 -06:00
Bjorn Neergaard
a480b37621
seccomp: add name_to_handle_at to allowlist
Based on the analysis on [the previous PR][1].

  [1]: https://github.com/moby/moby/pull/45766#pullrequestreview-1493908145

Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
(cherry picked from commit b335e3d305)
Resolved conflicts:
	profiles/seccomp/default_linux.go
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-06-28 05:48:28 -06:00
Vitor Anjos
45a8248070
remove name_to_handle_at(2) from filtered syscalls
Signed-off-by: Vitor Anjos <bartier@users.noreply.github.com>
(cherry picked from commit fdc9b7cceb)
Resolved conflicts:
	profiles/seccomp/default_linux.go
Co-Authored-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-06-27 13:22:05 -06:00
Sebastiaan van Stijn
cf806e4436
Merge pull request #45773 from thaJeztah/20.10_backport_fix_docker_py
[20.10 backport] testing: temporarily pin docker-py tests to use "bullseye"
2023-06-20 23:43:01 +02:00
Sebastiaan van Stijn
6ee44341b0
testing: temporarily pin docker-py tests to use "bullseye"
The official Python images on Docker Hub switched to debian bookworm,
which is now the current stable version of Debian.

However, the location of the apt repository config file changed, which
causes the Dockerfile build to fail;

    Loaded image: emptyfs:latest
    Loaded image ID: sha256:0df1207206e5288f4a989a2f13d1f5b3c4e70467702c1d5d21dfc9f002b7bd43
    INFO: Building docker-sdk-python3:5.0.3...
    tests/Dockerfile:6
    --------------------
       5 |     ARG APT_MIRROR
       6 | >>> RUN sed -ri "s/(httpredir|deb).debian.org/${APT_MIRROR:-deb.debian.org}/g" /etc/apt/sources.list \
       7 | >>>     && sed -ri "s/(security).debian.org/${APT_MIRROR:-security.debian.org}/g" /etc/apt/sources.list
       8 |
    --------------------
    ERROR: failed to solve: process "/bin/sh -c sed -ri \"s/(httpredir|deb).debian.org/${APT_MIRROR:-deb.debian.org}/g\" /etc/apt/sources.list     && sed -ri \"s/(security).debian.org/${APT_MIRROR:-security.debian.org}/g\" /etc/apt/sources.list" did not complete successfully: exit code: 2

This needs to be fixed in docker-py, but in the meantime, we can pin to
the bullseye variant.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 19d860fa9d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-20 11:42:21 +02:00
Bjorn Neergaard
2ae14baf02
Merge pull request #45750 from thaJeztah/20.10_update_go1.19.10
[20.10] update go to go1.19.10
2023-06-14 15:07:32 -06:00
Sebastiaan van Stijn
99f10dec91
[20.10] update go to go1.19.10
go1.19.10 (released 2023-06-06) includes four security fixes to the cmd/go and
runtime packages, as well as bug fixes to the compiler, the go command, and the
runtime. See the Go 1.19.10 milestone on our issue tracker for details:

https://github.com/golang/go/issues?q=milestone%3AGo1.19.10+label%3ACherryPickApproved

full diff: https://github.com/golang/go/compare/go1.19.9...go1.19.10

These minor releases include 3 security fixes following the security policy:

- cmd/go: cgo code injection
  The go command may generate unexpected code at build time when using cgo. This
  may result in unexpected behavior when running a go program which uses cgo.

  This may occur when running an untrusted module which contains directories with
  newline characters in their names. Modules which are retrieved using the go command,
  i.e. via "go get", are not affected (modules retrieved using GOPATH-mode, i.e.
  GO111MODULE=off, may be affected).

  Thanks to Juho Nurminen of Mattermost for reporting this issue.

  This is CVE-2023-29402 and Go issue https://go.dev/issue/60167.

- runtime: unexpected behavior of setuid/setgid binaries

  The Go runtime didn't act any differently when a binary had the setuid/setgid
  bit set. On Unix platforms, if a setuid/setgid binary was executed with standard
  I/O file descriptors closed, opening any files could result in unexpected
  content being read/written with elevated prilieges. Similarly if a setuid/setgid
  program was terminated, either via panic or signal, it could leak the contents
  of its registers.

  Thanks to Vincent Dehors from Synacktiv for reporting this issue.

  This is CVE-2023-29403 and Go issue https://go.dev/issue/60272.

- cmd/go: improper sanitization of LDFLAGS

  The go command may execute arbitrary code at build time when using cgo. This may
  occur when running "go get" on a malicious module, or when running any other
  command which builds untrusted code. This is can by triggered by linker flags,
  specified via a "#cgo LDFLAGS" directive.

  Thanks to Juho Nurminen of Mattermost for reporting this issue.

  This is CVE-2023-29404 and CVE-2023-29405 and Go issues https://go.dev/issue/60305 and https://go.dev/issue/60306.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-14 13:19:22 +02:00
Cory Snider
168fae3264
Merge pull request #45717 from ameyag/20.10-grpc-fix
[20.10 backport] grpc: make sure typed errors handler is installed
2023-06-09 15:07:05 -04:00
Cory Snider
1e57f01055
Merge pull request #45716 from ameyag/20.10-apparmor-template
[20.10 backport] Fix AppArmor profile docker-default /proc/sys rule
2023-06-09 15:05:45 -04:00
Bjorn Neergaard
c78a918e43
Merge pull request #45710 from neersighted/check_config_bpf/20.10
[20.10 backport] contrib/check-config: require xt_bpf for encrypted overlay
2023-06-09 07:06:06 -06:00
Sebastiaan van Stijn
50a03fdc6a
api/server/router/grpc: fix some nits in NewRouter()
These were changes I drafted when reviewing 7c731e02a9,
and had these stashed in my local git;

- rename receiver to prevent "unconsistent receiver name" warnings
- make NewRouter() slightly more idiomatic, and wrap the options,
  to make them easier to read.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 758714ed6d)
Signed-off-by: Ameya Gawde <agawde@mirantis.com>
2023-06-08 11:58:20 -07:00
Tonis Tiigi
a222dbfd5c
grpc: make sure typed errors handler is installed
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit 7c731e02a9)
Signed-off-by: Ameya Gawde <agawde@mirantis.com>
2023-06-08 11:57:53 -07:00
Phil Sphicas
b1c04fc6cd
Fix AppArmor profile docker-default /proc/sys rule
The current docker-default AppArmor profile intends to block write
access to everything in `/proc`, except for `/proc/<pid>` and
`/proc/sys/kernel/shm*`.

Currently the rules block access to everything in `/proc/sys`, and do
not successfully allow access to `/proc/sys/kernel/shm*`. Specifically,
a path like /proc/sys/kernel/shmmax matches this part of the pattern:

    deny @{PROC}/{[^1-9][^0-9][^0-9][^0-9]*     }/** w,
         /proc  / s     y     s     /     kernel /shmmax

This patch updates the rule so that it works as intended.

Closes #39791

Signed-off-by: Phil Sphicas <phil.sphicas@att.com>
(cherry picked from commit 66f14e4ae9)
Signed-off-by: Ameya Gawde <agawde@mirantis.com>
2023-06-08 10:23:22 -07:00
Bjorn Neergaard
a388019ce8
contrib/check-config: move xt_bpf check to overlay section
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
(cherry picked from commit 800ea039ec)
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-06-06 16:53:50 -06:00
Bjorn Neergaard
d5515dc7e9
contrib/check-config: check for xt_bpf
We omit xt_u32 as it's optional; since we will remove support for this
module in the future, it's simpler to check for xt_bpf, which will
become the new baseline.

Related issues:
* https://github.com/microsoft/WSL/issues/10029#issuecomment-1574440255
* https://github.com/docker/for-win/issues/13450#issuecomment-1574443139

Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
(cherry picked from commit 1910fdde81)
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-06-06 16:53:14 -06:00
Bjorn Neergaard
73854c21b6
Merge pull request #45615 from thaJeztah/20.10_backport_skip_criu
[20.10 backport] Dockerfile: temporarily skip CRIU stage
2023-05-25 13:27:13 -06:00
Sebastiaan van Stijn
af0477880c
Dockerfile: temporarily skip CRIU stage
The package repository currently has issues;

    => ERROR https://download.opensuse.org/repositories/devel:/tools:/criu/Debian_11/Release.key

The only test currently using this binary is currently skipped, as the test
was broken;
6e98a7f2c9/integration/container/checkpoint_test.go (L32-L33)

So let's disable this stage for the time being.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit d3d2823edf)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-25 12:23:03 +02:00
Sebastiaan van Stijn
0b2c7ae6cb
Merge pull request #45606 from neersighted/backport/45603/20.10
[20.10 backport] hack/make/.binary: don't use "netgo" when building Windows binaries
2023-05-25 01:49:54 +02:00
Sebastiaan van Stijn
3d5ebbecda
hack/make/.binary: don't use "netgo" when building Windows binaries
Starting with go1.19, the Go runtime on Windows now supports the `netgo` build-
flag to use a native Go DNS resolver. Prior to that version, the build-flag
only had an effect on non-Windows platforms. When using the `netgo` build-flag,
the Windows's host resolver is not used, and as a result, custom entries in
`etc/hosts` are ignored, which is a change in behavior from binaries compiled
with older versions of the Go runtime.

From the go1.19 release notes: https://go.dev/doc/go1.19#net

> Resolver.PreferGo is now implemented on Windows and Plan 9. It previously
> only worked on Unix platforms. Combined with Dialer.Resolver and Resolver.Dial,
> it's now possible to write portable programs and be in control of all DNS name
> lookups when dialing.
>
> The net package now has initial support for the netgo build tag on Windows.
> When used, the package uses the Go DNS client (as used by Resolver.PreferGo)
> instead of asking Windows for DNS results. The upstream DNS server it discovers
> from Windows may not yet be correct with complex system network configurations,
> however.

Our Windows binaries are compiled with the "static" (`make/binary-daemon`)
script, which has the `netgo` option set by default. This patch unsets the
`netgo` option when cross-compiling for Windows.

Co-authored-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
(cherry picked from commit 53d1b12bc0)
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-05-24 11:01:55 -06:00
Akihiro Suda
0f8ffc4b89
Merge pull request #45406 from thaJeztah/20.10_backport_runc_binary_1.1.6
[20.10 backport] update runc binary to v1.1.7
2023-05-16 11:37:10 +09:00
Akihiro Suda
562e795bf2
Merge pull request #45479 from thaJeztah/20.10_containerd_binary_1.6.21
[20.10] update containerd binary to v1.6.21
2023-05-16 11:36:58 +09:00
Sebastiaan van Stijn
66ce8ec8e0
Merge pull request #45411 from thaJeztah/20.10_backport_jenkinsfile_ubuntu_2004
[20.10 backport] Jenkinsfile: use Ubuntu 20.04
2023-05-12 23:26:33 +01:00
Sebastiaan van Stijn
9b6ec6dbba
update containerd binary to v1.6.21
release notes: https://github.com/containerd/containerd/releases/tag/v1.6.21

Notable Updates

- update runc binary to v1.1.7
- Remove entry for container from container store on error
- oci: partially restore comment on read-only mounts for uid/gid uses
- windows: Add ArgsEscaped support for CRI
- oci: Use WithReadonlyTempMount when adding users/groups
- archive: consistently respect value of WithSkipDockerManifest

full diff: https://github.com/containerd/containerd/compare/c0efc63d3907...v1.6.21

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit edadebe177)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-05 21:04:03 +02:00
Sebastiaan van Stijn
db9ab6438d
update runc binary to v1.1.7
release notes: https://github.com/opencontainers/runc/releases/tag/v1.1.7
full diff: https://github.com/opencontainers/runc/compare/v1.1.6...v1.1.7

This is the seventh patch release in the 1.1.z release of runc, and is
the last planned release of the 1.1.z series. It contains a fix for
cgroup device rules with systemd when handling device rules for devices
that don't exist (though for devices whose drivers don't correctly
register themselves in the kernel -- such as the NVIDIA devices -- the
full fix only works with systemd v240+).

- When used with systemd v240+, systemd cgroup drivers no longer skip
  DeviceAllow rules if the device does not exist (a regression introduced
  in runc 1.1.3). This fix also reverts the workaround added in runc 1.1.5,
  removing an extra warning emitted by runc run/start.
- The source code now has a new file, runc.keyring, which contains the keys
  used to sign runc releases.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 2d0e899819)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-05 21:00:05 +02:00
Sebastiaan van Stijn
b73d5f066d
update runc binary to v1.1.6
release notes: https://github.com/opencontainers/runc/releases/tag/v1.1.6
full diff: https://github.com/opencontainers/runc/compare/v1.1.5...v1.1.6

This is the sixth patch release in the 1.1.z series of runc, which fixes
a series of cgroup-related issues.

Note that this release can no longer be built from sources using Go
1.16. Using a latest maintained Go 1.20.x or Go 1.19.x release is
recommended. Go 1.17 can still be used.

- systemd cgroup v1 and v2 drivers were deliberately ignoring UnitExist error
  from systemd while trying to create a systemd unit, which in some scenarios
  may result in a container not being added to the proper systemd unit and
  cgroup.
- systemd cgroup v2 driver was incorrectly translating cpuset range from spec's
  resources.cpu.cpus to systemd unit property (AllowedCPUs) in case of more
  than 8 CPUs, resulting in the wrong AllowedCPUs setting.
- systemd cgroup v1 driver was prefixing container's cgroup path with the path
  of PID 1 cgroup, resulting in inability to place PID 1 in a non-root cgroup.
- runc run/start may return "permission denied" error when starting a rootless
  container when the file to be executed does not have executable bit set for
  the user, not taking the CAP_DAC_OVERRIDE capability into account. This is
  a regression in runc 1.1.4, as well as in Go 1.20 and 1.20.1
- cgroup v1 drivers are now aware of misc controller.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit d0efca893b)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-05 20:59:10 +02:00
Sebastiaan van Stijn
db2f293976
[20.10] Jenkinsfile: update all stages to use ubuntu 20.04
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-05 01:18:42 +02:00
Sebastiaan van Stijn
575c0faa50
Jenkinsfile: use Ubuntu 20.04 for DCO stage
Also switching to use arm64, as all amd64 stages have moved to GitHub actions,
so using arm64 allows the same machine to be used for tests after the DCO check
completed.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 419c47a80a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-05 01:18:41 +02:00
Sebastiaan van Stijn
3275e2a936
Merge pull request #45466 from corhere/backport-20.10/copy-xattr-enotsupp
[20.10 backport] d/graphdriver/copy: support src fs w/o xattr support
2023-05-05 01:13:00 +02:00
Cory Snider
14160332da d/graphdriver/copy: support src fs w/o xattr support
Treat copying extended attributes from a source filesystem which does
not support extended attributes as a no-op, same as if the file did not
possess the extended attribute. Only fail copying extended attributes if
the source file has the attribute and the destination filesystem does
not support xattrs.

Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit 2b6761fd3e)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-05-04 15:28:49 -04:00
Sebastiaan van Stijn
5df983c7db
Merge pull request #45457 from thaJeztah/20.10_update_go_1.19.9
[20.10] update go to go1.19.9
2023-05-04 15:33:05 +02:00
Sebastiaan van Stijn
7f91a52b89
[20.10] update go to go1.19.9
go1.19.9 (released 2023-05-02) includes three security fixes to the html/template
package, as well as bug fixes to the compiler, the runtime, and the crypto/tls
and syscall packages. See the Go 1.19.9 milestone on our issue tracker for details.

https://github.com/golang/go/issues?q=milestone%3AGo1.19.9+label%3ACherryPickApproved

release notes: https://go.dev/doc/devel/release#go1.19.9
full diff: https://github.com/golang/go/compare/go1.19.8...go1.19.9

from the announcement:

> These minor releases include 3 security fixes following the security policy:
>
>- html/template: improper sanitization of CSS values
>
>   Angle brackets (`<>`) were not considered dangerous characters when inserted
>   into CSS contexts. Templates containing multiple actions separated by a '/'
>   character could result in unexpectedly closing the CSS context and allowing
>   for injection of unexpected HMTL, if executed with untrusted input.
>
>   Thanks to Juho Nurminen of Mattermost for reporting this issue.
>
>   This is CVE-2023-24539 and Go issue https://go.dev/issue/59720.
>
> - html/template: improper handling of JavaScript whitespace
>
>   Not all valid JavaScript whitespace characters were considered to be
>   whitespace. Templates containing whitespace characters outside of the character
>   set "\t\n\f\r\u0020\u2028\u2029" in JavaScript contexts that also contain
>   actions may not be properly sanitized during execution.
>
>   Thanks to Juho Nurminen of Mattermost for reporting this issue.
>
>   This is CVE-2023-24540 and Go issue https://go.dev/issue/59721.
>
> - html/template: improper handling of empty HTML attributes
>
>   Templates containing actions in unquoted HTML attributes (e.g. "attr={{.}}")
>   executed with empty input could result in output that would have unexpected
>   results when parsed due to HTML normalization rules. This may allow injection
>   of arbitrary attributes into tags.
>
>   Thanks to Juho Nurminen of Mattermost for reporting this issue.
>
>   This is CVE-2023-29400 and Go issue https://go.dev/issue/59722.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-03 21:06:41 +02:00
Sebastiaan van Stijn
0fdb994c8e
Merge pull request #45424 from thaJeztah/20.10_update_containerd_1.6.20
[20.10] update containerd binary to v1.6.20
2023-04-28 02:33:45 +02:00
Sebastiaan van Stijn
4fc8f8b4d8
update containerd binary to v1.6.20
Notable Updates

- Disable looking up usernames and groupnames on host
- Add support for Windows ArgsEscaped images
- Update hcsshim to v0.9.8
- Fix debug flag in shim
- Add WithReadonlyTempMount to support readonly temporary mounts
- Update ttrpc to fix file descriptor leak
- Update runc binary to v1.1.5
= Update image config to support ArgsEscaped

full diff: https://github.com/containerd/containerd/compare/v1.6.19...v1.6.20

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 389e18081d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-28 00:57:43 +02:00
Sebastiaan van Stijn
bb3ba157db
[20.10] update containerd binary to v1.6.19
Update hcsshim to v0.9.7 to include fix for graceful termination and pause containers

- release notes: https://github.com/containerd/containerd/releases/tag/v1.6.19
- full diff: https://github.com/containerd/containerd/compare/v1.6.18...v1.6.19

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 3125aa0aef)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-28 00:56:46 +02:00
Sebastiaan van Stijn
1176e7f90d
[20.10] update containerd binary to v1.6.18
update containerd binary to v1.6.18, which includes fixes for CVE-2023-25153
and CVE-2023-25173.

1.6.18 release notes: https://github.com/containerd/containerd/releases/tag/v1.6.18

> - Fix OCI image importer memory exhaustion (GHSA-259w-8hf6-59c2)
> - Fix supplementary groups not being set up properly (GHSA-hmfx-3pcx-653p)
> - Revert removal of /sbin/apparmor_parser check
> - Update Go to 1.19.6

full diff: https://github.com/containerd/containerd/compare/v1.6.17...v1.6.18

1.6.17 release notes: https://github.com/containerd/containerd/releases/tag/v1.6.17

> - Add network plugin metrics
> - Update mkdir permission on /etc/cni to 0755 instead of 0700
> - Export remote snapshotter label handler
> - Add support for default hosts.toml configuration

full diff: https://github.com/containerd/containerd/compare/v1.6.16...v1.6.17

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 52d667794f)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-28 00:55:43 +02:00
Tianon Gravi
206d38a9c1
Merge pull request #45238 from thaJeztah/20.10_backport_update_runc_binary_1.1.5
[20.10 backport] update runc binary to v1.1.5
2023-04-27 22:43:36 +00:00
Cory Snider
19b062885b
Merge pull request #45398 from corhere/vendor-libnetwork-for-20.10.25
[20.10] vendor: github.com/docker/libnetwork 3f0048413d95802b9c6c836eba06bfc54f9dbd03
2023-04-25 20:07:52 +02:00
Cory Snider
6ff576e8ca vendor: github.com/docker/libnetwork 3f0048413d95802b9c6c836eba06bfc54f9dbd03
full diff: 05b93e0d3a...3f0048413d

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-04-25 12:16:45 -04:00
Sebastiaan van Stijn
7a778373c8
update runc binary to v1.1.5
release notes: https://github.com/opencontainers/runc/releases/tag/v1.1.5
diff: https://github.com/opencontainers/runc/compare/v1.1.4...v1.1.5

This is the fifth patch release in the 1.1.z series of runc, which fixes
three CVEs found in runc.

* CVE-2023-25809 is a vulnerability involving rootless containers where
  (under specific configurations), the container would have write access
  to the /sys/fs/cgroup/user.slice/... cgroup hierarchy. No other
  hierarchies on the host were affected. This vulnerability was
  discovered by Akihiro Suda.
  <https://github.com/opencontainers/runc/security/advisories/GHSA-m8cg-xc2p-r3fc>

* CVE-2023-27561 was a regression which effectively re-introduced
  CVE-2019-19921. This bug was present from v1.0.0-rc95 to v1.1.4. This
  regression was discovered by @Beuc.
  <https://github.com/advisories/GHSA-vpvm-3wq2-2wvm>

* CVE-2023-28642 is a variant of CVE-2023-27561 and was fixed by the same
  patch. This variant of the above vulnerability was reported by Lei
  Wang.
  <https://github.com/opencontainers/runc/security/advisories/GHSA-g2j6-57v7-gm8c>

In addition, the following other fixes are included in this release:

* Fix the inability to use `/dev/null` when inside a container.
* Fix changing the ownership of host's `/dev/null` caused by fd redirection
  (a regression in 1.1.1).
* Fix rare runc exec/enter unshare error on older kernels, including
  CentOS < 7.7.
* nsexec: Check for errors in `write_log()`.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 77be7b777c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-21 12:01:47 +02:00
Bjorn Neergaard
ac1cd0423f
Merge pull request #45349 from PettitWesley/awslogs-non-blocking-bug-20.10
[20.10 backport] awslogs: fix non-blocking log drop bug
2023-04-20 20:46:46 +02:00
Wesley Pettit
ad45ece6fe
awslogs: fix non-blocking log drop bug
Previously, the AWSLogs driver attempted to implement
non-blocking itself. Non-blocking is supposed to
implemented solely by the Docker RingBuffer that
wraps the log driver.

Please see issue and explanation here:
https://github.com/moby/moby/issues/45217

Signed-off-by: Wesley Pettit <wppttt@amazon.com>
(cherry picked from commit c8f8d11ac4)
2023-04-17 13:32:48 -07:00
Akihiro Suda
de57aecf4a
Merge pull request #45276 from thaJeztah/20.10_bump_go1.19.8
[20.10] update go to go1.19.8
2023-04-06 10:25:58 +09:00
Sebastiaan van Stijn
a09b3e9cf9
[20.10] update go to go1.19.8
go1.19.8 (released 2023-04-04) includes security fixes to the go/parser,
html/template, mime/multipart, net/http, and net/textproto packages, as well as
bug fixes to the linker, the runtime, and the time package. See the Go 1.19.8
milestone on our issue tracker for details:

https://github.com/golang/go/issues?q=milestone%3AGo1.19.8+label%3ACherryPickApproved

full diff: https://github.com/golang/go/compare/go1.19.7...go1.19.8

Further details from the announcement on the mailing list:

We have just released Go versions 1.20.3 and 1.19.8, minor point releases.
These minor releases include 4 security fixes following the security policy:

- go/parser: infinite loop in parsing

  Calling any of the Parse functions on Go source code which contains `//line`
  directives with very large line numbers can cause an infinite loop due to
  integer overflow.
  Thanks to Philippe Antoine (Catena cyber) for reporting this issue.
  This is CVE-2023-24537 and Go issue https://go.dev/issue/59180.

- html/template: backticks not treated as string delimiters

  Templates did not properly consider backticks (`) as Javascript string
  delimiters, and as such did not escape them as expected. Backticks are
  used, since ES6, for JS template literals. If a template contained a Go
  template action within a Javascript template literal, the contents of the
  action could be used to terminate the literal, injecting arbitrary Javascript
  code into the Go template.

  As ES6 template literals are rather complex, and themselves can do string
  interpolation, we've decided to simply disallow Go template actions from being
  used inside of them (e.g. "var a = {{.}}"), since there is no obviously safe
  way to allow this behavior. This takes the same approach as
  github.com/google/safehtml. Template.Parse will now return an Error when it
  encounters templates like this, with a currently unexported ErrorCode with a
  value of 12. This ErrorCode will be exported in the next major release.

  Users who rely on this behavior can re-enable it using the GODEBUG flag
  jstmpllitinterp=1, with the caveat that backticks will now be escaped. This
  should be used with caution.

  Thanks to Sohom Datta, Manipal Institute of Technology, for reporting this issue.

  This is CVE-2023-24538 and Go issue https://go.dev/issue/59234.

- net/http, net/textproto: denial of service from excessive memory allocation

  HTTP and MIME header parsing could allocate large amounts of memory, even when
  parsing small inputs.

  Certain unusual patterns of input data could cause the common function used to
  parse HTTP and MIME headers to allocate substantially more memory than
  required to hold the parsed headers. An attacker can exploit this behavior to
  cause an HTTP server to allocate large amounts of memory from a small request,
  potentially leading to memory exhaustion and a denial of service.
  Header parsing now correctly allocates only the memory required to hold parsed
  headers.

  Thanks to Jakob Ackermann (@das7pad) for discovering this issue.

  This is CVE-2023-24534 and Go issue https://go.dev/issue/58975.

- net/http, net/textproto, mime/multipart: denial of service from excessive resource consumption

  Multipart form parsing can consume large amounts of CPU and memory when
  processing form inputs containing very large numbers of parts. This stems from
  several causes:

  mime/multipart.Reader.ReadForm limits the total memory a parsed multipart form
  can consume. ReadForm could undercount the amount of memory consumed, leading
  it to accept larger inputs than intended. Limiting total memory does not
  account for increased pressure on the garbage collector from large numbers of
  small allocations in forms with many parts. ReadForm could allocate a large
  number of short-lived buffers, further increasing pressure on the garbage
  collector. The combination of these factors can permit an attacker to cause an
  program that parses multipart forms to consume large amounts of CPU and
  memory, potentially resulting in a denial of service. This affects programs
  that use mime/multipart.Reader.ReadForm, as well as form parsing in the
  net/http package with the Request methods FormFile, FormValue,
  ParseMultipartForm, and PostFormValue.

  ReadForm now does a better job of estimating the memory consumption of parsed
  forms, and performs many fewer short-lived allocations.

  In addition, mime/multipart.Reader now imposes the following limits on the
  size of parsed forms:

  Forms parsed with ReadForm may contain no more than 1000 parts. This limit may
  be adjusted with the environment variable GODEBUG=multipartmaxparts=. Form
  parts parsed with NextPart and NextRawPart may contain no more than 10,000
  header fields. In addition, forms parsed with ReadForm may contain no more
  than 10,000 header fields across all parts. This limit may be adjusted with
  the environment variable GODEBUG=multipartmaxheaders=.

  Thanks to Jakob Ackermann for discovering this issue.

  This is CVE-2023-24536 and Go issue https://go.dev/issue/59153.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-05 15:58:33 +02:00
Sebastiaan van Stijn
5d6db84223
Merge pull request from GHSA-232p-vwff-86mp
[20.10] vendor: libnetwork c5aa85f9b25f0acaec8591ced679cb9fb5b9e32c
2023-04-04 20:03:51 +02:00
Sebastiaan van Stijn
d2bc43a75b
Merge pull request #45242 from neersighted/go1.19.7/20.10
[20.10 backport] update to go1.19.7
2023-03-30 22:43:46 +02:00
Sebastiaan van Stijn
9aa5d55a8b
update to go1.19.7
Includes a security fix for crypto/elliptic (CVE-2023-24532).

> go1.19.7 (released 2023-03-07) includes a security fix to the crypto/elliptic
> package, as well as bug fixes to the linker, the runtime, and the crypto/x509
> and syscall packages. See the Go 1.19.7 milestone on our issue tracker for
> details.

https://go.dev/doc/devel/release#go1.19.minor

From the announcement:

> We have just released Go versions 1.20.2 and 1.19.7, minor point releases.
>
> These minor releases include 1 security fixes following the security policy:
>
> - crypto/elliptic: incorrect P-256 ScalarMult and ScalarBaseMult results
    >
    >   The ScalarMult and ScalarBaseMult methods of the P256 Curve may return an
    >   incorrect result if called with some specific unreduced scalars (a scalar larger
    >   than the order of the curve).
    >
    >   This does not impact usages of crypto/ecdsa or crypto/ecdh.
>
> This is CVE-2023-24532 and Go issue https://go.dev/issue/58647.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit c48f7fd12c)
Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
2023-03-30 13:00:44 -06:00
Sebastiaan van Stijn
83679bb638
Merge pull request #45216 from corhere/backport-20.10/containerd-fifo_v1.1
[20.10 backport] Upgrade containerd/fifo to v1.1.0
2023-03-30 00:46:19 +02:00
Sebastiaan van Stijn
b4f0442da2
Merge pull request #45219 from vvoland/test-windows-execstartfails-2010
[20.10 backport] integration-cli: Enable TestExecStartFails on Windows
2023-03-28 23:04:15 +02:00
Bjorn Neergaard
ba043e8691
Merge pull request #44990 from thaJeztah/20.10_backport_update_go1.19
[20.10 backport] update to go1.19.6
2023-03-28 11:16:50 -06:00
Paweł Gronowski
b56fe59505
integration-cli: Enable TestExecStartFails on Windows
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit be34e93f20)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-03-28 11:12:34 +02:00
Sebastiaan van Stijn
d9433ee096
Merge pull request #45197 from vvoland/integration-restart-race-2010
[backport 20.10] TestDaemonRestartKillContainers: Fix races
2023-03-27 23:38:23 +02:00
Cory Snider
a9c02c238f Upgrade containerd/fifo to v1.1.0
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-03-27 17:04:05 -04:00
Cory Snider
bbec6704dc [20.10] vendor: libnetwork c5aa85f9b25f0acaec8591ced679cb9fb5b9e32c
The commit used to build the docker-proxy binary is not updated as the
build script pulls from the public libnetwork repo but the
aforementioned commit only exists in a private fork until after the
security vulnerabilities being fixed have been publicly released. The
vulnerable code is not used in the proxy binary anyway.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-03-23 12:06:05 -04:00
Paweł Gronowski
e55d9b155d
StartWithLogFile: Fix d.cmd race
Use `exec.Command` created by this function instead of obtaining it from
daemon struct. This prevents a race condition where `daemon.Kill` is
called before the goroutine has the chance to call `cmd.Wait`.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 88992de283)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-03-22 15:43:20 +01:00
Paweł Gronowski
d26e4dab48
TestDaemonRestartKillContainers: Fix loop capture
TestDaemonRestartKillContainers test was always executing the last case
(`container created should not be restarted`) because the iterated
variables were not copied correctly.
Capture iterated values by value correctly and rename c to tc.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit fed1c96e10)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-03-22 15:43:19 +01:00
Sebastiaan van Stijn
98c9e3f438 update to go1.19.6
go1.19.6 (released 2023-02-14) includes security fixes to the crypto/tls,
mime/multipart, net/http, and path/filepath packages, as well as bug fixes to
the go command, the linker, the runtime, and the crypto/x509, net/http, and
time packages. See the Go 1.19.6 milestone on our issue tracker for details:

https://github.com/golang/go/issues?q=milestone%3AGo1.19.6+label%3ACherryPickApproved

From the announcement on the security mailing:

We have just released Go versions 1.20.1 and 1.19.6, minor point releases.

These minor releases include 4 security fixes following the security policy:

- path/filepath: path traversal in filepath.Clean on Windows

  On Windows, the filepath.Clean function could transform an invalid path such
  as a/../c:/b into the valid path c:\b. This transformation of a relative (if
  invalid) path into an absolute path could enable a directory traversal attack.
  The filepath.Clean function will now transform this path into the relative
  (but still invalid) path .\c:\b.

  This is CVE-2022-41722 and Go issue https://go.dev/issue/57274.

- net/http, mime/multipart: denial of service from excessive resource
  consumption

  Multipart form parsing with mime/multipart.Reader.ReadForm can consume largely
  unlimited amounts of memory and disk files. This also affects form parsing in
  the net/http package with the Request methods FormFile, FormValue,
  ParseMultipartForm, and PostFormValue.

  ReadForm takes a maxMemory parameter, and is documented as storing "up to
  maxMemory bytes +10MB (reserved for non-file parts) in memory". File parts
  which cannot be stored in memory are stored on disk in temporary files. The
  unconfigurable 10MB reserved for non-file parts is excessively large and can
  potentially open a denial of service vector on its own. However, ReadForm did
  not properly account for all memory consumed by a parsed form, such as map
  ntry overhead, part names, and MIME headers, permitting a maliciously crafted
  form to consume well over 10MB. In addition, ReadForm contained no limit on
  the number of disk files created, permitting a relatively small request body
  to create a large number of disk temporary files.

  ReadForm now properly accounts for various forms of memory overhead, and
  should now stay within its documented limit of 10MB + maxMemory bytes of
  memory consumption. Users should still be aware that this limit is high and
  may still be hazardous.

  ReadForm now creates at most one on-disk temporary file, combining multiple
  form parts into a single temporary file. The mime/multipart.File interface
  type's documentation states, "If stored on disk, the File's underlying
  concrete type will be an *os.File.". This is no longer the case when a form
  contains more than one file part, due to this coalescing of parts into a
  single file. The previous behavior of using distinct files for each form part
  may be reenabled with the environment variable
  GODEBUG=multipartfiles=distinct.

  Users should be aware that multipart.ReadForm and the http.Request methods
  that call it do not limit the amount of disk consumed by temporary files.
  Callers can limit the size of form data with http.MaxBytesReader.

  This is CVE-2022-41725 and Go issue https://go.dev/issue/58006.

- crypto/tls: large handshake records may cause panics

  Both clients and servers may send large TLS handshake records which cause
  servers and clients, respectively, to panic when attempting to construct
  responses.

  This affects all TLS 1.3 clients, TLS 1.2 clients which explicitly enable
  session resumption (by setting Config.ClientSessionCache to a non-nil value),
  and TLS 1.3 servers which request client certificates (by setting
  Config.ClientAuth
  > = RequestClientCert).

  This is CVE-2022-41724 and Go issue https://go.dev/issue/58001.

- net/http: avoid quadratic complexity in HPACK decoding

  A maliciously crafted HTTP/2 stream could cause excessive CPU consumption
  in the HPACK decoder, sufficient to cause a denial of service from a small
  number of small requests.

  This issue is also fixed in golang.org/x/net/http2 v0.7.0, for users manually
  configuring HTTP/2.

  This is CVE-2022-41723 and Go issue https://go.dev/issue/57855.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 94feb31516)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-02-27 11:28:45 -05:00
Sebastiaan van Stijn
5b48f300dd update to go1.19.5
go1.19.5 (released 2023-01-10) includes fixes to the compiler, the linker,
and the crypto/x509, net/http, sync/atomic, and syscall packages. See the
Go 1.19.5 milestone on the issue tracker for details:

https://github.com/golang/go/issues?q=milestone%3AGo1.19.5+label%3ACherryPickApproved

full diff: https://github.com/golang/go/compare/go1.19.4...go1.19.5

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 155e8d7d78)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-02-27 11:28:45 -05:00
Sebastiaan van Stijn
82b0ac1166 update to go1.19.4
Includes security fixes for net/http (CVE-2022-41717, CVE-2022-41720),
and os (CVE-2022-41720).

These minor releases include 2 security fixes following the security policy:

- os, net/http: avoid escapes from os.DirFS and http.Dir on Windows

  The os.DirFS function and http.Dir type provide access to a tree of files
  rooted at a given directory. These functions permitted access to Windows
  device files under that root. For example, os.DirFS("C:/tmp").Open("COM1")
  would open the COM1 device.
  Both os.DirFS and http.Dir only provide read-only filesystem access.

  In addition, on Windows, an os.DirFS for the directory \(the root of the
  current drive) can permit a maliciously crafted path to escape from the
  drive and access any path on the system.

  The behavior of os.DirFS("") has changed. Previously, an empty root was
  treated equivalently to "/", so os.DirFS("").Open("tmp") would open the
  path "/tmp". This now returns an error.

  This is CVE-2022-41720 and Go issue https://go.dev/issue/56694.

- net/http: limit canonical header cache by bytes, not entries

  An attacker can cause excessive memory growth in a Go server accepting
  HTTP/2 requests.

  HTTP/2 server connections contain a cache of HTTP header keys sent by
  the client. While the total number of entries in this cache is capped,
  an attacker sending very large keys can cause the server to allocate
  approximately 64 MiB per open connection.

  This issue is also fixed in golang.org/x/net/http2 vX.Y.Z, for users
  manually configuring HTTP/2.

  Thanks to Josselin Costanzi for reporting this issue.

  This is CVE-2022-41717 and Go issue https://go.dev/issue/56350.

View the release notes for more information:
https://go.dev/doc/devel/release#go1.19.4

And the milestone on the issue tracker:
https://github.com/golang/go/issues?q=milestone%3AGo1.19.4+label%3ACherryPickApproved

Full diff: https://github.com/golang/go/compare/go1.19.3...go1.19.4

The golang.org/x/net fix is in 1e63c2f08a

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 52bc1ad744)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-02-27 11:28:45 -05:00
Cory Snider
4701ca9f71 Update to Go 1.19.3 to address CVE-2022-41716
On Windows, syscall.StartProcess and os/exec.Cmd did not properly
    check for invalid environment variable values. A malicious
    environment variable value could exploit this behavior to set a
    value for a different environment variable. For example, the
    environment variable string "A=B\x00C=D" set the variables "A=B" and
    "C=D".

    Thanks to RyotaK (https://twitter.com/ryotkak) for reporting this
    issue.

    This is CVE-2022-41716 and Go issue https://go.dev/issue/56284.

This Go release also fixes https://github.com/golang/go/issues/56309, a
runtime bug which can cause random memory corruption when a goroutine
exits with runtime.LockOSThread() set. This fix is necessary to unblock
work to replace certain uses of pkg/reexec with unshared OS threads.

Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit f9d4589976)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-02-27 11:28:45 -05:00
Cory Snider
091b8896be vendor: update stdlib archive/tar for go1.19.2
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-02-27 11:28:45 -05:00
Sebastiaan van Stijn
1c8c16524f Update to go 1.19.2 to address CVE-2022-2879, CVE-2022-2880, CVE-2022-41715
From the mailing list:

We have just released Go versions 1.19.2 and 1.18.7, minor point releases.

These minor releases include 3 security fixes following the security policy:

- archive/tar: unbounded memory consumption when reading headers

  Reader.Read did not set a limit on the maximum size of file headers.
  A maliciously crafted archive could cause Read to allocate unbounded
  amounts of memory, potentially causing resource exhaustion or panics.
  Reader.Read now limits the maximum size of header blocks to 1 MiB.

  Thanks to Adam Korczynski (ADA Logics) and OSS-Fuzz for reporting this issue.

  This is CVE-2022-2879 and Go issue https://go.dev/issue/54853.

- net/http/httputil: ReverseProxy should not forward unparseable query parameters

  Requests forwarded by ReverseProxy included the raw query parameters from the
  inbound request, including unparseable parameters rejected by net/http. This
  could permit query parameter smuggling when a Go proxy forwards a parameter
  with an unparseable value.

  ReverseProxy will now sanitize the query parameters in the forwarded query
  when the outbound request's Form field is set after the ReverseProxy.Director
  function returns, indicating that the proxy has parsed the query parameters.
  Proxies which do not parse query parameters continue to forward the original
  query parameters unchanged.

  Thanks to Gal Goldstein (Security Researcher, Oxeye) and
  Daniel Abeles (Head of Research, Oxeye) for reporting this issue.

  This is CVE-2022-2880 and Go issue https://go.dev/issue/54663.

- regexp/syntax: limit memory used by parsing regexps

  The parsed regexp representation is linear in the size of the input,
  but in some cases the constant factor can be as high as 40,000,
  making relatively small regexps consume much larger amounts of memory.

  Each regexp being parsed is now limited to a 256 MB memory footprint.
  Regular expressions whose representation would use more space than that
  are now rejected. Normal use of regular expressions is unaffected.

  Thanks to Adam Korczynski (ADA Logics) and OSS-Fuzz for reporting this issue.

  This is CVE-2022-41715 and Go issue https://go.dev/issue/55949.

View the release notes for more information: https://go.dev/doc/devel/release#go1.19.2

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 7b4e4c08b5)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-02-27 11:28:45 -05:00
Sebastiaan van Stijn
6cc1ef32a2 Update to go 1.19.1 to address CVE-2022-27664, CVE-2022-32190
From the mailing list:

We have just released Go versions 1.19.1 and 1.18.6, minor point releases.
These minor releases include 2 security fixes following the security policy:

- net/http: handle server errors after sending GOAWAY
  A closing HTTP/2 server connection could hang forever waiting for a clean
  shutdown that was preempted by a subsequent fatal error. This failure mode
  could be exploited to cause a denial of service.

  Thanks to Bahruz Jabiyev, Tommaso Innocenti, Anthony Gavazzi, Steven Sprecher,
  and Kaan Onarlioglu for reporting this.

  This is CVE-2022-27664 and Go issue https://go.dev/issue/54658.

- net/url: JoinPath does not strip relative path components in all circumstances
  JoinPath and URL.JoinPath would not remove `../` path components appended to a
  relative path. For example, `JoinPath("https://go.dev", "../go")` returned the
  URL `https://go.dev/../go`, despite the JoinPath documentation stating that
  `../` path elements are cleaned from the result.

  Thanks to q0jt for reporting this issue.

  This is CVE-2022-32190 and Go issue https://go.dev/issue/54385.

Release notes:

go1.19.1 (released 2022-09-06) includes security fixes to the net/http and
net/url packages, as well as bug fixes to the compiler, the go command, the pprof
command, the linker, the runtime, and the crypto/tls and crypto/x509 packages.
See the Go 1.19.1 milestone on the issue tracker for details.

https://github.com/golang/go/issues?q=milestone%3AGo1.19.1+label%3ACherryPickApproved

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 1eadbdd9fa)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-02-27 11:28:45 -05:00
Sebastiaan van Stijn
721358e0cb vendor: update stdlib archive/tar for go1.19
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-02-27 11:28:45 -05:00
Sebastiaan van Stijn
5091f13a5d update to golang 1.19
also ran gofmt with go1.19

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 58413c15cb)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-02-27 11:28:45 -05:00
Cory Snider
dfd89ede4b
Merge pull request #45073 from corhere/20.10_go1.19_prereqs
[20.10 backport] Prepare for Go 1.19 upgrade
2023-02-27 11:27:41 -05:00
Sebastiaan van Stijn
6b1c02b983 golangci-lint: update to v1.49.0
Remove the "deadcode", "structcheck", and "varcheck" linters, as they are
deprecated:

    WARN [runner] The linter 'deadcode' is deprecated (since v1.49.0) due to: The owner seems to have abandoned the linter.  Replaced by unused.
    WARN [runner] The linter 'structcheck' is deprecated (since v1.49.0) due to: The owner seems to have abandoned the linter.  Replaced by unused.
    WARN [runner] The linter 'varcheck' is deprecated (since v1.49.0) due to: The owner seems to have abandoned the linter.  Replaced by unused.
    WARN [linters context] structcheck is disabled because of generics. You can track the evolution of the generics support by following the https://github.com/golangci/golangci-lint/issues/2649.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 2f1c382a6d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 3ce520ec80)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-02-24 17:05:42 -05:00
Sebastiaan van Stijn
9d08a57a9c set ReadHeaderTimeout to address G112: Potential Slowloris Attack (gosec)
After discussing in the maintainers meeting, we concluded that Slowloris attacks
are not a real risk other than potentially having some additional goroutines
lingering around, so setting a long timeout to satisfy the linter, and to at
least have "some" timeout.

    api/server/server.go:60:10: G112: Potential Slowloris Attack because ReadHeaderTimeout is not configured in the http.Server (gosec)
                srv: &http.Server{
                    Addr: addr,
                },
    daemon/metrics_unix.go:34:13: G114: Use of net/http serve function that has no support for setting timeouts (gosec)
            if err := http.Serve(l, mux); err != nil && !strings.Contains(err.Error(), "use of closed network connection") {
                      ^
    cmd/dockerd/metrics.go:27:13: G114: Use of net/http serve function that has no support for setting timeouts (gosec)
            if err := http.Serve(l, mux); err != nil && !strings.Contains(err.Error(), "use of closed network connection") {
                      ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 55fd77f724)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 997ec12ec8)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-02-24 17:05:42 -05:00
Sebastiaan van Stijn
d855447c46 linting: host:port in url should be constructed with net.JoinHostPort
integration-cli/docker_cli_daemon_test.go:545:54: host:port in url should be constructed with net.JoinHostPort and not directly with fmt.Sprintf (nosprintfhostport)
            cmdArgs = append(cmdArgs, "--tls=false", "--host", fmt.Sprintf("tcp://%s:%s", l.daemon, l.port))
                                                               ^
    opts/hosts_test.go:35:31: host:port in url should be constructed with net.JoinHostPort and not directly with fmt.Sprintf (nosprintfhostport)
            "tcp://:5555":              fmt.Sprintf("tcp://%s:5555", DefaultHTTPHost),
                                        ^
    opts/hosts_test.go:91:30: host:port in url should be constructed with net.JoinHostPort and not directly with fmt.Sprintf (nosprintfhostport)
            ":5555":                   fmt.Sprintf("tcp://%s:5555", DefaultHTTPHost),
                                       ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 306b8c89e8)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit e9e7491f2b)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-02-24 17:05:42 -05:00
Sebastiaan van Stijn
2ad43faba5 linting: gosec: fix or suppress G112, G114 in test code
Updating test-code only; set ReadHeaderTimeout for some, or suppress the linter
error for others.

     contrib/httpserver/server.go:11:12: G114: Use of net/http serve function that has no support for setting timeouts (gosec)
        log.Panic(http.ListenAndServe(":80", nil))
                  ^
     integration/plugin/logging/cmd/close_on_start/main.go:42:12: G112: Potential Slowloris Attack because ReadHeaderTimeout is not configured in the http.Server (gosec)
        server := http.Server{
            Addr:    l.Addr().String(),
            Handler: mux,
        }
     integration/plugin/logging/cmd/discard/main.go:17:12: G112: Potential Slowloris Attack because ReadHeaderTimeout is not configured in the http.Server (gosec)
        server := http.Server{
            Addr:    l.Addr().String(),
            Handler: mux,
        }
     integration/plugin/logging/cmd/dummy/main.go:14:12: G112: Potential Slowloris Attack because ReadHeaderTimeout is not configured in the http.Server (gosec)
        server := http.Server{
            Addr:    l.Addr().String(),
            Handler: http.NewServeMux(),
        }
     integration/plugin/volumes/cmd/dummy/main.go:14:12: G112: Potential Slowloris Attack because ReadHeaderTimeout is not configured in the http.Server (gosec)
        server := http.Server{
            Addr:    l.Addr().String(),
            Handler: http.NewServeMux(),
        }
     testutil/fixtures/plugin/basic/basic.go:25:12: G112: Potential Slowloris Attack because ReadHeaderTimeout is not configured in the http.Server (gosec)
        server := http.Server{
            Addr:    l.Addr().String(),
            Handler: http.NewServeMux(),
        }
     volume/testutils/testutils.go:170:5: G114: Use of net/http serve function that has no support for setting timeouts (gosec)
        go http.Serve(l, mux)
           ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 31fb92c609)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 2609d4e252)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-02-24 17:05:42 -05:00
Sebastiaan van Stijn
9a5d1b295e fix formatting of "nolint" tags for go1.19
The correct formatting for machine-readable comments is;

    //<some alphanumeric identifier>:<options>[,<option>...][ // comment]

Which basically means:

- MUST NOT have a space before `<identifier>` (e.g. `nolint`)
- Identified MUST be alphanumeric
- MUST be followed by a colon
- MUST be followed by at least one `<option>`
- Optionally additional `<options>` (comma-separated)
- Optionally followed by a comment

Any other format will not be considered a machine-readable comment by `gofmt`,
and thus formatted as a regular comment. Note that this also means that a
`//nolint` (without anything after it) is considered invalid, same for `//#nosec`
(starts with a `#`).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 4f08346686)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit e34ab5200d)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-02-24 17:05:41 -05:00
Sebastiaan van Stijn
c8c40abbba gofmt GoDoc comments with go1.19
Older versions of Go don't format comments, so committing this as
a separate commit, so that we can already make these changes before
we upgrade to Go 1.19.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 52c1a2fae8)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit cdbca4061b)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-02-24 17:05:41 -05:00
Sebastiaan van Stijn
78d8e65d2a replace golint with revive, as it's deprecated
WARN [runner] The linter 'golint' is deprecated (since v1.41.0) due to: The repository of the linter has been archived by the owner.  Replaced by revive.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 1cab8eda24)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-02-24 17:05:41 -05:00
Sebastiaan van Stijn
432fbc8638 hack: update golangci-lint to v1.46.2
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 65e1adc219)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-02-24 17:05:41 -05:00
Sebastiaan van Stijn
057ea3492e fix some minor linting issues
distribution/pull_v2_test.go:213:4: S1038: should use t.Fatalf(...) instead of t.Fatal(fmt.Sprintf(...)) (gosimple)
                t.Fatal(fmt.Sprintf("expected formatPlatform to show windows platform with a version, but got '%s'", result))
                ^
    integration-cli/docker_cli_build_test.go:5951:3: S1038: should use c.Skipf(...) instead of c.Skip(fmt.Sprintf(...)) (gosimple)
            c.Skip(fmt.Sprintf("Bug fixed in 18.06 or higher.Skipping it for %s", testEnv.DaemonInfo.ServerVersion))
            ^
    integration-cli/docker_cli_daemon_test.go:240:3: S1038: should use c.Skipf(...) instead of c.Skip(fmt.Sprintf(...)) (gosimple)
            c.Skip(fmt.Sprintf("New base device size (%v) must be greater than (%s)", units.HumanSize(float64(newBasesizeBytes)), units.HumanSize(float64(oldBasesizeBytes))))
            ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 968ff5ab44)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-02-24 17:05:41 -05:00
Sebastiaan van Stijn
2461d88305 linting: error strings should not be capitalized (revive)
client/request.go:183:28: error-strings: error strings should not be capitalized or end with punctuation or a newline (revive)
                    err = errors.Wrap(err, "In the default daemon configuration on Windows, the docker client must be run with elevated privileges to connect.")
                                           ^
    client/request.go:186:28: error-strings: error strings should not be capitalized or end with punctuation or a newline (revive)
                    err = errors.Wrap(err, "This error may indicate that the docker daemon is not running.")
                                           ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 10c56efa97)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-02-24 17:05:41 -05:00
Kazuyoshi Kato
063042449e hack: remove a workaround for go-tools
The false positive has been fixed.

Signed-off-by: Kazuyoshi Kato <katokazu@amazon.com>
(cherry picked from commit 4bdc208449)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-02-24 16:11:56 -05:00
Kazuyoshi Kato
7125951236 hack: do not exclude SQL-related checks
moby doesn't have SQLs right now.

Signed-off-by: Kazuyoshi Kato <katokazu@amazon.com>
(cherry picked from commit 22f19f8243)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-02-24 16:11:55 -05:00
Kazuyoshi Kato
ddae0d6381 hack: remove non-existent directories from golangci-lint's skip-dirs
Signed-off-by: Kazuyoshi Kato <katokazu@amazon.com>
(cherry picked from commit 136f93ea05)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-02-24 16:11:55 -05:00
Eng Zer Jun
a916414b0b refactor: move from io/ioutil to io and os package
The io/ioutil package has been deprecated in Go 1.16. This commit
replaces the existing io/ioutil functions with their new definitions in
io and os packages.

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
(cherry picked from commit c55a4ac779)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-02-24 16:11:55 -05:00
Sebastiaan van Stijn
9f156997b0
Merge pull request #44946 from thaJeztah/20.10_backport_dockerfile_git_dubious
[20.10 backport] Dockerfile: configure code dir as "safe" directory
2023-02-14 20:32:10 +01:00
Sebastiaan van Stijn
0312e468da
Dockerfile: configure code dir as "safe" directory
CI is failing when bind-mounting source from the host into the dev-container;

    fatal: detected dubious ownership in repository at '/go/src/github.com/docker/docker'
    To add an exception for this directory, call:

        git config --global --add safe.directory /go/src/github.com/docker/docker

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 21677816a0)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-02-07 14:05:53 +01:00
Akihiro Suda
b3ff678200
Merge pull request #44882 from thaJeztah/20.10_containerd_binary_1.6.16
[20.10] update containerd binary to v1.6.16
2023-01-31 02:00:29 +09:00
Sebastiaan van Stijn
526ae907e2
[20.10] update containerd binary to v1.6.16
Notable Updates

- Fix push error propagation
- Fix slice append error with HugepageLimits for Linux
- Update default seccomp profile for PKU and CAP_SYS_NICE
- Fix overlayfs error when upperdirlabel option is set

full diff: https://github.com/containerd/containerd/compare/v1.6.15...v1.6.16

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-01-30 09:17:32 +01:00
Bjorn Neergaard
6051f14291
Merge pull request #44846 from thaJeztah/20.10_bump_libnetwork
[20.10] vendor: libnetwork 05b93e0d3a95952f70c113b0bc5bdb538d7afdd7
2023-01-18 09:24:28 -07:00
Sebastiaan van Stijn
e3b9b535bd
[20.10] vendor: libnetwork 05b93e0d3a95952f70c113b0bc5bdb538d7afdd7
full diff: 374259e831...05b93e0d3a

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-01-18 13:25:55 +01:00
Sebastiaan van Stijn
d0a3f3a376
Merge pull request #44795 from thaJeztah/20.10_bump_go_1.18.10
[20.10] update to go1.18.10
2023-01-11 16:25:23 +01:00
Sebastiaan van Stijn
de10f917ea
Merge pull request #44797 from neersighted/libnetwork_revendor
[20.10] vendor: github.com/docker/libnetwork 374259e8316124ccf1fc38c0c0f3430f8d0e9c76
2023-01-11 09:41:28 +01:00
Bjorn Neergaard
26a4ccf99e
vendor: github.com/docker/libnetwork 374259e8316124ccf1fc38c0c0f3430f8d0e9c76
Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
2023-01-10 18:14:10 -07:00
Sebastiaan van Stijn
625903f3fd
[20.10] update to go1.18.10
go1.18.10 (released 2023-01-10) includes fixes to cgo, the compiler, the linker,
and the crypto/x509, net/http, and syscall packages. See the Go 1.18.10 milestone
on the issue tracker for details:

https://github.com/golang/go/issues?q=milestone%3AGo1.18.10+label%3ACherryPickApproved

full diff: https://github.com/golang/go/compare/go1.18.9...go1.18.10

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-01-11 00:21:42 +01:00
Sebastiaan van Stijn
864cc1c871
Merge pull request #44765 from vvoland/fix-test-race-logs-goroutines-20
[20.10 backport] integration-cli: Fix hanging TestLogsFollowGoroutines*
2023-01-10 15:30:39 +01:00
Samuel Karp
8771b4be5f
Merge pull request #44767 from thaJeztah/20.10_containerd_binary_1.6.15 2023-01-09 13:23:34 -08:00
Sebastiaan van Stijn
208fb1ca7b
Merge pull request #44755 from vvoland/buildflags-nopie-whenrace-2010
[20.10 backport] hack/make: Don't add -buildmode=pie with -race
2023-01-09 13:30:14 +01:00
Sebastiaan van Stijn
efbc878c47
[20.10] update containerd binary to v1.6.15
release notes: https://github.com/containerd/containerd/releases/tag/v1.6.15

> Welcome to the v1.6.15 release of containerd!
>
> The fifteenth patch release for containerd 1.6 fixes an issue with CNI in the CRI plugin
>
> Notable Updates
>
> - Fix no CNI info for pod sandbox on restart in CRI plugin

full diff: https://github.com/containerd/containerd/compare/v1.6.14...v1.6.15

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-01-09 13:13:40 +01:00
Paweł Gronowski
91e767cc1e
integration-cli: Fix hanging TestLogsFollowGoroutines*
cmd.Wait is called twice from different goroutines which can cause the
test to hang completely. Fix by calling Wait only once and sending its
return value over a channel.

In TestLogsFollowGoroutinesWithStdout also added additional closes and
process kills to ensure that we don't leak anything in case test returns
early because of failed test assertion.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit deb4910c5b)
2023-01-09 10:31:06 +01:00
Paweł Gronowski
d645297358
hack/make: Don't add -buildmode=pie with -race
Make it possible to add `-race` to the BUILDFLAGS without making the
build fail with error:
"-buildmode=pie not supported when -race is enabled"

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-01-05 20:32:46 +01:00
Sebastiaan van Stijn
6ddb3ba414
Merge pull request #44718 from neersighted/backport/44668/20.10
[20.10 backport] api: can marshal and unmarshal when args.fields is empty
2023-01-02 18:29:02 +01:00
Yamazaki Masashi
ad703aa0ab
api: can marshal and unmarshal when args.fields is empty
Signed-off-by: Yamazaki Masashi <masi19bw@gmail.com>
Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
(cherry picked from commit 5fed968b1d)
Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
2022-12-29 15:42:24 -07:00
Sebastiaan van Stijn
ae6a9319b0
Merge pull request #44712 from neersighted/20.10_revert_vsock_filter
[20.10] Revert "seccomp: block socket calls to AF_VSOCK in default profile"
2022-12-29 23:13:50 +01:00
Sebastiaan van Stijn
d46fe8029b
Merge pull request #44650 from thaJeztah/20.10_backport_builder_host_gateway
[20.10 backport] builder: handle host-gateway with extra hosts
2022-12-29 23:10:02 +01:00
Sebastiaan van Stijn
d3bf4595cd
Merge pull request #44682 from thaJeztah/20.10_update_libnetwork
[20.10] vendor: github.com/docker/libnetwork 1f3b98be6833a93f254aa0f765ff55d407dfdd69
2022-12-29 22:54:14 +01:00
Bjorn Neergaard
f91836cb26
Merge pull request #44709 from zhsj/20.10-btrfs-progs-6.1
[20.10 backport] daemon/graphdriver/btrfs: workaround field rename in btrfs-progs 6.1
2022-12-29 14:47:31 -07:00
Bjorn Neergaard
dcf27af59b
Revert "seccomp: block socket calls to AF_VSOCK in default profile"
This reverts commit 57b229012a.

This change, while favorable from a security standpoint, caused a
regression for users of the 20.10 branch of Moby. As such, we are
reverting it to ensure stability and compatibility for the affected
users.

However, users of AF_VSOCK in containers should recognize that this
(special) address family is not currently namespaced in any version of
the Linux kernel, and may result in unexpected behavior, like VMs
communicating directly with host hypervisors.

Future branches, including the 23.0 branch, will continue to filter
AF_VSOCK. Users who need to allow containers to communicate over the
unnamespaced AF_VSOCK will need to turn off seccomp confinement or set a
custom seccomp profile.

It is our hope that future mechanisms will make this more
ergonomic/maintainable for end users, and that future kernels will
support namespacing of AF_VSOCK.

Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
2022-12-29 13:16:57 -07:00
Sebastiaan van Stijn
7f3a95d360
vendor: github.com/docker/libnetwork 1f3b98be6833a93f254aa0f765ff55d407dfdd69
- fix linting issues
- update to go1.18.9, gofmt, and regenerate proto
- processEndpointCreate: Fix deadlock between getSvcRecords and processEndpointCreate

full diff: dcdf8f176d...1f3b98be68

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-29 19:50:39 +01:00
Sebastiaan van Stijn
313e53aafd
Merge pull request #44691 from thaJeztah/20.10_fix_vanity_url
[20.10] fix broken vanity-URL for code.cloudfoundry.org
2022-12-29 19:50:05 +01:00
Sebastiaan van Stijn
0c207b0516
[20.10] fix broken vanity-URL for code.cloudfoundry.org
The vanity URL looks to be misconfigured;

    2022-12-22T00:01:12.571Z] 2022/12/22 00:01:12 unrecognized import path "code.cloudfoundry.org/clock" (https fetch: Get "https://code.cloudfoundry.org/clock?go-get=1": x509: certificate is valid for *.de.a9sapp.eu, de.a9sapp.eu, not code.cloudfoundry.org)

This patch updates vendor.conf to fetch the code directly from GitHub.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-29 14:45:04 +01:00
Shengjing Zhu
97ae178741 daemon/graphdriver/btrfs: workaround field rename in btrfs-progs 6.1
Closes: #44698

Signed-off-by: Shengjing Zhu <zhsj@debian.org>
(cherry picked from commit ffbbe3d103)
Signed-off-by: Shengjing Zhu <zhsj@debian.org>
2022-12-29 21:21:26 +08:00
Sebastiaan van Stijn
ddab6b3621
Merge pull request #44673 from thaJeztah/20.10_update_containerd_1.6.14
[20.10] update containerd binary to v1.6.14
2022-12-20 10:56:02 +01:00
Sebastiaan van Stijn
2f212873ac
update containerd binary to v1.6.14
Notable Updates

- Fix `memory.memsw.limit_in_bytes: no such file or directory` error in CRI plugin

full diff: https://github.com/containerd/containerd/compare/v1.6.13...v1.6.14

See the changelog for complete list of changes:
https://github.com/containerd/containerd/releases/tag/v1.6.14

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-19 18:35:28 +01:00
Sebastiaan van Stijn
42c8b31499
Merge pull request #44656 from thaJeztah/20.10_containerd_binary_1.6.13
[20.10] update containerd binary to v1.6.13
2022-12-15 23:17:04 +01:00
Sebastiaan van Stijn
ff29c403a3
update containerd binary to v1.6.13
Notable Updates

- Update overlay snapshotter to check for tmpfs when evaluating usage of userxattr
- Update hcsschim to v0.9.6 to fix resource leak on exec
- Make swapping disabled with memory limit in CRI plugin
- Allow clients to remove created tasks with PID 0
- Fix concurrent map iteration and map write in CRI port forwarding
- Check for nil HugepageLimits to avoid panic in CRI plugin

See the changelog for complete list of changes:
https://github.com/containerd/containerd/releases/tag/v1.6.13

full diff: https://github.com/containerd/containerd/compare/v1.6.12...v1.6.13

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-15 19:29:13 +01:00
Sebastiaan van Stijn
02343226e5
Merge pull request #44488 from thaJeztah/20.10_backport_update_gotestsum
[20.10] update gotestsum to v1.8.2
2022-12-15 15:13:54 +01:00
CrazyMax
877a5e6867
builder: handle host-gateway with extra hosts
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 521b8c02cc)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-15 13:30:26 +01:00
Sebastiaan van Stijn
edca413033
[20.10] update gotestsum to v1.8.2
release notes: https://github.com/gotestyourself/gotestsum/releases/tag/v1.8.2

- Show shuffle seed
- Update tests, and cleanup formats
- Update dependencies
- Test against go1.19, remove go1.15
- Add project name to junit.xml output
- Adding in support for s390x and ppc64le

full diff: https://github.com/gotestyourself/gotestsum/compare/v1.7.0...v1.8.2

(adapted from commit 882ddf4b16)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-08 19:53:20 +01:00
Sebastiaan van Stijn
6112b23b36
Merge pull request #44476 from sbuckfelder/20.10_UPDATE
[20.10] vendor: github.com/moby/libnetwork dcdf8f176d1e13ad719e913e796fb698d846de98
2022-12-08 19:51:17 +01:00
Sebastiaan van Stijn
194e73feb9
Merge pull request #44607 from thaJeztah/20.10_containerd_binary_1.6.12
[20.10] update containerd binary to v1.6.12 (addresses CVE-2022-23471)
2022-12-08 10:32:35 +01:00
Sebastiaan van Stijn
a9fdcd577f
[20.10] update containerd binary to v1.6.12 (addresses CVE-2022-23471)
The twelfth patch release for containerd 1.6 contains a fix for CVE-2022-23471.

Notable Updates

- Fix goroutine leak during Exec in CRI plugin (GHSA-2qjp-425j-52j9)

full diff: https://github.com/containerd/containerd/compare/v1.6.11...v1.6.12

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-08 00:48:56 +01:00
Sebastiaan van Stijn
48f955ddb9
Merge pull request #44597 from thaJeztah/20.10_containerd_1.6.11
[20.10] update containerd binary to v1.6.11
2022-12-07 13:49:24 +01:00
Sebastiaan van Stijn
50d4d98737
Merge pull request #44569 from thaJeztah/20.10_backport_relax_checkSupportedMediaType
[20.10 backport] distribution: checkSupportedMediaType: allow additional media-types
2022-12-07 13:29:51 +01:00
Sebastiaan van Stijn
17451d2fdc
Merge pull request #44593 from thaJeztah/20.10_update_go_1.18.9
[20.10] update to go1.18.9
2022-12-07 13:23:00 +01:00
Sebastiaan van Stijn
565fcdd56c
[20.10] update containerd binary to v1.6.11
Welcome to the v1.6.11 release of containerd!

The eleventh patch release for containerd 1.6 contains a various fixes and updates.

Notable Updates

- Add pod UID annotation in CRI plugin
- Fix nil pointer deference for Windows containers in CRI plugin
- Fix lease labels unexpectedly overwriting expiration
- Fix for simultaneous diff creation using the same parent snapshot

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-06 23:35:20 +01:00
Sebastiaan van Stijn
f8b0d77bfe
[20.10] update to go1.18.9
Includes security fixes for net/http (CVE-2022-41717, CVE-2022-41720),
and os (CVE-2022-41720).

These minor releases include 2 security fixes following the security policy:

- os, net/http: avoid escapes from os.DirFS and http.Dir on Windows

  The os.DirFS function and http.Dir type provide access to a tree of files
  rooted at a given directory. These functions permitted access to Windows
  device files under that root. For example, os.DirFS("C:/tmp").Open("COM1")
  would open the COM1 device.
  Both os.DirFS and http.Dir only provide read-only filesystem access.

  In addition, on Windows, an os.DirFS for the directory \(the root of the
  current drive) can permit a maliciously crafted path to escape from the
  drive and access any path on the system.

  The behavior of os.DirFS("") has changed. Previously, an empty root was
  treated equivalently to "/", so os.DirFS("").Open("tmp") would open the
  path "/tmp". This now returns an error.

  This is CVE-2022-41720 and Go issue https://go.dev/issue/56694.

- net/http: limit canonical header cache by bytes, not entries

  An attacker can cause excessive memory growth in a Go server accepting
  HTTP/2 requests.

  HTTP/2 server connections contain a cache of HTTP header keys sent by
  the client. While the total number of entries in this cache is capped,
  an attacker sending very large keys can cause the server to allocate
  approximately 64 MiB per open connection.

  This issue is also fixed in golang.org/x/net/http2 vX.Y.Z, for users
  manually configuring HTTP/2.

  Thanks to Josselin Costanzi for reporting this issue.

  This is CVE-2022-41717 and Go issue https://go.dev/issue/56350.

View the release notes for more information:
https://go.dev/doc/devel/release#go1.18.9

And the milestone on the issue tracker:
https://github.com/golang/go/issues?q=milestone%3AGo1.18.9+label%3ACherryPickApproved

Full diff: https://github.com/golang/go/compare/go1.18.8...go1.18.9

The golang.org/x/net fix is in 1e63c2f08a

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-06 22:59:44 +01:00
Sebastiaan van Stijn
cfb3ebe96e
Merge pull request #44564 from thaJeztah/20.10_backport_seccomp_block_af_vsock
[20.10 backport] seccomp: block socket calls to AF_VSOCK in default profile
2022-12-05 13:31:18 +01:00
Sebastiaan van Stijn
966327c1e5
distribution: checkSupportedMediaType: allow additional media-types
This addresses a regression introduced in 407e3a4552,
which turned out to be "too strict", as there's old images that use, for example;

    docker pull python:3.5.1-alpine
    3.5.1-alpine: Pulling from library/python
    unsupported media type application/octet-stream

Before 407e3a4552, such mediatypes were accepted;

    docker pull python:3.5.1-alpine
    3.5.1-alpine: Pulling from library/python
    e110a4a17941: Pull complete
    30dac23631f0: Pull complete
    202fc3980a36: Pull complete
    Digest: sha256:f88925c97b9709dd6da0cb2f811726da9d724464e9be17a964c70f067d2aa64a
    Status: Downloaded newer image for python:3.5.1-alpine
    docker.io/library/python:3.5.1-alpine

This patch copies the additional media-types, using the list of types that
were added in a215e15cb1, which fixed a
similar issue.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit a6a539497a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-02 01:31:15 +01:00
Sebastiaan van Stijn
a01576ec4a
seccomp: block socket calls to AF_VSOCK in default profile
This syncs the seccomp-profile with the latest changes in containerd's
profile, applying the same changes as 17a9324035

Some background from the associated ticket:

> We want to use vsock for guest-host communication on KubeVirt
> (https://github.com/kubevirt/kubevirt). In KubeVirt we run VMs in pods.
>
> However since anyone can just connect from any pod to any VM with the
> default seccomp settings, we cannot limit connection attempts to our
> privileged node-agent.
>
> ### Describe the solution you'd like
> We want to deny the `socket` syscall for the `AF_VSOCK` family by default.
>
> I see in [1] and [2] that AF_VSOCK was actually already blocked for some
> time, but that got reverted since some architectures support the `socketcall`
> syscall which can't be restricted properly. However we are mostly interested
> in `arm64` and `amd64` where limiting `socket` would probably be enough.
>
> ### Additional context
> I know that in theory we could use our own seccomp profiles, but we would want
> to provide security for as many users as possible which use KubeVirt, and there
> it would be very helpful if this protection could be added by being part of the
> DefaultRuntime profile to easily ensure that it is active for all pods [3].
>
> Impact on existing workloads: It is unlikely that this will disturb any existing
> workload, becuase VSOCK is almost exclusively used for host-guest commmunication.
> However if someone would still use it: Privileged pods would still be able to
> use `socket` for `AF_VSOCK`, custom seccomp policies could be applied too.
> Further it was already blocked for quite some time and the blockade got lifted
> due to reasons not related to AF_VSOCK.
>
> The PR in KubeVirt which adds VSOCK support for additional context: [4]
>
> [1]: https://github.com/moby/moby/pull/29076#commitcomment-21831387
> [2]: dcf2632945
> [3]: https://kubernetes.io/docs/tutorials/security/seccomp/#enable-the-use-of-runtimedefault-as-the-default-seccomp-profile-for-all-workloads
> [4]: https://github.com/kubevirt/kubevirt/pull/8546

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 57b229012a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-01 14:32:05 +01:00
Scott Buckfelder
72b9ba4f24
[20.10] vendor: github.com/moby/libnetwork dcdf8f176d1e13ad719e913e796fb698d846de98
Co-authored-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Scott Buckfelder <buckscot@amazon.com>
2022-11-23 20:26:17 +01:00
Sebastiaan van Stijn
6dfa15b24b
Merge pull request #44472 from thaJeztah/20.10_backport_containerd_v1.6.10
[20.10 backport] update containerd binary to v1.6.10
2022-11-17 10:46:40 +01:00
Jintao Zhang
7ebfee940f
update containerd binary to v1.6.10
Signed-off-by: Jintao Zhang <zhangjintao9020@gmail.com>
(cherry picked from commit a5979a2106)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-16 14:04:59 +01:00
Samuel Karp
4ed81ac0e2
Merge pull request #44412 from thaJeztah/20.10_bump_go_1.18.8
fixes https://github.com/golang/go/issues/56309
2022-11-08 19:13:48 -08:00
Sebastiaan van Stijn
5ea9cf68ae
Merge pull request #44413 from thaJeztah/20.10_backport_oci_artifacts_error
[20.10 backport] distribution: Error when pulling OCI artifacts
2022-11-07 14:27:34 +01:00
Paweł Gronowski
f292bf0f4f
distribution: Error when pulling OCI artifacts
Currently an attempt to pull a reference which resolves to an OCI
artifact (Helm chart for example), results in a bit unrelated error
message `invalid rootfs in image configuration`.

This provides a more meaningful error in case a user attempts to
download a media type which isn't image related.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 407e3a4552)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-05 18:20:18 +01:00
Sebastiaan van Stijn
0211f9e44d
[20.10] update to Go 1.18.8 to address CVE-2022-41716
On Windows, syscall.StartProcess and os/exec.Cmd did not properly
    check for invalid environment variable values. A malicious
    environment variable value could exploit this behavior to set a
    value for a different environment variable. For example, the
    environment variable string "A=B\x00C=D" set the variables "A=B" and
    "C=D".

    Thanks to RyotaK (https://twitter.com/ryotkak) for reporting this
    issue.

    This is CVE-2022-41716 and Go issue https://go.dev/issue/56284.

This Go release also fixes https://github.com/golang/go/issues/56309, a
runtime bug which can cause random memory corruption when a goroutine
exits with runtime.LockOSThread() set. This fix is necessary to unblock
work to replace certain uses of pkg/reexec with unshared OS threads.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-05 17:55:06 +01:00
Sebastiaan van Stijn
3056208812
Merge pull request #44358 from thaJeztah/20.10_backport_busybox_w32_img
[20.10 backport] integration: download busybox-w32 from GitHub Release
2022-10-25 07:44:15 -04:00
CrazyMax
f99cb8297b
integration: download busybox-w32 from GitHub Release
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 4f1d1422de)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-24 17:06:12 -04:00
Sebastiaan van Stijn
24aa4c9c43
Merge pull request #44353 from thaJeztah/20.10_backport_update_containerd_binary
[20.10 backport] update containerd binary to v1.6.9
2022-10-24 15:50:29 -04:00
Sebastiaan van Stijn
3f9dc25f5c
update containerd binary to v1.6.9
release notes: https://github.com/containerd/containerd/releases/tag/v1.6.9

full diff: containerd/containerd@v1.6.8...v1.6.9

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit ac79a02ace)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-24 13:58:16 -04:00
Tianon Gravi
753cd92a31
Merge pull request #44326 from thaJeztah/20.10_bump_term_ansiterm
[20.10 backport] vendor: moby/term, Azure/go-ansiterm for golang.org/x/sys/windows compatibility
2022-10-22 00:28:04 +00:00
Sebastiaan van Stijn
82f583c8af
Merge pull request #44339 from crazy-max/20.10_update_buildkit
[20.10] vendor: github.com/moby/buildkit eeb7b65ab7d651770a5ec52a06ea7c96eb97a249 (v0.8 branch)
2022-10-22 00:16:12 +02:00
Sebastiaan van Stijn
87ccd38cea
vendor: moby/term, Azure/go-ansiterm for golang.org/x/sys/windows compatibility
- winterm: GetStdFile(): Added compatibility with "golang.org/x/sys/windows"
- winterm: fix GetStdFile() falltrough
- update deprecation message to refer to the correct replacement
- add go.mod
- Fix int overflow
- Convert int to string using rune()

full diff:

- bea5bbe245...3f7ff695ad
- d6e3b3328b...d185dfc1b5

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit af1e74555a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-21 19:43:02 +02:00
CrazyMax
e83e465ae2
[20.10] vendor: github.com/moby/buildkit eeb7b65ab7d651770a5ec52a06ea7c96eb97a249 (v0.8 branch)
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-21 17:39:46 +02:00
Sebastiaan van Stijn
e3fef9aea9
Merge pull request #44340 from thaJeztah/20.10_backport_skip_tests
[20.10 backport] skip TestImagePullStoredfDigestForOtherRepo() on Windows and rootless
2022-10-21 11:30:04 +02:00
Sebastiaan van Stijn
9c84417c1b
skip TestImagePullStoredfDigestForOtherRepo() on Windows and rootless
- On Windows, we don't build and run a local  test registry (we're not running
  docker-in-docker), so we need to skip this test.
- On rootless, networking doesn't support this (currently)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 4f43cb660a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-21 01:52:32 +02:00
Sebastiaan van Stijn
93c2c3bc60
Merge pull request #44332 from corhere/backport-20.10/fix-git-windows-environ
[20.10] builder: fix running git commands on Windows
2022-10-20 21:16:42 +02:00
Cory Snider
5b5b5c6f13 builder: add missing doc comment
Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-10-20 14:03:36 -04:00
Cory Snider
05e25f7892 builder: fix running git commands on Windows
Setting cmd.Env overrides the default of passing through the parent
process' environment, which works out fine most of the time, except when
it doesn't. For whatever reason, leaving out all the environment causes
git-for-windows sh.exe subprocesses to enter an infinite loop of
access violations during Cygwin initialization in certain environments
(specifically, our very own dev container image).

Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-10-20 13:41:42 -04:00
Sebastiaan van Stijn
2b7f97a974
Merge pull request #44313 from tianon/20.10-distributable
[20.10 backport] registry: allow "allow-nondistributable-artifacts" for Docker Hub
2022-10-19 23:27:58 +02:00
Sebastiaan van Stijn
03df974ae9
Merge pull request #6 from moby/ghsa-ambiguous-pull-by-digest
[20.10] Validate digest in repo for pull by digest
2022-10-18 19:26:23 +02:00
Sebastiaan van Stijn
3adff51f85
Merge pull request #9 from moby/update_buildkit
[20.10] vendor moby/buildkit v0.8.3-31-gc0149372
2022-10-18 19:17:40 +02:00
Sebastiaan van Stijn
d7c33ad827
Merge pull request #8 from moby/20.10_fix_git_file_leak
[20.10 backport] builder: Isolate Git from local system
2022-10-18 19:08:22 +02:00
Sebastiaan van Stijn
2f3bf18014
[20.10] vendor moby/buildkit v0.8.3-31-gc0149372
full diff: 3a1eeca59a...c014937225

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-18 10:12:37 +02:00
Sebastiaan van Stijn
6699afa549 registry: allow "allow-nondistributable-artifacts" for Docker Hub
Previously, Docker Hub was excluded when configuring "allow-nondistributable-artifacts".
With the updated policy announced by Microsoft, we can remove this restriction;
https://techcommunity.microsoft.com/t5/containers/announcing-windows-container-base-image-redistribution-rights/ba-p/3645201

There are plans to deprecated support for foreign layers altogether in the OCI,
and we should consider to make this option the default, but as that requires
deprecating the option (and possibly keeping an "opt-out" option), we can look
at that separately.

(cherry picked from commit 30e5333ce3)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Tianon Gravi <admwiggin@gmail.com>
2022-10-17 15:12:13 -07:00
Brian Goff
4b9902bad4
Validate digest in repo for pull by digest
This is accomplished by storing the distribution source in the content
labels. If the distribution source is not found then we check to the
registry to see if the digest exists in the repo, if it does exist then
the puller will use it.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-17 17:47:46 +02:00
Cory Snider
c0d1188c14 builder: make git config isolation opt-in
While it is undesirable for the system or user git config to be used
when the daemon clones a Git repo, it could break workflows if it was
unconditionally applied to docker/cli as well.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-10-13 17:34:23 -04:00
Cory Snider
9f5f3abcee builder: isolate git from local system
Prevent git commands we run from reading the user or system
configuration, or cloning submodules from the local filesystem.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-10-13 17:34:20 -04:00
Cory Snider
10db4c2db7 builder: explicitly set CWD for all git commands
Keep It Simple! Set the working directory for git commands by...setting
the git process's working directory. Git commands can be run in the
parent process's working directory by passing the empty string.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-10-13 17:34:20 -04:00
Cory Snider
8816c3c2aa builder: modernize TestCheckoutGit
Make the test more debuggable by logging all git command output and
running each table-driven test case as a subtest.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-10-13 17:34:20 -04:00
Sebastiaan van Stijn
c964641a0d
Merge pull request #44122 from thaJeztah/20.10_bump_buildkit
[20.10] vendor: github.com/moby/buildkit 3a1eeca59a9263613d996ead67d53a4b7d45723d (v0.8 branch)
2022-10-06 20:53:28 +02:00
Brian Goff
3bb99735b6
Merge pull request #44127 from thaJeztah/20.10_backport_image_spec_no_literal
[20.10] Update uses of Image platform fields in OCI image-spec
2022-10-06 11:18:55 -07:00
Sebastiaan van Stijn
435c40ce15
Merge pull request #44247 from thaJeztah/20.10_bump_go_1.18.7
[20.10] Update to go 1.18.7 to address CVE-2022-2879, CVE-2022-2880, CVE-2022-41715
2022-10-06 20:13:47 +02:00
Sebastiaan van Stijn
11bdbf40b9
[20.10] Update to go 1.18.7 to address CVE-2022-2879, CVE-2022-2880, CVE-2022-41715
From the mailing list:

We have just released Go versions 1.19.2 and 1.18.7, minor point releases.

These minor releases include 3 security fixes following the security policy:

- archive/tar: unbounded memory consumption when reading headers

  Reader.Read did not set a limit on the maximum size of file headers.
  A maliciously crafted archive could cause Read to allocate unbounded
  amounts of memory, potentially causing resource exhaustion or panics.
  Reader.Read now limits the maximum size of header blocks to 1 MiB.

  Thanks to Adam Korczynski (ADA Logics) and OSS-Fuzz for reporting this issue.

  This is CVE-2022-2879 and Go issue https://go.dev/issue/54853.

- net/http/httputil: ReverseProxy should not forward unparseable query parameters

  Requests forwarded by ReverseProxy included the raw query parameters from the
  inbound request, including unparseable parameters rejected by net/http. This
  could permit query parameter smuggling when a Go proxy forwards a parameter
  with an unparseable value.

  ReverseProxy will now sanitize the query parameters in the forwarded query
  when the outbound request's Form field is set after the ReverseProxy.Director
  function returns, indicating that the proxy has parsed the query parameters.
  Proxies which do not parse query parameters continue to forward the original
  query parameters unchanged.

  Thanks to Gal Goldstein (Security Researcher, Oxeye) and
  Daniel Abeles (Head of Research, Oxeye) for reporting this issue.

  This is CVE-2022-2880 and Go issue https://go.dev/issue/54663.

- regexp/syntax: limit memory used by parsing regexps

  The parsed regexp representation is linear in the size of the input,
  but in some cases the constant factor can be as high as 40,000,
  making relatively small regexps consume much larger amounts of memory.

  Each regexp being parsed is now limited to a 256 MB memory footprint.
  Regular expressions whose representation would use more space than that
  are now rejected. Normal use of regular expressions is unaffected.

  Thanks to Adam Korczynski (ADA Logics) and OSS-Fuzz for reporting this issue.

  This is CVE-2022-41715 and Go issue https://go.dev/issue/55949.

View the release notes for more information: https://go.dev/doc/devel/release#go1.18.7

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-04 22:20:50 +02:00
Sebastiaan van Stijn
35eaf7ecc4
Merge pull request #44238 from cpuguy83/20.10_fix_restore_volumerefs
[20.10] Fix live-restore w/ restart policies + volume refs
2022-10-03 23:32:53 +02:00
Brian Goff
66ddb7f91c Fix live-restore w/ restart policies + volume refs
Before this change restarting the daemon in live-restore with running
containers + a restart policy meant that volume refs were not restored.
This specifically happens when the container is still running *and*
there is a restart policy that would make sure the container was running
again on restart.

The bug allows volumes to be removed even though containers are
referencing them. 😱

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit 4c0e0979b4)
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2022-10-03 18:41:34 +00:00
Sebastiaan van Stijn
f219cb5ded
Merge pull request #44218 from thaJeztah/20.10_backport_more_robust_rootless
[20.10 backport] docker-rootless-setuptools.sh fixes
2022-09-29 22:09:04 +02:00
Sebastiaan van Stijn
c003392582
contrib: make dockerd-rootless-setuptool.sh more robust
The `docker` CLI currently doesn't handle situations where the current context
(as defined in `~/.docker/config.json`) is invalid or doesn't exist. As loading
(and checking) the context happens during initialization of the CLI, this
prevents `docker context` commands from being used, which makes it complicated
to fix the situation. For example, running `docker context use <correct context>`
would fail, which makes it not possible to update the `~/.docker/config.json`,
unless doing so manually.

For example, given the following `~/.docker/config.json`:

```json
{
        "currentContext": "nosuchcontext"
}
```

All of the commands below fail:

```bash
docker context inspect rootless
Current context "nosuchcontext" is not found on the file system, please check your config file at /Users/thajeztah/.docker/config.json

docker context rm --force rootless
Current context "nosuchcontext" is not found on the file system, please check your config file at /Users/thajeztah/.docker/config.json

docker context use default
Current context "nosuchcontext" is not found on the file system, please check your config file at /Users/thajeztah/.docker/config.json
```

While these things should be fixed, this patch updates the script to switch
the context using the `--context` flag; this flag is taken into account when
initializing the CLI, so that having an invalid context configured won't
block `docker context` commands from being executed. Given that all `context`
commands are local operations, "any" context can be used (it doesn't need to
make a connection with the daemon).

With this patch, those commands can now be run (and won't fail for the wrong
reason);

```bash
 docker --context=default context inspect -f "{{.Name}}" rootless
rootless

docker --context=default context inspect -f "{{.Name}}" rootless-doesnt-exist
context "rootless-doesnt-exist" does not exist
```

One other issue may also cause things to fail during uninstall; trying to remove
a context that doesn't exist will fail (even with the `-f` / `--force` option
set);

```bash
docker --context=default context rm blablabla
Error: context "blablabla": not found
```

While this is "ok" in most circumstances, it also means that (potentially) the
current context is not reset to "default", so this patch adds an explicit
`docker context use`, as well as unsetting the `DOCKER_HOST` and `DOCKER_CONTEXT`
environment variables.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit e2114731e7)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-29 10:05:56 +02:00
Mathieu PATUREL
53313be0f3
docker-rootless-setuptools.sh: use context after install
Signed-off-by: Mathieu PATUREL <mathieu.paturel@gmail.com>
(cherry picked from commit 7c17ad8735)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-29 10:05:51 +02:00
Sebastiaan van Stijn
4163c55082
Merge pull request #44202 from crazy-max/20.10_backport_api-fix-logo
[20.10 backport] swagger: update links to logo
2022-09-27 17:54:20 +02:00
CrazyMax
9c486bd267
swagger: update links to logo
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 7f3602f1c9)
2022-09-27 11:57:16 +02:00
Sebastiaan van Stijn
956161115f
Merge pull request #44167 from thaJeztah/20.10_backport_update_containerd_console
[20.10 backport] vendor: github.com/containerd/console v1.0.2
2022-09-22 20:16:05 +02:00
Sebastiaan van Stijn
668ab11a67
Merge pull request #44145 from neersighted/bump_memberlist_20.10
[20.10] vendor: github.com/hashicorp/memberlist v0.4.0
2022-09-21 17:51:18 +02:00
Sebastiaan van Stijn
fa17fab895
vendor: github.com/containerd/console v1.0.2
Contains fix for s390x and support for FreeBSD

full diff: https://github.com/containerd/console/compare/v1.0.1...v1.0.2

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 948e201c1c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-21 16:41:46 +02:00
Bjorn Neergaard
481bee51b5 vendor: github.com/armon/go-metrics v0.4.1
from github.com/hashicorp/memberlist v0.4.0 go.mod

Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
2022-09-21 08:12:24 -06:00
Bjorn Neergaard
39ba2873e8 vendor: github.com/google/btree v1.1.2
from github.com/hashicorp/memberlist v0.4.0 go.mod

Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
2022-09-21 08:12:24 -06:00
Bjorn Neergaard
c2755f40cd vendor: github.com/hasicorp/memberlist v0.4.0
Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
2022-09-21 08:12:24 -06:00
Sebastiaan van Stijn
cc3aa33f54
Merge pull request #44123 from linux-on-ibm-z/20.10-CI
[20.10] Dockerfile: switch to bullseye, add missing libseccomp-dev, remove build pack
2022-09-21 14:41:13 +02:00
Prabhav Thali
5ba3208ec7 Dockerfile: Update Dockerfile syntax, switch to bullseye, add missing libseccomp-dev, remove build pack
Signed-off-by: Prabhav Thali <Prabhav.Thali1@ibm.com>
2022-09-20 16:18:21 +05:30
Sebastiaan van Stijn
6d6a236286
[20.10] Update uses of Image platform fields in OCI image-spec
The OCI image spec is considering to change the Image struct and embedding the
Platform type (see opencontainers/image-spec#959) in the go implementation.
Moby currently uses some struct-literals to propagate the platform fields,
which will break once those changes in the OCI spec are merged.

Ideally (once that change arrives) we would update the code to set the Platform
information as a whole, instead of assigning related fields individually, but
in some cases in the code, image platform information is only partially set
(for example, OSVersion and OSFeatures are not preserved in all cases). This
may be on purpose, so needs to be reviewed.

This patch keeps the current behavior (assigning only specific fields), but
removes the use of struct-literals to make the code compatible with the
upcoming changes in the image-spec module.

(similar to commit 3cb933db9d)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-10 12:25:17 +02:00
Sebastiaan van Stijn
2570784169
[20.10] vendor: github.com/moby/buildkit 3a1eeca59a9263613d996ead67d53a4b7d45723d (v0.8 branch)
- update to go1.18
- metadata: hold lock on storageitem update
- cache: avoid concurrent maps write on prune
- update containerd to latest of docker-20.10 branch

full diff: bc07b2b81b...3a1eeca59a

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-09 13:11:08 +02:00
Sebastiaan van Stijn
e42327a6d3
Merge pull request #44120 from thaJeztah/20.10_backport_update_tests
[20.10 backport] Update some tests for supplementary group permissions
2022-09-09 00:50:10 +02:00
Sebastiaan van Stijn
fcd4df906b
Update some tests for supplementary group permissions
Update tests checking for groups to adjust for new policy updated in
de7af816e7, which caused those tests
to fail:

    === FAIL: amd64.integration-cli TestDockerSwarmSuite/TestSwarmServiceWithGroup (1.94s)
    docker_cli_swarm_test.go:311: assertion failed: uid=0(root) gid=0(root) groups=0(root),10(wheel),29(audio),50(staff),777 (string) != uid=0(root) gid=0(root) groups=10(wheel),29(audio),50(staff),777
(string)
    --- FAIL: TestDockerSwarmSuite/TestSwarmServiceWithGroup (1.94s)

    === FAIL: amd64.integration-cli TestDockerCLIRunSuite/TestRunGroupAdd (0.41s)
    docker_cli_run_test.go:1091: expected output uid=0(root) gid=0(root) groups=10(wheel),29(audio),50(staff),777 received uid=0(root) gid=0(root) groups=0(root),10(wheel),29(audio),50(staff),777
    --- FAIL: TestDockerCLIRunSuite/TestRunGroupAdd (0.41s)

    === FAIL: amd64.integration-cli TestDockerCLIRunSuite/TestRunUserByIDZero (0.41s)
    docker_cli_run_test.go:790: expected daemon user got uid=0(root) gid=0(root) groups=0(root),10(wheel)
    --- FAIL: TestDockerCLIRunSuite/TestRunUserByIDZero (0.41s)

    === FAIL: amd64.integration-cli TestDockerCLIRunSuite (195.70s)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit c7e77dba7f)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-08 23:29:34 +02:00
Sebastiaan van Stijn
bb0197c829
Merge pull request from GHSA-rc4r-wh2q-q6c4
[20.10 backport] AdditionalGids must include effective group ID
2022-09-08 21:29:35 +02:00
Brian Goff
d348775bd5
Merge pull request #44109 from rumpl/20.10-fix-local-context
[20.10 backport] fix local context
2022-09-08 11:11:32 -07:00
Djordje Lukic
6a0186b357 Wrap local calls to the content and lease service
The wrapper sets the default namespace in the context if none is
provided, this is needed because we are calling these services directly
and not trough GRPC that has an interceptor to set the default namespace
to all calls.

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
(cherry picked from commit 878906630b)
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2022-09-08 16:56:51 +02:00
Sebastiaan van Stijn
b73e9c2c8a
Merge pull request #44098 from thaJeztah/20.10_backport_swagger_updates
[20.10 backport] assorted swagger updates in documentation
2022-09-07 20:47:06 +02:00
Samuel Karp
4855c28780
Merge pull request #44101 from thaJeztah/20.10_backport_bump_golang_1.18.6 2022-09-06 20:45:51 -07:00
Sebastiaan van Stijn
3d4616f943
Update to go 1.18.6 to address CVE-2022-27664, CVE-2022-32190
From the mailing list:

We have just released Go versions 1.19.1 and 1.18.6, minor point releases.
These minor releases include 2 security fixes following the security policy:

- net/http: handle server errors after sending GOAWAY
  A closing HTTP/2 server connection could hang forever waiting for a clean
  shutdown that was preempted by a subsequent fatal error. This failure mode
  could be exploited to cause a denial of service.

  Thanks to Bahruz Jabiyev, Tommaso Innocenti, Anthony Gavazzi, Steven Sprecher,
  and Kaan Onarlioglu for reporting this.

  This is CVE-2022-27664 and Go issue https://go.dev/issue/54658.

- net/url: JoinPath does not strip relative path components in all circumstances
  JoinPath and URL.JoinPath would not remove `../` path components appended to a
  relative path. For example, `JoinPath("https://go.dev", "../go")` returned the
  URL `https://go.dev/../go`, despite the JoinPath documentation stating that
  `../` path elements are cleaned from the result.

  Thanks to q0jt for reporting this issue.

  This is CVE-2022-32190 and Go issue https://go.dev/issue/54385.

Release notes:

go1.18.6 (released 2022-09-06) includes security fixes to the net/http package,
as well as bug fixes to the compiler, the go command, the pprof command, the
runtime, and the crypto/tls, encoding/xml, and net packages. See the Go 1.18.6
milestone on the issue tracker for details;

https://github.com/golang/go/issues?q=milestone%3AGo1.18.6+label%3ACherryPickApproved

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit cba36a064d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-06 22:25:44 +02:00
Sebastiaan van Stijn
d56101e0d2
Merge pull request #44093 from thaJeztah/20.10_backport_remove_tereshkova
[20.10] cleanup namesgenerator
2022-09-06 21:57:49 +02:00
Niel Drummond
23c7d84b84
docs: api: adjust ContainerWaitResponse error as optional
Signed-off-by: Niel Drummond <niel@drummond.lu>
(cherry picked from commit 2359da802b)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-06 17:07:53 +02:00
Sebastiaan van Stijn
3e9e79d134
docs: api: document ImageSummary fields (api v1.39-v1.41)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit b9de761c26)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-06 17:07:22 +02:00
Sebastiaan van Stijn
fdd438ae03
api: docs: improve documentation of ContainerConfig type (API v1.30-v1.41)
ContainerConfig is used in multiple locations (for example, both for
Image.Config and Image.ContainerConfig). Unfortunately, swagger does
not allow documenting individual uses if a type is used; for this type,
the content is _optional_ when used as Image.ContainerConfig (which is
set by the classic builder, which does a "commit" of a container, but
not used when building an image with BuildKit).

This patch attempts to address this confusion by documenting that
"it may be empty (or fields not propagated) if it's used for the
Image.ContainerConfig field".

Perhaps alternatives are possible (aliasing the type?) but we can
look at those in a follow-up.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 07dba5d9fe)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-06 17:06:55 +02:00
Sebastiaan van Stijn
97014a8db5
namesgenerator: remove Valentina Tereshkova
While the name generator has been frozen for new additions in 624b3cfbe8,
this person has become controversial. Our intent is for this list to be inclusive
and non-controversial.

This patch removes the name from the list.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 0f052eb4f5)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-06 13:57:50 +02:00
Nicolas De Loof
e44d7f735e
AdditionalGids must include effective group ID
otherwise this one won't be considered for permission checks

Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
(cherry picked from commit 25345f2c04)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-06 12:25:37 +02:00
Sebastiaan van Stijn
b0d57e0ed4
Merge pull request #44067 from thaJeztah/20.10_update_containerd_vendor
[20.10] vendor: update containerd to latest of docker-20.10 branch
2022-09-04 17:14:53 +02:00
Sebastiaan van Stijn
9e7662e4a7
[20.10] vendor: update containerd to latest of docker-20.10 branch
This brings the containerd vendoring up-to-date with the latest changes from
the docker-20.10 branch in our fork (https://github.com/moby/containerd). This
adds some fixes that were included in another fork that was used in the BuildKit
repository, which have now been ported to our fork as well.

Relevant changes:

- docker: avoid concurrent map access panic
- overlay: support "userxattr" option (kernel 5.11) (does not affect vendored code)

full diff: 7cfa023d95...96c5ae04b6

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-02 02:14:23 +02:00
Akihiro Suda
f7f1f960ea
Merge pull request #44081 from thaJeztah/20.10_backport_tarsplit
[20.10 backport] vendor: update tar-split to v0.11.2
2022-09-02 01:40:47 +09:00
Tonis Tiigi
7dac25a3a9
vendor: update tar-split to v0.11.2
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit 21faae85ee)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-01 17:11:09 +02:00
Sebastiaan van Stijn
d346080204
Merge pull request #44041 from thaJeztah/20.10_backport_ontainerd_binary_1.6.8
[20.10 backport] update containerd binary to v1.6.8
2022-08-26 13:07:00 +02:00
Sebastiaan van Stijn
1c7b0dabc5
Merge pull request #44038 from thaJeztah/20.10_backport_update_runc_1.1.4
[20.10 backport] update runc to v1.1.4
2022-08-26 13:06:03 +02:00
Sebastiaan van Stijn
8bd86a0699
update containerd binary to v1.6.8
release notes: https://github.com/containerd/containerd/releases/tag/v1.6.8

full diff: https://github.com/containerd/containerd/compare/v1.6.7...v1.6.8

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit d52ffce38f)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-26 00:36:23 +02:00
Akihiro Suda
6c8dd6a6f2
update runc to v1.1.4
release notes: https://github.com/opencontainers/runc/releases/tag/v1.1.4

full diff: https://github.com/opencontainers/runc/compare/v1.1.3...v1.1.4

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit bd98bf38e9)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-26 00:25:00 +02:00
Samuel Karp
0bd91f300e
Merge pull request #44018 from thaJeztah/20.10_backport_healthcheck_timeout 2022-08-25 10:20:59 -07:00
Cory Snider
418c141e64
[20.10 backport] daemon: kill exec process on ctx cancel
Terminating the exec process when the context is canceled has been
broken since Docker v17.11 so nobody has been able to depend upon that
behaviour in five years of releases. We are thus free from backwards-
compatibility constraints.

conflicts:

- minor conflict in daemon/exec.go, as 2ec2b65e45
  is not in the 20.10 branch, so had to cast the signal to an int.
- minor conflict in daemon/health.go, where a comment was updated, which was
  added in bdc6473d2d, which is not in the
  20.10 branch
- remove the skip.If() from TestHealthCheckProcessKilled, as the 20.10 branch
  is not testing on Windows with containerd (and the RuntimeIsWindowsContainerd
  does not exist), but kept a "FIXME" comment.

Co-authored-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Co-authored-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Signed-off-by: Cory Snider <csnider@mirantis.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 4b84a33217)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-23 22:30:01 +02:00
Sebastiaan van Stijn
62fd7187d0
Merge pull request #43991 from thaJeztah/20.10_seccomp_updates
[20.10 backport] seccomp updates
2022-08-18 21:28:19 +02:00
Sebastiaan van Stijn
7d4cc78c02
Merge pull request #43976 from thaJeztah/20.10_backport_bump_golang_1.18
[20.10 backport] update to golang 1.18.5
2022-08-18 21:27:25 +02:00
Djordje Lukic
d127287d92
Allow different syscalls from kernels 5.12 -> 5.16
Kernel 5.12:

    mount_setattr: needs CAP_SYS_ADMIN

Kernel 5.14:

    quotactl_fd: needs CAP_SYS_ADMIN
    memfd_secret: always allowed

Kernel 5.15:

    process_mrelease: always allowed

Kernel 5.16:

    futex_waitv: always allowed

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
(cherry picked from commit 7de9f4f82d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-18 18:58:09 +02:00
Tudor Brindus
57db169641
seccomp: add support for Landlock syscalls in default policy
This commit allows the Landlock[0] system calls in the default seccomp
policy.

Landlock was introduced in kernel 5.13, to fill the gap that inspecting
filepaths passed as arguments to filesystem system calls is not really
possible with pure `seccomp` (unless involving `ptrace`).

Allowing Landlock by default fits in with allowing `seccomp` for
containerized applications to voluntarily restrict their access rights
to files within the container.

[0]: https://www.kernel.org/doc/html/latest/userspace-api/landlock.html

Signed-off-by: Tudor Brindus <me@tbrindus.ca>
(cherry picked from commit af819bf623)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-18 18:55:16 +02:00
Sebastiaan van Stijn
7ba8ca042c
Update golang to 1.18.5
Update Go runtime to 1.18.5 to address CVE-2022-32189.

Full diff: https://github.com/golang/go/compare/go1.18.4...go1.18.5

--------------------------------------------------------

From the security announcement:
https://groups.google.com/g/golang-announce/c/YqYYG87xB10

We have just released Go versions 1.18.5 and 1.17.13, minor point
releases.

These minor releases include 1 security fixes following the security
policy:

encoding/gob & math/big: decoding big.Float and big.Rat can panic

Decoding big.Float and big.Rat types can panic if the encoded message is
too short.

This is CVE-2022-32189 and Go issue https://go.dev/issue/53871.

View the release notes for more information:
https://go.dev/doc/devel/release#go1.18.5

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit f1d71f7cc3)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-18 16:55:28 +02:00
Sebastiaan van Stijn
f2a3c3bcef
update golang to 1.18.4
go1.18.4 (released 2022-07-12) includes security fixes to the compress/gzip,
encoding/gob, encoding/xml, go/parser, io/fs, net/http, and path/filepath
packages, as well as bug fixes to the compiler, the go command, the linker,
the runtime, and the runtime/metrics package. See the Go 1.18.4 milestone on the
issue tracker for details:

https://github.com/golang/go/issues?q=milestone%3AGo1.18.4+label%3ACherryPickApproved

This update addresses:

CVE-2022-1705, CVE-2022-1962, CVE-2022-28131, CVE-2022-30630, CVE-2022-30631,
CVE-2022-30632, CVE-2022-30633, CVE-2022-30635, and CVE-2022-32148.

Full diff: https://github.com/golang/go/compare/go1.18.3...go1.18.4

From the security announcement;
https://groups.google.com/g/golang-announce/c/nqrv9fbR0zE

We have just released Go versions 1.18.4 and 1.17.12, minor point releases. These
minor releases include 9 security fixes following the security policy:

- net/http: improper sanitization of Transfer-Encoding header

  The HTTP/1 client accepted some invalid Transfer-Encoding headers as indicating
  a "chunked" encoding. This could potentially allow for request smuggling, but
  only if combined with an intermediate server that also improperly failed to
  reject the header as invalid.

  This is CVE-2022-1705 and https://go.dev/issue/53188.

- When `httputil.ReverseProxy.ServeHTTP` was called with a `Request.Header` map
  containing a nil value for the X-Forwarded-For header, ReverseProxy would set
  the client IP as the value of the X-Forwarded-For header, contrary to its
  documentation. In the more usual case where a Director function set the
  X-Forwarded-For header value to nil, ReverseProxy would leave the header
  unmodified as expected.

  This is https://go.dev/issue/53423 and CVE-2022-32148.

  Thanks to Christian Mehlmauer for reporting this issue.

- compress/gzip: stack exhaustion in Reader.Read

  Calling Reader.Read on an archive containing a large number of concatenated
  0-length compressed files can cause a panic due to stack exhaustion.

  This is CVE-2022-30631 and Go issue https://go.dev/issue/53168.

- encoding/xml: stack exhaustion in Unmarshal

  Calling Unmarshal on a XML document into a Go struct which has a nested field
  that uses the any field tag can cause a panic due to stack exhaustion.

  This is CVE-2022-30633 and Go issue https://go.dev/issue/53611.

- encoding/xml: stack exhaustion in Decoder.Skip

  Calling Decoder.Skip when parsing a deeply nested XML document can cause a
  panic due to stack exhaustion. The Go Security team discovered this issue, and
  it was independently reported by Juho Nurminen of Mattermost.

  This is CVE-2022-28131 and Go issue https://go.dev/issue/53614.

- encoding/gob: stack exhaustion in Decoder.Decode

  Calling Decoder.Decode on a message which contains deeply nested structures
  can cause a panic due to stack exhaustion.

  This is CVE-2022-30635 and Go issue https://go.dev/issue/53615.

- path/filepath: stack exhaustion in Glob

  Calling Glob on a path which contains a large number of path separators can
  cause a panic due to stack exhaustion.

  Thanks to Juho Nurminen of Mattermost for reporting this issue.

  This is CVE-2022-30632 and Go issue https://go.dev/issue/53416.

- io/fs: stack exhaustion in Glob

  Calling Glob on a path which contains a large number of path separators can
  cause a panic due to stack exhaustion.

  This is CVE-2022-30630 and Go issue https://go.dev/issue/53415.

- go/parser: stack exhaustion in all Parse* functions

  Calling any of the Parse functions on Go source code which contains deeply
  nested types or declarations can cause a panic due to stack exhaustion.

  Thanks to Juho Nurminen of Mattermost for reporting this issue.

  This is CVE-2022-1962 and Go issue https://go.dev/issue/53616.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 34b8670b1a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-18 16:55:27 +02:00
Sebastiaan van Stijn
a99c9cd852
update golang to 1.18.3
go1.18.3 (released 2022-06-01) includes security fixes to the crypto/rand,
crypto/tls, os/exec, and path/filepath packages, as well as bug fixes to the
compiler, and the crypto/tls and text/template/parse packages. See the Go
1.18.3 milestone on our issue tracker for details:

https://github.com/golang/go/issues?q=milestone%3AGo1.18.3+label%3ACherryPickApproved

Hello gophers,

We have just released Go versions 1.18.3 and 1.17.11, minor point releases.

These minor releases include 4 security fixes following the security policy:

- crypto/rand: rand.Read hangs with extremely large buffers
  On Windows, rand.Read will hang indefinitely if passed a buffer larger than
  1 << 32 - 1 bytes.

  Thanks to Davis Goodin and Quim Muntal, working at Microsoft on the Go toolset,
  for reporting this issue.

  This is [CVE-2022-30634][CVE-2022-30634] and Go issue https://go.dev/issue/52561.
- crypto/tls: session tickets lack random ticket_age_add
  Session tickets generated by crypto/tls did not contain a randomly generated
  ticket_age_add. This allows an attacker that can observe TLS handshakes to
  correlate successive connections by comparing ticket ages during session
  resumption.

  Thanks to GitHub user nervuri for reporting this.

  This is [CVE-2022-30629][CVE-2022-30629] and Go issue https://go.dev/issue/52814.
- `os/exec`: empty `Cmd.Path` can result in running unintended binary on Windows

  If, on Windows, `Cmd.Run`, `cmd.Start`, `cmd.Output`, or `cmd.CombinedOutput`
  are executed when Cmd.Path is unset and, in the working directory, there are
  binaries named either "..com" or "..exe", they will be executed.

  Thanks to Chris Darroch, brian m. carlson, and Mikhail Shcherbakov for reporting
  this.

  This is [CVE-2022-30580][CVE-2022-30580] and Go issue https://go.dev/issue/52574.
- `path/filepath`: Clean(`.\c:`) returns `c:` on Windows

  On Windows, the `filepath.Clean` function could convert an invalid path to a
  valid, absolute path. For example, Clean(`.\c:`) returned `c:`.

  Thanks to Unrud for reporting this issue.

  This is [CVE-2022-29804][CVE-2022-29804] and Go issue https://go.dev/issue/52476.

[CVE-2022-30634]: https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-30634
[CVE-2022-30629]: https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-30629
[CVE-2022-30580]: https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-30580
[CVE-2022-29804]: https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-29804

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit c1a9ffc97a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-18 16:55:25 +02:00
Sebastiaan van Stijn
82939f536b
update golang to 1.18.2
go1.18.2 (released 2022-05-10) includes security fixes to the syscall package,
as well as bug fixes to the compiler, runtime, the go command, and the crypto/x509,
go/types, net/http/httptest, reflect, and sync/atomic packages. See the Go 1.18.2
milestone on the issue tracker for details:

https://github.com/golang/go/issues?q=milestone%3AGo1.18.2+label%3ACherryPickApproved

Full diff: http://github.com/golang/go/compare/go1.18.1...go1.18.2

Includes fixes for:

- CVE-2022-29526 (http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-29526);
  (description at https://go.dev/issue/52313).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit d294078dd0)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-18 16:55:23 +02:00
Sebastiaan van Stijn
ecd1aa081f
update golang to 1.18.1
go1.18.1 (released 2022-04-12) includes security fixes to the crypto/elliptic,
crypto/x509, and encoding/pem packages, as well as bug fixes to the compiler,
linker, runtime, the go command, vet, and the bytes, crypto/x509, and go/types
packages. See the Go 1.18.1 milestone on the issue tracker for details:

https://github.com/golang/go/issues?q=milestone%3AGo1.18.1+label%3ACherryPickApproved

Includes fixes for:

- CVE-2022-24675 (https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-24675)
- CVE-2022-27536 (https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-27536)
- CVE-2022-28327 (https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-28327)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit ef64db5021)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-18 16:55:21 +02:00
Sebastiaan van Stijn
7ba67d05a8
[20.10] vendor: update archive/tar for go 1.18
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-18 16:54:03 +02:00
Sebastiaan van Stijn
0bc432241e
update golang to 1.18.0
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 590e34eaf4)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-18 08:44:23 +02:00
Sebastiaan van Stijn
bb95d09f9a
staticcheck: ignore "SA1019: strings.Title is deprecated"
This function is marked deprecated in Go 1.18; however, the suggested replacement
brings in a large amount of new code, and most strings we generate will be ASCII,
so this would only be in case it's used for some user-provided string. We also
don't have a language to use, so would be using the "default".

Adding a `//nolint` comment to suppress the linting failure instead.

    daemon/logger/templates/templates.go:23:14: SA1019: strings.Title is deprecated: The rule Title uses for word boundaries does not handle Unicode punctuation properly. Use golang.org/x/text/cases instead. (staticcheck)
        "title":    strings.Title,
                    ^
    pkg/plugins/pluginrpc-gen/template.go:67:9: SA1019: strings.Title is deprecated: The rule Title uses for word boundaries does not handle Unicode punctuation properly. Use golang.org/x/text/cases instead. (staticcheck)
        return strings.Title(s)
               ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 4203a97aad)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-18 08:44:20 +02:00
Sebastiaan van Stijn
32debe0986
Merge pull request #43975 from thaJeztah/20.10_backport_update_golangci_lint
[20.10 backport] Dockerfile: update golangci-lint v1.44.0
2022-08-18 08:43:53 +02:00
Sebastiaan van Stijn
a7299ae72c
Dockerfile: update golangci-lint v1.44.0
Looks like this may be needed for Go 1.18

Also updating the golangci-lint configuration to account for updated
exclusion rules.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 646ace6ee3)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-17 19:03:10 +02:00
Sebastiaan van Stijn
d97fd533cf
integration-cli: SA5011: possible nil pointer dereference (staticcheck)
I think the original intent here was to make passing t optional (62a856e912),
but it looks like that's not done anywhere, so let's remove it.

    integration-cli/docker_utils_test.go:81:2: SA5011: possible nil pointer dereference (staticcheck)
        c.Helper()
        ^
    integration-cli/docker_utils_test.go:84:5: SA5011(related information): this check suggests that the pointer can be nil (staticcheck)
        if c != nil {
           ^
    integration-cli/docker_utils_test.go:106:2: SA5011: possible nil pointer dereference (staticcheck)
        c.Helper()
        ^
    integration-cli/docker_utils_test.go:108:5: SA5011(related information): this check suggests that the pointer can be nil (staticcheck)
        if c != nil {
           ^
    integration-cli/docker_utils_test.go:116:2: SA5011: possible nil pointer dereference (staticcheck)
        c.Helper()
        ^
    integration-cli/docker_utils_test.go:118:5: SA5011(related information): this check suggests that the pointer can be nil (staticcheck)
        if c != nil {
           ^
    integration-cli/docker_utils_test.go:126:2: SA5011: possible nil pointer dereference (staticcheck)
        c.Helper()
        ^
    integration-cli/docker_utils_test.go:128:5: SA5011(related information): this check suggests that the pointer can be nil (staticcheck)
        if c != nil {
           ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 89f63f476b)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-17 19:03:09 +02:00
Sebastiaan van Stijn
e6aee04a88
client.NewClientWithOpts(): remove redundant type assertion (gosimple)
As caught by gosimple:

    client/client.go:138:14: S1040: type assertion to the same type: c.client.Transport already has type http.RoundTripper (gosimple)
        if _, ok := c.client.Transport.(http.RoundTripper); !ok {
                    ^

This check was originally added in dc9f5c2ca3, to
check if the passed option was a `http.Transport`, and later changed in
e345cd12f9 to check for `http.RoundTripper` instead.

Client.client is a http.Client, for which the Transport field is a RoundTripper,
so this check is redundant.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 99935ff803)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-17 19:03:06 +02:00
Sebastiaan van Stijn
0523323c28
daemon/logger/awslogs: suppress false positive on hardcoded creds (gosec)
daemon/logger/awslogs/cloudwatchlogs.go:42:2: G101: Potential hardcoded credentials (gosec)
        credentialsEndpointKey = "awslogs-credentials-endpoint"
        ^
    daemon/logger/awslogs/cloudwatchlogs.go:67:2: G101: Potential hardcoded credentials (gosec)
        credentialsEndpoint = "http://169.254.170.2"
        ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit b88f4e2604)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-17 19:03:04 +02:00
Sebastiaan van Stijn
adeb29c64c
client/request.go:157:8: SA1019: err.Temporary is deprecated (staticcheck)
It's deprecated in Go 1.18:

    client/request.go:157:8: SA1019: err.Temporary is deprecated: Temporary errors are not well-defined. Most "temporary" errors are timeouts, and the few exceptions are surprising. Do not use this method. (staticcheck)
        if !err.Temporary() {
            ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 2cff05e960)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-17 19:03:02 +02:00
Sebastiaan van Stijn
50361d91a6
registry: trimV1Address(): simplify trimming trailing slash
strings.TrimSuffix() does exactly the same as this code, but is
a bit more readable.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 4a52c46e37)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-17 19:02:54 +02:00
Sebastiaan van Stijn
ae3a9337dd
golangci.yml: do not limit max reported issues
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 594c972fc5)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-17 18:18:38 +02:00
Sebastiaan van Stijn
9820255a1c
golangci.yml: skip some tests
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 22ce0f8faa)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-17 18:16:06 +02:00
Sebastiaan van Stijn
d223f37300
golangci.yml: update regex for ignoring SA1019
The message changed from "is deprecated" to "has been deprecated":

    client/hijack.go:85:16: SA1019: httputil.NewClientConn has been deprecated since Go 1.0: Use the Client or Transport in package net/http instead. (staticcheck)
        clientconn := httputil.NewClientConn(conn, nil)
                      ^
    integration/plugin/authz/authz_plugin_test.go:180:7: SA1019: httputil.NewClientConn has been deprecated since Go 1.0: Use the Client or Transport in package net/http instead. (staticcheck)
        c := httputil.NewClientConn(conn, nil)
             ^
    integration/plugin/authz/authz_plugin_test.go:479:12: SA1019: httputil.NewClientConn has been deprecated since Go 1.0: Use the Client or Transport in package net/http instead. (staticcheck)
        client := httputil.NewClientConn(conn, nil)
                  ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit ea74765a58)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-17 18:13:03 +02:00
Sebastiaan van Stijn
ec3bfba89d
graphdriver: temporarily ignore unsafeptr: possible misuse of reflect.SliceHeader
Probably needs a similar change as c208f03fbd,
but this code makes my head spin, so for now suppressing, and created a
tracking issue:

    daemon/graphdriver/graphtest/graphtest_unix.go:305:12: unsafeptr: possible misuse of reflect.SliceHeader (govet)
        header := *(*reflect.SliceHeader)(unsafe.Pointer(&buf))
                  ^
    daemon/graphdriver/graphtest/graphtest_unix.go:308:36: unsafeptr: possible misuse of reflect.SliceHeader (govet)
        data := *(*[]byte)(unsafe.Pointer(&header))
                                          ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit e6dabfa977)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-17 18:12:51 +02:00
Sebastiaan van Stijn
f2f387b131
daemon: var-declaration: should omit type bool (revive)
daemon/list.go:556:18: var-declaration: should omit type bool from declaration of var shouldSkip; it will be inferred from the right-hand side (revive)
                shouldSkip    bool = true
                              ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit d61b7c1211)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-17 18:12:41 +02:00
Sebastiaan van Stijn
2fb7c9fea7
daemon/config: error strings should not be capitalized
daemon/config/config_unix.go:92:21: error-strings: error strings should not be capitalized or end with punctuation or a newline (revive)
            return fmt.Errorf("Default cgroup namespace mode (%v) is invalid. Use \"host\" or \"private\".", cm) // nolint: golint
                              ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 16ced7622b)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-17 18:12:30 +02:00
Sebastiaan van Stijn
fa6954cb98
reformat "nolint" comments
Unlike regular comments, nolint comments should not have a leading space.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit bb17074119)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-17 18:12:14 +02:00
Sebastiaan van Stijn
45fa675a35
if-return: redundant if ...; err != nil check (revive)
builder/builder-next/adapters/snapshot/snapshot.go:386:3: if-return: redundant if ...; err != nil check, just return error instead. (revive)
            if err := b.Put(keyIsCommitted, []byte{}); err != nil {
                return err
            }

    plugin/fetch_linux.go:112:2: if-return: redundant if ...; err != nil check, just return error instead. (revive)
        if err := images.Dispatch(ctx, images.Handlers(handlers...), nil, desc); err != nil {
            return err
        }

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit dd1374f7b2)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-17 18:12:02 +02:00
Sebastiaan van Stijn
9e88f8435a
daemon/splunk: ignore G402: TLS MinVersion too low for now
daemon/logger/splunk/splunk.go:173:16: G402: TLS MinVersion too low. (gosec)
    	tlsConfig := &tls.Config{}
    	              ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 4004a39d53)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-17 18:11:52 +02:00
Sebastiaan van Stijn
2de90ebbe4
pkg/archive: RebaseArchiveEntries(): ignore G110
pkg/archive/copy.go:357:16: G110: Potential DoS vulnerability via decompression bomb (gosec)
                if _, err = io.Copy(rebasedTar, srcTar); err != nil {
                            ^

Ignoring GoSec G110. See https://github.com/securego/gosec/pull/433
and https://cure53.de/pentest-report_opa.pdf, which recommends to
replace io.Copy with io.CopyN7. The latter allows to specify the
maximum number of bytes that should be read. By properly defining
the limit, it can be assured that a GZip compression bomb cannot
easily cause a Denial-of-Service.
After reviewing, this should not affect us, because here we do not
read into memory.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 7b071e0557)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-17 18:11:42 +02:00
Sebastiaan van Stijn
14b475d091
daemon/stats: fix notRunningErr / notFoundErr detected as unused (false positive)
Also looks like a false positive, but given that these were basically
testing for the `errdefs.Conflict` and `errdefs.NotFound` interfaces, I
replaced these with those;

    daemon/stats/collector.go:154:6: type `notRunningErr` is unused (unused)
    type notRunningErr interface {
         ^
    daemon/stats/collector.go:159:6: type `notFoundErr` is unused (unused)
    type notFoundErr interface {
         ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 09191c0936)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-17 18:11:31 +02:00
Sebastiaan van Stijn
db7b3f4737
unused: ignore false positives
integration/build/build_session_test.go:92:6: func `testBuildWithSession` is unused (unused)
    func testBuildWithSession(t *testing.T, client dclient.APIClient, daemonHost string, dir, dockerfile string) (outStr string) {
         ^
    integration/container/checkpoint_test.go:23:6: func `containerExec` is unused (unused)
    func containerExec(t *testing.T, client client.APIClient, cID string, cmd []string) {
         ^
    integration/network/service_test.go:295:6: func `swarmIngressReady` is unused (unused)
    func swarmIngressReady(client client.NetworkAPIClient) func(log poll.LogT) poll.Result {
         ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 7c91fd4240)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-17 18:11:20 +02:00
Sebastiaan van Stijn
b6de0ca7c5
G601: Implicit memory aliasing in for loop
daemon/cluster/executor/container/adapter.go:446:42: G601: Implicit memory aliasing in for loop. (gosec)
            req := c.container.volumeCreateRequest(&mount)
                                                   ^
    daemon/network.go:577:10: G601: Implicit memory aliasing in for loop. (gosec)
                np := &n
                      ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit b4c0c7c076)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-17 18:11:09 +02:00
Sebastiaan van Stijn
e8b838e99f
gosec: G601: Implicit memory aliasing in for loop
plugin/v2/plugin.go:141:50: G601: Implicit memory aliasing in for loop. (gosec)
                    updateSettingsEnv(&p.PluginObj.Settings.Env, &s)
                                                                 ^
    libcontainerd/remote/client.go:572:13: G601: Implicit memory aliasing in for loop. (gosec)
                cpDesc = &m
                         ^
    distribution/push_v2.go:400:34: G601: Implicit memory aliasing in for loop. (gosec)
                (metadata.CheckV2MetadataHMAC(&mountCandidate, pd.hmacKey) ||
                                              ^
    builder/dockerfile/builder.go:261:84: G601: Implicit memory aliasing in for loop. (gosec)
            currentCommandIndex = printCommand(b.Stdout, currentCommandIndex, totalCommands, &meta)
                                                                                             ^
    builder/dockerfile/builder.go:278:46: G601: Implicit memory aliasing in for loop. (gosec)
            if err := initializeStage(dispatchRequest, &stage); err != nil {
                                                       ^
    daemon/container.go:283:40: G601: Implicit memory aliasing in for loop. (gosec)
            if err := parser.ValidateMountConfig(&cfg); err != nil {
                                                 ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit d13997b4ba)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-17 18:10:55 +02:00
Sebastiaan van Stijn
2ddf6e598a
gosimple: S1039: unnecessary use of fmt.Sprintf
pkg/devicemapper/devmapper.go:383:28: S1039: unnecessary use of fmt.Sprintf (gosimple)
        if err := task.setMessage(fmt.Sprintf("@cancel_deferred_remove")); err != nil {
                                  ^
    integration/plugin/graphdriver/external_test.go:321:18: S1039: unnecessary use of fmt.Sprintf (gosimple)
                http.Error(w, fmt.Sprintf("missing id"), 409)
                              ^
    integration-cli/docker_api_stats_test.go:70:31: S1039: unnecessary use of fmt.Sprintf (gosimple)
            _, body, err := request.Get(fmt.Sprintf("/info"))
                                        ^
    integration-cli/docker_cli_build_test.go:4547:19: S1039: unnecessary use of fmt.Sprintf (gosimple)
                "--build-arg", fmt.Sprintf("FOO1=fromcmd"),
                               ^
    integration-cli/docker_cli_build_test.go:4548:19: S1039: unnecessary use of fmt.Sprintf (gosimple)
                "--build-arg", fmt.Sprintf("FOO2="),
                               ^
    integration-cli/docker_cli_build_test.go:4549:19: S1039: unnecessary use of fmt.Sprintf (gosimple)
                "--build-arg", fmt.Sprintf("FOO3"), // set in env
                               ^
    integration-cli/docker_cli_build_test.go:4668:32: S1039: unnecessary use of fmt.Sprintf (gosimple)
            cli.WithFlags("--build-arg", fmt.Sprintf("tag=latest")))
                                         ^
    integration-cli/docker_cli_build_test.go:4690:32: S1039: unnecessary use of fmt.Sprintf (gosimple)
            cli.WithFlags("--build-arg", fmt.Sprintf("baz=abc")))
                                         ^
    pkg/jsonmessage/jsonmessage_test.go:255:4: S1039: unnecessary use of fmt.Sprintf (gosimple)
                fmt.Sprintf("ID: status\n"),
                ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit f77213efc2)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-17 18:10:32 +02:00
Sebastiaan van Stijn
fadf8bbdff
staticcheck: SA4001: &*x will be simplified to x. It will not copy x
daemon/volumes_unix_test.go:228:13: SA4001: &*x will be simplified to x. It will not copy x. (staticcheck)
                mp:      &(*c.MountPoints["/jambolan"]), // copy the mountpoint, expect no changes
                         ^
    daemon/logger/local/local_test.go:214:22: SA4001: &*x will be simplified to x. It will not copy x. (staticcheck)
            dst.PLogMetaData = &(*src.PLogMetaData)
                               ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit f7433d6190)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-17 18:10:19 +02:00
Sebastiaan van Stijn
7573e32577
client: S1031: unnecessary nil check around range (gosimple)
client/request.go:245:2: S1031: unnecessary nil check around range (gosimple)
        if headers != nil {
        ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit b92be7e297)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-17 18:10:06 +02:00
Sebastiaan van Stijn
e738a57a6d
daemon/logger/journald: fix linting errors
daemon/logger/journald/read.go:128:3 comment on exported function `CErr` should be of the form `CErr ...`

    daemon/logger/journald/read.go:131:36: unnecessary conversion (unconvert)
            return C.GoString(C.strerror(C.int(-ret)))
	                                  ^
    daemon/logger/journald/read.go:380:2: S1023: redundant `return` statement (gosimple)
        return
        ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit d43bcc8974)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-17 18:09:54 +02:00
Sebastiaan van Stijn
34f6b94255
gosec: G404: Use of weak random number generator
These should be ok to ignore for the purpose they're used

    pkg/namesgenerator/names-generator.go:843:36: G404: Use of weak random number generator (math/rand instead of crypto/rand) (gosec)
        name := fmt.Sprintf("%s_%s", left[rand.Intn(len(left))], right[rand.Intn(len(right))])
                                          ^
    pkg/namesgenerator/names-generator.go:849:36: G404: Use of weak random number generator (math/rand instead of crypto/rand) (gosec)
            name = fmt.Sprintf("%s%d", name, rand.Intn(10))
                                             ^
    testutil/stringutils.go:11:18: G404: Use of weak random number generator (math/rand instead of crypto/rand) (gosec)
            b[i] = letters[rand.Intn(len(letters))]
                           ^
    pkg/namesgenerator/names-generator.go:849:36: G404: Use of weak random number generator (math/rand instead of crypto/rand) (gosec)
            name = fmt.Sprintf("%s%d", name, rand.Intn(10))
                                             ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 6b0ecacd92)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-17 18:09:41 +02:00
Sebastiaan van Stijn
a77093b4fe
Merge pull request #43911 from thaJeztah/20.10_backport_buildkit_api_docs
[20.10 backport] api: swagger: document BuildCache fields (API v1.39-v1.41)
2022-08-08 16:52:54 +02:00
Tianon Gravi
5e6d42aa9d
Merge pull request #43918 from thaJeztah/20.10_backport_bump_containerd_1.6.7_binary
[20.10 backport] update containerd binary to v1.6.7, runc to v1.3.1
2022-08-05 15:09:01 -07:00
Tianon Gravi
a345492301
Merge pull request #43859 from olljanat/20.10_backport_restore-custom-nat-networks
[20.10 backport] Windows: Re-create custom NAT networks after restart if missing from HNS
2022-08-05 09:22:11 -07:00
Sebastiaan van Stijn
a6d7b61c8b
update containerd binary to v1.6.7
full diff: https://github.com/containerd/containerd/v1.6.6...v1.6.7

Welcome to the v1.6.7 release of containerd!

The seventh patch release for containerd 1.6 contains various fixes,
includes a new version of runc and adds support for ppc64le and riscv64
(requires unreleased runc 1.2) builds.

Notable Updates

- Update runc to v1.1.3
- Seccomp: Allow clock_settime64 with CAP_SYS_TIME
- Fix WWW-Authenticate parsing
- Support RISC-V 64 and ppc64le builds
- Windows: Update hcsshim to v0.9.4 to fix regression with HostProcess stats
- Windows: Fix shim logs going to panic.log file
- Allow ptrace(2) by default for kernels >= 4.8

See the changelog for complete list of changes

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 4e46d9f963)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-05 00:20:28 +02:00
Sebastiaan van Stijn
b4ba1ee22f
update runc binary to v1.1.3
full diff: https://github.com/opencontainers/runc/compare/v1.1.2...v1.1.3

This is the third release of the 1.1.z series of runc, and contains
various minor improvements and bugfixes.

- Our seccomp `-ENOSYS` stub now correctly handles multiplexed syscalls on
  s390 and s390x. This solves the issue where syscalls the host kernel did not
  support would return `-EPERM` despite the existence of the `-ENOSYS` stub
  code (this was due to how s390x does syscall multiplexing).
- Retry on dbus disconnect logic in libcontainer/cgroups/systemd now works as
  intended; this fix does not affect runc binary itself but is important for
  libcontainer users such as Kubernetes.
- Inability to compile with recent clang due to an issue with duplicate
  constants in libseccomp-golang.
- When using systemd cgroup driver, skip adding device paths that don't exist,
  to stop systemd from emitting warnings about those paths.
- Socket activation was failing when more than 3 sockets were used.
- Various CI fixes.
- Allow to bind mount `/proc/sys/kernel/ns_last_pid` to inside container.
- runc static binaries are now linked against libseccomp v2.5.4.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 2293de1c82)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-05 00:20:16 +02:00
Sebastiaan van Stijn
195761c1fa
Merge pull request #43889 from thaJeztah/20.10_backport_api_fix_missing_platform
[20.10 backport] api: swagger: add missing "platform" query-arg on create
2022-08-04 21:00:31 +02:00
Sebastiaan van Stijn
da8828c4b3
api: swagger: fix invalid example value (API v1.39-v1.41)
This was introduced in 43956c1bfc

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 30295c1750)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-04 17:54:11 +02:00
Sebastiaan van Stijn
1c0ca366dd
Merge pull request #43905 from thaJeztah/20.10_bump_golang_1.17.13
[20.10] Update golang to 1.17.13
2022-08-04 16:53:43 +02:00
Sebastiaan van Stijn
9501d91e19
api: swagger: document BuildCache fields (API v1.39-v1.41)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 43956c1bfc)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-04 12:52:00 +02:00
Sebastiaan van Stijn
61fdea902b
api: swagger: document BuildCache fields.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit dc2b34af6a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-04 12:51:56 +02:00
Sebastiaan van Stijn
c77432c889
[20.10] Update golang to 1.17.13
Update Go runtime to 1.17.13 to address CVE-2022-32189.

Full diff: https://github.com/golang/go/compare/go1.17.12...go1.17.13

--------------------------------------------------------

From the security announcement:
https://groups.google.com/g/golang-announce/c/YqYYG87xB10

We have just released Go versions 1.18.5 and 1.17.13, minor point
releases.

These minor releases include 1 security fixes following the security
policy:

encoding/gob & math/big: decoding big.Float and big.Rat can panic

Decoding big.Float and big.Rat types can panic if the encoded message is
too short.

This is CVE-2022-32189 and Go issue https://go.dev/issue/53871.

View the release notes for more information:
https://go.dev/doc/devel/release#go1.17.13

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-03 20:15:00 +02:00
Sebastiaan van Stijn
2833aa1e4b
docs: api: add missing "platform" query-arg on create (v1.41)
Commit 7a9cb29fb9 added a new "platform" query-
parameter to the `POST /containers/create` endpoint, but did not update the
swagger file and documentation.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 1000e4ee7d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-29 23:22:04 +02:00
Sebastiaan van Stijn
a8c28260ad
api: swagger: add missing "platform" query-arg on create
Commit 7a9cb29fb9 added a new "platform" query-
parameter to the `POST /containers/create` endpoint, but did not update the
swagger file and documentation.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 3dae8e9fc2)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-29 23:22:00 +02:00
Akihiro Suda
a70ae13d3b
Merge pull request #43876 from thaJeztah/20.10_backport_42655_vfs_storage_driver
[20.10 backport] Fix file capabilities dropping in Dockerfile
2022-07-29 07:28:23 +09:00
Illo Abdulrahim
cfdc075b1c
Fix file capabilities droping in Dockerfile
doCopyXattrs() never reached due to copyXattrs boolean being false, as
a result file capabilities not being copied.

moved copyXattr() out of doCopyXattrs()

Signed-off-by: Illo Abdulrahim <abdulrahim.illo@nokia.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 31f654a704)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-28 09:39:53 +02:00
Olli Janatuinen
2daa6bb6b3 Windows: Re-create custom NAT networks after restart if missing from HNS
Signed-off-by: Olli Janatuinen <olli.janatuinen@gmail.com>
(cherry picked from commit 67c36d5)
Signed-off-by: Olli Janatuinen <olli.janatuinen@gmail.com>
2022-07-25 03:29:01 -07:00
Sebastiaan van Stijn
21b91d3fd5
Merge pull request #43824 from AkihiroSuda/libnetwork-20220716
[20.10] vendor: libnetwork 0dde5c895075df6e3630e76f750a447cf63f4789
2022-07-17 12:17:00 +02:00
Akihiro Suda
903cd53ce4
vendor: libnetwork 0dde5c895075df6e3630e76f750a447cf63f4789
f6ccccb1c0...0dde5c8950

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2022-07-16 22:29:27 +09:00
Akihiro Suda
55cbb2fc57
Merge pull request #43803 from thaJeztah/20.10_update_golang_1.17.12
[20.10] update golang to 1.17.12
2022-07-14 22:25:32 +09:00
Sebastiaan van Stijn
eccaf6d368
[20.10] update golang to 1.17.12
go1.17.12 (released 2022-07-12) includes security fixes to the compress/gzip,
encoding/gob, encoding/xml, go/parser, io/fs, net/http, and path/filepath
packages, as well as bug fixes to the compiler, the go command, the runtime,
and the runtime/metrics package. See the Go 1.17.12 milestone on the issue
tracker for details:

https://github.com/golang/go/issues?q=milestone%3AGo1.17.12+label%3ACherryPickApproved

This update addresses:

CVE-2022-1705, CVE-2022-1962, CVE-2022-28131, CVE-2022-30630, CVE-2022-30631,
CVE-2022-30632, CVE-2022-30633, CVE-2022-30635, and CVE-2022-32148.

Full diff: https://github.com/golang/go/compare/go1.17.11...go1.17.12

From the security announcement;
https://groups.google.com/g/golang-announce/c/nqrv9fbR0zE

We have just released Go versions 1.18.4 and 1.17.12, minor point releases. These
minor releases include 9 security fixes following the security policy:

- net/http: improper sanitization of Transfer-Encoding header

  The HTTP/1 client accepted some invalid Transfer-Encoding headers as indicating
  a "chunked" encoding. This could potentially allow for request smuggling, but
  only if combined with an intermediate server that also improperly failed to
  reject the header as invalid.

  This is CVE-2022-1705 and https://go.dev/issue/53188.

- When `httputil.ReverseProxy.ServeHTTP` was called with a `Request.Header` map
  containing a nil value for the X-Forwarded-For header, ReverseProxy would set
  the client IP as the value of the X-Forwarded-For header, contrary to its
  documentation. In the more usual case where a Director function set the
  X-Forwarded-For header value to nil, ReverseProxy would leave the header
  unmodified as expected.

  This is https://go.dev/issue/53423 and CVE-2022-32148.

  Thanks to Christian Mehlmauer for reporting this issue.

- compress/gzip: stack exhaustion in Reader.Read

  Calling Reader.Read on an archive containing a large number of concatenated
  0-length compressed files can cause a panic due to stack exhaustion.

  This is CVE-2022-30631 and Go issue https://go.dev/issue/53168.

- encoding/xml: stack exhaustion in Unmarshal

  Calling Unmarshal on a XML document into a Go struct which has a nested field
  that uses the any field tag can cause a panic due to stack exhaustion.

  This is CVE-2022-30633 and Go issue https://go.dev/issue/53611.

- encoding/xml: stack exhaustion in Decoder.Skip

  Calling Decoder.Skip when parsing a deeply nested XML document can cause a
  panic due to stack exhaustion. The Go Security team discovered this issue, and
  it was independently reported by Juho Nurminen of Mattermost.

  This is CVE-2022-28131 and Go issue https://go.dev/issue/53614.

- encoding/gob: stack exhaustion in Decoder.Decode

  Calling Decoder.Decode on a message which contains deeply nested structures
  can cause a panic due to stack exhaustion.

  This is CVE-2022-30635 and Go issue https://go.dev/issue/53615.

- path/filepath: stack exhaustion in Glob

  Calling Glob on a path which contains a large number of path separators can
  cause a panic due to stack exhaustion.

  Thanks to Juho Nurminen of Mattermost for reporting this issue.

  This is CVE-2022-30632 and Go issue https://go.dev/issue/53416.

- io/fs: stack exhaustion in Glob

  Calling Glob on a path which contains a large number of path separators can
  cause a panic due to stack exhaustion.

  This is CVE-2022-30630 and Go issue https://go.dev/issue/53415.

- go/parser: stack exhaustion in all Parse* functions

  Calling any of the Parse functions on Go source code which contains deeply
  nested types or declarations can cause a panic due to stack exhaustion.

  Thanks to Juho Nurminen of Mattermost for reporting this issue.

  This is CVE-2022-1962 and Go issue https://go.dev/issue/53616.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-13 10:46:39 +02:00
Sebastiaan van Stijn
3949ff121e
Merge pull request #43770 from xinfengliu/20.10_backport_update_continuity
[20.10] vendor: github.com/containerd/continuity v0.3.0
2022-07-06 14:52:17 +02:00
Xinfeng Liu
ff7feeac37 vendor: github.com/containerd/continuity v0.3.0
Signed-off-by: Xinfeng Liu <Xinfeng.Liu@gmail.com>
2022-07-06 10:36:05 +08:00
Tianon Gravi
a89b84221c
Merge pull request #43701 from thaJeztah/20.10_backport_update_containerd_1.6.6
[20.10 backport] update containerd binary to v1.6.6
2022-06-06 15:32:38 -07:00
Sebastiaan van Stijn
6f3f2b6d08
update containerd binary to v1.6.6
Welcome to the v1.6.6 release of containerd!

The sixth patch release for containerd 1.6 includes a fix for
[CVE-2022-31030](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-31030).

Notable Updates

- Fix ExecSync handler to cap console output size ([GHSA-5ffw-gxpp-mxpf](https://github.com/containerd/containerd/security/advisories/GHSA-5ffw-gxpp-mxpf))

full diff: https://github.com/containerd/containerd/compare/v1.6.5...v1.6.6

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit a7e3182757)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-06-06 22:40:17 +02:00
Akihiro Suda
3fba09221e
Merge pull request #43692 from thaJeztah/20.0_backport_bump_containerd_binary_1.6.5
[20.10 backport] update containerd binary to v1.6.5
2022-06-05 13:39:45 +09:00
Sebastiaan van Stijn
678cc00583
Merge pull request #43589 from thaJeztah/20.10_backport_bump_runc
[20.10 backport] update runc binary to v1.1.2
2022-06-04 22:36:22 +02:00
Sebastiaan van Stijn
b3bcb15da8
update containerd binary to v1.6.5
Welcome to the v1.6.5 release of containerd!

The fifth patch release for containerd 1.6 includes a few fixes and updated
version of runc.

Notable Updates

- Fix for older CNI plugins not reporting version
- Fix mount path handling for CRI plugin on Windows

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit a747cd3702)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-06-04 22:30:35 +02:00
Tianon Gravi
59d0ff35f8
Merge pull request #43686 from dfr/backport-freebsd-mknod
[20.10 backport] Merge fixes to pkg/system to support FreeBSD's mknod
2022-06-03 15:34:32 -07:00
Tianon Gravi
434eecd2cc
Merge pull request #43683 from neersighted/update-libnetwork
[20.10] Backport sysctl performance tuning in libnetwork sandboxes
2022-06-03 11:00:12 -07:00
Tonis Tiigi
f55b030fa0 system: unbreak build for darwin
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit 26dafe43a4)
Signed-off-by: Doug Rabson <dfr@rabson.org>
2022-06-03 11:42:23 +01:00
Artem Khramov
63ab12cd3a Port pkg/system/mknod.go to FreeBSD
Because FreeBSD uses 64-bit device nodes (see
https://reviews.freebsd.org/rS318736), Linux implementation of
`system.Mknod` & `system.Mkdev` is not sufficient.

This change adds freebsd-specific implementations for `Mknod` and
Mkdev`.

Signed-off-by: Artem Khramov <akhramov@pm.me>
(cherry picked from commit f3d3994a4b)
Signed-off-by: Doug Rabson <dfr@rabson.org>
2022-06-03 11:42:12 +01:00
Bjorn Neergaard
081e538fbd vendor: libnetwork f6ccccb1c082a432c2a5814aaedaca56af33d9ea
Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
2022-06-02 13:37:46 -06:00
Sebastiaan van Stijn
4159fa6860
Merge pull request #43677 from thaJeztah/20.10_update_golang_1.17.11
[20.10] update golang to 1.17.11
2022-06-02 20:10:46 +02:00
Sebastiaan van Stijn
8e9d647c01
[20.10] update golang to 1.17.11
go1.17.11 (released 2022-06-01) includes security fixes to the crypto/rand,
crypto/tls, os/exec, and path/filepath packages, as well as bug fixes to the
crypto/tls package. See the Go 1.17.11 milestone on our issue tracker for details.

https://github.com/golang/go/issues?q=milestone%3AGo1.17.11+label%3ACherryPickApproved

Hello gophers,

We have just released Go versions 1.18.3 and 1.17.11, minor point releases.

These minor releases include 4 security fixes following the security policy:

- crypto/rand: rand.Read hangs with extremely large buffers
  On Windows, rand.Read will hang indefinitely if passed a buffer larger than
  1 << 32 - 1 bytes.

  Thanks to Davis Goodin and Quim Muntal, working at Microsoft on the Go toolset,
  for reporting this issue.

  This is [CVE-2022-30634][CVE-2022-30634] and Go issue https://go.dev/issue/52561.
- crypto/tls: session tickets lack random ticket_age_add
  Session tickets generated by crypto/tls did not contain a randomly generated
  ticket_age_add. This allows an attacker that can observe TLS handshakes to
  correlate successive connections by comparing ticket ages during session
  resumption.

  Thanks to GitHub user nervuri for reporting this.

  This is [CVE-2022-30629][CVE-2022-30629] and Go issue https://go.dev/issue/52814.
- `os/exec`: empty `Cmd.Path` can result in running unintended binary on Windows

  If, on Windows, `Cmd.Run`, `cmd.Start`, `cmd.Output`, or `cmd.CombinedOutput`
  are executed when Cmd.Path is unset and, in the working directory, there are
  binaries named either "..com" or "..exe", they will be executed.

  Thanks to Chris Darroch, brian m. carlson, and Mikhail Shcherbakov for reporting
  this.

  This is [CVE-2022-30580][CVE-2022-30580] and Go issue https://go.dev/issue/52574.
- `path/filepath`: Clean(`.\c:`) returns `c:` on Windows

  On Windows, the `filepath.Clean` function could convert an invalid path to a
  valid, absolute path. For example, Clean(`.\c:`) returned `c:`.

  Thanks to Unrud for reporting this issue.

  This is [CVE-2022-29804][CVE-2022-29804] and Go issue https://go.dev/issue/52476.

[CVE-2022-30634]: https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-30634
[CVE-2022-30629]: https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-30629
[CVE-2022-30580]: https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-30580
[CVE-2022-29804]: https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-29804

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-06-02 09:19:56 +02:00
Tianon Gravi
90033ab093
Merge pull request #43644 from ameyag/20.10-win-port-conflict
[20.10 backport] Fix for Windows port conflict with published ports in host mode for overlay.
2022-05-26 11:31:41 -07:00
Ameya Gawde
87ead7fd2a
vendor: hcsshim a11a2c44e8a4aa9d66314b1d759ef582df5ab5e8
Signed-off-by: Ameya Gawde <agawde@mirantis.com>
2022-05-25 15:33:23 -07:00
Ameya Gawde
27f8322324
vendor: libnetwork 2dab5620d4462865c6151e573b3e7fa5d3b8458b
Signed-off-by: Ameya Gawde <agawde@mirantis.com>
2022-05-25 15:28:25 -07:00
Sebastiaan van Stijn
506d28dca8
Merge pull request #43612 from thaJeztah/20.10_backport_websocket_streams_docs_touch_ups
[20.10 backport] docs: api: /containers/{id}/attach/ws: remove unsupported query-args < v1.42
2022-05-19 23:44:59 +02:00
Sebastiaan van Stijn
829951ec19
docs: api: /containers/{id}/attach/ws: remove unsupported query-args < v1.42
These query-args were documented, but not actually supported until
ea6760138c (API v1.42).

This removes them from the documentation, as these arguments were ignored
(and defaulted to `true` (enabled))

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit a5a77979dd)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-05-18 13:50:08 +02:00
Sebastiaan van Stijn
7307a0d583
Merge pull request #43606 from rumpl/20.10_backport_rename-reservation
[20.10 backport]  Rename Reservation to Reservations in the open API
2022-05-17 23:21:03 +02:00
Djordje Lukic
6cbe73bfc0
Rename Reservation to Reservations in the open API
The correct name for this property is, and always was "Reservations"

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2022-05-17 14:45:24 +02:00
Sebastiaan van Stijn
d9ed3d7e28
update runc binary to v1.1.2
This is the second patch release of the runc 1.1 release branch. It
fixes CVE-2022-29162, a minor security issue (which appears to not be
exploitable) related to process capabilities.

This is a similar bug to the ones found and fixed in Docker and
containerd recently (CVE-2022-24769).

- A bug was found in runc where runc exec --cap executed processes with
  non-empty inheritable Linux process capabilities, creating an atypical Linux
  environment. For more information, see GHSA-f3fp-gc8g-vw66 and CVE-2022-29162.
- runc spec no longer sets any inheritable capabilities in the created
  example OCI spec (config.json) file.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit bc0fd3f617)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-05-12 14:44:34 +02:00
Samuel Karp
f756502055
Merge pull request #43582 from thaJeztah/20.10_bump_golang_1.17.10 2022-05-12 00:33:06 -07:00
Sebastiaan van Stijn
a15acb4bd6
[20.10] vendor: golang.org/x/sys v0.0.0-20220412211240-33da011f77ad
Includes fixes for:

- CVE-2022-29526 (http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-29526);
  (description at https://go.dev/issue/52313).

full diff: 63515b42dc...33da011f77

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-05-11 14:17:46 +02:00
Sebastiaan van Stijn
5f2e0b79ad
[20.10] update golang to 1.17.10
go1.17.10 (released 2022-05-10) includes security fixes to the syscall package,
as well as bug fixes to the compiler, runtime, and the crypto/x509 and net/http/httptest
packages. See the Go 1.17.10 milestone on the issue tracker for details:

https://github.com/golang/go/issues?q=milestone%3AGo1.17.10+label%3ACherryPickApproved

Full diff: http://github.com/golang/go/compare/go1.17.9...go1.17.10

Includes fixes for:

- CVE-2022-29526 (http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-29526);
  (description at https://go.dev/issue/52313).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-05-11 14:12:56 +02:00
Sebastiaan van Stijn
462cd7de50
Merge pull request #43567 from 42wim/fixstats
[20.10] vendor: update github.com/containerd/cgroups and github.com/cilium/ebpf (for fixing empty `docker stats`)
2022-05-10 09:17:07 +02:00
Wim
be7855fdbe
vendor: update github.com/containerd/cgroups and github.com/cilium/ebpf
Fixes
- https://github.com/docker/for-linux/issues/1284
- https://github.com/containerd/containerd/issues/6700
- https://github.com/moby/moby/issues/43387

Update to cgroups v1.0.1 which has the current proto for cgroupsv1
Need to update cilium/ebpf dependency to v0.4.0

Signed-off-by: Wim <wim@42.be>
2022-05-07 17:58:57 +02:00
Sebastiaan van Stijn
4433bf67ba
Merge pull request #43561 from thaJeztah/20.10_backport_bump_containerd_binary_1.6.4
[20.10 backport] update containerd binary to v1.6.4
2022-05-04 20:25:18 +02:00
Sebastiaan van Stijn
414a9e24a7
update containerd binary to v1.6.4
Notable Updates

- Update go-cni to fix teardown regression
- Fix broken SELinux relabeling for Kubernetes volume mounts

full diff: https://github.com/containerd/containerd/compare/v1.6.3...v1.6.4

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 6e376e32d8)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-05-04 11:02:22 +02:00
Akihiro Suda
0809f5fafb
Merge pull request #43433 from thaJeztah/20.10_backport_update_containerd_runc
[20.10 backport] update containerd binary to v1.6.3, runc v1.1.1
2022-04-28 13:11:49 +09:00
Sebastiaan van Stijn
47b6a924b6
update containerd binary to v1.6.3
Release notes:

Welcome to the v1.6.3 release of containerd!

The third patch release for containerd 1.6 includes various fixes and updates.

Notable Updates

- Fix panic when configuring tracing plugin
- Improve image pull performance in CRI plugin
- Check for duplicate nspath
- Fix deadlock in cgroup metrics collector
- Mount devmapper xfs file system with "nouuid" option
- Make the temp mount as ready only in container WithVolumes
- Fix deadlock from leaving transaction open in native snapshotter
- Monitor OOMKill events to prevent missing container events

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit a9be008f00)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-27 21:19:59 +02:00
Sebastiaan van Stijn
6d7c2b2d26
update containerd binary to v1.6.2
The second patch release for containerd 1.6 includes a fix for
 [CVE-2022-24769](https://github.com/containerd/containerd/security/advisories/GHSA-c9cp-9c75-9v8c).

Notable Updates

- **Fix the inheritable capability defaults** ([GHSA-c9cp-9c75-9v8c](https://github.com/containerd/containerd/security/advisories/GHSA-c9cp-9c75-9v8c))

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit e69529c94a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-27 21:19:57 +02:00
Sebastiaan van Stijn
91708bf704
update containerd binary to v1.6.1
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 06abe8dd2d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-27 21:19:55 +02:00
Sebastiaan van Stijn
53ae17008e
Revert "[20.10] update containerd binary to 1.5.11"
This reverts commit 7dfe7a1752.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-27 21:19:53 +02:00
Akihiro Suda
961b9a78d5
update runc binary to v1.1.1
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit ffc903d7a6)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-27 21:19:51 +02:00
Akihiro Suda
97972dac5f
update runc binary to v1.1.0
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit c55eb6b824)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-27 21:19:47 +02:00
Akihiro Suda
2929771a53
Merge pull request #43499 from thaJeztah/20.10_bump_golang_1.17.9
[20.10] update golang to 1.17.9
2022-04-20 09:37:53 +09:00
Sebastiaan van Stijn
033a819714
[20.10] update golang to 1.17.9
go1.17.9 (released 2022-04-12) includes security fixes to the crypto/elliptic
and encoding/pem packages, as well as bug fixes to the linker and runtime. See
the Go 1.17.9 milestone on the issue tracker for details:

Includes fixes for:

- CVE-2022-24675 (https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-24675)
- CVE-2022-28327 (https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-28327)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-19 09:16:33 +02:00
Tianon Gravi
ecf2eee0f6
Merge pull request #43486 from thaJeztah/20.10_backport_fix_dco_check
[20.10 backport] Jenkinsfile: add workaround for CVE-2022-24765
2022-04-13 11:39:23 -07:00
Sebastiaan van Stijn
a80884126b
Jenkinsfile: add workaround for CVE-2022-24765
git published an advisory Yesterday, which (as a counter-measure)
requires the git repository's directory to be owned by the current
user, and otherwise produce an error:

    fatal: unsafe repository ('/workspace' is owned by someone else)
    To add an exception for this directory, call:

        git config --global --add safe.directory /workspace

The DCO check is run within a container, which is running as `root`
(to allow packages to be installed), but because of this, the user
does not match the files that are bind-mounted from the host (as they
are checked out by Jenkins, using a different user).

To work around this issue, this patch configures git to consider the
`/workspace` directory as "safe". We configure it in the `--system`
configuration so that it takes effect for "all users" inside the
container.

More details on the advisory can be found on GitHub's blog:
https://github.blog/2022-04-12-git-security-vulnerability-announced/

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit efe03aa2d8)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-13 16:07:32 +02:00
Brian Goff
086a3fa036
Merge pull request #43468 from thaJeztah/20.10_backport_update_go_1.17
[20.10 backport] update to go 1.17.8
2022-04-12 09:41:51 -07:00
Sebastiaan van Stijn
09d6fcdfec
update to go 1.17.8 to address CVE-2022-24921
Addresses [CVE-2022-24921](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-24921)

go1.17.8 (released 2022-03-03) includes a security fix to the regexp/syntax package,
as well as bug fixes to the compiler, runtime, the go command, and the crypto/x509,
and net packages. See the Go 1.17.8 milestone on the issue tracker for details:

https://github.com/golang/go/issues?q=milestone%3AGo1.17.8+label%3ACherryPickApproved

full diff: https://github.com/golang/go/compare/go1.17.7...go1.17.8

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit e781cf5f64)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-07 23:28:10 +02:00
Sebastiaan van Stijn
5957684b2c
Update Go to 1.17.7
Includes security fixes for crypto/elliptic (CVE-2022-23806), math/big (CVE-2022-23772),
and cmd/go (CVE-2022-23773).

go1.17.7 (released 2022-02-10) includes security fixes to the crypto/elliptic,
math/big packages and to the go command, as well as bug fixes to the compiler,
linker, runtime, the go command, and the debug/macho, debug/pe, and net/http/httptest
packages. See the Go 1.17.7 milestone on our issue tracker for details:

https://github.com/golang/go/issues?q=milestone%3AGo1.17.7+label%3ACherryPickApproved

full diff: https://github.com/golang/go/compare/go1.17.6...go1.17.7

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit cad6c8f7f1)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-07 23:28:08 +02:00
Sebastiaan van Stijn
55b72c70ba
Update Go to 1.17.6
go1.17.6 (released 2022-01-06) includes fixes to the compiler, linker, runtime,
and the crypto/x509, net/http, and reflect packages. See the Go 1.17.6 milestone
on our issue tracker for details:

https://github.com/golang/go/issues?q=milestone%3AGo1.17.6+label%3ACherryPickApproved

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit f85ae526f0)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-07 23:28:06 +02:00
Sebastiaan van Stijn
fdf3020bd5
Update Go to 1.17.5
go1.17.5 (released 2021-12-09) includes security fixes to the syscall and net/http
packages. See the Go 1.17.5 milestone on the issue tracker for details:

https://github.com/golang/go/issues?q=milestone%3AGo1.17.5+label%3ACherryPickApproved

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit d620cb6afc)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-07 23:28:04 +02:00
Sebastiaan van Stijn
36e164ba80
Update Go to 1.17.4
go1.17.4 (released 2021-12-02) includes fixes to the compiler, linker, runtime,
and the go/types, net/http, and time packages. See the Go 1.17.4 milestone on
the issue tracker for details:

https://github.com/golang/go/issues?q=milestone%3AGo1.17.4+label%3ACherryPickApproved

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 6bb3891c60)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-07 23:28:02 +02:00
Sebastiaan van Stijn
ecfba8f588
Update Go to 1.17.3
go1.17.3 (released 2021-11-04) includes security fixes to the archive/zip and
debug/macho packages, as well as bug fixes to the compiler, linker, runtime, the
go command, the misc/wasm directory, and to the net/http and syscall packages.
See the Go 1.17.3 milestone on our issue tracker for details.

From the announcement e-mail:

[security] Go 1.17.3 and Go 1.16.10 are released

We have just released Go versions 1.17.3 and 1.16.10, minor point releases.
These minor releases include two security fixes following the security policy:

- archive/zip: don't panic on (*Reader).Open
  Reader.Open (the API implementing io/fs.FS introduced in Go 1.16) can be made
  to panic by an attacker providing either a crafted ZIP archive containing
  completely invalid names or an empty filename argument.
  Thank you to Colin Arnott, SiteHost and Noah Santschi-Cooney, Sourcegraph Code
  Intelligence Team for reporting this issue. This is CVE-2021-41772 and Go issue
  golang.org/issue/48085.
- debug/macho: invalid dynamic symbol table command can cause panic
  Malformed binaries parsed using Open or OpenFat can cause a panic when calling
  ImportedSymbols, due to an out-of-bounds slice operation.
  Thanks to Burak Çarıkçı - Yunus Yıldırım (CT-Zer0 Crypttech) for reporting this
  issue. This is CVE-2021-41771 and Go issue golang.org/issue/48990.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit ce668d6c1e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-07 23:28:01 +02:00
Sebastiaan van Stijn
4e14dcc125
Update Go to 1.17.2
go1.17.2 (released 2021-10-07) includes a security fix to the linker and misc/wasm
directory, as well as bug fixes to the compiler, the runtime, the go command, and
to the time and text/template packages. See the Go 1.17.2 milestone on our issue
tracker for details:

https://github.com/golang/go/issues?q=milestone%3AGo1.17.2+label%3ACherryPickApproved

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit e7fb0c8201)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-07 23:27:59 +02:00
Sebastiaan van Stijn
c32b5ece31
Update Go to 1.17.1
This includes additional fixes for CVE-2021-39293.

go1.17.1 (released 2021-09-09) includes a security fix to the archive/zip package,
as well as bug fixes to the compiler, linker, the go command, and to the crypto/rand,
embed, go/types, html/template, and net/http packages. See the Go 1.17.1 milestone
on the issue tracker for details:

https://github.com/golang/go/issues?q=milestone%3AGo1.17.1+label%3ACherryPickApproved

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 0050ddd43b)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-07 23:27:57 +02:00
Sebastiaan van Stijn
7096508811
vendor: update archive/tar to match Go 1.17.0
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit aa606307b7)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-07 23:27:55 +02:00
Sebastiaan van Stijn
a1150245cc
Update to Go 1.17.0, and gofmt with Go 1.17
Movified from 686be57d0a, and re-ran
gofmt again to address for files not present in 20.10 and vice-versa.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 686be57d0a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-07 23:27:50 +02:00
Sebastiaan van Stijn
95cc7115fb
hack/vendor.sh: allow go version to be specified with .0
Golang '.0' releases are released without a trailing .0 (i.e. go1.17
is equal to go1.17.0). For the base image, we want to specify the go
version including their patch release (golang:1.17 is equivalent to
go1.17.x), so adjust the script to also accept the trailing .0, because
otherwise the download-URL is not found:

    hack/vendor.sh archive/tar
    update vendored copy of archive/tar
    downloading: https://golang.org/dl/go1.17.0.src.tar.gz
    curl: (22) The requested URL returned error: 404

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 9ed88a0801)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-07 23:26:34 +02:00
Sebastiaan van Stijn
949c33b1c5
vendor: golang.org/x/sys 63515b42dcdf9544f4e6a02fd7632793fde2f72d (for Go 1.17)
Go 1.17 requires golang.org/x/sys a76c4d0a0096537dc565908b53073460d96c8539 (May 8,
2021) or later, see https://github.com/golang/go/issues/45702. While this seems
to affect macOS only, let's update to the latest version.

full diff: d19ff857e8...63515b42dc

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit d48c8b70a1)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-07 23:26:32 +02:00
Sebastiaan van Stijn
8392285876
vendor: golang.org/x/sys d19ff857e887eacb631721f188c7d365c2331456
full diff: b64e53b001...d19ff857e8

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit f0d3e905b6)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-07 23:26:29 +02:00
Brian Goff
9ee5abed9c
Merge pull request #43470 from thaJeztah/20.10_backport_bump_buildx_0.8.2
[20.10 backport] testing: install buildx from binary, and update to v0.8.2
2022-04-07 12:36:23 -07:00
Sebastiaan van Stijn
4e81bcf380
Makefile: update buildx to v0.8.2
release notes: https://github.com/docker/buildx/releases/tag/v0.8.2

Notable changes:

- Update Compose spec used by buildx bake to v1.2.1 to fix parsing ports definition
- Fix possible crash on handling progress streams from BuildKit v0.10
- Fix parsing groups in buildx bake when already loaded by a parent group

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit ae7d3efafd)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-07 10:56:43 +02:00
Sebastiaan van Stijn
74e699c8d3
Makefile: update buildx version to v0.6.0
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 22316ff1d8)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-07 10:56:21 +02:00
Sebastiaan van Stijn
bc3cc2e7ac
Makefile: install buildx from binary release, instead of building
This was originally added in 833444c0d6,
at which time buildx did not yet have a release, so we had to build
from source.

Now that buildx has binary releases on GitHub, we should be able to
consume those binaries instead of building.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 151ec207b9)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-07 10:55:22 +02:00
Sebastiaan van Stijn
811795ccd7
Merge pull request #43454 from thaJeztah/20.10_update_distribution
[20.10] vendor: github.com/docker/distribution v2.8.1
2022-04-05 13:44:22 +02:00
Sebastiaan van Stijn
ae7f230408
Merge pull request #43460 from thaJeztah/20.10_backport_docs_fixes
[20.10 backport] docs: assorted fixes in swagger files
2022-04-05 13:43:10 +02:00
Sebastiaan van Stijn
492fac20af
api: docs: fix indentation of HostConfig.SecurityOpt (v1.39-v1.41)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 0759f013a4)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-04 19:30:05 +02:00
Sebastiaan van Stijn
3cba2682d8
api: docs: move ContainerWaitResponse to definitions (v1.39-v1.41)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 352d1bb8e0)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-04 19:30:02 +02:00
Sebastiaan van Stijn
55e71450ae
api: docs: move VolumeCreateOptions to definitions (v1.39-v1.41)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 444b27910c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-04 19:30:00 +02:00
Sebastiaan van Stijn
c54362cd64
api: docs: move Volume examples inline (v1.39-v1.41)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 5c0aae359f)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-04 19:29:57 +02:00
Daniel Black
c60ff9b296
doc: server API Correct ImagesCreate - platform parameter added in 1.32
closes: docker/docker.github.io#9305

Signed-off-by: Daniel Black <daniel@linux.ibm.com>
(cherry picked from commit 521ac858e7)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-04 19:29:27 +02:00
Sebastiaan van Stijn
7a45f7a8cc
docs: cleanup swagger API with multiple examples (v1.25-v1.41)
Applies the changes from 3671cb90a3 to
the swagger files used for the documentation.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 8ac2f84f9a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-04 19:29:11 +02:00
Niel Drummond
29bb9204bf
api: docs: add IPAMConfig on IPAM (v1.41)
Signed-off-by: Niel Drummond <niel@drummond.lu>
(cherry picked from commit abc2f095ab)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-04 19:28:54 +02:00
Sebastiaan van Stijn
77f6564369
api: docs: document MountPoint fields (v1.25-v1.41)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 0a438f2612)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-04 19:28:14 +02:00
Sebastiaan van Stijn
51ea235ab8
api: docs: remove deprecated RootFS.BaseLayer (API v1.25-v1.41)
This field was used when Windows did not yet support regular images, and required
the base-image to pre-exist on the Windows machine (as those layers were not yet
allowed to be distributed).

Commit f342b27145 (docker 1.13.0, API v1.25) removed
usage of the field. The field was not documented in the API, but because it was not
removed from the Golang structs in the API, ended up in the API documentation when
we switched to using Swagger instead of plain MarkDown for the API docs.

Given that the field was never set in any of these API versions, and had an "omitempty",
it was never actually returned in a response, so should be fine to remove from these
API docs.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 8e9c8ff7f2)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-04 19:27:54 +02:00
Sebastiaan van Stijn
3d6b4ae572
Correct type of Mounts in ContainerSummary in docs (v1.25-v1.40)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit c04dff7623)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-04 19:27:27 +02:00
Michael Weidmann
6e8b9809b7
Correct type of Mounts in ContainerSummary in docs
Signed-off-by: Michael Weidmann <michaelweidmann@web.de>
(cherry picked from commit 16e3ca35eb)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-04 19:27:24 +02:00
Sebastiaan van Stijn
621a98dac0
api: docs: fix warning about comment indentation (API v1.40-v1.41)
5428:7 warning comment not indented like content (comments-indentation)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit d19dd22257)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-04 19:27:02 +02:00
Sebastiaan van Stijn
bb9ef98060
api: docs: update docs for /images/{name}/json (API v1.39-v1.41)
- rename definition in swagger from `Image` to `ImageInspect` to match the go type
- improve (or add) documentation for various fields
- move example values in-line in the "definitions" section
- remove the `required` fields from `ImageInspect`, as the type is only used as
  response type (not to make requests).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 9565606222)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-04 19:26:59 +02:00
Sebastiaan van Stijn
88ca5cec4e
daemon: fix error-message for minimum allowed kernel-memory limit
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 3c44ade6d0)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-04 19:26:18 +02:00
Sebastiaan van Stijn
3ea996abd7
docs: add missing KernelMemoryTCP to api v1.40 and v1.41
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 9f3c238c29)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-04 19:26:16 +02:00
Sebastiaan van Stijn
b475bc95cd
docs/api: add missing 400 response for POST /containers/{id}/wait
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 5e2b7dea02)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-04 19:26:14 +02:00
Sebastiaan van Stijn
ae07b3cc96
docs/api: update /containers/{id}/wait "condition" parameter (v1.30-v1.41)
This patch updates the swagger, and:

- adds an enum definition to document valid values (instead of describing them)
- updates the description to mention both "omitted" and "empty" values (although
  the former is already implicitly covered by the field being "optional" and
  having a default value).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 41b137962d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-04 19:26:12 +02:00
Sebastiaan van Stijn
19555fa92d
[20.10] vendor: github.com/docker/distribution v2.8.1
equivalent of b92af14a1c, for 20.10.x

no changes to code we use, but the v2.8.0 module was borked

full diff: https://github.com/docker/distribution/compare/v2.8.0...v2.8.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-04 11:17:29 +02:00
Tianon Gravi
4e188b8a49
Merge pull request #43426 from thaJeztah/20.10_backport_info_counters_rwmutex
[20.10 backport] daemon: use RWMutex for stateCounter
2022-03-25 15:49:29 -07:00
Sebastiaan van Stijn
339d7bdd7d
Merge pull request #43406 from dims/cherry-pick-thaJeztah-errdefs_reduce_client_deps_alternative
[20.10] errdefs: move GetHTTPErrorStatusCode to api/server/httpstatus
2022-03-25 14:52:59 +01:00
Sebastiaan van Stijn
32fe0bbb91
daemon: use RWMutex for stateCounter
Use an RWMutex to allow concurrent reads of these counters

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 699174347c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-25 09:38:53 +01:00
Tianon Gravi
1881be2de5
Merge pull request #43360 from Nossnevs/backport-21.x-linux-routeoverlaps-link-only-42598
[20.10 backport] Only check if route overlaps routes with scope: LINK
2022-03-24 16:54:51 -07:00
Sebastiaan van Stijn
af05807928
Merge pull request #43356 from awmirantis/20.10-bump-swarmkit-increase-config-size
[20.10] Bump swarmkit for change to max config size
2022-03-24 20:23:48 +01:00
Sebastiaan van Stijn
ed8fb00b65
errdefs: move GetHTTPErrorStatusCode to api/server/httpstatus
This reverts the changes made in 2a9c987e5a, which
moved the GetHTTPErrorStatusCode() utility to the errdefs package.

While it seemed to make sense at the time to have the errdefs package provide
conversion both from HTTP status codes errdefs and the reverse, a side-effect
of the move was that the errdefs package now had a dependency on various external
modules, to handle conversio of errors coming from those sub-systems, such as;

- github.com/containerd/containerd
- github.com/docker/distribution
- google.golang.org/grpc

This patch moves the conversion from (errdef-) errors to HTTP status-codes to a
 api/server/httpstatus package, which is only used by the API server, and should
not be needed by client-code using the errdefs package.

The MakeErrorHandler() utility was moved to the API server itself, as that's the
only place it's used. While the same applies to the GetHTTPErrorStatusCode func,
I opted for keeping that in its own package for a slightly cleaner interface.

Why not move it into the api/server/httputils package?

The api/server/httputils package is also imported in the client package, which
uses the httputils.ParseForm() and httputils.HijackConnection() functions as
part of the TestTLSCloseWriter() test. While this is only used in tests, I
wanted to avoid introducing the indirect depdencencies outside of the api/server
code.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 85b7df5ead93a79ed6c8ef83535c153f65ef116d)
Signed-off-by: Davanum Srinivas <davanum@gmail.com>
(cherry picked from commit b9af850d5d232d2d8e0800f4f0d7ceceb5bf84ff)
Signed-off-by: Davanum Srinivas <davanum@gmail.com>
2022-03-24 14:16:40 -04:00
tonic
3bd611d7a5
log error message when receiving an unexpected type error
Signed-off-by: tonic <tonicbupt@gmail.com>
(cherry picked from commit 24aaa7f8c9)
Signed-off-by: Davanum Srinivas <davanum@gmail.com>
2022-03-24 14:16:29 -04:00
Sebastiaan van Stijn
66a96760b2
Merge pull request #43407 from dims/cherry-pick-remove_containerd_from_client
[20.10] Remove containerd "platform" dependency from client
2022-03-24 19:12:03 +01:00
Sebastiaan van Stijn
87a90dc786
Merge pull request #43417 from thaJeztah/20.10_update_containerd_binary_1.5.11
[20.10] update containerd binary to 1.5.11
2022-03-24 02:37:51 +01:00
Sebastiaan van Stijn
7dfe7a1752
[20.10] update containerd binary to 1.5.11
The eleventh patch release for containerd 1.5 includes a fix for
[CVE-2022-24769](https://github.com/containerd/containerd/security/advisories/GHSA-c9cp-9c75-9v8c).

Notable Updates:

- **Fix the inheritable capability defaults** ([GHSA-c9cp-9c75-9v8c](https://github.com/containerd/containerd/security/advisories/GHSA-c9cp-9c75-9v8c))

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-24 00:50:43 +01:00
Sebastiaan van Stijn
7f375bcff4
Merge pull request from GHSA-2mm7-x5h6-5pvq
[20.10] oci: inheritable capability set should be empty
2022-03-23 22:10:17 +01:00
Sebastiaan van Stijn
cf0abad081
Merge pull request #43416 from thaJeztah/20.10_update_containerd2
[20.10] vendor: containerd 7cfa023d95d37076d5ab035003d4839f4b6ba791
2022-03-23 21:26:07 +01:00
Sebastiaan van Stijn
af953d2f38
[20.10] vendor: containerd 7cfa023d95d37076d5ab035003d4839f4b6ba791
updates the vendoring from the latest commit of the ambiguous-manifest-moby-20.10
branch in our fork.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-23 19:41:05 +01:00
Sebastiaan van Stijn
5f9753ae73
client: remove containerd "platform" dependency
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 9d7495c2b3500565986e3ab8d571c57e296a980d)
Signed-off-by: Davanum Srinivas <davanum@gmail.com>
2022-03-22 16:34:19 -04:00
Sebastiaan van Stijn
4df345e65d
client: remove unused Platform field from configWrapper
This field was added in 7a9cb29fb9,
but appears to be unused, so removing it.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 3994e0ce7855b0dc845c558304e4c1e7a89f0929)
Signed-off-by: Davanum Srinivas <davanum@gmail.com>
2022-03-22 16:34:08 -04:00
Samuel Karp
dd38613d0c
oci: inheritable capability set should be empty
The Linux kernel never sets the Inheritable capability flag to anything
other than empty.  Moby should have the same behavior, and leave it to
userspace code within the container to set a non-empty value if desired.

Reported-by: Andrew G. Morgan <morgan@kernel.org>
Signed-off-by: Samuel Karp <skarp@amazon.com>
(cherry picked from commit 0d9a37d0c2)
Signed-off-by: Samuel Karp <skarp@amazon.com>
2022-03-17 14:17:00 -07:00
Mikael Svensson
2825bf7123
Only check if route overlaps routes with scope: LINK
Signed-off-by: Mikael Svensson <mikael.svensson@nasdaq.com>
2022-03-17 09:54:27 +01:00
Adam Williams
f5c56eaca8 [20.10] bump swarmkit for config size increase
Signed-off-by: Adam Williams <awilliams@mirantis.com>
2022-03-10 10:49:27 -08:00
Sebastiaan van Stijn
906f57ff5b
Merge pull request #43333 from pete-woods/20.10-backport-43291-schema-download-retry
[20.10 backport] distribution: retry downloading schema config on retryable error
2022-03-06 21:52:27 +01:00
Pete Woods
ce3b6d1ae9
distribution: retry downloading schema config on retryable error
fixes #43267

Signed-off-by: Pete Woods <pete.woods@circleci.com>

# Conflicts:
#	distribution/pull_v2.go
#	distribution/pull_v2_test.go
#	distribution/xfer/transfer.go
2022-03-05 20:29:10 +00:00
Sebastiaan van Stijn
c3dec607c3
Merge pull request #43329 from thaJeztah/20.10_backport_update_containerd_binary_1.5.10
[20.10 backport] update containerd binary to v1.5.10
2022-03-04 21:17:02 +01:00
Sebastiaan van Stijn
180f3b9cc7
update containerd binary to v1.5.10
Welcome to the v1.5.10 release of containerd!

The tenth patch release for containerd 1.5 includes a fix for [CVE-2022-23648][1]
and other issues.

Notable Updates

- Use fs.RootPath when mounting volumes (GHSA-crp2-qrr5-8pq7)
- Return init pid when clean dead shim in runc.v1/v2 shims
- Handle sigint/sigterm in shimv2
- Use readonly mount to read user/group info

[1]: https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-23648
[2]: https://github.com/containerd/containerd/security/advisories/GHSA-crp2-qrr5-8pq7

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 2c8f0a0c99)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-04 19:36:34 +01:00
Samuel Karp
409d7ac7f2
Merge pull request #43326 from thaJeztah/20.10_bump_go_1.16.15 2022-03-04 10:09:10 -08:00
Sebastiaan van Stijn
0fa0d70489
[20.10] update to go 1.16.15 to address CVE-2022-24921
Addresses [CVE-2022-24921](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-24921)

go1.16.15 (released 2022-03-03) includes a security fix to the regexp/syntax package,
as well as bug fixes to the compiler, runtime, the go command, and to the net package.
See the Go 1.16.15 milestone on the issue tracker for details:

https://github.com/golang/go/issues?q=milestone%3AGo1.16.15+label%3ACherryPickApproved

full diff: https://github.com/golang/go/compare/go1.16.14...go1.16.15

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-04 16:42:56 +01:00
Brian Goff
c4f6167be1
Merge pull request #43259 from thaJeztah/20.10_backport_fix_lock
[20.10 backport] vendor: bump etcd v3.3.27
2022-03-03 11:12:52 -08:00
Sebastiaan van Stijn
9a57c5c4c9
Merge pull request #43260 from thaJeztah/20.10_backport_docs_fixes
[20.10 backport] docs: assorted docs and swagger fixes
2022-03-03 20:10:01 +01:00
Sebastiaan van Stijn
4679ffd01c
docs: remove HostConfig.LxcConf field from API docs
Commit 3b5fac462d  / docker 1.10 removed support
for the LXC runtime, and removed the corresponding fields from the API (v1.22).

This patch removes the `HostConfig.LxcConf` field from the API documentation.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 839e2ecc1b)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-02-18 13:21:13 +01:00
Sebastiaan van Stijn
fbbe1fcfd8
api: remove HostConfig.LxcConf field
Commit 3b5fac462d  / docker 1.10 removed support
for the LXC runtime, and removed the corresponding fields from the API (v1.22).

This patch removes the `HostConfig.LxcConf` field from the swagger definition.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 75a1ad0c9f)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-02-18 13:21:10 +01:00
Sebastiaan van Stijn
a5a697143d
docs: fixing /exec/{id}/resize response code in API documentation
This takes the changes from 1a933e113d and
834272f978, and applies them to older API
versions in the docs directory (which are used for the actual documentation).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 2145f3ba2c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-02-18 13:20:25 +01:00
Thomas LEDOS
42b13c09a6
Added error codes for exec resize in API docs
Signed-off-by: Thomas LEDOS <thomas.ledos92@gmail.com>
(cherry picked from commit 834272f978)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-02-18 13:20:09 +01:00
Thomas LEDOS
aa69e8ed3f
Fixed exec resize success code in API docs
Signed-off-by: Thomas LEDOS <thomas.ledos92@gmail.com>
(cherry picked from commit 1a933e113d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-02-18 13:19:58 +01:00
Felipe Ruhland
ca5ba4890e
Fix typo on the v1.41 changelog
Signed-off-by: Felipe Ruhland <felipe.ruhland@gmail.com>
(cherry picked from commit eefb6c672a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-02-18 13:19:21 +01:00
Kir Kolyshkin
db214206e5
vendor: bump etcd v3.3.27
Bump etcd to v3.3.27, which includes https://github.com/etcd-io/etcd/pull/12552,
to fix https://github.com/moby/moby/issues/31182

Full diff: https://github.com/coreos/etcd/compare/v3.3.25...v3.3.27

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit ffd5a20ab8)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-02-18 12:55:15 +01:00
Sebastiaan van Stijn
df6a536193
vendor: github.com/coreos/etcd v3.3.25
full diff: https://github.com/coreos/etcd/compare/v3.3.12...v3.3.25

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 2bef937507)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-02-18 12:54:48 +01:00
Sebastiaan van Stijn
3d5fb9425e
Merge pull request #43167 from thaJeztah/20.10_backport_jenkinsfile_remove_rs1
[20.10 backport] Jenkinsfile: remove Windows RS1 as it reached end of support
2022-02-17 20:10:45 +01:00
Sebastiaan van Stijn
1eef444005
Merge pull request #43171 from thaJeztah/20.10_backport_api_docs_fixes
[20.10 backport] docs: update API v1.41 and v1.40 docs with fixes from api/swagger
2022-02-17 20:10:26 +01:00
Brian Goff
7f44d606f9
Merge pull request #43166 from thaJeztah/20.10_backport_fix_update_sync
[20.10 backport] Fix for lack of syncronization in daemon/update.go
2022-02-17 11:08:56 -08:00
Tianon Gravi
c9e1608dd4
Merge pull request #43243 from thaJeztah/20.10_update_go_1.16.14
[20.10] Update Go to 1.16.14
2022-02-15 13:13:37 -08:00
Sebastiaan van Stijn
9502dec32f
[20.10] Update Go to 1.16.14
Includes security fixes for crypto/elliptic (CVE-2022-23806), math/big (CVE-2022-23772),
and cmd/go (CVE-2022-23773).

go1.16.14 (released 2022-02-10) includes security fixes to the crypto/elliptic,
math/big packages and to the go command, as well as bug fixes to the compiler,
linker, runtime, the go command, and the debug/macho, debug/pe, net/http/httptest,
and testing packages. See the Go 1.16.14 milestone on our issue tracker for details:

https://github.com/golang/go/issues?q=milestone%3AGo1.16.14+label%3ACherryPickApproved

full diff: https://github.com/golang/go/compare/go1.16.13...go1.16.14

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-02-15 16:22:00 +01:00
Akihiro Suda
879dd468dc
Merge pull request #43215 from thaJeztah/20.10_backport_fix_overlay_fuse_permissions
[20.10 backport] daemon/graphdriver/fuse-overlayfs: Init(): fix directory permissions (staticcheck)
2022-02-12 12:10:29 +09:00
Sebastiaan van Stijn
9edb93886a
Merge pull request #43151 from thaJeztah/20.10_backport_containerd_15
[20.10 backport] update containerd binary v1.5.9, runc v1.0.3, and some script changes
2022-02-10 20:36:31 +01:00
Sebastiaan van Stijn
878b9de935
daemon/graphdriver/fuse-overlayfs: Init(): fix directory permissions (staticcheck)
daemon/graphdriver/fuse-overlayfs/fuseoverlayfs.go:101:63: SA9002: file mode '700' evaluates to 01274; did you mean '0700'? (staticcheck)
        if err := idtools.MkdirAllAndChown(path.Join(home, linkDir), 700, currentID); err != nil {
                                                                     ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit f9fb5d4f25)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-02-08 13:38:29 +01:00
Sebastiaan van Stijn
606c29418a
Merge pull request #43177 from thaJeztah/20.10_update_distribution
[20.10] vendor: github.com/docker/distribution v2.8.0
2022-02-08 09:40:46 +01:00
Sebastiaan van Stijn
d1b3497bfa
[20.10] vendor: github.com/docker/distribution v2.8.0
full diff: 58f99e93b7...distribution:v2.8.0

(taking my own fork for the diff link, as the samuelkarp fork didn't have a reference to the upstream)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-02-07 18:37:21 +01:00
Brian Goff
dd61d8dc4b
Merge pull request #43165 from thaJeztah/20.10_backport_logs_fixes
[20.10 backport] daemon/logger: read the length header correctly
2022-01-27 10:15:00 -08:00
Sebastiaan van Stijn
f9344b45fe
api/swagger: move DistributionInspect to definitions
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 16cb04c2e8)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-20 16:03:21 +01:00
Sebastiaan van Stijn
13cb9d9723
api/swagger: rename PluginPrivilegeItem to PluginPrivilege
To match the name in Go

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit ebd709f80c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-20 16:03:19 +01:00
Sebastiaan van Stijn
7f9760e10c
api/swagger: fix up event-types and move to definitions
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 0b5a2e3c87)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-20 16:03:18 +01:00
Sebastiaan van Stijn
e6739a2884
api/swagger: move DistributionInspect to definitions
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit a65804adc4)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-20 16:03:17 +01:00
Sebastiaan van Stijn
d445d24804
api/swagger: rename PluginPrivilegeItem to PluginPrivilege
To match the name in Go

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 24a43d934c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-20 16:03:16 +01:00
Sebastiaan van Stijn
5e38ae84b2
api/swagger: fix up event-types and move to definitions
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 92ed6ca356)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-20 16:03:14 +01:00
Tobias Gesellchen
012fdff916
Fix swagger docs to match the opencontainers image-spec
Signed-off-by: Tobias Gesellchen <tobias@gesellix.de>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 62cf748df9)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-20 16:03:13 +01:00
Tobias Gesellchen
13cbf7fbb7
Extract PluginPrivilegeItem as explicit type definition
Signed-off-by: Tobias Gesellchen <tobias@gesellix.de>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 6153c2f08d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-20 16:03:12 +01:00
Tobias Gesellchen
8fbcf0611b
Use explicit object names for improved swagger based code generation
Signed-off-by: Tobias Gesellchen <tobias@gesellix.de>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit be93e50a58)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-20 16:03:11 +01:00
Tobias Gesellchen
b8bee972c4
Fix ContainerSummary swagger docs
Signed-off-by: Tobias Gesellchen <tobias@gesellix.de>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 8168d1877d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-20 16:03:09 +01:00
Tobias Gesellchen
7092a6091c
Add "changes" query parameter for /image/create to swagger docs
Signed-off-by: Tobias Gesellchen <tobias@gesellix.de>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 0e719f8a00)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-20 16:03:08 +01:00
Tobias Gesellchen
adf1e470a1
Add RestartPolicy "no" to swagger docs
Signed-off-by: Tobias Gesellchen <tobias@gesellix.de>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit d3d78c1ae3)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-20 16:03:07 +01:00
Tobias Gesellchen
b4b469eac2
Fix swagger docs to match the opencontainers image-spec
Signed-off-by: Tobias Gesellchen <tobias@gesellix.de>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit c0bc82cef1)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-20 16:03:06 +01:00
Tobias Gesellchen
2c38a2a635
Extract PluginPrivilegeItem as explicit type definition
Signed-off-by: Tobias Gesellchen <tobias@gesellix.de>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 7248ebcd7e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-20 16:03:04 +01:00
Tobias Gesellchen
621bfddd6e
Use explicit object names for improved swagger based code generation
Signed-off-by: Tobias Gesellchen <tobias@gesellix.de>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 6e421a1823)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-20 16:03:03 +01:00
Tobias Gesellchen
8a5240a8aa
Fix ContainerSummary swagger docs
Signed-off-by: Tobias Gesellchen <tobias@gesellix.de>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 9ac2d04617)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-20 16:03:02 +01:00
Tobias Gesellchen
751cf68e36
Add "changes" query parameter for /image/create to swagger docs
Signed-off-by: Tobias Gesellchen <tobias@gesellix.de>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 050f5f29f7)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-20 16:03:00 +01:00
Tobias Gesellchen
a961b76aef
Add RestartPolicy "no" to swagger docs
Signed-off-by: Tobias Gesellchen <tobias@gesellix.de>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 90294e9bdb)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-20 16:02:49 +01:00
Sebastiaan van Stijn
ad21bcd94e
Jenkinsfile: remove Windows RS1 as it reached end of support
It was already disabled by default, but removing it now that it reached
end of the line.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 9326ea5b99)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-20 10:14:29 +01:00
dmytro.iakovliev
b2684c1857
Fix for lack of syncromization in daemon/update.go
Signed-off-by: dmytro.iakovliev <dmytro.iakovliev@zodiacsystems.com>
(cherry picked from commit 58825ffc32)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-20 09:55:17 +01:00
Kazuyoshi Kato
8268f70ebb
daemon/logger: replace flaky TestFollowLogsHandleDecodeErr
Signed-off-by: Kazuyoshi Kato <katokazu@amazon.com>
(cherry picked from commit c91e09bee2)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-20 09:40:19 +01:00
Kazuyoshi Kato
78d0b936b8
daemon/logger: refactor followLogs to write more unit tests
followLogs() is getting really long (170+ lines) and complex.
The function has multiple inner functions that mutate its variables.

To refactor the function, this change introduces follow{} struct.
The inner functions are now defined as ordinal methods, which are
accessible from tests.

Signed-off-by: Kazuyoshi Kato <katokazu@amazon.com>
(cherry picked from commit 7a10f5a558)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-20 09:40:16 +01:00
Kazuyoshi Kato
39519221c2
daemon/logger: test followLogs' handleDecodeErr case
Signed-off-by: Kazuyoshi Kato <katokazu@amazon.com>
(cherry picked from commit f2e458ebc5)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-20 09:38:36 +01:00
Kazuyoshi Kato
ada1b01de1
daemon/logger: read the length header correctly
Before this change, if Decode() couldn't read a log record fully,
the subsequent invocation of Decode() would read the record's non-header part
as a header and cause a huge heap allocation.

This change prevents such a case by having the intermediate buffer in
the decoder struct.

Fixes #42125.

Signed-off-by: Kazuyoshi Kato <katokazu@amazon.com>
(cherry picked from commit 48d387a757)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-20 09:38:32 +01:00
Sebastiaan van Stijn
829f071228
update containerd binary to v1.5.9
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit df3ea5da03)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-20 09:28:34 +01:00
Akihiro Suda
3e5eea4192
update runc binary to v1.0.3
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit 53397ac539)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-20 09:28:32 +01:00
Sebastiaan van Stijn
bd42e17284
update containerd binary to v1.5.8
The eighth patch release for containerd 1.5 contains a mitigation for CVE-2021-41190
as well as several fixes and updates.

Notable Updates

* Handle ambiguous OCI manifest parsing
* Filter selinux xattr for image volumes in CRI plugin
* Use DeactiveLayer to unlock layers that cannot be renamed in Windows snapshotter
* Fix pull failure on unexpected EOF
* Close task IO before waiting on delete
* Log a warning for ignored invalid image labels rather than erroring
* Update pull to handle of non-https urls in descriptors

See the changelog for complete list of changes

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit aef782f348)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-20 09:28:30 +01:00
Sebastiaan van Stijn
3fd0b8d6eb
Update containerd binary to v1.5.7
The seventh patch release for containerd 1.5 is a security release to fix CVE-2021-41103.

Notable Updates:

- Fix insufficiently restricted permissions on container root and plugin directories
  GHSA-c2h3-6mxw-7mvq

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit fa4a9702be)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-20 09:28:28 +01:00
Sebastiaan van Stijn
3700adb70a
Update containerd binary to v1.5.6
- Install apparmor parser for arm64 and update seccomp to 2.5.1
- Update runc binary to 1.0.2
- Update hcsshim to v0.8.21 to fix layer issue on Windows Server 2019
- Add support for 'clone3' syscall to fix issue with certain images when seccomp is enabled
- Add image config labels in CRI container creation
- Fix panic in metadata content writer on copy error

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit b746a2bf9b)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-20 09:28:26 +01:00
Sebastiaan van Stijn
0f37f2989b
Dockerfile: update gotestsum to v1.7.0
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 1b8db06785)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-20 09:28:21 +01:00
Sebastiaan van Stijn
2716336abd
Dockerfile: use "go install" to install utilities
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 14ff070469)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-20 09:27:36 +01:00
Sebastiaan van Stijn
b35a1707e3
Dockerfile: use version for some utilities instead of commit-sha
The golangci-lint, gotestsum, shfmt, and vndr utilities should generally
be ok to be pinned by version instead of a specific sha. Also rename
the corresponding env-vars / build-args accordingly:

- GOLANGCI_LINT_COMMIT -> GOLANGCI_LINT_VERSION
- GOTESTSUM_COMMIT -> GOTESTSUM_VERSION
- SHFMT_COMMIT -> SHFMT_VERSION
- VNDR_COMMIT -> VNDR_VERSION
- CONTAINERD_COMMIT -> CONTAINERD_VERSION
- RUNC_COMMIT -> RUNC_VERSION
- ROOTLESS_COMMIT -> ROOTLESS_VERSION

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit a7a7c732c0)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-20 09:27:30 +01:00
Sebastiaan van Stijn
2a18825cdf
Dockerfile: remove GOPROXY override (was for go < 1.14)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 3cec4b8cd4)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-20 09:24:51 +01:00
Sebastiaan van Stijn
0f925d5d3d
remove deprecated "nokmem" build-tag for runc
This build-tag was removed in 52390d6804,
which is part of runc v1.0.0-rc94 and up, so no longer relevant.

the kmem options are now always disabled in runc.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 3c7c18a499)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-20 09:24:49 +01:00
Sebastiaan van Stijn
fb45fe614d
info: remove "expected" check for tini version
These checks were added when we required a specific version of containerd
and runc (different versions were known to be incompatible). I don't think
we had a similar requirement for tini, so this check was redundant. Let's
remove the check altogether.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit b585c64e2b)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-20 09:24:48 +01:00
Sebastiaan van Stijn
fd32c70031
update containerd binary to v1.5.5
Welcome to the v1.5.5 release of containerd!

The fifth patch release for containerd 1.5 updates runc to 1.0.1 and contains
other minor updates.

Notable Updates

- Update runc binary to 1.0.1
- Update pull logic to try next mirror on non-404 response
- Update pull authorization logic on redirect

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 4a07b89e9a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-20 09:24:46 +01:00
Sebastiaan van Stijn
302114634c
update containerd binary v1.4.8
Update to containerd 1.4.8 to address [CVE-2021-32760][1].

[1]: https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-32760

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit cf1328cd46)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-20 09:24:44 +01:00
Sebastiaan van Stijn
1cd13dcb6c
Update containerd binary to v1.5.3
full diff: https://github.com/containerd/containerd/compare/v1.5.2...v1.5.3

Welcome to the v1.5.3 release of containerd!

The third patch release for containerd 1.5 updates runc to 1.0.0 and contains
various other fixes.

Notable Updates

- Update runc binary to 1.0.0
- Send pod UID to CNI plugins as K8S_POD_UID
- Fix invalid validation error checking
- Fix error on image pull resume
- Fix User Agent sent to registry authentication server
- Fix symlink resolution for disk mounts on Windows

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 5ae2af41ee)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-20 09:24:42 +01:00
Sebastiaan van Stijn
5f09d5c76a
update containerd binary to v1.5.2
full diff: https://github.com/containerd/containerd/compare/v1.5.1...v1.5.2

The second patch release for containerd 1.5 is a security release to update
runc for CVE-2021-30465

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 8e3186fc8f)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-20 09:24:40 +01:00
Sebastiaan van Stijn
23f23c99ed
update containerd binary to v1.5.1
full diff: https://github.com/containerd/containerd/compare/v1.5.0...v1.5.1

Notable Updates

- Update runc to rc94
- Fix registry mirror authorization logic in CRI plugin
- Fix regression in cri-cni-release to include cri tools

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 22c0291333)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-20 09:24:38 +01:00
Sebastiaan van Stijn
f036a34c5b
update containerd binary to v1.5.0
Welcome to the v1.5.0 release of containerd!

The sixth major release of containerd includes many stability improvements
and code organization changes to make contribution easier and make future
features cleaner to develop. This includes bringing CRI development into the
main containerd repository and switching to Go modules. This release also
brings support for the Node Resource Interface (NRI).

Highlights
--------------------------------------------------------------------------------

*Project Organization*

- Merge containerd/cri codebase into containerd/containerd
- Move to Go modules
- Remove selinux build tag
- Add json log format output option for daemon log

*Snapshots*

- Add configurable overlayfs path
- Separate overlay implementation from plugin
- Native snapshotter configuration and plugin separation
- Devmapper snapshotter configuration and plugin separation
- AUFS snapshotter configuration and plugin separation
- ZFS snapshotter configuration and plugin separation
- Pass custom snapshot labels when creating snapshot
- Add platform check for snapshotter support when unpacking
- Handle loopback mounts
- Support userxattr mount option for overlay in user namespace
- ZFS snapshotter implementation of usage

*Distribution*

- Improve registry response errors
- Improve image pull performance over HTTP 1.1
- Registry configuration package
- Add support for layers compressed with zstd
- Allow arm64 to fallback to arm (v8, v7, v6, v5)

*Runtime*

- Add annotations to containerd task update API
- Add logging binary support when terminal is true
- Runtime support on FreeBSD

*Windows*

- Implement windowsDiff.Compare to allow outputting OCI images
- Optimize WCOW snapshotter to commit writable layers as read-only parent layers
- Optimize LCOW snapshotter use of scratch layers

*CRI*

- Add NRI injection points cri#1552
- Add support for registry host directory configuration
- Update privileged containers to use current capabilities instead of known capabilities
- Add pod annotations to CNI call
- Enable ocicrypt by default
- Support PID NamespaceMode_TARGET

Impactful Client Updates
--------------------------------------------------------------------------------

This release has changes which may affect projects which import containerd.

*Switch to Go modules*

containerd and all containerd sub-repositories are now using Go modules. This
should help make importing easier for handling transitive dependencies. As of
this release, containerd still does not guarantee client library compatibility
for 1.x versions, although best effort is made to minimize impact from changes
to exported Go packages.

*CRI plugin moved to main repository*

With the CRI plugin moving into the main repository, imports under github.com/containerd/cri/
can now be found github.com/containerd/containerd/pkg/cri/.
There are no changes required for end users of CRI.

*Library changes*

oci

The WithAllCapabilities has been removed and replaced with WithAllCurrentCapabilities
and WithAllKnownCapabilities. WithAllKnownCapabilities has similar
functionality to the previous WithAllCapabilities with added support for newer
capabilities. WithAllCurrentCapabilities can be used to give privileged
containers the same set of permissions as the calling process, preventing errors
when privileged containers attempt to get more permissions than given to the
caller.

*Configuration changes*

New registry.config_path for CRI plugin

registry.config_path specifies a directory to look for registry hosts
configuration. When resolving an image name during pull operations, the CRI
plugin will look in the <registry.config_path>/<image hostname>/ directory
for host configuration. An optional hosts.toml file in that directory may be
used to configure which hosts will be used for the pull operation as well
host-specific configurations. Updates under that directory do not require
restarting the containerd daemon.

Enable registry.config_path in the containerd configuration file.

    [plugins."io.containerd.grpc.v1.cri".registry]
       config_path = "/etc/containerd/certs.d"
    Configure registry hosts, such as /etc/containerd/certs.d/docker.io/hosts.toml
    for any image under the docker.io namespace (any image on Docker Hub).

    server = "https://registry-1.docker.io"

    [host."https://public-mirror.example.com"]
      capabilities = ["pull"]
    [host."https://docker-mirror.internal"]
      capabilities = ["pull", "resolve"]
      ca = "docker-mirror.crt"

If no hosts.toml configuration exists in the host directory, it will fallback
to check certificate files based on Docker's certificate file
pattern (".crt" files for CA certificates and ".cert"/".key" files for client
certificates).

*Deprecation of registry.mirrors and registry.configs in CRI plugin*

Mirroring and TLS can now be configured using the new registry.config_path
option. Existing configurations may be migrated to new host directory
configuration. These fields are only deprecated with no planned removal,
however, these configurations cannot be used while registry.config_path is
defined.

*Version 1 schema is deprecated*

Version 2 of the containerd configuration toml is recommended format and the
default. Starting this version, a deprecation warning will be logged when
version 1 is used.

To check version, see the version value in the containerd toml configuration.

    version=2

FreeBSD Runtime Support (Experimental)
--------------------------------------------------------------------------------

This release includes changes that allow containerd to run on FreeBSD with a
compatible runtime, such as runj. This
support should be considered experimental and currently there are no official
binary releases for FreeBSD. The runtimes used by containerd are maintained
separately and have their own stability guarantees. The containerd project
strives to be compatible with any runtime which aims to implement containerd's
shim API and OCI runtime specification.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 9b2f55bc1c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-20 09:24:36 +01:00
Sebastiaan van Stijn
1dd37750a6
Revert "[20.10] update containerd binary to v1.4.5"
This reverts commit 01f734cb4f.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-20 09:24:34 +01:00
Sebastiaan van Stijn
b097d29705
Revert "[20.10] update containerd binary to v1.4.6"
This reverts commit 56541eca9a.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-20 09:24:32 +01:00
Sebastiaan van Stijn
de656f9da4
Revert "[20.10] update containerd binary to v1.4.7"
This reverts commit 793340a33a.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-20 09:24:30 +01:00
Sebastiaan van Stijn
9e36f77577
Revert "[20.10] update containerd binary v1.4.8"
This reverts commit 067918a8c3.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-20 09:24:28 +01:00
Sebastiaan van Stijn
eb2acf2fb3
Revert "[20.10] update containerd binary to v1.4.9"
This reverts commit e8fb8f7acd.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-20 09:24:26 +01:00
Sebastiaan van Stijn
4e838e50ea
Revert "[20.10] update containerd binary to v1.4.10"
This reverts commit 6835d15f55.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-20 09:24:24 +01:00
Sebastiaan van Stijn
79fd9c1541
Revert "[20.10] update containerd binary to v1.4.11"
This reverts commit 129a2000cf.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-20 09:24:22 +01:00
Sebastiaan van Stijn
13de46fd4b
Revert "[20.10] update containerd binary to v1.4.12"
This reverts commit d47de2a4c7.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-20 09:24:19 +01:00
Sebastiaan van Stijn
22ff2ed34b
Merge pull request #43147 from PettitWesley/backport-fluentd-fix
[20.10 backport] backport fluentd log driver async connect fix
2022-01-20 09:23:15 +01:00
Sebastiaan van Stijn
b106f7dfd0
Merge pull request #43153 from thaJeztah/20.10_bump_go_1.16.13
[20.10] update Go to 1.16.13
2022-01-18 17:37:54 +01:00
Sebastiaan van Stijn
aa92e697cb
[20.10] update Go to 1.16.13
go1.16.13 (released 2022-01-06) includes fixes to the compiler, linker, runtime,
and the net/http package. See the Go 1.16.13 milestone on our issue tracker for
details:

https://github.com/golang/go/issues?q=milestone%3AGo1.16.13+label%3ACherryPickApproved

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-13 16:44:11 +01:00
Albin Kerouanton
f9df098e76 fluentd: Turn ForceStopAsyncSend true when async connect is used
The flag ForceStopAsyncSend was added to fluent logger lib in v1.5.0 (at
this time named AsyncStop) to tell fluentd to abort sending logs
asynchronously as soon as possible, when its Close() method is called.
However this flag was broken because of the way the lib was handling it
(basically, the lib could be stucked in retry-connect loop without
checking this flag).

Since fluent logger lib v1.7.0, calling Close() (when ForceStopAsyncSend
is true) will really stop all ongoing send/connect procedure,
wherever it's stucked.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
(cherry picked from commit bd61629b6b)
Signed-off-by: Wesley <wppttt@amazon.com>
2022-01-13 01:06:12 +00:00
Albin Kerouanton
81fc02b7e1 vendor: github.com/fluent/fluent-logger-golang v1.8.0
Updates the fluent logger library to v1.8.0. Following PRs/commits were
merged since last bump:

* [Add callback for error handling when using
  async](https://github.com/fluent/fluent-logger-golang/pull/97)
* [Fix panic when accessing unexported struct
  field](https://github.com/fluent/fluent-logger-golang/pull/99)
* [Properly stop logger during (re)connect
  failure](https://github.com/fluent/fluent-logger-golang/pull/82)
* [Support a TLS-enabled connection](e5d6aa13b7)

See https://github.com/fluent/fluent-logger-golang/compare/v1.6.1..v1.8.0

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
(cherry picked from commit e24d61b7ef)
Signed-off-by: Wesley <wppttt@amazon.com>
2022-01-13 01:05:52 +00:00
Cam
d6f3add5c6 vendor: github.com/fluent/fluent-logger-golang 1.6.1
Updates the fluent logger library. Namely this fixes a couple places
where the library could panic when closing and writing to channels.

see https://github.com/fluent/fluent-logger-golang/pull/93 and
https://github.com/fluent/fluent-logger-golang/pull/95

closes #40829
closes #32567

Signed-off-by: Cam <gh@sparr.email>
(cherry picked from commit a6a98d6928)
Signed-off-by: Wesley <wppttt@amazon.com>
2022-01-13 01:05:12 +00:00
Tianon Gravi
b1fc0c84de
Merge pull request #43084 from AkihiroSuda/cherrypick-42736
[20.10 backport] daemon.WithCommonOptions() fix detection of user-namespaces
2022-01-07 16:09:12 -08:00
Sebastiaan van Stijn
660b9962e4
daemon.WithCommonOptions() fix detection of user-namespaces
Commit dae652e2e5 added support for non-privileged
containers to use ICMP_PROTO (used for `ping`). This option cannot be set for
containers that have user-namespaces enabled.

However, the detection looks to be incorrect; HostConfig.UsernsMode was added
in 6993e891d1 / ee2183881b,
and the property only has meaning if the daemon is running with user namespaces
enabled. In other situations, the property has no meaning.
As a result of the above, the sysctl would only be set for containers running
with UsernsMode=host on a daemon running with user-namespaces enabled.

This patch adds a check if the daemon has user-namespaces enabled (RemappedRoot
having a non-empty value), or if the daemon is running inside a user namespace
(e.g. rootless mode) to fix the detection.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit a826ca3aef)

---
The cherry-pick was almost clean but `userns.RunningInUserNS()` -> `sys.RunningInUserNS()`.

Fix docker/buildx issue 561
---

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-12-15 18:20:07 +09:00
Justin Cormack
459d0dfbbb
Merge pull request #43077 from thaJeztah/20.10_bump_go_1.16.12
[20.10] update Go to 1.16.12
2021-12-12 10:11:51 +00:00
Sebastiaan van Stijn
a621bc007b
[20.10] update Go to 1.16.12
go1.16.12 (released 2021-12-09) includes security fixes to the syscall and net/http
packages. See the Go 1.16.12 milestone on the issue tracker for details:

https://github.com/golang/go/issues?q=milestone%3AGo1.16.12+label%3ACherryPickApproved

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-12-12 01:15:00 +01:00
Brian Goff
4bed71ae9a
Merge pull request #43063 from thaJeztah/20.10_bump_go_1.16.11
[20.10] update Go to 1.16.11
2021-12-08 10:09:17 -08:00
Sebastiaan van Stijn
f4daf9dd08
[20.10] update Go to 1.16.11
go1.16.11 (released 2021-12-02) includes fixes to the compiler, runtime, and the
net/http, net/http/httptest, and time packages. See the Go 1.16.11 milestone on
the issue tracker for details:

https://github.com/golang/go/issues?q=milestone%3AGo1.16.11+label%3ACherryPickApproved

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-12-06 10:16:12 +01:00
Sebastiaan van Stijn
847da184ad
Merge pull request #43004 from AkihiroSuda/cherrypick-42152
[20.10 backport] info: unset cgroup-related fields when CgroupDriver == none
2021-11-18 01:21:59 +01:00
Sebastiaan van Stijn
4a27cd1a1b
Merge pull request #43027 from thaJeztah/20.10_backport_update_image_spec
[20.10 backport] vendor: github.com/opencontainers/image-spec v1.0.2
2021-11-18 01:16:31 +01:00
Sebastiaan van Stijn
7568123fc4
Merge pull request #43023 from thaJeztah/20.10_bump_buildkit
[20.10] vendor: github.com/moby/buildkit v0.8.3-4-gbc07b2b8
2021-11-18 00:18:46 +01:00
Sebastiaan van Stijn
c98869341b
Merge pull request #43024 from thaJeztah/20.10_containerd_1.4.12
[20.10] update containerd binary to v1.4.12
2021-11-18 00:04:51 +01:00
Sebastiaan van Stijn
dc015972bb
vendor: github.com/opencontainers/image-spec v1.0.2
- Bring mediaType out of reserved status
- specs-go: adding mediaType to the index and manifest structures

full diff: https://github.com/opencontainers/image-spec/compare/v1.0.1...v1.0.2

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit cef0a7c14e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-11-18 00:03:29 +01:00
Sebastiaan van Stijn
11b3bfee6c
Merge pull request #43028 from thaJeztah/20.10_fix_vendor_conf
[20.10] fix vendor validation
2021-11-17 23:08:36 +01:00
Sebastiaan van Stijn
e0108db2bd
[20.10] fix vendor validation
Looks like vndr didn't like the replace rule missing a scheme;

    github.com/docker/distribution: Err: exit status 128, out: fatal: repository 'github.com/samuelkarp/docker-distribution' does not exist
    github.com/containerd/containerd: Err: exit status 128, out: fatal: repository 'github.com/moby/containerd' does not exist

While at it, I also replaced the schem for go-immutable-radix, because GitHub
is deprecating the git:// protocol.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-11-17 22:13:44 +01:00
Sebastiaan van Stijn
d47de2a4c7
[20.10] update containerd binary to v1.4.12
The twelfth patch release for containerd 1.4 contains a few minor bug fixes
and an update to mitigate CVE-2021-41190.

Notable Updates

* Handle ambiguous OCI manifest parsing GHSA-5j5w-g665-5m35
* Update pull to try next mirror for non-404 errors
* Update pull to handle of non-https urls in descriptors

See the changelog for complete list of changes

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-11-17 21:01:41 +01:00
Sebastiaan van Stijn
da9c983789
[20.10] vendor: github.com/moby/buildkit v0.8.3-4-gbc07b2b8
imageutil: make mediatype detection more stricter to mitigate CVE-2021-41190.

full diff: 244e8cde63...bc07b2b81b

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-11-17 20:40:17 +01:00
Sebastiaan van Stijn
10106a0f66
Merge pull request from GHSA-xmmx-7jpf-fx42
[20.10] vendor: update github.com/docker/distribution and github.com/containerd/containerd
2021-11-17 20:34:50 +01:00
Samuel Karp
c1f352c4b1
distribution: validate blob type
Signed-off-by: Samuel Karp <skarp@amazon.com>
2021-11-15 14:25:52 -08:00
Samuel Karp
c96ed28f2f
vendor: update github.com/containerd/containerd
Signed-off-by: Samuel Karp <skarp@amazon.com>
2021-11-15 14:25:52 -08:00
Akihiro Suda
7bd682c48c
Merge pull request #43008 from thaJeztah/20.10_backport_fix_TestBuildUserNamespaceValidateCapabilitiesAreV2
[20.10 backport] TestBuildUserNamespaceValidateCapabilitiesAreV2: cleanup daemon storage
2021-11-11 15:36:36 +09:00
Sebastiaan van Stijn
7677aeafd7
TestBuildUserNamespaceValidateCapabilitiesAreV2: cleanup daemon storage
This should help with Jenkins failing to clean up the Workspace:

- make sure "cleanup" is also called in the defer for all daemons. keeping
  the daemon's storage around prevented Jenkins from cleaning up.
- close client connections and some readers (just to be sure)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit eea2758761)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-11-10 14:12:12 +01:00
Sebastiaan van Stijn
34eb6fbe60
testutil: daemon.Cleanup(): cleanup more directories
The storage-driver directory caused Jenkins cleanup to fail. While at it, also
removing other directories that we do not include in the "bundles" that are
stored as Jenkins artifacts.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 1a15a1a061)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-11-10 14:12:09 +01:00
Sebastiaan van Stijn
c7b97c306a
Merge pull request #42987 from thaJeztah/20.10_backport_create_panic_log_without_readonly
[20.10 backport] [Windows]] cmd/dockerd: create panic.log file without readonly flag
2021-11-09 22:05:55 +01:00
Akihiro Suda
0e76a0a418
info: unset cgroup-related fields when CgroupDriver == none
Fix issue 42151

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit 039e9670cb)
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-11-09 18:43:44 +09:00
Tianon Gravi
d9b2f4d0c8
Merge pull request #42989 from thaJeztah/20.10_bump_go_1.16.10
[20.10] Update Go to 1.16.10
2021-11-08 13:05:28 -08:00
Sebastiaan van Stijn
c7edd308ad
[20.10] Update Go to 1.16.10
go1.16.10 (released 2021-11-04) includes security fixes to the archive/zip and
debug/macho packages, as well as bug fixes to the compiler, linker, runtime, the
misc/wasm directory, and to the net/http package. See the Go 1.16.10 milestone
for details: https://github.com/golang/go/issues?q=milestone%3AGo1.16.10+label%3ACherryPickApproved

From the announcement e-mail:

[security] Go 1.17.3 and Go 1.16.10 are released

We have just released Go versions 1.17.3 and 1.16.10, minor point releases.
These minor releases include two security fixes following the security policy:

- archive/zip: don't panic on (*Reader).Open
  Reader.Open (the API implementing io/fs.FS introduced in Go 1.16) can be made
  to panic by an attacker providing either a crafted ZIP archive containing
  completely invalid names or an empty filename argument.
  Thank you to Colin Arnott, SiteHost and Noah Santschi-Cooney, Sourcegraph Code
  Intelligence Team for reporting this issue. This is CVE-2021-41772 and Go issue
  golang.org/issue/48085.
- debug/macho: invalid dynamic symbol table command can cause panic
  Malformed binaries parsed using Open or OpenFat can cause a panic when calling
  ImportedSymbols, due to an out-of-bounds slice operation.
  Thanks to Burak Çarıkçı - Yunus Yıldırım (CT-Zer0 Crypttech) for reporting this
  issue. This is CVE-2021-41771 and Go issue golang.org/issue/48990.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-11-05 11:10:49 +01:00
Samuel Karp
b3456925ca
vendor: update github.com/docker/distribution
Signed-off-by: Samuel Karp <skarp@amazon.com>
2021-11-04 14:41:33 -07:00
Aleksandr Chebotov
6611c72b65
cmd/dockerd: create panic.log file without readonly flag
Signed-off-by: Aleksandr Chebotov <v-aleche@microsoft.com>
(cherry picked from commit b865204042)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-11-03 14:30:08 +01:00
Sebastiaan van Stijn
9a309c6165
Merge pull request #42971 from thaJeztah/20.10_backport_fix_TestCreateServiceSecretFileMode
[20.10 backport] Fix race in TestCreateServiceSecretFileMode, TestCreateServiceConfigFileMode
2021-11-03 14:29:53 +01:00
Sebastiaan van Stijn
4b9a3dac46
Fix race in TestCreateServiceSecretFileMode, TestCreateServiceConfigFileMode
Looks like this test was broken from the start, and fully relied on a race
condition. (Test was added in 65ee7fff02)

The problem is in the service's command: `ls -l /etc/config || /bin/top`, which
will either:

- exit immediately if the secret is mounted correctly at `/etc/config` (which it should)
- keep running with `/bin/top` if the above failed

After the service is created, the test enters a race-condition, checking for 1
task to be running (which it ocassionally is), after which it proceeds, and looks
up the list of tasks of the service, to get the log output of `ls -l /etc/config`.

This is another race: first of all, the original filter for that task lookup did
not filter by `running`, so it would pick "any" task of the service (either failed,
running, or "completed" (successfully exited) tasks).

In the meantime though, SwarmKit kept reconciling the service, and creating new
tasks, so even if the test was able to get the ID of the correct task, that task
may already have been exited, and removed (task-limit is 5 by default), so only
if the test was "lucky", it would be able to get the logs, but of course, chances
were likely that it would be "too late", and the task already gone.

The problem can be easily reproduced when running the steps manually:

    echo 'CONFIG' | docker config create myconfig -

    docker service create --config source=myconfig,target=/etc/config,mode=0777 --name myservice busybox sh -c 'ls -l /etc/config || /bin/top'

The above creates the service, but it keeps retrying, because each task exits
immediately (followed by SwarmKit reconciling and starting a new task);

    mjntpfkkyuuc1dpay4h00c4oo
    overall progress: 0 out of 1 tasks
    1/1: ready     [======================================>            ]
    verify: Detected task failure
    ^COperation continuing in background.
    Use `docker service ps mjntpfkkyuuc1dpay4h00c4oo` to check progress.

And checking the tasks for the service reveals that tasks exit cleanly (no error),
but _do exit_, so swarm just keeps up reconciling, and spinning up new tasks;

    docker service ps myservice --no-trunc
    ID                          NAME              IMAGE                                                                                    NODE             DESIRED STATE   CURRENT STATE                     ERROR     PORTS
    2wmcuv4vffnet8nybg3he4v9n   myservice.1       busybox:latest@sha256:f7ca5a32c10d51aeda3b4d01c61c6061f497893d7f6628b92f822f7117182a57   docker-desktop   Ready           Ready less than a second ago
    5p8b006uec125iq2892lxay64    \_ myservice.1   busybox:latest@sha256:f7ca5a32c10d51aeda3b4d01c61c6061f497893d7f6628b92f822f7117182a57   docker-desktop   Shutdown        Complete less than a second ago
    k8lpsvlak4b3nil0zfkexw61p    \_ myservice.1   busybox:latest@sha256:f7ca5a32c10d51aeda3b4d01c61c6061f497893d7f6628b92f822f7117182a57   docker-desktop   Shutdown        Complete 6 seconds ago
    vsunl5pi7e2n9ol3p89kvj6pn    \_ myservice.1   busybox:latest@sha256:f7ca5a32c10d51aeda3b4d01c61c6061f497893d7f6628b92f822f7117182a57   docker-desktop   Shutdown        Complete 11 seconds ago
    orxl8b6kt2l6dfznzzd4lij4s    \_ myservice.1   busybox:latest@sha256:f7ca5a32c10d51aeda3b4d01c61c6061f497893d7f6628b92f822f7117182a57   docker-desktop   Shutdown        Complete 17 seconds ago

This patch changes the service's command to `sleep`, so that a successful task
(after successfully performing `ls -l /etc/config`) continues to be running until
the service is deleted. With that change, the service should (usually) reconcile
immediately, which removes the race condition, and should also make it faster :)

This patch changes the tests to use client.ServiceLogs() instead of using the
service's tasklist to directly access container logs. This should also fix some
failures that happened if some tasks failed to start before reconciling, in which
case client.TaskList() (with the current filters), could return more tasks than
anticipated (as it also contained the exited tasks);

    === RUN   TestCreateServiceSecretFileMode
        create_test.go:291: assertion failed: 2 (int) != 1 (int)
    --- FAIL: TestCreateServiceSecretFileMode (7.88s)
    === RUN   TestCreateServiceConfigFileMode
        create_test.go:355: assertion failed: 2 (int) != 1 (int)
    --- FAIL: TestCreateServiceConfigFileMode (7.87s)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 13cff6d583)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-10-27 12:30:35 +02:00
Sebastiaan van Stijn
e2f740de44
Merge pull request #42959 from thaJeztah/20.10_backport_fix_racey_health_test
[20.10 backport] Fix racey TestHealthKillContainer
2021-10-22 17:32:27 +02:00
Brian Goff
acb4f263b3
Fix racey TestHealthKillContainer
Before this change if you assume that things work the way the test
expects them to (it does not, but lets assume for now) we aren't really
testing anything because we are testing that a container is healthy
before and after we send a signal. This will give false positives even
if there is a bug in the underlying code. Sending a signal can take any
amount of time to cause a container to exit or to trigger healthchecks
to stop or whatever.

Now lets remove the assumption that things are working as expected,
because they are not.
In this case, `top` (which is what is running in the container) is
actually exiting when it receives `USR1`.
This totally invalidates the test.

We need more control and knowledge as to what is happening in the
container to properly test this.
This change introduces a custom script which traps `USR1` and flips the
health status each time the signal is received.
We then send the signal twice so that we know the change has occurred
and check that the value has flipped so that we know the change has
actually occurred.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit 27ba755f70)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-10-22 09:30:08 +02:00
Sebastiaan van Stijn
8ab80e4a20
Merge pull request #42957 from thaJeztah/20.10_backport_rootless_quotes
[20.10 backport] rootless script fixes
2021-10-22 08:17:00 +02:00
Sebastiaan van Stijn
52f5b613ba
Merge pull request #42956 from cpuguy83/20.10-backport-kill-fixes
[20.10] Backport fixes for kill/stop handling
2021-10-21 20:10:22 +02:00
Sebastiaan van Stijn
a4b1ae5153
Merge pull request #42954 from crazy-max/20.10_build-local-normalized
[20.10 backport] buildkit: normalize build target and local platform
2021-10-21 20:07:21 +02:00
Chenyang Yan
59d2a2c397
dockerd-rootless.sh: Fix variable not double quotes cause unexpected behavior
```
$ cat test.sh

echo "orign value=$XDG_RUNTIME_DIR"

echo "1. with [ ] not quote ..."
[ -w $XDG_RUNTIME_DIR ]
echo "get 1 ret_code: $?"

echo "2. with [ ] and quote ..."
[ -w "$XDG_RUNTIME_DIR" ]
echo "get 2 ret_code: $?"

$ sh ./test.sh
orign value=
1. with [ ] not quote ...
get 1 ret_code: 0
2. with [ ] and quote ...
get 2 ret_code: 1

$ bash ./test.sh
orign value=
1. with [ ] not quote ...
get 1 ret_code: 0
2. with [ ] and quote ...
get 2 ret_code: 1
```

Signed-off-by: Chenyang Yan <memory.yancy@gmail.com>
(cherry picked from commit a8ce4d47c3)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-10-21 19:51:55 +02:00
Hugo Barrera
2c6aa5aad9
Remove needless check
Starting `dockerd-rootless.sh` checks that `$HOME` is writeable, but does not
require it to be so.
Make the check more precise, and check that it actually exists and is a
directory.

Signed-off-by: Hugo Osvaldo Barrera <hugo@barrera.io>
(cherry picked from commit 3980d0462d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-10-21 19:51:49 +02:00
Cam
3285c27503 Fix log statement 'failed to exit' timeout accuracy
log statement should reflect how long it actually waited, not how long
it theoretically could wait based on the 'seconds' integer passed in.

Signed-off-by: Cam <gh@sparr.email>
(cherry picked from commit d15ce134ef)
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2021-10-20 22:35:44 +00:00
Cam
a4bcd4c64f docker daemon container stop refactor
this refactors the Stop command to fix a few issues and behaviors that
dont seem completely correct:

1. first it fixes a situation where stop could hang forever (#41579)
2. fixes a behavior where if sending the
stop signal failed, then the code directly sends a -9 signal. If that
fails, it returns without waiting for the process to exit or going
through the full docker kill codepath.
3. fixes a behavior where if sending the stop signal failed, then the
code sends a -9 signal. If that succeeds, then we still go through the
same stop waiting process, and may even go through the docker kill path
again, even though we've already sent a -9.
4. fixes a behavior where the code would wait the full 30 seconds after
sending a stop signal, even if we already know the stop signal failed.

fixes #41579

Signed-off-by: Cam <gh@sparr.email>
(cherry picked from commit 8e362b75cb)
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2021-10-20 22:35:40 +00:00
Cam
bed624fdc9 docker kill: fix bug where failed kills didnt fallback to unix kill
1. fixes #41587
2. removes potential infinite Wait and goroutine leak at end of kill
function

fixes #41587

Signed-off-by: Cam <gh@sparr.email>
(cherry picked from commit e57a365ab1)
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2021-10-20 22:33:15 +00:00
CrazyMax
80b7e8b5d7 buildkit: normalize build target and local platform
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit b4e056d556)
2021-10-20 12:47:07 +02:00
Sebastiaan van Stijn
9f5b26fb86
Merge pull request #42944 from kevpar/20.10_update-winio
[20.10] vendor: Update go-winio to v0.4.20
2021-10-18 14:50:48 +02:00
Kevin Parsons
c2b9a32875 vendor: Update go-winio to v0.4.20
Updates go-winio to the latest v0.4.x version. The main important fix
here is to go-winio's backuptar package. This is needed to fix a bug in
sparse file handling in container layers, which was exposed by a recent
change in Windows.

go-winio v0.4.20: https://github.com/microsoft/go-winio/releases/tag/v0.4.20

Signed-off-by: Kevin Parsons <kevpar@microsoft.com>
2021-10-15 15:24:08 -07:00
Sebastiaan van Stijn
dc084ac10e
Merge pull request #42923 from thaJeztah/20.10_bump_go_1.16.9
[20.10] Update Go to 1.16.9
2021-10-15 12:57:25 +02:00
Sebastiaan van Stijn
a8a4b81d6a
Merge pull request #42901 from thaJeztah/20.10_update_containerd_1.4.10
[20.10] update containerd binary to v1.4.11
2021-10-14 20:32:09 +02:00
Sebastiaan van Stijn
c580a02873
[20.10] Update Go to 1.16.9
go1.16.9 (released 2021-10-07) includes a security fix to the linker and misc/wasm
directory, as well as bug fixes to the runtime and to the text/template package.
See the Go 1.16.9 milestone on our issue tracker for details:

https://github.com/golang/go/issues?q=milestone%3AGo1.16.9+label%3ACherryPickApproved

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-10-08 15:14:58 +02:00
Sebastiaan van Stijn
da3769688a
Merge pull request #42875 from awmirantis/bump_swarmkit_fix_rollback
[20.10] Bump swarmkit to get fix for rollback
2021-10-07 15:15:42 +02:00
Tianon Gravi
59beba295e
Merge pull request #42844 from AkihiroSuda/cherrypick-42764
[20.10 backport] update runc binary to v1.0.2
2021-10-06 17:33:46 -07:00
Tianon Gravi
e086edaadf
Merge pull request #42760 from AkihiroSuda/cherrypick-42708
[20.10 backport] bump up rootlesskit to v0.14.4
2021-10-06 17:32:53 -07:00
Sebastiaan van Stijn
129a2000cf
[20.10] update containerd binary to v1.4.11
The eleventh patch release for containerd 1.4 is a security release to fix CVE-2021-41103.

Notable Updates

- Fix insufficiently restricted permissions on container root and plugin directories GHSA-c2h3-6mxw-7mvq

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-10-04 21:15:47 +02:00
Sebastiaan van Stijn
6835d15f55
[20.10] update containerd binary to v1.4.10
- Update runc to v1.0.2
- Update hcsshim to v0.8.21
- Support "clone3" in default seccomp profile
- Fix panic in metadata content writer on copy error

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-10-04 21:14:20 +02:00
Sebastiaan van Stijn
9cd3585bf5
Merge pull request #42843 from AkihiroSuda/cherrypick-42791
[20.10 backport] cmd/dockerd: add the link of "the documentation"
2021-10-04 21:10:52 +02:00
Sebastiaan van Stijn
1c2ca1f624
Merge pull request #42869 from pete-woods/backport-quota-package-cgo-fix
[20.10 backport] quota: adjust build-tags to allow build without CGO
2021-10-04 21:10:12 +02:00
Sebastiaan van Stijn
977283509f
Merge pull request #42836 from tianon/20.10-clone3
[20.10 backport] seccomp: add support for "clone3" syscall in default policy
2021-10-04 21:08:57 +02:00
Sebastiaan van Stijn
79ea9d3080
Merge pull request #5 from moby/20.10_bump_go_1.16.8
[20.10] Update Go to 1.16.8
2021-09-23 20:26:18 +02:00
Adam Williams
5730c139f7 Bump swarmkit to get fix for rollback
Signed-off-by: Adam Williams <awilliams@mirantis.com>
2021-09-22 11:21:01 -07:00
Tibor Vass
59f10e3435
quota: adjust build-tags to allow build without CGO
This is to allow quota package (without tests) to be built without cgo.
makeBackingFsDev was used in helpers but not defined in projectquota_unsupported.go

Also adjust some GoDoc to follow the standard format.

Signed-off-by: Tibor Vass <tibor@docker.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 7cf079acdb)
Signed-off-by: Pete Woods <pete.woods@circleci.com>
2021-09-20 14:19:22 +01:00
Sebastiaan van Stijn
fa78afebcf
Update Go to 1.16.8
This includes additional fixes for CVE-2021-39293.

go1.16.8 (released 2021-09-09) includes a security fix to the archive/zip package,
as well as bug fixes to the archive/zip, go/internal/gccgoimporter, html/template,
net/http, and runtime/pprof packages. See the Go 1.16.8 milestone on the issue
tracker for details:

https://github.com/golang/go/issues?q=milestone%3AGo1.16.8+label%3ACherryPickApproved

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-09-15 13:48:01 +02:00
Tianon Gravi
567c01f6d1 seccomp: add support for "clone3" syscall in default policy
This is a backport of 9f6b562dd1, adapted to avoid the refactoring that happened in d92739713c.

Original commit message is as follows:

> If no seccomp policy is requested, then the built-in default policy in
> dockerd applies. This has no rule for "clone3" defined, nor any default
> errno defined. So when runc receives the config it attempts to determine
> a default errno, using logic defined in its commit:
>
>   opencontainers/runc@7a8d716
>
> As explained in the above commit message, runc uses a heuristic to
> decide which errno to return by default:
>
> [quote]
>   The solution applied here is to prepend a "stub" filter which returns
>   -ENOSYS if the requested syscall has a larger syscall number than any
>   syscall mentioned in the filter. The reason for this specific rule is
>   that syscall numbers are (roughly) allocated sequentially and thus newer
>   syscalls will (usually) have a larger syscall number -- thus causing our
>   filters to produce -ENOSYS if the filter was written before the syscall
>   existed.
> [/quote]
>
> Unfortunately clone3 appears to one of the edge cases that does not
> result in use of ENOSYS, instead ending up with the historical EPERM
> errno.
>
> Latest glibc (2.33.9000, in Fedora 35 rawhide) will attempt to use
> clone3 by default. If it sees ENOSYS then it will automatically
> fallback to using clone. Any other errno is treated as a fatal
> error. Thus when docker seccomp policy triggers EPERM from clone3,
> no fallback occurs and programs are thus unable to spawn threads.
>
> The clone3 syscall is much more complicated than clone, most notably its
> flags are not exposed as a directly argument any more. Instead they are
> hidden inside a struct. This means that seccomp filters are unable to
> apply policy based on values seen in flags. Thus we can't directly
> replicate the current "clone" filtering for "clone3". We can at least
> ensure "clone3" returns ENOSYS errno, to trigger fallback to "clone"
> at which point we can filter on flags.

Signed-off-by: Tianon Gravi <admwiggin@gmail.com>
Co-authored-by: Daniel P. Berrangé <berrange@redhat.com>
2021-09-13 08:56:21 -07:00
Akihiro Suda
07728cd2bd
update runc binary to v1.0.2
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit 14189170d1)
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-09-13 15:19:53 +09:00
Akihiro Suda
964768f200
cmd/dockerd: add the link of "the documentation"
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit 1a67e9572e)
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-09-13 15:17:42 +09:00
Sebastiaan van Stijn
bce32e5c93
Merge pull request #4 from moby/20.10-ghsa-v994-f8vw-g7j4-chroot-mkdir
[20.10] chrootarchive: don't create parent dirs outside of chroot
2021-09-09 20:50:12 +02:00
Sebastiaan van Stijn
f0ab919f51
Merge pull request #2 from moby/20.10-GHSA-3fwx-pjgw-3558_0701-perms
[20.10] Lock down docker root dir perms.
2021-09-09 20:45:08 +02:00
Tonis Tiigi
80f1169eca chrootarchive: don't create parent dirs outside of chroot
If chroot is used with a special root directory then create
destination directory within chroot. This works automatically
already due to extractor creating parent paths and is only
used currently with cp where parent paths are actually required
and error will be shown to user before reaching this point.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit 52d285184068998c22632bfb869f6294b5613a58)
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2021-08-19 20:49:31 +00:00
Brian Goff
93ac040bf0 Lock down docker root dir perms.
Do not use 0701 perms.
0701 dir perms allows anyone to traverse the docker dir.
It happens to allow any user to execute, as an example, suid binaries
from image rootfs dirs because it allows traversal AND critically
container users need to be able to do execute things.

0701 on lower directories also happens to allow any user to modify
     things in, for instance, the overlay upper dir which neccessarily
     has 0755 permissions.

This changes to use 0710 which allows users in the group to traverse.
In userns mode the UID owner is (real) root and the GID is the remapped
root's GID.

This prevents anyone but the remapped root to traverse our directories
(which is required for userns with runc).

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit ef7237442147441a7cadcda0600be1186d81ac73)
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2021-08-19 20:40:15 +00:00
Akihiro Suda
b0c0b73798
bump up rootlesskit to v0.14.4
Fixes `panic: tap2vif: read: read /dev/net/tun: not pollable` on early
start up of RootlessKit with VPNKit.

Changes:
- https://github.com/rootless-containers/rootlesskit/releases/tag/v0.14.4
- https://github.com/rootless-containers/rootlesskit/releases/tag/v0.14.3

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit 9499acc360)
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-08-19 16:21:01 +09:00
Sebastiaan van Stijn
d24c6dc5cf
Merge pull request #42721 from thaJeztah/20.10_backport_bump_go_1.16.7
[20.10 backport] Update Go to 1.16.7
2021-08-17 14:12:42 +02:00
Sebastiaan van Stijn
decb56ac89
Update Go to 1.16.7
go1.16.7 (released 2021-08-05) includes a security fix to the net/http/httputil
package, as well as bug fixes to the compiler, the linker, the runtime, the go
command, and the net/http package. See the Go 1.16.7 milestone on the issue
tracker for details:

https://github.com/golang/go/issues?q=milestone%3AGo1.16.7+label%3ACherryPickApproved

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit b1f7ffea9f)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-08-07 18:16:20 +02:00
Sebastiaan van Stijn
75249d88bc
Merge pull request #42695 from thaJeztah/20.10_update_containerd_1.4.9
[20.10] update containerd binary to v1.4.9
2021-07-30 03:30:57 +02:00
Brian Goff
af8e58faef
Merge pull request #42659 from AkihiroSuda/runc-v1.0.1-2010
[20.10 backport] update runc binary to v1.0.1
2021-07-29 10:48:18 -07:00
Sebastiaan van Stijn
e8fb8f7acd
[20.10] update containerd binary to v1.4.9
Welcome to the v1.4.9 release of containerd!

The ninth patch release for containerd 1.4 updates runc to 1.0.1 and contains
other minor updates.

Notable Updates

- Update runc binary to 1.0.1
- Update pull authorization logic on redirect
- Fix user agent used for fetching registry authentication tokens

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-29 19:43:43 +02:00
Akihiro Suda
4cfeb27f78
update runc binary to v1.0.1
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit f50c7644cf)
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-07-20 13:51:38 +09:00
Brian Goff
013d6655bb
Merge pull request #42657 from thaJeztah/20.10_containerd_1.4.8 2021-07-19 16:46:25 -07:00
Sebastiaan van Stijn
067918a8c3
[20.10] update containerd binary v1.4.8
Update to containerd 1.4.8 to address [CVE-2021-32760][1].

[1]: https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-32760

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-19 21:21:53 +02:00
Akihiro Suda
e7bf9923d4
Merge pull request #42643 from thaJeztah/20.10_backport_bump_go116 2021-07-18 18:50:31 +09:00
Sebastiaan van Stijn
b0da207af4
Bump go 1.16.6 (addresses CVE-2021-34558)
This addresses CVE-2021-34558: https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-34558

go1.16.6 (released 2021-07-12) includes a security fix to the crypto/tls package,
as well as bug fixes to the compiler, and the net and net/http packages. See the
Go 1.16.6 milestone on the issue tracker for details:

https://github.com/golang/go/issues?q=milestone%3AGo1.16.6+label%3ACherryPickApproved

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit fe6f1a4067)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-16 19:45:20 +02:00
Sebastiaan van Stijn
abe8c4e80d
updated vendored archive/tar to go1.16.5
result of: `hack/vendor.sh archive/tar`

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 3ed804aeca)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-16 19:45:18 +02:00
Sebastiaan van Stijn
7c6645b32b
update archive/tar patch for go 1.16
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit f400e84a43)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-16 19:45:14 +02:00
Sebastiaan van Stijn
55c363ef48
Bump go 1.16.5
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit ae5ddd257c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-16 19:45:10 +02:00
Tianon Gravi
0fbb0f869a
Merge pull request #42642 from thaJeztah/20.10_backport_test_changes_for_go116
[20.10 backport] various test-changes for Go 1.16
2021-07-16 17:32:35 +00:00
Sebastiaan van Stijn
8b0913935c
integration: ensurePlugin: disable go modules when building plugin
=== RUN   TestServicePlugin
        plugin_test.go:42: assertion failed: error is not nil: error building basic plugin bin: no required module provides package github.com/docker/docker/testutil/fixtures/plugin/basic: go.mod file not found in current directory or any parent directory; see 'go help modules'
            : exit status 1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 7070df3a3e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-15 22:25:29 +02:00
Sebastiaan van Stijn
09a7efb1f7
hack/ci/windows.ps1: disable go modules
INFO: Running integration tests at 05/17/2021 12:54:50...
    INFO: DOCKER_HOST at tcp://127.0.0.1:2357
    INFO: Integration API tests being run from the host:
    INFO: make.ps1 starting at 05/17/2021 12:54:50
    powershell.exe : go: cannot find main module, but found vendor.conf in D:\gopath\src\github.com\docker\docker
    At D:\gopath\src\github.com\docker\docker@tmp\durable-1ed00396\powershellWrapper.ps1:3 char:1
    + & powershell -NoProfile -NonInteractive -ExecutionPolicy Bypass -Comm ...
    + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        + CategoryInfo          : NotSpecified: (go: cannot find...m\docker\docker:String) [], RemoteException
        + FullyQualifiedErrorId : NativeCommandError

    	to create a module there, run:
    	go mod init
    INFO: make.ps1 ended at 05/17/2021 12:54:51

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 8bae2278ba)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-15 22:25:18 +02:00
Sebastiaan van Stijn
6793ff26d8
pkg/fileutils: TestMatches: remove cases no longer valid for go1.16
These tests were no longer valid on Go 1.16; related to https://tip.golang.org/doc/go1.16#path/filepath

> The Match and Glob functions now return an error if the unmatched part of
> the pattern has a syntax error. Previously, the functions returned early on
> a failed match, and thus did not report any later syntax error in the pattern.

Causing the test to fail:

    === RUN   TestMatches
        fileutils_test.go:388: assertion failed: error is not nil: syntax error in pattern: pattern="a\\" text="a"
    --- FAIL: TestMatches (0.00s)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 2842639e0e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-15 22:24:50 +02:00
Sebastiaan van Stijn
ab9a92f79c
Update test certificates
Updates the certificates to account for current versions of Go expecting
SANs to be used instead of the Common Name field:

    FAIL: s390x.integration.plugin.authz TestAuthZPluginTLS (0.53s)
    [2020-07-26T09:36:58.638Z]     authz_plugin_test.go:132: assertion failed:
        error is not nil: error during connect: Get "https://localhost:4271/v1.41/version":
        x509: certificate relies on legacy Common Name field, use SANs or temporarily enable Common Name matching with GODEBUG=x509ignoreCN=0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit fe54215fb3)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-15 22:24:21 +02:00
Sebastiaan van Stijn
1d4a06e610
hack: add script to regenerate certificates
Certificates were originally added in c000cb6471,
but did not include a script to generate them. Current versions of Go expect
certificates to use SAN instead of Common Name fields, so updating the script
to include those;

    x509: certificate relies on legacy Common Name field, use SANs or temporarily
    enable Common Name matching with GODEBUG=x509ignoreCN=0

Some fields were updated to be a bit more descriptive (instead of "replaceme"),
and the `-text` option was used to include a human-readable variant of the
content.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 2fea30f146)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-15 22:23:47 +02:00
moby
feaca9816a
hack/vendor: add check for vendored archive/tar
Also allow re-vendoring using `./hack/vendor.sh archive/tar`

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 31b2c3bbd9)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-15 22:23:01 +02:00
Sebastiaan van Stijn
cc5a381cbc
Merge pull request #42633 from thaJeztah/20.10_backport_warn_on_non_matching_platform
[20.10 backport] docker pull: warn when pulled single-arch image does not match --platform
2021-07-15 20:46:08 +02:00
Sebastiaan van Stijn
8b224ca06c
Merge pull request #42462 from AkihiroSuda/cherrypick-rootless-selinux-42334
[20.10 backport] rootless:  avoid /run/xtables.lock EACCES on SELinux hosts  ; disable overlay2 if running with SELinux ; fix "x509: certificate signed by unknown authority" on openSUSE Tumbleweed
2021-07-15 20:45:36 +02:00
Sebastiaan van Stijn
a5044765b9
Merge pull request #42479 from AkihiroSuda/cherrypick-42071
[20.10 backport] Fix setting swaplimit=true without checking
2021-07-15 20:44:23 +02:00
Sebastiaan van Stijn
8a2af96969
Merge pull request #42591 from thaJeztah/20.10_backport_update_s390x_ubuntu_2004
[20.10 backport] Run s390x tests on Ubuntu 20.04
2021-07-15 20:42:09 +02:00
Sebastiaan van Stijn
c37c7b5c95
Merge pull request #42613 from thaJeztah/20.10_update_hcsshim
[20.10] vendor github.com/Microsoft/hcsshim 64a2b71405dacf76c95600f4c756a991ad09cf7c (moby branch)
2021-07-15 20:39:53 +02:00
Brian Goff
883cc3682d
Merge pull request #42602 from thaJeztah/20.10_backport_swagger_404
[20.10 backport] API: fix 404 status description on container create
2021-07-15 11:34:29 -07:00
Brian Goff
a1a73d1477
Merge pull request #42568 from thaJeztah/20.10_backport_runc_v1.0.0
[20.10 backport] update runc binary to v1.0.0 GA
2021-07-15 11:32:20 -07:00
Brian Goff
555bf586c7
Merge pull request #42637 from thaJeztah/20.10_update_containerd
[20.10] update containerd binary to v1.4.7
2021-07-15 11:30:13 -07:00
Sebastiaan van Stijn
4b407e4140
Merge pull request #42595 from thaJeztah/20.10_backport_update_swagger_fork
[20.10 backport] Dockerfile: update go-swagger to fix validation on Go1.16
2021-07-14 10:11:59 +02:00
Sebastiaan van Stijn
793340a33a
[20.10] update containerd binary to v1.4.7
full diff: https://github.com/containerd/containerd/compare/v1.4.6...v1.4.7

Welcome to the v1.4.7 release of containerd!

The seventh patch release for containerd 1.4 updates runc to 1.0.0 and contains
various other fixes.

Notable Updates

- Update runc binary to 1.0.0
- Fix invalid validation error checking
- Fix error on image pull resume
- Fix symlink resolution for disk mounts on Windows

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-13 23:41:22 +02:00
Sebastiaan van Stijn
7429792eed
docker pull: warn when pulled single-arch image does not match --platform
This takes the same approach as was implemented on `docker build`, where a warning
is printed if `FROM --platform=...` is used (added in 399695305c)

Before:

    docker rmi armhf/busybox
    docker pull --platform=linux/s390x armhf/busybox

    Using default tag: latest
    latest: Pulling from armhf/busybox
    d34a655120f5: Pull complete
    Digest: sha256:8e51389cdda2158935f2b231cd158790c33ae13288c3106909324b061d24d6d1
    Status: Downloaded newer image for armhf/busybox:latest
    docker.io/armhf/busybox:latest

With this change:

    docker rmi armhf/busybox
    docker pull --platform=linux/s390x armhf/busybox

    Using default tag: latest
    latest: Pulling from armhf/busybox
    d34a655120f5: Pull complete
    Digest: sha256:8e51389cdda2158935f2b231cd158790c33ae13288c3106909324b061d24d6d1
    Status: Downloaded newer image for armhf/busybox:latest
    WARNING: image with reference armhf/busybox was found but does not match the specified platform: wanted linux/s390x, actual: linux/arm64
    docker.io/armhf/busybox:latest

And daemon logs print:

   WARN[2021-04-26T11:19:37.153572667Z] ignoring platform mismatch on single-arch image  error="image with reference armhf/busybox was found but does not match the specified platform: wanted linux/s390x, actual: linux/arm64" image=armhf/busybox

When pulling without specifying `--platform, no warning is currently printed (but we can add a warning in future);

    docker rmi armhf/busybox
    docker pull armhf/busybox

    Using default tag: latest
    latest: Pulling from armhf/busybox
    d34a655120f5: Pull complete
    Digest: sha256:8e51389cdda2158935f2b231cd158790c33ae13288c3106909324b061d24d6d1
    Status: Downloaded newer image for armhf/busybox:latest
    docker.io/armhf/busybox:latest

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 424c0eb3c0)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-13 17:06:58 +02:00
Sebastiaan van Stijn
72b66d56a5
[20.10] vendor github.com/Microsoft/hcsshim 64a2b71405dacf76c95600f4c756a991ad09cf7c (moby branch)
Brings in microsoft/hcsshim#1065, which fixes #42610.

full diff: 89a9a3b524...64a2b71405

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-09 20:06:40 +02:00
Sebastiaan van Stijn
50c392c9ff
API: fix 404 status description on container create
This updates the current swagger file, and all docs versions
with the same fix as ff1d9a3ec5

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 68b095d4df)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-07 11:50:43 +02:00
Matt Morrison
025e3a7898
Update v1.41.yaml
fix containers/create 404 response description

Signed-off-by: Matt Morrison <3241034+Emdot@users.noreply.github.com>
(cherry picked from commit ff1d9a3ec5)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-07 11:50:26 +02:00
Akihiro Suda
b9cf7b7db5
rootless: fix "x509: certificate signed by unknown authority" on openSUSE Tumbleweed
openSUSE Tumbleweed was facing "x509: certificate signed by unknown authority" error,
as `/etc/ssl/ca-bundle.pem` is provided as a symlink to `../../var/lib/ca-certificates/ca-bundle.pem`,
which was not supported by `rootlesskit --copy-up=/etc` .

See rootless-containers/rootlesskit issues 225

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit 8610d8ce4c)
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-07-06 18:57:43 +09:00
Akihiro Suda
869b50e10b
rootless: disable overlay2 if running with SELinux
Kernel 5.11 introduced support for rootless overlayfs, but incompatible with SELinux.

On the other hand, fuse-overlayfs is compatible.

Close issue 42333

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit 4300a52606)
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-07-06 18:57:39 +09:00
Akihiro Suda
44f95c7126
dockerd-rootless.sh: avoid /run/xtables.lock EACCES on SELinux hosts
Previously, running dockerd-rootless.sh on SELinux-enabled hosts
was failing with "can't open lock file /run/xtables.lock: Permission denied" error.
(issue 41230).

This commit avoids hitting the error by relabeling /run in the RootlessKit child.
The actual /run on the parent is unaffected.

e6fc34b71a/libpod/networking_linux.go (L396-L401)

Tested on Fedora 34

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit cdaf82ba3f)
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-07-06 18:57:38 +09:00
Sebastiaan van Stijn
78bb0f445a
Dockerfile: update go-swagger to fix validation on Go1.16
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 42d2048b9d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-03 17:23:07 +02:00
Stefan Scherer
618f6a79ab
Run s390x tests on Ubuntu 20.04
Signed-off-by: Stefan Scherer <stefan.scherer@docker.com>
(cherry picked from commit 7a6cac2b23)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-02 14:41:13 +02:00
Akihiro Suda
872cb16edb
update runc binary to v1.0.0 GA
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit 64badfc018)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-06-24 22:19:37 +02:00
Sebastiaan van Stijn
46a7ebc540
Merge pull request #42561 from ameyag/20.10-logbroker
[20.10] vendor: swarmkit to fix deadlock in log broker (bump_20.10)
2021-06-24 15:23:30 +02:00
Ameya Gawde
4d42e18c05
vendor: swarmkit to fix deadlock in log broker
Signed-off-by: Ameya Gawde <agawde@mirantis.com>
2021-06-23 16:45:51 -07:00
Akihiro Suda
87e28a6171
Merge pull request #42538 from ameyag/20.10-overlap-ip
[20.10 backport] Fix possible overlapping IPs
2021-06-19 18:05:46 +09:00
Drew Erny
89edb68e89
Fix possible overlapping IPs
A node is no longer using its load balancer IP address when it no longer
has tasks that use the network that requires that load balancer. When
this occurs, the swarmkit manager will free that IP in IPAM, and may
reaassign it.

When a task shuts down cleanly, it attempts removal of the networks it
uses, and if it is the last task using those networks, this removal
succeeds, and the load balancer IP is freed.

However, this behavior is absent if the container fails. Removal of the
networks is never attempted.

To address this issue, I amend the executor. Whenever a node load
balancer IP is removed or changed, that information is passedd to the
executor by way of the Configure method. By keeping track of the set of
node NetworkAttachments from the previous call to Configure, we can
determine which, if any, have been removed or changed.

At first, this seems to create a race, by which a task can be attempting
to start and the network is removed right out from under it. However,
this is already addressed in the controller. The controller will attempt
to recreate missing networks before starting a task.

Signed-off-by: Drew Erny <derny@mirantis.com>
(cherry picked from commit 0d9b0ed678)
Signed-off-by: Ameya Gawde <agawde@mirantis.com>
2021-06-18 10:13:59 -07:00
Tianon Gravi
4d29d58a65
Merge pull request #42507 from thaJeztah/20.10_backport_disable_power_z
[20.10 backport] Jenkinsfile: skip ppc64le and s390x by default on pull requests
2021-06-10 12:06:52 -07:00
Sebastiaan van Stijn
523f8b397c
Jenkinsfile: skip ppc64le and s390x by default on pull requests
This changes CI to skip these platforms by default. The ppc64le and s390x
machines are "pet machines", configuration may be outdated, and these
machines are known to be flaky.

Building and verifying packages for these platforms is being handed
over to the IBM team.

We can still run these platforms for specific pull requests by selecting
the checkboxes.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 82c7e906ea)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-06-10 14:05:07 +02:00
Jakub Guzik
a57fc0eb15
Fix setting swaplimit=true without checking
Signed-off-by: Jakub Guzik <jakubmguzik@gmail.com>
(cherry picked from commit 7ef6ece774)
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-06-07 17:27:35 +09:00
Brian Goff
b0f5bc36fe
Merge pull request #42352 from AkihiroSuda/cherrypick-41724
[20.10 backport] Use v2 capabilities in layer archives
2021-06-01 15:34:42 -07:00
Brian Goff
497c50a575
Merge pull request #42448 from thaJeztah/20.10_backport_update_buildkit
[20.10 backport] vendor: github.com/moby/buildkit v0.8.3-3-g244e8cde
2021-06-01 15:13:27 -07:00
Sebastiaan van Stijn
6474dada20
vendor: github.com/moby/buildkit v0.8.3-3-g244e8cde
full diff: https://github.com/moby/buildkit/compare/v0.8.3...v0.8.3-3-g244e8cde

- Transform relative mountpoints for exec mounts in the executor
- Add test for handling relative mountpoints

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 61b04b3a02)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-05-31 16:08:36 +02:00
Sebastiaan van Stijn
895eaacdd4
vendor: github.com/moby/buildkit v0.8.3
full diff: https://github.com/moby/buildkit/compare/v0.8.2...v0.8.3

- vendor containerd (required for rootless overlayfs on kernel 5.11)
    - not included to avoid depending on a fork
- Add retry on image push 5xx errors
- contenthash: include basename in content checksum for wildcards
- Fix missing mounts in execOp cache map
- Add regression test for run cache not considering mounts
- Add hack to preserve Dockerfile RUN cache compatibility after mount cache bugfix

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 79ee285d76)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-05-31 16:08:33 +02:00
Sebastiaan van Stijn
eab37e5676
Merge pull request #42413 from thaJeztah/20.10_backport_bump_libnetwork
[20.10 backport] vendor: github.com/docker/libnetwork 64b7a4574d1426139437d20e81c0b6d391130ec8
2021-05-27 10:44:33 +02:00
Akihiro Suda
d844987b77
Merge pull request #42421 from thaJeztah/20.10_backport_more_ignore 2021-05-27 02:46:50 +09:00
Sebastiaan van Stijn
003e3c0551
pkg/signal: ignore SIGURG on all platforms
Other Unix platforms (e.g. Darwin) are also affected by the Go
runtime sending SIGURG.

This patch changes how we match the signal by just looking for the
"URG" name, which should handle any platform that has this signal
defined in the SignalMap.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 05f520dd3c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-05-25 21:42:21 +02:00
Sebastiaan van Stijn
95551168ac
vendor: github.com/ishidawataru/sctp f2269e66cdee387bd321445d5d300893449805be
full diff: 6e2cb13661...f2269e66cd

- support SO_SNDBUF/SO_RCVBUF handling
- Support Go Modules
- license clarificaton
- ci: drop 1.6, 1.7, 1.8 support
- Add support for SocketConfig
- support goarch mips64le architecture.
- fix possible socket leak when bind fails

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 22b9e2a7e5)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-05-25 11:39:46 +02:00
Sebastiaan van Stijn
d29a55c6c3
vendor: github.com/docker/libnetwork 64b7a4574d1426139437d20e81c0b6d391130ec8
Update libnetwork to make `docker run -p 80:80` functional again on environments
with kernel boot parameter `ipv6.disable=1`.

full diff: b3507428be...64b7a4574d

- fix port forwarding with ipv6.disable=1
    - fixes moby/moby/42288 Docker 20.10.6: all containers stopped and cannot start if ipv6 is disabled on host
    - fixes docker/libnetwork/2629 Network issue with IPv6 following update to version 20.10.6
    - fixesdocker/for-linux/1233 Since 20.10.6 it's not possible to run docker on a machine with disabled IPv6 interfaces
- vendor: github.com/ishidawataru/sctp f2269e66cdee387bd321445d5d300893449805be
- Enforce order of lock acquisitions on network/controller, fixes #2632
    - fixes docker/libnetwork/2632 Name resolution stuck due to deadlock between different network struct methods
    - fixes moby/moby/42032 Docker deamon get's stuck, can't serve DNS requests

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit e4109b3b6b)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-05-25 11:39:44 +02:00
Brian Goff
3db8385b18
Merge pull request #42382 from thaJeztah/20.10_backport_schema1_cache_fix
[20.10 backport] builder-next: relax second cache key requirements for schema1
2021-05-20 12:08:53 -07:00
Brian Goff
df371af560
Merge pull request #42401 from thaJeztah/20.10_catch_almost_all
[20.10 backport] pkg/signal.CatchAll: ignore SIGURG on Linux
2021-05-20 11:08:47 -07:00
Sebastiaan van Stijn
c1ebb81514
Merge pull request #42398 from thaJeztah/20.10_update_containerd_1.4.6
[20.10] update containerd binary to v1.4.6
2021-05-20 12:33:05 +02:00
Sebastiaan van Stijn
41cf01fa93
pkg/signal.CatchAll: ignore SIGURG on Linux
Do not handle SIGURG on Linux, as in go1.14+, the go runtime issues
SIGURG as an interrupt to support preemptable system calls on Linux.

This issue was caught in TestCatchAll, which could fail when updating to Go 1.14 or above;

    === Failed
    === FAIL: pkg/signal TestCatchAll (0.01s)
        signal_linux_test.go:32: assertion failed: urgent I/O condition (string) != continued (string)
        signal_linux_test.go:32: assertion failed: continued (string) != hangup (string)
        signal_linux_test.go:32: assertion failed: hangup (string) != child exited (string)
        signal_linux_test.go:32: assertion failed: child exited (string) != illegal instruction (string)
        signal_linux_test.go:32: assertion failed: illegal instruction (string) != floating point exception (string)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit b7ebf32ba3)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-05-20 10:11:39 +02:00
Sebastiaan van Stijn
a23a880c6e
Merge pull request #42395 from thaJeztah/20.10_backport_runc_rc95
[20.10 backport] update runc binary to v1.0.0-rc95
2021-05-19 20:49:52 +02:00
Sebastiaan van Stijn
56541eca9a
[20.10] update containerd binary to v1.4.6
full diff: https://github.com/containerd/containerd/compare/v1.4.5...v1.4.6

The sixth patch release for containerd 1.4 is a security release to update
runc for CVE-2021-30465

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-05-19 20:43:04 +02:00
Sebastiaan van Stijn
fb179ff098
update runc binary to v1.0.0-rc95
full diff: https://github.com/opencontainers/runc/compare/v1.0.0-rc94...v1.0.0-rc95

Release notes:

This release of runc contains a fix for CVE-2021-30465, and users are
strongly recommended to update (especially if you are providing
semi-limited access to spawn containers to untrusted users).

Aside from this security fix, only a few other changes were made since
v1.0.0-rc94 (the only user-visible change was the addition of support
for defaultErrnoRet in seccomp profiles).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit efec2bb368)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-05-19 13:20:30 +02:00
Akihiro Suda
989c08c367
Merge pull request #42388 from thaJeztah/20.10_backport_update_runc
[20.10 backport] Update runc binary to v1.0.0-rc94
2021-05-19 12:26:26 +09:00
Akihiro Suda
4c801fdb7d
integration: remove KernelMemory tests
Starting with runc v1.0.0-rc94, runc no longer supports KernelMemory.

52390d6804

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit 2f0d6664a1)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-05-18 09:25:38 +02:00
Jintao Zhang
6174e3cf22
Update runc binary to v1.0.0-rc94
Signed-off-by: Jintao Zhang <zhangjintao9020@gmail.com>
(cherry picked from commit 8c019e830a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-05-18 09:25:36 +02:00
Akihiro Suda
afbb1277a3
Swarm config: use absolute paths for mount destination strings
Needed for runc >= 1.0.0-rc94.

See runc issue 2928.

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 9303376242)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-05-18 09:25:34 +02:00
Tonis Tiigi
94c1890d39
builder-next: relax second cache key requirements for schema1
Schema1 images can not have a config based cache key
before the layers are pulled. Avoid validation and reuse
manifest digest as a second key.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit 85167fc634)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-05-17 14:58:16 +02:00
Akihiro Suda
e4b9915803
Merge pull request #42372 from thaJeztah/20.10_containerd_1.4.5
[20.10] update containerd binary to v1.4.5
2021-05-13 12:51:54 +09:00
Sebastiaan van Stijn
01f734cb4f
[20.10] update containerd binary to v1.4.5
release notes: https://github.com/containerd/containerd/releases/tag/v1.4.5

- Update runc to rc94
- Fix leaking socket path in runc shim v2
- Fix cleanup logic in new container in runc shim v2
- Fix registry mirror authorization logic in CRI plugin
- Add support for userxattr in overlay snapshotter for kernel 5.11+

(Note that the update to runc is done separately)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-05-12 10:46:44 +02:00
Sebastiaan van Stijn
e3cb5adc0d
Merge pull request #42292 from cpuguy83/20.10_hcsshim_no_error_details
[20.10] Bump hcsshim for error details fix
2021-05-06 16:35:18 +02:00
Eric Mountain
2a0c446866
Use v2 capabilities in layer archives
When building images in a user-namespaced container, v3 capabilities are
stored including the root UID of the creator of the user-namespace.

This UID does not make sense outside the build environment however. If
the image is run in a non-user-namespaced runtime, or if a user-namespaced
runtime uses a different UID, the capabilities requested by the effective
bit will not be honoured by `execve(2)` due to this mismatch.

Instead, we convert v3 capabilities to v2, dropping the root UID on the
fly.

Signed-off-by: Eric Mountain <eric.mountain@datadoghq.com>
(cherry picked from commit 95eb490780)
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-05-05 21:46:31 +09:00
Brian Goff
614a70a11b
Merge pull request #42294 from AkihiroSuda/rootlesskit-0.14.2-2010
[20.10 backport] bump up rootlesskit to v0.14.2 (Fix `Timed out proxy starting the userland proxy.` error with `DOCKERD_ROOTLESS_ROOTLESSKIT_PORT_DRIVER=slirp4netns`)
2021-04-30 09:18:11 -07:00
Tianon Gravi
114310d76a
Merge pull request #42342 from AkihiroSuda/dind-fix-cgroup2-evac-2010
[20.10 backport] hack/dind: fix cgroup v2 evacuation with `docker run --init`
2021-04-30 08:10:12 -07:00
Akihiro Suda
21391bb7f7
hack/dind: fix cgroup v2 evacuation with docker run --init
Evacuate all the processes in `/sys/fs/cgroup/cgroup.procs`, not just PID 1.

Before:
```console
$ docker run --rm --privileged --init $(docker build -q .) cat /sys/fs/cgroup/cgroup.subtree_control
sed: couldn't flush stdout: Device or resource busy
```

After:
```console
$ docker run --rm --privileged --init $(docker build -q .) cat /sys/fs/cgroup/cgroup.subtree_control
cpuset cpu io memory hugetlb pids rdma
```

Fix docker-library/docker issue 308

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit 42b1175eda)
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-04-30 19:08:07 +09:00
Brian Goff
12b03bcb27 Error string match: do not match command path
Whether or not the command path is in the error message is a an
implementation detail.
For example, on Windows the only reason this ever matched was because it
dumped the entire container config into the error message, but this had
nothing to do with the actual error.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit 225e046d9d)
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2021-04-27 18:46:33 +00:00
Tianon Gravi
ce82693823
Merge pull request #42324 from AkihiroSuda/whichless-2010
[20.10 backport] dockerd-rootless.sh: use `command -v` instead of `which`
2021-04-26 12:47:40 -07:00
Akihiro Suda
8a7f77cb2f
dockerd-rootless.sh: use command -v instead of which
`which` binary is often missing

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit e928692c69)
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-04-26 16:04:08 +09:00
Tianon Gravi
4af54f15ed
Merge pull request #42297 from thaJeztah/20.10_backport_update_yamllint
[20.10 backport] Dockerfile: update yamllint to v1.26.1 to fix build
2021-04-23 11:47:44 -07:00
Akihiro Suda
9ca66776fa
bump up rootlesskit to v0.14.2
Fix `Timed out proxy starting the userland proxy.` error with `DOCKERD_ROOTLESS_ROOTLESSKIT_PORT_DRIVER=slirp4netns`.
(https://github.com/rootless-containers/rootlesskit/issues/250)

Full changes: https://github.com/rootless-containers/rootlesskit/compare/v0.14.1...v0.14.2

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit 11bddf330d4fec818e17333c360c25e8641f221d)
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-04-21 18:11:00 +09:00
Sebastiaan van Stijn
08b27e45d8
Dockerfile: update yamllint to v1.26.1 to fix build
Installation of yamllint started failing, on non-amd64 builds, which could
be if the version we were using wasn't specific enough about a dependency
to install.

    copying Cython/Utility/CppSupport.cpp -> build/lib.linux-aarch64-3.7/Cython/Utility
    running build_ext
    building 'Cython.Plex.Scanners' extension
    creating build/temp.linux-aarch64-3.7
    creating build/temp.linux-aarch64-3.7/tmp
    creating build/temp.linux-aarch64-3.7/tmp/pip-install-jasgbmp7
    creating build/temp.linux-aarch64-3.7/tmp/pip-install-jasgbmp7/Cython
    creating build/temp.linux-aarch64-3.7/tmp/pip-install-jasgbmp7/Cython/Cython
    creating build/temp.linux-aarch64-3.7/tmp/pip-install-jasgbmp7/Cython/Cython/Plex
    aarch64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/usr/include/python3.7m -c /tmp/pip-install-jasgbmp7/Cython/Cython/Plex/Scanners.c -o build/temp.linux-aarch64-3.7/tmp/pip-install-jasgbmp7/Cython/Cython/Plex/Scanners.o
         /tmp/pip-install-jasgbmp7/Cython/Cython/Plex/Scanners.c:21:10: fatal error: Python.h: No such file or directory
          #include "Python.h"
                   ^~~~~~~~~~
         compilation terminated.
         error: command 'aarch64-linux-gnu-gcc' failed with exit status 1

         ----------------------------------------
     Command "/usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-jasgbmp7/Cython/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-if5qclwe/install-record.txt --single-version-externally-managed --prefix /tmp/pip-build-env-_dtiuyfw --compile" failed with error code 1 in /tmp/pip-install-jasgbmp7/Cython/

      ----------------------------------------
    Command "/usr/bin/python3 -m pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-_dtiuyfw --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- setuptools wheel Cython" failed with error code 1 in None
    #22 ERROR: executor failed running [/bin/sh -c pip3 install yamllint==1.16.0]: exit code: 1

Trying if updating to the latest version fixes this.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit c35cefb489)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-04-15 22:27:44 +02:00
Brian Goff
404ede5737 Bump hcsshim for error details fix
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2021-04-14 18:57:05 +00:00
Tibor Vass
8728dd246c
Merge pull request #42263 from AkihiroSuda/move-cgroup2-out-of-experimental-20.10
[20.10 backport] Move cgroup v2 out of experimental
2021-04-09 15:06:18 -07:00
Tibor Vass
94854bcbd6
Merge pull request #42109 from AkihiroSuda/rootless-add-context-2010
[20.10 backport] dockerd-rootless-setuptool.sh: create CLI context "rootless"
2021-04-07 16:58:09 -07:00
Tibor Vass
ffd037de36
Merge pull request #42253 from AkihiroSuda/btrfs-allow-unprivileged-20.10
[20.10 backport] btrfs: Allow unprivileged user to delete subvolumes (kernel >= 4.18)
2021-04-07 16:55:56 -07:00
Sebastiaan van Stijn
419b3706ea
Merge pull request #42256 from cpuguy83/20.10_plugin_layer_mediatype
[20.10 backport] Use docker media type for plugin layers
2021-04-07 21:26:18 +02:00
Tibor Vass
76b0df9b6e
Merge pull request #42257 from thaJeztah/20.10_backport_fix_testinspect
[20.10 backport] Fix TestInspect(), and pin arm64 machines to a specific Ubuntu version
2021-04-07 01:23:47 -07:00
Akihiro Suda
255c79a1e8
Move cgroup v2 out of experimental
We have upgraded runc to rc93 and added CI for cgroup 2.
So we can move cgroup v2 out of experimental.

Fix issue 41916

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit 1d2a660093)
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-04-07 13:55:48 +09:00
Stefan Scherer
f2c0b3688a
Pin arm64 machines to a specific Ubuntu version
Signed-off-by: Stefan Scherer <stefan.scherer@docker.com>
(cherry picked from commit b7c3548c82)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-04-06 18:55:24 +02:00
Sebastiaan van Stijn
29ff2af2d3
Fix flaky TestInspect
This test has been flaky for a long time, failing with:

    --- FAIL: TestInspect (12.04s)
        inspect_test.go:39: timeout hit after 10s: waiting for tasks to enter run state. task failed with error: task: non-zero exit (1)

While looking through logs, noticed tasks were started, entering RUNNING stage,
and then exited, to be started again.

    state.transition="STARTING->RUNNING"
    ...
    msg="fatal task error" error="task: non-zero exit (1)"
    ...
    state.transition="RUNNING->FAILED"

Looking for possible reasons, first considering network issues (possibly we ran
out of IP addresses or networking not cleaned up), then I spotted the issue.

The service is started with;

    Command:         []string{"/bin/top"},
    Args:            []string{"-u", "root"},

The `-u root` is not an argument for the service, but for `/bin/top`. While the
Ubuntu/Debian/GNU version `top` has a -u/-U option;

    docker run --rm ubuntu:20.04 top -h 2>&1 | grep '\-u'
      top -hv | -bcEHiOSs1 -d secs -n max -u|U user -p pid(s) -o field -w [cols]

The *busybox* version of top does not:

    docker run --rm busybox top --help 2>&1 | grep '\-u'

So running `top -u root` would cause the task to fail;

    docker run --rm busybox top -u root
    top: invalid option -- u
    ...

    echo $?
    1

As a result, the service went into a crash-loop, and because the `poll.WaitOn()`
was running with a short interval, in many cases would _just_ find the RUNNING
state, perform the `service inspect`, and pass, but in other cases, it would not
be that lucky, and continue polling untill we reached the 10 seconds timeout,
and mark the test as failed.

Looking for history of this option (was it previously using a different image?) I
found this was added in 6cd6d8646a, but probably
just missed during review.

Given that the option is only set to have "something" to inspect, I replaced
the `-u root` with `-d 5`, which makes top refresh with a 5 second interval.

Note that there is another test (`TestServiceListWithStatuses) that uses the same
spec, however, that test is skipped based on API version of the test-daemon, and
(to be looked into), when performing that check, no API version is known, causing
the test to (always?) be skipped:

    === RUN   TestServiceListWithStatuses
        --- SKIP: TestServiceListWithStatuses (0.00s)
            list_test.go:34: versions.LessThan(testEnv.DaemonInfo.ServerVersion, "1.41")

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 00cb3073f4)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-04-06 18:55:06 +02:00
Brian Goff
60310e2409 Use docker media type for plugin layers
This was changed as part of a refactor to use containerd dist code. The
problem is the OCI media types are not compatible with older versions of
Docker.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit a876ede24f)
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2021-04-06 16:52:54 +00:00
Akihiro Suda
8088859bab
btrfs: Allow unprivileged user to delete subvolumes (kernel >= 4.18)
Fix issue 41762

Cherry-pick "drivers: btrfs: Allow unprivileged user to delete subvolumes" from containers/storage
831e32b6bd

> In btrfs, subvolume can be deleted by IOC_SNAP_DESTROY ioctl but there
> is one catch: unprivileged IOC_SNAP_DESTROY call is restricted by default.
>
> This is because IOC_SNAP_DESTROY only performs permission checks on
> the top directory(subvolume) and unprivileged user might delete dirs/files
> which cannot be deleted otherwise. This restriction can be relaxed if
> user_subvol_rm_allowed mount option is used.
>
> Although the above ioctl had been the only way to delete a subvolume,
> btrfs now allows deletion of subvolume just like regular directory
> (i.e. rmdir sycall) since kernel 4.18.
>
> So if we fail to cleanup subvolume in subvolDelete(), just fallback to
> system.EnsureRmoveall() to try to cleanup subvolumes again.
> (Note: quota needs privilege, so if quota is enabled we do not fallback)
>
> This fix will allow non-privileged container works with btrfs backend.

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit 62b5194f62)
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-04-06 14:45:01 +09:00
Tibor Vass
88bd96d6e5
Merge pull request #42233 from AkihiroSuda/fix-rootless-bind-EPERM-20.10
[20.10 backport] rootless: bind mount: fix "operation not permitted"
2021-04-01 07:41:54 -07:00
Tibor Vass
b6f4e8ba12
Merge pull request #42235 from AkihiroSuda/fix-overlay2-nativediff-2010
[20.10 backport] rootless: overlay2: fix "createDirWithOverlayOpaque(...) ... input/output error"
2021-04-01 06:23:42 -07:00
Tibor Vass
d6ca8a8e16
Merge pull request #42236 from thaJeztah/20.10_backport_specconv_fix_trimspace
[20.10 backport] rootless: fix getCurrentOOMScoreAdj
2021-04-01 05:31:04 -07:00
Tibor Vass
3499626899
Merge pull request #42232 from AkihiroSuda/rootlesskit-0.14.1-20.10
[20.10 backport] bump up rootlesskit to v0.14.1 (Fix `DOCKERD_ROOTLESS_ROOTLESSKIT_PORT_DRIVER=slirp4netns` regression)
2021-04-01 04:08:47 -07:00
Akihiro Suda
d22dde8eb1
rootless: fix getCurrentOOMScoreAdj
`getCurrentOOMScoreAdj()` was broken because `strconv.Atoi()` was called
without trimming "\n".

Fix issue 40068: `rootless docker in kubernetes: "getting the final child's pid from
pipe caused \"EOF\": unknown"

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit d6ddfb6118)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-04-01 12:11:27 +02:00
Akihiro Suda
c1e7924f7c
archive: do not use overlayWhiteoutConverter for UserNS
overlay2 no longer sets `archive.OverlayWhiteoutFormat` when
running in UserNS, so we can remove the complicated logic in the
archive package.

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit 6322dfc217)
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-04-01 19:00:42 +09:00
Akihiro Suda
22dc1597b9
overlay2: doesSupportNativeDiff: add fast path for userns
When running in userns, returns error (i.e. "use naive, not native")
immediately.

No substantial change to the logic.

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit 67aa418df2)
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-04-01 19:00:37 +09:00
Akihiro Suda
daae27bfce
overlay2: call d.naiveDiff.ApplyDiff when useNaiveDiff==true
Previously, `d.naiveDiff.ApplyDiff` was not used even when
`useNaiveDiff()==true`

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit dd97134232)
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-04-01 19:00:32 +09:00
Akihiro Suda
e974cb638c
rootless: bind mount: fix "operation not permitted"
The following was failing previously, because `getUnprivilegedMountFlags()` was not called:
```console
$ sudo mount -t tmpfs -o noexec none /tmp/foo
$ $ docker --context=rootless run -it --rm -v /tmp/foo:/mnt:ro alpine
docker: Error response from daemon: OCI runtime create failed: container_linux.go:367: starting container process caused: process_linux.go:520: container init caused: rootfs_linux.go:60: mounting "/tmp/foo" to rootfs at "/home/suda/.local/share/docker/overlay2/b8e7ea02f6ef51247f7f10c7fb26edbfb308d2af8a2c77915260408ed3b0a8ec/merged/mnt" caused: operation not permitted: unknown.
```

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit 248f98ef5e)
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-04-01 18:45:23 +09:00
Akihiro Suda
7022b1e12e
bump up rootlesskit to v0.14.1
Fix `DOCKERD_ROOTLESS_ROOTLESSKIT_PORT_DRIVER=slirp4netns` regression.

Full changes: https://github.com/rootless-containers/rootlesskit/compare/v0.14.0...v0.14.1

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit 45021ee354)
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-04-01 18:39:07 +09:00
Tibor Vass
cdd71c6736
Merge pull request #42205 from thaJeztah/20.10_backport_bump_libnetwork
[20.10 backport] vendor: docker/libnetwork b3507428be5b458cb0e2b4086b13531fb0706e46
2021-04-01 02:37:21 -07:00
Akihiro Suda
4ccd7382e0
Merge pull request #42141 from thaJeztah/20.10_backport_vpnkit_arm64 2021-03-26 09:15:42 +09:00
Sebastiaan van Stijn
88470052e7
vendor: docker/libnetwork b3507428be5b458cb0e2b4086b13531fb0706e46
full diff: fa125a3512...b3507428be

- fixed IPv6 iptables rules for enabled firewalld (libnetwork#2609)
    - fixes "Docker uses 'iptables' instead of 'ip6tables' for IPv6 NAT rule, crashes"
- Fix regression in docker-proxy
    - introduced in "Fix IPv6 Port Forwarding for the Bridge Driver" (libnetwork#2604)
    - fixes/addresses: "IPv4 and IPv6 addresses are not bound by default anymore" (libnetwork#2607)
    - fixes/addresses "IPv6 is no longer proxied by default anymore" (moby#41858)
- Use hostIP to decide on Portmapper version
    - fixes docker-proxy not being stopped correctly

Port mapping of containers now contain separatet mappings for IPv4 and IPv6 addresses, when
listening on "any" IP address. Various tests had to be updated to take multiple mappings into
account.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 0450728267)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-03-25 22:29:47 +01:00
Sebastiaan van Stijn
d26ed2c33b
fix assertPortList normalizing being too strict
The normalizing was updated with the output of the "docker port" command
in mind, but we're normalizing the "expected" output, which is passed
without the "->" in front of the mapping, causing some tests to fail;

    === RUN   TestDockerSuite/TestPortHostBinding
        --- FAIL: TestDockerSuite/TestPortHostBinding (1.21s)
            docker_cli_port_test.go:324: assertion failed: error is not nil: |:::9876!=[::]:9876|
    === RUN   TestDockerSuite/TestPortList
        --- FAIL: TestDockerSuite/TestPortList (0.96s)
            docker_cli_port_test.go:25: assertion failed: error is not nil: |:::9876!=[::]:9876|

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit c8599a6537)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-03-25 22:29:44 +01:00
Sebastiaan van Stijn
8926328927
Merge pull request #42197 from thaJeztah/20.10_backport_improve_build_errors
[20.10 backport] builder: produce error when using unsupported Dockerfile option
2021-03-25 21:27:00 +01:00
Sebastiaan van Stijn
eef17706da
Merge pull request #42194 from thaJeztah/20.10_backport_rootlesskit_goproxy
[20.10 backport] hack: use GOPROXY for rootlesskit to workaround issue with old git on CentOS/RHEL 7
2021-03-25 20:55:02 +01:00
Sebastiaan van Stijn
6b1da9492f
Merge pull request #42186 from thaJeztah/20.10_backport_update_rootlesskit
[20.10 backport] update rootlesskit to v0.14.0
2021-03-25 20:49:46 +01:00
Brian Goff
60aa0f2f6b
Merge pull request #42079 from thaJeztah/20.10_backport_update_docs_links
[20.10 backport] Update documentation links
2021-03-25 12:48:52 -07:00
Sebastiaan van Stijn
122ef5ce94
Merge pull request #42072 from AkihiroSuda/prohibit-rootless-as-root-2010
[20.10 backport] dockerd-rootless.sh: prohibit running as root
2021-03-25 20:46:59 +01:00
Akihiro Suda
7c84bf929b
Merge pull request #42196 from thaJeztah/20.10_backport_ci_fixes
[20.10 backport] CI: update tests to be more resilient against CLI output format and for libnetwork changes
2021-03-25 13:23:58 +09:00
Sebastiaan van Stijn
915b239519
builder: produce error when using unsupported Dockerfile option
With the promotion of the experimental Dockerfile syntax to "stable", the Dockerfile
syntax now includes some options that are supported by BuildKit, but not (yet)
supported in the classic builder.

As a result, parsing a Dockerfile may succeed, but any flag that's known to BuildKit,
but not supported by the classic builder is silently ignored;

    $ mkdir buildkit_flags && cd buildkit_flags
    $ touch foo.txt

For example, `RUN --mount`:

    DOCKER_BUILDKIT=0 docker build --no-cache -f- . <<EOF
    FROM busybox
    RUN --mount=type=cache,target=/foo echo hello
    EOF

    Sending build context to Docker daemon  2.095kB
    Step 1/2 : FROM busybox
     ---> 219ee5171f80
    Step 2/2 : RUN --mount=type=cache,target=/foo echo hello
     ---> Running in 022fdb856bc8
    hello
    Removing intermediate container 022fdb856bc8
     ---> e9f0988844d1
    Successfully built e9f0988844d1

Or `COPY --chmod` (same for `ADD --chmod`):

    DOCKER_BUILDKIT=0 docker build --no-cache -f- . <<EOF
    FROM busybox
    COPY --chmod=0777 /foo.txt /foo.txt
    EOF

    Sending build context to Docker daemon  2.095kB
    Step 1/2 : FROM busybox
     ---> 219ee5171f80
    Step 2/2 : COPY --chmod=0777 /foo.txt /foo.txt
     ---> 8b7117932a2a
    Successfully built 8b7117932a2a

Note that unknown flags still produce and error, for example, the below fails because `--hello` is an unknown flag;

    DOCKER_BUILDKIT=0 docker build -<<EOF
    FROM busybox
    RUN --hello echo hello
    EOF

    Sending build context to Docker daemon  2.048kB
    Error response from daemon: dockerfile parse error line 2: Unknown flag: hello

With this patch applied
----------------------------

With this patch applied, flags that are known in the Dockerfile spec, but are not
supported by the classic builder, produce an error, which includes a link to the
documentation how to enable BuildKit:

    DOCKER_BUILDKIT=0 docker build --no-cache -f- . <<EOF
    FROM busybox
    RUN --mount=type=cache,target=/foo echo hello
    EOF

    Sending build context to Docker daemon  2.048kB
    Step 1/2 : FROM busybox
     ---> b97242f89c8a
    Step 2/2 : RUN --mount=type=cache,target=/foo echo hello
    the --mount option requires BuildKit. Refer to https://docs.docker.com/go/buildkit/ to learn how to build images with BuildKit enabled

    DOCKER_BUILDKIT=0 docker build --no-cache -f- . <<EOF
    FROM busybox
    COPY --chmod=0777 /foo.txt /foo.txt
    EOF

    Sending build context to Docker daemon  2.095kB
    Step 1/2 : FROM busybox
     ---> b97242f89c8a
    Step 2/2 : COPY --chmod=0777 /foo.txt /foo.txt
    the --chmod option requires BuildKit. Refer to https://docs.docker.com/go/buildkit/ to learn how to build images with BuildKit enabled

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit a09c0276a2)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-03-24 23:57:54 +01:00
Sebastiaan van Stijn
ef2351b416
integration-cli: rely less on "docker port" output format
Also re-formatting some lines for readability.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit c6038b4884)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-03-24 22:04:10 +01:00
Sebastiaan van Stijn
86d98f5711
integration: update getExternalAddress to prefer IPv4
Rootlesskit doesn't currently handle IPv6 addresses, causing TestNetworkLoopbackNat
and TestNetworkNat to fail;

    Error starting userland proxy:
    error while calling PortManager.AddPort(): listen tcp: address :::8080: too many colons in address

This patch:

- Updates `getExternalAddress()` to pick IPv4 address if both IPv6 and IPv4 are found
- Update TestNetworkNat to net.JoinHostPort(), so that square brackets are used for
  IPv6 addresses (e.g. `[::]:8080`)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit f845b98ca6)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-03-24 22:04:08 +01:00
Sebastiaan van Stijn
b41e2d4dc1
integration/container: wrap some long lines for readability
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 54ca929a70)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-03-24 22:04:06 +01:00
Tibor Vass
407a61cdb2
hack: use GOPROXY for rootlesskit to workaround issue with old git on CentOS/RHEL 7
Since rootlesskit removed vendor folder, building it has to rely on go mod.

Dockerfile in docker-ce-packaging uses GOPROXY=direct, which makes "go mod"
commands use git to fetch modules. "go mod" in Go versions before 1.14.1 are
incompatible with older git versions, including the version of git that ships
with CentOS/RHEL 7 (which have git 1.8), see golang/go#38373

This patch switches rootlesskit install script to set GOPROXY to
https://proxy.golang.org so that git is not required for downloading modules.

Once all our code has upgraded to Go 1.14+, this workaround should be
removed.

Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit cbc6cefdcb)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-03-24 09:28:31 +01:00
Sebastiaan van Stijn
653b58cc8a
Merge pull request #42111 from thaJeztah/20.10_backport_dockerfile_stable
[20.10 backport] Dockerfile: switch to "stable" dockerfile front-end
2021-03-22 22:53:03 +01:00
Sebastiaan van Stijn
a35e1f451e
update rootlesskit to v0.14.0
full diff: https://github.com/rootless-containers/rootlesskit/compare/v0.13.1...v0.14.0

v0.14.0 Changes (since v0.13.2)
--------------------------------------

- CLI: improve --help output
- API: support GET /info
- Port API: support specifying IP version explicitly ("tcp4", "tcp6")
- rootlesskit-docker-proxy: support libnetwork >= 20201216 convention
- Allow vendoring with moby/sys/mountinfo@v0.1.3 as well as @v0.4.0
- Remove socat port driver
    - socat driver has been deprecated since v0.7.1 (Dec 2019)
- New experimental flag: --ipv6
    - Enables IPv6 routing (slirp4netns --enable-ipv6). Unrelated to port driver.

v0.13.2
--------------------------------------

- Fix cleaning up crashed state dir
- Update Go to 1.16
- Misc fixes

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit e166af959d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-03-22 22:19:06 +01:00
Sebastiaan van Stijn
9f2f85c53b
Merge pull request #42176 from cpuguy83/20.10_testPushMultipleTags
[20.10] TestPushMultipleTags: Add support for 20.10 CLI
2021-03-22 15:31:21 +01:00
Akihiro Suda
257cf10ee1
Merge pull request #42175 from cpuguy83/20.10_TestDockerNetworkFlagAlias 2021-03-21 01:11:32 +09:00
Sebastiaan van Stijn
5a697ae130
Merge pull request #42174 from thaJeztah/20.10_backport_41820_fix_json_unexpected_eof
[20.10 backport] Fix handling for json-file io.UnexpectedEOF
2021-03-20 10:10:24 +01:00
Akihiro Suda
cdb77eca0e
Merge pull request #42168 from AkihiroSuda/ovl-k511-2010
[20.10 backport] overlay2: support "userxattr" option (kernel 5.11)
2021-03-20 12:31:02 +09:00
Brian Goff
9780942e20 Remove cli test for duplicate --net/--network opts
This seems to be testing a strange case, specifically that one can set
the `--net` and `--network` in the same command with the same network.

Indeed this used to work with older CLIs but newer ones error out when
validating the request before sending it to the daemon.

Opening this for discussion because:

1. This doesn't seem to be testing anything at all related to the rest
   of the test
2. Not really providing any value here.
3. Is testing that a technically invalid option is successful (whether
   the option should be valid as it relates to the CLI accepting it is
   debatable).
4. Such a case seems fringe and even a bug in whatever is calling the
   CLI with such options.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit e31086320e)
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2021-03-19 23:13:48 +00:00
Brian Goff
e1ee2823ec TestPushMultipleTags: Add support for 20.10 CLI
In 20.10 we no longer implicitly push all tags and require a
"--all-tags" flag, so add this to the test when the CLI is >= 20.10

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit 601707a655)
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2021-03-19 23:08:40 +00:00
Brian Goff
969bde2009
jsonfile: more defensive reader implementation
Tonis mentioned that we can run into issues if there is more error
handling added here. This adds a custom reader implementation which is
like io.MultiReader except it does not cache EOF's.
What got us into trouble in the first place is `io.MultiReader` will
always return EOF once it has received an EOF, however the error
handling that we are going for is to recover from an EOF because the
underlying file is a file which can have more data added to it after
EOF.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit 5a664dc87d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-03-19 18:18:55 +01:00
Brian Goff
cb501700e8
Fix handling for json-file io.UnexpectedEOF
When the multireader hits EOF, we will always get EOF from it, so we
cannot store the multrireader fro later error handling, only for the
decoder.

Thanks @tobiasstadler for pointing this error out.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit 4be98a38e7)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-03-19 18:18:52 +01:00
Sebastiaan van Stijn
6d72bb6acf
Merge pull request #42147 from thaJeztah/20.10_backport_41704_update_libseccomp
[20.10 backport] Use buster backports to build with libseccomp-2.4.4
2021-03-19 13:51:56 +01:00
Akihiro Suda
2d39a44c1c
overlayutils/userxattr.go: add "fast path" for kernel >= 5.11.0
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit a8008f7313)
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-03-19 03:36:07 +09:00
Akihiro Suda
95d2b686be
overlay2: support "userxattr" option (kernel 5.11)
The "userxattr" option is needed for mounting overlayfs inside a user namespace with kernel >= 5.11.

The "userxattr" option is NOT needed for the initial user namespace (aka "the host").

Also, Ubuntu (since circa 2015) and Debian (since 10) with kernel < 5.11 can mount the overlayfs in a user namespace without the "userxattr" option.

The corresponding kernel commit: 2d2f2d7322ff43e0fe92bf8cccdc0b09449bf2e1
> **ovl: user xattr**
>
> Optionally allow using "user.overlay." namespace instead of "trusted.overlay."
> ...
> Disable redirect_dir and metacopy options, because these would allow privilege escalation through direct manipulation of the
> "user.overlay.redirect" or "user.overlay.metacopy" xattrs.

Fix issue 42055

Related to containerd/containerd PR 5076

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit 11ef8d3ba9)
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-03-19 03:35:59 +09:00
Jeremy Huntwork
074270703c
Use buster backports to build with libseccomp-2.4.4
Fixes #41704

The latest released versions of the static binaries (20.10.3) are still unable
to use faccessat2 with musl-1.2.2 even though this was addressed in #41353 and
related issues. The underlying cause seems to be that the build system
here still uses the default version of libseccomp shipped with buster.
An updated version is available in buster backports:
https://packages.debian.org/buster-backports/libseccomp-dev

Signed-off-by: Jeremy Huntwork <jhuntwork@lightcubesolutions.com>
(cherry picked from commit 1600e851b5)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-03-15 08:58:30 +01:00
Akihiro Suda
fed6ba2790
Include VPNkit binary for arm64
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit 088e6ee790)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-03-12 09:02:45 +01:00
Tibor Vass
59ad0b6a86
Merge pull request #42124 from thaJeztah/20.10_backport_bump_containerd
[20.10 backport] update containerd binary to v1.4.4
2021-03-08 16:23:30 -08:00
Sebastiaan van Stijn
2ab3cd8c9e
update containerd binary to v1.4.4
full diff: https://github.com/containerd/containerd/compare/v1.4.3...v1.4.4

Release notes:

The fourth patch release for `containerd` 1.4 contains a fix for CVE-2021-21334
along with various other minor issues.
See [GHSA-36xw-fx78-c5r4](https://github.com/containerd/containerd/security/advisories/GHSA-36xw-fx78-c5r4)
for more details related to CVE-2021-21334.

Notable Updates

- Fix container create in CRI to prevent possible environment variable leak between containers
- Update shim server to return grpc NotFound error
- Add bounds on max `oom_score_adj` value for shim's AdjustOOMScore
- Update task manager to use fresh context when calling shim shutdown
- Update Docker resolver to avoid possible concurrent map access panic
- Update shim's log file open flags to avoid containerd hang on syscall open
- Fix incorrect usage calculation

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 1a49393403)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-03-08 18:58:05 +01:00
Akihiro Suda
bc795b19bf
Merge pull request #42103 from thaJeztah/20.10_backport_skip_test 2021-03-04 00:23:01 +09:00
Sebastiaan van Stijn
d3188dc164
Dockerfile: switch to "stable" dockerfile front-end
The `RUN --mount` options have been promoted to the stable channel,
so we can switch from "experimental" to "stable".

Note that the syntax directive should no longer be needed now, but
it's good practice to add a syntax-directive, to allow building on
older versions of docker.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 083dbe9fcd)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-03-03 11:11:54 +01:00
Akihiro Suda
98273a606a
dockerd-rootless-setuptool.sh: create CLI context "rootless"
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit f2f1c0fe38)
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-03-03 13:51:28 +09:00
Shengjing Zhu
a0670c6d3d
pkg/archive: TestUntarParentPathPermissions requires root
=== RUN   TestUntarParentPathPermissions
    archive_unix_test.go:171: assertion failed: error is not nil: chown /tmp/TestUntarParentPathPermissions694189715/foo: operation not permitted

Signed-off-by: Shengjing Zhu <zhsj@debian.org>
(cherry picked from commit f23c1c297d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-03-01 22:05:09 +01:00
Sebastiaan van Stijn
04d9b581e9
Update documentation links
- Using "/go/" redirects for some topics, which allows us to
  redirect to new locations if topics are moved around in the
  documentation.
- Updated some old URLs to their new location.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 328de0b8d9)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-02-25 21:54:39 +01:00
Tibor Vass
363e9a88a1
Merge pull request #42061 from thaJeztah/20.10_backport_bump_buildkit
[20.10 backport] vendor: github.com/moby/buildkit v0.8.2
2021-02-24 22:27:18 -08:00
Akihiro Suda
1015b5b438
dockerd-rootless.sh: prohibit running as root
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit 9351e19658)
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-02-25 13:33:15 +09:00
Tibor Vass
35f5f9e624 builder: fix incorrect cache match for inline cache with empty layers
See https://github.com/moby/buildkit/pull/1993

Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 9bf93e90fa)
Signed-off-by: Tibor Vass <tibor@docker.com>
2021-02-25 01:50:37 +00:00
Tibor Vass
035cb276d9
Merge pull request #42070 from thaJeztah/20.10_backport_rootless_typo_guard
[20.10 backport] dockerd-rootless.sh: add typo guard
2021-02-24 17:42:50 -08:00
Sebastiaan van Stijn
3ce37a6aa4 vendor: github.com/moby/buildkit v0.8.2
full diff: 68bb095353...9065b18ba4

- fix seccomp compatibility in 32bit arm
    - fixes Unable to build alpine:edge containers for armv7
    - fixes Buildx failing to build for arm/v7 platform on arm64 machine
- resolver: avoid error caching on token fetch
    - fixes "Error: i/o timeout should not be cached"
- fileop: fix checksum to contain indexes of inputs
- frontend/dockerfile: add RunCommand.FlagsUsed field
    - relates to [20.10] Classic builder silently ignores unsupported Dockerfile command flags
- update qemu emulators
    - relates to "Impossible to run git clone inside buildx with non x86 architecture"
- Fix reference count issues on typed errors with mount references
    - fixes errors on releasing mounts with typed execerror refs
    - fixes / addresses invalid mutable ref when using shared cache mounts
- dockerfile/docs: fix frontend image tags
- git: set token only for main remote access
    - fixes "Loading repositories with submodules is repeated. Failed to clone submodule from googlesource"
- allow skipping empty layer detection on cache export

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 9962a3f74e)
Signed-off-by: Tibor Vass <tibor@docker.com>
2021-02-25 01:41:11 +00:00
Akihiro Suda
5e8c1b4f7d
dockerd-rootless.sh: add typo guard
`dockerd-rootless.sh install` is a common typo of `dockerd-rootless-setuptool.sh install`.

Now `dockerd-rootless.sh install` shows human-readable error.

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit 8dc6c109b5)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-02-24 22:13:50 +01:00
Tibor Vass
7f547e15c7
Merge pull request #42060 from thaJeztah/20.10_backport_bump_swarmkit
[20.10 backport] Update Swarmkit to pick up fixes to heartbeat period and stalled tasks
2021-02-24 12:46:03 -08:00
Tibor Vass
830471acf5
Merge pull request #42066 from thaJeztah/20.10_backport_check_config
[20.10 backport] check-config.sh: add NETFILTER_XT_MARK
2021-02-24 12:45:33 -08:00
Tibor Vass
7ae42f5797
Merge pull request #42065 from thaJeztah/20.10_backport_lease_blobs_fixes
[20.10 backport] builder: fix blobs releasing via leases after pull
2021-02-24 12:44:51 -08:00
Sebastiaan van Stijn
f3d130d743
Merge pull request #42049 from thaJeztah/20.10_backport_builder_pull_fix
[20.10 backport] builder: fix pull synchronization regression
2021-02-23 21:15:32 +01:00
Piotr Karbowski
a24d92f95b
check-config.sh: add NETFILTER_XT_MARK
Points out another symbol that Docker might need. in this case Docker's
mesh network in swarm mode does not route Virtual IPs if it's unset.

From /var/logs/docker.log:
time="2021-02-19T18:15:39+01:00" level=error msg="set up rule failed, [-t mangle -A INPUT -d 10.0.1.2/32 -j MARK --set-mark 257]:  (iptables failed: iptables --wait -t mang
le -A INPUT
-d 10.0.1.2/32 -j MARK --set-mark 257: iptables v1.8.7 (legacy): unknown option \"--set-mark\"\nTry `iptables -h' or 'iptables --help' for more information.\n (exit status 2))"

Bug: https://github.com/moby/libnetwork/issues/2227
Bug: https://github.com/docker/for-linux/issues/644
Bug: https://github.com/docker/for-linux/issues/525
Signed-off-by: Piotr Karbowski <piotr.karbowski@protonmail.ch>
(cherry picked from commit e8ceb97646)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-02-23 19:25:47 +01:00
Tonis Tiigi
80019e1b0e
builder: fix blobs releasing via leases after pull
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit 5c01d06f72)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-02-23 19:22:35 +01:00
Tibor Vass
dc1606ad79
Merge pull request #42046 from thaJeztah/20.10_labels_regex_length_check
[20.10 backport] Check the length of the correct variable #42039
2021-02-23 10:00:58 -08:00
Adam Williams
2a220f1f3d
Update Swarmkit to pick up fixes to heartbeat period and stalled tasks
Signed-off-by: Adam Williams <awilliams@mirantis.com>
(cherry picked from commit cbd2f726bf)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-02-23 09:57:37 +01:00
Sebastiaan van Stijn
148e6c9514
Merge pull request #42017 from thaJeztah/20.10_backport_build_fixes
[20.10 backport]: avoid creating parent dirs for XGlobalHeader, and fix permissions
2021-02-22 20:04:04 +01:00
Tonis Tiigi
da1a672102
builder: fix pull synchronization regression
Config resolution was synchronized based on a wrong key as ref
variable is initialized only after in the same function. Using
the right key isn't fully correct either as the synchronized method
changes properties of the puller instance and can't be just skipped.
Added better error handling for the same case as well.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit b53ea19c49)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-02-19 10:08:04 +01:00
Nathan Carlson
0e001154f9
Check the length of the correct variable #42039
Signed-off-by: Nathan Carlson <carl4403@umn.edu>
(cherry picked from commit 8d73c1ad68)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-02-18 22:23:34 +01:00
Sebastiaan van Stijn
df2cfb4d33
Merge pull request #42045 from cpuguy83/20.10_fallback_manifest_on_bad_plat
[20.10] Fallback to manifest list when no platform match
2021-02-18 21:37:34 +01:00
Tibor Vass
7f6776fb5e
Merge pull request #41971 from thaJeztah/20.10_backport_seccomp_update
[20.10 backport] profiles: seccomp: update to Linux 5.11 syscall list
2021-02-18 12:36:47 -08:00
Tibor Vass
caa48de224
Merge pull request #41974 from thaJeztah/20.10_backport_for_linux_1169_plugins_custom_runtime-panic
[20.10 backport] Add shim config for custom runtimes for plugins
2021-02-18 12:36:21 -08:00
Tibor Vass
6a86c25cf0
Merge pull request #41972 from thaJeztah/20.10_backport_net_leak_fix
[20.10 backport] builder: ensure libnetwork state file do not leak
2021-02-18 12:34:14 -08:00
Tibor Vass
ff486ae873
Merge pull request #41973 from thaJeztah/20.10_backport_fix_builder_inconsisent_platform
[20.10 backport] Fix builder inconsistent error on buggy platform
2021-02-18 12:32:53 -08:00
Tibor Vass
b55d9e1b91
Merge pull request #41976 from thaJeztah/20.10_backport_reuse
[20.10 backport] replace json.Unmarshal with NewFromJSON in Create
2021-02-18 12:30:18 -08:00
Tibor Vass
b81e649d2b
Merge pull request #41977 from thaJeztah/20.10_backport_minor_fixes
[20.10 backport] assorted small fixes, docs changes, and contrib
2021-02-18 12:29:07 -08:00
Tibor Vass
5bb85a962a
Merge pull request #42001 from thaJeztah/20.10_backport_fix_cgroup_rule_panic
[20.10 backport] Fix daemon panic when starting container with invalid device cgroup rule
2021-02-18 12:27:38 -08:00
Tibor Vass
6de7dbd225
Merge pull request #42012 from thaJeztah/20.10_backport_fix_nanocpus_casing
[20.10 backport] api/docs: fix NanoCPUs casing in swagger
2021-02-18 12:26:04 -08:00
Tibor Vass
8e2c5fc178
Merge pull request #42013 from thaJeztah/20.10_backport_42003_fix_userns_uid_username_match
[20.10 backport] Fix userns-remap option when username & UID match
2021-02-18 12:25:13 -08:00
Tibor Vass
f88c4aeaa0
Merge pull request #42014 from thaJeztah/20.10_backport_bump_runc_binary
[20.10 backport] update runc binary to v1.0.0-rc93
2021-02-18 12:24:02 -08:00
Tibor Vass
c981698f9a
Merge pull request #42025 from thaJeztah/20.10_backport_bump_rootlesskit
[20.10 backport] Update rootlesskit to v0.13.1 to fix handling of IPv6 addresses
2021-02-18 12:17:55 -08:00
Tibor Vass
d6ae06a70a
Merge pull request #42042 from thaJeztah/20.10_backport_docker_dind_integration_test_fix_subnet_missmatch
[20.10 backport] Update TestDaemonRestartWithLiveRestore: fix docker0 subnet missmatch
2021-02-18 12:15:05 -08:00
Brian Goff
3beb2e4422 Move cpu variant checks into platform matcher
Wrap platforms.Only and fallback to our ignore mismatches due to  empty
CPU variants. This just cleans things up and makes the logic re-usable
in other places.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit 50f39e7247)
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2021-02-18 20:12:07 +00:00
Brian Goff
0caf485abb Fallback to manifest list when no platform match
In some cases, in fact many in the wild, an image may have the incorrect
platform on the image config.
This can lead to failures to run an image, particularly when a user
specifies a `--platform`.
Typically what we see in the wild is a manifest list with an an entry
for, as an example, linux/arm64 pointing to an image config that has
linux/amd64 on it.

This change falls back to looking up the manifest list for an image to
see if the manifest list shows the image as the correct one for that
platform.

In order to accomplish this we need to traverse the leases associated
with an image. Each image, if pulled with Docker 20.10, will have the
manifest list stored in the containerd content store with the resource
assigned to a lease keyed on the image ID.
So we look up the lease for the image, then look up the assocated
resources to find the manifest list, then check the manifest list for a
platform match, then ensure that manifest referes to our image config.

This is only used as a fallback when a user specified they want a
particular platform and the image config that we have does not match
that platform.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit 4be5453215)
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2021-02-18 20:12:00 +00:00
Brian Goff
24e1d7fa59
Merge pull request #41975 from thaJeztah/20.10_backport_41794_sized_logger
[20.10 backport] Handle long log messages correctly on SizedLogger
2021-02-17 16:51:24 -08:00
Aleksa Sarai
a6a88b3145
profiles: seccomp: update to Linux 5.11 syscall list
These syscalls (some of which have been in Linux for a while but were
missing from the profile) fall into a few buckets:

 * close_range(2), epoll_pwait2(2) are just extensions of existing "safe
   for everyone" syscalls.

 * The mountv2 API syscalls (fs*(2), move_mount(2), open_tree(2)) are
   all equivalent to aspects of mount(2) and thus go into the
   CAP_SYS_ADMIN category.

 * process_madvise(2) is similar to the other process_*(2) syscalls and
   thus goes in the CAP_SYS_PTRACE category.

Signed-off-by: Aleksa Sarai <asarai@suse.de>
(cherry picked from commit 54eff4354b)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-02-17 21:22:12 +01:00
Tonis Tiigi
e3750357a5
builder: ensure libnetwork state file do not leak
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit 7c7e168902)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-02-17 21:21:25 +01:00
Brian Goff
ab5711e619
Fix builder inconsistent error on buggy platform
When pulling an image by platform, it is possible for the image's
configured platform to not match what was in the manifest list.
The image itself is buggy because either the manifest list is incorrect
or the image config is incorrect. In any case, this is preventing people
from upgrading because many times users do not have control over these
buggy images.

This was not a problem in 19.03 because we did not compare on platform
before. It just assumed if we had the image it was the one we wanted
regardless of platform, which has its own problems.

Example Dockerfile that has this problem:

```Dockerfile
FROM --platform=linux/arm64 k8s.gcr.io/build-image/debian-iptables:buster-v1.3.0
RUN echo hello
```

This fails the first time you try to build after it finishes pulling but
before performing the `RUN` command.
On the second attempt it works because the image is already there and
does not hit the code that errors out on platform mismatch (Actually it
ignores errors if an image is returned at all).

Must be run with the classic builder (DOCKER_BUILDKIT=0).

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit 399695305c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-02-17 21:20:46 +01:00
Brian Goff
df2a989769
Add shim config for custom runtimes for plugins
This fixes a panic when an admin specifies a custom default runtime,
when a plugin is started the shim config is nil.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit 2903863a1d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-02-17 21:20:03 +01:00
Kazuyoshi Kato
d13e162a63
Handle long log messages correctly on SizedLogger
Loggers that implement BufSize() (e.g. awslogs) uses the method to
tell Copier about the maximum log line length. However loggerWithCache
and RingBuffer hide the method by wrapping loggers.

As a result, Copier uses its default 16KB limit which breaks log
lines > 16kB even the destinations can handle that.

This change implements BufSize() on loggerWithCache and RingBuffer to
make sure these logger wrappes don't hide the method on the underlying
loggers.

Fixes #41794.

Signed-off-by: Kazuyoshi Kato <katokazu@amazon.com>
(cherry picked from commit bb11365e96)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-02-17 21:19:02 +01:00
Jim Lin
34446d0343
replace json.Unmarshal with NewFromJSON in Create
Signed-off-by: Jim Lin <b04705003@ntu.edu.tw>
(cherry picked from commit c9ec21e17a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-02-17 21:18:19 +01:00
Sebastiaan van Stijn
c00fb1383f
docs: fix double "the" in existing API versions
Backport of 2db5676c6e to the swagger files
used in the documentation

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 240d0b37bb)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-02-17 21:17:42 +01:00
Frederico F. de Oliveira
b7e6803ec4
swagger.yaml: Remove extra 'the' wrapped by newline
This PR was originally proposed by @phillc here: https://github.com/docker/engine/pull/456

Signed-off-by: FreddieOliveira <fredf_oliveira@ufu.br>
(cherry picked from commit 2db5676c6e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-02-17 21:17:40 +01:00
Kir Kolyshkin
420de4c569
contrib/check-config.sh: fix INET_XFRM_MODE_TRANSPORT
This parameter was removed by kernel commit 4c145dce260137,
which made its way to kernel v5.3-rc1. Since that commit,
the functionality is built-in (i.e. it is available as long
as CONFIG_XFRM is on).

Make the check conditional.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 06d9020fac)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-02-17 21:17:39 +01:00
Kir Kolyshkin
8412078b1e
contrib/check-config.sh: fix IOSCHED_CFQ CFQ_GROUP_IOSCHED
These config options are removed by kernel commit f382fb0bcef4,
which made its way into kernel v5.0-rc1.

Make the check conditional.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 18e0543587)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-02-17 21:17:37 +01:00
Kir Kolyshkin
bb0866f04e
contrib/check-config.sh: fix MEMCG_SWAP_ENABLED
Kernel commit 2d1c498072de69e (which made its way into kernel v5.8-rc1)
removed CONFIG_MEMCG_SWAP_ENABLED Kconfig option, making swap accounting
always enabled (unless swapaccount=0 boot option is provided).

Make the check conditional.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 070f9d9dd3)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-02-17 21:17:35 +01:00
Kir Kolyshkin
db47bec3c7
contrib/check-config.sh: fix NF_NAT_NEEDED
CONFIG_NF_NAT_NEEDED was removed in kernel commit 4806e975729f99c7,
which made its way into v5.2-rc1. The functionality is now under
NF_NAT which we already check for.

Make the check for NF_NAT_NEEDED conditional.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 03da41152a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-02-17 21:17:33 +01:00
Kir Kolyshkin
6bc47ca4b4
contrib/check-config.sh: fix NF_NAT_IPV4
CONFIG_NF_NAT_IPV4 was removed in kernel commit 3bf195ae6037e310,
which made its way into v5.1-rc1. The functionality is now under
NF_NAT which we already check for.

Make the check for NF_NAT_IPV4 conditional.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit eeb53c1f22)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-02-17 21:17:31 +01:00
Kir Kolyshkin
491642e696
contrib/check-config.sh: support for cgroupv2
Before:

> Generally Necessary:
> - cgroup hierarchy: nonexistent??
>     (see https://github.com/tianon/cgroupfs-mount)

After:

> Generally Necessary:
> - cgroup hierarchy: cgroupv2

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 76b59065ae)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-02-17 21:17:30 +01:00
gunadhya
cda6988478
Fix Error in daemon_unix.go and docker_cli_run_unit_test.go
Signed-off-by: gunadhya <6939749+gunadhya@users.noreply.github.com>
(cherry picked from commit 64465f3b5f)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-02-17 21:17:28 +01:00
Sebastiaan van Stijn
18543cd8c8
Merge pull request #42000 from thaJeztah/20.10_backport_fix_dockerfile_simple
[20.10 backport] Dockerfile.simple: Fix compile docker binary error with btrfs
2021-02-17 21:17:02 +01:00
Sebastiaan van Stijn
1640d7b986
Fix daemon panic when starting container with invalid device cgroup rule
This fixes a panic when an invalid "device cgroup rule" is passed, resulting
in an "index out of range".

This bug was introduced in the original implementation in 1756af6faf,
but was not reproducible when using the CLI, because the same commit also added
client-side validation on the flag before making an API request. The following
example, uses an invalid rule (`c *:*  rwm` - two spaces before the permissions);

```console
$ docker run --rm --network=host --device-cgroup-rule='c *:*  rwm' busybox
invalid argument "c *:*  rwm" for "--device-cgroup-rule" flag: invalid device cgroup format 'c *:*  rwm'
```

Doing the same, but using the API results in a daemon panic when starting the container;

Create a container with an invalid device cgroup rule:

```console
curl -v \
  --unix-socket /var/run/docker.sock \
  "http://localhost/v1.41/containers/create?name=foobar" \
  -H "Content-Type: application/json" \
  -d '{"Image":"busybox:latest", "HostConfig":{"DeviceCgroupRules": ["c *:*  rwm"]}}'
```

Start the container:

```console
curl -v \
  --unix-socket /var/run/docker.sock \
  -X POST \
  "http://localhost/v1.41/containers/foobar/start"
```

Observe the daemon logs:

```
2021-01-22 12:53:03.313806 I | http: panic serving @: runtime error: index out of range [0] with length 0
goroutine 571 [running]:
net/http.(*conn).serve.func1(0xc000cb2d20)
	/usr/local/go/src/net/http/server.go:1795 +0x13b
panic(0x2f32380, 0xc000aebfc0)
	/usr/local/go/src/runtime/panic.go:679 +0x1b6
github.com/docker/docker/oci.AppendDevicePermissionsFromCgroupRules(0xc000175c00, 0x8, 0x8, 0xc0000bd380, 0x1, 0x4, 0x0, 0x0, 0xc0000e69c0, 0x0, ...)
	/go/src/github.com/docker/docker/oci/oci.go:34 +0x64f
```

This patch:

- fixes the panic, allowing the daemon to return an error on container start
- adds a unit-test to validate various permutations
- adds a "todo" to verify the regular expression (and handling) of the "a" (all) value

We should also consider performing this validation when _creating_ the container,
so that an error is produced early.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 5cc1753f2c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-02-17 21:16:01 +01:00
Sebastiaan van Stijn
6e3f2acdac
docs: fix NanoCPUs casing
While the field in the Go struct is named `NanoCPUs`, it has a JSON label to
use `NanoCpus`, which was added in the original pull request (not clear what
the reason was); 846baf1fd3

Some notes:

- Golang processes field names case-insensitive, so when *using* the API,
  both cases should work, but when inspecting a container, the field is
  returned as `NanoCpus`.
- This only affects Containers.Resources. The `Limits` and `Reservation`
  for SwarmKit services and SwarmKit "nodes" do not override the name
  for JSON, so have the canonical (`NanoCPUs`) casing.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 8e2343ffd4)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-02-17 21:15:21 +01:00
Sebastiaan van Stijn
ad777ff3bc
api: fix NanoCPUs casing in swagger
While the field in the Go struct is named `NanoCPUs`, it has a JSON label to
use `NanoCpus`, which was added in the original pull request (not clear what
the reason was); 846baf1fd3

Some notes:

- Golang processes field names case-insensitive, so when *using* the API,
  both cases should work, but when inspecting a container, the field is
  returned as `NanoCpus`.
- This only affects Containers.Resources. The `Limits` and `Reservation`
  for SwarmKit services and SwarmKit "nodes" do not override the name
  for JSON, so have the canonical (`NanoCPUs`) casing.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 2bd46ed7e5)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-02-17 21:15:19 +01:00
Grant Millar
94d2467613
Fix userns-remap option when username & UID match
Signed-off-by: Grant Millar <rid@cylo.io>
(cherry picked from commit 2ad187fd4a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-02-17 21:14:40 +01:00
Sebastiaan van Stijn
acb8a48a3c
update runc binary to v1.0.0-rc93
full diff: https://github.com/opencontainers/runc/compare/v1.0.0-rc92...v1.0.0-rc93
release notes: https://github.com/opencontainers/runc/releases/tag/v1.0.0-rc93

Release notes for runc v1.0.0-rc93
-------------------------------------------------

This is the last feature-rich RC release and we are in a feature-freeze until
1.0. 1.0.0~rc94 will be released in a few weeks with minimal bug fixes only,
and 1.0.0 will be released soon afterwards.

- runc's cgroupv2 support is no longer considered experimental. It is now
  believed to be fully ready for production deployments. In addition, runc's
  cgroup code has been improved:
    - The systemd cgroup driver has been improved to be more resilient and
      handle more systemd properties correctly.
    - We now make use of openat2(2) when possible to improve the security of
      cgroup operations (in future runc will be wholesale ported to libpathrs to
      get this protection in all codepaths).
- runc's mountinfo parsing code has been reworked significantly, making
  container startup times significantly faster and less wasteful in general.
- runc now has special handling for seccomp profiles to avoid making new
  syscalls unusable for glibc. This is done by installing a custom prefix to
  all seccomp filters which returns -ENOSYS for syscalls that are newer than
  any syscall in the profile (meaning they have a larger syscall number).

  This should not cause any regressions (because previously users would simply
  get -EPERM rather than -ENOSYS, and the rule applied above is the most
  conservative rule possible) but please report any regressions you find as a
  result of this change -- in particular, programs which have special fallback
  code that is only run in the case of -EPERM.
- runc now supports the following new runtime-spec features:
    - The umask of a container can now be specified.
    - The new Linux 5.9 capabilities (CAP_PERFMON, CAP_BPF, and
      CAP_CHECKPOINT_RESTORE) are now supported.
    - The "unified" cgroup configuration option, which allows users to explicitly
      specify the limits based on the cgroup file names rather than abstracting
      them through OCI configuration. This is currently limited in scope to
      cgroupv2.
- Various rootless containers improvements:
    - runc will no longer cause conflicts if a user specifies a custom device
      which conflicts with a user-configured device -- the user device takes
      precedence.
    - runc no longer panics if /sys/fs/cgroup is missing in rootless mode.
- runc --root is now always treated as local to the current working directory.
- The --no-pivot-root hardening was improved to handle nested mounts properly
  (please note that we still strongly recommend that users do not use
  --no-pivot-root -- it is still an insecure option).
- A large number of code cleanliness and other various cleanups, including
  fairly large changes to our tests and CI to make them all run more
  efficiently.

For packagers the following changes have been made which will have impact on
your packaging of runc:

- The "selinux" and "apparmor" buildtags have been removed, and now all runc
  builds will have SELinux and AppArmor support enabled. Note that "seccomp"
  is still optional (though we very highly recommend you enable it).
- make install DESTDIR= now functions correctly.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 28e5a3c5a4)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-02-17 21:13:50 +01:00
Sebastiaan van Stijn
5d442b1cb7
pkg/archive: Unpack() use 0755 permissions for missing directories
Commit edb62a3ace fixed a bug in MkdirAllAndChown()
that caused the specified permissions to not be applied correctly. As a result
of that bug, the configured umask would be applied.

When extracting archives, Unpack() used 0777 permissions when creating missing
parent directories for files that were extracted.
Before edb62a3ace, this resulted in actual
permissions of those directories to be 0755 on most configurations (using a
default 022 umask).

Creating these directories should not depend on the host's umask configuration.
This patch changes the permissions to 0755 to match the previous behavior,
and to reflect the original intent of using 0755 as default.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 25ada76437)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-02-17 21:12:57 +01:00
Tonis Tiigi
5db18e0aba
archive: avoid creating parent dirs for XGlobalHeader
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit ba7906aef3)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-02-17 21:12:55 +01:00
Sebastiaan van Stijn
94feac18d2
Update rootlesskit to v0.13.1 to fix handling of IPv6 addresses
v0.13.1

- Refactor `ParsePortSpec` to handle IPv6 addresses, and improve validation

v0.13.0

- `rootlesskit --pidns`: fix propagating exit status
- Support cgroup2 evacuation, e.g., `systemd-run -p Delegate=yes --user -t rootlesskit --cgroupns --pidns --evacuate-cgroup2=evac --net=slirp4netns bash`

v0.12.0

- Port forwarding API now supports setting `ChildIP`
- The `vendor` directory is no longer included in this repo. Run `go mod vendor` if you need

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit e32ae1973a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-02-17 21:11:17 +01:00
Alexis Ries
cc377d27ac
Update TestDaemonRestartWithLiveRestore: fix docker0 subnet missmatch
Fix docker0 subnet missmatch when running from docker in docker (dind)

Signed-off-by: Alexis Ries <ries.alexis@gmail.com>
(cherry picked from commit 96e103feb1)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-02-17 21:07:36 +01:00
Brian Goff
fae366b323
Merge pull request #41970 from thaJeztah/20.10_backport_testing_fixes 2021-02-17 09:37:19 -08:00
Brian Goff
dfce527001
Merge pull request #42030 from thaJeztah/20.10_backport_cgroup2ci_jenkins 2021-02-16 09:37:37 -08:00
Sebastiaan van Stijn
fc07fecfb5
TestBuildUserNamespaceValidateCapabilitiesAreV2: verify build completed
Check if the `docker build` completed successfully before continuing.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit fa480403c7)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-02-16 14:20:02 +01:00
Sebastiaan van Stijn
f7893961de
TestBuildUserNamespaceValidateCapabilitiesAreV2: use correct image name
This currently doesn't make a difference, because load.FrozenImagesLinux()
currently loads all frozen images, not just the specified one, but in case
that is fixed/implemented at some point.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 26965fbfa0)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-02-16 14:20:00 +01:00
Akihiro Suda
d31b2141ae
Jenkinsfile: add cgroup2
Thanks to Stefan Scherer for setting up the Jenkins nodes.

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit c23b99f4db)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-02-16 09:22:30 +01:00
Akihiro Suda
5de9bc7e01
TestInspectOomKilledTrue: skip on cgroup v2
The test fails intermittently on cgroup v2.

```
=== FAIL: amd64.integration.container TestInspectOomKilledTrue (0.53s)
    kill_test.go:171: assertion failed: true (true bool) != false (inspect.State.OOMKilled bool)
```

Tracked in issue 41929

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit c316dd7cc5)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-02-16 09:22:27 +01:00
Tibor Vass
11443bc681
Merge pull request #41957 from AkihiroSuda/cherrypick-41892-2010
[20.10 backport] pkg/archive: allow mknodding FIFO inside userns
2021-02-11 11:56:54 -08:00
Lei Jiang
ff49cb3e33
Dockerfile.simple: Fix compile docker binary error with btrfs
Use the image build from Dockerfile.simple to build docker binary failed
with not find <brtfs/ioctl.h>, we need to install libbtrfs-dev to fix this.
```
Building: bundles/dynbinary-daemon/dockerd-dev
GOOS="" GOARCH="" GOARM=""
.gopath/src/github.com/docker/docker/daemon/graphdriver/btrfs/btrfs.go:8:10: fatal error: btrfs/ioctl.h: No such file or directory
 #include <btrfs/ioctl.h>

```

Signed-off-by: Lei Jitang <leijitang@outlook.com>
(cherry picked from commit dd7ee8ea3e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-02-08 17:47:03 +01:00
Sebastiaan van Stijn
49e706e14c
Dockerfile.buildx: update buildx to v0.5.1
full diff: https://github.com/docker/buildx/compare/v0.3.1...v0.5.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 30b20a6bdd)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-02-03 13:54:27 +01:00
Sebastiaan van Stijn
0211909bde
testing: update docker-py 4.4.1
run docker-py integration tests of the latest release;

full diff: https://github.com/docker/docker-py/compare/4.3.0...4.4.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 14fb165085)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-02-03 13:54:22 +01:00
Sebastiaan van Stijn
faf6442f80
integration: fix TestBuildUserNamespaceValidateCapabilitiesAreV2 not using frozen image
Commit f2f5106c92 added this test to verify loading
of images that were built with user-namespaces enabled.

However, because this test spins up a new daemon, not the daemon that's set up by
the test-suite's `TestMain()` (which loads the frozen images).

As a result, the `debian:bullseye` image was pulled from Docker Hub when running
the test;

    Calling POST /v1.41/images/load?quiet=1
    Applying tar in /go/src/github.com/docker/docker/bundles/test-integration/TestBuildUserNamespaceValidateCapabilitiesAreV2/d4d366b15997b/root/165536.165536/overlay2/3f7f9375197667acaf7bc810b34689c21f8fed9c52c6765c032497092ca023d6/diff" storage-driver=overlay
    Applied tar sha256:845f0e5159140e9dbcad00c0326c2a506fbe375aa1c229c43f082867d283149c to 3f7f9375197667acaf7bc810b34689c21f8fed9c52c6765c032497092ca023d6, size: 5922359
    Calling POST /v1.41/build?buildargs=null&cachefrom=null&cgroupparent=&cpuperiod=0&cpuquota=0&cpusetcpus=&cpusetmems=&cpushares=0&dockerfile=&labels=null&memory=0&memswap=0&networkmode=&rm=0&shmsize=0&t=capabilities%3A1.0&target=&ulimits=null&version=
    Trying to pull debian from https://registry-1.docker.io v2
    Fetching manifest from remote" digest="sha256:f169dbadc9021fc0b08e371d50a772809286a167f62a8b6ae86e4745878d283d" error="<nil>" remote="docker.io/library/debian:bullseye
    Pulling ref from V2 registry: debian:bullseye
    ...

This patch updates `TestBuildUserNamespaceValidateCapabilitiesAreV2` to load the
frozen image. `StartWithBusybox` is also changed to `Start`, because the test
is not using the busybox image, so there's no need to load it.

In a followup, we should probably add some utilities to make this easier to set up
(and to allow passing the list frozen images that we want to load, without having
to "hard-code" the image name to load).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 46dfc31342)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-02-03 13:54:16 +01:00
Brian Goff
f0e526f43e
Make test work with rootless mode
Using `d.Kill()` with rootless mode causes the restarted daemon to not
be able to start containerd (it times out).

Originally this was SIGKILLing the daemon because we were hoping to not
have to manipulate on disk state, but since we need to anyway we can
shut it down normally.

I also tested this to ensure the test fails correctly without the fix
that the test was added to check for.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit e6591a9c7a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-02-03 13:54:09 +01:00
Brian Goff
11ecfe8a81
Merge pull request #41959 from AkihiroSuda/cherrypick-41917-2010
[20.10 backport] TestCgroupNamespacesRunOlderClient: support cgroup v2
2021-02-02 10:54:01 -08:00
Brian Goff
49df387b71
Merge pull request #41958 from AkihiroSuda/cherrypick-41894-2010 2021-02-02 10:52:42 -08:00
Brian Goff
54f561aeb9
Merge pull request #41956 from AkihiroSuda/cherrypick-41947-2010
[20.10 backport] rootless: prevent the service hanging when stopping (set systemd KillMode to mixed)
2021-02-02 10:51:08 -08:00
Akihiro Suda
519a55f491
TestCgroupNamespacesRunOlderClient: support cgroup v2
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-02-02 14:34:08 +09:00
Akihiro Suda
b6a6a35684
docker info: adjust warning strings for cgroup v2
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit 00225e220f)
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-02-02 14:32:13 +09:00
Akihiro Suda
25bd941ae4
docker info: silence unhandleable warnings
The following warnings in `docker info` are now discarded,
because there is no action user can actually take.

On cgroup v1:
- "WARNING: No blkio weight support"
- "WARNING: No blkio weight_device support"

On cgroup v2:
- "WARNING: No kernel memory TCP limit support"
- "WARNING: No oom kill disable support"

`docker run` still prints warnings when the missing feature is being attempted to use.

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit 8086443a44)
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-02-02 14:32:00 +09:00
Akihiro Suda
a287e76e15
pkg/archive: allow mknodding FIFO inside userns
Fix #41803

Also attempt to mknod devices.
Mknodding devices are likely to fail, but still worth trying when
running with a seccomp user notification.

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit d5d5cccb7e)
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-02-02 14:29:49 +09:00
Akihiro Suda
58283298d7
rootless: set systemd KillMode to mixed
Now `systemctl --user stop docker` completes just with in 1 or 2 seconds.

Fix issue 41944 ("Docker rootless does not exit properly if containers are running")

See systemd.kill(5) https://www.freedesktop.org/software/systemd/man/systemd.kill.html

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit 05566adf71)
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-02-02 14:26:47 +09:00
Tibor Vass
46229ca1d8 Use golang.org/x/sys/execabs
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 7ca0cb7ffa)
Signed-off-by: Tibor Vass <tibor@docker.com>
2021-01-28 21:33:12 +00:00
Brian Goff
a7d4af84bd pull: Validate layer digest format
Otherwise a malformed or empty digest may cause a panic.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2021-01-28 21:33:12 +00:00
Brian Goff
611eb6ffb3 buildkit: Apply apparmor profile
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2021-01-28 21:33:12 +00:00
Tibor Vass
4afe620fac vendor buildkit 68bb095353c65bc3993fd534c26cf77fe05e61b1
Signed-off-by: Tibor Vass <tibor@docker.com>
2021-01-28 20:20:56 +00:00
Brian Goff
e908cc3901 Use real root with 0701 perms
Various dirs in /var/lib/docker contain data that needs to be mounted
into a container. For this reason, these dirs are set to be owned by the
remapped root user, otherwise there can be permissions issues.
However, this uneccessarily exposes these dirs to an unprivileged user
on the host.

Instead, set the ownership of these dirs to the real root (or rather the
UID/GID of dockerd) with 0701 permissions, which allows the remapped
root to enter the directories but not read/write to them.
The remapped root needs to enter these dirs so the container's rootfs
can be configured... e.g. to mount /etc/resolve.conf.

This prevents an unprivileged user from having read/write access to
these dirs on the host.
The flip side of this is now any user can enter these directories.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2021-01-26 17:23:32 +00:00
Brian Goff
bfedd27259 Do not set DOCKER_TMP to be owned by remapped root
The remapped root does not need access to this dir.
Having this owned by the remapped root opens the host up to an
uprivileged user on the host being able to escalate privileges.

While it would not be normal for the remapped UID to be used outside of
the container context, it could happen.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2021-01-26 17:23:32 +00:00
Brian Goff
edb62a3ace Ensure MkdirAllAndChown also sets perms
Generally if we ever need to change perms of a dir, between versions,
this ensures the permissions actually change when we think it should
change without having to handle special cases if it already existed.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2021-01-26 17:23:32 +00:00
9384 changed files with 535376 additions and 1142827 deletions

22
.DEREK.yml Normal file
View file

@ -0,0 +1,22 @@
curators:
- aboch
- alexellis
- andrewhsu
- anonymuse
- arkodg
- chanwit
- ehazlett
- fntlnz
- gianarb
- kolyshkin
- mgoelzer
- olljanat
- programmerq
- rheinwein
- ripcurld0
- thajeztah
features:
- comments
- pr_description_required

View file

@ -1,21 +0,0 @@
{
"name": "moby",
"build": {
"context": "..",
"dockerfile": "../Dockerfile",
"target": "devcontainer"
},
"workspaceFolder": "/go/src/github.com/docker/docker",
"workspaceMount": "source=${localWorkspaceFolder},target=/go/src/github.com/docker/docker,type=bind,consistency=cached",
"remoteUser": "root",
"runArgs": ["--privileged"],
"customizations": {
"vscode": {
"extensions": [
"golang.go"
]
}
}
}

View file

@ -1,4 +1,6 @@
.git
bundles/
cli/winresources/**/winres.json
cli/winresources/**/*.syso
.go-pkg-cache
.gopath
bundles
vendor/pkg

3
.gitattributes vendored
View file

@ -1,3 +0,0 @@
Dockerfile* linguist-language=Dockerfile
vendor.mod linguist-language=Go-Module
vendor.sum linguist-language=Go-Checksums

3
.github/CODEOWNERS vendored
View file

@ -5,6 +5,9 @@
builder/** @tonistiigi
contrib/mkimage/** @tianon
daemon/graphdriver/devmapper/** @rhvgoyal
daemon/graphdriver/lcow/** @johnstep
daemon/graphdriver/overlay/** @dmcgowan
daemon/graphdriver/overlay2/** @dmcgowan
daemon/graphdriver/windows/** @johnstep
daemon/logger/awslogs/** @samuelkarp

70
.github/ISSUE_TEMPLATE.md vendored Normal file
View file

@ -0,0 +1,70 @@
<!--
If you are reporting a new issue, make sure that we do not have any duplicates
already open. You can ensure this by searching the issue list for this
repository. If there is a duplicate, please close your issue and add a comment
to the existing issue instead.
If you suspect your issue is a bug, please edit your issue description to
include the BUG REPORT INFORMATION shown below. If you fail to provide this
information within 7 days, we cannot debug your issue and will close it. We
will, however, reopen it if you later provide the information.
For more information about reporting issues, see
https://github.com/moby/moby/blob/master/CONTRIBUTING.md#reporting-other-issues
---------------------------------------------------
GENERAL SUPPORT INFORMATION
---------------------------------------------------
The GitHub issue tracker is for bug reports and feature requests.
General support for **docker** can be found at the following locations:
- Docker Support Forums - https://forums.docker.com
- Slack - community.docker.com #general channel
- Post a question on StackOverflow, using the Docker tag
General support for **moby** can be found at the following locations:
- Moby Project Forums - https://forums.mobyproject.org
- Slack - community.docker.com #moby-project channel
- Post a question on StackOverflow, using the Moby tag
---------------------------------------------------
BUG REPORT INFORMATION
---------------------------------------------------
Use the commands below to provide key information from your environment:
You do NOT have to include this information if this is a FEATURE REQUEST
-->
**Description**
<!--
Briefly describe the problem you are having in a few paragraphs.
-->
**Steps to reproduce the issue:**
1.
2.
3.
**Describe the results you received:**
**Describe the results you expected:**
**Additional information you deem important (e.g. issue happens only occasionally):**
**Output of `docker version`:**
```
(paste your output here)
```
**Output of `docker info`:**
```
(paste your output here)
```
**Additional environment details (AWS, VirtualBox, physical, etc.):**

View file

@ -1,146 +0,0 @@
name: Bug report
description: Create a report to help us improve
labels:
- kind/bug
- status/0-triage
body:
- type: markdown
attributes:
value: |
Thank you for taking the time to report a bug!
If this is a security issue please report it to the [Docker Security team](mailto:security@docker.com).
- type: textarea
id: description
attributes:
label: Description
description: Please give a clear and concise description of the bug
validations:
required: true
- type: textarea
id: repro
attributes:
label: Reproduce
description: Steps to reproduce the bug
placeholder: |
1. docker run ...
2. docker kill ...
3. docker rm ...
validations:
required: true
- type: textarea
id: expected
attributes:
label: Expected behavior
description: What is the expected behavior?
placeholder: |
E.g. "`docker rm` should remove the container and cleanup all associated data"
- type: textarea
id: version
attributes:
label: docker version
description: Output of `docker version`
render: bash
placeholder: |
Client:
Version: 20.10.17
API version: 1.41
Go version: go1.17.11
Git commit: 100c70180fde3601def79a59cc3e996aa553c9b9
Built: Mon Jun 6 21:36:39 UTC 2022
OS/Arch: linux/amd64
Context: default
Experimental: true
Server:
Engine:
Version: 20.10.17
API version: 1.41 (minimum version 1.12)
Go version: go1.17.11
Git commit: a89b84221c8560e7a3dee2a653353429e7628424
Built: Mon Jun 6 22:32:38 2022
OS/Arch: linux/amd64
Experimental: true
containerd:
Version: 1.6.6
GitCommit: 10c12954828e7c7c9b6e0ea9b0c02b01407d3ae1
runc:
Version: 1.1.2
GitCommit: a916309fff0f838eb94e928713dbc3c0d0ac7aa4
docker-init:
Version: 0.19.0
GitCommit:
validations:
required: true
- type: textarea
id: info
attributes:
label: docker info
description: Output of `docker info`
render: bash
placeholder: |
Client:
Context: default
Debug Mode: false
Plugins:
buildx: Docker Buildx (Docker Inc., 0.8.2)
compose: Docker Compose (Docker Inc., 2.6.0)
Server:
Containers: 4
Running: 2
Paused: 0
Stopped: 2
Images: 80
Server Version: 20.10.17
Storage Driver: overlay2
Backing Filesystem: xfs
Supports d_type: true
Native Overlay Diff: false
userxattr: false
Logging Driver: local
Cgroup Driver: cgroupfs
Cgroup Version: 1
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: runc io.containerd.runc.v2 io.containerd.runtime.v1.linux
Default Runtime: runc
Init Binary: docker-init
containerd version: 10c12954828e7c7c9b6e0ea9b0c02b01407d3ae1
runc version: a916309fff0f838eb94e928713dbc3c0d0ac7aa4
init version:
Security Options:
apparmor
seccomp
Profile: default
Kernel Version: 5.13.0-1031-azure
Operating System: Ubuntu 20.04.4 LTS
OSType: linux
Architecture: x86_64
CPUs: 4
Total Memory: 15.63GiB
Name: dev
ID: UC44:2RFL:7NQ5:GGFW:34O5:DYRE:CLOH:VLGZ:64AZ:GFXC:PY6H:SAHY
Docker Root Dir: /var/lib/docker
Debug Mode: true
File Descriptors: 46
Goroutines: 134
System Time: 2022-07-06T18:07:54.812439392Z
EventsListeners: 0
Registry: https://index.docker.io/v1/
Labels:
Experimental: true
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: true
validations:
required: true
- type: textarea
id: additional
attributes:
label: Additional Info
description: Additional info you want to provide such as logs, system info, environment, etc.
validations:
required: false

View file

@ -1,8 +0,0 @@
blank_issues_enabled: false
contact_links:
- name: Security and Vulnerabilities
url: https://github.com/moby/moby/blob/master/SECURITY.md
about: Please report any security issues or vulnerabilities responsibly to the Docker security team. Please do not use the public issue tracker.
- name: Questions and Discussions
url: https://github.com/moby/moby/discussions/new
about: Use Github Discussions to ask questions and/or open discussion topics.

View file

@ -1,13 +0,0 @@
name: Feature request
description: Missing functionality? Come tell us about it!
labels:
- kind/feature
- status/0-triage
body:
- type: textarea
id: description
attributes:
label: Description
description: What is the feature you want to see?
validations:
required: true

View file

@ -22,12 +22,9 @@ Please provide the following information:
**- Description for the changelog**
<!--
Write a short (one line) summary that describes the changes in this
pull request for inclusion in the changelog.
It must be placed inside the below triple backticks section:
pull request for inclusion in the changelog:
-->
```markdown changelog
```
**- A picture of a cute animal (not mandatory but encouraged)**

View file

@ -1,27 +0,0 @@
name: 'Setup Runner'
description: 'Composite action to set up the GitHub Runner for jobs in the test.yml workflow'
runs:
using: composite
steps:
- run: |
sudo modprobe ip_vs
sudo modprobe ipv6
sudo modprobe ip6table_filter
sudo modprobe -r overlay
sudo modprobe overlay redirect_dir=off
shell: bash
- run: |
if [ ! -e /etc/docker/daemon.json ]; then
echo '{}' | sudo tee /etc/docker/daemon.json >/dev/null
fi
DOCKERD_CONFIG=$(jq '.+{"experimental":true,"live-restore":true,"ipv6":true,"fixed-cidr-v6":"2001:db8:1::/64"}' /etc/docker/daemon.json)
sudo tee /etc/docker/daemon.json <<<"$DOCKERD_CONFIG" >/dev/null
sudo service docker restart
shell: bash
- run: |
./contrib/check-config.sh || true
shell: bash
- run: |
docker info
shell: bash

View file

@ -1,14 +0,0 @@
name: 'Setup Tracing'
description: 'Composite action to set up the tracing for test jobs'
runs:
using: composite
steps:
- run: |
set -e
# Jaeger is set up on Windows through an inline run step. If you update Jaeger here, don't forget to update
# the version set in .github/workflows/.windows.yml.
docker run -d --net=host --name jaeger -e COLLECTOR_OTLP_ENABLED=true jaegertracing/all-in-one:1.46
docker0_ip="$(ip -f inet addr show docker0 | grep -Po 'inet \K[\d.]+')"
echo "OTEL_EXPORTER_OTLP_ENDPOINT=http://${docker0_ip}:4318" >> "${GITHUB_ENV}"
shell: bash

View file

@ -1,48 +0,0 @@
# reusable workflow
name: .dco
# TODO: hide reusable workflow from the UI. Tracked in https://github.com/community/community/discussions/12025
on:
workflow_call:
env:
ALPINE_VERSION: 3.16
jobs:
run:
runs-on: ubuntu-20.04
steps:
-
name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 0
-
name: Dump context
uses: actions/github-script@v7
with:
script: |
console.log(JSON.stringify(context, null, 2));
-
name: Get base ref
id: base-ref
uses: actions/github-script@v7
with:
result-encoding: string
script: |
if (/^refs\/pull\//.test(context.ref) && context.payload?.pull_request?.base?.ref != undefined) {
return context.payload.pull_request.base.ref;
}
return context.ref.replace(/^refs\/heads\//g, '');
-
name: Validate
run: |
docker run --rm \
-v "$(pwd):/workspace" \
-e VALIDATE_REPO \
-e VALIDATE_BRANCH \
alpine:${{ env.ALPINE_VERSION }} sh -c 'apk add --no-cache -q bash git openssh-client && git config --system --add safe.directory /workspace && cd /workspace && hack/validate/dco'
env:
VALIDATE_REPO: ${{ github.server_url }}/${{ github.repository }}.git
VALIDATE_BRANCH: ${{ steps.base-ref.outputs.result }}

View file

@ -1,35 +0,0 @@
# reusable workflow
name: .test-prepare
# TODO: hide reusable workflow from the UI. Tracked in https://github.com/community/community/discussions/12025
on:
workflow_call:
outputs:
matrix:
description: Test matrix
value: ${{ jobs.run.outputs.matrix }}
jobs:
run:
runs-on: ubuntu-20.04
outputs:
matrix: ${{ steps.set.outputs.matrix }}
steps:
-
name: Checkout
uses: actions/checkout@v4
-
name: Create matrix
id: set
uses: actions/github-script@v7
with:
script: |
let matrix = ['graphdriver'];
if ("${{ contains(github.event.pull_request.labels.*.name, 'containerd-integration') || github.event_name != 'pull_request' }}" == "true") {
matrix.push('snapshotter');
}
await core.group(`Set matrix`, async () => {
core.info(`matrix: ${JSON.stringify(matrix)}`);
core.setOutput('matrix', JSON.stringify(matrix));
});

View file

@ -1,445 +0,0 @@
# reusable workflow
name: .test
# TODO: hide reusable workflow from the UI. Tracked in https://github.com/community/community/discussions/12025
on:
workflow_call:
inputs:
storage:
required: true
type: string
default: "graphdriver"
env:
GO_VERSION: "1.21.9"
GOTESTLIST_VERSION: v0.3.1
TESTSTAT_VERSION: v0.1.25
ITG_CLI_MATRIX_SIZE: 6
DOCKER_EXPERIMENTAL: 1
DOCKER_GRAPHDRIVER: ${{ inputs.storage == 'snapshotter' && 'overlayfs' || 'overlay2' }}
TEST_INTEGRATION_USE_SNAPSHOTTER: ${{ inputs.storage == 'snapshotter' && '1' || '' }}
jobs:
unit:
runs-on: ubuntu-20.04
continue-on-error: ${{ github.event_name != 'pull_request' }}
timeout-minutes: 120
steps:
-
name: Checkout
uses: actions/checkout@v4
-
name: Set up runner
uses: ./.github/actions/setup-runner
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
-
name: Build dev image
uses: docker/bake-action@v4
with:
targets: dev
set: |
dev.cache-from=type=gha,scope=dev
-
name: Test
run: |
make -o build test-unit
-
name: Prepare reports
if: always()
run: |
mkdir -p bundles /tmp/reports
find bundles -path '*/root/*overlay2' -prune -o -type f \( -name '*-report.json' -o -name '*.log' -o -name '*.out' -o -name '*.prof' -o -name '*-report.xml' \) -print | xargs sudo tar -czf /tmp/reports.tar.gz
tar -xzf /tmp/reports.tar.gz -C /tmp/reports
sudo chown -R $(id -u):$(id -g) /tmp/reports
tree -nh /tmp/reports
-
name: Send to Codecov
uses: codecov/codecov-action@v4
with:
directory: ./bundles
env_vars: RUNNER_OS
flags: unit
token: ${{ secrets.CODECOV_TOKEN }} # used to upload coverage reports: https://github.com/moby/buildkit/pull/4660#issue-2142122533
-
name: Upload reports
if: always()
uses: actions/upload-artifact@v4
with:
name: test-reports-unit-${{ inputs.storage }}
path: /tmp/reports/*
retention-days: 1
unit-report:
runs-on: ubuntu-20.04
continue-on-error: ${{ github.event_name != 'pull_request' }}
timeout-minutes: 10
if: always()
needs:
- unit
steps:
-
name: Set up Go
uses: actions/setup-go@v5
with:
go-version: ${{ env.GO_VERSION }}
-
name: Download reports
uses: actions/download-artifact@v4
with:
name: test-reports-unit-${{ inputs.storage }}
path: /tmp/reports
-
name: Install teststat
run: |
go install github.com/vearutop/teststat@${{ env.TESTSTAT_VERSION }}
-
name: Create summary
run: |
find /tmp/reports -type f -name '*-go-test-report.json' -exec teststat -markdown {} \+ >> $GITHUB_STEP_SUMMARY
docker-py:
runs-on: ubuntu-20.04
continue-on-error: ${{ github.event_name != 'pull_request' }}
timeout-minutes: 120
steps:
-
name: Checkout
uses: actions/checkout@v4
-
name: Set up runner
uses: ./.github/actions/setup-runner
-
name: Set up tracing
uses: ./.github/actions/setup-tracing
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
-
name: Build dev image
uses: docker/bake-action@v4
with:
targets: dev
set: |
dev.cache-from=type=gha,scope=dev
-
name: Test
run: |
make -o build test-docker-py
-
name: Prepare reports
if: always()
run: |
mkdir -p bundles /tmp/reports
find bundles -path '*/root/*overlay2' -prune -o -type f \( -name '*-report.json' -o -name '*.log' -o -name '*.out' -o -name '*.prof' -o -name '*-report.xml' \) -print | xargs sudo tar -czf /tmp/reports.tar.gz
tar -xzf /tmp/reports.tar.gz -C /tmp/reports
sudo chown -R $(id -u):$(id -g) /tmp/reports
tree -nh /tmp/reports
curl -sSLf localhost:16686/api/traces?service=integration-test-client > /tmp/reports/jaeger-trace.json
-
name: Test daemon logs
if: always()
run: |
cat bundles/test-docker-py/docker.log
-
name: Upload reports
if: always()
uses: actions/upload-artifact@v4
with:
name: test-reports-docker-py-${{ inputs.storage }}
path: /tmp/reports/*
retention-days: 1
integration-flaky:
runs-on: ubuntu-20.04
continue-on-error: ${{ github.event_name != 'pull_request' }}
timeout-minutes: 120
steps:
-
name: Checkout
uses: actions/checkout@v4
-
name: Set up runner
uses: ./.github/actions/setup-runner
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
-
name: Build dev image
uses: docker/bake-action@v4
with:
targets: dev
set: |
dev.cache-from=type=gha,scope=dev
-
name: Test
run: |
make -o build test-integration-flaky
env:
TEST_SKIP_INTEGRATION_CLI: 1
integration:
runs-on: ${{ matrix.os }}
continue-on-error: ${{ github.event_name != 'pull_request' }}
timeout-minutes: 120
strategy:
fail-fast: false
matrix:
os:
- ubuntu-20.04
- ubuntu-22.04
mode:
- ""
- rootless
- systemd
#- rootless-systemd FIXME: https://github.com/moby/moby/issues/44084
steps:
-
name: Checkout
uses: actions/checkout@v4
-
name: Set up runner
uses: ./.github/actions/setup-runner
-
name: Set up tracing
uses: ./.github/actions/setup-tracing
-
name: Prepare
run: |
CACHE_DEV_SCOPE=dev
if [[ "${{ matrix.mode }}" == *"rootless"* ]]; then
echo "DOCKER_ROOTLESS=1" >> $GITHUB_ENV
fi
if [[ "${{ matrix.mode }}" == *"systemd"* ]]; then
echo "SYSTEMD=true" >> $GITHUB_ENV
CACHE_DEV_SCOPE="${CACHE_DEV_SCOPE}systemd"
fi
echo "CACHE_DEV_SCOPE=${CACHE_DEV_SCOPE}" >> $GITHUB_ENV
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
-
name: Build dev image
uses: docker/bake-action@v4
with:
targets: dev
set: |
dev.cache-from=type=gha,scope=${{ env.CACHE_DEV_SCOPE }}
-
name: Test
run: |
make -o build test-integration
env:
TEST_SKIP_INTEGRATION_CLI: 1
TESTCOVERAGE: 1
-
name: Prepare reports
if: always()
run: |
reportsName=${{ matrix.os }}
if [ -n "${{ matrix.mode }}" ]; then
reportsName="$reportsName-${{ matrix.mode }}"
fi
reportsPath="/tmp/reports/$reportsName"
echo "TESTREPORTS_NAME=$reportsName" >> $GITHUB_ENV
mkdir -p bundles $reportsPath
find bundles -path '*/root/*overlay2' -prune -o -type f \( -name '*-report.json' -o -name '*.log' -o -name '*.out' -o -name '*.prof' -o -name '*-report.xml' \) -print | xargs sudo tar -czf /tmp/reports.tar.gz
tar -xzf /tmp/reports.tar.gz -C $reportsPath
sudo chown -R $(id -u):$(id -g) $reportsPath
tree -nh $reportsPath
curl -sSLf localhost:16686/api/traces?service=integration-test-client > $reportsPath/jaeger-trace.json
-
name: Send to Codecov
uses: codecov/codecov-action@v4
with:
directory: ./bundles/test-integration
env_vars: RUNNER_OS
flags: integration,${{ matrix.mode }}
token: ${{ secrets.CODECOV_TOKEN }} # used to upload coverage reports: https://github.com/moby/buildkit/pull/4660#issue-2142122533
-
name: Test daemon logs
if: always()
run: |
cat bundles/test-integration/docker.log
-
name: Upload reports
if: always()
uses: actions/upload-artifact@v4
with:
name: test-reports-integration-${{ inputs.storage }}-${{ env.TESTREPORTS_NAME }}
path: /tmp/reports/*
retention-days: 1
integration-report:
runs-on: ubuntu-20.04
continue-on-error: ${{ github.event_name != 'pull_request' }}
timeout-minutes: 10
if: always()
needs:
- integration
steps:
-
name: Set up Go
uses: actions/setup-go@v5
with:
go-version: ${{ env.GO_VERSION }}
-
name: Download reports
uses: actions/download-artifact@v4
with:
path: /tmp/reports
pattern: test-reports-integration-${{ inputs.storage }}-*
merge-multiple: true
-
name: Install teststat
run: |
go install github.com/vearutop/teststat@${{ env.TESTSTAT_VERSION }}
-
name: Create summary
run: |
find /tmp/reports -type f -name '*-go-test-report.json' -exec teststat -markdown {} \+ >> $GITHUB_STEP_SUMMARY
integration-cli-prepare:
runs-on: ubuntu-20.04
continue-on-error: ${{ github.event_name != 'pull_request' }}
outputs:
matrix: ${{ steps.tests.outputs.matrix }}
steps:
-
name: Checkout
uses: actions/checkout@v4
-
name: Set up Go
uses: actions/setup-go@v5
with:
go-version: ${{ env.GO_VERSION }}
-
name: Install gotestlist
run:
go install github.com/crazy-max/gotestlist/cmd/gotestlist@${{ env.GOTESTLIST_VERSION }}
-
name: Create matrix
id: tests
working-directory: ./integration-cli
run: |
# This step creates a matrix for integration-cli tests. Tests suites
# are distributed in integration-cli job through a matrix. There is
# also overrides being added to the matrix like "./..." to run
# "Test integration" step exclusively and specific tests suites that
# take a long time to run.
matrix="$(gotestlist -d ${{ env.ITG_CLI_MATRIX_SIZE }} -o "./..." -o "DockerSwarmSuite" -o "DockerNetworkSuite|DockerExternalVolumeSuite" ./...)"
echo "matrix=$matrix" >> $GITHUB_OUTPUT
-
name: Show matrix
run: |
echo ${{ steps.tests.outputs.matrix }}
integration-cli:
runs-on: ubuntu-20.04
continue-on-error: ${{ github.event_name != 'pull_request' }}
timeout-minutes: 120
needs:
- integration-cli-prepare
strategy:
fail-fast: false
matrix:
test: ${{ fromJson(needs.integration-cli-prepare.outputs.matrix) }}
steps:
-
name: Checkout
uses: actions/checkout@v4
-
name: Set up runner
uses: ./.github/actions/setup-runner
-
name: Set up tracing
uses: ./.github/actions/setup-tracing
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
-
name: Build dev image
uses: docker/bake-action@v4
with:
targets: dev
set: |
dev.cache-from=type=gha,scope=dev
-
name: Test
run: |
make -o build test-integration
env:
TEST_SKIP_INTEGRATION: 1
TESTCOVERAGE: 1
TESTFLAGS: "-test.run (${{ matrix.test }})/"
-
name: Prepare reports
if: always()
run: |
reportsName=$(echo -n "${{ matrix.test }}" | sha256sum | cut -d " " -f 1)
reportsPath=/tmp/reports/$reportsName
echo "TESTREPORTS_NAME=$reportsName" >> $GITHUB_ENV
mkdir -p bundles $reportsPath
echo "${{ matrix.test }}" | tr -s '|' '\n' | tee -a "$reportsPath/tests.txt"
find bundles -path '*/root/*overlay2' -prune -o -type f \( -name '*-report.json' -o -name '*.log' -o -name '*.out' -o -name '*.prof' -o -name '*-report.xml' \) -print | xargs sudo tar -czf /tmp/reports.tar.gz
tar -xzf /tmp/reports.tar.gz -C $reportsPath
sudo chown -R $(id -u):$(id -g) $reportsPath
tree -nh $reportsPath
curl -sSLf localhost:16686/api/traces?service=integration-test-client > $reportsPath/jaeger-trace.json
-
name: Send to Codecov
uses: codecov/codecov-action@v4
with:
directory: ./bundles/test-integration
env_vars: RUNNER_OS
flags: integration-cli
token: ${{ secrets.CODECOV_TOKEN }} # used to upload coverage reports: https://github.com/moby/buildkit/pull/4660#issue-2142122533
-
name: Test daemon logs
if: always()
run: |
cat bundles/test-integration/docker.log
-
name: Upload reports
if: always()
uses: actions/upload-artifact@v4
with:
name: test-reports-integration-cli-${{ inputs.storage }}-${{ env.TESTREPORTS_NAME }}
path: /tmp/reports/*
retention-days: 1
integration-cli-report:
runs-on: ubuntu-20.04
continue-on-error: ${{ github.event_name != 'pull_request' }}
timeout-minutes: 10
if: always()
needs:
- integration-cli
steps:
-
name: Set up Go
uses: actions/setup-go@v5
with:
go-version: ${{ env.GO_VERSION }}
-
name: Download reports
uses: actions/download-artifact@v4
with:
path: /tmp/reports
pattern: test-reports-integration-cli-${{ inputs.storage }}-*
merge-multiple: true
-
name: Install teststat
run: |
go install github.com/vearutop/teststat@${{ env.TESTSTAT_VERSION }}
-
name: Create summary
run: |
find /tmp/reports -type f -name '*-go-test-report.json' -exec teststat -markdown {} \+ >> $GITHUB_STEP_SUMMARY

View file

@ -1,551 +0,0 @@
# reusable workflow
name: .windows
# TODO: hide reusable workflow from the UI. Tracked in https://github.com/community/community/discussions/12025
on:
workflow_call:
inputs:
os:
required: true
type: string
storage:
required: true
type: string
default: "graphdriver"
send_coverage:
required: false
type: boolean
default: false
env:
GO_VERSION: "1.21.9"
GOTESTLIST_VERSION: v0.3.1
TESTSTAT_VERSION: v0.1.25
WINDOWS_BASE_IMAGE: mcr.microsoft.com/windows/servercore
WINDOWS_BASE_TAG_2019: ltsc2019
WINDOWS_BASE_TAG_2022: ltsc2022
TEST_IMAGE_NAME: moby:test
TEST_CTN_NAME: moby
DOCKER_BUILDKIT: 0
ITG_CLI_MATRIX_SIZE: 6
jobs:
build:
runs-on: ${{ inputs.os }}
env:
GOPATH: ${{ github.workspace }}\go
GOBIN: ${{ github.workspace }}\go\bin
BIN_OUT: ${{ github.workspace }}\out
defaults:
run:
working-directory: ${{ env.GOPATH }}/src/github.com/docker/docker
steps:
-
name: Checkout
uses: actions/checkout@v4
with:
path: ${{ env.GOPATH }}/src/github.com/docker/docker
-
name: Env
run: |
Get-ChildItem Env: | Out-String
-
name: Init
run: |
New-Item -ItemType "directory" -Path "${{ github.workspace }}\go-build"
New-Item -ItemType "directory" -Path "${{ github.workspace }}\go\pkg\mod"
If ("${{ inputs.os }}" -eq "windows-2019") {
echo "WINDOWS_BASE_IMAGE_TAG=${{ env.WINDOWS_BASE_TAG_2019 }}" | Out-File -FilePath $Env:GITHUB_ENV -Encoding utf-8 -Append
} ElseIf ("${{ inputs.os }}" -eq "windows-2022") {
echo "WINDOWS_BASE_IMAGE_TAG=${{ env.WINDOWS_BASE_TAG_2022 }}" | Out-File -FilePath $Env:GITHUB_ENV -Encoding utf-8 -Append
}
-
name: Cache
uses: actions/cache@v4
with:
path: |
~\AppData\Local\go-build
~\go\pkg\mod
${{ github.workspace }}\go-build
${{ env.GOPATH }}\pkg\mod
key: ${{ inputs.os }}-${{ github.job }}-${{ hashFiles('**/vendor.sum') }}
restore-keys: |
${{ inputs.os }}-${{ github.job }}-
-
name: Docker info
run: |
docker info
-
name: Build base image
run: |
& docker build `
--build-arg WINDOWS_BASE_IMAGE `
--build-arg WINDOWS_BASE_IMAGE_TAG `
--build-arg GO_VERSION `
-t ${{ env.TEST_IMAGE_NAME }} `
-f Dockerfile.windows .
-
name: Build binaries
run: |
& docker run --name ${{ env.TEST_CTN_NAME }} -e "DOCKER_GITCOMMIT=${{ github.sha }}" `
-v "${{ github.workspace }}\go-build:C:\Users\ContainerAdministrator\AppData\Local\go-build" `
-v "${{ github.workspace }}\go\pkg\mod:C:\gopath\pkg\mod" `
${{ env.TEST_IMAGE_NAME }} hack\make.ps1 -Daemon -Client
-
name: Copy artifacts
run: |
New-Item -ItemType "directory" -Path "${{ env.BIN_OUT }}"
docker cp "${{ env.TEST_CTN_NAME }}`:c`:\gopath\src\github.com\docker\docker\bundles\docker.exe" ${{ env.BIN_OUT }}\
docker cp "${{ env.TEST_CTN_NAME }}`:c`:\gopath\src\github.com\docker\docker\bundles\dockerd.exe" ${{ env.BIN_OUT }}\
docker cp "${{ env.TEST_CTN_NAME }}`:c`:\gopath\bin\gotestsum.exe" ${{ env.BIN_OUT }}\
docker cp "${{ env.TEST_CTN_NAME }}`:c`:\containerd\bin\containerd.exe" ${{ env.BIN_OUT }}\
docker cp "${{ env.TEST_CTN_NAME }}`:c`:\containerd\bin\containerd-shim-runhcs-v1.exe" ${{ env.BIN_OUT }}\
-
name: Upload artifacts
uses: actions/upload-artifact@v4
with:
name: build-${{ inputs.storage }}-${{ inputs.os }}
path: ${{ env.BIN_OUT }}/*
if-no-files-found: error
retention-days: 2
unit-test:
runs-on: ${{ inputs.os }}
timeout-minutes: 120
env:
GOPATH: ${{ github.workspace }}\go
GOBIN: ${{ github.workspace }}\go\bin
defaults:
run:
working-directory: ${{ env.GOPATH }}/src/github.com/docker/docker
steps:
-
name: Checkout
uses: actions/checkout@v4
with:
path: ${{ env.GOPATH }}/src/github.com/docker/docker
-
name: Env
run: |
Get-ChildItem Env: | Out-String
-
name: Init
run: |
New-Item -ItemType "directory" -Path "${{ github.workspace }}\go-build"
New-Item -ItemType "directory" -Path "${{ github.workspace }}\go\pkg\mod"
New-Item -ItemType "directory" -Path "bundles"
If ("${{ inputs.os }}" -eq "windows-2019") {
echo "WINDOWS_BASE_IMAGE_TAG=${{ env.WINDOWS_BASE_TAG_2019 }}" | Out-File -FilePath $Env:GITHUB_ENV -Encoding utf-8 -Append
} ElseIf ("${{ inputs.os }}" -eq "windows-2022") {
echo "WINDOWS_BASE_IMAGE_TAG=${{ env.WINDOWS_BASE_TAG_2022 }}" | Out-File -FilePath $Env:GITHUB_ENV -Encoding utf-8 -Append
}
-
name: Cache
uses: actions/cache@v4
with:
path: |
~\AppData\Local\go-build
~\go\pkg\mod
${{ github.workspace }}\go-build
${{ env.GOPATH }}\pkg\mod
key: ${{ inputs.os }}-${{ github.job }}-${{ hashFiles('**/vendor.sum') }}
restore-keys: |
${{ inputs.os }}-${{ github.job }}-
-
name: Docker info
run: |
docker info
-
name: Build base image
run: |
& docker build `
--build-arg WINDOWS_BASE_IMAGE `
--build-arg WINDOWS_BASE_IMAGE_TAG `
--build-arg GO_VERSION `
-t ${{ env.TEST_IMAGE_NAME }} `
-f Dockerfile.windows .
-
name: Test
run: |
& docker run --name ${{ env.TEST_CTN_NAME }} -e "DOCKER_GITCOMMIT=${{ github.sha }}" `
-v "${{ github.workspace }}\go-build:C:\Users\ContainerAdministrator\AppData\Local\go-build" `
-v "${{ github.workspace }}\go\pkg\mod:C:\gopath\pkg\mod" `
-v "${{ env.GOPATH }}\src\github.com\docker\docker\bundles:C:\gopath\src\github.com\docker\docker\bundles" `
${{ env.TEST_IMAGE_NAME }} hack\make.ps1 -TestUnit
-
name: Send to Codecov
if: inputs.send_coverage
uses: codecov/codecov-action@v4
with:
working-directory: ${{ env.GOPATH }}\src\github.com\docker\docker
directory: bundles
env_vars: RUNNER_OS
flags: unit
token: ${{ secrets.CODECOV_TOKEN }} # used to upload coverage reports: https://github.com/moby/buildkit/pull/4660#issue-2142122533
-
name: Upload reports
if: always()
uses: actions/upload-artifact@v4
with:
name: ${{ inputs.os }}-${{ inputs.storage }}-unit-reports
path: ${{ env.GOPATH }}\src\github.com\docker\docker\bundles\*
retention-days: 1
unit-test-report:
runs-on: ubuntu-latest
if: always()
needs:
- unit-test
steps:
-
name: Set up Go
uses: actions/setup-go@v5
with:
go-version: ${{ env.GO_VERSION }}
-
name: Download artifacts
uses: actions/download-artifact@v4
with:
name: ${{ inputs.os }}-${{ inputs.storage }}-unit-reports
path: /tmp/artifacts
-
name: Install teststat
run: |
go install github.com/vearutop/teststat@${{ env.TESTSTAT_VERSION }}
-
name: Create summary
run: |
find /tmp/artifacts -type f -name '*-go-test-report.json' -exec teststat -markdown {} \+ >> $GITHUB_STEP_SUMMARY
integration-test-prepare:
runs-on: ubuntu-latest
outputs:
matrix: ${{ steps.tests.outputs.matrix }}
steps:
-
name: Checkout
uses: actions/checkout@v4
-
name: Set up Go
uses: actions/setup-go@v5
with:
go-version: ${{ env.GO_VERSION }}
-
name: Install gotestlist
run:
go install github.com/crazy-max/gotestlist/cmd/gotestlist@${{ env.GOTESTLIST_VERSION }}
-
name: Create matrix
id: tests
working-directory: ./integration-cli
run: |
# This step creates a matrix for integration-cli tests. Tests suites
# are distributed in integration-test job through a matrix. There is
# also an override being added to the matrix like "./..." to run
# "Test integration" step exclusively.
matrix="$(gotestlist -d ${{ env.ITG_CLI_MATRIX_SIZE }} -o "./..." ./...)"
echo "matrix=$matrix" >> $GITHUB_OUTPUT
-
name: Show matrix
run: |
echo ${{ steps.tests.outputs.matrix }}
integration-test:
runs-on: ${{ inputs.os }}
continue-on-error: ${{ inputs.storage == 'snapshotter' && github.event_name != 'pull_request' }}
timeout-minutes: 120
needs:
- build
- integration-test-prepare
strategy:
fail-fast: false
matrix:
storage:
- ${{ inputs.storage }}
runtime:
- builtin
- containerd
test: ${{ fromJson(needs.integration-test-prepare.outputs.matrix) }}
exclude:
- storage: snapshotter
runtime: builtin
env:
GOPATH: ${{ github.workspace }}\go
GOBIN: ${{ github.workspace }}\go\bin
BIN_OUT: ${{ github.workspace }}\out
defaults:
run:
working-directory: ${{ env.GOPATH }}/src/github.com/docker/docker
steps:
-
name: Checkout
uses: actions/checkout@v4
with:
path: ${{ env.GOPATH }}/src/github.com/docker/docker
-
name: Set up Jaeger
run: |
# Jaeger is set up on Linux through the setup-tracing action. If you update Jaeger here, don't forget to
# update the version set in .github/actions/setup-tracing/action.yml.
Invoke-WebRequest -Uri "https://github.com/jaegertracing/jaeger/releases/download/v1.46.0/jaeger-1.46.0-windows-amd64.tar.gz" -OutFile ".\jaeger-1.46.0-windows-amd64.tar.gz"
tar -zxvf ".\jaeger-1.46.0-windows-amd64.tar.gz"
Start-Process '.\jaeger-1.46.0-windows-amd64\jaeger-all-in-one.exe'
echo "OTEL_EXPORTER_OTLP_ENDPOINT=http://127.0.0.1:4318" | Out-File -FilePath $Env:GITHUB_ENV -Encoding utf-8 -Append
shell: pwsh
-
name: Env
run: |
Get-ChildItem Env: | Out-String
-
name: Download artifacts
uses: actions/download-artifact@v4
with:
name: build-${{ inputs.storage }}-${{ inputs.os }}
path: ${{ env.BIN_OUT }}
-
name: Init
run: |
New-Item -ItemType "directory" -Path "bundles"
If ("${{ inputs.os }}" -eq "windows-2019") {
echo "WINDOWS_BASE_IMAGE_TAG=${{ env.WINDOWS_BASE_TAG_2019 }}" | Out-File -FilePath $Env:GITHUB_ENV -Encoding utf-8 -Append
} ElseIf ("${{ inputs.os }}" -eq "windows-2022") {
echo "WINDOWS_BASE_IMAGE_TAG=${{ env.WINDOWS_BASE_TAG_2022 }}" | Out-File -FilePath $Env:GITHUB_ENV -Encoding utf-8 -Append
}
Write-Output "${{ env.BIN_OUT }}" | Out-File -FilePath $env:GITHUB_PATH -Encoding utf8 -Append
$testName = ([System.BitConverter]::ToString((New-Object System.Security.Cryptography.SHA256Managed).ComputeHash([System.Text.Encoding]::UTF8.GetBytes("${{ matrix.test }}"))) -replace '-').ToLower()
echo "TESTREPORTS_NAME=$testName" | Out-File -FilePath $Env:GITHUB_ENV -Encoding utf-8 -Append
-
# removes docker service that is currently installed on the runner. we
# could use Uninstall-Package but not yet available on Windows runners.
# more info: https://github.com/actions/virtual-environments/blob/d3a5bad25f3b4326c5666bab0011ac7f1beec95e/images/win/scripts/Installers/Install-Docker.ps1#L11
name: Removing current daemon
run: |
if (Get-Service docker -ErrorAction SilentlyContinue) {
$dockerVersion = (docker version -f "{{.Server.Version}}")
Write-Host "Current installed Docker version: $dockerVersion"
# remove service
Stop-Service -Force -Name docker
Remove-Service -Name docker
# removes event log entry. we could use "Remove-EventLog -LogName -Source docker"
# but this cmd is not available atm
$ErrorActionPreference = "SilentlyContinue"
& reg delete "HKLM\SYSTEM\CurrentControlSet\Services\EventLog\Application\docker" /f 2>&1 | Out-Null
$ErrorActionPreference = "Stop"
Write-Host "Service removed"
}
-
name: Starting containerd
if: matrix.runtime == 'containerd'
run: |
Write-Host "Generating config"
& "${{ env.BIN_OUT }}\containerd.exe" config default | Out-File "$env:TEMP\ctn.toml" -Encoding ascii
Write-Host "Creating service"
New-Item -ItemType Directory "$env:TEMP\ctn-root" -ErrorAction SilentlyContinue | Out-Null
New-Item -ItemType Directory "$env:TEMP\ctn-state" -ErrorAction SilentlyContinue | Out-Null
Start-Process -Wait "${{ env.BIN_OUT }}\containerd.exe" `
-ArgumentList "--log-level=debug", `
"--config=$env:TEMP\ctn.toml", `
"--address=\\.\pipe\containerd-containerd", `
"--root=$env:TEMP\ctn-root", `
"--state=$env:TEMP\ctn-state", `
"--log-file=$env:TEMP\ctn.log", `
"--register-service"
Write-Host "Starting service"
Start-Service -Name containerd
Start-Sleep -Seconds 5
Write-Host "Service started successfully!"
-
name: Starting test daemon
run: |
Write-Host "Creating service"
If ("${{ matrix.runtime }}" -eq "containerd") {
$runtimeArg="--containerd=\\.\pipe\containerd-containerd"
echo "DOCKER_WINDOWS_CONTAINERD_RUNTIME=1" | Out-File -FilePath $Env:GITHUB_ENV -Encoding utf-8 -Append
}
New-Item -ItemType Directory "$env:TEMP\moby-root" -ErrorAction SilentlyContinue | Out-Null
New-Item -ItemType Directory "$env:TEMP\moby-exec" -ErrorAction SilentlyContinue | Out-Null
Start-Process -Wait -NoNewWindow "${{ env.BIN_OUT }}\dockerd" `
-ArgumentList $runtimeArg, "--debug", `
"--host=npipe:////./pipe/docker_engine", `
"--data-root=$env:TEMP\moby-root", `
"--exec-root=$env:TEMP\moby-exec", `
"--pidfile=$env:TEMP\docker.pid", `
"--register-service"
If ("${{ inputs.storage }}" -eq "snapshotter") {
# Make the env-var visible to the service-managed dockerd, as there's no CLI flag for this option.
& reg add "HKLM\SYSTEM\CurrentControlSet\Services\docker" /v Environment /t REG_MULTI_SZ /s '@' /d TEST_INTEGRATION_USE_SNAPSHOTTER=1
echo "TEST_INTEGRATION_USE_SNAPSHOTTER=1" | Out-File -FilePath $Env:GITHUB_ENV -Encoding utf-8 -Append
}
Write-Host "Starting service"
Start-Service -Name docker
Write-Host "Service started successfully!"
-
name: Waiting for test daemon to start
run: |
$tries=20
Write-Host "Waiting for the test daemon to start..."
While ($true) {
$ErrorActionPreference = "SilentlyContinue"
& "${{ env.BIN_OUT }}\docker" version
$ErrorActionPreference = "Stop"
If ($LastExitCode -eq 0) {
break
}
$tries--
If ($tries -le 0) {
Throw "Failed to get a response from the daemon"
}
Write-Host -NoNewline "."
Start-Sleep -Seconds 1
}
Write-Host "Test daemon started and replied!"
env:
DOCKER_HOST: npipe:////./pipe/docker_engine
-
name: Docker info
run: |
& "${{ env.BIN_OUT }}\docker" info
env:
DOCKER_HOST: npipe:////./pipe/docker_engine
-
name: Building contrib/busybox
run: |
& "${{ env.BIN_OUT }}\docker" build -t busybox `
--build-arg WINDOWS_BASE_IMAGE `
--build-arg WINDOWS_BASE_IMAGE_TAG `
.\contrib\busybox\
env:
DOCKER_HOST: npipe:////./pipe/docker_engine
-
name: List images
run: |
& "${{ env.BIN_OUT }}\docker" images
env:
DOCKER_HOST: npipe:////./pipe/docker_engine
-
name: Set up Go
uses: actions/setup-go@v5
with:
go-version: ${{ env.GO_VERSION }}
-
name: Test integration
if: matrix.test == './...'
run: |
.\hack\make.ps1 -TestIntegration
env:
DOCKER_HOST: npipe:////./pipe/docker_engine
GO111MODULE: "off"
TEST_CLIENT_BINARY: ${{ env.BIN_OUT }}\docker
-
name: Test integration-cli
if: matrix.test != './...'
run: |
.\hack\make.ps1 -TestIntegrationCli
env:
DOCKER_HOST: npipe:////./pipe/docker_engine
GO111MODULE: "off"
TEST_CLIENT_BINARY: ${{ env.BIN_OUT }}\docker
INTEGRATION_TESTRUN: ${{ matrix.test }}
-
name: Send to Codecov
if: inputs.send_coverage
uses: codecov/codecov-action@v4
with:
working-directory: ${{ env.GOPATH }}\src\github.com\docker\docker
directory: bundles
env_vars: RUNNER_OS
flags: integration,${{ matrix.runtime }}
token: ${{ secrets.CODECOV_TOKEN }} # used to upload coverage reports: https://github.com/moby/buildkit/pull/4660#issue-2142122533
-
name: Docker info
run: |
& "${{ env.BIN_OUT }}\docker" info
env:
DOCKER_HOST: npipe:////./pipe/docker_engine
-
name: Stop containerd
if: always() && matrix.runtime == 'containerd'
run: |
$ErrorActionPreference = "SilentlyContinue"
Stop-Service -Force -Name containerd
$ErrorActionPreference = "Stop"
-
name: Containerd logs
if: always() && matrix.runtime == 'containerd'
run: |
Copy-Item "$env:TEMP\ctn.log" -Destination ".\bundles\containerd.log"
Get-Content "$env:TEMP\ctn.log" | Out-Host
-
name: Stop daemon
if: always()
run: |
$ErrorActionPreference = "SilentlyContinue"
Stop-Service -Force -Name docker
$ErrorActionPreference = "Stop"
-
# as the daemon is registered as a service we have to check the event
# logs against the docker provider.
name: Daemon event logs
if: always()
run: |
Get-WinEvent -ea SilentlyContinue `
-FilterHashtable @{ProviderName= "docker"; LogName = "application"} |
Sort-Object @{Expression="TimeCreated";Descending=$false} |
ForEach-Object {"$($_.TimeCreated.ToUniversalTime().ToString("o")) [$($_.LevelDisplayName)] $($_.Message)"} |
Tee-Object -file ".\bundles\daemon.log"
-
name: Download Jaeger traces
if: always()
run: |
Invoke-WebRequest `
-Uri "http://127.0.0.1:16686/api/traces?service=integration-test-client" `
-OutFile ".\bundles\jaeger-trace.json"
-
name: Upload reports
if: always()
uses: actions/upload-artifact@v4
with:
name: ${{ inputs.os }}-${{ inputs.storage }}-integration-reports-${{ matrix.runtime }}-${{ env.TESTREPORTS_NAME }}
path: ${{ env.GOPATH }}\src\github.com\docker\docker\bundles\*
retention-days: 1
integration-test-report:
runs-on: ubuntu-latest
continue-on-error: ${{ inputs.storage == 'snapshotter' && github.event_name != 'pull_request' }}
if: always()
needs:
- integration-test
strategy:
fail-fast: false
matrix:
storage:
- ${{ inputs.storage }}
runtime:
- builtin
- containerd
exclude:
- storage: snapshotter
runtime: builtin
steps:
-
name: Set up Go
uses: actions/setup-go@v5
with:
go-version: ${{ env.GO_VERSION }}
-
name: Download reports
uses: actions/download-artifact@v4
with:
path: /tmp/reports
pattern: ${{ inputs.os }}-${{ inputs.storage }}-integration-reports-${{ matrix.runtime }}-*
merge-multiple: true
-
name: Install teststat
run: |
go install github.com/vearutop/teststat@${{ env.TESTSTAT_VERSION }}
-
name: Create summary
run: |
find /tmp/reports -type f -name '*-go-test-report.json' -exec teststat -markdown {} \+ >> $GITHUB_STEP_SUMMARY

View file

@ -1,191 +0,0 @@
name: bin-image
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
on:
workflow_dispatch:
push:
branches:
- 'master'
- '[0-9]+.[0-9]+'
tags:
- 'v*'
pull_request:
env:
MOBYBIN_REPO_SLUG: moby/moby-bin
DOCKER_GITCOMMIT: ${{ github.sha }}
VERSION: ${{ github.ref }}
PLATFORM: Moby Engine - Nightly
PRODUCT: moby-bin
PACKAGER_NAME: The Moby Project
jobs:
validate-dco:
if: ${{ !startsWith(github.ref, 'refs/tags/v') }}
uses: ./.github/workflows/.dco.yml
prepare:
runs-on: ubuntu-20.04
outputs:
platforms: ${{ steps.platforms.outputs.matrix }}
steps:
-
name: Checkout
uses: actions/checkout@v4
-
name: Docker meta
id: meta
uses: docker/metadata-action@v5
with:
images: |
${{ env.MOBYBIN_REPO_SLUG }}
### versioning strategy
## push semver tag v23.0.0
# moby/moby-bin:23.0.0
# moby/moby-bin:latest
## push semver prelease tag v23.0.0-beta.1
# moby/moby-bin:23.0.0-beta.1
## push on master
# moby/moby-bin:master
## push on 23.0 branch
# moby/moby-bin:23.0
## any push
# moby/moby-bin:sha-ad132f5
tags: |
type=semver,pattern={{version}}
type=ref,event=branch
type=ref,event=pr
type=sha
-
name: Rename meta bake definition file
# see https://github.com/docker/metadata-action/issues/381#issuecomment-1918607161
run: |
bakeFile="${{ steps.meta.outputs.bake-file }}"
mv "${bakeFile#cwd://}" "/tmp/bake-meta.json"
-
name: Upload meta bake definition
uses: actions/upload-artifact@v4
with:
name: bake-meta
path: /tmp/bake-meta.json
if-no-files-found: error
retention-days: 1
-
name: Create platforms matrix
id: platforms
run: |
echo "matrix=$(docker buildx bake bin-image-cross --print | jq -cr '.target."bin-image-cross".platforms')" >>${GITHUB_OUTPUT}
build:
runs-on: ubuntu-20.04
needs:
- validate-dco
- prepare
if: always() && !contains(needs.*.result, 'failure') && !contains(needs.*.result, 'cancelled')
strategy:
fail-fast: false
matrix:
platform: ${{ fromJson(needs.prepare.outputs.platforms) }}
steps:
-
name: Prepare
run: |
platform=${{ matrix.platform }}
echo "PLATFORM_PAIR=${platform//\//-}" >> $GITHUB_ENV
-
name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 0
-
name: Download meta bake definition
uses: actions/download-artifact@v4
with:
name: bake-meta
path: /tmp
-
name: Set up QEMU
uses: docker/setup-qemu-action@v3
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
-
name: Login to Docker Hub
if: github.event_name != 'pull_request' && github.repository == 'moby/moby'
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKERHUB_MOBYBIN_USERNAME }}
password: ${{ secrets.DOCKERHUB_MOBYBIN_TOKEN }}
-
name: Build
id: bake
uses: docker/bake-action@v4
with:
files: |
./docker-bake.hcl
/tmp/bake-meta.json
targets: bin-image
set: |
*.platform=${{ matrix.platform }}
*.output=type=image,name=${{ env.MOBYBIN_REPO_SLUG }},push-by-digest=true,name-canonical=true,push=${{ github.event_name != 'pull_request' && github.repository == 'moby/moby' }}
*.tags=
-
name: Export digest
if: github.event_name != 'pull_request' && github.repository == 'moby/moby'
run: |
mkdir -p /tmp/digests
digest="${{ fromJSON(steps.bake.outputs.metadata)['bin-image']['containerimage.digest'] }}"
touch "/tmp/digests/${digest#sha256:}"
-
name: Upload digest
if: github.event_name != 'pull_request' && github.repository == 'moby/moby'
uses: actions/upload-artifact@v4
with:
name: digests-${{ env.PLATFORM_PAIR }}
path: /tmp/digests/*
if-no-files-found: error
retention-days: 1
merge:
runs-on: ubuntu-20.04
needs:
- build
if: always() && !contains(needs.*.result, 'failure') && !contains(needs.*.result, 'cancelled') && github.event_name != 'pull_request' && github.repository == 'moby/moby'
steps:
-
name: Download meta bake definition
uses: actions/download-artifact@v4
with:
name: bake-meta
path: /tmp
-
name: Download digests
uses: actions/download-artifact@v4
with:
path: /tmp/digests
pattern: digests-*
merge-multiple: true
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
-
name: Login to Docker Hub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKERHUB_MOBYBIN_USERNAME }}
password: ${{ secrets.DOCKERHUB_MOBYBIN_TOKEN }}
-
name: Create manifest list and push
working-directory: /tmp/digests
run: |
set -x
docker buildx imagetools create $(jq -cr '.target."docker-metadata-action".tags | map("-t " + .) | join(" ")' /tmp/bake-meta.json) \
$(printf '${{ env.MOBYBIN_REPO_SLUG }}@sha256:%s ' *)
-
name: Inspect image
run: |
set -x
docker buildx imagetools inspect ${{ env.MOBYBIN_REPO_SLUG }}:$(jq -cr '.target."docker-metadata-action".args.DOCKER_META_VERSION' /tmp/bake-meta.json)

View file

@ -1,139 +0,0 @@
name: buildkit
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
on:
workflow_dispatch:
push:
branches:
- 'master'
- '[0-9]+.[0-9]+'
pull_request:
env:
GO_VERSION: "1.21.9"
DESTDIR: ./build
jobs:
validate-dco:
uses: ./.github/workflows/.dco.yml
build:
runs-on: ubuntu-20.04
needs:
- validate-dco
steps:
-
name: Checkout
uses: actions/checkout@v4
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
-
name: Build
uses: docker/bake-action@v4
with:
targets: binary
-
name: Upload artifacts
uses: actions/upload-artifact@v4
with:
name: binary
path: ${{ env.DESTDIR }}
if-no-files-found: error
retention-days: 1
test:
runs-on: ubuntu-20.04
timeout-minutes: 120
needs:
- build
env:
TEST_IMAGE_BUILD: "0"
TEST_IMAGE_ID: "buildkit-tests"
strategy:
fail-fast: false
matrix:
worker:
- dockerd
- dockerd-containerd
pkg:
- client
- cmd/buildctl
- solver
- frontend
- frontend/dockerfile
typ:
- integration
steps:
-
name: Prepare
run: |
disabledFeatures="cache_backend_azblob,cache_backend_s3"
if [ "${{ matrix.worker }}" = "dockerd" ]; then
disabledFeatures="${disabledFeatures},merge_diff"
fi
echo "BUILDKIT_TEST_DISABLE_FEATURES=${disabledFeatures}" >> $GITHUB_ENV
# Expose `ACTIONS_RUNTIME_TOKEN` and `ACTIONS_CACHE_URL`, which is used
# in BuildKit's test suite to skip/unskip cache exporters:
# https://github.com/moby/buildkit/blob/567a99433ca23402d5e9b9f9124005d2e59b8861/client/client_test.go#L5407-L5411
-
name: Expose GitHub Runtime
uses: crazy-max/ghaction-github-runtime@v3
-
name: Checkout
uses: actions/checkout@v4
with:
path: moby
-
name: BuildKit ref
run: |
echo "$(./hack/buildkit-ref)" >> $GITHUB_ENV
working-directory: moby
-
name: Checkout BuildKit ${{ env.BUILDKIT_REF }}
uses: actions/checkout@v4
with:
repository: ${{ env.BUILDKIT_REPO }}
ref: ${{ env.BUILDKIT_REF }}
path: buildkit
-
name: Set up QEMU
uses: docker/setup-qemu-action@v3
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
-
name: Download binary artifacts
uses: actions/download-artifact@v4
with:
name: binary
path: ./buildkit/build/moby/
-
name: Update daemon.json
run: |
sudo rm -f /etc/docker/daemon.json
sudo service docker restart
docker version
docker info
-
name: Build test image
uses: docker/bake-action@v4
with:
workdir: ./buildkit
targets: integration-tests
set: |
*.output=type=docker,name=${{ env.TEST_IMAGE_ID }}
-
name: Test
run: |
./hack/test ${{ matrix.typ }}
env:
CONTEXT: "."
TEST_DOCKERD: "1"
TEST_DOCKERD_BINARY: "./build/moby/dockerd"
TESTPKGS: "./${{ matrix.pkg }}"
TESTFLAGS: "-v --parallel=1 --timeout=30m --run=//worker=${{ matrix.worker }}$"
working-directory: buildkit

View file

@ -1,113 +0,0 @@
name: ci
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
on:
workflow_dispatch:
push:
branches:
- 'master'
- '[0-9]+.[0-9]+'
pull_request:
env:
DESTDIR: ./build
jobs:
validate-dco:
uses: ./.github/workflows/.dco.yml
build:
runs-on: ubuntu-20.04
needs:
- validate-dco
strategy:
fail-fast: false
matrix:
target:
- binary
- dynbinary
steps:
-
name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 0
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
-
name: Build
uses: docker/bake-action@v4
with:
targets: ${{ matrix.target }}
-
name: List artifacts
run: |
tree -nh ${{ env.DESTDIR }}
-
name: Check artifacts
run: |
find ${{ env.DESTDIR }} -type f -exec file -e ascii -- {} +
prepare-cross:
runs-on: ubuntu-latest
needs:
- validate-dco
outputs:
matrix: ${{ steps.platforms.outputs.matrix }}
steps:
-
name: Checkout
uses: actions/checkout@v4
-
name: Create matrix
id: platforms
run: |
matrix="$(docker buildx bake binary-cross --print | jq -cr '.target."binary-cross".platforms')"
echo "matrix=$matrix" >> $GITHUB_OUTPUT
-
name: Show matrix
run: |
echo ${{ steps.platforms.outputs.matrix }}
cross:
runs-on: ubuntu-20.04
needs:
- validate-dco
- prepare-cross
strategy:
fail-fast: false
matrix:
platform: ${{ fromJson(needs.prepare-cross.outputs.matrix) }}
steps:
-
name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 0
-
name: Prepare
run: |
platform=${{ matrix.platform }}
echo "PLATFORM_PAIR=${platform//\//-}" >> $GITHUB_ENV
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
-
name: Build
uses: docker/bake-action@v4
with:
targets: all
set: |
*.platform=${{ matrix.platform }}
-
name: List artifacts
run: |
tree -nh ${{ env.DESTDIR }}
-
name: Check artifacts
run: |
find ${{ env.DESTDIR }} -type f -exec file -e ascii -- {} +

View file

@ -1,177 +0,0 @@
name: test
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
on:
workflow_dispatch:
push:
branches:
- 'master'
- '[0-9]+.[0-9]+'
pull_request:
env:
GO_VERSION: "1.21.9"
GIT_PAGER: "cat"
PAGER: "cat"
jobs:
validate-dco:
uses: ./.github/workflows/.dco.yml
build-dev:
runs-on: ubuntu-20.04
needs:
- validate-dco
strategy:
fail-fast: false
matrix:
mode:
- ""
- systemd
steps:
-
name: Prepare
run: |
if [ "${{ matrix.mode }}" = "systemd" ]; then
echo "SYSTEMD=true" >> $GITHUB_ENV
fi
-
name: Checkout
uses: actions/checkout@v4
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
-
name: Build dev image
uses: docker/bake-action@v4
with:
targets: dev
set: |
*.cache-from=type=gha,scope=dev${{ matrix.mode }}
*.cache-to=type=gha,scope=dev${{ matrix.mode }},mode=max
*.output=type=cacheonly
test:
needs:
- build-dev
- validate-dco
uses: ./.github/workflows/.test.yml
secrets: inherit
strategy:
fail-fast: false
matrix:
storage:
- graphdriver
- snapshotter
with:
storage: ${{ matrix.storage }}
validate-prepare:
runs-on: ubuntu-20.04
needs:
- validate-dco
outputs:
matrix: ${{ steps.scripts.outputs.matrix }}
steps:
-
name: Checkout
uses: actions/checkout@v4
-
name: Create matrix
id: scripts
run: |
scripts=$(cd ./hack/validate && jq -nc '$ARGS.positional - ["all", "default", "dco"] | map(select(test("[.]")|not)) + ["generate-files"]' --args *)
echo "matrix=$scripts" >> $GITHUB_OUTPUT
-
name: Show matrix
run: |
echo ${{ steps.scripts.outputs.matrix }}
validate:
runs-on: ubuntu-20.04
timeout-minutes: 120
needs:
- validate-prepare
- build-dev
strategy:
fail-fast: true
matrix:
script: ${{ fromJson(needs.validate-prepare.outputs.matrix) }}
steps:
-
name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 0
-
name: Set up runner
uses: ./.github/actions/setup-runner
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
-
name: Build dev image
uses: docker/bake-action@v4
with:
targets: dev
set: |
dev.cache-from=type=gha,scope=dev
-
name: Validate
run: |
make -o build validate-${{ matrix.script }}
smoke-prepare:
runs-on: ubuntu-20.04
needs:
- validate-dco
outputs:
matrix: ${{ steps.platforms.outputs.matrix }}
steps:
-
name: Checkout
uses: actions/checkout@v4
-
name: Create matrix
id: platforms
run: |
matrix="$(docker buildx bake binary-smoketest --print | jq -cr '.target."binary-smoketest".platforms')"
echo "matrix=$matrix" >> $GITHUB_OUTPUT
-
name: Show matrix
run: |
echo ${{ steps.platforms.outputs.matrix }}
smoke:
runs-on: ubuntu-20.04
needs:
- smoke-prepare
strategy:
fail-fast: false
matrix:
platform: ${{ fromJson(needs.smoke-prepare.outputs.matrix) }}
steps:
-
name: Checkout
uses: actions/checkout@v4
-
name: Prepare
run: |
platform=${{ matrix.platform }}
echo "PLATFORM_PAIR=${platform//\//-}" >> $GITHUB_ENV
-
name: Set up QEMU
uses: docker/setup-qemu-action@v3
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
-
name: Test
uses: docker/bake-action@v4
with:
targets: binary-smoketest
set: |
*.platform=${{ matrix.platform }}

View file

@ -1,62 +0,0 @@
name: validate-pr
on:
pull_request:
types: [opened, edited, labeled, unlabeled]
jobs:
check-area-label:
runs-on: ubuntu-20.04
steps:
- name: Missing `area/` label
if: contains(join(github.event.pull_request.labels.*.name, ','), 'impact/') && !contains(join(github.event.pull_request.labels.*.name, ','), 'area/')
run: |
echo "::error::Every PR with an 'impact/*' label should also have an 'area/*' label"
exit 1
- name: OK
run: exit 0
check-changelog:
if: contains(join(github.event.pull_request.labels.*.name, ','), 'impact/')
runs-on: ubuntu-20.04
env:
PR_BODY: |
${{ github.event.pull_request.body }}
steps:
- name: Check changelog description
run: |
# Extract the `markdown changelog` note code block
block=$(echo -n "$PR_BODY" | tr -d '\r' | awk '/^```markdown changelog$/{flag=1;next}/^```$/{flag=0}flag')
# Strip empty lines
desc=$(echo "$block" | awk NF)
if [ -z "$desc" ]; then
echo "::error::Changelog section is empty. Please provide a description for the changelog."
exit 1
fi
len=$(echo -n "$desc" | wc -c)
if [[ $len -le 6 ]]; then
echo "::error::Description looks too short: $desc"
exit 1
fi
echo "This PR will be included in the release notes with the following note:"
echo "$desc"
check-pr-branch:
runs-on: ubuntu-20.04
env:
PR_TITLE: ${{ github.event.pull_request.title }}
steps:
# Backports or PR that target a release branch directly should mention the target branch in the title, for example:
# [X.Y backport] Some change that needs backporting to X.Y
# [X.Y] Change directly targeting the X.Y branch
- name: Get branch from PR title
id: title_branch
run: echo "$PR_TITLE" | sed -n 's/^\[\([0-9]*\.[0-9]*\)[^]]*\].*/branch=\1/p' >> $GITHUB_OUTPUT
- name: Check release branch
if: github.event.pull_request.base.ref != steps.title_branch.outputs.branch && !(github.event.pull_request.base.ref == 'master' && steps.title_branch.outputs.branch == '')
run: echo "::error::PR title suggests targetting the ${{ steps.title_branch.outputs.branch }} branch, but is opened against ${{ github.event.pull_request.base.ref }}" && exit 1

View file

@ -1,33 +0,0 @@
name: windows-2019
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
on:
schedule:
- cron: '0 10 * * *'
workflow_dispatch:
jobs:
validate-dco:
uses: ./.github/workflows/.dco.yml
test-prepare:
uses: ./.github/workflows/.test-prepare.yml
needs:
- validate-dco
run:
needs:
- test-prepare
uses: ./.github/workflows/.windows.yml
secrets: inherit
strategy:
fail-fast: false
matrix:
storage: ${{ fromJson(needs.test-prepare.outputs.matrix) }}
with:
os: windows-2019
storage: ${{ matrix.storage }}
send_coverage: false

View file

@ -1,36 +0,0 @@
name: windows-2022
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
on:
workflow_dispatch:
push:
branches:
- 'master'
- '[0-9]+.[0-9]+'
pull_request:
jobs:
validate-dco:
uses: ./.github/workflows/.dco.yml
test-prepare:
uses: ./.github/workflows/.test-prepare.yml
needs:
- validate-dco
run:
needs:
- test-prepare
uses: ./.github/workflows/.windows.yml
secrets: inherit
strategy:
fail-fast: false
matrix:
storage: ${{ fromJson(needs.test-prepare.outputs.matrix) }}
with:
os: windows-2022
storage: ${{ matrix.storage }}
send_coverage: true

33
.gitignore vendored
View file

@ -1,28 +1,23 @@
# If you want to ignore files created by your editor/tools, please consider a
# [global .gitignore](https://help.github.com/articles/ignoring-files).
*~
*.bak
# Docker project generated files to ignore
# if you want to ignore files created by your editor/tools,
# please consider a global .gitignore https://help.github.com/articles/ignoring-files
*.exe
*.exe~
*.gz
*.orig
test.main
.*.swp
.DS_Store
thumbs.db
# local repository customization
.envrc
# a .bashrc may be added to customize the build environment
.bashrc
.editorconfig
# build artifacts
.gopath/
.go-pkg-cache/
autogen/
bundles/
cli/winresources/*/*.syso
cli/winresources/*/winres.json
cmd/dockerd/dockerd
contrib/builder/rpm/*/changelog
# ci artifacts
*.exe
*.gz
vendor/pkg/
go-test-report.json
junit-report.xml
profile.out
test.main
junit-report.xml

View file

@ -1,137 +0,0 @@
linters:
enable:
- depguard
- dupword # Checks for duplicate words in the source code.
- goimports
- gosec
- gosimple
- govet
- importas
- ineffassign
- misspell
- revive
- staticcheck
- typecheck
- unconvert
- unused
disable:
- errcheck
run:
concurrency: 2
modules-download-mode: vendor
skip-dirs:
- docs
linters-settings:
dupword:
ignore:
- "true" # some tests use this as expected output
- "false" # some tests use this as expected output
- "root" # for tests using "ls" output with files owned by "root:root"
importas:
# Do not allow unaliased imports of aliased packages.
no-unaliased: true
alias:
# Enforce alias to prevent it accidentally being used instead of our
# own errdefs package (or vice-versa).
- pkg: github.com/containerd/containerd/errdefs
alias: cerrdefs
- pkg: github.com/opencontainers/image-spec/specs-go/v1
alias: ocispec
govet:
check-shadowing: false
depguard:
rules:
main:
deny:
- pkg: io/ioutil
desc: The io/ioutil package has been deprecated, see https://go.dev/doc/go1.16#ioutil
- pkg: "github.com/stretchr/testify/assert"
desc: Use "gotest.tools/v3/assert" instead
- pkg: "github.com/stretchr/testify/require"
desc: Use "gotest.tools/v3/assert" instead
- pkg: "github.com/stretchr/testify/suite"
desc: Do not use
revive:
rules:
# FIXME make sure all packages have a description. Currently, there's many packages without.
- name: package-comments
disabled: true
issues:
# The default exclusion rules are a bit too permissive, so copying the relevant ones below
exclude-use-default: false
exclude-rules:
# We prefer to use an "exclude-list" so that new "default" exclusions are not
# automatically inherited. We can decide whether or not to follow upstream
# defaults when updating golang-ci-lint versions.
# Unfortunately, this means we have to copy the whole exclusion pattern, as
# (unlike the "include" option), the "exclude" option does not take exclusion
# ID's.
#
# These exclusion patterns are copied from the default excluses at:
# https://github.com/golangci/golangci-lint/blob/v1.46.2/pkg/config/issues.go#L10-L104
# EXC0001
- text: "Error return value of .((os\\.)?std(out|err)\\..*|.*Close|.*Flush|os\\.Remove(All)?|.*print(f|ln)?|os\\.(Un)?Setenv). is not checked"
linters:
- errcheck
# EXC0006
- text: "Use of unsafe calls should be audited"
linters:
- gosec
# EXC0007
- text: "Subprocess launch(ed with variable|ing should be audited)"
linters:
- gosec
# EXC0008
# TODO: evaluate these and fix where needed: G307: Deferring unsafe method "*os.File" on type "Close" (gosec)
- text: "(G104|G307)"
linters:
- gosec
# EXC0009
- text: "(Expect directory permissions to be 0750 or less|Expect file permissions to be 0600 or less)"
linters:
- gosec
# EXC0010
- text: "Potential file inclusion via variable"
linters:
- gosec
# Looks like the match in "EXC0007" above doesn't catch this one
# TODO: consider upstreaming this to golangci-lint's default exclusion rules
- text: "G204: Subprocess launched with a potential tainted input or cmd arguments"
linters:
- gosec
# Looks like the match in "EXC0009" above doesn't catch this one
# TODO: consider upstreaming this to golangci-lint's default exclusion rules
- text: "G306: Expect WriteFile permissions to be 0600 or less"
linters:
- gosec
# Exclude some linters from running on tests files.
- path: _test\.go
linters:
- errcheck
- gosec
# Suppress golint complaining about generated types in api/types/
- text: "type name will be used as (container|volume)\\.(Container|Volume).* by other packages, and that stutters; consider calling this"
path: "api/types/(volume|container)/"
linters:
- revive
# FIXME temporarily suppress these (see https://github.com/gotestyourself/gotest.tools/issues/272)
- text: "SA1019: (assert|cmp|is)\\.ErrorType is deprecated"
linters:
- staticcheck
# Maximum issues count per one linter. Set to 0 to disable. Default is 50.
max-issues-per-linter: 0
# Maximum count of issues with the same text. Set to 0 to disable. Default is 3.
max-same-issues: 0

232
.mailmap
View file

@ -1,22 +1,15 @@
# This file lists the canonical name and email of contributors, and is used to
# generate AUTHORS (in hack/generate-authors.sh).
#
# To find new duplicates, regenerate AUTHORS and scan for name duplicates, or
# run the following to find email duplicates:
# git log --format='%aE - %aN' | sort -uf | awk -v IGNORECASE=1 '$1 in a {print a[$1]; print}; {a[$1]=$0}'
#
# For an explanation of this file format, consult gitmailmap(5).
# Generate AUTHORS: hack/generate-authors.sh
# Tip for finding duplicates (besides scanning the output of AUTHORS for name
# duplicates that aren't also email duplicates): scan the output of:
# git log --format='%aE - %aN' | sort -uf
#
# For explanation on this file format: man git-shortlog
<21551195@zju.edu.cn> <hsinko@users.noreply.github.com>
<mr.wrfly@gmail.com> <wrfly@users.noreply.github.com>
Aaron L. Xu <liker.xu@foxmail.com>
Aaron L. Xu <liker.xu@foxmail.com> <likexu@harmonycloud.cn>
Aaron Lehmann <alehmann@netflix.com>
Aaron Lehmann <alehmann@netflix.com> <aaron.lehmann@docker.com>
Abhinandan Prativadi <aprativadi@gmail.com>
Abhinandan Prativadi <aprativadi@gmail.com> <abhi@docker.com>
Abhinandan Prativadi <aprativadi@gmail.com> abhi <user.email>
Abhishek Chanda <abhishek.becs@gmail.com>
Abhishek Chanda <abhishek.becs@gmail.com> <abhishek.chanda@emc.com>
Ada Mancini <ada@docker.com>
Abhinandan Prativadi <abhi@docker.com>
Adam Dobrawy <naczelnik@jawnosc.tk>
Adam Dobrawy <naczelnik@jawnosc.tk> <ad-m@users.noreply.github.com>
Adrien Gallouët <adrien@gallouet.fr> <angt@users.noreply.github.com>
@ -29,38 +22,22 @@ Akihiro Matsushima <amatsusbit@gmail.com> <amatsus@users.noreply.github.com>
Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp> <suda.akihiro@lab.ntt.co.jp>
Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp> <suda.kyoto@gmail.com>
Akshay Moghe <akshay.moghe@gmail.com>
Albin Kerouanton <albinker@gmail.com>
Albin Kerouanton <albinker@gmail.com> <albin@akerouanton.name>
Albin Kerouanton <albinker@gmail.com> <557933+akerouanton@users.noreply.github.com>
Aleksa Sarai <asarai@suse.de>
Aleksa Sarai <asarai@suse.de> <asarai@suse.com>
Aleksa Sarai <asarai@suse.de> <cyphar@cyphar.com>
Aleksandrs Fadins <aleks@s-ko.net>
Alessandro Boch <aboch@tetrationanalytics.com>
Alessandro Boch <aboch@tetrationanalytics.com> <aboch@docker.com>
Alessandro Boch <aboch@tetrationanalytics.com> <aboch@socketplane.io>
Alessandro Boch <aboch@tetrationanalytics.com> <aboch@users.noreply.github.com>
Alex Chan <alex@alexwlchan.net>
Alex Chan <alex@alexwlchan.net> <alex.chan@metaswitch.com>
Alex Chen <alexchenunix@gmail.com> <root@localhost.localdomain>
Alex Ellis <alexellis2@gmail.com>
Alex Goodman <wagoodman@gmail.com> <wagoodman@users.noreply.github.com>
Alexander Larsson <alexl@redhat.com> <alexander.larsson@gmail.com>
Alexander Morozov <lk4d4math@gmail.com>
Alexander Morozov <lk4d4math@gmail.com> <lk4d4@docker.com>
Alexander Morozov <lk4d4@docker.com>
Alexander Morozov <lk4d4@docker.com> <lk4d4math@gmail.com>
Alexandre Beslic <alexandre.beslic@gmail.com> <abronan@docker.com>
Alexandre González <agonzalezro@gmail.com>
Alexis Ries <ries.alexis@gmail.com>
Alexis Ries <ries.alexis@gmail.com> <alexis.ries.ext@orange.com>
Alexis Thomas <fr.alexisthomas@gmail.com>
Alicia Lauerman <alicia@eta.im> <allydevour@me.com>
Allen Sun <allensun.shl@alibaba-inc.com> <allen.sun@daocloud.io>
Allen Sun <allensun.shl@alibaba-inc.com> <shlallen1990@gmail.com>
Anca Iordache <anca.iordache@docker.com>
Andrea Denisse Gómez <crypto.andrea@protonmail.ch>
Andrew Kim <taeyeonkim90@gmail.com>
Andrew Kim <taeyeonkim90@gmail.com> <akim01@fortinet.com>
Andrew Weiss <andrew.weiss@docker.com> <andrew.weiss@microsoft.com>
Andrew Weiss <andrew.weiss@docker.com> <andrew.weiss@outlook.com>
Andrey Kolomentsev <andrey.kolomentsev@docker.com>
@ -68,8 +45,6 @@ Andrey Kolomentsev <andrey.kolomentsev@docker.com> <andrey.kolomentsev@gmail.com
André Martins <aanm90@gmail.com> <martins@noironetworks.com>
Andy Rothfusz <github@developersupport.net> <github@metaliveblog.com>
Andy Smith <github@anarkystic.com>
Andy Zhang <andy.zhangtao@hotmail.com>
Andy Zhang <andy.zhangtao@hotmail.com> <ztao@tibco-support.com>
Ankush Agarwal <ankushagarwal11@gmail.com> <ankushagarwal@users.noreply.github.com>
Antonio Murdaca <antonio.murdaca@gmail.com> <amurdaca@redhat.com>
Antonio Murdaca <antonio.murdaca@gmail.com> <me@runcom.ninja>
@ -79,16 +54,11 @@ Antonio Murdaca <antonio.murdaca@gmail.com> <runcom@users.noreply.github.com>
Anuj Bahuguna <anujbahuguna.dev@gmail.com>
Anuj Bahuguna <anujbahuguna.dev@gmail.com> <abahuguna@fiberlink.com>
Anusha Ragunathan <anusha.ragunathan@docker.com> <anusha@docker.com>
Anyu Wang <wanganyu@outlook.com>
Arko Dasgupta <arko@tetrate.io>
Arko Dasgupta <arko@tetrate.io> <arko.dasgupta@docker.com>
Arko Dasgupta <arko@tetrate.io> <arkodg@users.noreply.github.com>
Arnaud Porterie <icecrime@gmail.com>
Arnaud Porterie <icecrime@gmail.com> <arnaud.porterie@docker.com>
Arnaud Rebillout <arnaud.rebillout@collabora.com>
Arnaud Rebillout <arnaud.rebillout@collabora.com> <elboulangero@gmail.com>
Arko Dasgupta <arko.dasgupta@docker.com>
Arko Dasgupta <arko.dasgupta@docker.com> <arkodg@users.noreply.github.com>
Arnaud Porterie <arnaud.porterie@docker.com>
Arnaud Porterie <arnaud.porterie@docker.com> <icecrime@gmail.com>
Arthur Gautier <baloo@gandi.net> <superbaloo+registrations.github@superbaloo.net>
Artur Meyster <arthurfbi@yahoo.com>
Avi Miller <avi.miller@oracle.com> <avi.miller@gmail.com>
Ben Bonnefoy <frenchben@docker.com>
Ben Golub <ben.golub@dotcloud.com>
@ -104,22 +74,14 @@ Bily Zhang <xcoder@tenxcloud.com>
Bin Liu <liubin0329@gmail.com>
Bin Liu <liubin0329@gmail.com> <liubin0329@users.noreply.github.com>
Bingshen Wang <bingshen.wbs@alibaba-inc.com>
Bjorn Neergaard <bjorn@neersighted.com>
Bjorn Neergaard <bjorn@neersighted.com> <bjorn.neergaard@docker.com>
Bjorn Neergaard <bjorn@neersighted.com> <bneergaard@mirantis.com>
Boaz Shuster <ripcurld.github@gmail.com>
Bojun Zhu <bojun.zhu@foxmail.com>
Boqin Qin <bobbqqin@gmail.com>
Boshi Lian <farmer1992@gmail.com>
Brandon Philips <brandon.philips@coreos.com> <brandon@ifup.co>
Brandon Philips <brandon.philips@coreos.com> <brandon@ifup.org>
Brent Salisbury <brent.salisbury@docker.com> <brent@docker.com>
Brian Goff <cpuguy83@gmail.com>
Brian Goff <cpuguy83@gmail.com> <bgoff@cpuguy83-mbp.home>
Brian Goff <cpuguy83@gmail.com> <bgoff@cpuguy83-mbp.local>
Brian Goff <cpuguy83@gmail.com> <brian.goff@microsoft.com>
Brian Goff <cpuguy83@gmail.com> <cpuguy@hey.com>
Cameron Sparr <gh@sparr.email>
Carlos de Paula <me@carlosedp.com>
Chander Govindarajan <chandergovind@gmail.com>
Chao Wang <wangchao.fnst@cn.fujitsu.com> <chaowang@localhost.localdomain>
@ -134,8 +96,6 @@ Chris Dias <cdias@microsoft.com>
Chris McKinnel <chris.mckinnel@tangentlabs.co.uk>
Chris Price <cprice@mirantis.com>
Chris Price <cprice@mirantis.com> <chris.price@docker.com>
Chris Telfer <ctelfer@docker.com>
Chris Telfer <ctelfer@docker.com> <ctelfer@users.noreply.github.com>
Christopher Biscardi <biscarch@sketcht.com>
Christopher Latham <sudosurootdev@gmail.com>
Christy Norman <christy@linux.vnet.ibm.com>
@ -145,8 +105,6 @@ Cristian Ariza <dev@cristianrz.com>
Cristian Staretu <cristian.staretu@gmail.com>
Cristian Staretu <cristian.staretu@gmail.com> <unclejack@users.noreply.github.com>
Cristian Staretu <cristian.staretu@gmail.com> <unclejacksons@gmail.com>
cui fliter <imcusg@gmail.com>
cui fliter <imcusg@gmail.com> cuishuang <imcusg@gmail.com>
CUI Wei <ghostplant@qq.com> cuiwei13 <cuiwei13@pku.edu.cn>
Daehyeok Mun <daehyeok@gmail.com>
Daehyeok Mun <daehyeok@gmail.com> <daehyeok@daehyeok-ui-MacBook-Air.local>
@ -173,23 +131,16 @@ Dattatraya Kumbhar <dattatraya.kumbhar@gslab.com>
Dave Goodchild <buddhamagnet@gmail.com>
Dave Henderson <dhenderson@gmail.com> <Dave.Henderson@ca.ibm.com>
Dave Tucker <dt@docker.com> <dave@dtucker.co.uk>
David Dooling <dooling@gmail.com>
David Dooling <dooling@gmail.com> <david.dooling@docker.com>
David M. Karr <davidmichaelkarr@gmail.com>
David Sheets <dsheets@docker.com> <sheets@alum.mit.edu>
David Sissitka <me@dsissitka.com>
David Williamson <david.williamson@docker.com> <davidwilliamson@users.noreply.github.com>
Derek Ch <denc716@gmail.com>
Derek McGowan <derek@mcg.dev>
Derek McGowan <derek@mcg.dev> <derek@mcgstyle.net>
Deshi Xiao <dxiao@redhat.com> <dsxiao@dataman-inc.com>
Deshi Xiao <dxiao@redhat.com> <xiaods@gmail.com>
Dhilip Kumars <dhilip.kumar.s@huawei.com>
Diego Siqueira <dieg0@live.com>
Diogo Monica <diogo@docker.com> <diogo.monica@gmail.com>
Dmitry Sharshakov <d3dx12.xx@gmail.com>
Dmitry Sharshakov <d3dx12.xx@gmail.com> <sh7dm@outlook.com>
Dmytro Iakovliev <dmytro.iakovliev@zodiacsystems.com>
Dominic Yin <yindongchao@inspur.com>
Dominik Honnef <dominik@honnef.co> <dominikh@fork-bomb.org>
Doug Davis <dug@us.ibm.com> <duglin@users.noreply.github.com>
@ -199,9 +150,6 @@ Drew Erny <derny@mirantis.com> <drew.erny@docker.com>
Elan Ruusamäe <glen@pld-linux.org>
Elan Ruusamäe <glen@pld-linux.org> <glen@delfi.ee>
Elango Sivanandam <elango.siva@docker.com>
Elango Sivanandam <elango.siva@docker.com> <elango@docker.com>
Eli Uriegas <seemethere101@gmail.com>
Eli Uriegas <seemethere101@gmail.com> <eli.uriegas@docker.com>
Eric G. Noriega <enoriega@vizuri.com> <egnoriega@users.noreply.github.com>
Eric Hanchrow <ehanchrow@ine.com> <eric.hanchrow@gmail.com>
Eric Rosenberg <ehaydenr@gmail.com> <ehaydenr@users.noreply.github.com>
@ -221,14 +169,10 @@ Felix Hupfeld <felix@quobyte.com> <quofelix@users.noreply.github.com>
Felix Ruess <felix.ruess@gmail.com> <felix.ruess@roboception.de>
Feng Yan <fy2462@gmail.com>
Fengtu Wang <wangfengtu@huawei.com> <wangfengtu@huawei.com>
Filipe Pina <hzlu1ot0@duck.com>
Filipe Pina <hzlu1ot0@duck.com> <636320+fopina@users.noreply.github.com>
Francisco Carriedo <fcarriedo@gmail.com>
Frank Rosquin <frank.rosquin+github@gmail.com> <frank.rosquin@gmail.com>
Frank Yang <yyb196@gmail.com>
Frederick F. Kautz IV <fkautz@redhat.com> <fkautz@alumni.cmu.edu>
Fu JinLin <withlin@yeah.net>
Gabriel Goller <gabrielgoller123@gmail.com>
Gabriel Nicolas Avellaneda <avellaneda.gabriel@gmail.com>
Gaetan de Villele <gdevillele@gmail.com>
Gang Qiao <qiaohai8866@gmail.com> <1373319223@qq.com>
@ -248,62 +192,42 @@ Guillaume J. Charmes <guillaume.charmes@docker.com> <guillaume.charmes@dotcloud.
Guillaume J. Charmes <guillaume.charmes@docker.com> <guillaume@charmes.net>
Guillaume J. Charmes <guillaume.charmes@docker.com> <guillaume@docker.com>
Guillaume J. Charmes <guillaume.charmes@docker.com> <guillaume@dotcloud.com>
Gunadhya S. <6939749+gunadhya@users.noreply.github.com>
Guoqiang QI <guoqiang.qi1@gmail.com>
Guri <odg0318@gmail.com>
Gurjeet Singh <gurjeet@singh.im> <singh.gurjeet@gmail.com>
Gustav Sinder <gustav.sinder@gmail.com>
Günther Jungbluth <gunther@gameslabs.net>
Hakan Özler <hakan.ozler@kodcu.com>
Hao Shu Wei <haoshuwei24@gmail.com>
Hao Shu Wei <haoshuwei24@gmail.com> <haoshuwei1989@163.com>
Hao Shu Wei <haoshuwei24@gmail.com> <haosw@cn.ibm.com>
Hao Shu Wei <haosw@cn.ibm.com>
Hao Shu Wei <haosw@cn.ibm.com> <haoshuwei1989@163.com>
Harald Albers <github@albersweb.de> <albers@users.noreply.github.com>
Harald Niesche <harald@niesche.de>
Harold Cooper <hrldcpr@gmail.com>
Harry Zhang <harryz@hyper.sh>
Harry Zhang <harryz@hyper.sh> <harryzhang@zju.edu.cn>
Harry Zhang <harryz@hyper.sh> <resouer@163.com>
Harry Zhang <harryz@hyper.sh> <resouer@gmail.com>
Harry Zhang <resouer@163.com>
Harshal Patil <harshal.patil@in.ibm.com> <harche@users.noreply.github.com>
He Simei <hesimei@zju.edu.cn>
Helen Xie <chenjg@harmonycloud.cn>
Hiroyuki Sasagawa <hs19870702@gmail.com>
Hollie Teal <hollie@docker.com>
Hollie Teal <hollie@docker.com> <hollie.teal@docker.com>
Hollie Teal <hollie@docker.com> <hollietealok@users.noreply.github.com>
hsinko <21551195@zju.edu.cn> <hsinko@users.noreply.github.com>
Hu Keping <hukeping@huawei.com>
Huajin Tong <fliterdashen@gmail.com>
Hui Kang <hkang.sunysb@gmail.com>
Hui Kang <hkang.sunysb@gmail.com> <kangh@us.ibm.com>
Huu Nguyen <huu@prismskylabs.com> <whoshuu@gmail.com>
Hyeongkyu Lee <hyeongkyu.lee@navercorp.com>
Hyzhou Zhy <hyzhou.zhy@alibaba-inc.com>
Hyzhou Zhy <hyzhou.zhy@alibaba-inc.com> <1187766782@qq.com>
Ian Campbell <ian.campbell@docker.com>
Ian Campbell <ian.campbell@docker.com> <ijc@docker.com>
Ilya Khlopotov <ilya.khlopotov@gmail.com>
Iskander Sharipov <quasilyte@gmail.com>
Ivan Babrou <ibobrik@gmail.com>
Ivan Markin <sw@nogoegst.net> <twim@riseup.net>
Jack Laxson <jackjrabbit@gmail.com>
Jacob Atzen <jacob@jacobatzen.dk> <jatzen@gmail.com>
Jacob Tomlinson <jacob@tom.linson.uk> <jacobtomlinson@users.noreply.github.com>
Jaivish Kothari <janonymous.codevulture@gmail.com>
Jake Moshenko <jake@devtable.com>
Jakub Drahos <jdrahos@pulsepoint.com>
Jakub Drahos <jdrahos@pulsepoint.com> <jack.drahos@gmail.com>
James Nesbitt <jnesbitt@mirantis.com>
James Nesbitt <jnesbitt@mirantis.com> <james.nesbitt@wunderkraut.com>
Jamie Hannaford <jamie@limetree.org> <jamie.hannaford@rackspace.com>
Jan Götte <jaseg@jaseg.net>
Jana Radhakrishnan <mrjana@docker.com>
Jana Radhakrishnan <mrjana@docker.com> <mrjana@socketplane.io>
Javier Bassi <javierbassi@gmail.com>
Javier Bassi <javierbassi@gmail.com> <CrimsonGlory@users.noreply.github.com>
Jay Lim <jay@imjching.com>
Jay Lim <jay@imjching.com> <imjching@hotmail.com>
Jean Rouge <rougej+github@gmail.com> <jer329@cornell.edu>
Jean-Baptiste Barth <jeanbaptiste.barth@gmail.com>
Jean-Baptiste Dalido <jeanbaptiste@appgratis.com>
@ -333,12 +257,11 @@ Joffrey F <joffrey@docker.com> <joffrey@dotcloud.com>
Johan Euphrosine <proppy@google.com> <proppy@aminche.com>
John Harris <john@johnharris.io>
John Howard <github@lowenna.com>
John Howard <github@lowenna.com> <10522484+lowenna@users.noreply.github.com>
John Howard <github@lowenna.com> <jhoward@microsoft.com>
John Howard <github@lowenna.com> <jhoward@ntdev.microsoft.com>
John Howard <github@lowenna.com> <jhowardmsft@users.noreply.github.com>
John Howard <github@lowenna.com> <John.Howard@microsoft.com>
John Howard <github@lowenna.com> <john.howard@microsoft.com>
John Howard <github@lowenna.com> <john@lowenna.com>
John Stephens <johnstep@docker.com> <johnstep@users.noreply.github.com>
Jon Surrell <jon.surrell@gmail.com> <jon.surrell@automattic.com>
Jonathan Choy <jonathan.j.choy@gmail.com>
@ -358,12 +281,9 @@ Josh Wilson <josh.wilson@fivestars.com> <jcwilson@users.noreply.github.com>
Joyce Jang <mail@joycejang.com>
Julien Bordellier <julienbordellier@gmail.com> <git@julienbordellier.com>
Julien Bordellier <julienbordellier@gmail.com> <me@julienbordellier.com>
Jun Du <dujun5@huawei.com>
Justin Cormack <justin.cormack@docker.com>
Justin Cormack <justin.cormack@docker.com> <justin.cormack@unikernel.com>
Justin Cormack <justin.cormack@docker.com> <justin@specialbusservice.com>
Justin Keller <85903732+jk-vb@users.noreply.github.com>
Justin Keller <85903732+jk-vb@users.noreply.github.com> <jkeller@vb-jkeller-mbp.local>
Justin Simonelis <justin.p.simonelis@gmail.com> <justin.simonelis@PTS-JSIMON2.toronto.exclamation.com>
Justin Terry <juterry@microsoft.com>
Jérôme Petazzoni <jerome.petazzoni@docker.com> <jerome.petazzoni@dotcloud.com>
@ -380,9 +300,6 @@ Ken Cochrane <kencochrane@gmail.com> <KenCochrane@gmail.com>
Ken Herner <kherner@progress.com> <chosenken@gmail.com>
Ken Reese <krrgithub@gmail.com>
Kenfe-Mickaël Laventure <mickael.laventure@gmail.com>
Kevin Alvarez <github@crazymax.dev>
Kevin Alvarez <github@crazymax.dev> <1951866+crazy-max@users.noreply.github.com>
Kevin Alvarez <github@crazymax.dev> <crazy-max@users.noreply.github.com>
Kevin Feyrer <kevin.feyrer@btinternet.com> <kevinfeyrer@users.noreply.github.com>
Kevin Kern <kaiwentan@harmonycloud.cn>
Kevin Meredith <kevin.m.meredith@gmail.com>
@ -393,17 +310,11 @@ Konrad Kleine <konrad.wilhelm.kleine@gmail.com> <kwk@users.noreply.github.com>
Konstantin Gribov <grossws@gmail.com>
Konstantin Pelykh <kpelykh@zettaset.com>
Kotaro Yoshimatsu <kotaro.yoshimatsu@gmail.com>
Kunal Kushwaha <kushwaha_kunal_v7@lab.ntt.co.jp>
Kunal Kushwaha <kushwaha_kunal_v7@lab.ntt.co.jp> <btkushuwahak@KUNAL-PC.swh.swh.nttdata.co.jp>
Kunal Kushwaha <kushwaha_kunal_v7@lab.ntt.co.jp> <kunal.kushwaha@gmail.com>
Kyle Squizzato <ksquizz@gmail.com>
Kyle Squizzato <ksquizz@gmail.com> <kyle.squizzato@docker.com>
Lajos Papp <lajos.papp@sequenceiq.com> <lalyos@yahoo.com>
Lei Gong <lgong@alauda.io>
Lei Jitang <leijitang@huawei.com>
Lei Jitang <leijitang@huawei.com> <leijitang@gmail.com>
Lei Jitang <leijitang@huawei.com> <leijitang@outlook.com>
Leiiwang <u2takey@gmail.com>
Liang Mingqiang <mqliang.zju@gmail.com>
Liang-Chi Hsieh <viirya@gmail.com>
Liao Qingwei <liaoqingwei@huawei.com>
@ -420,11 +331,8 @@ Lyn <energylyn@zju.edu.cn>
Lynda O'Leary <lyndaoleary29@gmail.com>
Lynda O'Leary <lyndaoleary29@gmail.com> <lyndaoleary@hotmail.com>
Ma Müller <mueller-ma@users.noreply.github.com>
Madhan Raj Mookkandy <MadhanRaj.Mookkandy@microsoft.com>
Madhan Raj Mookkandy <MadhanRaj.Mookkandy@microsoft.com> <madhanm@corp.microsoft.com>
Madhan Raj Mookkandy <MadhanRaj.Mookkandy@microsoft.com> <madhanm@microsoft.com>
Madhu Venugopal <mavenugo@gmail.com> <madhu@docker.com>
Madhu Venugopal <mavenugo@gmail.com> <madhu@socketplane.io>
Madhu Venugopal <madhu@socketplane.io> <madhu@docker.com>
Mageee <fangpuyi@foxmail.com> <21521230.zju.edu.cn>
Mansi Nahar <mmn4185@rit.edu> <mansi.nahar@macbookpro-mansinahar.local>
Mansi Nahar <mmn4185@rit.edu> <mansinahar@users.noreply.github.com>
@ -437,12 +345,10 @@ Markan Patel <mpatel678@gmail.com>
Markus Kortlang <hyp3rdino@googlemail.com> <markus.kortlang@lhsystems.com>
Martin Redmond <redmond.martin@gmail.com> <martin@tinychat.com>
Martin Redmond <redmond.martin@gmail.com> <xgithub@redmond5.com>
Maru Newby <mnewby@thesprawl.net>
Mary Anthony <mary.anthony@docker.com> <mary@docker.com>
Mary Anthony <mary.anthony@docker.com> <moxieandmore@gmail.com>
Mary Anthony <mary.anthony@docker.com> moxiegirl <mary@docker.com>
Masato Ohba <over.rye@gmail.com>
Mathieu Paturel <mathieu.paturel@gmail.com>
Matt Bentley <matt.bentley@docker.com> <mbentley@mbentley.net>
Matt Schurenko <matt.schurenko@gmail.com>
Matt Williams <mattyw@me.com>
@ -454,23 +360,15 @@ Matthias Kühnle <git.nivoc@neverbox.com> <kuehnle@online.de>
Mauricio Garavaglia <mauricio@medallia.com> <mauriciogaravaglia@gmail.com>
Maxwell <csuhp007@gmail.com>
Maxwell <csuhp007@gmail.com> <csuhqg@foxmail.com>
Menghui Chen <menghui.chen@alibaba-inc.com>
Michael Beskin <mrbeskin@gmail.com>
Michael Crosby <crosbymichael@gmail.com>
Michael Crosby <crosbymichael@gmail.com> <crosby.michael@gmail.com>
Michael Crosby <crosbymichael@gmail.com> <michael@crosbymichael.com>
Michael Crosby <crosbymichael@gmail.com> <michael@docker.com>
Michael Crosby <crosbymichael@gmail.com> <michael@thepasture.io>
Michael Crosby <michael@docker.com> <crosby.michael@gmail.com>
Michael Crosby <michael@docker.com> <crosbymichael@gmail.com>
Michael Crosby <michael@docker.com> <michael@crosbymichael.com>
Michael Hudson-Doyle <michael.hudson@canonical.com> <michael.hudson@linaro.org>
Michael Huettermann <michael@huettermann.net>
Michael Käufl <docker@c.michael-kaeufl.de> <michael-k@users.noreply.github.com>
Michael Nussbaum <michael.nussbaum@getbraintree.com>
Michael Nussbaum <michael.nussbaum@getbraintree.com> <code@getbraintree.com>
Michael Spetsiotis <michael_spets@hotmail.com>
Michael Stapelberg <michael+gh@stapelberg.de>
Michael Stapelberg <michael+gh@stapelberg.de> <stapelberg@google.com>
Michal Kostrzewa <michal.kostrzewa@codilime.com>
Michal Kostrzewa <michal.kostrzewa@codilime.com> <kostrzewa.michal@o2.pl>
Michal Minář <miminar@redhat.com>
Michał Gryko <github@odkurzacz.org>
Michiel de Jong <michiel@unhosted.org>
@ -478,24 +376,14 @@ Mickaël Fortunato <morsi.morsicus@gmail.com>
Miguel Angel Alvarez Cabrerizo <doncicuto@gmail.com> <30386061+doncicuto@users.noreply.github.com>
Miguel Angel Fernández <elmendalerenda@gmail.com>
Mihai Borobocea <MihaiBorob@gmail.com> <MihaiBorobocea@gmail.com>
Mikael Davranche <mikael.davranche@corp.ovh.com>
Mikael Davranche <mikael.davranche@corp.ovh.com> <mikael.davranche@corp.ovh.net>
Mike Casas <mkcsas0@gmail.com> <mikecasas@users.noreply.github.com>
Mike Goelzer <mike.goelzer@docker.com> <mgoelzer@docker.com>
Milas Bowman <devnull@milas.dev>
Milas Bowman <devnull@milas.dev> <milasb@gmail.com>
Milas Bowman <devnull@milas.dev> <milas.bowman@docker.com>
Milind Chawre <milindchawre@gmail.com>
Misty Stanley-Jones <misty@docker.com> <misty@apache.org>
Mohammad Banikazemi <MBanikazemi@gmail.com>
Mohammad Banikazemi <MBanikazemi@gmail.com> <mb@us.ibm.com>
Mohd Sadiq <mohdsadiq058@gmail.com> <mohdsadiq058@gmail.com>
Mohd Sadiq <mohdsadiq058@gmail.com> <42430865+msadiq058@users.noreply.github.com>
Mohit Soni <mosoni@ebay.com> <mohitsoni1989@gmail.com>
Moorthy RS <rsmoorthy@gmail.com> <rsmoorthy@users.noreply.github.com>
Moysés Borges <moysesb@gmail.com>
Moysés Borges <moysesb@gmail.com> <moyses.furtado@wplex.com.br>
mrfly <mr.wrfly@gmail.com> <wrfly@users.noreply.github.com>
Nace Oroz <orkica@gmail.com>
Natasha Jarus <linuxmercedes@gmail.com>
Nathan LeClaire <nathan.leclaire@docker.com> <nathan.leclaire@gmail.com>
@ -512,8 +400,6 @@ Oh Jinkyun <tintypemolly@gmail.com> <tintypemolly@Ohui-MacBook-Pro.local>
Oliver Reason <oli@overrateddev.co>
Olli Janatuinen <olli.janatuinen@gmail.com>
Olli Janatuinen <olli.janatuinen@gmail.com> <olljanat@users.noreply.github.com>
Onur Filiz <onur.filiz@microsoft.com>
Onur Filiz <onur.filiz@microsoft.com> <ofiliz@users.noreply.github.com>
Ouyang Liduo <oyld0210@163.com>
Patrick Stapleton <github@gdi2290.com>
Paul Liljenberg <liljenberg.paul@gmail.com> <letters@paulnotcom.se>
@ -524,20 +410,14 @@ Peter Dave Hello <hsu@peterdavehello.org> <PeterDaveHello@users.noreply.github.c
Peter Jaffe <pjaffe@nevo.com>
Peter Nagy <xificurC@gmail.com> <pnagy@gratex.com>
Peter Waller <p@pwaller.net> <peter@scraperwiki.com>
Phil Estes <estesp@gmail.com>
Phil Estes <estesp@gmail.com> <estesp@amazon.com>
Phil Estes <estesp@gmail.com> <estesp@linux.vnet.ibm.com>
Phil Estes <estesp@linux.vnet.ibm.com> <estesp@gmail.com>
Philip Alexander Etling <paetling@gmail.com>
Philipp Gillé <philipp.gille@gmail.com> <philippgille@users.noreply.github.com>
Prasanna Gautam <prasannagautam@gmail.com>
Puneet Pruthi <puneet.pruthi@oracle.com>
Puneet Pruthi <puneet.pruthi@oracle.com> <puneetpruthi@gmail.com>
Qiang Huang <h.huangqiang@huawei.com>
Qiang Huang <h.huangqiang@huawei.com> <qhuang@10.0.2.15>
Qin TianHuan <tianhuan@bingotree.cn>
Ray Tsang <rayt@google.com> <saturnism@users.noreply.github.com>
Renaud Gaubert <rgaubert@nvidia.com> <renaud.gaubert@gmail.com>
Richard Scothern <richard.scothern@gmail.com>
Robert Terhaar <rterhaar@atlanticdynamic.com> <robbyt@users.noreply.github.com>
Roberto G. Hashioka <roberto.hashioka@docker.com> <roberto_hashioka@hotmail.com>
Roberto Muñoz Fernández <robertomf@gmail.com> <roberto.munoz.fernandez.contractor@bbva.com>
@ -545,30 +425,17 @@ Robin Thoni <robin@rthoni.com>
Roman Dudin <katrmr@gmail.com> <decadent@users.noreply.github.com>
Rong Zhang <rongzhang@alauda.io>
Rongxiang Song <tinysong1226@gmail.com>
Rony Weng <ronyweng@synology.com>
Ross Boucher <rboucher@gmail.com>
Rui Cao <ruicao@alauda.io>
Runshen Zhu <runshen.zhu@gmail.com>
Ryan Stelly <ryan.stelly@live.com>
Ryoga Saito <contact@proelbtn.com>
Ryoga Saito <contact@proelbtn.com> <proelbtn@users.noreply.github.com>
Sainath Grandhi <sainath.grandhi@intel.com>
Sainath Grandhi <sainath.grandhi@intel.com> <saiallforums@gmail.com>
Sakeven Jiang <jc5930@sina.cn>
Samuel Karp <me@samuelkarp.com> <skarp@amazon.com>
Sandeep Bansal <sabansal@microsoft.com>
Sandeep Bansal <sabansal@microsoft.com> <msabansal@microsoft.com>
Santhosh Manohar <santhosh@docker.com>
Sargun Dhillon <sargun@netflix.com> <sargun@sargun.me>
Satoshi Tagomori <tagomoris@gmail.com>
Sean Lee <seanlee@tw.ibm.com> <scaleoutsean@users.noreply.github.com>
Sebastiaan van Stijn <github@gone.nl>
Sebastiaan van Stijn <github@gone.nl> <moby@example.com>
Sebastiaan van Stijn <github@gone.nl> <sebastiaan@ws-key-sebas3.dpi1.dpi>
Sebastiaan van Stijn <github@gone.nl> <thaJeztah@users.noreply.github.com>
Sebastian Thomschke <sebthom@users.noreply.github.com>
Seongyeol Lim <seongyeol37@gmail.com>
Serhii Nakon <serhii.n@thescimus.com>
Shaun Kaasten <shaunk@gmail.com>
Shawn Landden <shawn@churchofgit.com> <shawnlandden@gmail.com>
Shengbo Song <thomassong@tencent.com>
@ -577,10 +444,10 @@ Shih-Yuan Lee <fourdollars@gmail.com>
Shishir Mahajan <shishir.mahajan@redhat.com> <smahajan@redhat.com>
Shu-Wai Chow <shu-wai.chow@seattlechildrens.org>
Shukui Yang <yangshukui@huawei.com>
Shuwei Hao <haosw@cn.ibm.com>
Shuwei Hao <haosw@cn.ibm.com> <haoshuwei24@gmail.com>
Sidhartha Mani <sidharthamn@gmail.com>
Sjoerd Langkemper <sjoerd-github@linuxonly.nl> <sjoerd@byte.nl>
Smark Meng <smark@freecoop.net>
Smark Meng <smark@freecoop.net> <smarkm@users.noreply.github.com>
Solomon Hykes <solomon@docker.com> <s@docker.com>
Solomon Hykes <solomon@docker.com> <solomon.hykes@dotcloud.com>
Solomon Hykes <solomon@docker.com> <solomon@dotcloud.com>
@ -611,48 +478,29 @@ Sven Dowideit <SvenDowideit@home.org.au> <SvenDowideit@fosiki.com>
Sven Dowideit <SvenDowideit@home.org.au> <SvenDowideit@home.org.au>
Sven Dowideit <SvenDowideit@home.org.au> <SvenDowideit@users.noreply.github.com>
Sven Dowideit <SvenDowideit@home.org.au> <¨SvenDowideit@home.org.au¨>
Sylvain Baubeau <lebauce@gmail.com>
Sylvain Baubeau <lebauce@gmail.com> <sbaubeau@redhat.com>
Sylvain Bellemare <sylvain@ascribe.io>
Sylvain Bellemare <sylvain@ascribe.io> <sylvain.bellemare@ezeep.com>
Takuto Sato <tockn.jp@gmail.com>
Tangi Colin <tangicolin@gmail.com>
Tejesh Mehta <tejesh.mehta@gmail.com> <tj@init.me>
Terry Chu <zue.hterry@gmail.com>
Terry Chu <zue.hterry@gmail.com> <jubosh.tw@gmail.com>
Thatcher Peskens <thatcher@docker.com>
Thatcher Peskens <thatcher@docker.com> <thatcher@dotcloud.com>
Thatcher Peskens <thatcher@docker.com> <thatcher@gmx.net>
Thiago Alves Silva <thiago.alves@aurea.com>
Thiago Alves Silva <thiago.alves@aurea.com> <thiagoalves@users.noreply.github.com>
Thomas Gazagnaire <thomas@gazagnaire.org> <thomas@gazagnaire.com>
Thomas Ledos <thomas.ledos92@gmail.com>
Thomas Léveil <thomasleveil@gmail.com>
Thomas Léveil <thomasleveil@gmail.com> <thomasleveil@users.noreply.github.com>
Tibor Vass <teabee89@gmail.com> <tibor@docker.com>
Tibor Vass <teabee89@gmail.com> <tiborvass@users.noreply.github.com>
Till Claassen <pixelistik@users.noreply.github.com>
Tim Bart <tim@fewagainstmany.com>
Tim Bosse <taim@bosboot.org> <maztaim@users.noreply.github.com>
Tim Potter <tpot@hpe.com>
Tim Potter <tpot@hpe.com> <tpot@Tims-MacBook-Pro.local>
Tim Ruffles <oi@truffles.me.uk> <timruffles@googlemail.com>
Tim Terhorst <mynamewastaken+git@gmail.com>
Tim Wagner <tim.wagner@freenet.ag>
Tim Wagner <tim.wagner@freenet.ag> <33624860+herrwagner@users.noreply.github.com>
Tim Zju <21651152@zju.edu.cn>
Timothy Hobbs <timothyhobbs@seznam.cz>
Toli Kuznets <toli@docker.com>
Tom Barlow <tomwbarlow@gmail.com>
Tom Denham <tom@tomdee.co.uk>
Tom Denham <tom@tomdee.co.uk> <tom.denham@metaswitch.com>
Tom Sweeney <tsweeney@redhat.com>
Tom Wilkie <tom.wilkie@gmail.com>
Tom Wilkie <tom.wilkie@gmail.com> <tom@weave.works>
Tõnis Tiigi <tonistiigi@gmail.com>
Trace Andreason <tandreason@gmail.com>
Trapier Marshall <tmarshall@mirantis.com>
Trapier Marshall <tmarshall@mirantis.com> <trapier.marshall@docker.com>
Trishna Guha <trishnaguha17@gmail.com>
Tristan Carel <tristan@cogniteev.com>
Tristan Carel <tristan@cogniteev.com> <tristan.carel@gmail.com>
@ -666,15 +514,12 @@ Victor Vieux <victor.vieux@docker.com> <victor@docker.com>
Victor Vieux <victor.vieux@docker.com> <victor@dotcloud.com>
Victor Vieux <victor.vieux@docker.com> <victorvieux@gmail.com>
Victor Vieux <victor.vieux@docker.com> <vieux@docker.com>
Vikas Choudhary <choudharyvikas16@gmail.com>
Vikram bir Singh <vsingh@mirantis.com>
Vikram bir Singh <vsingh@mirantis.com> <vikrambir.singh@docker.com>
Viktor Vojnovski <viktor.vojnovski@amadeus.com> <vojnovski@gmail.com>
Vincent Batts <vbatts@redhat.com> <vbatts@hashbangbash.com>
Vincent Bernat <vincent@bernat.ch>
Vincent Bernat <vincent@bernat.ch> <bernat@luffy.cx>
Vincent Bernat <vincent@bernat.ch> <Vincent.Bernat@exoscale.ch>
Vincent Bernat <vincent@bernat.ch> <vincent@bernat.im>
Vincent Bernat <Vincent.Bernat@exoscale.ch> <bernat@luffy.cx>
Vincent Bernat <Vincent.Bernat@exoscale.ch> <vincent@bernat.im>
Vincent Boulineau <vincent.boulineau@datadoghq.com>
Vincent Demeester <vincent.demeester@docker.com> <vincent+github@demeester.fr>
Vincent Demeester <vincent.demeester@docker.com> <vincent@demeester.fr>
@ -683,8 +528,6 @@ Vishnu Kannan <vishnuk@google.com>
Vitaly Ostrosablin <vostrosablin@virtuozzo.com>
Vitaly Ostrosablin <vostrosablin@virtuozzo.com> <tmp6154@yandex.ru>
Vladimir Rutsky <altsysrq@gmail.com> <iamironbob@gmail.com>
Vladislav Kolesnikov <vkolesnikov@beget.ru>
Vladislav Kolesnikov <vkolesnikov@beget.ru> <prime@vladqa.ru>
Walter Stanish <walter@pratyeka.org>
Wang Chao <chao.wang@ucloud.cn>
Wang Chao <chao.wang@ucloud.cn> <wcwxyz@gmail.com>
@ -696,27 +539,18 @@ Wang Yuexiao <wang.yuexiao@zte.com.cn>
Wayne Chang <wayne@neverfear.org>
Wayne Song <wsong@docker.com> <wsong@users.noreply.github.com>
Wei Wu <wuwei4455@gmail.com> cizixs <cizixs@163.com>
Wei-Ting Kuo <waitingkuo0527@gmail.com>
Wen Cheng Ma <wenchma@cn.ibm.com>
Wenjun Tang <tangwj2@lenovo.com> <dodia@163.com>
Wewang Xiaorenfine <wang.xiaoren@zte.com.cn>
Will Weaver <monkey@buildingbananas.com>
Wing-Kam Wong <wingkwong.code@gmail.com>
WuLonghui <wlh6666@qq.com>
Xian Chaobo <xianchaobo@huawei.com>
Xian Chaobo <xianchaobo@huawei.com> <jimmyxian2004@yahoo.com.cn>
Xianglin Gao <xlgao@zju.edu.cn>
Xianjie <guxianjie@gmail.com>
Xianjie <guxianjie@gmail.com> <datastream@datastream-laptop.local>
Xianlu Bird <xianlubird@gmail.com>
Xiao YongBiao <xyb4638@gmail.com>
Xiao Zhang <xiaozhang0210@hotmail.com>
Xiaodong Liu <liuxiaodong@loongson.cn>
Xiaodong Zhang <a4012017@sina.com>
Xiaohua Ding <xiao_hua_ding@sina.cn>
Xiaoyu Zhang <zhang.xiaoyu33@zte.com.cn>
Xinfeng Liu <XinfengLiu@icloud.com>
Xinfeng Liu <XinfengLiu@icloud.com> <xinfeng.liu@gmail.com>
Xuecong Liao <satorulogic@gmail.com>
Yamasaki Masahide <masahide.y@gmail.com>
Yao Zaiyong <yaozaiyong@hotmail.com>
@ -733,15 +567,9 @@ Yu Changchun <yuchangchun1@huawei.com>
Yu Chengxia <yuchengxia@huawei.com>
Yu Peng <yu.peng36@zte.com.cn>
Yu Peng <yu.peng36@zte.com.cn> <yupeng36@zte.com.cn>
Yuan Sun <sunyuan3@huawei.com>
Yue Zhang <zy675793960@yeah.net>
Yufei Xiong <yufei.xiong@qq.com>
Zach Gershman <zachgersh@gmail.com>
Zach Gershman <zachgersh@gmail.com> <zachgersh@users.noreply.github.com>
Zachary Jaffee <zjaffee@us.ibm.com> <zij@case.edu>
Zachary Jaffee <zjaffee@us.ibm.com> <zjaffee@apache.org>
Zhang Kun <zkazure@gmail.com>
Zhang Wentao <zhangwentao234@huawei.com>
ZhangHang <stevezhang2014@gmail.com>
Zhenkun Bi <bi.zhenkun@zte.com.cn>
Zhou Hao <zhouhao@cn.fujitsu.com>
@ -751,5 +579,3 @@ Ziheng Liu <lzhfromustc@gmail.com>
Zou Yu <zouyu7@huawei.com>
Zuhayr Elahi <zuhayr.elahi@docker.com>
Zuhayr Elahi <zuhayr.elahi@docker.com> <elahi.zuhayr@gmail.com>
정재영 <jjy600901@gmail.com>
정재영 <jjy600901@gmail.com> <43400316+J-jaeyoung@users.noreply.github.com>

351
AUTHORS

File diff suppressed because it is too large Load diff

3609
CHANGELOG.md Normal file

File diff suppressed because it is too large Load diff

View file

@ -72,7 +72,7 @@ anybody starts working on it.
We are always thrilled to receive pull requests. We do our best to process them
quickly. If your pull request is not accepted on the first try,
don't get discouraged! Our contributor's guide explains [the review process we
use for simple changes](https://docs.docker.com/contribute/overview/).
use for simple changes](https://docs.docker.com/opensource/workflow/make-a-contribution/).
### Design and cleanup proposals
@ -101,8 +101,9 @@ the contributors guide.
<td>
<p>
Register for the Docker Community Slack at
<a href="https://dockr.ly/comm-slack" target="_blank">https://dockr.ly/comm-slack</a>.
<a href="https://community.docker.com/registrations/groups/4316" target="_blank">https://community.docker.com/registrations/groups/4316</a>.
We use the #moby-project channel for general discussion, and there are separate channels for other Moby projects such as #containerd.
Archives are available at <a href="https://dockercommunity.slackarchive.io/" target="_blank">https://dockercommunity.slackarchive.io/</a>.
</p>
</td>
</tr>
@ -309,6 +310,36 @@ Don't forget: being a maintainer is a time investment. Make sure you
will have time to make yourself available. You don't have to be a
maintainer to make a difference on the project!
### Manage issues and pull requests using the Derek bot
If you want to help label, assign, close or reopen issues or pull requests
without commit rights, ask a maintainer to add your Github handle to the
`.DEREK.yml` file. [Derek](https://github.com/alexellis/derek) is a bot that extends
Github's user permissions to help non-committers to manage issues and pull requests simply by commenting.
For example:
* Labels
```
Derek add label: kind/question
Derek remove label: status/claimed
```
* Assign work
```
Derek assign: username
Derek unassign: me
```
* Manage issues and PRs
```
Derek close
Derek reopen
```
## Moby community guidelines
We want to keep the Moby community awesome, growing and collaborative. We need
@ -422,6 +453,6 @@ The rules:
guidelines. Since you've read all the rules, you now know that.
If you are having trouble getting into the mood of idiomatic Go, we recommend
reading through [Effective Go](https://go.dev/doc/effective_go). The
[Go Blog](https://go.dev/blog/) is also a great resource. Drinking the
reading through [Effective Go](https://golang.org/doc/effective_go.html). The
[Go Blog](https://blog.golang.org) is also a great resource. Drinking the
kool-aid is a lot easier than going thirsty.

View file

@ -1,127 +1,86 @@
# syntax=docker/dockerfile:1.7
ARG GO_VERSION=1.21.9
ARG BASE_DEBIAN_DISTRO="bookworm"
ARG GOLANG_IMAGE="golang:${GO_VERSION}-${BASE_DEBIAN_DISTRO}"
ARG XX_VERSION=1.4.0
ARG VPNKIT_VERSION=0.5.0
ARG DOCKERCLI_REPOSITORY="https://github.com/docker/cli.git"
ARG DOCKERCLI_VERSION=v26.0.0
# cli version used for integration-cli tests
ARG DOCKERCLI_INTEGRATION_REPOSITORY="https://github.com/docker/cli.git"
ARG DOCKERCLI_INTEGRATION_VERSION=v17.06.2-ce
ARG BUILDX_VERSION=0.13.1
ARG COMPOSE_VERSION=v2.25.0
# syntax=docker/dockerfile:1
ARG CROSS="false"
ARG SYSTEMD="false"
ARG DOCKER_STATIC=1
ARG GO_VERSION=1.20.10
ARG DEBIAN_FRONTEND=noninteractive
ARG VPNKIT_VERSION=0.5.0
ARG DOCKER_BUILDTAGS="apparmor seccomp"
# REGISTRY_VERSION specifies the version of the registry to download from
# https://hub.docker.com/r/distribution/distribution. This version of
# the registry is used to test schema 2 manifests. Generally, the version
# specified here should match a current release.
ARG REGISTRY_VERSION=2.8.3
ARG BASE_DEBIAN_DISTRO="bullseye"
ARG GOLANG_IMAGE="golang:${GO_VERSION}-${BASE_DEBIAN_DISTRO}"
# delve is currently only supported on linux/amd64 and linux/arm64;
# https://github.com/go-delve/delve/blob/v1.8.1/pkg/proc/native/support_sentinel.go#L1-L6
ARG DELVE_SUPPORTED=${TARGETPLATFORM#linux/amd64} DELVE_SUPPORTED=${DELVE_SUPPORTED#linux/arm64}
ARG DELVE_SUPPORTED=${DELVE_SUPPORTED:+"unsupported"}
ARG DELVE_SUPPORTED=${DELVE_SUPPORTED:-"supported"}
# cross compilation helper
FROM --platform=$BUILDPLATFORM tonistiigi/xx:${XX_VERSION} AS xx
# dummy stage to make sure the image is built for deps that don't support some
# architectures
FROM --platform=$BUILDPLATFORM busybox AS build-dummy
RUN mkdir -p /build
FROM scratch AS binary-dummy
COPY --from=build-dummy /build /build
# base
FROM --platform=$BUILDPLATFORM ${GOLANG_IMAGE} AS base
COPY --from=xx / /
FROM ${GOLANG_IMAGE} AS base
RUN echo 'Binary::apt::APT::Keep-Downloaded-Packages "true";' > /etc/apt/apt.conf.d/keep-cache
RUN apt-get update && apt-get install --no-install-recommends -y file
ARG APT_MIRROR
RUN test -n "$APT_MIRROR" && sed -ri "s/(httpredir|deb|security).debian.org/${APT_MIRROR}/g" /etc/apt/sources.list || true
ENV GO111MODULE=off
ENV GOTOOLCHAIN=local
FROM base AS criu
ADD --chmod=0644 https://download.opensuse.org/repositories/devel:/tools:/criu/Debian_11/Release.key /etc/apt/trusted.gpg.d/criu.gpg.asc
ARG DEBIAN_FRONTEND
# Install dependency packages specific to criu
RUN --mount=type=cache,sharing=locked,id=moby-criu-aptlib,target=/var/lib/apt \
--mount=type=cache,sharing=locked,id=moby-criu-aptcache,target=/var/cache/apt \
echo 'deb https://download.opensuse.org/repositories/devel:/tools:/criu/Debian_12/ /' > /etc/apt/sources.list.d/criu.list \
&& apt-get update \
&& apt-get install -y --no-install-recommends criu \
&& install -D /usr/sbin/criu /build/criu \
&& /build/criu --version
apt-get update && apt-get install -y --no-install-recommends \
libcap-dev \
libnet-dev \
libnl-3-dev \
libprotobuf-c-dev \
libprotobuf-dev \
protobuf-c-compiler \
protobuf-compiler \
python3-protobuf
# registry
FROM base AS registry-src
WORKDIR /usr/src/registry
RUN git init . && git remote add origin "https://github.com/distribution/distribution.git"
# Install CRIU for checkpoint/restore support
ARG CRIU_VERSION=3.14
RUN mkdir -p /usr/src/criu \
&& curl -sSL https://github.com/checkpoint-restore/criu/archive/v${CRIU_VERSION}.tar.gz | tar -C /usr/src/criu/ -xz --strip-components=1 \
&& cd /usr/src/criu \
&& make \
&& make PREFIX=/build/ install-criu
FROM base AS registry
WORKDIR /go/src/github.com/docker/distribution
# REGISTRY_VERSION_SCHEMA1 specifies the version of the registry to build and
# install from the https://github.com/docker/distribution repository. This is
# an older (pre v2.3.0) version of the registry that only supports schema1
# manifests. This version of the registry is not working on arm64, so installation
# is skipped on that architecture.
ARG REGISTRY_VERSION_SCHEMA1=v2.1.0
ARG TARGETPLATFORM
RUN --mount=from=registry-src,src=/usr/src/registry,rw \
--mount=type=cache,target=/root/.cache/go-build,id=registry-build-$TARGETPLATFORM \
# Install two versions of the registry. The first one is a recent version that
# supports both schema 1 and 2 manifests. The second one is an older version that
# only supports schema1 manifests. This allows integration-cli tests to cover
# push/pull with both schema1 and schema2 manifests.
# The old version of the registry is not working on arm64, so installation is
# skipped on that architecture.
ENV REGISTRY_COMMIT_SCHEMA1 ec87e9b6971d831f0eff752ddb54fb64693e51cd
ENV REGISTRY_COMMIT 47a064d4195a9b56133891bbb13620c3ac83a827
RUN --mount=type=cache,target=/root/.cache/go-build \
--mount=type=cache,target=/go/pkg/mod \
--mount=type=tmpfs,target=/go/src <<EOT
set -ex
export GOPATH="/go/src/github.com/docker/distribution/Godeps/_workspace:$GOPATH"
# Make the /build directory no matter what so that it doesn't fail on arm64 or
# any other platform where we don't build this registry
mkdir /build
case $TARGETPLATFORM in
linux/amd64|linux/arm/v7|linux/ppc64le|linux/s390x)
git fetch -q --depth 1 origin "${REGISTRY_VERSION_SCHEMA1}" +refs/tags/*:refs/tags/*
git checkout -q FETCH_HEAD
CGO_ENABLED=0 xx-go build -o /build/registry-v2-schema1 -v ./cmd/registry
xx-verify /build/registry-v2-schema1
;;
esac
EOT
FROM distribution/distribution:$REGISTRY_VERSION AS registry-v2
RUN mkdir /build && mv /bin/registry /build/registry-v2
# go-swagger
FROM base AS swagger-src
WORKDIR /usr/src/swagger
# Currently uses a fork from https://github.com/kolyshkin/go-swagger/tree/golang-1.13-fix
# TODO: move to under moby/ or fix upstream go-swagger to work for us.
RUN git init . && git remote add origin "https://github.com/kolyshkin/go-swagger.git"
# GO_SWAGGER_COMMIT specifies the version of the go-swagger binary to build and
# install. Go-swagger is used in CI for validating swagger.yaml in hack/validate/swagger-gen
ARG GO_SWAGGER_COMMIT=c56166c036004ba7a3a321e5951ba472b9ae298c
RUN git fetch -q --depth 1 origin "${GO_SWAGGER_COMMIT}" && git checkout -q FETCH_HEAD
--mount=type=tmpfs,target=/go/src/ \
set -x \
&& git clone https://github.com/docker/distribution.git . \
&& git checkout -q "$REGISTRY_COMMIT" \
&& GOPATH="/go/src/github.com/docker/distribution/Godeps/_workspace:$GOPATH" \
go build -buildmode=pie -o /build/registry-v2 github.com/docker/distribution/cmd/registry \
&& case $(dpkg --print-architecture) in \
amd64|armhf|ppc64*|s390x) \
git checkout -q "$REGISTRY_COMMIT_SCHEMA1"; \
GOPATH="/go/src/github.com/docker/distribution/Godeps/_workspace:$GOPATH"; \
go build -buildmode=pie -o /build/registry-v2-schema1 github.com/docker/distribution/cmd/registry; \
;; \
esac
FROM base AS swagger
WORKDIR /go/src/github.com/go-swagger/go-swagger
ARG TARGETPLATFORM
RUN --mount=from=swagger-src,src=/usr/src/swagger,rw \
--mount=type=cache,target=/root/.cache/go-build,id=swagger-build-$TARGETPLATFORM \
WORKDIR $GOPATH/src/github.com/go-swagger/go-swagger
# Install go-swagger for validating swagger.yaml
# This is https://github.com/kolyshkin/go-swagger/tree/golang-1.13-fix
# TODO: move to under moby/ or fix upstream go-swagger to work for us.
ENV GO_SWAGGER_COMMIT c56166c036004ba7a3a321e5951ba472b9ae298c
RUN --mount=type=cache,target=/root/.cache/go-build \
--mount=type=cache,target=/go/pkg/mod \
--mount=type=tmpfs,target=/go/src/ <<EOT
set -e
xx-go build -o /build/swagger ./cmd/swagger
xx-verify /build/swagger
EOT
--mount=type=tmpfs,target=/go/src/ \
set -x \
&& git clone https://github.com/kolyshkin/go-swagger.git . \
&& git checkout -q "$GO_SWAGGER_COMMIT" \
&& go build -o /build/swagger github.com/go-swagger/go-swagger/cmd/swagger
# frozen-images
# See also frozenImages in "testutil/environment/protect.go" (which needs to
# be updated when adding images to this list)
FROM debian:${BASE_DEBIAN_DISTRO} AS frozen-images
ARG DEBIAN_FRONTEND
RUN --mount=type=cache,sharing=locked,id=moby-frozen-images-aptlib,target=/var/lib/apt \
--mount=type=cache,sharing=locked,id=moby-frozen-images-aptcache,target=/var/cache/apt \
apt-get update && apt-get install -y --no-install-recommends \
@ -131,380 +90,171 @@ RUN --mount=type=cache,sharing=locked,id=moby-frozen-images-aptlib,target=/var/l
# Get useful and necessary Hub images so we can "docker load" locally instead of pulling
COPY contrib/download-frozen-image-v2.sh /
ARG TARGETARCH
ARG TARGETVARIANT
RUN /download-frozen-image-v2.sh /build \
busybox:latest@sha256:95cf004f559831017cdf4628aaf1bb30133677be8702a8c5f2994629f637a209 \
busybox:glibc@sha256:1f81263701cddf6402afe9f33fca0266d9fff379e59b1748f33d3072da71ee85 \
debian:bookworm-slim@sha256:2bc5c236e9b262645a323e9088dfa3bb1ecb16cc75811daf40a23a824d665be9 \
debian:bullseye-slim@sha256:dacf278785a4daa9de07596ec739dbc07131e189942772210709c5c0777e8437 \
hello-world:latest@sha256:d58e752213a51785838f9eed2b7a498ffa1cb3aa7f946dda11af39286c3db9a9 \
arm32v7/hello-world:latest@sha256:50b8560ad574c779908da71f7ce370c0a2471c098d44d1c8f6b513c5a55eeeb1
# See also frozenImages in "testutil/environment/protect.go" (which needs to be updated when adding images to this list)
# delve
FROM base AS delve-src
WORKDIR /usr/src/delve
RUN git init . && git remote add origin "https://github.com/go-delve/delve.git"
# DELVE_VERSION specifies the version of the Delve debugger binary
# from the https://github.com/go-delve/delve repository.
# It can be used to run Docker with a possibility of
# attaching debugger to it.
ARG DELVE_VERSION=v1.21.1
RUN git fetch -q --depth 1 origin "${DELVE_VERSION}" +refs/tags/*:refs/tags/* && git checkout -q FETCH_HEAD
FROM base AS cross-false
FROM base AS delve-supported
WORKDIR /usr/src/delve
ARG TARGETPLATFORM
RUN --mount=from=delve-src,src=/usr/src/delve,rw \
--mount=type=cache,target=/root/.cache/go-build,id=delve-build-$TARGETPLATFORM \
--mount=type=cache,target=/go/pkg/mod <<EOT
set -e
GO111MODULE=on xx-go build -o /build/dlv ./cmd/dlv
xx-verify /build/dlv
EOT
FROM binary-dummy AS delve-unsupported
FROM delve-${DELVE_SUPPORTED} AS delve
FROM base AS tomll
# GOTOML_VERSION specifies the version of the tomll binary to build and install
# from the https://github.com/pelletier/go-toml repository. This binary is used
# in CI in the hack/validate/toml script.
#
# When updating this version, consider updating the github.com/pelletier/go-toml
# dependency in vendor.mod accordingly.
ARG GOTOML_VERSION=v1.8.1
RUN --mount=type=cache,target=/root/.cache/go-build \
--mount=type=cache,target=/go/pkg/mod \
GOBIN=/build/ GO111MODULE=on go install "github.com/pelletier/go-toml/cmd/tomll@${GOTOML_VERSION}" \
&& /build/tomll --help
FROM base AS gowinres
# GOWINRES_VERSION defines go-winres tool version
ARG GOWINRES_VERSION=v0.3.1
RUN --mount=type=cache,target=/root/.cache/go-build \
--mount=type=cache,target=/go/pkg/mod \
GOBIN=/build/ GO111MODULE=on go install "github.com/tc-hib/go-winres@${GOWINRES_VERSION}" \
&& /build/go-winres --help
# containerd
FROM base AS containerd-src
WORKDIR /usr/src/containerd
RUN git init . && git remote add origin "https://github.com/containerd/containerd.git"
# CONTAINERD_VERSION is used to build containerd binaries, and used for the
# integration tests. The distributed docker .deb and .rpm packages depend on a
# separate (containerd.io) package, which may be a different version as is
# specified here. The containerd golang package is also pinned in vendor.mod.
# When updating the binary version you may also need to update the vendor
# version to pick up bug fixes or new APIs, however, usually the Go packages
# are built from a commit from the master branch.
ARG CONTAINERD_VERSION=v1.7.15
RUN git fetch -q --depth 1 origin "${CONTAINERD_VERSION}" +refs/tags/*:refs/tags/* && git checkout -q FETCH_HEAD
FROM base AS containerd-build
WORKDIR /go/src/github.com/containerd/containerd
ARG TARGETPLATFORM
RUN --mount=type=cache,sharing=locked,id=moby-containerd-aptlib,target=/var/lib/apt \
--mount=type=cache,sharing=locked,id=moby-containerd-aptcache,target=/var/cache/apt \
apt-get update && xx-apt-get install -y --no-install-recommends \
gcc \
libbtrfs-dev \
libsecret-1-dev \
pkg-config
ARG DOCKER_STATIC
RUN --mount=from=containerd-src,src=/usr/src/containerd,rw \
--mount=type=cache,target=/root/.cache/go-build,id=containerd-build-$TARGETPLATFORM <<EOT
set -e
export CC=$(xx-info)-gcc
export CGO_ENABLED=$([ "$DOCKER_STATIC" = "1" ] && echo "0" || echo "1")
xx-go --wrap
make $([ "$DOCKER_STATIC" = "1" ] && echo "STATIC=1") binaries
xx-verify $([ "$DOCKER_STATIC" = "1" ] && echo "--static") bin/containerd
xx-verify $([ "$DOCKER_STATIC" = "1" ] && echo "--static") bin/containerd-shim-runc-v2
xx-verify $([ "$DOCKER_STATIC" = "1" ] && echo "--static") bin/ctr
mkdir /build
mv bin/containerd bin/containerd-shim-runc-v2 bin/ctr /build
EOT
FROM containerd-build AS containerd-linux
FROM binary-dummy AS containerd-windows
FROM containerd-${TARGETOS} AS containerd
FROM base AS golangci_lint
ARG GOLANGCI_LINT_VERSION=v1.55.2
RUN --mount=type=cache,target=/root/.cache/go-build \
--mount=type=cache,target=/go/pkg/mod \
GOBIN=/build/ GO111MODULE=on go install "github.com/golangci/golangci-lint/cmd/golangci-lint@${GOLANGCI_LINT_VERSION}" \
&& /build/golangci-lint --version
FROM base AS gotestsum
ARG GOTESTSUM_VERSION=v1.8.2
RUN --mount=type=cache,target=/root/.cache/go-build \
--mount=type=cache,target=/go/pkg/mod \
GOBIN=/build/ GO111MODULE=on go install "gotest.tools/gotestsum@${GOTESTSUM_VERSION}" \
&& /build/gotestsum --version
FROM base AS shfmt
ARG SHFMT_VERSION=v3.8.0
RUN --mount=type=cache,target=/root/.cache/go-build \
--mount=type=cache,target=/go/pkg/mod \
GOBIN=/build/ GO111MODULE=on go install "mvdan.cc/sh/v3/cmd/shfmt@${SHFMT_VERSION}" \
&& /build/shfmt --version
FROM base AS gopls
RUN --mount=type=cache,target=/root/.cache/go-build \
--mount=type=cache,target=/go/pkg/mod \
GOBIN=/build/ GO111MODULE=on go install "golang.org/x/tools/gopls@latest" \
&& /build/gopls version
FROM base AS dockercli
WORKDIR /go/src/github.com/docker/cli
ARG DOCKERCLI_REPOSITORY
ARG DOCKERCLI_VERSION
ARG TARGETPLATFORM
RUN --mount=source=hack/dockerfile/cli.sh,target=/download-or-build-cli.sh \
--mount=type=cache,id=dockercli-git-$TARGETPLATFORM,sharing=locked,target=./.git \
--mount=type=cache,target=/root/.cache/go-build,id=dockercli-build-$TARGETPLATFORM \
rm -f ./.git/*.lock \
&& /download-or-build-cli.sh ${DOCKERCLI_VERSION} ${DOCKERCLI_REPOSITORY} /build \
&& /build/docker --version
FROM base AS dockercli-integration
WORKDIR /go/src/github.com/docker/cli
ARG DOCKERCLI_INTEGRATION_REPOSITORY
ARG DOCKERCLI_INTEGRATION_VERSION
ARG TARGETPLATFORM
RUN --mount=source=hack/dockerfile/cli.sh,target=/download-or-build-cli.sh \
--mount=type=cache,id=dockercli-git-$TARGETPLATFORM,sharing=locked,target=./.git \
--mount=type=cache,target=/root/.cache/go-build,id=dockercli-build-$TARGETPLATFORM \
rm -f ./.git/*.lock \
&& /download-or-build-cli.sh ${DOCKERCLI_INTEGRATION_VERSION} ${DOCKERCLI_INTEGRATION_REPOSITORY} /build \
&& /build/docker --version
# runc
FROM base AS runc-src
WORKDIR /usr/src/runc
RUN git init . && git remote add origin "https://github.com/opencontainers/runc.git"
# RUNC_VERSION should match the version that is used by the containerd version
# that is used. If you need to update runc, open a pull request in the containerd
# project first, and update both after that is merged. When updating RUNC_VERSION,
# consider updating runc in vendor.mod accordingly.
ARG RUNC_VERSION=v1.1.12
RUN git fetch -q --depth 1 origin "${RUNC_VERSION}" +refs/tags/*:refs/tags/* && git checkout -q FETCH_HEAD
FROM base AS runc-build
WORKDIR /go/src/github.com/opencontainers/runc
ARG TARGETPLATFORM
RUN --mount=type=cache,sharing=locked,id=moby-runc-aptlib,target=/var/lib/apt \
--mount=type=cache,sharing=locked,id=moby-runc-aptcache,target=/var/cache/apt \
apt-get update && xx-apt-get install -y --no-install-recommends \
dpkg-dev \
gcc \
libc6-dev \
libseccomp-dev \
pkg-config
ARG DOCKER_STATIC
RUN --mount=from=runc-src,src=/usr/src/runc,rw \
--mount=type=cache,target=/root/.cache/go-build,id=runc-build-$TARGETPLATFORM <<EOT
set -e
xx-go --wrap
CGO_ENABLED=1 make "$([ "$DOCKER_STATIC" = "1" ] && echo "static" || echo "runc")"
xx-verify $([ "$DOCKER_STATIC" = "1" ] && echo "--static") runc
mkdir /build
mv runc /build/
EOT
FROM runc-build AS runc-linux
FROM binary-dummy AS runc-windows
FROM runc-${TARGETOS} AS runc
# tini
FROM base AS tini-src
WORKDIR /usr/src/tini
RUN git init . && git remote add origin "https://github.com/krallin/tini.git"
# TINI_VERSION specifies the version of tini (docker-init) to build. This
# binary is used when starting containers with the `--init` option.
ARG TINI_VERSION=v0.19.0
RUN git fetch -q --depth 1 origin "${TINI_VERSION}" +refs/tags/*:refs/tags/* && git checkout -q FETCH_HEAD
FROM base AS tini-build
WORKDIR /go/src/github.com/krallin/tini
RUN --mount=type=cache,sharing=locked,id=moby-tini-aptlib,target=/var/lib/apt \
--mount=type=cache,sharing=locked,id=moby-tini-aptcache,target=/var/cache/apt \
apt-get update && apt-get install -y --no-install-recommends cmake
ARG TARGETPLATFORM
RUN --mount=type=cache,sharing=locked,id=moby-tini-aptlib,target=/var/lib/apt \
--mount=type=cache,sharing=locked,id=moby-tini-aptcache,target=/var/cache/apt \
xx-apt-get install -y --no-install-recommends \
gcc \
libc6-dev \
pkg-config
RUN --mount=from=tini-src,src=/usr/src/tini,rw \
--mount=type=cache,target=/root/.cache/go-build,id=tini-build-$TARGETPLATFORM <<EOT
set -e
CC=$(xx-info)-gcc cmake .
make tini-static
xx-verify --static tini-static
mkdir /build
mv tini-static /build/docker-init
EOT
FROM tini-build AS tini-linux
FROM binary-dummy AS tini-windows
FROM tini-${TARGETOS} AS tini
# rootlesskit
FROM base AS rootlesskit-src
WORKDIR /usr/src/rootlesskit
RUN git init . && git remote add origin "https://github.com/rootless-containers/rootlesskit.git"
# When updating, also update vendor.mod and hack/dockerfile/install/rootlesskit.installer accordingly.
ARG ROOTLESSKIT_VERSION=v2.0.2
RUN git fetch -q --depth 1 origin "${ROOTLESSKIT_VERSION}" +refs/tags/*:refs/tags/* && git checkout -q FETCH_HEAD
FROM base AS rootlesskit-build
WORKDIR /go/src/github.com/rootless-containers/rootlesskit
ARG TARGETPLATFORM
RUN --mount=type=cache,sharing=locked,id=moby-rootlesskit-aptlib,target=/var/lib/apt \
--mount=type=cache,sharing=locked,id=moby-rootlesskit-aptcache,target=/var/cache/apt \
apt-get update && xx-apt-get install -y --no-install-recommends \
gcc \
libc6-dev \
pkg-config
ENV GO111MODULE=on
ARG DOCKER_STATIC
RUN --mount=from=rootlesskit-src,src=/usr/src/rootlesskit,rw \
--mount=type=cache,target=/go/pkg/mod \
--mount=type=cache,target=/root/.cache/go-build,id=rootlesskit-build-$TARGETPLATFORM <<EOT
set -e
export CGO_ENABLED=$([ "$DOCKER_STATIC" = "1" ] && echo "0" || echo "1")
xx-go build -o /build/rootlesskit -ldflags="$([ "$DOCKER_STATIC" != "1" ] && echo "-linkmode=external")" ./cmd/rootlesskit
xx-verify $([ "$DOCKER_STATIC" = "1" ] && echo "--static") /build/rootlesskit
xx-go build -o /build/rootlesskit-docker-proxy -ldflags="$([ "$DOCKER_STATIC" != "1" ] && echo "-linkmode=external")" ./cmd/rootlesskit-docker-proxy
xx-verify $([ "$DOCKER_STATIC" = "1" ] && echo "--static") /build/rootlesskit-docker-proxy
EOT
COPY --link ./contrib/dockerd-rootless.sh /build/
COPY --link ./contrib/dockerd-rootless-setuptool.sh /build/
FROM rootlesskit-build AS rootlesskit-linux
FROM binary-dummy AS rootlesskit-windows
FROM rootlesskit-${TARGETOS} AS rootlesskit
FROM base AS crun
ARG CRUN_VERSION=1.12
RUN --mount=type=cache,sharing=locked,id=moby-crun-aptlib,target=/var/lib/apt \
--mount=type=cache,sharing=locked,id=moby-crun-aptcache,target=/var/cache/apt \
FROM --platform=linux/amd64 base AS cross-true
ARG DEBIAN_FRONTEND
RUN dpkg --add-architecture arm64
RUN dpkg --add-architecture armel
RUN dpkg --add-architecture armhf
RUN dpkg --add-architecture ppc64el
RUN dpkg --add-architecture s390x
RUN --mount=type=cache,sharing=locked,id=moby-cross-true-aptlib,target=/var/lib/apt \
--mount=type=cache,sharing=locked,id=moby-cross-true-aptcache,target=/var/cache/apt \
apt-get update && apt-get install -y --no-install-recommends \
autoconf \
automake \
build-essential \
libcap-dev \
libprotobuf-c-dev \
crossbuild-essential-arm64 \
crossbuild-essential-armel \
crossbuild-essential-armhf \
crossbuild-essential-ppc64el \
crossbuild-essential-s390x
FROM cross-${CROSS} as dev-base
FROM dev-base AS runtime-dev-cross-false
ARG DEBIAN_FRONTEND
RUN --mount=type=cache,sharing=locked,id=moby-cross-false-aptlib,target=/var/lib/apt \
--mount=type=cache,sharing=locked,id=moby-cross-false-aptcache,target=/var/cache/apt \
apt-get update && apt-get install -y --no-install-recommends \
binutils-mingw-w64 \
g++-mingw-w64-x86-64 \
libapparmor-dev \
libbtrfs-dev \
libdevmapper-dev \
libseccomp-dev \
libsystemd-dev \
libtool \
libudev-dev \
libyajl-dev \
python3 \
;
RUN --mount=type=tmpfs,target=/tmp/crun-build \
git clone https://github.com/containers/crun.git /tmp/crun-build && \
cd /tmp/crun-build && \
git checkout -q "${CRUN_VERSION}" && \
./autogen.sh && \
./configure --bindir=/build && \
make -j install
libudev-dev
# vpnkit
# use dummy scratch stage to avoid build to fail for unsupported platforms
FROM scratch AS vpnkit-windows
FROM scratch AS vpnkit-linux-386
FROM scratch AS vpnkit-linux-arm
FROM scratch AS vpnkit-linux-ppc64le
FROM scratch AS vpnkit-linux-riscv64
FROM scratch AS vpnkit-linux-s390x
FROM djs55/vpnkit:${VPNKIT_VERSION} AS vpnkit-linux-amd64
FROM djs55/vpnkit:${VPNKIT_VERSION} AS vpnkit-linux-arm64
FROM vpnkit-linux-${TARGETARCH} AS vpnkit-linux
FROM vpnkit-${TARGETOS} AS vpnkit
# containerutility
FROM base AS containerutil-src
WORKDIR /usr/src/containerutil
RUN git init . && git remote add origin "https://github.com/docker-archive/windows-container-utility.git"
ARG CONTAINERUTILITY_VERSION=aa1ba87e99b68e0113bd27ec26c60b88f9d4ccd9
RUN git fetch -q --depth 1 origin "${CONTAINERUTILITY_VERSION}" +refs/tags/*:refs/tags/* && git checkout -q FETCH_HEAD
FROM base AS containerutil-build
WORKDIR /usr/src/containerutil
ARG TARGETPLATFORM
RUN xx-apt-get install -y --no-install-recommends \
gcc \
g++ \
libc6-dev \
pkg-config
RUN --mount=from=containerutil-src,src=/usr/src/containerutil,rw \
--mount=type=cache,target=/root/.cache/go-build,id=containerutil-build-$TARGETPLATFORM <<EOT
set -e
CC="$(xx-info)-gcc" CXX="$(xx-info)-g++" make
xx-verify --static containerutility.exe
mkdir /build
mv containerutility.exe /build/
EOT
FROM binary-dummy AS containerutil-linux
FROM containerutil-build AS containerutil-windows-amd64
FROM containerutil-windows-${TARGETARCH} AS containerutil-windows
FROM containerutil-${TARGETOS} AS containerutil
FROM docker/buildx-bin:${BUILDX_VERSION} as buildx
FROM docker/compose-bin:${COMPOSE_VERSION} as compose
FROM base AS dev-systemd-false
COPY --link --from=frozen-images /build/ /docker-frozen-images
COPY --link --from=swagger /build/ /usr/local/bin/
COPY --link --from=delve /build/ /usr/local/bin/
COPY --link --from=tomll /build/ /usr/local/bin/
COPY --link --from=gowinres /build/ /usr/local/bin/
COPY --link --from=tini /build/ /usr/local/bin/
COPY --link --from=registry /build/ /usr/local/bin/
COPY --link --from=registry-v2 /build/ /usr/local/bin/
# Skip the CRIU stage for now, as the opensuse package repository is sometimes
# unstable, and we're currently not using it in CI.
#
# FIXME(thaJeztah): re-enable this stage when https://github.com/moby/moby/issues/38963 is resolved (see https://github.com/moby/moby/pull/38984)
# COPY --link --from=criu /build/ /usr/local/bin/
COPY --link --from=gotestsum /build/ /usr/local/bin/
COPY --link --from=golangci_lint /build/ /usr/local/bin/
COPY --link --from=shfmt /build/ /usr/local/bin/
COPY --link --from=runc /build/ /usr/local/bin/
COPY --link --from=containerd /build/ /usr/local/bin/
COPY --link --from=rootlesskit /build/ /usr/local/bin/
COPY --link --from=vpnkit / /usr/local/bin/
COPY --link --from=containerutil /build/ /usr/local/bin/
COPY --link --from=crun /build/ /usr/local/bin/
COPY --link hack/dockerfile/etc/docker/ /etc/docker/
COPY --link --from=buildx /buildx /usr/local/libexec/docker/cli-plugins/docker-buildx
COPY --link --from=compose /docker-compose /usr/libexec/docker/cli-plugins/docker-compose
ENV PATH=/usr/local/cli:$PATH
ENV TEST_CLIENT_BINARY=/usr/local/cli-integration/docker
ENV CONTAINERD_ADDRESS=/run/docker/containerd/containerd.sock
ENV CONTAINERD_NAMESPACE=moby
WORKDIR /go/src/github.com/docker/docker
VOLUME /var/lib/docker
VOLUME /home/unprivilegeduser/.local/share/docker
# Wrap all commands in the "docker-in-docker" script to allow nested containers
ENTRYPOINT ["hack/dind"]
FROM dev-systemd-false AS dev-systemd-true
RUN --mount=type=cache,sharing=locked,id=moby-dev-aptlib,target=/var/lib/apt \
--mount=type=cache,sharing=locked,id=moby-dev-aptcache,target=/var/cache/apt \
FROM --platform=linux/amd64 runtime-dev-cross-false AS runtime-dev-cross-true
ARG DEBIAN_FRONTEND
# These crossbuild packages rely on gcc-<arch>, but this doesn't want to install
# on non-amd64 systems, so other architectures cannot crossbuild amd64.
RUN --mount=type=cache,sharing=locked,id=moby-cross-true-aptlib,target=/var/lib/apt \
--mount=type=cache,sharing=locked,id=moby-cross-true-aptcache,target=/var/cache/apt \
apt-get update && apt-get install -y --no-install-recommends \
dbus \
dbus-user-session \
systemd \
systemd-sysv
ENTRYPOINT ["hack/dind-systemd"]
libapparmor-dev:arm64 \
libapparmor-dev:armel \
libapparmor-dev:armhf \
libapparmor-dev:ppc64el \
libapparmor-dev:s390x \
libseccomp-dev:arm64 \
libseccomp-dev:armel \
libseccomp-dev:armhf \
libseccomp-dev:ppc64el \
libseccomp-dev:s390x
FROM dev-systemd-${SYSTEMD} AS dev-base
FROM runtime-dev-cross-${CROSS} AS runtime-dev
FROM base AS tomlv
ARG TOMLV_COMMIT
RUN --mount=type=cache,target=/root/.cache/go-build \
--mount=type=cache,target=/go/pkg/mod \
--mount=type=bind,src=hack/dockerfile/install,target=/tmp/install \
PREFIX=/build /tmp/install/install.sh tomlv
FROM base AS vndr
ARG VNDR_VERSION
RUN --mount=type=cache,target=/root/.cache/go-build \
--mount=type=cache,target=/go/pkg/mod \
--mount=type=bind,src=hack/dockerfile/install,target=/tmp/install \
PREFIX=/build /tmp/install/install.sh vndr
FROM dev-base AS containerd
ARG DEBIAN_FRONTEND
RUN --mount=type=cache,sharing=locked,id=moby-containerd-aptlib,target=/var/lib/apt \
--mount=type=cache,sharing=locked,id=moby-containerd-aptcache,target=/var/cache/apt \
apt-get update && apt-get install -y --no-install-recommends \
libbtrfs-dev
ARG CONTAINERD_VERSION
RUN --mount=type=cache,target=/root/.cache/go-build \
--mount=type=cache,target=/go/pkg/mod \
--mount=type=bind,src=hack/dockerfile/install,target=/tmp/install \
PREFIX=/build /tmp/install/install.sh containerd
FROM dev-base AS proxy
ARG LIBNETWORK_COMMIT
RUN --mount=type=cache,target=/root/.cache/go-build \
--mount=type=cache,target=/go/pkg/mod \
--mount=type=bind,src=hack/dockerfile/install,target=/tmp/install \
PREFIX=/build /tmp/install/install.sh proxy
FROM base AS golangci_lint
ARG GOLANGCI_LINT_VERSION
RUN --mount=type=cache,target=/root/.cache/go-build \
--mount=type=cache,target=/go/pkg/mod \
--mount=type=bind,src=hack/dockerfile/install,target=/tmp/install \
PREFIX=/build /tmp/install/install.sh golangci_lint
FROM base AS gotestsum
ARG GOTESTSUM_VERSION
RUN --mount=type=cache,target=/root/.cache/go-build \
--mount=type=cache,target=/go/pkg/mod \
--mount=type=bind,src=hack/dockerfile/install,target=/tmp/install \
PREFIX=/build /tmp/install/install.sh gotestsum
FROM base AS shfmt
ARG SHFMT_VERSION
RUN --mount=type=cache,target=/root/.cache/go-build \
--mount=type=cache,target=/go/pkg/mod \
--mount=type=bind,src=hack/dockerfile/install,target=/tmp/install \
PREFIX=/build /tmp/install/install.sh shfmt
FROM dev-base AS dockercli
ARG DOCKERCLI_CHANNEL
ARG DOCKERCLI_VERSION
RUN --mount=type=cache,target=/root/.cache/go-build \
--mount=type=cache,target=/go/pkg/mod \
--mount=type=bind,src=hack/dockerfile/install,target=/tmp/install \
PREFIX=/build /tmp/install/install.sh dockercli
FROM runtime-dev AS runc
ARG RUNC_VERSION
ARG RUNC_BUILDTAGS
RUN --mount=type=cache,target=/root/.cache/go-build \
--mount=type=cache,target=/go/pkg/mod \
--mount=type=bind,src=hack/dockerfile/install,target=/tmp/install \
PREFIX=/build /tmp/install/install.sh runc
FROM dev-base AS tini
ARG DEBIAN_FRONTEND
ARG TINI_VERSION
RUN --mount=type=cache,sharing=locked,id=moby-tini-aptlib,target=/var/lib/apt \
--mount=type=cache,sharing=locked,id=moby-tini-aptcache,target=/var/cache/apt \
apt-get update && apt-get install -y --no-install-recommends \
cmake \
vim-common
RUN --mount=type=cache,target=/root/.cache/go-build \
--mount=type=cache,target=/go/pkg/mod \
--mount=type=bind,src=hack/dockerfile/install,target=/tmp/install \
PREFIX=/build /tmp/install/install.sh tini
FROM dev-base AS rootlesskit
ARG ROOTLESSKIT_VERSION
RUN --mount=type=cache,target=/root/.cache/go-build \
--mount=type=cache,target=/go/pkg/mod \
--mount=type=bind,src=hack/dockerfile/install,target=/tmp/install \
PREFIX=/build /tmp/install/install.sh rootlesskit
COPY ./contrib/dockerd-rootless.sh /build
COPY ./contrib/dockerd-rootless-setuptool.sh /build
FROM --platform=amd64 djs55/vpnkit:${VPNKIT_VERSION} AS vpnkit-amd64
FROM --platform=arm64 djs55/vpnkit:${VPNKIT_VERSION} AS vpnkit-arm64
FROM scratch AS vpnkit
COPY --from=vpnkit-amd64 /vpnkit /build/vpnkit.x86_64
COPY --from=vpnkit-arm64 /vpnkit /build/vpnkit.aarch64
# TODO: Some of this is only really needed for testing, it would be nice to split this up
FROM runtime-dev AS dev-systemd-false
ARG DEBIAN_FRONTEND
RUN groupadd -r docker
RUN useradd --create-home --gid docker unprivilegeduser \
&& mkdir -p /home/unprivilegeduser/.local/share/docker \
@ -534,138 +284,125 @@ RUN --mount=type=cache,sharing=locked,id=moby-dev-aptlib,target=/var/lib/apt \
libnet1 \
libnl-3-200 \
libprotobuf-c1 \
libyajl2 \
net-tools \
patch \
pigz \
python3-pip \
python3-setuptools \
python3-wheel \
sudo \
systemd-journal-remote \
thin-provisioning-tools \
uidmap \
vim \
vim-common \
xfsprogs \
xz-utils \
zip \
zstd
zip
# Switch to use iptables instead of nftables (to match the CI hosts)
# TODO use some kind of runtime auto-detection instead if/when nftables is supported (https://github.com/moby/moby/issues/26824)
RUN update-alternatives --set iptables /usr/sbin/iptables-legacy || true \
&& update-alternatives --set ip6tables /usr/sbin/ip6tables-legacy || true \
&& update-alternatives --set arptables /usr/sbin/arptables-legacy || true
RUN pip3 install yamllint==1.26.1
COPY --from=dockercli /build/ /usr/local/cli
COPY --from=frozen-images /build/ /docker-frozen-images
COPY --from=swagger /build/ /usr/local/bin/
COPY --from=tomlv /build/ /usr/local/bin/
COPY --from=tini /build/ /usr/local/bin/
COPY --from=registry /build/ /usr/local/bin/
# Skip the CRIU stage for now, as the opensuse package repository is sometimes
# unstable, and we're currently not using it in CI.
#
# FIXME(thaJeztah): re-enable this stage when https://github.com/moby/moby/issues/38963 is resolved (see https://github.com/moby/moby/pull/38984)
# COPY --from=criu /build/ /usr/local/
COPY --from=vndr /build/ /usr/local/bin/
COPY --from=gotestsum /build/ /usr/local/bin/
COPY --from=golangci_lint /build/ /usr/local/bin/
COPY --from=shfmt /build/ /usr/local/bin/
COPY --from=runc /build/ /usr/local/bin/
COPY --from=containerd /build/ /usr/local/bin/
COPY --from=rootlesskit /build/ /usr/local/bin/
COPY --from=vpnkit /build/ /usr/local/bin/
COPY --from=proxy /build/ /usr/local/bin/
ENV PATH=/usr/local/cli:$PATH
ARG DOCKER_BUILDTAGS
ENV DOCKER_BUILDTAGS="${DOCKER_BUILDTAGS}"
WORKDIR /go/src/github.com/docker/docker
VOLUME /var/lib/docker
VOLUME /home/unprivilegeduser/.local/share/docker
# Wrap all commands in the "docker-in-docker" script to allow nested containers
ENTRYPOINT ["hack/dind"]
FROM dev-systemd-false AS dev-systemd-true
RUN --mount=type=cache,sharing=locked,id=moby-dev-aptlib,target=/var/lib/apt \
--mount=type=cache,sharing=locked,id=moby-dev-aptcache,target=/var/cache/apt \
apt-get update && apt-get install --no-install-recommends -y \
gcc \
pkg-config \
dpkg-dev \
libapparmor-dev \
libseccomp-dev \
libsecret-1-dev \
libsystemd-dev \
libudev-dev \
yamllint
COPY --link --from=dockercli /build/ /usr/local/cli
COPY --link --from=dockercli-integration /build/ /usr/local/cli-integration
apt-get update && apt-get install -y --no-install-recommends \
dbus \
dbus-user-session \
systemd \
systemd-sysv
RUN mkdir -p hack \
&& curl -o hack/dind-systemd https://raw.githubusercontent.com/AkihiroSuda/containerized-systemd/b70bac0daeea120456764248164c21684ade7d0d/docker-entrypoint.sh \
&& chmod +x hack/dind-systemd
ENTRYPOINT ["hack/dind-systemd"]
FROM base AS build
COPY --from=gowinres /build/ /usr/local/bin/
WORKDIR /go/src/github.com/docker/docker
ENV GO111MODULE=off
ENV CGO_ENABLED=1
RUN --mount=type=cache,sharing=locked,id=moby-build-aptlib,target=/var/lib/apt \
--mount=type=cache,sharing=locked,id=moby-build-aptcache,target=/var/cache/apt \
apt-get update && apt-get install --no-install-recommends -y \
clang \
lld \
llvm
ARG TARGETPLATFORM
RUN --mount=type=cache,sharing=locked,id=moby-build-aptlib,target=/var/lib/apt \
--mount=type=cache,sharing=locked,id=moby-build-aptcache,target=/var/cache/apt \
xx-apt-get install --no-install-recommends -y \
dpkg-dev \
gcc \
libapparmor-dev \
libc6-dev \
libseccomp-dev \
libsecret-1-dev \
libsystemd-dev \
libudev-dev \
pkg-config
ARG DOCKER_BUILDTAGS
ARG DOCKER_DEBUG
FROM dev-systemd-${SYSTEMD} AS dev
FROM runtime-dev AS binary-base
ARG DOCKER_GITCOMMIT=HEAD
ARG DOCKER_LDFLAGS
ARG DOCKER_STATIC
ENV DOCKER_GITCOMMIT=${DOCKER_GITCOMMIT}
ARG VERSION
ENV VERSION=${VERSION}
ARG PLATFORM
ENV PLATFORM=${PLATFORM}
ARG PRODUCT
ENV PRODUCT=${PRODUCT}
ARG DEFAULT_PRODUCT_LICENSE
ARG PACKAGER_NAME
# PREFIX overrides DEST dir in make.sh script otherwise it fails because of
# read only mount in current work dir
ENV PREFIX=/tmp
RUN <<EOT
# in bullseye arm64 target does not link with lld so configure it to use ld instead
if [ "$(xx-info arch)" = "arm64" ]; then
XX_CC_PREFER_LINKER=ld xx-clang --setup-target-triple
fi
EOT
RUN --mount=type=bind,target=.,rw \
--mount=type=tmpfs,target=cli/winresources/dockerd \
--mount=type=tmpfs,target=cli/winresources/docker-proxy \
--mount=type=cache,target=/root/.cache/go-build,id=moby-build-$TARGETPLATFORM <<EOT
set -e
target=$([ "$DOCKER_STATIC" = "1" ] && echo "binary" || echo "dynbinary")
xx-go --wrap
PKG_CONFIG=$(xx-go env PKG_CONFIG) ./hack/make.sh $target
xx-verify $([ "$DOCKER_STATIC" = "1" ] && echo "--static") /tmp/bundles/${target}-daemon/dockerd$([ "$(xx-info os)" = "windows" ] && echo ".exe")
xx-verify $([ "$DOCKER_STATIC" = "1" ] && echo "--static") /tmp/bundles/${target}-daemon/docker-proxy$([ "$(xx-info os)" = "windows" ] && echo ".exe")
mkdir /build
mv /tmp/bundles/${target}-daemon/* /build/
EOT
ENV DEFAULT_PRODUCT_LICENSE=${DEFAULT_PRODUCT_LICENSE}
ARG DOCKER_BUILDTAGS
ENV DOCKER_BUILDTAGS="${DOCKER_BUILDTAGS}"
ENV PREFIX=/build
# TODO: This is here because hack/make.sh binary copies these extras binaries
# from $PATH into the bundles dir.
# It would be nice to handle this in a different way.
COPY --from=tini /build/ /usr/local/bin/
COPY --from=runc /build/ /usr/local/bin/
COPY --from=containerd /build/ /usr/local/bin/
COPY --from=rootlesskit /build/ /usr/local/bin/
COPY --from=proxy /build/ /usr/local/bin/
COPY --from=vpnkit /build/ /usr/local/bin/
WORKDIR /go/src/github.com/docker/docker
FROM binary-base AS build-binary
RUN --mount=type=cache,target=/root/.cache/go-build \
--mount=type=bind,target=/go/src/github.com/docker/docker \
hack/make.sh binary
FROM binary-base AS build-dynbinary
RUN --mount=type=cache,target=/root/.cache/go-build \
--mount=type=bind,target=/go/src/github.com/docker/docker \
hack/make.sh dynbinary
FROM binary-base AS build-cross
ARG DOCKER_CROSSPLATFORMS
RUN --mount=type=cache,target=/root/.cache/go-build \
--mount=type=bind,target=/go/src/github.com/docker/docker \
--mount=type=tmpfs,target=/go/src/github.com/docker/docker/autogen \
hack/make.sh cross
# usage:
# > docker buildx bake binary
# > DOCKER_STATIC=0 docker buildx bake binary
# or
# > make binary
# > make dynbinary
FROM scratch AS binary
COPY --from=build /build/ /
COPY --from=build-binary /build/bundles/ /
# usage:
# > docker buildx bake all
FROM scratch AS all
COPY --link --from=tini /build/ /
COPY --link --from=runc /build/ /
COPY --link --from=containerd /build/ /
COPY --link --from=rootlesskit /build/ /
COPY --link --from=containerutil /build/ /
COPY --link --from=vpnkit / /
COPY --link --from=build /build /
FROM scratch AS dynbinary
COPY --from=build-dynbinary /build/bundles/ /
# smoke tests
# usage:
# > docker buildx bake binary-smoketest
FROM --platform=$TARGETPLATFORM base AS smoketest
WORKDIR /usr/local/bin
COPY --from=build /build .
RUN <<EOT
set -ex
file dockerd
dockerd --version
file docker-proxy
docker-proxy --version
EOT
FROM scratch AS cross
COPY --from=build-cross /build/bundles/ /
# devcontainer is a stage used by .devcontainer/devcontainer.json
FROM dev-base AS devcontainer
COPY --link . .
COPY --link --from=gopls /build/ /usr/local/bin/
# usage:
# > make shell
# > SYSTEMD=true make shell
FROM dev-base AS dev
COPY --link . .
FROM dev AS final
COPY . /go/src/github.com/docker/docker

84
Dockerfile.e2e Normal file
View file

@ -0,0 +1,84 @@
ARG GO_VERSION=1.20.10
FROM golang:${GO_VERSION}-alpine AS base
ENV GO111MODULE=off
RUN apk --no-cache add \
bash \
btrfs-progs-dev \
build-base \
curl \
lvm2-dev \
jq
RUN mkdir -p /build/
RUN mkdir -p /go/src/github.com/docker/docker/
WORKDIR /go/src/github.com/docker/docker/
FROM base AS frozen-images
# Get useful and necessary Hub images so we can "docker load" locally instead of pulling
COPY contrib/download-frozen-image-v2.sh /
RUN /download-frozen-image-v2.sh /build \
busybox:latest@sha256:95cf004f559831017cdf4628aaf1bb30133677be8702a8c5f2994629f637a209 \
busybox:latest@sha256:95cf004f559831017cdf4628aaf1bb30133677be8702a8c5f2994629f637a209 \
busybox:glibc@sha256:1f81263701cddf6402afe9f33fca0266d9fff379e59b1748f33d3072da71ee85 \
debian:bullseye-slim@sha256:dacf278785a4daa9de07596ec739dbc07131e189942772210709c5c0777e8437 \
hello-world:latest@sha256:d58e752213a51785838f9eed2b7a498ffa1cb3aa7f946dda11af39286c3db9a9
# See also frozenImages in "testutil/environment/protect.go" (which needs to be updated when adding images to this list)
FROM base AS dockercli
ENV INSTALL_BINARY_NAME=dockercli
COPY hack/dockerfile/install/install.sh ./install.sh
COPY hack/dockerfile/install/$INSTALL_BINARY_NAME.installer ./
RUN PREFIX=/build ./install.sh $INSTALL_BINARY_NAME
# Build DockerSuite.TestBuild* dependency
FROM base AS contrib
COPY contrib/syscall-test /build/syscall-test
COPY contrib/httpserver/Dockerfile /build/httpserver/Dockerfile
COPY contrib/httpserver contrib/httpserver
RUN CGO_ENABLED=0 go build -buildmode=pie -o /build/httpserver/httpserver github.com/docker/docker/contrib/httpserver
# Build the integration tests and copy the resulting binaries to /build/tests
FROM base AS builder
# Set tag and add sources
COPY . .
# Copy test sources tests that use assert can print errors
RUN mkdir -p /build${PWD} && find integration integration-cli -name \*_test.go -exec cp --parents '{}' /build${PWD} \;
# Build and install test binaries
ARG DOCKER_GITCOMMIT=undefined
RUN hack/make.sh build-integration-test-binary
RUN mkdir -p /build/tests && find . -name test.main -exec cp --parents '{}' /build/tests \;
## Generate testing image
FROM alpine:3.10 as runner
ENV DOCKER_REMOTE_DAEMON=1
ENV DOCKER_INTEGRATION_DAEMON_DEST=/
ENTRYPOINT ["/scripts/run.sh"]
# Add an unprivileged user to be used for tests which need it
RUN addgroup docker && adduser -D -G docker unprivilegeduser -s /bin/ash
# GNU tar is used for generating the emptyfs image
RUN apk --no-cache add \
bash \
ca-certificates \
g++ \
git \
iptables \
pigz \
tar \
xz
COPY hack/test/e2e-run.sh /scripts/run.sh
COPY hack/make/.ensure-emptyfs /scripts/ensure-emptyfs.sh
COPY integration/testdata /tests/integration/testdata
COPY integration/build/testdata /tests/integration/build/testdata
COPY integration-cli/fixtures /tests/integration-cli/fixtures
COPY --from=frozen-images /build/ /docker-frozen-images
COPY --from=dockercli /build/ /usr/bin/
COPY --from=contrib /build/ /tests/contrib/
COPY --from=builder /build/ /

View file

@ -5,14 +5,14 @@
# This represents the bare minimum required to build and test Docker.
ARG GO_VERSION=1.21.9
ARG GO_VERSION=1.20.10
ARG BASE_DEBIAN_DISTRO="bookworm"
ARG GOLANG_IMAGE="golang:${GO_VERSION}-${BASE_DEBIAN_DISTRO}"
FROM ${GOLANG_IMAGE}
FROM golang:${GO_VERSION}-buster
ENV GO111MODULE=off
ENV GOTOOLCHAIN=local
# allow replacing httpredir or deb mirror
ARG APT_MIRROR=deb.debian.org
RUN sed -ri "s/(httpredir|deb).debian.org/$APT_MIRROR/g" /etc/apt/sources.list
# Compile and runtime deps
# https://github.com/docker/docker/blob/master/project/PACKAGERS.md#build-dependencies
@ -21,8 +21,11 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
build-essential \
curl \
cmake \
gcc \
git \
libapparmor-dev \
libbtrfs-dev \
libdevmapper-dev \
libseccomp-dev \
ca-certificates \
e2fsprogs \
@ -33,6 +36,7 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
xfsprogs \
xz-utils \
\
aufs-tools \
vim-common \
&& rm -rf /var/lib/apt/lists/*

View file

@ -154,31 +154,29 @@
# The number of build steps below are explicitly minimised to improve performance.
ARG WINDOWS_BASE_IMAGE=mcr.microsoft.com/windows/servercore
ARG WINDOWS_BASE_IMAGE_TAG=ltsc2022
FROM ${WINDOWS_BASE_IMAGE}:${WINDOWS_BASE_IMAGE_TAG}
# Extremely important - do not change the following line to reference a "specific" image,
# such as `mcr.microsoft.com/windows/servercore:ltsc2019`. If using this Dockerfile in process
# isolated containers, the kernel of the host must match the container image, and hence
# would fail between Windows Server 2016 (aka RS1) and Windows Server 2019 (aka RS5).
# It is expected that the image `microsoft/windowsservercore:latest` is present, and matches
# the hosts kernel version before doing a build.
FROM microsoft/windowsservercore
# Use PowerShell as the default shell
SHELL ["powershell", "-Command", "$ErrorActionPreference = 'Stop'; $ProgressPreference = 'SilentlyContinue';"]
ARG GO_VERSION=1.21.9
ARG GO_VERSION=1.20.10
ARG GOTESTSUM_VERSION=v1.8.2
ARG GOWINRES_VERSION=v0.3.1
ARG CONTAINERD_VERSION=v1.7.15
# Environment variable notes:
# - GO_VERSION must be consistent with 'Dockerfile' used by Linux.
# - CONTAINERD_VERSION must be consistent with 'hack/dockerfile/install/containerd.installer' used by Linux.
# - FROM_DOCKERFILE is used for detection of building within a container.
ENV GO_VERSION=${GO_VERSION} `
CONTAINERD_VERSION=${CONTAINERD_VERSION} `
GIT_VERSION=2.11.1 `
GOPATH=C:\gopath `
GO111MODULE=off `
GOTOOLCHAIN=local `
FROM_DOCKERFILE=1 `
GOTESTSUM_VERSION=${GOTESTSUM_VERSION} `
GOWINRES_VERSION=${GOWINRES_VERSION}
GOTESTSUM_VERSION=${GOTESTSUM_VERSION}
RUN `
Function Test-Nano() { `
@ -213,15 +211,15 @@ RUN `
} `
} `
`
setx /M PATH $('C:\git\cmd;C:\git\usr\bin;'+$Env:PATH+';C:\gcc\bin;C:\go\bin;C:\containerd\bin'); `
setx /M PATH $('C:\git\cmd;C:\git\usr\bin;'+$Env:PATH+';C:\gcc\bin;C:\go\bin'); `
`
Write-Host INFO: Downloading git...; `
$location='https://www.nuget.org/api/v2/package/GitForWindows/'+$Env:GIT_VERSION; `
Download-File $location C:\gitsetup.zip; `
`
Write-Host INFO: Downloading go...; `
$dlGoVersion=$Env:GO_VERSION; `
Download-File "https://go.dev/dl/go${dlGoVersion}.windows-amd64.zip" C:\go.zip; `
$dlGoVersion=$Env:GO_VERSION -replace '\.0$',''; `
Download-File "https://golang.org/dl/go${dlGoVersion}.windows-amd64.zip" C:\go.zip; `
`
Write-Host INFO: Downloading compiler 1 of 3...; `
Download-File https://raw.githubusercontent.com/moby/docker-tdmgcc/master/gcc.zip C:\gcc.zip; `
@ -254,16 +252,6 @@ RUN `
Remove-Item C:\binutils.zip; `
Remove-Item C:\gitsetup.zip; `
`
Write-Host INFO: Downloading containerd; `
Install-Package -Force 7Zip4PowerShell; `
$location='https://github.com/containerd/containerd/releases/download/'+$Env:CONTAINERD_VERSION+'/containerd-'+$Env:CONTAINERD_VERSION.TrimStart('v')+'-windows-amd64.tar.gz'; `
Download-File $location C:\containerd.tar.gz; `
New-Item -Path C:\containerd -ItemType Directory; `
Expand-7Zip C:\containerd.tar.gz C:\; `
Expand-7Zip C:\containerd.tar C:\containerd; `
Remove-Item C:\containerd.tar.gz; `
Remove-Item C:\containerd.tar; `
`
# Ensure all directories exist that we will require below....
$srcDir = """$Env:GOPATH`\src\github.com\docker\docker\bundles"""; `
Write-Host INFO: Ensuring existence of directory $srcDir...; `
@ -288,22 +276,6 @@ RUN `
`
Install-GoTestSum
RUN `
Function Install-GoWinres() { `
$Env:GO111MODULE = 'on'; `
$tmpGobin = "${Env:GOBIN_TMP}"; `
$Env:GOBIN = """${Env:GOPATH}`\bin"""; `
Write-Host "INFO: Installing go-winres version $Env:GOWINRES_VERSION in $Env:GOBIN"; `
&go install "github.com/tc-hib/go-winres@${Env:GOWINRES_VERSION}"; `
$Env:GOBIN = "${tmpGobin}"; `
$Env:GO111MODULE = 'off'; `
if ($LASTEXITCODE -ne 0) { `
Throw '"go-winres install failed..."'; `
} `
} `
`
Install-GoWinres
# Make PowerShell the default entrypoint
ENTRYPOINT ["powershell.exe"]

560
Jenkinsfile vendored
View file

@ -8,14 +8,20 @@ pipeline {
timestamps()
}
parameters {
booleanParam(name: 'unit_validate', defaultValue: true, description: 'amd64 (x86_64) unit tests and vendor check')
booleanParam(name: 'validate_force', defaultValue: false, description: 'force validation steps to be run, even if no changes were detected')
booleanParam(name: 'amd64', defaultValue: true, description: 'amd64 (x86_64) Build/Test')
booleanParam(name: 'rootless', defaultValue: true, description: 'amd64 (x86_64) Build/Test (Rootless mode)')
booleanParam(name: 'cgroup2', defaultValue: true, description: 'amd64 (x86_64) Build/Test (cgroup v2)')
booleanParam(name: 'arm64', defaultValue: true, description: 'ARM (arm64) Build/Test')
booleanParam(name: 'windowsRS5', defaultValue: true, description: 'Windows 2019 (RS5) Build/Test')
booleanParam(name: 'dco', defaultValue: true, description: 'Run the DCO check')
}
environment {
DOCKER_BUILDKIT = '1'
DOCKER_EXPERIMENTAL = '1'
DOCKER_GRAPHDRIVER = 'overlay2'
CHECK_CONFIG_COMMIT = '33a3680e08d1007e72c3b3f1454f823d8e9948ee'
CHECK_CONFIG_COMMIT = '78405559cfe5987174aa2cb6463b9b2c1b917255'
TESTDEBUG = '0'
TIMEOUT = '120m'
}
@ -49,6 +55,490 @@ pipeline {
}
stage('Build') {
parallel {
stage('unit-validate') {
when {
beforeAgent true
expression { params.unit_validate }
}
agent { label 'amd64 && ubuntu-2004 && overlay2' }
environment {
// On master ("non-pull-request"), force running some validation checks (vendor, swagger),
// even if no files were changed. This allows catching problems caused by pull-requests
// that were merged out-of-sequence.
TEST_FORCE_VALIDATE = sh returnStdout: true, script: 'if [ "${BRANCH_NAME%%-*}" != "PR" ] || [ "${CHANGE_TARGET:-master}" != "master" ] || [ "${validate_force}" = "true" ]; then echo "1"; fi'
}
stages {
stage("Print info") {
steps {
sh 'docker version'
sh 'docker info'
sh '''
echo "check-config.sh version: ${CHECK_CONFIG_COMMIT}"
curl -fsSL -o ${WORKSPACE}/check-config.sh "https://raw.githubusercontent.com/moby/moby/${CHECK_CONFIG_COMMIT}/contrib/check-config.sh" \
&& bash ${WORKSPACE}/check-config.sh || true
'''
}
}
stage("Build dev image") {
steps {
sh 'docker build --force-rm --build-arg APT_MIRROR -t docker:${GIT_COMMIT} .'
}
}
stage("Validate") {
steps {
sh '''
docker run --rm -t --privileged \
-v "$WORKSPACE/bundles:/go/src/github.com/docker/docker/bundles" \
-v "$WORKSPACE/.git:/go/src/github.com/docker/docker/.git" \
--name docker-pr$BUILD_NUMBER \
-e DOCKER_EXPERIMENTAL \
-e DOCKER_GITCOMMIT=${GIT_COMMIT} \
-e DOCKER_GRAPHDRIVER \
-e TEST_FORCE_VALIDATE \
-e VALIDATE_REPO=${GIT_URL} \
-e VALIDATE_BRANCH=${CHANGE_TARGET} \
docker:${GIT_COMMIT} \
hack/validate/default
'''
}
}
stage("Docker-py") {
steps {
sh '''
docker run --rm -t --privileged \
-v "$WORKSPACE/bundles:/go/src/github.com/docker/docker/bundles" \
--name docker-pr$BUILD_NUMBER \
-e DOCKER_EXPERIMENTAL \
-e DOCKER_GITCOMMIT=${GIT_COMMIT} \
-e DOCKER_GRAPHDRIVER \
-e VALIDATE_REPO=${GIT_URL} \
-e VALIDATE_BRANCH=${CHANGE_TARGET} \
docker:${GIT_COMMIT} \
hack/make.sh \
dynbinary-daemon \
test-docker-py
'''
}
post {
always {
junit testResults: 'bundles/test-docker-py/junit-report.xml', allowEmptyResults: true
sh '''
echo "Ensuring container killed."
docker rm -vf docker-pr$BUILD_NUMBER || true
'''
sh '''
echo 'Chowning /workspace to jenkins user'
docker run --rm -v "$WORKSPACE:/workspace" busybox chown -R "$(id -u):$(id -g)" /workspace
'''
catchError(buildResult: 'SUCCESS', stageResult: 'FAILURE', message: 'Failed to create bundles.tar.gz') {
sh '''
bundleName=docker-py
echo "Creating ${bundleName}-bundles.tar.gz"
tar -czf ${bundleName}-bundles.tar.gz bundles/test-docker-py/*.xml bundles/test-docker-py/*.log
'''
archiveArtifacts artifacts: '*-bundles.tar.gz', allowEmptyArchive: true
}
}
}
}
stage("Static") {
steps {
sh '''
docker run --rm -t --privileged \
-v "$WORKSPACE/bundles:/go/src/github.com/docker/docker/bundles" \
--name docker-pr$BUILD_NUMBER \
-e DOCKER_GITCOMMIT=${GIT_COMMIT} \
-e DOCKER_GRAPHDRIVER \
docker:${GIT_COMMIT} \
hack/make.sh binary-daemon
'''
}
}
stage("Cross") {
steps {
sh '''
docker run --rm -t --privileged \
-v "$WORKSPACE/bundles:/go/src/github.com/docker/docker/bundles" \
--name docker-pr$BUILD_NUMBER \
-e DOCKER_GITCOMMIT=${GIT_COMMIT} \
-e DOCKER_GRAPHDRIVER \
docker:${GIT_COMMIT} \
hack/make.sh cross
'''
}
}
// needs to be last stage that calls make.sh for the junit report to work
stage("Unit tests") {
steps {
sh '''
docker run --rm -t --privileged \
-v "$WORKSPACE/bundles:/go/src/github.com/docker/docker/bundles" \
--name docker-pr$BUILD_NUMBER \
-e DOCKER_EXPERIMENTAL \
-e DOCKER_GITCOMMIT=${GIT_COMMIT} \
-e DOCKER_GRAPHDRIVER \
-e VALIDATE_REPO=${GIT_URL} \
-e VALIDATE_BRANCH=${CHANGE_TARGET} \
docker:${GIT_COMMIT} \
hack/test/unit
'''
}
post {
always {
junit testResults: 'bundles/junit-report.xml', allowEmptyResults: true
}
}
}
stage("Validate vendor") {
steps {
sh '''
docker run --rm -t --privileged \
-v "$WORKSPACE/.git:/go/src/github.com/docker/docker/.git" \
--name docker-pr$BUILD_NUMBER \
-e DOCKER_EXPERIMENTAL \
-e DOCKER_GITCOMMIT=${GIT_COMMIT} \
-e DOCKER_GRAPHDRIVER \
-e TEST_FORCE_VALIDATE \
-e VALIDATE_REPO=${GIT_URL} \
-e VALIDATE_BRANCH=${CHANGE_TARGET} \
docker:${GIT_COMMIT} \
hack/validate/vendor
'''
}
}
}
post {
always {
sh '''
echo 'Ensuring container killed.'
docker rm -vf docker-pr$BUILD_NUMBER || true
'''
sh '''
echo 'Chowning /workspace to jenkins user'
docker run --rm -v "$WORKSPACE:/workspace" busybox chown -R "$(id -u):$(id -g)" /workspace
'''
catchError(buildResult: 'SUCCESS', stageResult: 'FAILURE', message: 'Failed to create bundles.tar.gz') {
sh '''
bundleName=unit
echo "Creating ${bundleName}-bundles.tar.gz"
tar -czvf ${bundleName}-bundles.tar.gz bundles/junit-report.xml bundles/go-test-report.json bundles/profile.out
'''
archiveArtifacts artifacts: '*-bundles.tar.gz', allowEmptyArchive: true
}
}
cleanup {
sh 'make clean'
deleteDir()
}
}
}
stage('amd64') {
when {
beforeAgent true
expression { params.amd64 }
}
agent { label 'amd64 && ubuntu-2004 && overlay2' }
stages {
stage("Print info") {
steps {
sh 'docker version'
sh 'docker info'
sh '''
echo "check-config.sh version: ${CHECK_CONFIG_COMMIT}"
curl -fsSL -o ${WORKSPACE}/check-config.sh "https://raw.githubusercontent.com/moby/moby/${CHECK_CONFIG_COMMIT}/contrib/check-config.sh" \
&& bash ${WORKSPACE}/check-config.sh || true
'''
}
}
stage("Build dev image") {
steps {
sh '''
# todo: include ip_vs in base image
sudo modprobe ip_vs
docker build --force-rm --build-arg APT_MIRROR -t docker:${GIT_COMMIT} .
'''
}
}
stage("Run tests") {
steps {
sh '''#!/bin/bash
# bash is needed so 'jobs -p' works properly
# it also accepts setting inline envvars for functions without explicitly exporting
set -x
run_tests() {
[ -n "$TESTDEBUG" ] && rm= || rm=--rm;
docker run $rm -t --privileged \
-v "$WORKSPACE/bundles/${TEST_INTEGRATION_DEST}:/go/src/github.com/docker/docker/bundles" \
-v "$WORKSPACE/bundles/dynbinary-daemon:/go/src/github.com/docker/docker/bundles/dynbinary-daemon" \
-v "$WORKSPACE/.git:/go/src/github.com/docker/docker/.git" \
--name "$CONTAINER_NAME" \
-e KEEPBUNDLE=1 \
-e TESTDEBUG \
-e TESTFLAGS \
-e TEST_SKIP_INTEGRATION \
-e TEST_SKIP_INTEGRATION_CLI \
-e DOCKER_GITCOMMIT=${GIT_COMMIT} \
-e DOCKER_GRAPHDRIVER \
-e TIMEOUT \
-e VALIDATE_REPO=${GIT_URL} \
-e VALIDATE_BRANCH=${CHANGE_TARGET} \
docker:${GIT_COMMIT} \
hack/make.sh \
"$1" \
test-integration
}
trap "exit" INT TERM
trap 'pids=$(jobs -p); echo "Remaining pids to kill: [$pids]"; [ -z "$pids" ] || kill $pids' EXIT
CONTAINER_NAME=docker-pr$BUILD_NUMBER
docker run --rm -t --privileged \
-v "$WORKSPACE/bundles:/go/src/github.com/docker/docker/bundles" \
-v "$WORKSPACE/.git:/go/src/github.com/docker/docker/.git" \
--name ${CONTAINER_NAME}-build \
-e DOCKER_EXPERIMENTAL \
-e DOCKER_GITCOMMIT=${GIT_COMMIT} \
-e DOCKER_GRAPHDRIVER \
docker:${GIT_COMMIT} \
hack/make.sh \
dynbinary-daemon
# flaky + integration
TEST_INTEGRATION_DEST=1 CONTAINER_NAME=${CONTAINER_NAME}-1 TEST_SKIP_INTEGRATION_CLI=1 run_tests test-integration-flaky &
# integration-cli first set
TEST_INTEGRATION_DEST=2 CONTAINER_NAME=${CONTAINER_NAME}-2 TEST_SKIP_INTEGRATION=1 TESTFLAGS="-test.run Test(DockerSuite|DockerNetworkSuite|DockerHubPullSuite|DockerRegistrySuite|DockerSchema1RegistrySuite|DockerRegistryAuthTokenSuite|DockerRegistryAuthHtpasswdSuite)/" run_tests &
# integration-cli second set
TEST_INTEGRATION_DEST=3 CONTAINER_NAME=${CONTAINER_NAME}-3 TEST_SKIP_INTEGRATION=1 TESTFLAGS="-test.run Test(DockerSwarmSuite|DockerDaemonSuite|DockerExternalVolumeSuite)/" run_tests &
c=0
for job in $(jobs -p); do
wait ${job} || c=$?
done
exit $c
'''
}
post {
always {
junit testResults: 'bundles/**/*-report.xml', allowEmptyResults: true
}
}
}
}
post {
always {
sh '''
echo "Ensuring container killed."
cids=$(docker ps -aq -f name=docker-pr${BUILD_NUMBER}-*)
[ -n "$cids" ] && docker rm -vf $cids || true
'''
sh '''
echo "Chowning /workspace to jenkins user"
docker run --rm -v "$WORKSPACE:/workspace" busybox chown -R "$(id -u):$(id -g)" /workspace
'''
catchError(buildResult: 'SUCCESS', stageResult: 'FAILURE', message: 'Failed to create bundles.tar.gz') {
sh '''
bundleName=amd64
echo "Creating ${bundleName}-bundles.tar.gz"
# exclude overlay2 directories
find bundles -path '*/root/*overlay2' -prune -o -type f \\( -name '*-report.json' -o -name '*.log' -o -name '*.prof' -o -name '*-report.xml' \\) -print | xargs tar -czf ${bundleName}-bundles.tar.gz
'''
archiveArtifacts artifacts: '*-bundles.tar.gz', allowEmptyArchive: true
}
}
cleanup {
sh 'make clean'
deleteDir()
}
}
}
stage('rootless') {
when {
beforeAgent true
expression { params.rootless }
}
agent { label 'amd64 && ubuntu-2004 && overlay2' }
stages {
stage("Print info") {
steps {
sh 'docker version'
sh 'docker info'
sh '''
echo "check-config.sh version: ${CHECK_CONFIG_COMMIT}"
curl -fsSL -o ${WORKSPACE}/check-config.sh "https://raw.githubusercontent.com/moby/moby/${CHECK_CONFIG_COMMIT}/contrib/check-config.sh" \
&& bash ${WORKSPACE}/check-config.sh || true
'''
}
}
stage("Build dev image") {
steps {
sh '''
docker build --force-rm --build-arg APT_MIRROR -t docker:${GIT_COMMIT} .
'''
}
}
stage("Integration tests") {
environment {
DOCKER_ROOTLESS = '1'
TEST_SKIP_INTEGRATION_CLI = '1'
}
steps {
sh '''
docker run --rm -t --privileged \
-v "$WORKSPACE/bundles:/go/src/github.com/docker/docker/bundles" \
--name docker-pr$BUILD_NUMBER \
-e DOCKER_GITCOMMIT=${GIT_COMMIT} \
-e DOCKER_GRAPHDRIVER \
-e DOCKER_EXPERIMENTAL \
-e DOCKER_ROOTLESS \
-e TEST_SKIP_INTEGRATION_CLI \
-e TIMEOUT \
-e VALIDATE_REPO=${GIT_URL} \
-e VALIDATE_BRANCH=${CHANGE_TARGET} \
docker:${GIT_COMMIT} \
hack/make.sh \
dynbinary \
test-integration
'''
}
post {
always {
junit testResults: 'bundles/**/*-report.xml', allowEmptyResults: true
}
}
}
}
post {
always {
sh '''
echo "Ensuring container killed."
docker rm -vf docker-pr$BUILD_NUMBER || true
'''
sh '''
echo "Chowning /workspace to jenkins user"
docker run --rm -v "$WORKSPACE:/workspace" busybox chown -R "$(id -u):$(id -g)" /workspace
'''
catchError(buildResult: 'SUCCESS', stageResult: 'FAILURE', message: 'Failed to create bundles.tar.gz') {
sh '''
bundleName=amd64-rootless
echo "Creating ${bundleName}-bundles.tar.gz"
# exclude overlay2 directories
find bundles -path '*/root/*overlay2' -prune -o -type f \\( -name '*-report.json' -o -name '*.log' -o -name '*.prof' -o -name '*-report.xml' \\) -print | xargs tar -czf ${bundleName}-bundles.tar.gz
'''
archiveArtifacts artifacts: '*-bundles.tar.gz', allowEmptyArchive: true
}
}
cleanup {
sh 'make clean'
deleteDir()
}
}
}
stage('cgroup2') {
when {
beforeAgent true
expression { params.cgroup2 }
}
agent { label 'amd64 && ubuntu-2004 && cgroup2' }
stages {
stage("Print info") {
steps {
sh 'docker version'
sh 'docker info'
}
}
stage("Build dev image") {
steps {
sh '''
docker build --force-rm --build-arg APT_MIRROR --build-arg SYSTEMD=true -t docker:${GIT_COMMIT} .
'''
}
}
stage("Integration tests") {
environment {
DOCKER_SYSTEMD = '1' // recommended cgroup driver for v2
TEST_SKIP_INTEGRATION_CLI = '1' // CLI tests do not support v2
}
steps {
sh '''
docker run --rm -t --privileged \
-v "$WORKSPACE/bundles:/go/src/github.com/docker/docker/bundles" \
--name docker-pr$BUILD_NUMBER \
-e DOCKER_GITCOMMIT=${GIT_COMMIT} \
-e DOCKER_GRAPHDRIVER \
-e DOCKER_EXPERIMENTAL \
-e DOCKER_SYSTEMD \
-e TEST_SKIP_INTEGRATION_CLI \
-e TIMEOUT \
-e VALIDATE_REPO=${GIT_URL} \
-e VALIDATE_BRANCH=${CHANGE_TARGET} \
docker:${GIT_COMMIT} \
hack/make.sh \
dynbinary \
test-integration
'''
}
post {
always {
junit testResults: 'bundles/**/*-report.xml', allowEmptyResults: true
}
}
}
}
post {
always {
sh '''
echo "Ensuring container killed."
docker rm -vf docker-pr$BUILD_NUMBER || true
'''
sh '''
echo "Chowning /workspace to jenkins user"
docker run --rm -v "$WORKSPACE:/workspace" busybox chown -R "$(id -u):$(id -g)" /workspace
'''
catchError(buildResult: 'SUCCESS', stageResult: 'FAILURE', message: 'Failed to create bundles.tar.gz') {
sh '''
bundleName=amd64-cgroup2
echo "Creating ${bundleName}-bundles.tar.gz"
# exclude overlay2 directories
find bundles -path '*/root/*overlay2' -prune -o -type f \\( -name '*-report.json' -o -name '*.log' -o -name '*.prof' -o -name '*-report.xml' \\) -print | xargs tar -czf ${bundleName}-bundles.tar.gz
'''
archiveArtifacts artifacts: '*-bundles.tar.gz', allowEmptyArchive: true
}
}
cleanup {
sh 'make clean'
deleteDir()
}
}
}
stage('arm64') {
when {
beforeAgent true
@ -78,9 +568,6 @@ pipeline {
}
stage("Unit tests") {
steps {
sh '''
sudo modprobe ip6table_filter
'''
sh '''
docker run --rm -t --privileged \
-v "$WORKSPACE/bundles:/go/src/github.com/docker/docker/bundles" \
@ -96,7 +583,7 @@ pipeline {
}
post {
always {
junit testResults: 'bundles/junit-report*.xml', allowEmptyResults: true
junit testResults: 'bundles/junit-report.xml', allowEmptyResults: true
}
}
}
@ -111,7 +598,6 @@ pipeline {
-e DOCKER_GITCOMMIT=${GIT_COMMIT} \
-e DOCKER_GRAPHDRIVER \
-e TESTDEBUG \
-e TEST_INTEGRATION_USE_SNAPSHOTTER \
-e TEST_SKIP_INTEGRATION_CLI \
-e TIMEOUT \
-e VALIDATE_REPO=${GIT_URL} \
@ -159,6 +645,68 @@ pipeline {
}
}
}
stage('win-RS5') {
when {
beforeAgent true
expression { params.windowsRS5 }
}
environment {
DOCKER_BUILDKIT = '0'
DOCKER_DUT_DEBUG = '1'
SKIP_VALIDATION_TESTS = '1'
SOURCES_DRIVE = 'd'
SOURCES_SUBDIR = 'gopath'
TESTRUN_DRIVE = 'd'
TESTRUN_SUBDIR = "CI"
WINDOWS_BASE_IMAGE = 'mcr.microsoft.com/windows/servercore'
WINDOWS_BASE_IMAGE_TAG = 'ltsc2019'
}
agent {
node {
customWorkspace 'd:\\gopath\\src\\github.com\\docker\\docker'
label 'windows-2019'
}
}
stages {
stage("Print info") {
steps {
sh 'docker version'
sh 'docker info'
}
}
stage("Run tests") {
steps {
powershell '''
$ErrorActionPreference = 'Stop'
Invoke-WebRequest https://github.com/moby/docker-ci-zap/blob/master/docker-ci-zap.exe?raw=true -OutFile C:/Windows/System32/docker-ci-zap.exe
./hack/ci/windows.ps1
exit $LastExitCode
'''
}
}
}
post {
always {
junit testResults: 'bundles/junit-report-*.xml', allowEmptyResults: true
catchError(buildResult: 'SUCCESS', stageResult: 'FAILURE', message: 'Failed to create bundles.tar.gz') {
powershell '''
cd $env:WORKSPACE
$bundleName="windowsRS5-integration"
Write-Host -ForegroundColor Green "Creating ${bundleName}-bundles.zip"
# archiveArtifacts does not support env-vars to , so save the artifacts in a fixed location
Compress-Archive -Path "bundles/CIDUT.out", "bundles/CIDUT.err", "bundles/junit-report-*.xml" -CompressionLevel Optimal -DestinationPath "${bundleName}-bundles.zip"
'''
archiveArtifacts artifacts: '*-bundles.zip', allowEmptyArchive: true
}
}
cleanup {
sh 'make clean'
deleteDir()
}
}
}
}
}
}

View file

@ -24,23 +24,18 @@
# subsystem maintainers accountable. If ownership is unclear, they are the de facto owners.
people = [
"akerouanton",
"akihirosuda",
"anusha",
"coolljt0725",
"corhere",
"cpuguy83",
"crazy-max",
"crosbymichael",
"estesp",
"johnstep",
"justincormack",
"kolyshkin",
"laurazard",
"lowenna",
"mhbauer",
"neersighted",
"rumpl",
"runcom",
"samuelkarp",
"stevvooe",
"thajeztah",
"tianon",
@ -49,7 +44,6 @@
"unclejack",
"vdemeester",
"vieux",
"vvoland",
"yongtang"
]
@ -68,15 +62,11 @@
people = [
"alexellis",
"andrewhsu",
"bsousaa",
"dmcgowan",
"fntlnz",
"gianarb",
"olljanat",
"programmerq",
"ripcurld",
"robmry",
"sam-thibault",
"samwhited",
"thajeztah"
]
@ -113,17 +103,6 @@
# and tweets as @calavera.
"calavera",
# Michael Crosby was "chief maintainer" of the Docker project.
# During his time as a maintainer, Michael contributed to many
# milestones of the project; he was release captain of Docker v1.0.0,
# started the development of "libcontainer" (what later became runc)
# and containerd, as well as demoing cool hacks such as live migrating
# a game server container with checkpoint/restore.
#
# Michael is currently a maintainer of containerd, but you may see
# him around in other projects on GitHub.
"crosbymichael",
# Before becoming a maintainer, Daniel Nephin was a core contributor
# to "Fig" (now known as Docker Compose). As a maintainer for both the
# Engine and Docker CLI, Daniel contributed many features, among which
@ -187,16 +166,6 @@
# check out her open source projects on GitHub https://github.com/jessfraz (a must-try).
"jessfraz",
# As a maintainer, John Howard managed to make the impossible possible;
# to run Docker on Windows. After facing many challenges, teaching
# fellow-maintainers that 'Windows is not Linux', and many changes in
# Windows Server to facilitate containers, native Windows containers
# saw the light of day in 2015.
#
# John is now enjoying life without containers: playing piano, painting,
# and walking his dogs, but you may occasionally see him drop by on GitHub.
"lowenna",
# Alexander Morozov contributed many features to Docker, worked on the premise of
# what later became containerd (and worked on that too), and made a "stupid" Go
# vendor tool specifically for docker/docker needs: vndr (https://github.com/LK4D4/vndr).
@ -284,11 +253,6 @@
Email = "aaron.lehmann@docker.com"
GitHub = "aaronlehmann"
[people.akerouanton]
Name = "Albin Kerouanton"
Email = "albinker@gmail.com"
GitHub = "akerouanton"
[people.alexellis]
Name = "Alex Ellis"
Email = "alexellis2@gmail.com"
@ -318,11 +282,6 @@
Name = "Anusha Ragunathan"
Email = "anusha@docker.com"
GitHub = "anusha-ragunathan"
[people.bsousaa]
Name = "Bruno de Sousa"
Email = "bruno.sousa@docker.com"
GitHub = "bsousaa"
[people.calavera]
Name = "David Calavera"
@ -334,21 +293,11 @@
Email = "leijitang@huawei.com"
GitHub = "coolljt0725"
[people.corhere]
Name = "Cory Snider"
Email = "csnider@mirantis.com"
GitHub = "corhere"
[people.cpuguy83]
Name = "Brian Goff"
Email = "cpuguy83@gmail.com"
GitHub = "cpuguy83"
[people.crazy-max]
Name = "Kevin Alvarez"
Email = "contact@crazymax.dev"
GitHub = "crazy-max"
[people.crosbymichael]
Name = "Michael Crosby"
Email = "crosbymichael@gmail.com"
@ -359,11 +308,6 @@
Email = "dnephin@gmail.com"
GitHub = "dnephin"
[people.dmcgowan]
Name = "Derek McGowan"
Email = "derek@mcgstyle.net"
GitHub = "dmcgowan"
[people.duglin]
Name = "Doug Davis"
Email = "dug@us.ibm.com"
@ -424,11 +368,6 @@
Email = "kolyshkin@gmail.com"
GitHub = "kolyshkin"
[people.laurazard]
Name = "Laura Brehm"
Email = "laura.brehm@docker.com"
GitHub = "laurazard"
[people.lk4d4]
Name = "Alexander Morozov"
Email = "lk4d4@docker.com"
@ -464,11 +403,6 @@
Email = "mrjana@docker.com"
GitHub = "mrjana"
[people.neersighted]
Name = "Bjorn Neergaard"
Email = "bjorn@neersighted.com"
GitHub = "neersighted"
[people.olljanat]
Name = "Olli Janatuinen"
Email = "olli.janatuinen@gmail.com"
@ -479,36 +413,16 @@
Email = "jeff@docker.com"
GitHub = "programmerq"
[people.robmry]
Name = "Rob Murray"
Email = "rob.murray@docker.com"
GitHub = "robmry"
[people.ripcurld]
Name = "Boaz Shuster"
Email = "ripcurld.github@gmail.com"
GitHub = "ripcurld"
[people.rumpl]
Name = "Djordje Lukic"
Email = "djordje.lukic@docker.com"
GitHub = "rumpl"
[people.runcom]
Name = "Antonio Murdaca"
Email = "runcom@redhat.com"
GitHub = "runcom"
[people.sam-thibault]
Name = "Sam Thibault"
Email = "sam.thibault@docker.com"
GitHub = "sam-thibault"
[people.samuelkarp]
Name = "Samuel Karp"
Email = "me@samuelkarp.com"
GitHub = "samuelkarp"
[people.samwhited]
Name = "Sam Whited"
Email = "sam@samwhited.com"
@ -573,11 +487,6 @@
Name = "Vishnu Kannan"
Email = "vishnuk@google.com"
GitHub = "vishh"
[people.vvoland]
Name = "Paweł Gronowski"
Email = "pawel.gronowski@docker.com"
GitHub = "vvoland"
[people.yongtang]
Name = "Yong Tang"

169
Makefile
View file

@ -1,13 +1,31 @@
.PHONY: all binary dynbinary build cross help install manpages run shell test test-docker-py test-integration test-unit validate validate-% win
.PHONY: all binary dynbinary build cross help install manpages run shell test test-docker-py test-integration test-unit validate win
BUILDX_VERSION ?= v0.8.2
ifdef USE_BUILDX
BUILDX ?= $(shell command -v buildx)
BUILDX ?= $(shell command -v docker-buildx)
DOCKER_BUILDX_CLI_PLUGIN_PATH ?= ~/.docker/cli-plugins/docker-buildx
BUILDX ?= $(shell if [ -x "$(DOCKER_BUILDX_CLI_PLUGIN_PATH)" ]; then echo $(DOCKER_BUILDX_CLI_PLUGIN_PATH); fi)
endif
ifndef USE_BUILDX
DOCKER_BUILDKIT := 1
export DOCKER_BUILDKIT
endif
BUILDX ?= bundles/buildx
DOCKER ?= docker
BUILDX ?= $(DOCKER) buildx
# set the graph driver as the current graphdriver if not set
DOCKER_GRAPHDRIVER := $(if $(DOCKER_GRAPHDRIVER),$(DOCKER_GRAPHDRIVER),$(shell docker info -f '{{ .Driver }}' 2>&1))
DOCKER_GRAPHDRIVER := $(if $(DOCKER_GRAPHDRIVER),$(DOCKER_GRAPHDRIVER),$(shell docker info 2>&1 | grep "Storage Driver" | sed 's/.*: //'))
export DOCKER_GRAPHDRIVER
DOCKER_GITCOMMIT := $(shell git rev-parse HEAD)
# get OS/Arch of docker engine
DOCKER_OSARCH := $(shell bash -c 'source hack/make/.detect-daemon-osarch && echo $${DOCKER_ENGINE_OSARCH}')
DOCKERFILE := $(shell bash -c 'source hack/make/.detect-daemon-osarch && echo $${DOCKERFILE}')
DOCKER_GITCOMMIT := $(shell git rev-parse --short HEAD || echo unsupported)
export DOCKER_GITCOMMIT
# allow overriding the repository and branch that validation scripts are running
@ -16,9 +34,6 @@ export VALIDATE_REPO
export VALIDATE_BRANCH
export VALIDATE_ORIGIN_BRANCH
export PAGER
export GIT_PAGER
# env vars passed through directly to Docker's build scripts
# to allow things like `make KEEPBUNDLE=1 binary` easily
# `project/PACKAGERS.md` have some limited documentation of some of these
@ -27,9 +42,10 @@ export GIT_PAGER
# option of "go build". For example, a built-in graphdriver priority list
# can be changed during build time like this:
#
# make DOCKER_LDFLAGS="-X github.com/docker/docker/daemon/graphdriver.priority=overlay2,zfs" dynbinary
# make DOCKER_LDFLAGS="-X github.com/docker/docker/daemon/graphdriver.priority=overlay2,devicemapper" dynbinary
#
DOCKER_ENVS := \
-e DOCKER_CROSSPLATFORMS \
-e BUILDFLAGS \
-e KEEPBUNDLE \
-e DOCKER_BUILD_ARGS \
@ -39,10 +55,6 @@ DOCKER_ENVS := \
-e DOCKER_BUILDKIT \
-e DOCKER_BASH_COMPLETION_PATH \
-e DOCKER_CLI_PATH \
-e DOCKERCLI_VERSION \
-e DOCKERCLI_REPOSITORY \
-e DOCKERCLI_INTEGRATION_VERSION \
-e DOCKERCLI_INTEGRATION_REPOSITORY \
-e DOCKER_DEBUG \
-e DOCKER_EXPERIMENTAL \
-e DOCKER_GITCOMMIT \
@ -55,16 +67,10 @@ DOCKER_ENVS := \
-e DOCKER_TEST_HOST \
-e DOCKER_USERLANDPROXY \
-e DOCKERD_ARGS \
-e DELVE_PORT \
-e GITHUB_ACTIONS \
-e TEST_FORCE_VALIDATE \
-e TEST_INTEGRATION_DIR \
-e TEST_INTEGRATION_USE_SNAPSHOTTER \
-e TEST_INTEGRATION_FAIL_FAST \
-e TEST_SKIP_INTEGRATION \
-e TEST_SKIP_INTEGRATION_CLI \
-e TEST_IGNORE_CGROUP_CHECK \
-e TESTCOVERAGE \
-e TESTDEBUG \
-e TESTDIRS \
-e TESTFLAGS \
@ -75,16 +81,16 @@ DOCKER_ENVS := \
-e VALIDATE_REPO \
-e VALIDATE_BRANCH \
-e VALIDATE_ORIGIN_BRANCH \
-e HTTP_PROXY \
-e HTTPS_PROXY \
-e NO_PROXY \
-e http_proxy \
-e https_proxy \
-e no_proxy \
-e VERSION \
-e PLATFORM \
-e DEFAULT_PRODUCT_LICENSE \
-e PRODUCT \
-e PACKAGER_NAME \
-e PAGER \
-e GIT_PAGER \
-e OTEL_EXPORTER_OTLP_ENDPOINT \
-e OTEL_EXPORTER_OTLP_PROTOCOL \
-e OTEL_SERVICE_NAME
-e PRODUCT
# note: we _cannot_ add "-e DOCKER_BUILDTAGS" here because even if it's unset in the shell, that would shadow the "ENV DOCKER_BUILDTAGS" set in our Dockerfile, which is very important for our official builds
# to allow `make BIND_DIR=. shell` or `make BIND_DIR= test`
@ -102,7 +108,7 @@ DOCKER_MOUNT := $(if $(DOCKER_BINDDIR_MOUNT_OPTS),$(DOCKER_MOUNT):$(DOCKER_BINDD
# Note that `BIND_DIR` will already be set to `bundles` if `DOCKER_HOST` is not set (see above BIND_DIR line), in such case this will do nothing since `DOCKER_MOUNT` will already be set.
DOCKER_MOUNT := $(if $(DOCKER_MOUNT),$(DOCKER_MOUNT),-v /go/src/github.com/docker/docker/bundles) -v "$(CURDIR)/.git:/go/src/github.com/docker/docker/.git"
DOCKER_MOUNT_CACHE := -v docker-dev-cache:/root/.cache -v docker-mod-cache:/go/pkg/mod/
DOCKER_MOUNT_CACHE := -v docker-dev-cache:/root/.cache
DOCKER_MOUNT_CLI := $(if $(DOCKER_CLI_PATH),-v $(shell dirname $(DOCKER_CLI_PATH)):/usr/local/cli,)
DOCKER_MOUNT_BASH_COMPLETION := $(if $(DOCKER_BASH_COMPLETION_PATH),-v $(shell dirname $(DOCKER_BASH_COMPLETION_PATH)):/usr/local/completion/bash,)
DOCKER_MOUNT := $(DOCKER_MOUNT) $(DOCKER_MOUNT_CACHE) $(DOCKER_MOUNT_CLI) $(DOCKER_MOUNT_BASH_COMPLETION)
@ -111,11 +117,12 @@ endif # ifndef DOCKER_MOUNT
# This allows to set the docker-dev container name
DOCKER_CONTAINER_NAME := $(if $(CONTAINER_NAME),--name $(CONTAINER_NAME),)
DOCKER_IMAGE := docker-dev
GIT_BRANCH := $(shell git rev-parse --abbrev-ref HEAD 2>/dev/null)
GIT_BRANCH_CLEAN := $(shell echo $(GIT_BRANCH) | sed -e "s/[^[:alnum:]]/-/g")
DOCKER_IMAGE := docker-dev$(if $(GIT_BRANCH_CLEAN),:$(GIT_BRANCH_CLEAN))
DOCKER_PORT_FORWARD := $(if $(DOCKER_PORT),-p "$(DOCKER_PORT)",)
DELVE_PORT_FORWARD := $(if $(DELVE_PORT),-p "$(DELVE_PORT)",)
DOCKER_FLAGS := $(DOCKER) run --rm --privileged $(DOCKER_CONTAINER_NAME) $(DOCKER_ENVS) $(DOCKER_MOUNT) $(DOCKER_PORT_FORWARD) $(DELVE_PORT_FORWARD)
DOCKER_FLAGS := $(DOCKER) run --rm -i --privileged $(DOCKER_CONTAINER_NAME) $(DOCKER_ENVS) $(DOCKER_MOUNT) $(DOCKER_PORT_FORWARD)
SWAGGER_DOCS_PORT ?= 9000
@ -132,42 +139,53 @@ ifeq ($(INTERACTIVE), 1)
DOCKER_FLAGS += -t
endif
# on GitHub Runners input device is not a TTY but we allocate a pseudo-one,
# otherwise keep STDIN open even if not attached if not a GitHub Runner.
ifeq ($(GITHUB_ACTIONS),true)
DOCKER_FLAGS += -t
else
DOCKER_FLAGS += -i
endif
DOCKER_RUN_DOCKER := $(DOCKER_FLAGS) "$(DOCKER_IMAGE)"
DOCKER_BUILD_ARGS += --build-arg=GO_VERSION
DOCKER_BUILD_ARGS += --build-arg=DOCKERCLI_VERSION
DOCKER_BUILD_ARGS += --build-arg=DOCKERCLI_REPOSITORY
DOCKER_BUILD_ARGS += --build-arg=DOCKERCLI_INTEGRATION_VERSION
DOCKER_BUILD_ARGS += --build-arg=DOCKERCLI_INTEGRATION_REPOSITORY
ifdef DOCKER_SYSTEMD
DOCKER_BUILD_ARGS += --build-arg=SYSTEMD=true
endif
BUILD_OPTS := ${DOCKER_BUILD_ARGS} ${DOCKER_BUILD_OPTS}
BUILD_OPTS := ${DOCKER_BUILD_ARGS} ${DOCKER_BUILD_OPTS} -f "$(DOCKERFILE)"
ifdef USE_BUILDX
BUILD_OPTS += $(BUILDX_BUILD_EXTRA_OPTS)
BUILD_CMD := $(BUILDX) build
BAKE_CMD := $(BUILDX) bake
else
BUILD_CMD := $(DOCKER) build
endif
# This is used for the legacy "build" target and anything still depending on it
BUILD_CROSS =
ifdef DOCKER_CROSS
BUILD_CROSS = --build-arg CROSS=$(DOCKER_CROSS)
endif
ifdef DOCKER_CROSSPLATFORMS
BUILD_CROSS = --build-arg CROSS=true
endif
VERSION_AUTOGEN_ARGS = --build-arg VERSION --build-arg DOCKER_GITCOMMIT --build-arg PRODUCT --build-arg PLATFORM --build-arg DEFAULT_PRODUCT_LICENSE
default: binary
all: build ## validate all checks, build linux binaries, run all tests,\ncross build non-linux binaries, and generate archives
$(DOCKER_RUN_DOCKER) bash -c 'hack/validate/default && hack/make.sh'
binary: bundles ## build statically linked linux binaries
$(BAKE_CMD) binary
# This is only used to work around read-only bind mounts of the source code into
# binary build targets. We end up mounting a tmpfs over autogen which allows us
# to write build-time generated assets even though the source is mounted read-only
# ...But in order to do so, this dir needs to already exist.
autogen:
mkdir -p autogen
dynbinary: bundles ## build dynamically linked linux binaries
$(BAKE_CMD) dynbinary
binary: buildx autogen ## build statically linked linux binaries
$(BUILD_CMD) $(BUILD_OPTS) --output=bundles/ --target=$@ $(VERSION_AUTOGEN_ARGS) .
cross: bundles ## cross build the binaries
$(BAKE_CMD) binary-cross
dynbinary: buildx autogen ## build dynamically linked linux binaries
$(BUILD_CMD) $(BUILD_OPTS) --output=bundles/ --target=$@ $(VERSION_AUTOGEN_ARGS) .
cross: BUILD_OPTS += --build-arg CROSS=true --build-arg DOCKER_CROSSPLATFORMS
cross: buildx autogen ## cross build the binaries for darwin, freebsd and\nwindows
$(BUILD_CMD) $(BUILD_OPTS) --output=bundles/ --target=$@ $(VERSION_AUTOGEN_ARGS) .
bundles:
mkdir bundles
@ -176,8 +194,8 @@ bundles:
clean: clean-cache
.PHONY: clean-cache
clean-cache: ## remove the docker volumes that are used for caching in the dev-container
docker volume rm -f docker-dev-cache docker-mod-cache
clean-cache:
docker volume rm -f docker-dev-cache
help: ## this help
@awk 'BEGIN {FS = ":.*?## "} /^[a-zA-Z0-9_-]+:.*?## / {gsub("\\\\n",sprintf("\n%22c",""), $$2);printf "\033[36m%-20s\033[0m %s\n", $$1, $$2}' $(MAKEFILE_LIST)
@ -190,18 +208,21 @@ run: build ## run the docker daemon in a container
.PHONY: build
ifeq ($(BIND_DIR), .)
build: shell_target := --target=dev-base
else
build: shell_target := --target=dev
else
build: shell_target := --target=final
endif
build: bundles
$(BUILD_CMD) $(BUILD_OPTS) $(shell_target) --load -t "$(DOCKER_IMAGE)" .
ifdef USE_BUILDX
build: buildx_load := --load
endif
build: buildx
$(BUILD_CMD) $(BUILD_OPTS) $(shell_target) $(buildx_load) $(BUILD_CROSS) -t "$(DOCKER_IMAGE)" .
shell: build ## start a shell inside the build env
$(DOCKER_RUN_DOCKER) bash
test: build test-unit ## run the unit, integration and docker-py tests
$(DOCKER_RUN_DOCKER) hack/make.sh dynbinary test-integration test-docker-py
$(DOCKER_RUN_DOCKER) hack/make.sh dynbinary cross test-integration test-docker-py
test-docker-py: build ## run the docker-py tests
$(DOCKER_RUN_DOCKER) hack/make.sh dynbinary test-docker-py
@ -225,16 +246,8 @@ test-unit: build ## run the unit tests
validate: build ## validate DCO, Seccomp profile generation, gofmt,\n./pkg/ isolation, golint, tests, tomls, go vet and vendor
$(DOCKER_RUN_DOCKER) hack/validate/all
validate-generate-files:
$(BUILD_CMD) --target "validate" \
--output "type=cacheonly" \
--file "./hack/dockerfiles/generate-files.Dockerfile" .
validate-%: build ## validate specific check
$(DOCKER_RUN_DOCKER) hack/validate/$*
win: bundles ## cross build the binary for windows
$(BAKE_CMD) --set *.platform=windows/amd64 binary
win: build ## cross build the binary for windows
$(DOCKER_RUN_DOCKER) DOCKER_CROSSPLATFORMS=windows/amd64 hack/make.sh cross
.PHONY: swagger-gen
swagger-gen:
@ -250,17 +263,15 @@ swagger-docs: ## preview the API documentation
@docker run --rm -v $(PWD)/api/swagger.yaml:/usr/share/nginx/html/swagger.yaml \
-e 'REDOC_OPTIONS=hide-hostname="true" lazy-rendering' \
-p $(SWAGGER_DOCS_PORT):80 \
bfirsh/redoc:1.14.0
bfirsh/redoc:1.6.2
.PHONY: generate-files
generate-files:
$(eval $@_TMP_OUT := $(shell mktemp -d -t moby-output.XXXXXXXXXX))
@if [ -z "$($@_TMP_OUT)" ]; then \
echo "Temp dir is not set"; \
exit 1; \
fi
$(BUILD_CMD) --target "update" \
--output "type=local,dest=$($@_TMP_OUT)" \
--file "./hack/dockerfiles/generate-files.Dockerfile" .
cp -R "$($@_TMP_OUT)"/. .
rm -rf "$($@_TMP_OUT)"/*
.PHONY: buildx
ifdef USE_BUILDX
ifeq ($(BUILDX), bundles/buildx)
buildx: bundles/buildx ## build buildx cli tool
endif
endif
bundles/buildx: bundles ## build buildx CLI tool
curl -fsSL https://raw.githubusercontent.com/moby/buildkit/70deac12b5857a1aa4da65e90b262368e2f71500/hack/install-buildx | VERSION="$(BUILDX_VERSION)" BINDIR="$(@D)" bash
$@ version

View file

@ -14,7 +14,7 @@ Moby is an open project guided by strong principles, aiming to be modular, flexi
It is open to the community to help set its direction.
- Modular: the project includes lots of components that have well-defined functions and APIs that work together.
- Batteries included but swappable: Moby includes enough components to build fully featured container systems, but its modular architecture ensures that most of the components can be swapped by different implementations.
- Batteries included but swappable: Moby includes enough components to build fully featured container system, but its modular architecture ensures that most of the components can be swapped by different implementations.
- Usable security: Moby provides secure defaults without compromising usability.
- Developer focused: The APIs are intended to be functional and useful to build powerful tools.
They are not necessarily intended as end user tools but as components aimed at developers.

View file

@ -37,6 +37,6 @@ There is hopefully enough example material in the file for you to copy a similar
When you make edits to `swagger.yaml`, you may want to check the generated API documentation to ensure it renders correctly.
Run `make swagger-docs` and a preview will be running at `http://localhost:9000`. Some of the styling may be incorrect, but you'll be able to ensure that it is generating the correct documentation.
Run `make swagger-docs` and a preview will be running at `http://localhost`. Some of the styling may be incorrect, but you'll be able to ensure that it is generating the correct documentation.
The production documentation is generated by vendoring `swagger.yaml` into [docker/docker.github.io](https://github.com/docker/docker.github.io).

View file

@ -2,17 +2,8 @@ package api // import "github.com/docker/docker/api"
// Common constants for daemon and client.
const (
// DefaultVersion of the current REST API.
DefaultVersion = "1.45"
// MinSupportedAPIVersion is the minimum API version that can be supported
// by the API server, specified as "major.minor". Note that the daemon
// may be configured with a different minimum API version, as returned
// in [github.com/docker/docker/api/types.Version.MinAPIVersion].
//
// API requests for API versions lower than the configured version produce
// an error.
MinSupportedAPIVersion = "1.24"
// DefaultVersion of Current REST API
DefaultVersion = "1.41"
// NoBaseImageSpecifier is the symbol used by the FROM
// command to specify that no base image is to be used.

7
api/common_unix.go Normal file
View file

@ -0,0 +1,7 @@
//go:build !windows
// +build !windows
package api // import "github.com/docker/docker/api"
// MinVersion represents Minimum REST API version supported
const MinVersion = "1.12"

8
api/common_windows.go Normal file
View file

@ -0,0 +1,8 @@
package api // import "github.com/docker/docker/api"
// MinVersion represents Minimum REST API version supported
// Technically the first daemon API version released on Windows is v1.25 in
// engine version 1.13. However, some clients are explicitly using downlevel
// APIs (e.g. docker-compose v2.1 file format) and that is just too restrictive.
// Hence also allowing 1.24 on Windows.
const MinVersion string = "1.24"

View file

@ -5,7 +5,7 @@ import (
"fmt"
"strconv"
"github.com/distribution/reference"
"github.com/docker/distribution/reference"
"github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/backend"
"github.com/docker/docker/api/types/events"
@ -21,7 +21,7 @@ import (
// ImageComponent provides an interface for working with images
type ImageComponent interface {
SquashImage(from string, to string) (string, error)
TagImage(context.Context, image.ID, reference.Named) error
TagImageWithReference(image.ID, reference.Named) error
}
// Builder defines interface for running a build
@ -54,7 +54,7 @@ func (b *Backend) Build(ctx context.Context, config backend.BuildConfig) (string
options := config.Options
useBuildKit := options.Version == types.BuilderBuildKit
tags, err := sanitizeRepoAndTags(options.Tags)
tagger, err := NewTagger(b.imageComponent, config.ProgressWriter.StdoutFormatter, options.Tags)
if err != nil {
return "", err
}
@ -76,7 +76,7 @@ func (b *Backend) Build(ctx context.Context, config backend.BuildConfig) (string
return "", nil
}
imageID := build.ImageID
var imageID = build.ImageID
if options.Squash {
if imageID, err = squashBuild(build, b.imageComponent); err != nil {
return "", err
@ -92,8 +92,8 @@ func (b *Backend) Build(ctx context.Context, config backend.BuildConfig) (string
stdout := config.ProgressWriter.StdoutFormatter
fmt.Fprintf(stdout, "Successfully built %s\n", stringid.TruncateID(imageID))
}
if imageID != "" && !useBuildKit {
err = tagImages(ctx, b.imageComponent, config.ProgressWriter.StdoutFormatter, image.ID(imageID), tags)
if imageID != "" {
err = tagger.TagImages(image.ID(imageID))
}
return imageID, err
}
@ -104,7 +104,7 @@ func (b *Backend) PruneCache(ctx context.Context, opts types.BuildCachePruneOpti
if err != nil {
return nil, errors.Wrap(err, "failed to prune build cache")
}
b.eventsService.Log(events.ActionPrune, events.BuilderEventType, events.Actor{
b.eventsService.Log("prune", events.BuilderEventType, events.Actor{
Attributes: map[string]string{
"reclaimed": strconv.FormatInt(buildCacheSize, 10),
},

View file

@ -1,31 +1,55 @@
package build // import "github.com/docker/docker/api/server/backend/build"
import (
"context"
"fmt"
"io"
"github.com/distribution/reference"
"github.com/docker/distribution/reference"
"github.com/docker/docker/image"
"github.com/pkg/errors"
)
// tagImages creates image tags for the imageID.
func tagImages(ctx context.Context, ic ImageComponent, stdout io.Writer, imageID image.ID, repoAndTags []reference.Named) error {
for _, rt := range repoAndTags {
if err := ic.TagImage(ctx, imageID, rt); err != nil {
// Tagger is responsible for tagging an image created by a builder
type Tagger struct {
imageComponent ImageComponent
stdout io.Writer
repoAndTags []reference.Named
}
// NewTagger returns a new Tagger for tagging the images of a build.
// If any of the names are invalid tags an error is returned.
func NewTagger(backend ImageComponent, stdout io.Writer, names []string) (*Tagger, error) {
reposAndTags, err := sanitizeRepoAndTags(names)
if err != nil {
return nil, err
}
return &Tagger{
imageComponent: backend,
stdout: stdout,
repoAndTags: reposAndTags,
}, nil
}
// TagImages creates image tags for the imageID
func (bt *Tagger) TagImages(imageID image.ID) error {
for _, rt := range bt.repoAndTags {
if err := bt.imageComponent.TagImageWithReference(imageID, rt); err != nil {
return err
}
_, _ = fmt.Fprintln(stdout, "Successfully tagged", reference.FamiliarString(rt))
fmt.Fprintf(bt.stdout, "Successfully tagged %s\n", reference.FamiliarString(rt))
}
return nil
}
// sanitizeRepoAndTags parses the raw "t" parameter received from the client
// to a slice of repoAndTag. It removes duplicates, and validates each name
// to not contain a digest.
func sanitizeRepoAndTags(names []string) (repoAndTags []reference.Named, err error) {
uniqNames := map[string]struct{}{}
// to a slice of repoAndTag.
// It also validates each repoName and tag.
func sanitizeRepoAndTags(names []string) ([]reference.Named, error) {
var (
repoAndTags []reference.Named
// This map is used for deduplicating the "-t" parameter.
uniqNames = make(map[string]struct{})
)
for _, repo := range names {
if repo == "" {
continue
@ -36,12 +60,14 @@ func sanitizeRepoAndTags(names []string) (repoAndTags []reference.Named, err err
return nil, err
}
if _, ok := ref.(reference.Digested); ok {
if _, isCanonical := ref.(reference.Canonical); isCanonical {
return nil, errors.New("build tag cannot contain a digest")
}
ref = reference.TagNameOnly(ref)
nameWithTag := ref.String()
if _, exists := uniqNames[nameWithTag]; !exists {
uniqNames[nameWithTag] = struct{}{}
repoAndTags = append(repoAndTags, ref)

View file

@ -0,0 +1,34 @@
package server
import (
"net/http"
"github.com/docker/docker/api/server/httpstatus"
"github.com/docker/docker/api/server/httputils"
"github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/versions"
"github.com/gorilla/mux"
"google.golang.org/grpc/status"
)
// makeErrorHandler makes an HTTP handler that decodes a Docker error and
// returns it in the response.
func makeErrorHandler(err error) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
statusCode := httpstatus.FromError(err)
vars := mux.Vars(r)
if apiVersionSupportsJSONErrors(vars["version"]) {
response := &types.ErrorResponse{
Message: err.Error(),
}
_ = httputils.WriteJSON(w, statusCode, response)
} else {
http.Error(w, status.Convert(err).Message(), statusCode)
}
}
}
func apiVersionSupportsJSONErrors(version string) bool {
const firstAPIVersionWithJSONErrors = "1.23"
return version == "" || versions.GreaterThan(version, firstAPIVersionWithJSONErrors)
}

View file

@ -1,14 +1,13 @@
package httpstatus // import "github.com/docker/docker/api/server/httpstatus"
import (
"context"
"fmt"
"net/http"
cerrdefs "github.com/containerd/containerd/errdefs"
"github.com/containerd/log"
containerderrors "github.com/containerd/containerd/errdefs"
"github.com/docker/distribution/registry/api/errcode"
"github.com/docker/docker/errdefs"
"github.com/sirupsen/logrus"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/status"
)
@ -20,7 +19,7 @@ type causer interface {
// FromError retrieves status code from error message.
func FromError(err error) int {
if err == nil {
log.G(context.TODO()).WithError(err).Error("unexpected HTTP error handling")
logrus.WithFields(logrus.Fields{"error": err}).Error("unexpected HTTP error handling")
return http.StatusInternalServerError
}
@ -66,11 +65,10 @@ func FromError(err error) int {
return FromError(e.Cause())
}
log.G(context.TODO()).WithFields(log.Fields{
logrus.WithFields(logrus.Fields{
"module": "api",
"error": err,
"error_type": fmt.Sprintf("%T", err),
}).Debug("FIXME: Got an API for which error does not match any expected type!!!")
}).Debugf("FIXME: Got an API for which error does not match any expected type!!!: %+v", err)
}
if statusCode == 0 {
@ -134,17 +132,17 @@ func statusCodeFromDistributionError(err error) int {
// consumed directly (not through gRPC)
func statusCodeFromContainerdError(err error) int {
switch {
case cerrdefs.IsInvalidArgument(err):
case containerderrors.IsInvalidArgument(err):
return http.StatusBadRequest
case cerrdefs.IsNotFound(err):
case containerderrors.IsNotFound(err):
return http.StatusNotFound
case cerrdefs.IsAlreadyExists(err):
case containerderrors.IsAlreadyExists(err):
return http.StatusConflict
case cerrdefs.IsFailedPrecondition(err):
case containerderrors.IsFailedPrecondition(err):
return http.StatusPreconditionFailed
case cerrdefs.IsUnavailable(err):
case containerderrors.IsUnavailable(err):
return http.StatusServiceUnavailable
case cerrdefs.IsNotImplemented(err):
case containerderrors.IsNotImplemented(err):
return http.StatusNotImplemented
default:
return http.StatusInternalServerError

View file

@ -12,4 +12,5 @@ import (
// container configuration.
type ContainerDecoder interface {
DecodeConfig(src io.Reader) (*container.Config, *container.HostConfig, *network.NetworkingConfig, error)
DecodeHostConfig(src io.Reader) (*container.HostConfig, error)
}

View file

@ -1,12 +1,9 @@
package httputils // import "github.com/docker/docker/api/server/httputils"
import (
"fmt"
"net/http"
"strconv"
"strings"
"github.com/distribution/reference"
)
// BoolValue transforms a form value in different formats into a boolean type.
@ -44,38 +41,6 @@ func Int64ValueOrDefault(r *http.Request, field string, def int64) (int64, error
return def, nil
}
// RepoTagReference parses form values "repo" and "tag" and returns a valid
// reference with repository and tag.
// If repo is empty, then a nil reference is returned.
// If no tag is given, then the default "latest" tag is set.
func RepoTagReference(repo, tag string) (reference.NamedTagged, error) {
if repo == "" {
return nil, nil
}
ref, err := reference.ParseNormalizedNamed(repo)
if err != nil {
return nil, err
}
if _, isDigested := ref.(reference.Digested); isDigested {
return nil, fmt.Errorf("cannot import digest reference")
}
if tag != "" {
return reference.WithTag(ref, tag)
}
withDefaultTag := reference.TagNameOnly(ref)
namedTagged, ok := withDefaultTag.(reference.NamedTagged)
if !ok {
return nil, fmt.Errorf("unexpected reference: %q", ref.String())
}
return namedTagged, nil
}
// ArchiveOptions stores archive information for different operations.
type ArchiveOptions struct {
Name string

View file

@ -2,7 +2,6 @@ package httputils // import "github.com/docker/docker/api/server/httputils"
import (
"context"
"encoding/json"
"io"
"mime"
"net/http"
@ -10,6 +9,7 @@ import (
"github.com/docker/docker/errdefs"
"github.com/pkg/errors"
"github.com/sirupsen/logrus"
)
// APIVersionKey is the client's requested API version.
@ -49,50 +49,17 @@ func CheckForJSON(r *http.Request) error {
ct := r.Header.Get("Content-Type")
// No Content-Type header is ok as long as there's no Body
if ct == "" && (r.Body == nil || r.ContentLength == 0) {
return nil
if ct == "" {
if r.Body == nil || r.ContentLength == 0 {
return nil
}
}
// Otherwise it better be json
return matchesContentType(ct, "application/json")
}
// ReadJSON validates the request to have the correct content-type, and decodes
// the request's Body into out.
func ReadJSON(r *http.Request, out interface{}) error {
err := CheckForJSON(r)
if err != nil {
return err
}
if r.Body == nil || r.ContentLength == 0 {
// an empty body is not invalid, so don't return an error; see
// https://lists.w3.org/Archives/Public/ietf-http-wg/2010JulSep/0272.html
if matchesContentType(ct, "application/json") {
return nil
}
dec := json.NewDecoder(r.Body)
err = dec.Decode(out)
defer r.Body.Close()
if err != nil {
if err == io.EOF {
return errdefs.InvalidParameter(errors.New("invalid JSON: got EOF while reading request body"))
}
return errdefs.InvalidParameter(errors.Wrap(err, "invalid JSON"))
}
if dec.More() {
return errdefs.InvalidParameter(errors.New("unexpected content after JSON"))
}
return nil
}
// WriteJSON writes the value v to the http response stream as json with standard json encoding.
func WriteJSON(w http.ResponseWriter, code int, v interface{}) error {
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(code)
enc := json.NewEncoder(w)
enc.SetEscapeHTML(false)
return enc.Encode(v)
return errdefs.InvalidParameter(errors.Errorf("Content-Type specified (%s) must be 'application/json'", ct))
}
// ParseForm ensures the request form is parsed even with invalid content types.
@ -122,13 +89,10 @@ func VersionFromContext(ctx context.Context) string {
}
// matchesContentType validates the content type against the expected one
func matchesContentType(contentType, expectedType string) error {
func matchesContentType(contentType, expectedType string) bool {
mimetype, _, err := mime.ParseMediaType(contentType)
if err != nil {
return errdefs.InvalidParameter(errors.Wrapf(err, "malformed Content-Type header (%s)", contentType))
logrus.Errorf("Error parsing media type: %s error: %v", contentType, err)
}
if mimetype != expectedType {
return errdefs.InvalidParameter(errors.Errorf("unsupported Content-Type header (%s): must be '%s'", contentType, expectedType))
}
return nil
return err == nil && mimetype == expectedType
}

View file

@ -1,130 +1,18 @@
package httputils // import "github.com/docker/docker/api/server/httputils"
import (
"net/http"
"strings"
"testing"
)
import "testing"
// matchesContentType
func TestJsonContentType(t *testing.T) {
err := matchesContentType("application/json", "application/json")
if err != nil {
t.Error(err)
if !matchesContentType("application/json", "application/json") {
t.Fail()
}
err = matchesContentType("application/json; charset=utf-8", "application/json")
if err != nil {
t.Error(err)
if !matchesContentType("application/json; charset=utf-8", "application/json") {
t.Fail()
}
expected := "unsupported Content-Type header (dockerapplication/json): must be 'application/json'"
err = matchesContentType("dockerapplication/json", "application/json")
if err == nil || err.Error() != expected {
t.Errorf(`expected "%s", got "%v"`, expected, err)
}
expected = "malformed Content-Type header (foo;;;bar): mime: invalid media parameter"
err = matchesContentType("foo;;;bar", "application/json")
if err == nil || err.Error() != expected {
t.Errorf(`expected "%s", got "%v"`, expected, err)
if matchesContentType("dockerapplication/json", "application/json") {
t.Fail()
}
}
func TestReadJSON(t *testing.T) {
t.Run("nil body", func(t *testing.T) {
req, err := http.NewRequest(http.MethodPost, "https://example.com/some/path", nil)
if err != nil {
t.Error(err)
}
foo := struct{}{}
err = ReadJSON(req, &foo)
if err != nil {
t.Error(err)
}
})
t.Run("empty body", func(t *testing.T) {
req, err := http.NewRequest(http.MethodPost, "https://example.com/some/path", strings.NewReader(""))
if err != nil {
t.Error(err)
}
foo := struct{ SomeField string }{}
err = ReadJSON(req, &foo)
if err != nil {
t.Error(err)
}
if foo.SomeField != "" {
t.Errorf("expected: '', got: %s", foo.SomeField)
}
})
t.Run("with valid request", func(t *testing.T) {
req, err := http.NewRequest(http.MethodPost, "https://example.com/some/path", strings.NewReader(`{"SomeField":"some value"}`))
if err != nil {
t.Error(err)
}
req.Header.Set("Content-Type", "application/json")
foo := struct{ SomeField string }{}
err = ReadJSON(req, &foo)
if err != nil {
t.Error(err)
}
if foo.SomeField != "some value" {
t.Errorf("expected: 'some value', got: %s", foo.SomeField)
}
})
t.Run("with whitespace", func(t *testing.T) {
req, err := http.NewRequest(http.MethodPost, "https://example.com/some/path", strings.NewReader(`
{"SomeField":"some value"}
`))
if err != nil {
t.Error(err)
}
req.Header.Set("Content-Type", "application/json")
foo := struct{ SomeField string }{}
err = ReadJSON(req, &foo)
if err != nil {
t.Error(err)
}
if foo.SomeField != "some value" {
t.Errorf("expected: 'some value', got: %s", foo.SomeField)
}
})
t.Run("with extra content", func(t *testing.T) {
req, err := http.NewRequest(http.MethodPost, "https://example.com/some/path", strings.NewReader(`{"SomeField":"some value"} and more content`))
if err != nil {
t.Error(err)
}
req.Header.Set("Content-Type", "application/json")
foo := struct{ SomeField string }{}
err = ReadJSON(req, &foo)
if err == nil {
t.Error("expected an error, got none")
}
expected := "unexpected content after JSON"
if err.Error() != expected {
t.Errorf("expected: '%s', got: %s", expected, err.Error())
}
})
t.Run("invalid JSON", func(t *testing.T) {
req, err := http.NewRequest(http.MethodPost, "https://example.com/some/path", strings.NewReader(`{invalid json`))
if err != nil {
t.Error(err)
}
req.Header.Set("Content-Type", "application/json")
foo := struct{ SomeField string }{}
err = ReadJSON(req, &foo)
if err == nil {
t.Error("expected an error, got none")
}
expected := "invalid JSON: invalid character 'i' looking for beginning of object key string"
if err.Error() != expected {
t.Errorf("expected: '%s', got: %s", expected, err.Error())
}
})
}

View file

@ -0,0 +1,15 @@
package httputils // import "github.com/docker/docker/api/server/httputils"
import (
"encoding/json"
"net/http"
)
// WriteJSON writes the value v to the http response stream as json with standard json encoding.
func WriteJSON(w http.ResponseWriter, code int, v interface{}) error {
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(code)
enc := json.NewEncoder(w)
enc.SetEscapeHTML(false)
return enc.Encode(v)
}

View file

@ -7,8 +7,8 @@ import (
"net/url"
"sort"
"github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/backend"
"github.com/docker/docker/api/types/container"
"github.com/docker/docker/pkg/ioutils"
"github.com/docker/docker/pkg/jsonmessage"
"github.com/docker/docker/pkg/stdcopy"
@ -16,7 +16,7 @@ import (
// WriteLogStream writes an encoded byte stream of log messages from the
// messages channel, multiplexing them with a stdcopy.Writer if mux is true
func WriteLogStream(_ context.Context, w io.Writer, msgs <-chan *backend.LogMessage, config *container.LogsOptions, mux bool) {
func WriteLogStream(_ context.Context, w io.Writer, msgs <-chan *backend.LogMessage, config *types.ContainerLogsOptions, mux bool) {
wf := ioutils.NewWriteFlusher(w)
defer wf.Close()

View file

@ -1,9 +1,9 @@
package server // import "github.com/docker/docker/api/server"
import (
"github.com/containerd/log"
"github.com/docker/docker/api/server/httputils"
"github.com/docker/docker/api/server/middleware"
"github.com/sirupsen/logrus"
)
// handlerWithGlobalMiddlewares wraps the handler function for a request with
@ -16,7 +16,7 @@ func (s *Server) handlerWithGlobalMiddlewares(handler httputils.APIFunc) httputi
next = m.WrapHandler(next)
}
if log.GetLevel() == log.DebugLevel {
if s.cfg.Logging && logrus.GetLevel() == logrus.DebugLevel {
next = middleware.DebugRequestMiddleware(next)
}

View file

@ -4,8 +4,7 @@ import (
"context"
"net/http"
"github.com/containerd/log"
"github.com/docker/docker/api/types/registry"
"github.com/sirupsen/logrus"
)
// CORSMiddleware injects CORS headers to each request
@ -29,9 +28,9 @@ func (c CORSMiddleware) WrapHandler(handler func(ctx context.Context, w http.Res
corsHeaders = "*"
}
log.G(ctx).Debugf("CORS header is enabled and set to: %s", corsHeaders)
logrus.Debugf("CORS header is enabled and set to: %s", corsHeaders)
w.Header().Add("Access-Control-Allow-Origin", corsHeaders)
w.Header().Add("Access-Control-Allow-Headers", "Origin, X-Requested-With, Content-Type, Accept, "+registry.AuthHeader)
w.Header().Add("Access-Control-Allow-Headers", "Origin, X-Requested-With, Content-Type, Accept, X-Registry-Auth")
w.Header().Add("Access-Control-Allow-Methods", "HEAD, GET, POST, DELETE, PUT, OPTIONS")
return handler(ctx, w, r, vars)
}

View file

@ -8,15 +8,15 @@ import (
"net/http"
"strings"
"github.com/containerd/log"
"github.com/docker/docker/api/server/httputils"
"github.com/docker/docker/pkg/ioutils"
"github.com/sirupsen/logrus"
)
// DebugRequestMiddleware dumps the request to logger
func DebugRequestMiddleware(handler func(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error) func(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
return func(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
log.G(ctx).Debugf("Calling %s %s", r.Method, r.RequestURI)
logrus.Debugf("Calling %s %s", r.Method, r.RequestURI)
if r.Method != http.MethodPost {
return handler(ctx, w, r, vars)
@ -44,9 +44,9 @@ func DebugRequestMiddleware(handler func(ctx context.Context, w http.ResponseWri
maskSecretKeys(postForm)
formStr, errMarshal := json.Marshal(postForm)
if errMarshal == nil {
log.G(ctx).Debugf("form data: %s", string(formStr))
logrus.Debugf("form data: %s", string(formStr))
} else {
log.G(ctx).Debugf("form data: %q", postForm)
logrus.Debugf("form data: %q", postForm)
}
}

View file

@ -6,7 +6,6 @@ import (
"net/http"
"runtime"
"github.com/docker/docker/api"
"github.com/docker/docker/api/server/httputils"
"github.com/docker/docker/api/types/versions"
)
@ -14,40 +13,19 @@ import (
// VersionMiddleware is a middleware that
// validates the client and server versions.
type VersionMiddleware struct {
serverVersion string
// defaultAPIVersion is the default API version provided by the API server,
// specified as "major.minor". It is usually configured to the latest API
// version [github.com/docker/docker/api.DefaultVersion].
//
// API requests for API versions greater than this version are rejected by
// the server and produce a [versionUnsupportedError].
defaultAPIVersion string
// minAPIVersion is the minimum API version provided by the API server,
// specified as "major.minor".
//
// API requests for API versions lower than this version are rejected by
// the server and produce a [versionUnsupportedError].
minAPIVersion string
serverVersion string
defaultVersion string
minVersion string
}
// NewVersionMiddleware creates a VersionMiddleware with the given versions.
func NewVersionMiddleware(serverVersion, defaultAPIVersion, minAPIVersion string) (*VersionMiddleware, error) {
if versions.LessThan(defaultAPIVersion, api.MinSupportedAPIVersion) || versions.GreaterThan(defaultAPIVersion, api.DefaultVersion) {
return nil, fmt.Errorf("invalid default API version (%s): must be between %s and %s", defaultAPIVersion, api.MinSupportedAPIVersion, api.DefaultVersion)
// NewVersionMiddleware creates a new VersionMiddleware
// with the default versions.
func NewVersionMiddleware(s, d, m string) VersionMiddleware {
return VersionMiddleware{
serverVersion: s,
defaultVersion: d,
minVersion: m,
}
if versions.LessThan(minAPIVersion, api.MinSupportedAPIVersion) || versions.GreaterThan(minAPIVersion, api.DefaultVersion) {
return nil, fmt.Errorf("invalid minimum API version (%s): must be between %s and %s", minAPIVersion, api.MinSupportedAPIVersion, api.DefaultVersion)
}
if versions.GreaterThan(minAPIVersion, defaultAPIVersion) {
return nil, fmt.Errorf("invalid API version: the minimum API version (%s) is higher than the default version (%s)", minAPIVersion, defaultAPIVersion)
}
return &VersionMiddleware{
serverVersion: serverVersion,
defaultAPIVersion: defaultAPIVersion,
minAPIVersion: minAPIVersion,
}, nil
}
type versionUnsupportedError struct {
@ -67,20 +45,21 @@ func (e versionUnsupportedError) InvalidParameter() {}
func (v VersionMiddleware) WrapHandler(handler func(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error) func(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
return func(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
w.Header().Set("Server", fmt.Sprintf("Docker/%s (%s)", v.serverVersion, runtime.GOOS))
w.Header().Set("API-Version", v.defaultAPIVersion)
w.Header().Set("API-Version", v.defaultVersion)
w.Header().Set("OSType", runtime.GOOS)
apiVersion := vars["version"]
if apiVersion == "" {
apiVersion = v.defaultAPIVersion
apiVersion = v.defaultVersion
}
if versions.LessThan(apiVersion, v.minAPIVersion) {
return versionUnsupportedError{version: apiVersion, minVersion: v.minAPIVersion}
if versions.LessThan(apiVersion, v.minVersion) {
return versionUnsupportedError{version: apiVersion, minVersion: v.minVersion}
}
if versions.GreaterThan(apiVersion, v.defaultAPIVersion) {
return versionUnsupportedError{version: apiVersion, maxVersion: v.defaultAPIVersion}
if versions.GreaterThan(apiVersion, v.defaultVersion) {
return versionUnsupportedError{version: apiVersion, maxVersion: v.defaultVersion}
}
ctx = context.WithValue(ctx, httputils.APIVersionKey{}, apiVersion)
return handler(ctx, w, r, vars)
}
}

View file

@ -2,82 +2,27 @@ package middleware // import "github.com/docker/docker/api/server/middleware"
import (
"context"
"fmt"
"net/http"
"net/http/httptest"
"runtime"
"testing"
"github.com/docker/docker/api"
"github.com/docker/docker/api/server/httputils"
"gotest.tools/v3/assert"
is "gotest.tools/v3/assert/cmp"
)
func TestNewVersionMiddlewareValidation(t *testing.T) {
tests := []struct {
doc, defaultVersion, minVersion, expectedErr string
}{
{
doc: "defaults",
defaultVersion: api.DefaultVersion,
minVersion: api.MinSupportedAPIVersion,
},
{
doc: "invalid default lower than min",
defaultVersion: api.MinSupportedAPIVersion,
minVersion: api.DefaultVersion,
expectedErr: fmt.Sprintf("invalid API version: the minimum API version (%s) is higher than the default version (%s)", api.DefaultVersion, api.MinSupportedAPIVersion),
},
{
doc: "invalid default too low",
defaultVersion: "0.1",
minVersion: api.MinSupportedAPIVersion,
expectedErr: fmt.Sprintf("invalid default API version (0.1): must be between %s and %s", api.MinSupportedAPIVersion, api.DefaultVersion),
},
{
doc: "invalid default too high",
defaultVersion: "9999.9999",
minVersion: api.DefaultVersion,
expectedErr: fmt.Sprintf("invalid default API version (9999.9999): must be between %s and %s", api.MinSupportedAPIVersion, api.DefaultVersion),
},
{
doc: "invalid minimum too low",
defaultVersion: api.MinSupportedAPIVersion,
minVersion: "0.1",
expectedErr: fmt.Sprintf("invalid minimum API version (0.1): must be between %s and %s", api.MinSupportedAPIVersion, api.DefaultVersion),
},
{
doc: "invalid minimum too high",
defaultVersion: api.DefaultVersion,
minVersion: "9999.9999",
expectedErr: fmt.Sprintf("invalid minimum API version (9999.9999): must be between %s and %s", api.MinSupportedAPIVersion, api.DefaultVersion),
},
}
for _, tc := range tests {
tc := tc
t.Run(tc.doc, func(t *testing.T) {
_, err := NewVersionMiddleware("1.2.3", tc.defaultVersion, tc.minVersion)
if tc.expectedErr == "" {
assert.Check(t, err)
} else {
assert.Check(t, is.Error(err, tc.expectedErr))
}
})
}
}
func TestVersionMiddlewareVersion(t *testing.T) {
expectedVersion := "<not set>"
defaultVersion := "1.10.0"
minVersion := "1.2.0"
expectedVersion := defaultVersion
handler := func(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
v := httputils.VersionFromContext(ctx)
assert.Check(t, is.Equal(expectedVersion, v))
return nil
}
m, err := NewVersionMiddleware("1.2.3", api.DefaultVersion, api.MinSupportedAPIVersion)
assert.NilError(t, err)
m := NewVersionMiddleware(defaultVersion, defaultVersion, minVersion)
h := m.WrapHandler(handler)
req, _ := http.NewRequest(http.MethodGet, "/containers/json", nil)
@ -90,19 +35,19 @@ func TestVersionMiddlewareVersion(t *testing.T) {
errString string
}{
{
expectedVersion: api.DefaultVersion,
expectedVersion: "1.10.0",
},
{
reqVersion: api.MinSupportedAPIVersion,
expectedVersion: api.MinSupportedAPIVersion,
reqVersion: "1.9.0",
expectedVersion: "1.9.0",
},
{
reqVersion: "0.1",
errString: fmt.Sprintf("client version 0.1 is too old. Minimum supported API version is %s, please upgrade your client to a newer version", api.MinSupportedAPIVersion),
errString: "client version 0.1 is too old. Minimum supported API version is 1.2.0, please upgrade your client to a newer version",
},
{
reqVersion: "9999.9999",
errString: fmt.Sprintf("client version 9999.9999 is too new. Maximum supported API version is %s", api.DefaultVersion),
errString: "client version 9999.9999 is too new. Maximum supported API version is 1.10.0",
},
}
@ -126,8 +71,9 @@ func TestVersionMiddlewareWithErrorsReturnsHeaders(t *testing.T) {
return nil
}
m, err := NewVersionMiddleware("1.2.3", api.DefaultVersion, api.MinSupportedAPIVersion)
assert.NilError(t, err)
defaultVersion := "1.10.0"
minVersion := "1.2.0"
m := NewVersionMiddleware(defaultVersion, defaultVersion, minVersion)
h := m.WrapHandler(handler)
req, _ := http.NewRequest(http.MethodGet, "/containers/json", nil)
@ -135,12 +81,12 @@ func TestVersionMiddlewareWithErrorsReturnsHeaders(t *testing.T) {
ctx := context.Background()
vars := map[string]string{"version": "0.1"}
err = h(ctx, resp, req, vars)
err := h(ctx, resp, req, vars)
assert.Check(t, is.ErrorContains(err, ""))
hdr := resp.Result().Header
assert.Check(t, is.Contains(hdr.Get("Server"), "Docker/1.2.3"))
assert.Check(t, is.Contains(hdr.Get("Server"), "Docker/"+defaultVersion))
assert.Check(t, is.Contains(hdr.Get("Server"), runtime.GOOS))
assert.Check(t, is.Equal(hdr.Get("API-Version"), api.DefaultVersion))
assert.Check(t, is.Equal(hdr.Get("API-Version"), defaultVersion))
assert.Check(t, is.Equal(hdr.Get("OSType"), runtime.GOOS))
}

View file

@ -15,6 +15,7 @@ type Backend interface {
// Prune build cache
PruneCache(context.Context, types.BuildCachePruneOptions) (*types.BuildCachePruneReport, error)
Cancel(context.Context, string) error
}

View file

@ -1,24 +1,24 @@
package build // import "github.com/docker/docker/api/server/router/build"
import (
"runtime"
"github.com/docker/docker/api/server/router"
"github.com/docker/docker/api/types"
)
// buildRouter is a router to talk with the build controller
type buildRouter struct {
backend Backend
daemon experimentalProvider
routes []router.Route
backend Backend
daemon experimentalProvider
routes []router.Route
features *map[string]bool
}
// NewRouter initializes a new build router
func NewRouter(b Backend, d experimentalProvider) router.Router {
func NewRouter(b Backend, d experimentalProvider, features *map[string]bool) router.Router {
r := &buildRouter{
backend: b,
daemon: d,
backend: b,
daemon: d,
features: features,
}
r.initRoutes()
return r
@ -37,24 +37,17 @@ func (r *buildRouter) initRoutes() {
}
}
// BuilderVersion derives the default docker builder version from the config.
//
// The default on Linux is version "2" (BuildKit), but the daemon can be
// configured to recommend version "1" (classic Builder). Windows does not
// yet support BuildKit for native Windows images, and uses "1" (classic builder)
// as a default.
//
// This value is only a recommendation as advertised by the daemon, and it is
// up to the client to choose which builder to use.
// BuilderVersion derives the default docker builder version from the config
// Note: it is valid to have BuilderVersion unset which means it is up to the
// client to choose which builder to use.
func BuilderVersion(features map[string]bool) types.BuilderVersion {
// TODO(thaJeztah) move the default to daemon/config
if runtime.GOOS == "windows" {
return types.BuilderV1
}
bv := types.BuilderBuildKit
if v, ok := features["buildkit"]; ok && !v {
bv = types.BuilderV1
var bv types.BuilderVersion
if v, ok := features["buildkit"]; ok {
if v {
bv = types.BuilderBuildKit
} else {
bv = types.BuilderV1
}
}
return bv
}

View file

@ -14,26 +14,28 @@ import (
"strings"
"sync"
"github.com/containerd/log"
"github.com/docker/docker/api/server/httputils"
"github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/backend"
"github.com/docker/docker/api/types/container"
"github.com/docker/docker/api/types/filters"
"github.com/docker/docker/api/types/registry"
"github.com/docker/docker/api/types/versions"
"github.com/docker/docker/errdefs"
"github.com/docker/docker/pkg/ioutils"
"github.com/docker/docker/pkg/progress"
"github.com/docker/docker/pkg/streamformatter"
units "github.com/docker/go-units"
"github.com/pkg/errors"
"github.com/sirupsen/logrus"
)
type invalidParam struct {
error
type invalidIsolationError string
func (e invalidIsolationError) Error() string {
return fmt.Sprintf("Unsupported isolation: %q", string(e))
}
func (e invalidParam) InvalidParameter() {}
func (e invalidIsolationError) InvalidParameter() {}
func newImageBuildOptions(ctx context.Context, r *http.Request) (*types.ImageBuildOptions, error) {
options := &types.ImageBuildOptions{
@ -42,7 +44,6 @@ func newImageBuildOptions(ctx context.Context, r *http.Request) (*types.ImageBui
SuppressOutput: httputils.BoolValue(r, "q"),
NoCache: httputils.BoolValue(r, "nocache"),
ForceRemove: httputils.BoolValue(r, "forcerm"),
PullParent: httputils.BoolValue(r, "pull"),
MemorySwap: httputils.Int64ValueOrZero(r, "memswap"),
Memory: httputils.Int64ValueOrZero(r, "memory"),
CPUShares: httputils.Int64ValueOrZero(r, "cpushares"),
@ -63,18 +64,20 @@ func newImageBuildOptions(ctx context.Context, r *http.Request) (*types.ImageBui
}
if runtime.GOOS != "windows" && options.SecurityOpt != nil {
// SecurityOpt only supports "credentials-spec" on Windows, and not used on other platforms.
return nil, invalidParam{errors.New("security options are not supported on " + runtime.GOOS)}
return nil, errdefs.InvalidParameter(errors.New("The daemon on this platform does not support setting security options on build"))
}
if httputils.BoolValue(r, "forcerm") {
version := httputils.VersionFromContext(ctx)
if httputils.BoolValue(r, "forcerm") && versions.GreaterThanOrEqualTo(version, "1.12") {
options.Remove = true
} else if r.FormValue("rm") == "" {
} else if r.FormValue("rm") == "" && versions.GreaterThanOrEqualTo(version, "1.12") {
options.Remove = true
} else {
options.Remove = httputils.BoolValue(r, "rm")
}
version := httputils.VersionFromContext(ctx)
if httputils.BoolValue(r, "pull") && versions.GreaterThanOrEqualTo(version, "1.16") {
options.PullParent = true
}
if versions.GreaterThanOrEqualTo(version, "1.32") {
options.Platform = r.FormValue("platform")
}
@ -83,7 +86,7 @@ func newImageBuildOptions(ctx context.Context, r *http.Request) (*types.ImageBui
if outputsJSON != "" {
var outputs []types.ImageBuildOutput
if err := json.Unmarshal([]byte(outputsJSON), &outputs); err != nil {
return nil, invalidParam{errors.Wrap(err, "invalid outputs specified")}
return nil, err
}
options.Outputs = outputs
}
@ -100,14 +103,14 @@ func newImageBuildOptions(ctx context.Context, r *http.Request) (*types.ImageBui
if i := r.FormValue("isolation"); i != "" {
options.Isolation = container.Isolation(i)
if !options.Isolation.IsValid() {
return nil, invalidParam{errors.Errorf("unsupported isolation: %q", i)}
return nil, invalidIsolationError(options.Isolation)
}
}
if ulimitsJSON := r.FormValue("ulimits"); ulimitsJSON != "" {
buildUlimits := []*units.Ulimit{}
var buildUlimits = []*units.Ulimit{}
if err := json.Unmarshal([]byte(ulimitsJSON), &buildUlimits); err != nil {
return nil, invalidParam{errors.Wrap(err, "error reading ulimit settings")}
return nil, errors.Wrap(errdefs.InvalidParameter(err), "error reading ulimit settings")
}
options.Ulimits = buildUlimits
}
@ -125,25 +128,25 @@ func newImageBuildOptions(ctx context.Context, r *http.Request) (*types.ImageBui
// so that it can print a warning about "foo" being unused if there is
// no "ARG foo" in the Dockerfile.
if buildArgsJSON := r.FormValue("buildargs"); buildArgsJSON != "" {
buildArgs := map[string]*string{}
var buildArgs = map[string]*string{}
if err := json.Unmarshal([]byte(buildArgsJSON), &buildArgs); err != nil {
return nil, invalidParam{errors.Wrap(err, "error reading build args")}
return nil, errors.Wrap(errdefs.InvalidParameter(err), "error reading build args")
}
options.BuildArgs = buildArgs
}
if labelsJSON := r.FormValue("labels"); labelsJSON != "" {
labels := map[string]string{}
var labels = map[string]string{}
if err := json.Unmarshal([]byte(labelsJSON), &labels); err != nil {
return nil, invalidParam{errors.Wrap(err, "error reading labels")}
return nil, errors.Wrap(errdefs.InvalidParameter(err), "error reading labels")
}
options.Labels = labels
}
if cacheFromJSON := r.FormValue("cachefrom"); cacheFromJSON != "" {
cacheFrom := []string{}
var cacheFrom = []string{}
if err := json.Unmarshal([]byte(cacheFromJSON), &cacheFrom); err != nil {
return nil, invalidParam{errors.Wrap(err, "error reading cache-from")}
return nil, err
}
options.CacheFrom = cacheFrom
}
@ -166,7 +169,7 @@ func parseVersion(s string) (types.BuilderVersion, error) {
case types.BuilderBuildKit:
return types.BuilderBuildKit, nil
default:
return "", invalidParam{errors.Errorf("invalid version %q", s)}
return "", errors.Errorf("invalid version %q", s)
}
}
@ -176,7 +179,7 @@ func (br *buildRouter) postPrune(ctx context.Context, w http.ResponseWriter, r *
}
fltrs, err := filters.FromJSON(r.Form.Get("filters"))
if err != nil {
return err
return errors.Wrap(err, "could not parse filters")
}
ksfv := r.FormValue("keep-storage")
if ksfv == "" {
@ -184,7 +187,7 @@ func (br *buildRouter) postPrune(ctx context.Context, w http.ResponseWriter, r *
}
ks, err := strconv.Atoi(ksfv)
if err != nil {
return invalidParam{errors.Wrapf(err, "keep-storage is in bytes and expects an integer, got %v", ksfv)}
return errors.Wrapf(err, "keep-storage is in bytes and expects an integer, got %v", ksfv)
}
opts := types.BuildCachePruneOptions{
@ -205,7 +208,7 @@ func (br *buildRouter) postCancel(ctx context.Context, w http.ResponseWriter, r
id := r.FormValue("id")
if id == "" {
return invalidParam{errors.New("build ID not provided")}
return errors.Errorf("build ID not provided")
}
return br.backend.Cancel(ctx, id)
@ -235,6 +238,7 @@ func (br *buildRouter) postBuild(ctx context.Context, w http.ResponseWriter, r *
defer func() { _ = output.Close() }()
errf := func(err error) error {
if httputils.BoolValue(r, "q") && notVerboseBuffer.Len() > 0 {
_, _ = output.Write(notVerboseBuffer.Bytes())
}
@ -246,7 +250,7 @@ func (br *buildRouter) postBuild(ctx context.Context, w http.ResponseWriter, r *
}
_, err = output.Write(streamformatter.FormatError(err))
if err != nil {
log.G(ctx).Warnf("could not write error response: %v", err)
logrus.Warnf("could not write error response: %v", err)
}
return nil
}
@ -258,7 +262,7 @@ func (br *buildRouter) postBuild(ctx context.Context, w http.ResponseWriter, r *
buildOptions.AuthConfigs = getAuthConfigs(r.Header)
if buildOptions.Squash && !br.daemon.HasExperimental() {
return invalidParam{errors.New("squash is only supported with experimental mode")}
return errdefs.InvalidParameter(errors.New("squash is only supported with experimental mode"))
}
out := io.Writer(output)
@ -292,8 +296,8 @@ func (br *buildRouter) postBuild(ctx context.Context, w http.ResponseWriter, r *
return nil
}
func getAuthConfigs(header http.Header) map[string]registry.AuthConfig {
authConfigs := map[string]registry.AuthConfig{}
func getAuthConfigs(header http.Header) map[string]types.AuthConfig {
authConfigs := map[string]types.AuthConfig{}
authConfigsEncoded := header.Get("X-Registry-Config")
if authConfigsEncoded == "" {

View file

@ -1,10 +1,10 @@
package checkpoint // import "github.com/docker/docker/api/server/router/checkpoint"
import "github.com/docker/docker/api/types/checkpoint"
import "github.com/docker/docker/api/types"
// Backend for Checkpoint
type Backend interface {
CheckpointCreate(container string, config checkpoint.CreateOptions) error
CheckpointDelete(container string, config checkpoint.DeleteOptions) error
CheckpointList(container string, config checkpoint.ListOptions) ([]checkpoint.Summary, error)
CheckpointCreate(container string, config types.CheckpointCreateOptions) error
CheckpointDelete(container string, config types.CheckpointDeleteOptions) error
CheckpointList(container string, config types.CheckpointListOptions) ([]types.Checkpoint, error)
}

View file

@ -2,10 +2,11 @@ package checkpoint // import "github.com/docker/docker/api/server/router/checkpo
import (
"context"
"encoding/json"
"net/http"
"github.com/docker/docker/api/server/httputils"
"github.com/docker/docker/api/types/checkpoint"
"github.com/docker/docker/api/types"
)
func (s *checkpointRouter) postContainerCheckpoint(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
@ -13,8 +14,10 @@ func (s *checkpointRouter) postContainerCheckpoint(ctx context.Context, w http.R
return err
}
var options checkpoint.CreateOptions
if err := httputils.ReadJSON(r, &options); err != nil {
var options types.CheckpointCreateOptions
decoder := json.NewDecoder(r.Body)
if err := decoder.Decode(&options); err != nil {
return err
}
@ -32,9 +35,10 @@ func (s *checkpointRouter) getContainerCheckpoints(ctx context.Context, w http.R
return err
}
checkpoints, err := s.backend.CheckpointList(vars["name"], checkpoint.ListOptions{
checkpoints, err := s.backend.CheckpointList(vars["name"], types.CheckpointListOptions{
CheckpointDir: r.Form.Get("dir"),
})
if err != nil {
return err
}
@ -47,10 +51,11 @@ func (s *checkpointRouter) deleteContainerCheckpoint(ctx context.Context, w http
return err
}
err := s.backend.CheckpointDelete(vars["name"], checkpoint.DeleteOptions{
err := s.backend.CheckpointDelete(vars["name"], types.CheckpointDeleteOptions{
CheckpointDir: r.Form.Get("dir"),
CheckpointID: vars["checkpoint"],
})
if err != nil {
return err
}

View file

@ -17,29 +17,30 @@ type execBackend interface {
ContainerExecCreate(name string, config *types.ExecConfig) (string, error)
ContainerExecInspect(id string) (*backend.ExecInspect, error)
ContainerExecResize(name string, height, width int) error
ContainerExecStart(ctx context.Context, name string, options container.ExecStartOptions) error
ContainerExecStart(ctx context.Context, name string, stdin io.Reader, stdout io.Writer, stderr io.Writer) error
ExecExists(name string) (bool, error)
}
// copyBackend includes functions to implement to provide container copy functionality.
type copyBackend interface {
ContainerArchivePath(name string, path string) (content io.ReadCloser, stat *types.ContainerPathStat, err error)
ContainerExport(ctx context.Context, name string, out io.Writer) error
ContainerCopy(name string, res string) (io.ReadCloser, error)
ContainerExport(name string, out io.Writer) error
ContainerExtractToDir(name, path string, copyUIDGID, noOverwriteDirNonDir bool, content io.Reader) error
ContainerStatPath(name string, path string) (stat *types.ContainerPathStat, err error)
}
// stateBackend includes functions to implement to provide container state lifecycle functionality.
type stateBackend interface {
ContainerCreate(ctx context.Context, config backend.ContainerCreateConfig) (container.CreateResponse, error)
ContainerKill(name string, signal string) error
ContainerCreate(config types.ContainerCreateConfig) (container.ContainerCreateCreatedBody, error)
ContainerKill(name string, sig uint64) error
ContainerPause(name string) error
ContainerRename(oldName, newName string) error
ContainerResize(name string, height, width int) error
ContainerRestart(ctx context.Context, name string, options container.StopOptions) error
ContainerRm(name string, config *backend.ContainerRmConfig) error
ContainerStart(ctx context.Context, name string, checkpoint string, checkpointDir string) error
ContainerStop(ctx context.Context, name string, options container.StopOptions) error
ContainerRestart(name string, seconds *int) error
ContainerRm(name string, config *types.ContainerRmConfig) error
ContainerStart(name string, hostConfig *container.HostConfig, checkpoint string, checkpointDir string) error
ContainerStop(name string, seconds *int) error
ContainerUnpause(name string) error
ContainerUpdate(name string, hostConfig *container.HostConfig) (container.ContainerUpdateOKBody, error)
ContainerWait(ctx context.Context, name string, condition containerpkg.WaitCondition) (<-chan containerpkg.StateStatus, error)
@ -47,12 +48,13 @@ type stateBackend interface {
// monitorBackend includes functions to implement to provide containers monitoring functionality.
type monitorBackend interface {
ContainerChanges(ctx context.Context, name string) ([]archive.Change, error)
ContainerInspect(ctx context.Context, name string, size bool, version string) (interface{}, error)
ContainerLogs(ctx context.Context, name string, config *container.LogsOptions) (msgs <-chan *backend.LogMessage, tty bool, err error)
ContainerChanges(name string) ([]archive.Change, error)
ContainerInspect(name string, size bool, version string) (interface{}, error)
ContainerLogs(ctx context.Context, name string, config *types.ContainerLogsOptions) (msgs <-chan *backend.LogMessage, tty bool, err error)
ContainerStats(ctx context.Context, name string, config *backend.ContainerStatsConfig) error
ContainerTop(name string, psArgs string) (*container.ContainerTopOKBody, error)
Containers(ctx context.Context, config *container.ListOptions) ([]*types.Container, error)
Containers(config *types.ContainerListOptions) ([]*types.Container, error)
}
// attachBackend includes function to implement to provide container attaching functionality.
@ -66,7 +68,7 @@ type systemBackend interface {
}
type commitBackend interface {
CreateImageFromContainer(ctx context.Context, name string, config *backend.CreateImageConfig) (imageID string, err error)
CreateImageFromContainer(name string, config *backend.CreateImageConfig) (imageID string, err error)
}
// Backend is all the methods that need to be implemented to provide container specific functionality.

View file

@ -56,6 +56,7 @@ func (r *containerRouter) initRoutes() {
router.NewPostRoute("/containers/{name:.*}/wait", r.postContainersWait),
router.NewPostRoute("/containers/{name:.*}/resize", r.postContainersResize),
router.NewPostRoute("/containers/{name:.*}/attach", r.postContainersAttach),
router.NewPostRoute("/containers/{name:.*}/copy", r.postContainersCopy), // Deprecated since 1.8, Errors out since 1.12
router.NewPostRoute("/containers/{name:.*}/exec", r.postContainerExecCreate),
router.NewPostRoute("/exec/{name:.*}/start", r.postContainerExecStart),
router.NewPostRoute("/exec/{name:.*}/resize", r.postContainerExecResize),

View file

@ -6,27 +6,24 @@ import (
"fmt"
"io"
"net/http"
"runtime"
"strconv"
"strings"
"syscall"
"github.com/containerd/containerd/platforms"
"github.com/containerd/log"
"github.com/docker/docker/api/server/httpstatus"
"github.com/docker/docker/api/server/httputils"
"github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/backend"
"github.com/docker/docker/api/types/container"
"github.com/docker/docker/api/types/filters"
"github.com/docker/docker/api/types/mount"
"github.com/docker/docker/api/types/network"
"github.com/docker/docker/api/types/versions"
containerpkg "github.com/docker/docker/container"
"github.com/docker/docker/errdefs"
"github.com/docker/docker/pkg/ioutils"
"github.com/docker/docker/runconfig"
ocispec "github.com/opencontainers/image-spec/specs-go/v1"
"github.com/docker/docker/pkg/signal"
specs "github.com/opencontainers/image-spec/specs-go/v1"
"github.com/pkg/errors"
"github.com/sirupsen/logrus"
"golang.org/x/net/websocket"
)
@ -39,24 +36,29 @@ func (s *containerRouter) postCommit(ctx context.Context, w http.ResponseWriter,
return err
}
// TODO: remove pause arg, and always pause in backend
pause := httputils.BoolValue(r, "pause")
version := httputils.VersionFromContext(ctx)
if r.FormValue("pause") == "" && versions.GreaterThanOrEqualTo(version, "1.13") {
pause = true
}
config, _, _, err := s.decoder.DecodeConfig(r.Body)
if err != nil && !errors.Is(err, io.EOF) { // Do not fail if body is empty.
if err != nil && err != io.EOF { // Do not fail if body is empty.
return err
}
ref, err := httputils.RepoTagReference(r.Form.Get("repo"), r.Form.Get("tag"))
if err != nil {
return errdefs.InvalidParameter(err)
}
imgID, err := s.backend.CreateImageFromContainer(ctx, r.Form.Get("container"), &backend.CreateImageConfig{
Pause: httputils.BoolValueOrDefault(r, "pause", true), // TODO(dnephin): remove pause arg, and always pause in backend
Tag: ref,
commitCfg := &backend.CreateImageConfig{
Pause: pause,
Repo: r.Form.Get("repo"),
Tag: r.Form.Get("tag"),
Author: r.Form.Get("author"),
Comment: r.Form.Get("comment"),
Config: config,
Changes: r.Form["changes"],
})
}
imgID, err := s.backend.CreateImageFromContainer(r.Form.Get("container"), commitCfg)
if err != nil {
return err
}
@ -73,7 +75,7 @@ func (s *containerRouter) getContainersJSON(ctx context.Context, w http.Response
return err
}
config := &container.ListOptions{
config := &types.ContainerListOptions{
All: httputils.BoolValue(r, "all"),
Size: httputils.BoolValue(r, "size"),
Since: r.Form.Get("since"),
@ -89,7 +91,7 @@ func (s *containerRouter) getContainersJSON(ctx context.Context, w http.Response
config.Limit = limit
}
containers, err := s.backend.Containers(ctx, config)
containers, err := s.backend.Containers(config)
if err != nil {
return err
}
@ -111,11 +113,14 @@ func (s *containerRouter) getContainersStats(ctx context.Context, w http.Respons
oneShot = httputils.BoolValueOrDefault(r, "one-shot", false)
}
return s.backend.ContainerStats(ctx, vars["name"], &backend.ContainerStatsConfig{
config := &backend.ContainerStatsConfig{
Stream: stream,
OneShot: oneShot,
OutStream: w,
})
Version: httputils.VersionFromContext(ctx),
}
return s.backend.ContainerStats(ctx, vars["name"], config)
}
func (s *containerRouter) getContainersLogs(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
@ -134,7 +139,7 @@ func (s *containerRouter) getContainersLogs(ctx context.Context, w http.Response
}
containerName := vars["name"]
logsConfig := &container.LogsOptions{
logsConfig := &types.ContainerLogsOptions{
Follow: httputils.BoolValue(r, "follow"),
Timestamps: httputils.BoolValue(r, "timestamps"),
Since: r.Form.Get("since"),
@ -150,12 +155,6 @@ func (s *containerRouter) getContainersLogs(ctx context.Context, w http.Response
return err
}
contentType := types.MediaTypeRawStream
if !tty && versions.GreaterThanOrEqualTo(httputils.VersionFromContext(ctx), "1.42") {
contentType = types.MediaTypeMultiplexedStream
}
w.Header().Set("Content-Type", contentType)
// if has a tty, we're not muxing streams. if it doesn't, we are. simple.
// this is the point of no return for writing a response. once we call
// WriteLogStream, the response has been started and errors will be
@ -165,9 +164,17 @@ func (s *containerRouter) getContainersLogs(ctx context.Context, w http.Response
}
func (s *containerRouter) getContainersExport(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
return s.backend.ContainerExport(ctx, vars["name"], w)
return s.backend.ContainerExport(vars["name"], w)
}
type bodyOnStartError struct{}
func (bodyOnStartError) Error() string {
return "starting container with non-empty request body was deprecated since API v1.22 and removed in v1.24"
}
func (bodyOnStartError) InvalidParameter() {}
func (s *containerRouter) postContainersStart(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
// If contentLength is -1, we can assumed chunked encoding
// or more technically that the length is unknown
@ -175,17 +182,33 @@ func (s *containerRouter) postContainersStart(ctx context.Context, w http.Respon
// net/http otherwise seems to swallow any headers related to chunked encoding
// including r.TransferEncoding
// allow a nil body for backwards compatibility
//
version := httputils.VersionFromContext(ctx)
var hostConfig *container.HostConfig
// A non-nil json object is at least 7 characters.
if r.ContentLength > 7 || r.ContentLength == -1 {
return errdefs.InvalidParameter(errors.New("starting container with non-empty request body was deprecated since API v1.22 and removed in v1.24"))
if versions.GreaterThanOrEqualTo(version, "1.24") {
return bodyOnStartError{}
}
if err := httputils.CheckForJSON(r); err != nil {
return err
}
c, err := s.decoder.DecodeHostConfig(r.Body)
if err != nil {
return err
}
hostConfig = c
}
if err := httputils.ParseForm(r); err != nil {
return err
}
if err := s.backend.ContainerStart(ctx, vars["name"], r.Form.Get("checkpoint"), r.Form.Get("checkpoint-dir")); err != nil {
checkpoint := r.Form.Get("checkpoint")
checkpointDir := r.Form.Get("checkpoint-dir")
if err := s.backend.ContainerStart(vars["name"], hostConfig, checkpoint, checkpointDir); err != nil {
return err
}
@ -198,37 +221,52 @@ func (s *containerRouter) postContainersStop(ctx context.Context, w http.Respons
return err
}
var (
options container.StopOptions
version = httputils.VersionFromContext(ctx)
)
if versions.GreaterThanOrEqualTo(version, "1.42") {
options.Signal = r.Form.Get("signal")
}
var seconds *int
if tmpSeconds := r.Form.Get("t"); tmpSeconds != "" {
valSeconds, err := strconv.Atoi(tmpSeconds)
if err != nil {
return err
}
options.Timeout = &valSeconds
seconds = &valSeconds
}
if err := s.backend.ContainerStop(ctx, vars["name"], options); err != nil {
if err := s.backend.ContainerStop(vars["name"], seconds); err != nil {
return err
}
w.WriteHeader(http.StatusNoContent)
return nil
}
func (s *containerRouter) postContainersKill(_ context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
func (s *containerRouter) postContainersKill(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
if err := httputils.ParseForm(r); err != nil {
return err
}
var sig syscall.Signal
name := vars["name"]
if err := s.backend.ContainerKill(name, r.Form.Get("signal")); err != nil {
return errors.Wrapf(err, "cannot kill container: %s", name)
// If we have a signal, look at it. Otherwise, do nothing
if sigStr := r.Form.Get("signal"); sigStr != "" {
var err error
if sig, err = signal.ParseSignal(sigStr); err != nil {
return errdefs.InvalidParameter(err)
}
}
if err := s.backend.ContainerKill(name, uint64(sig)); err != nil {
var isStopped bool
if errdefs.IsConflict(err) {
isStopped = true
}
// Return error that's not caused because the container is stopped.
// Return error if the container is not running and the api is >= 1.20
// to keep backwards compatibility.
version := httputils.VersionFromContext(ctx)
if versions.GreaterThanOrEqualTo(version, "1.20") || !isStopped {
return errors.Wrapf(err, "Cannot kill container: %s", name)
}
}
w.WriteHeader(http.StatusNoContent)
@ -240,26 +278,21 @@ func (s *containerRouter) postContainersRestart(ctx context.Context, w http.Resp
return err
}
var (
options container.StopOptions
version = httputils.VersionFromContext(ctx)
)
if versions.GreaterThanOrEqualTo(version, "1.42") {
options.Signal = r.Form.Get("signal")
}
var seconds *int
if tmpSeconds := r.Form.Get("t"); tmpSeconds != "" {
valSeconds, err := strconv.Atoi(tmpSeconds)
if err != nil {
return err
}
options.Timeout = &valSeconds
seconds = &valSeconds
}
if err := s.backend.ContainerRestart(ctx, vars["name"], options); err != nil {
if err := s.backend.ContainerRestart(vars["name"], seconds); err != nil {
return err
}
w.WriteHeader(http.StatusNoContent)
return nil
}
@ -304,18 +337,12 @@ func (s *containerRouter) postContainersWait(ctx context.Context, w http.Respons
if err := httputils.ParseForm(r); err != nil {
return err
}
if v := r.Form.Get("condition"); v != "" {
switch container.WaitCondition(v) {
case container.WaitConditionNotRunning:
waitCondition = containerpkg.WaitConditionNotRunning
case container.WaitConditionNextExit:
waitCondition = containerpkg.WaitConditionNextExit
case container.WaitConditionRemoved:
waitCondition = containerpkg.WaitConditionRemoved
legacyRemovalWaitPre134 = versions.LessThan(version, "1.34")
default:
return errdefs.InvalidParameter(errors.Errorf("invalid condition: %q", v))
}
switch container.WaitCondition(r.Form.Get("condition")) {
case container.WaitConditionNextExit:
waitCondition = containerpkg.WaitConditionNextExit
case container.WaitConditionRemoved:
waitCondition = containerpkg.WaitConditionRemoved
legacyRemovalWaitPre134 = versions.LessThan(version, "1.34")
}
}
@ -345,19 +372,19 @@ func (s *containerRouter) postContainersWait(ctx context.Context, w http.Respons
return nil
}
var waitError *container.WaitExitError
var waitError *container.ContainerWaitOKBodyError
if status.Err() != nil {
waitError = &container.WaitExitError{Message: status.Err().Error()}
waitError = &container.ContainerWaitOKBodyError{Message: status.Err().Error()}
}
return json.NewEncoder(w).Encode(&container.WaitResponse{
return json.NewEncoder(w).Encode(&container.ContainerWaitOKBody{
StatusCode: int64(status.ExitCode()),
Error: waitError,
})
}
func (s *containerRouter) getContainersChanges(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
changes, err := s.backend.ContainerChanges(ctx, vars["name"])
changes, err := s.backend.ContainerChanges(vars["name"])
if err != nil {
return err
}
@ -396,20 +423,19 @@ func (s *containerRouter) postContainerUpdate(ctx context.Context, w http.Respon
if err := httputils.ParseForm(r); err != nil {
return err
}
if err := httputils.CheckForJSON(r); err != nil {
return err
}
var updateConfig container.UpdateConfig
if err := httputils.ReadJSON(r, &updateConfig); err != nil {
decoder := json.NewDecoder(r.Body)
if err := decoder.Decode(&updateConfig); err != nil {
return err
}
if versions.LessThan(httputils.VersionFromContext(ctx), "1.40") {
updateConfig.PidsLimit = nil
}
if versions.GreaterThanOrEqualTo(httputils.VersionFromContext(ctx), "1.42") {
// Ignore KernelMemory removed in API 1.42.
updateConfig.KernelMemory = 0
}
if updateConfig.PidsLimit != nil && *updateConfig.PidsLimit <= 0 {
// Both `0` and `-1` are accepted to set "unlimited" when updating.
// Historically, any negative value was accepted, so treat them as
@ -444,72 +470,39 @@ func (s *containerRouter) postContainersCreate(ctx context.Context, w http.Respo
config, hostConfig, networkingConfig, err := s.decoder.DecodeConfig(r.Body)
if err != nil {
if errors.Is(err, io.EOF) {
return errdefs.InvalidParameter(errors.New("invalid JSON: got EOF while reading request body"))
}
return err
}
if config == nil {
return errdefs.InvalidParameter(runconfig.ErrEmptyConfig)
}
if hostConfig == nil {
hostConfig = &container.HostConfig{}
}
if networkingConfig == nil {
networkingConfig = &network.NetworkingConfig{}
}
if networkingConfig.EndpointsConfig == nil {
networkingConfig.EndpointsConfig = make(map[string]*network.EndpointSettings)
}
// The NetworkMode "default" is used as a way to express a container should
// be attached to the OS-dependant default network, in an OS-independent
// way. Doing this conversion as soon as possible ensures we have less
// NetworkMode to handle down the path (including in the
// backward-compatibility layer we have just below).
//
// Note that this is not the only place where this conversion has to be
// done (as there are various other places where containers get created).
if hostConfig.NetworkMode == "" || hostConfig.NetworkMode.IsDefault() {
hostConfig.NetworkMode = runconfig.DefaultDaemonNetworkMode()
if nw, ok := networkingConfig.EndpointsConfig[network.NetworkDefault]; ok {
networkingConfig.EndpointsConfig[hostConfig.NetworkMode.NetworkName()] = nw
delete(networkingConfig.EndpointsConfig, network.NetworkDefault)
}
}
version := httputils.VersionFromContext(ctx)
adjustCPUShares := versions.LessThan(version, "1.19")
// When using API 1.24 and under, the client is responsible for removing the container
if versions.LessThan(version, "1.25") {
if hostConfig != nil && versions.LessThan(version, "1.25") {
hostConfig.AutoRemove = false
}
if versions.LessThan(version, "1.40") {
if hostConfig != nil && versions.LessThan(version, "1.40") {
// Ignore BindOptions.NonRecursive because it was added in API 1.40.
for _, m := range hostConfig.Mounts {
if bo := m.BindOptions; bo != nil {
bo.NonRecursive = false
}
}
// Ignore KernelMemoryTCP because it was added in API 1.40.
hostConfig.KernelMemoryTCP = 0
// Older clients (API < 1.40) expects the default to be shareable, make them happy
if hostConfig.IpcMode.IsEmpty() {
hostConfig.IpcMode = container.IPCModeShareable
hostConfig.IpcMode = container.IpcMode("shareable")
}
}
if versions.LessThan(version, "1.41") {
if hostConfig != nil && versions.LessThan(version, "1.41") && !s.cgroup2 {
// Older clients expect the default to be "host" on cgroup v1 hosts
if !s.cgroup2 && hostConfig.CgroupnsMode.IsEmpty() {
hostConfig.CgroupnsMode = container.CgroupnsModeHost
if hostConfig.CgroupnsMode.IsEmpty() {
hostConfig.CgroupnsMode = container.CgroupnsMode("host")
}
}
var platform *ocispec.Platform
var platform *specs.Platform
if versions.GreaterThanOrEqualTo(version, "1.41") {
if v := r.Form.Get("platform"); v != "" {
p, err := platforms.Parse(v)
@ -520,106 +513,7 @@ func (s *containerRouter) postContainersCreate(ctx context.Context, w http.Respo
}
}
if versions.LessThan(version, "1.42") {
for _, m := range hostConfig.Mounts {
// Ignore BindOptions.CreateMountpoint because it was added in API 1.42.
if bo := m.BindOptions; bo != nil {
bo.CreateMountpoint = false
}
// These combinations are invalid, but weren't validated in API < 1.42.
// We reset them here, so that validation doesn't produce an error.
if o := m.VolumeOptions; o != nil && m.Type != mount.TypeVolume {
m.VolumeOptions = nil
}
if o := m.TmpfsOptions; o != nil && m.Type != mount.TypeTmpfs {
m.TmpfsOptions = nil
}
if bo := m.BindOptions; bo != nil {
// Ignore BindOptions.CreateMountpoint because it was added in API 1.42.
bo.CreateMountpoint = false
}
}
if runtime.GOOS == "linux" {
// ConsoleSize is not respected by Linux daemon before API 1.42
hostConfig.ConsoleSize = [2]uint{0, 0}
}
}
if versions.GreaterThanOrEqualTo(version, "1.42") {
// Ignore KernelMemory removed in API 1.42.
hostConfig.KernelMemory = 0
for _, m := range hostConfig.Mounts {
if o := m.VolumeOptions; o != nil && m.Type != mount.TypeVolume {
return errdefs.InvalidParameter(fmt.Errorf("VolumeOptions must not be specified on mount type %q", m.Type))
}
if o := m.BindOptions; o != nil && m.Type != mount.TypeBind {
return errdefs.InvalidParameter(fmt.Errorf("BindOptions must not be specified on mount type %q", m.Type))
}
if o := m.TmpfsOptions; o != nil && m.Type != mount.TypeTmpfs {
return errdefs.InvalidParameter(fmt.Errorf("TmpfsOptions must not be specified on mount type %q", m.Type))
}
}
}
if versions.LessThan(version, "1.43") {
// Ignore Annotations because it was added in API v1.43.
hostConfig.Annotations = nil
}
defaultReadOnlyNonRecursive := false
if versions.LessThan(version, "1.44") {
if config.Healthcheck != nil {
// StartInterval was added in API 1.44
config.Healthcheck.StartInterval = 0
}
// Set ReadOnlyNonRecursive to true because it was added in API 1.44
// Before that all read-only mounts were non-recursive.
// Keep that behavior for clients on older APIs.
defaultReadOnlyNonRecursive = true
for _, m := range hostConfig.Mounts {
if m.Type == mount.TypeBind {
if m.BindOptions != nil && m.BindOptions.ReadOnlyForceRecursive {
// NOTE: that technically this is a breaking change for older
// API versions, and we should ignore the new field.
// However, this option may be incorrectly set by a client with
// the expectation that the failing to apply recursive read-only
// is enforced, so we decided to produce an error instead,
// instead of silently ignoring.
return errdefs.InvalidParameter(errors.New("BindOptions.ReadOnlyForceRecursive needs API v1.44 or newer"))
}
}
}
// Creating a container connected to several networks is not supported until v1.44.
if len(networkingConfig.EndpointsConfig) > 1 {
l := make([]string, 0, len(networkingConfig.EndpointsConfig))
for k := range networkingConfig.EndpointsConfig {
l = append(l, k)
}
return errdefs.InvalidParameter(errors.Errorf("Container cannot be created with multiple network endpoints: %s", strings.Join(l, ", ")))
}
}
if versions.LessThan(version, "1.45") {
for _, m := range hostConfig.Mounts {
if m.VolumeOptions != nil && m.VolumeOptions.Subpath != "" {
return errdefs.InvalidParameter(errors.New("VolumeOptions.Subpath needs API v1.45 or newer"))
}
}
}
var warnings []string
if warn, err := handleMACAddressBC(config, hostConfig, networkingConfig, version); err != nil {
return err
} else if warn != "" {
warnings = append(warnings, warn)
}
if hostConfig.PidsLimit != nil && *hostConfig.PidsLimit <= 0 {
if hostConfig != nil && hostConfig.PidsLimit != nil && *hostConfig.PidsLimit <= 0 {
// Don't set a limit if either no limit was specified, or "unlimited" was
// explicitly set.
// Both `0` and `-1` are accepted as "unlimited", and historically any
@ -627,107 +521,28 @@ func (s *containerRouter) postContainersCreate(ctx context.Context, w http.Respo
hostConfig.PidsLimit = nil
}
ccr, err := s.backend.ContainerCreate(ctx, backend.ContainerCreateConfig{
Name: name,
Config: config,
HostConfig: hostConfig,
NetworkingConfig: networkingConfig,
Platform: platform,
DefaultReadOnlyNonRecursive: defaultReadOnlyNonRecursive,
ccr, err := s.backend.ContainerCreate(types.ContainerCreateConfig{
Name: name,
Config: config,
HostConfig: hostConfig,
NetworkingConfig: networkingConfig,
AdjustCPUShares: adjustCPUShares,
Platform: platform,
})
if err != nil {
return err
}
ccr.Warnings = append(ccr.Warnings, warnings...)
return httputils.WriteJSON(w, http.StatusCreated, ccr)
}
// handleMACAddressBC takes care of backward-compatibility for the container-wide MAC address by mutating the
// networkingConfig to set the endpoint-specific MACAddress field introduced in API v1.44. It returns a warning message
// or an error if the container-wide field was specified for API >= v1.44.
func handleMACAddressBC(config *container.Config, hostConfig *container.HostConfig, networkingConfig *network.NetworkingConfig, version string) (string, error) {
deprecatedMacAddress := config.MacAddress //nolint:staticcheck // ignore SA1019: field is deprecated, but still used on API < v1.44.
// For older versions of the API, migrate the container-wide MAC address to EndpointsConfig.
if versions.LessThan(version, "1.44") {
if deprecatedMacAddress == "" {
// If a MAC address is supplied in EndpointsConfig, discard it because the old API
// would have ignored it.
for _, ep := range networkingConfig.EndpointsConfig {
ep.MacAddress = ""
}
return "", nil
}
if !hostConfig.NetworkMode.IsBridge() && !hostConfig.NetworkMode.IsUserDefined() {
return "", runconfig.ErrConflictContainerNetworkAndMac
}
// There cannot be more than one entry in EndpointsConfig with API < 1.44.
// If there's no EndpointsConfig, create a place to store the configured address. It is
// safe to use NetworkMode as the network name, whether it's a name or id/short-id, as
// it will be normalised later and there is no other EndpointSettings object that might
// refer to this network/endpoint.
if len(networkingConfig.EndpointsConfig) == 0 {
nwName := hostConfig.NetworkMode.NetworkName()
networkingConfig.EndpointsConfig[nwName] = &network.EndpointSettings{}
}
// There's exactly one network in EndpointsConfig, either from the API or just-created.
// Migrate the container-wide setting to it.
// No need to check for a match between NetworkMode and the names/ids in EndpointsConfig,
// the old version of the API would have applied the address to this network anyway.
for _, ep := range networkingConfig.EndpointsConfig {
ep.MacAddress = deprecatedMacAddress
}
return "", nil
}
// The container-wide MacAddress parameter is deprecated and should now be specified in EndpointsConfig.
if deprecatedMacAddress == "" {
return "", nil
}
var warning string
if hostConfig.NetworkMode.IsBridge() || hostConfig.NetworkMode.IsUserDefined() {
nwName := hostConfig.NetworkMode.NetworkName()
// If there's no endpoint config, create a place to store the configured address.
if len(networkingConfig.EndpointsConfig) == 0 {
networkingConfig.EndpointsConfig[nwName] = &network.EndpointSettings{
MacAddress: deprecatedMacAddress,
}
} else {
// There is existing endpoint config - if it's not indexed by NetworkMode.Name(), we
// can't tell which network the container-wide settings was intended for. NetworkMode,
// the keys in EndpointsConfig and the NetworkID in EndpointsConfig may mix network
// name/id/short-id. It's not safe to create EndpointsConfig under the NetworkMode
// name to store the container-wide MAC address, because that may result in two sets
// of EndpointsConfig for the same network and one set will be discarded later. So,
// reject the request ...
ep, ok := networkingConfig.EndpointsConfig[nwName]
if !ok {
return "", errdefs.InvalidParameter(errors.New("if a container-wide MAC address is supplied, HostConfig.NetworkMode must match the identity of a network in NetworkSettings.Networks"))
}
// ep is the endpoint that needs the container-wide MAC address; migrate the address
// to it, or bail out if there's a mismatch.
if ep.MacAddress == "" {
ep.MacAddress = deprecatedMacAddress
} else if ep.MacAddress != deprecatedMacAddress {
return "", errdefs.InvalidParameter(errors.New("the container-wide MAC address must match the endpoint-specific MAC address for the main network, or be left empty"))
}
}
}
warning = "The container-wide MacAddress field is now deprecated. It should be specified in EndpointsConfig instead."
config.MacAddress = "" //nolint:staticcheck // ignore SA1019: field is deprecated, but still used on API < v1.44.
return warning, nil
}
func (s *containerRouter) deleteContainers(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
if err := httputils.ParseForm(r); err != nil {
return err
}
name := vars["name"]
config := &backend.ContainerRmConfig{
config := &types.ContainerRmConfig{
ForceRemove: httputils.BoolValue(r, "force"),
RemoveVolume: httputils.BoolValue(r, "v"),
RemoveLink: httputils.BoolValue(r, "link"),
@ -774,8 +589,7 @@ func (s *containerRouter) postContainersAttach(ctx context.Context, w http.Respo
return errdefs.InvalidParameter(errors.Errorf("error attaching to container %s, hijack connection missing", containerName))
}
contentType := types.MediaTypeRawStream
setupStreams := func(multiplexed bool) (io.ReadCloser, io.Writer, io.Writer, error) {
setupStreams := func() (io.ReadCloser, io.Writer, io.Writer, error) {
conn, _, err := hijacker.Hijack()
if err != nil {
return nil, nil, nil, err
@ -785,10 +599,7 @@ func (s *containerRouter) postContainersAttach(ctx context.Context, w http.Respo
conn.Write([]byte{})
if upgrade {
if multiplexed && versions.GreaterThanOrEqualTo(httputils.VersionFromContext(ctx), "1.42") {
contentType = types.MediaTypeMultiplexedStream
}
fmt.Fprintf(conn, "HTTP/1.1 101 UPGRADED\r\nContent-Type: "+contentType+"\r\nConnection: Upgrade\r\nUpgrade: tcp\r\n\r\n")
fmt.Fprintf(conn, "HTTP/1.1 101 UPGRADED\r\nContent-Type: application/vnd.docker.raw-stream\r\nConnection: Upgrade\r\nUpgrade: tcp\r\n\r\n")
} else {
fmt.Fprintf(conn, "HTTP/1.1 200 OK\r\nContent-Type: application/vnd.docker.raw-stream\r\n\r\n")
}
@ -812,16 +623,16 @@ func (s *containerRouter) postContainersAttach(ctx context.Context, w http.Respo
}
if err = s.backend.ContainerAttach(containerName, attachConfig); err != nil {
log.G(ctx).WithError(err).Errorf("Handler for %s %s returned error", r.Method, r.URL.Path)
logrus.Errorf("Handler for %s %s returned error: %v", r.Method, r.URL.Path, err)
// Remember to close stream if error happens
conn, _, errHijack := hijacker.Hijack()
if errHijack != nil {
log.G(ctx).WithError(err).Errorf("Handler for %s %s: unable to close stream; error when hijacking connection", r.Method, r.URL.Path)
} else {
if errHijack == nil {
statusCode := httpstatus.FromError(err)
statusText := http.StatusText(statusCode)
fmt.Fprintf(conn, "HTTP/1.1 %d %s\r\nContent-Type: %s\r\n\r\n%s\r\n", statusCode, statusText, contentType, err.Error())
fmt.Fprintf(conn, "HTTP/1.1 %d %s\r\nContent-Type: application/vnd.docker.raw-stream\r\n\r\n%s\r\n", statusCode, statusText, err.Error())
httputils.CloseStreams(conn)
} else {
logrus.Errorf("Error Hijacking: %v", err)
}
}
return nil
@ -841,7 +652,7 @@ func (s *containerRouter) wsContainersAttach(ctx context.Context, w http.Respons
version := httputils.VersionFromContext(ctx)
setupStreams := func(multiplexed bool) (io.ReadCloser, io.Writer, io.Writer, error) {
setupStreams := func() (io.ReadCloser, io.Writer, io.Writer, error) {
wsChan := make(chan *websocket.Conn)
h := func(conn *websocket.Conn) {
wsChan <- conn
@ -863,22 +674,15 @@ func (s *containerRouter) wsContainersAttach(ctx context.Context, w http.Respons
return conn, conn, conn, nil
}
useStdin, useStdout, useStderr := true, true, true
if versions.GreaterThanOrEqualTo(version, "1.42") {
useStdin = httputils.BoolValue(r, "stdin")
useStdout = httputils.BoolValue(r, "stdout")
useStderr = httputils.BoolValue(r, "stderr")
}
attachConfig := &backend.ContainerAttachConfig{
GetStreams: setupStreams,
UseStdin: useStdin,
UseStdout: useStdout,
UseStderr: useStderr,
Logs: httputils.BoolValue(r, "logs"),
Stream: httputils.BoolValue(r, "stream"),
DetachKeys: detachKeys,
MuxStreams: false, // never multiplex, as we rely on websocket to manage distinct streams
UseStdin: true,
UseStdout: true,
UseStderr: true,
MuxStreams: false, // TODO: this should be true since it's a single stream for both stdout and stderr
}
err = s.backend.ContainerAttach(containerName, attachConfig)
@ -886,9 +690,9 @@ func (s *containerRouter) wsContainersAttach(ctx context.Context, w http.Respons
select {
case <-started:
if err != nil {
log.G(ctx).Errorf("Error attaching websocket: %s", err)
logrus.Errorf("Error attaching websocket: %s", err)
} else {
log.G(ctx).Debug("websocket connection was closed by client")
logrus.Debug("websocket connection was closed by client")
}
return nil
default:
@ -903,7 +707,7 @@ func (s *containerRouter) postContainersPrune(ctx context.Context, w http.Respon
pruneFilters, err := filters.FromJSON(r.Form.Get("filters"))
if err != nil {
return err
return errdefs.InvalidParameter(err)
}
pruneReport, err := s.backend.ContainersPrune(ctx, pruneFilters)

View file

@ -1,160 +0,0 @@
package container
import (
"testing"
"github.com/docker/docker/api/types/container"
"github.com/docker/docker/api/types/network"
"gotest.tools/v3/assert"
is "gotest.tools/v3/assert/cmp"
)
func TestHandleMACAddressBC(t *testing.T) {
testcases := []struct {
name string
apiVersion string
ctrWideMAC string
networkMode container.NetworkMode
epConfig map[string]*network.EndpointSettings
expEpWithCtrWideMAC string
expEpWithNoMAC string
expCtrWideMAC string
expWarning string
expError string
}{
{
name: "old api ctr-wide mac mix id and name",
apiVersion: "1.43",
ctrWideMAC: "11:22:33:44:55:66",
networkMode: "aNetId",
epConfig: map[string]*network.EndpointSettings{"aNetName": {}},
expEpWithCtrWideMAC: "aNetName",
expCtrWideMAC: "11:22:33:44:55:66",
},
{
name: "old api clear ep mac",
apiVersion: "1.43",
networkMode: "aNetId",
epConfig: map[string]*network.EndpointSettings{"aNetName": {MacAddress: "11:22:33:44:55:66"}},
expEpWithNoMAC: "aNetName",
},
{
name: "old api no-network ctr-wide mac",
apiVersion: "1.43",
networkMode: "none",
ctrWideMAC: "11:22:33:44:55:66",
expError: "conflicting options: mac-address and the network mode",
expCtrWideMAC: "11:22:33:44:55:66",
},
{
name: "old api create ep",
apiVersion: "1.43",
networkMode: "aNetId",
ctrWideMAC: "11:22:33:44:55:66",
epConfig: map[string]*network.EndpointSettings{},
expEpWithCtrWideMAC: "aNetId",
expCtrWideMAC: "11:22:33:44:55:66",
},
{
name: "old api migrate ctr-wide mac",
apiVersion: "1.43",
ctrWideMAC: "11:22:33:44:55:66",
networkMode: "aNetName",
epConfig: map[string]*network.EndpointSettings{"aNetName": {}},
expEpWithCtrWideMAC: "aNetName",
expCtrWideMAC: "11:22:33:44:55:66",
},
{
name: "new api no macs",
apiVersion: "1.44",
networkMode: "aNetId",
epConfig: map[string]*network.EndpointSettings{"aNetName": {}},
},
{
name: "new api ep specific mac",
apiVersion: "1.44",
networkMode: "aNetName",
epConfig: map[string]*network.EndpointSettings{"aNetName": {MacAddress: "11:22:33:44:55:66"}},
},
{
name: "new api migrate ctr-wide mac to new ep",
apiVersion: "1.44",
ctrWideMAC: "11:22:33:44:55:66",
networkMode: "aNetName",
epConfig: map[string]*network.EndpointSettings{},
expEpWithCtrWideMAC: "aNetName",
expWarning: "The container-wide MacAddress field is now deprecated",
expCtrWideMAC: "",
},
{
name: "new api migrate ctr-wide mac to existing ep",
apiVersion: "1.44",
ctrWideMAC: "11:22:33:44:55:66",
networkMode: "aNetName",
epConfig: map[string]*network.EndpointSettings{"aNetName": {}},
expEpWithCtrWideMAC: "aNetName",
expWarning: "The container-wide MacAddress field is now deprecated",
expCtrWideMAC: "",
},
{
name: "new api mode vs name mismatch",
apiVersion: "1.44",
ctrWideMAC: "11:22:33:44:55:66",
networkMode: "aNetId",
epConfig: map[string]*network.EndpointSettings{"aNetName": {}},
expError: "if a container-wide MAC address is supplied, HostConfig.NetworkMode must match the identity of a network in NetworkSettings.Networks",
expCtrWideMAC: "11:22:33:44:55:66",
},
{
name: "new api mac mismatch",
apiVersion: "1.44",
ctrWideMAC: "11:22:33:44:55:66",
networkMode: "aNetName",
epConfig: map[string]*network.EndpointSettings{"aNetName": {MacAddress: "00:11:22:33:44:55"}},
expError: "the container-wide MAC address must match the endpoint-specific MAC address",
expCtrWideMAC: "11:22:33:44:55:66",
},
}
for _, tc := range testcases {
t.Run(tc.name, func(t *testing.T) {
cfg := &container.Config{
MacAddress: tc.ctrWideMAC, //nolint:staticcheck // ignore SA1019: field is deprecated, but still used on API < v1.44.
}
hostCfg := &container.HostConfig{
NetworkMode: tc.networkMode,
}
epConfig := make(map[string]*network.EndpointSettings, len(tc.epConfig))
for k, v := range tc.epConfig {
v := v
epConfig[k] = v
}
netCfg := &network.NetworkingConfig{
EndpointsConfig: epConfig,
}
warning, err := handleMACAddressBC(cfg, hostCfg, netCfg, tc.apiVersion)
if tc.expError == "" {
assert.Check(t, err)
} else {
assert.Check(t, is.ErrorContains(err, tc.expError))
}
if tc.expWarning == "" {
assert.Check(t, is.Equal(warning, ""))
} else {
assert.Check(t, is.Contains(warning, tc.expWarning))
}
if tc.expEpWithCtrWideMAC != "" {
got := netCfg.EndpointsConfig[tc.expEpWithCtrWideMAC].MacAddress
assert.Check(t, is.Equal(got, tc.ctrWideMAC))
}
if tc.expEpWithNoMAC != "" {
got := netCfg.EndpointsConfig[tc.expEpWithNoMAC].MacAddress
assert.Check(t, is.Equal(got, ""))
}
gotCtrWideMAC := cfg.MacAddress //nolint:staticcheck // ignore SA1019: field is deprecated, but still used on API < v1.44.
assert.Check(t, is.Equal(gotCtrWideMAC, tc.expCtrWideMAC))
})
}
}

View file

@ -6,15 +6,61 @@ import (
"context"
"encoding/base64"
"encoding/json"
"errors"
"io"
"net/http"
"github.com/docker/docker/api/server/httputils"
"github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/versions"
"github.com/docker/docker/errdefs"
gddohttputil "github.com/golang/gddo/httputil"
)
// setContainerPathStatHeader encodes the stat to JSON, base64 encode, and place in a header.
type pathError struct{}
func (pathError) Error() string {
return "Path cannot be empty"
}
func (pathError) InvalidParameter() {}
// postContainersCopy is deprecated in favor of getContainersArchive.
func (s *containerRouter) postContainersCopy(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
// Deprecated since 1.8, Errors out since 1.12
version := httputils.VersionFromContext(ctx)
if versions.GreaterThanOrEqualTo(version, "1.24") {
w.WriteHeader(http.StatusNotFound)
return nil
}
if err := httputils.CheckForJSON(r); err != nil {
return err
}
cfg := types.CopyConfig{}
if err := json.NewDecoder(r.Body).Decode(&cfg); err != nil {
if err == io.EOF {
return errdefs.InvalidParameter(errors.New("got EOF while reading request body"))
}
return errdefs.InvalidParameter(err)
}
if cfg.Resource == "" {
return pathError{}
}
data, err := s.backend.ContainerCopy(vars["name"], cfg.Resource)
if err != nil {
return err
}
defer data.Close()
w.Header().Set("Content-Type", "application/x-tar")
_, err = io.Copy(w, data)
return err
}
// // Encode the stat to JSON, base64 encode, and place in a header.
func setContainerPathStatHeader(stat *types.ContainerPathStat, header http.Header) error {
statJSON, err := json.Marshal(stat)
if err != nil {

View file

@ -2,18 +2,19 @@ package container // import "github.com/docker/docker/api/server/router/containe
import (
"context"
"encoding/json"
"errors"
"fmt"
"io"
"net/http"
"strconv"
"github.com/containerd/log"
"github.com/docker/docker/api/server/httputils"
"github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/container"
"github.com/docker/docker/api/types/versions"
"github.com/docker/docker/errdefs"
"github.com/docker/docker/pkg/stdcopy"
"github.com/sirupsen/logrus"
)
func (s *containerRouter) getExecByID(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
@ -37,26 +38,27 @@ func (s *containerRouter) postContainerExecCreate(ctx context.Context, w http.Re
if err := httputils.ParseForm(r); err != nil {
return err
}
if err := httputils.CheckForJSON(r); err != nil {
return err
}
name := vars["name"]
execConfig := &types.ExecConfig{}
if err := httputils.ReadJSON(r, execConfig); err != nil {
return err
if err := json.NewDecoder(r.Body).Decode(execConfig); err != nil {
if err == io.EOF {
return errdefs.InvalidParameter(errors.New("got EOF while reading request body"))
}
return errdefs.InvalidParameter(err)
}
if len(execConfig.Cmd) == 0 {
return execCommandError{}
}
version := httputils.VersionFromContext(ctx)
if versions.LessThan(version, "1.42") {
// Not supported by API versions before 1.42
execConfig.ConsoleSize = nil
}
// Register an instance of Exec in container.
id, err := s.backend.ContainerExecCreate(vars["name"], execConfig)
id, err := s.backend.ContainerExecCreate(name, execConfig)
if err != nil {
log.G(ctx).Errorf("Error setting up exec command in container %s: %v", vars["name"], err)
logrus.Errorf("Error setting up exec command in container %s: %v", name, err)
return err
}
@ -71,6 +73,13 @@ func (s *containerRouter) postContainerExecStart(ctx context.Context, w http.Res
return err
}
version := httputils.VersionFromContext(ctx)
if versions.GreaterThan(version, "1.21") {
if err := httputils.CheckForJSON(r); err != nil {
return err
}
}
var (
execName = vars["name"]
stdin, inStream io.ReadCloser
@ -78,28 +87,17 @@ func (s *containerRouter) postContainerExecStart(ctx context.Context, w http.Res
)
execStartCheck := &types.ExecStartCheck{}
if err := httputils.ReadJSON(r, execStartCheck); err != nil {
return err
if err := json.NewDecoder(r.Body).Decode(execStartCheck); err != nil {
if err == io.EOF {
return errdefs.InvalidParameter(errors.New("got EOF while reading request body"))
}
return errdefs.InvalidParameter(err)
}
if exists, err := s.backend.ExecExists(execName); !exists {
return err
}
if execStartCheck.ConsoleSize != nil {
version := httputils.VersionFromContext(ctx)
// Not supported before 1.42
if versions.LessThan(version, "1.42") {
execStartCheck.ConsoleSize = nil
}
// No console without tty
if !execStartCheck.Tty {
execStartCheck.ConsoleSize = nil
}
}
if !execStartCheck.Detach {
var err error
// Setting up the streaming http interface.
@ -110,11 +108,7 @@ func (s *containerRouter) postContainerExecStart(ctx context.Context, w http.Res
defer httputils.CloseStreams(inStream, outStream)
if _, ok := r.Header["Upgrade"]; ok {
contentType := types.MediaTypeRawStream
if !execStartCheck.Tty && versions.GreaterThanOrEqualTo(httputils.VersionFromContext(ctx), "1.42") {
contentType = types.MediaTypeMultiplexedStream
}
fmt.Fprint(outStream, "HTTP/1.1 101 UPGRADED\r\nContent-Type: "+contentType+"\r\nConnection: Upgrade\r\nUpgrade: tcp\r\n")
fmt.Fprint(outStream, "HTTP/1.1 101 UPGRADED\r\nContent-Type: application/vnd.docker.raw-stream\r\nConnection: Upgrade\r\nUpgrade: tcp\r\n")
} else {
fmt.Fprint(outStream, "HTTP/1.1 200 OK\r\nContent-Type: application/vnd.docker.raw-stream\r\n")
}
@ -133,21 +127,14 @@ func (s *containerRouter) postContainerExecStart(ctx context.Context, w http.Res
}
}
options := container.ExecStartOptions{
Stdin: stdin,
Stdout: stdout,
Stderr: stderr,
ConsoleSize: execStartCheck.ConsoleSize,
}
// Now run the user process in container.
// Maybe we should we pass ctx here if we're not detaching?
if err := s.backend.ContainerExecStart(context.Background(), execName, options); err != nil {
if err := s.backend.ContainerExecStart(context.Background(), execName, stdin, stdout, stderr); err != nil {
if execStartCheck.Detach {
return err
}
stdout.Write([]byte(err.Error() + "\r\n"))
log.G(ctx).Errorf("Error running exec %s in container: %v", execName, err)
logrus.Errorf("Error running exec %s in container: %v", execName, err)
}
return nil
}

View file

@ -12,7 +12,7 @@ func (s *containerRouter) getContainersByName(ctx context.Context, w http.Respon
displaySize := httputils.BoolValue(r, "size")
version := httputils.VersionFromContext(ctx)
json, err := s.backend.ContainerInspect(ctx, vars["name"], displaySize, version)
json, err := s.backend.ContainerInspect(vars["name"], displaySize, version)
if err != nil {
return err
}

View file

@ -3,13 +3,13 @@ package distribution // import "github.com/docker/docker/api/server/router/distr
import (
"context"
"github.com/distribution/reference"
"github.com/docker/distribution"
"github.com/docker/docker/api/types/registry"
"github.com/docker/distribution/reference"
"github.com/docker/docker/api/types"
)
// Backend is all the methods that need to be implemented
// to provide image specific functionality.
type Backend interface {
GetRepositories(context.Context, reference.Named, *registry.AuthConfig) ([]distribution.Repository, error)
GetRepository(context.Context, reference.Named, *types.AuthConfig) (distribution.Repository, bool, error)
}

View file

@ -2,20 +2,20 @@ package distribution // import "github.com/docker/docker/api/server/router/distr
import (
"context"
"encoding/base64"
"encoding/json"
"net/http"
"os"
"strings"
"github.com/distribution/reference"
"github.com/docker/distribution"
"github.com/docker/distribution/manifest/manifestlist"
"github.com/docker/distribution/manifest/schema1"
"github.com/docker/distribution/manifest/schema2"
"github.com/docker/distribution/reference"
"github.com/docker/docker/api/server/httputils"
"github.com/docker/docker/api/types/registry"
distributionpkg "github.com/docker/docker/distribution"
"github.com/docker/docker/api/types"
registrytypes "github.com/docker/docker/api/types/registry"
"github.com/docker/docker/errdefs"
ocispec "github.com/opencontainers/image-spec/specs-go/v1"
v1 "github.com/opencontainers/image-spec/specs-go/v1"
"github.com/pkg/errors"
)
@ -26,10 +26,25 @@ func (s *distributionRouter) getDistributionInfo(ctx context.Context, w http.Res
w.Header().Set("Content-Type", "application/json")
imgName := vars["name"]
var (
config = &types.AuthConfig{}
authEncoded = r.Header.Get("X-Registry-Auth")
distributionInspect registrytypes.DistributionInspect
)
if authEncoded != "" {
authJSON := base64.NewDecoder(base64.URLEncoding, strings.NewReader(authEncoded))
if err := json.NewDecoder(authJSON).Decode(&config); err != nil {
// for a search it is not an error if no auth was given
// to increase compatibility with the existing api it is defaulting to be empty
config = &types.AuthConfig{}
}
}
image := vars["name"]
// TODO why is reference.ParseAnyReference() / reference.ParseNormalizedNamed() not using the reference.ErrTagInvalidFormat (and so on) errors?
ref, err := reference.ParseAnyReference(imgName)
ref, err := reference.ParseAnyReference(image)
if err != nil {
return errdefs.InvalidParameter(err)
}
@ -39,58 +54,28 @@ func (s *distributionRouter) getDistributionInfo(ctx context.Context, w http.Res
// full image ID
return errors.Errorf("no manifest found for full image ID")
}
return errdefs.InvalidParameter(errors.Errorf("unknown image reference format: %s", imgName))
return errdefs.InvalidParameter(errors.Errorf("unknown image reference format: %s", image))
}
// For a search it is not an error if no auth was given. Ignore invalid
// AuthConfig to increase compatibility with the existing API.
authConfig, _ := registry.DecodeAuthConfig(r.Header.Get(registry.AuthHeader))
repos, err := s.backend.GetRepositories(ctx, namedRef, authConfig)
distrepo, _, err := s.backend.GetRepository(ctx, namedRef, config)
if err != nil {
return err
}
blobsrvc := distrepo.Blobs(ctx)
// Fetch the manifest; if a mirror is configured, try the mirror first,
// but continue with upstream on failure.
//
// FIXME(thaJeztah): construct "repositories" on-demand;
// GetRepositories() will attempt to connect to all endpoints (registries),
// but we may only need the first one if it contains the manifest we're
// looking for, or if the configured mirror is a pull-through mirror.
//
// Logic for this could be implemented similar to "distribution.Pull()",
// which uses the "pullEndpoints" utility to iterate over the list
// of endpoints;
//
// - https://github.com/moby/moby/blob/12c7411b6b7314bef130cd59f1c7384a7db06d0b/distribution/pull.go#L17-L31
// - https://github.com/moby/moby/blob/12c7411b6b7314bef130cd59f1c7384a7db06d0b/distribution/pull.go#L76-L152
var lastErr error
for _, repo := range repos {
distributionInspect, err := s.fetchManifest(ctx, repo, namedRef)
if err != nil {
lastErr = err
continue
}
return httputils.WriteJSON(w, http.StatusOK, distributionInspect)
}
return lastErr
}
func (s *distributionRouter) fetchManifest(ctx context.Context, distrepo distribution.Repository, namedRef reference.Named) (registry.DistributionInspect, error) {
var distributionInspect registry.DistributionInspect
if canonicalRef, ok := namedRef.(reference.Canonical); !ok {
namedRef = reference.TagNameOnly(namedRef)
taggedRef, ok := namedRef.(reference.NamedTagged)
if !ok {
return registry.DistributionInspect{}, errdefs.InvalidParameter(errors.Errorf("image reference not tagged: %s", namedRef))
return errdefs.InvalidParameter(errors.Errorf("image reference not tagged: %s", image))
}
descriptor, err := distrepo.Tags(ctx).Get(ctx, taggedRef.Tag())
if err != nil {
return registry.DistributionInspect{}, err
return err
}
distributionInspect.Descriptor = ocispec.Descriptor{
distributionInspect.Descriptor = v1.Descriptor{
MediaType: descriptor.MediaType,
Digest: descriptor.Digest,
Size: descriptor.Size,
@ -105,7 +90,7 @@ func (s *distributionRouter) fetchManifest(ctx context.Context, distrepo distrib
// we have a digest, so we can retrieve the manifest
mnfstsrvc, err := distrepo.Manifests(ctx)
if err != nil {
return registry.DistributionInspect{}, err
return err
}
mnfst, err := mnfstsrvc.Get(ctx, distributionInspect.Descriptor.Digest)
if err != nil {
@ -117,14 +102,14 @@ func (s *distributionRouter) fetchManifest(ctx context.Context, distrepo distrib
reference.ErrNameEmpty,
reference.ErrNameTooLong,
reference.ErrNameNotCanonical:
return registry.DistributionInspect{}, errdefs.InvalidParameter(err)
return errdefs.InvalidParameter(err)
}
return registry.DistributionInspect{}, err
return err
}
mediaType, payload, err := mnfst.Payload()
if err != nil {
return registry.DistributionInspect{}, err
return err
}
// update MediaType because registry might return something incorrect
distributionInspect.Descriptor.MediaType = mediaType
@ -136,7 +121,7 @@ func (s *distributionRouter) fetchManifest(ctx context.Context, distrepo distrib
switch mnfstObj := mnfst.(type) {
case *manifestlist.DeserializedManifestList:
for _, m := range mnfstObj.Manifests {
distributionInspect.Platforms = append(distributionInspect.Platforms, ocispec.Platform{
distributionInspect.Platforms = append(distributionInspect.Platforms, v1.Platform{
Architecture: m.Platform.Architecture,
OS: m.Platform.OS,
OSVersion: m.Platform.OSVersion,
@ -145,9 +130,8 @@ func (s *distributionRouter) fetchManifest(ctx context.Context, distrepo distrib
})
}
case *schema2.DeserializedManifest:
blobStore := distrepo.Blobs(ctx)
configJSON, err := blobStore.Get(ctx, mnfstObj.Config.Digest)
var platform ocispec.Platform
configJSON, err := blobsrvc.Get(ctx, mnfstObj.Config.Digest)
var platform v1.Platform
if err == nil {
err := json.Unmarshal(configJSON, &platform)
if err == nil && (platform.OS != "" || platform.Architecture != "") {
@ -155,14 +139,12 @@ func (s *distributionRouter) fetchManifest(ctx context.Context, distrepo distrib
}
}
case *schema1.SignedManifest:
if os.Getenv("DOCKER_ENABLE_DEPRECATED_PULL_SCHEMA_1_IMAGE") == "" {
return registry.DistributionInspect{}, distributionpkg.DeprecatedSchema1ImageError(namedRef)
}
platform := ocispec.Platform{
platform := v1.Platform{
Architecture: mnfstObj.Architecture,
OS: "linux",
}
distributionInspect.Platforms = append(distributionInspect.Platforms, platform)
}
return distributionInspect, nil
return httputils.WriteJSON(w, http.StatusOK, distributionInspect)
}

View file

@ -1,13 +1,8 @@
package grpc // import "github.com/docker/docker/api/server/router/grpc"
import (
"context"
"strings"
"github.com/docker/docker/api/server/router"
grpc_middleware "github.com/grpc-ecosystem/go-grpc-middleware"
"github.com/moby/buildkit/util/grpcerrors"
"go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc"
"golang.org/x/net/http2"
"google.golang.org/grpc"
)
@ -20,12 +15,12 @@ type grpcRouter struct {
// NewRouter initializes a new grpc http router
func NewRouter(backends ...Backend) router.Router {
unary := grpc.UnaryInterceptor(grpc_middleware.ChainUnaryServer(unaryInterceptor(), grpcerrors.UnaryServerInterceptor))
stream := grpc.StreamInterceptor(grpc_middleware.ChainStreamServer(otelgrpc.StreamServerInterceptor(), grpcerrors.StreamServerInterceptor)) //nolint:staticcheck // TODO(thaJeztah): ignore SA1019 for deprecated options: see https://github.com/moby/moby/issues/47437
r := &grpcRouter{
h2Server: &http2.Server{},
grpcServer: grpc.NewServer(unary, stream),
h2Server: &http2.Server{},
grpcServer: grpc.NewServer(
grpc.UnaryInterceptor(grpcerrors.UnaryServerInterceptor),
grpc.StreamInterceptor(grpcerrors.StreamServerInterceptor),
),
}
for _, b := range backends {
b.RegisterGRPC(r.grpcServer)
@ -44,17 +39,3 @@ func (gr *grpcRouter) initRoutes() {
router.NewPostRoute("/grpc", gr.serveGRPC),
}
}
func unaryInterceptor() grpc.UnaryServerInterceptor {
withTrace := otelgrpc.UnaryServerInterceptor() //nolint:staticcheck // TODO(thaJeztah): ignore SA1019 for deprecated options: see https://github.com/moby/moby/issues/47437
return func(ctx context.Context, req interface{}, info *grpc.UnaryServerInfo, handler grpc.UnaryHandler) (resp interface{}, err error) {
// This method is used by the clients to send their traces to buildkit so they can be included
// in the daemon trace and stored in the build history record. This method can not be traced because
// it would cause an infinite loop.
if strings.HasSuffix(info.FullMethod, "opentelemetry.proto.collector.trace.v1.TraceService/Export") {
return handler(ctx, req)
}
return withTrace(ctx, req, info, handler)
}
}

View file

@ -4,14 +4,11 @@ import (
"context"
"io"
"github.com/distribution/reference"
"github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/backend"
"github.com/docker/docker/api/types/filters"
"github.com/docker/docker/api/types/image"
"github.com/docker/docker/api/types/registry"
dockerimage "github.com/docker/docker/image"
ocispec "github.com/opencontainers/image-spec/specs-go/v1"
specs "github.com/opencontainers/image-spec/specs-go/v1"
)
// Backend is all the methods that need to be implemented
@ -23,25 +20,22 @@ type Backend interface {
}
type imageBackend interface {
ImageDelete(ctx context.Context, imageRef string, force, prune bool) ([]image.DeleteResponse, error)
ImageHistory(ctx context.Context, imageName string) ([]*image.HistoryResponseItem, error)
Images(ctx context.Context, opts image.ListOptions) ([]*image.Summary, error)
GetImage(ctx context.Context, refOrID string, options backend.GetImageOpts) (*dockerimage.Image, error)
TagImage(ctx context.Context, id dockerimage.ID, newRef reference.Named) error
ImageDelete(imageRef string, force, prune bool) ([]types.ImageDeleteResponseItem, error)
ImageHistory(imageName string) ([]*image.HistoryResponseItem, error)
Images(imageFilters filters.Args, all bool, withExtraAttrs bool) ([]*types.ImageSummary, error)
LookupImage(name string) (*types.ImageInspect, error)
TagImage(imageName, repository, tag string) (string, error)
ImagesPrune(ctx context.Context, pruneFilters filters.Args) (*types.ImagesPruneReport, error)
}
type importExportBackend interface {
LoadImage(ctx context.Context, inTar io.ReadCloser, outStream io.Writer, quiet bool) error
ImportImage(ctx context.Context, ref reference.Named, platform *ocispec.Platform, msg string, layerReader io.Reader, changes []string) (dockerimage.ID, error)
ExportImage(ctx context.Context, names []string, outStream io.Writer) error
LoadImage(inTar io.ReadCloser, outStream io.Writer, quiet bool) error
ImportImage(src string, repository, platform string, tag string, msg string, inConfig io.ReadCloser, outStream io.Writer, changes []string) error
ExportImage(names []string, outStream io.Writer) error
}
type registryBackend interface {
PullImage(ctx context.Context, ref reference.Named, platform *ocispec.Platform, metaHeaders map[string][]string, authConfig *registry.AuthConfig, outStream io.Writer) error
PushImage(ctx context.Context, ref reference.Named, metaHeaders map[string][]string, authConfig *registry.AuthConfig, outStream io.Writer) error
}
type Searcher interface {
Search(ctx context.Context, searchFilters filters.Args, term string, limit int, authConfig *registry.AuthConfig, headers map[string][]string) ([]registry.SearchResult, error)
PullImage(ctx context.Context, image, tag string, platform *specs.Platform, metaHeaders map[string][]string, authConfig *types.AuthConfig, outStream io.Writer) error
PushImage(ctx context.Context, image, tag string, metaHeaders map[string][]string, authConfig *types.AuthConfig, outStream io.Writer) error
SearchRegistryForImages(ctx context.Context, filtersArgs string, term string, limit int, authConfig *types.AuthConfig, metaHeaders map[string][]string) (*registry.SearchResults, error)
}

View file

@ -2,56 +2,43 @@ package image // import "github.com/docker/docker/api/server/router/image"
import (
"github.com/docker/docker/api/server/router"
"github.com/docker/docker/image"
"github.com/docker/docker/layer"
"github.com/docker/docker/reference"
)
// imageRouter is a router to talk with the image controller
type imageRouter struct {
backend Backend
searcher Searcher
referenceBackend reference.Store
imageStore image.Store
layerStore layer.Store
routes []router.Route
backend Backend
routes []router.Route
}
// NewRouter initializes a new image router
func NewRouter(backend Backend, searcher Searcher, referenceBackend reference.Store, imageStore image.Store, layerStore layer.Store) router.Router {
ir := &imageRouter{
backend: backend,
searcher: searcher,
referenceBackend: referenceBackend,
imageStore: imageStore,
layerStore: layerStore,
}
ir.initRoutes()
return ir
func NewRouter(backend Backend) router.Router {
r := &imageRouter{backend: backend}
r.initRoutes()
return r
}
// Routes returns the available routes to the image controller
func (ir *imageRouter) Routes() []router.Route {
return ir.routes
func (r *imageRouter) Routes() []router.Route {
return r.routes
}
// initRoutes initializes the routes in the image router
func (ir *imageRouter) initRoutes() {
ir.routes = []router.Route{
func (r *imageRouter) initRoutes() {
r.routes = []router.Route{
// GET
router.NewGetRoute("/images/json", ir.getImagesJSON),
router.NewGetRoute("/images/search", ir.getImagesSearch),
router.NewGetRoute("/images/get", ir.getImagesGet),
router.NewGetRoute("/images/{name:.*}/get", ir.getImagesGet),
router.NewGetRoute("/images/{name:.*}/history", ir.getImagesHistory),
router.NewGetRoute("/images/{name:.*}/json", ir.getImagesByName),
router.NewGetRoute("/images/json", r.getImagesJSON),
router.NewGetRoute("/images/search", r.getImagesSearch),
router.NewGetRoute("/images/get", r.getImagesGet),
router.NewGetRoute("/images/{name:.*}/get", r.getImagesGet),
router.NewGetRoute("/images/{name:.*}/history", r.getImagesHistory),
router.NewGetRoute("/images/{name:.*}/json", r.getImagesByName),
// POST
router.NewPostRoute("/images/load", ir.postImagesLoad),
router.NewPostRoute("/images/create", ir.postImagesCreate),
router.NewPostRoute("/images/{name:.*}/push", ir.postImagesPush),
router.NewPostRoute("/images/{name:.*}/tag", ir.postImagesTag),
router.NewPostRoute("/images/prune", ir.postImagesPrune),
router.NewPostRoute("/images/load", r.postImagesLoad),
router.NewPostRoute("/images/create", r.postImagesCreate),
router.NewPostRoute("/images/{name:.*}/push", r.postImagesPush),
router.NewPostRoute("/images/{name:.*}/tag", r.postImagesTag),
router.NewPostRoute("/images/prune", r.postImagesPrune),
// DELETE
router.NewDeleteRoute("/images/{name:.*}", ir.deleteImages),
router.NewDeleteRoute("/images/{name:.*}", r.deleteImages),
}
}

View file

@ -2,50 +2,41 @@ package image // import "github.com/docker/docker/api/server/router/image"
import (
"context"
"fmt"
"io"
"encoding/base64"
"encoding/json"
"net/http"
"net/url"
"strconv"
"strings"
"time"
"github.com/containerd/containerd/platforms"
"github.com/distribution/reference"
"github.com/docker/docker/api"
"github.com/docker/docker/api/server/httputils"
"github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/backend"
"github.com/docker/docker/api/types/filters"
imagetypes "github.com/docker/docker/api/types/image"
"github.com/docker/docker/api/types/registry"
"github.com/docker/docker/api/types/versions"
"github.com/docker/docker/builder/remotecontext"
"github.com/docker/docker/dockerversion"
"github.com/docker/docker/errdefs"
"github.com/docker/docker/image"
"github.com/docker/docker/pkg/ioutils"
"github.com/docker/docker/pkg/progress"
"github.com/docker/docker/pkg/streamformatter"
"github.com/opencontainers/go-digest"
ocispec "github.com/opencontainers/image-spec/specs-go/v1"
"github.com/docker/docker/pkg/system"
"github.com/docker/docker/registry"
specs "github.com/opencontainers/image-spec/specs-go/v1"
"github.com/pkg/errors"
)
// Creates an image from Pull or from Import
func (ir *imageRouter) postImagesCreate(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
func (s *imageRouter) postImagesCreate(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
if err := httputils.ParseForm(r); err != nil {
return err
}
var (
img = r.Form.Get("fromImage")
repo = r.Form.Get("repo")
tag = r.Form.Get("tag")
comment = r.Form.Get("message")
progressErr error
output = ioutils.NewWriteFlusher(w)
platform *ocispec.Platform
image = r.Form.Get("fromImage")
repo = r.Form.Get("repo")
tag = r.Form.Get("tag")
message = r.Form.Get("message")
err error
output = ioutils.NewWriteFlusher(w)
platform *specs.Platform
)
defer output.Close()
@ -53,16 +44,20 @@ func (ir *imageRouter) postImagesCreate(ctx context.Context, w http.ResponseWrit
version := httputils.VersionFromContext(ctx)
if versions.GreaterThanOrEqualTo(version, "1.32") {
if p := r.FormValue("platform"); p != "" {
sp, err := platforms.Parse(p)
apiPlatform := r.FormValue("platform")
if apiPlatform != "" {
sp, err := platforms.Parse(apiPlatform)
if err != nil {
return err
}
if err := system.ValidatePlatform(sp); err != nil {
return err
}
platform = &sp
}
}
if img != "" { // pull
if image != "" { // pull
metaHeaders := map[string][]string{}
for k, v := range r.Header {
if strings.HasPrefix(k, "X-Meta-") {
@ -70,92 +65,39 @@ func (ir *imageRouter) postImagesCreate(ctx context.Context, w http.ResponseWrit
}
}
// Special case: "pull -a" may send an image name with a
// trailing :. This is ugly, but let's not break API
// compatibility.
imgName := strings.TrimSuffix(img, ":")
ref, err := reference.ParseNormalizedNamed(imgName)
if err != nil {
return errdefs.InvalidParameter(err)
}
// TODO(thaJeztah) this could use a WithTagOrDigest() utility
if tag != "" {
// The "tag" could actually be a digest.
var dgst digest.Digest
dgst, err = digest.Parse(tag)
if err == nil {
ref, err = reference.WithDigest(reference.TrimNamed(ref), dgst)
} else {
ref, err = reference.WithTag(ref, tag)
}
if err != nil {
return errdefs.InvalidParameter(err)
authEncoded := r.Header.Get("X-Registry-Auth")
authConfig := &types.AuthConfig{}
if authEncoded != "" {
authJSON := base64.NewDecoder(base64.URLEncoding, strings.NewReader(authEncoded))
if err := json.NewDecoder(authJSON).Decode(authConfig); err != nil {
// for a pull it is not an error if no auth was given
// to increase compatibility with the existing api it is defaulting to be empty
authConfig = &types.AuthConfig{}
}
}
if err := validateRepoName(ref); err != nil {
return errdefs.Forbidden(err)
}
// For a pull it is not an error if no auth was given. Ignore invalid
// AuthConfig to increase compatibility with the existing API.
authConfig, _ := registry.DecodeAuthConfig(r.Header.Get(registry.AuthHeader))
progressErr = ir.backend.PullImage(ctx, ref, platform, metaHeaders, authConfig, output)
err = s.backend.PullImage(ctx, image, tag, platform, metaHeaders, authConfig, output)
} else { // import
src := r.Form.Get("fromSrc")
tagRef, err := httputils.RepoTagReference(repo, tag)
if err != nil {
return errdefs.InvalidParameter(err)
}
if len(comment) == 0 {
comment = "Imported from " + src
}
var layerReader io.ReadCloser
defer r.Body.Close()
if src == "-" {
layerReader = r.Body
} else {
if len(strings.Split(src, "://")) == 1 {
src = "http://" + src
}
u, err := url.Parse(src)
if err != nil {
return errdefs.InvalidParameter(err)
}
resp, err := remotecontext.GetWithStatusError(u.String())
if err != nil {
return err
}
output.Write(streamformatter.FormatStatus("", "Downloading from %s", u))
progressOutput := streamformatter.NewJSONProgressOutput(output, true)
layerReader = progress.NewProgressReader(resp.Body, progressOutput, resp.ContentLength, "", "Importing")
defer layerReader.Close()
}
var id image.ID
id, progressErr = ir.backend.ImportImage(ctx, tagRef, platform, comment, layerReader, r.Form["changes"])
if progressErr == nil {
output.Write(streamformatter.FormatStatus("", id.String()))
// 'err' MUST NOT be defined within this block, we need any error
// generated from the download to be available to the output
// stream processing below
os := ""
if platform != nil {
os = platform.OS
}
err = s.backend.ImportImage(src, repo, os, tag, message, r.Body, output, r.Form["changes"])
}
if progressErr != nil {
if err != nil {
if !output.Flushed() {
return progressErr
return err
}
_, _ = output.Write(streamformatter.FormatError(progressErr))
_, _ = output.Write(streamformatter.FormatError(err))
}
return nil
}
func (ir *imageRouter) postImagesPush(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
func (s *imageRouter) postImagesPush(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
metaHeaders := map[string][]string{}
for k, v := range r.Header {
if strings.HasPrefix(k, "X-Meta-") {
@ -165,47 +107,32 @@ func (ir *imageRouter) postImagesPush(ctx context.Context, w http.ResponseWriter
if err := httputils.ParseForm(r); err != nil {
return err
}
authConfig := &types.AuthConfig{}
var authConfig *registry.AuthConfig
if authEncoded := r.Header.Get(registry.AuthHeader); authEncoded != "" {
// the new format is to handle the authConfig as a header. Ignore invalid
// AuthConfig to increase compatibility with the existing API.
authConfig, _ = registry.DecodeAuthConfig(authEncoded)
authEncoded := r.Header.Get("X-Registry-Auth")
if authEncoded != "" {
// the new format is to handle the authConfig as a header
authJSON := base64.NewDecoder(base64.URLEncoding, strings.NewReader(authEncoded))
if err := json.NewDecoder(authJSON).Decode(authConfig); err != nil {
// to increase compatibility to existing api it is defaulting to be empty
authConfig = &types.AuthConfig{}
}
} else {
// the old format is supported for compatibility if there was no authConfig header
var err error
authConfig, err = registry.DecodeAuthConfigBody(r.Body)
if err != nil {
return errors.Wrap(err, "bad parameters and missing X-Registry-Auth")
if err := json.NewDecoder(r.Body).Decode(authConfig); err != nil {
return errors.Wrap(errdefs.InvalidParameter(err), "Bad parameters and missing X-Registry-Auth")
}
}
image := vars["name"]
tag := r.Form.Get("tag")
output := ioutils.NewWriteFlusher(w)
defer output.Close()
w.Header().Set("Content-Type", "application/json")
img := vars["name"]
tag := r.Form.Get("tag")
var ref reference.Named
// Tag is empty only in case PushOptions.All is true.
if tag != "" {
r, err := httputils.RepoTagReference(img, tag)
if err != nil {
return errdefs.InvalidParameter(err)
}
ref = r
} else {
r, err := reference.ParseNormalizedNamed(img)
if err != nil {
return errdefs.InvalidParameter(err)
}
ref = r
}
if err := ir.backend.PushImage(ctx, ref, metaHeaders, authConfig, output); err != nil {
if err := s.backend.PushImage(ctx, image, tag, metaHeaders, authConfig, output); err != nil {
if !output.Flushed() {
return err
}
@ -214,7 +141,7 @@ func (ir *imageRouter) postImagesPush(ctx context.Context, w http.ResponseWriter
return nil
}
func (ir *imageRouter) getImagesGet(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
func (s *imageRouter) getImagesGet(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
if err := httputils.ParseForm(r); err != nil {
return err
}
@ -230,7 +157,7 @@ func (ir *imageRouter) getImagesGet(ctx context.Context, w http.ResponseWriter,
names = r.Form["names"]
}
if err := ir.backend.ExportImage(ctx, names, output); err != nil {
if err := s.backend.ExportImage(names, output); err != nil {
if !output.Flushed() {
return err
}
@ -239,7 +166,7 @@ func (ir *imageRouter) getImagesGet(ctx context.Context, w http.ResponseWriter,
return nil
}
func (ir *imageRouter) postImagesLoad(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
func (s *imageRouter) postImagesLoad(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
if err := httputils.ParseForm(r); err != nil {
return err
}
@ -249,7 +176,7 @@ func (ir *imageRouter) postImagesLoad(ctx context.Context, w http.ResponseWriter
output := ioutils.NewWriteFlusher(w)
defer output.Close()
if err := ir.backend.LoadImage(ctx, r.Body, output, quiet); err != nil {
if err := s.backend.LoadImage(r.Body, output, quiet); err != nil {
_, _ = output.Write(streamformatter.FormatError(err))
}
return nil
@ -263,7 +190,7 @@ func (missingImageError) Error() string {
func (missingImageError) InvalidParameter() {}
func (ir *imageRouter) deleteImages(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
func (s *imageRouter) deleteImages(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
if err := httputils.ParseForm(r); err != nil {
return err
}
@ -277,7 +204,7 @@ func (ir *imageRouter) deleteImages(ctx context.Context, w http.ResponseWriter,
force := httputils.BoolValue(r, "force")
prune := !httputils.BoolValue(r, "noprune")
list, err := ir.backend.ImageDelete(ctx, name, force, prune)
list, err := s.backend.ImageDelete(name, force, prune)
if err != nil {
return err
}
@ -285,103 +212,16 @@ func (ir *imageRouter) deleteImages(ctx context.Context, w http.ResponseWriter,
return httputils.WriteJSON(w, http.StatusOK, list)
}
func (ir *imageRouter) getImagesByName(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
img, err := ir.backend.GetImage(ctx, vars["name"], backend.GetImageOpts{Details: true})
func (s *imageRouter) getImagesByName(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
imageInspect, err := s.backend.LookupImage(vars["name"])
if err != nil {
return err
}
imageInspect, err := ir.toImageInspect(img)
if err != nil {
return err
}
version := httputils.VersionFromContext(ctx)
if versions.LessThan(version, "1.44") {
imageInspect.VirtualSize = imageInspect.Size //nolint:staticcheck // ignore SA1019: field is deprecated, but still set on API < v1.44.
if imageInspect.Created == "" {
// backwards compatibility for Created not existing returning "0001-01-01T00:00:00Z"
// https://github.com/moby/moby/issues/47368
imageInspect.Created = time.Time{}.Format(time.RFC3339Nano)
}
}
if versions.GreaterThanOrEqualTo(version, "1.45") {
imageInspect.Container = "" //nolint:staticcheck // ignore SA1019: field is deprecated, but still set on API < v1.45.
imageInspect.ContainerConfig = nil //nolint:staticcheck // ignore SA1019: field is deprecated, but still set on API < v1.45.
}
return httputils.WriteJSON(w, http.StatusOK, imageInspect)
}
func (ir *imageRouter) toImageInspect(img *image.Image) (*types.ImageInspect, error) {
var repoTags, repoDigests []string
for _, ref := range img.Details.References {
switch ref.(type) {
case reference.NamedTagged:
repoTags = append(repoTags, reference.FamiliarString(ref))
case reference.Canonical:
repoDigests = append(repoDigests, reference.FamiliarString(ref))
}
}
comment := img.Comment
if len(comment) == 0 && len(img.History) > 0 {
comment = img.History[len(img.History)-1].Comment
}
// Make sure we output empty arrays instead of nil.
if repoTags == nil {
repoTags = []string{}
}
if repoDigests == nil {
repoDigests = []string{}
}
var created string
if img.Created != nil {
created = img.Created.Format(time.RFC3339Nano)
}
return &types.ImageInspect{
ID: img.ID().String(),
RepoTags: repoTags,
RepoDigests: repoDigests,
Parent: img.Parent.String(),
Comment: comment,
Created: created,
Container: img.Container, //nolint:staticcheck // ignore SA1019: field is deprecated, but still set on API < v1.45.
ContainerConfig: &img.ContainerConfig, //nolint:staticcheck // ignore SA1019: field is deprecated, but still set on API < v1.45.
DockerVersion: img.DockerVersion,
Author: img.Author,
Config: img.Config,
Architecture: img.Architecture,
Variant: img.Variant,
Os: img.OperatingSystem(),
OsVersion: img.OSVersion,
Size: img.Details.Size,
GraphDriver: types.GraphDriverData{
Name: img.Details.Driver,
Data: img.Details.Metadata,
},
RootFS: rootFSToAPIType(img.RootFS),
Metadata: imagetypes.Metadata{
LastTagTime: img.Details.LastUpdated,
},
}, nil
}
func rootFSToAPIType(rootfs *image.RootFS) types.RootFS {
var layers []string
for _, l := range rootfs.DiffIDs {
layers = append(layers, l.String())
}
return types.RootFS{
Type: rootfs.Type,
Layers: layers,
}
}
func (ir *imageRouter) getImagesJSON(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
func (s *imageRouter) getImagesJSON(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
if err := httputils.ParseForm(r); err != nil {
return err
}
@ -393,54 +233,23 @@ func (ir *imageRouter) getImagesJSON(ctx context.Context, w http.ResponseWriter,
version := httputils.VersionFromContext(ctx)
if versions.LessThan(version, "1.41") {
// NOTE: filter is a shell glob string applied to repository names.
filterParam := r.Form.Get("filter")
if filterParam != "" {
imageFilters.Add("reference", filterParam)
}
}
var sharedSize bool
if versions.GreaterThanOrEqualTo(version, "1.42") {
// NOTE: Support for the "shared-size" parameter was added in API 1.42.
sharedSize = httputils.BoolValue(r, "shared-size")
}
images, err := ir.backend.Images(ctx, imagetypes.ListOptions{
All: httputils.BoolValue(r, "all"),
Filters: imageFilters,
SharedSize: sharedSize,
})
images, err := s.backend.Images(imageFilters, httputils.BoolValue(r, "all"), false)
if err != nil {
return err
}
useNone := versions.LessThan(version, "1.43")
withVirtualSize := versions.LessThan(version, "1.44")
for _, img := range images {
if useNone {
if len(img.RepoTags) == 0 && len(img.RepoDigests) == 0 {
img.RepoTags = append(img.RepoTags, "<none>:<none>")
img.RepoDigests = append(img.RepoDigests, "<none>@<none>")
}
} else {
if img.RepoTags == nil {
img.RepoTags = []string{}
}
if img.RepoDigests == nil {
img.RepoDigests = []string{}
}
}
if withVirtualSize {
img.VirtualSize = img.Size //nolint:staticcheck // ignore SA1019: field is deprecated, but still set on API < v1.44.
}
}
return httputils.WriteJSON(w, http.StatusOK, images)
}
func (ir *imageRouter) getImagesHistory(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
history, err := ir.backend.ImageHistory(ctx, vars["name"])
func (s *imageRouter) getImagesHistory(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
name := vars["name"]
history, err := s.backend.ImageHistory(name)
if err != nil {
return err
}
@ -448,71 +257,56 @@ func (ir *imageRouter) getImagesHistory(ctx context.Context, w http.ResponseWrit
return httputils.WriteJSON(w, http.StatusOK, history)
}
func (ir *imageRouter) postImagesTag(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
func (s *imageRouter) postImagesTag(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
if err := httputils.ParseForm(r); err != nil {
return err
}
ref, err := httputils.RepoTagReference(r.Form.Get("repo"), r.Form.Get("tag"))
if ref == nil || err != nil {
return errdefs.InvalidParameter(err)
}
refName := reference.FamiliarName(ref)
if refName == string(digest.Canonical) {
return errdefs.InvalidParameter(errors.New("refusing to create an ambiguous tag using digest algorithm as name"))
}
img, err := ir.backend.GetImage(ctx, vars["name"], backend.GetImageOpts{})
if err != nil {
return errdefs.NotFound(err)
}
if err := ir.backend.TagImage(ctx, img.ID(), ref); err != nil {
if _, err := s.backend.TagImage(vars["name"], r.Form.Get("repo"), r.Form.Get("tag")); err != nil {
return err
}
w.WriteHeader(http.StatusCreated)
return nil
}
func (ir *imageRouter) getImagesSearch(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
func (s *imageRouter) getImagesSearch(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
if err := httputils.ParseForm(r); err != nil {
return err
}
var (
config *types.AuthConfig
authEncoded = r.Header.Get("X-Registry-Auth")
headers = map[string][]string{}
)
var limit int
if r.Form.Get("limit") != "" {
var err error
limit, err = strconv.Atoi(r.Form.Get("limit"))
if err != nil || limit < 0 {
return errdefs.InvalidParameter(errors.Wrap(err, "invalid limit specified"))
if authEncoded != "" {
authJSON := base64.NewDecoder(base64.URLEncoding, strings.NewReader(authEncoded))
if err := json.NewDecoder(authJSON).Decode(&config); err != nil {
// for a search it is not an error if no auth was given
// to increase compatibility with the existing api it is defaulting to be empty
config = &types.AuthConfig{}
}
}
searchFilters, err := filters.FromJSON(r.Form.Get("filters"))
if err != nil {
return err
}
// For a search it is not an error if no auth was given. Ignore invalid
// AuthConfig to increase compatibility with the existing API.
authConfig, _ := registry.DecodeAuthConfig(r.Header.Get(registry.AuthHeader))
headers := http.Header{}
for k, v := range r.Header {
k = http.CanonicalHeaderKey(k)
if strings.HasPrefix(k, "X-Meta-") {
headers[k] = v
}
}
headers.Set("User-Agent", dockerversion.DockerUserAgent(ctx))
res, err := ir.searcher.Search(ctx, searchFilters, r.Form.Get("term"), limit, authConfig, headers)
limit := registry.DefaultSearchLimit
if r.Form.Get("limit") != "" {
limitValue, err := strconv.Atoi(r.Form.Get("limit"))
if err != nil {
return err
}
limit = limitValue
}
query, err := s.backend.SearchRegistryForImages(ctx, r.Form.Get("filters"), r.Form.Get("term"), limit, config, headers)
if err != nil {
return err
}
return httputils.WriteJSON(w, http.StatusOK, res)
return httputils.WriteJSON(w, http.StatusOK, query.Results)
}
func (ir *imageRouter) postImagesPrune(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
func (s *imageRouter) postImagesPrune(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
if err := httputils.ParseForm(r); err != nil {
return err
}
@ -522,18 +316,9 @@ func (ir *imageRouter) postImagesPrune(ctx context.Context, w http.ResponseWrite
return err
}
pruneReport, err := ir.backend.ImagesPrune(ctx, pruneFilters)
pruneReport, err := s.backend.ImagesPrune(ctx, pruneFilters)
if err != nil {
return err
}
return httputils.WriteJSON(w, http.StatusOK, pruneReport)
}
// validateRepoName validates the name of a repository.
func validateRepoName(name reference.Named) error {
familiarName := reference.FamiliarName(name)
if familiarName == api.NoBaseImageSpecifier {
return fmt.Errorf("'%s' is a reserved name", familiarName)
}
return nil
}

View file

@ -4,15 +4,16 @@ import (
"context"
"github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/backend"
"github.com/docker/docker/api/types/filters"
"github.com/docker/docker/api/types/network"
"github.com/docker/libnetwork"
)
// Backend is all the methods that need to be implemented
// to provide network specific functionality.
type Backend interface {
GetNetworks(filters.Args, backend.NetworkListConfig) ([]types.NetworkResource, error)
FindNetwork(idName string) (libnetwork.Network, error)
GetNetworks(filters.Args, types.NetworkListConfig) ([]types.NetworkResource, error)
CreateNetwork(nc types.NetworkCreateRequest) (*types.NetworkCreateResponse, error)
ConnectContainerToNetwork(containerName, networkName string, endpointConfig *network.EndpointSettings) error
DisconnectContainerFromNetwork(containerName string, networkName string, force bool) error

View file

@ -0,0 +1 @@
package network // import "github.com/docker/docker/api/server/router/network"

View file

@ -2,19 +2,20 @@ package network // import "github.com/docker/docker/api/server/router/network"
import (
"context"
"encoding/json"
"io"
"net/http"
"strconv"
"strings"
"github.com/docker/docker/api/server/httputils"
"github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/backend"
"github.com/docker/docker/api/types/filters"
"github.com/docker/docker/api/types/network"
"github.com/docker/docker/api/types/versions"
"github.com/docker/docker/errdefs"
"github.com/docker/docker/libnetwork"
"github.com/docker/docker/libnetwork/scope"
"github.com/docker/libnetwork"
netconst "github.com/docker/libnetwork/datastore"
"github.com/pkg/errors"
)
@ -29,7 +30,7 @@ func (n *networkRouter) getNetworksList(ctx context.Context, w http.ResponseWrit
}
if err := network.ValidateFilters(filter); err != nil {
return err
return errdefs.InvalidParameter(err)
}
var list []types.NetworkResource
@ -40,7 +41,7 @@ func (n *networkRouter) getNetworksList(ctx context.Context, w http.ResponseWrit
// Combine the network list returned by Docker daemon if it is not already
// returned by the cluster manager
localNetworks, err := n.backend.GetNetworks(filter, backend.NetworkListConfig{Detailed: versions.LessThan(httputils.VersionFromContext(ctx), "1.28")})
localNetworks, err := n.backend.GetNetworks(filter, types.NetworkListConfig{Detailed: versions.LessThan(httputils.VersionFromContext(ctx), "1.28")})
if err != nil {
return err
}
@ -84,6 +85,10 @@ func (e ambigousResultsError) Error() string {
func (ambigousResultsError) InvalidParameter() {}
func nameConflict(name string) error {
return errdefs.Conflict(libnetwork.NetworkNameError(name))
}
func (n *networkRouter) getNetwork(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
if err := httputils.ParseForm(r); err != nil {
return err
@ -99,7 +104,7 @@ func (n *networkRouter) getNetwork(ctx context.Context, w http.ResponseWriter, r
return errors.Wrapf(invalidRequestError{err}, "invalid value for verbose: %s", v)
}
}
networkScope := r.URL.Query().Get("scope")
scope := r.URL.Query().Get("scope")
// In case multiple networks have duplicate names, return error.
// TODO (yongtang): should we wrap with version here for backward compatibility?
@ -115,23 +120,23 @@ func (n *networkRouter) getNetwork(ctx context.Context, w http.ResponseWriter, r
// TODO(@cpuguy83): All this logic for figuring out which network to return does not belong here
// Instead there should be a backend function to just get one network.
filter := filters.NewArgs(filters.Arg("idOrName", term))
if networkScope != "" {
filter.Add("scope", networkScope)
if scope != "" {
filter.Add("scope", scope)
}
networks, _ := n.backend.GetNetworks(filter, backend.NetworkListConfig{Detailed: true, Verbose: verbose})
for _, nw := range networks {
if nw.ID == term {
return httputils.WriteJSON(w, http.StatusOK, nw)
nw, _ := n.backend.GetNetworks(filter, types.NetworkListConfig{Detailed: true, Verbose: verbose})
for _, network := range nw {
if network.ID == term {
return httputils.WriteJSON(w, http.StatusOK, network)
}
if nw.Name == term {
if network.Name == term {
// No need to check the ID collision here as we are still in
// local scope and the network ID is unique in this scope.
listByFullName[nw.ID] = nw
listByFullName[network.ID] = network
}
if strings.HasPrefix(nw.ID, term) {
if strings.HasPrefix(network.ID, term) {
// No need to check the ID collision here as we are still in
// local scope and the network ID is unique in this scope.
listByPartialID[nw.ID] = nw
listByPartialID[network.ID] = network
}
}
@ -141,7 +146,7 @@ func (n *networkRouter) getNetwork(ctx context.Context, w http.ResponseWriter, r
// or if the get network was passed with a network name and scope as swarm
// return the network. Skipped using isMatchingScope because it is true if the scope
// is not set which would be case if the client API v1.30
if strings.HasPrefix(nwk.ID, term) || networkScope == scope.Swarm {
if strings.HasPrefix(nwk.ID, term) || (netconst.SwarmScope == scope) {
// If we have a previous match "backend", return it, we need verbose when enabled
// ex: overlay/partial_ID or name/swarm_scope
if nwv, ok := listByPartialID[nwk.ID]; ok {
@ -153,25 +158,25 @@ func (n *networkRouter) getNetwork(ctx context.Context, w http.ResponseWriter, r
}
}
networks, _ = n.cluster.GetNetworks(filter)
for _, nw := range networks {
if nw.ID == term {
return httputils.WriteJSON(w, http.StatusOK, nw)
nr, _ := n.cluster.GetNetworks(filter)
for _, network := range nr {
if network.ID == term {
return httputils.WriteJSON(w, http.StatusOK, network)
}
if nw.Name == term {
if network.Name == term {
// Check the ID collision as we are in swarm scope here, and
// the map (of the listByFullName) may have already had a
// network with the same ID (from local scope previously)
if _, ok := listByFullName[nw.ID]; !ok {
listByFullName[nw.ID] = nw
if _, ok := listByFullName[network.ID]; !ok {
listByFullName[network.ID] = network
}
}
if strings.HasPrefix(nw.ID, term) {
if strings.HasPrefix(network.ID, term) {
// Check the ID collision as we are in swarm scope here, and
// the map (of the listByPartialID) may have already had a
// network with the same ID (from local scope previously)
if _, ok := listByPartialID[nw.ID]; !ok {
listByPartialID[nw.ID] = nw
if _, ok := listByPartialID[network.ID]; !ok {
listByPartialID[network.ID] = network
}
}
}
@ -200,25 +205,39 @@ func (n *networkRouter) getNetwork(ctx context.Context, w http.ResponseWriter, r
}
func (n *networkRouter) postNetworkCreate(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
var create types.NetworkCreateRequest
if err := httputils.ParseForm(r); err != nil {
return err
}
var create types.NetworkCreateRequest
if err := httputils.ReadJSON(r, &create); err != nil {
if err := httputils.CheckForJSON(r); err != nil {
return err
}
if nws, err := n.cluster.GetNetworksByName(create.Name); err == nil && len(nws) > 0 {
return libnetwork.NetworkNameError(create.Name)
if err := json.NewDecoder(r.Body).Decode(&create); err != nil {
if err == io.EOF {
return errdefs.InvalidParameter(errors.New("got EOF while reading request body"))
}
return errdefs.InvalidParameter(err)
}
if nws, err := n.cluster.GetNetworksByName(create.Name); err == nil && len(nws) > 0 {
return nameConflict(create.Name)
}
// For a Swarm-scoped network, this call to backend.CreateNetwork is used to
// validate the configuration. The network will not be created but, if the
// configuration is valid, ManagerRedirectError will be returned and handled
// below.
nw, err := n.backend.CreateNetwork(create)
if err != nil {
var warning string
if _, ok := err.(libnetwork.NetworkNameError); ok {
// check if user defined CheckDuplicate, if set true, return err
// otherwise prepare a warning message
if create.CheckDuplicate {
return nameConflict(create.Name)
}
warning = libnetwork.NetworkNameError(create.Name).Error()
}
if _, ok := err.(libnetwork.ManagerRedirectError); !ok {
return err
}
@ -227,7 +246,8 @@ func (n *networkRouter) postNetworkCreate(ctx context.Context, w http.ResponseWr
return err
}
nw = &types.NetworkCreateResponse{
ID: id,
ID: id,
Warning: warning,
}
}
@ -235,15 +255,22 @@ func (n *networkRouter) postNetworkCreate(ctx context.Context, w http.ResponseWr
}
func (n *networkRouter) postNetworkConnect(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
var connect types.NetworkConnect
if err := httputils.ParseForm(r); err != nil {
return err
}
var connect types.NetworkConnect
if err := httputils.ReadJSON(r, &connect); err != nil {
if err := httputils.CheckForJSON(r); err != nil {
return err
}
if err := json.NewDecoder(r.Body).Decode(&connect); err != nil {
if err == io.EOF {
return errdefs.InvalidParameter(errors.New("got EOF while reading request body"))
}
return errdefs.InvalidParameter(err)
}
// Unlike other operations, we does not check ambiguity of the name/ID here.
// The reason is that, In case of attachable network in swarm scope, the actual local network
// may not be available at the time. At the same time, inside daemon `ConnectContainerToNetwork`
@ -252,15 +279,22 @@ func (n *networkRouter) postNetworkConnect(ctx context.Context, w http.ResponseW
}
func (n *networkRouter) postNetworkDisconnect(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
var disconnect types.NetworkDisconnect
if err := httputils.ParseForm(r); err != nil {
return err
}
var disconnect types.NetworkDisconnect
if err := httputils.ReadJSON(r, &disconnect); err != nil {
if err := httputils.CheckForJSON(r); err != nil {
return err
}
if err := json.NewDecoder(r.Body).Decode(&disconnect); err != nil {
if err == io.EOF {
return errdefs.InvalidParameter(errors.New("got EOF while reading request body"))
}
return errdefs.InvalidParameter(err)
}
return n.backend.DisconnectContainerFromNetwork(disconnect.Container, vars["id"], disconnect.Force)
}
@ -316,42 +350,42 @@ func (n *networkRouter) findUniqueNetwork(term string) (types.NetworkResource, e
listByPartialID := map[string]types.NetworkResource{}
filter := filters.NewArgs(filters.Arg("idOrName", term))
networks, _ := n.backend.GetNetworks(filter, backend.NetworkListConfig{Detailed: true})
for _, nw := range networks {
if nw.ID == term {
return nw, nil
nw, _ := n.backend.GetNetworks(filter, types.NetworkListConfig{Detailed: true})
for _, network := range nw {
if network.ID == term {
return network, nil
}
if nw.Name == term && !nw.Ingress {
if network.Name == term && !network.Ingress {
// No need to check the ID collision here as we are still in
// local scope and the network ID is unique in this scope.
listByFullName[nw.ID] = nw
listByFullName[network.ID] = network
}
if strings.HasPrefix(nw.ID, term) {
if strings.HasPrefix(network.ID, term) {
// No need to check the ID collision here as we are still in
// local scope and the network ID is unique in this scope.
listByPartialID[nw.ID] = nw
listByPartialID[network.ID] = network
}
}
networks, _ = n.cluster.GetNetworks(filter)
for _, nw := range networks {
if nw.ID == term {
return nw, nil
nr, _ := n.cluster.GetNetworks(filter)
for _, network := range nr {
if network.ID == term {
return network, nil
}
if nw.Name == term {
if network.Name == term {
// Check the ID collision as we are in swarm scope here, and
// the map (of the listByFullName) may have already had a
// network with the same ID (from local scope previously)
if _, ok := listByFullName[nw.ID]; !ok {
listByFullName[nw.ID] = nw
if _, ok := listByFullName[network.ID]; !ok {
listByFullName[network.ID] = network
}
}
if strings.HasPrefix(nw.ID, term) {
if strings.HasPrefix(network.ID, term) {
// Check the ID collision as we are in swarm scope here, and
// the map (of the listByPartialID) may have already had a
// network with the same ID (from local scope previously)
if _, ok := listByPartialID[nw.ID]; !ok {
listByPartialID[nw.ID] = nw
if _, ok := listByPartialID[network.ID]; !ok {
listByPartialID[network.ID] = network
}
}
}

View file

@ -5,25 +5,23 @@ import (
"io"
"net/http"
"github.com/distribution/reference"
"github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/backend"
"github.com/docker/distribution/reference"
enginetypes "github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/filters"
"github.com/docker/docker/api/types/registry"
"github.com/docker/docker/plugin"
)
// Backend for Plugin
type Backend interface {
Disable(name string, config *backend.PluginDisableConfig) error
Enable(name string, config *backend.PluginEnableConfig) error
List(filters.Args) ([]types.Plugin, error)
Inspect(name string) (*types.Plugin, error)
Remove(name string, config *backend.PluginRmConfig) error
Disable(name string, config *enginetypes.PluginDisableConfig) error
Enable(name string, config *enginetypes.PluginEnableConfig) error
List(filters.Args) ([]enginetypes.Plugin, error)
Inspect(name string) (*enginetypes.Plugin, error)
Remove(name string, config *enginetypes.PluginRmConfig) error
Set(name string, args []string) error
Privileges(ctx context.Context, ref reference.Named, metaHeaders http.Header, authConfig *registry.AuthConfig) (types.PluginPrivileges, error)
Pull(ctx context.Context, ref reference.Named, name string, metaHeaders http.Header, authConfig *registry.AuthConfig, privileges types.PluginPrivileges, outStream io.Writer, opts ...plugin.CreateOpt) error
Push(ctx context.Context, name string, metaHeaders http.Header, authConfig *registry.AuthConfig, outStream io.Writer) error
Upgrade(ctx context.Context, ref reference.Named, name string, metaHeaders http.Header, authConfig *registry.AuthConfig, privileges types.PluginPrivileges, outStream io.Writer) error
CreateFromContext(ctx context.Context, tarCtx io.ReadCloser, options *types.PluginCreateOptions) error
Privileges(ctx context.Context, ref reference.Named, metaHeaders http.Header, authConfig *enginetypes.AuthConfig) (enginetypes.PluginPrivileges, error)
Pull(ctx context.Context, ref reference.Named, name string, metaHeaders http.Header, authConfig *enginetypes.AuthConfig, privileges enginetypes.PluginPrivileges, outStream io.Writer, opts ...plugin.CreateOpt) error
Push(ctx context.Context, name string, metaHeaders http.Header, authConfig *enginetypes.AuthConfig, outStream io.Writer) error
Upgrade(ctx context.Context, ref reference.Named, name string, metaHeaders http.Header, authConfig *enginetypes.AuthConfig, privileges enginetypes.PluginPrivileges, outStream io.Writer) error
CreateFromContext(ctx context.Context, tarCtx io.ReadCloser, options *enginetypes.PluginCreateOptions) error
}

View file

@ -2,22 +2,25 @@ package plugin // import "github.com/docker/docker/api/server/router/plugin"
import (
"context"
"encoding/base64"
"encoding/json"
"io"
"net/http"
"strconv"
"strings"
"github.com/distribution/reference"
"github.com/docker/distribution/reference"
"github.com/docker/docker/api/server/httputils"
"github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/backend"
"github.com/docker/docker/api/types/filters"
"github.com/docker/docker/api/types/registry"
"github.com/docker/docker/errdefs"
"github.com/docker/docker/pkg/ioutils"
"github.com/docker/docker/pkg/streamformatter"
"github.com/pkg/errors"
)
func parseHeaders(headers http.Header) (map[string][]string, *registry.AuthConfig) {
func parseHeaders(headers http.Header) (map[string][]string, *types.AuthConfig) {
metaHeaders := map[string][]string{}
for k, v := range headers {
if strings.HasPrefix(k, "X-Meta-") {
@ -25,8 +28,16 @@ func parseHeaders(headers http.Header) (map[string][]string, *registry.AuthConfi
}
}
// Ignore invalid AuthConfig to increase compatibility with the existing API.
authConfig, _ := registry.DecodeAuthConfig(headers.Get(registry.AuthHeader))
// Get X-Registry-Auth
authEncoded := headers.Get("X-Registry-Auth")
authConfig := &types.AuthConfig{}
if authEncoded != "" {
authJSON := base64.NewDecoder(base64.URLEncoding, strings.NewReader(authEncoded))
if err := json.NewDecoder(authJSON).Decode(authConfig); err != nil {
authConfig = &types.AuthConfig{}
}
}
return metaHeaders, authConfig
}
@ -85,8 +96,12 @@ func (pr *pluginRouter) upgradePlugin(ctx context.Context, w http.ResponseWriter
}
var privileges types.PluginPrivileges
if err := httputils.ReadJSON(r, &privileges); err != nil {
return err
dec := json.NewDecoder(r.Body)
if err := dec.Decode(&privileges); err != nil {
return errors.Wrap(err, "failed to parse privileges")
}
if dec.More() {
return errors.New("invalid privileges")
}
metaHeaders, authConfig := parseHeaders(r.Header)
@ -120,8 +135,12 @@ func (pr *pluginRouter) pullPlugin(ctx context.Context, w http.ResponseWriter, r
}
var privileges types.PluginPrivileges
if err := httputils.ReadJSON(r, &privileges); err != nil {
return err
dec := json.NewDecoder(r.Body)
if err := dec.Decode(&privileges); err != nil {
return errors.Wrap(err, "failed to parse privileges")
}
if dec.More() {
return errors.New("invalid privileges")
}
metaHeaders, authConfig := parseHeaders(r.Header)
@ -187,8 +206,7 @@ func (pr *pluginRouter) createPlugin(ctx context.Context, w http.ResponseWriter,
}
options := &types.PluginCreateOptions{
RepoName: r.FormValue("name"),
}
RepoName: r.FormValue("name")}
if err := pr.backend.CreateFromContext(ctx, r.Body, options); err != nil {
return err
@ -208,7 +226,7 @@ func (pr *pluginRouter) enablePlugin(ctx context.Context, w http.ResponseWriter,
if err != nil {
return err
}
config := &backend.PluginEnableConfig{Timeout: timeout}
config := &types.PluginEnableConfig{Timeout: timeout}
return pr.backend.Enable(name, config)
}
@ -219,7 +237,7 @@ func (pr *pluginRouter) disablePlugin(ctx context.Context, w http.ResponseWriter
}
name := vars["name"]
config := &backend.PluginDisableConfig{
config := &types.PluginDisableConfig{
ForceDisable: httputils.BoolValue(r, "force"),
}
@ -232,7 +250,7 @@ func (pr *pluginRouter) removePlugin(ctx context.Context, w http.ResponseWriter,
}
name := vars["name"]
config := &backend.PluginRmConfig{
config := &types.PluginRmConfig{
ForceRemove: httputils.BoolValue(r, "force"),
}
return pr.backend.Remove(name, config)
@ -259,8 +277,11 @@ func (pr *pluginRouter) pushPlugin(ctx context.Context, w http.ResponseWriter, r
func (pr *pluginRouter) setPlugin(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
var args []string
if err := httputils.ReadJSON(r, &args); err != nil {
return err
if err := json.NewDecoder(r.Body).Decode(&args); err != nil {
if err == io.EOF {
return errdefs.InvalidParameter(errors.New("got EOF while reading request body"))
}
return errdefs.InvalidParameter(err)
}
if err := pr.backend.Set(vars["name"], args); err != nil {
return err

View file

@ -3,41 +3,46 @@ package swarm // import "github.com/docker/docker/api/server/router/swarm"
import (
"context"
"github.com/docker/docker/api/types"
basictypes "github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/backend"
"github.com/docker/docker/api/types/container"
"github.com/docker/docker/api/types/swarm"
types "github.com/docker/docker/api/types/swarm"
)
// Backend abstracts a swarm manager.
type Backend interface {
Init(req swarm.InitRequest) (string, error)
Join(req swarm.JoinRequest) error
Leave(ctx context.Context, force bool) error
Inspect() (swarm.Swarm, error)
Update(uint64, swarm.Spec, swarm.UpdateFlags) error
Init(req types.InitRequest) (string, error)
Join(req types.JoinRequest) error
Leave(force bool) error
Inspect() (types.Swarm, error)
Update(uint64, types.Spec, types.UpdateFlags) error
GetUnlockKey() (string, error)
UnlockSwarm(req swarm.UnlockRequest) error
GetServices(types.ServiceListOptions) ([]swarm.Service, error)
GetService(idOrName string, insertDefaults bool) (swarm.Service, error)
CreateService(swarm.ServiceSpec, string, bool) (*swarm.ServiceCreateResponse, error)
UpdateService(string, uint64, swarm.ServiceSpec, types.ServiceUpdateOptions, bool) (*swarm.ServiceUpdateResponse, error)
UnlockSwarm(req types.UnlockRequest) error
GetServices(basictypes.ServiceListOptions) ([]types.Service, error)
GetService(idOrName string, insertDefaults bool) (types.Service, error)
CreateService(types.ServiceSpec, string, bool) (*basictypes.ServiceCreateResponse, error)
UpdateService(string, uint64, types.ServiceSpec, basictypes.ServiceUpdateOptions, bool) (*basictypes.ServiceUpdateResponse, error)
RemoveService(string) error
ServiceLogs(context.Context, *backend.LogSelector, *container.LogsOptions) (<-chan *backend.LogMessage, error)
GetNodes(types.NodeListOptions) ([]swarm.Node, error)
GetNode(string) (swarm.Node, error)
UpdateNode(string, uint64, swarm.NodeSpec) error
ServiceLogs(context.Context, *backend.LogSelector, *basictypes.ContainerLogsOptions) (<-chan *backend.LogMessage, error)
GetNodes(basictypes.NodeListOptions) ([]types.Node, error)
GetNode(string) (types.Node, error)
UpdateNode(string, uint64, types.NodeSpec) error
RemoveNode(string, bool) error
GetTasks(types.TaskListOptions) ([]swarm.Task, error)
GetTask(string) (swarm.Task, error)
GetSecrets(opts types.SecretListOptions) ([]swarm.Secret, error)
CreateSecret(s swarm.SecretSpec) (string, error)
GetTasks(basictypes.TaskListOptions) ([]types.Task, error)
GetTask(string) (types.Task, error)
GetSecrets(opts basictypes.SecretListOptions) ([]types.Secret, error)
CreateSecret(s types.SecretSpec) (string, error)
RemoveSecret(idOrName string) error
GetSecret(id string) (swarm.Secret, error)
UpdateSecret(idOrName string, version uint64, spec swarm.SecretSpec) error
GetConfigs(opts types.ConfigListOptions) ([]swarm.Config, error)
CreateConfig(s swarm.ConfigSpec) (string, error)
GetSecret(id string) (types.Secret, error)
UpdateSecret(idOrName string, version uint64, spec types.SecretSpec) error
GetConfigs(opts basictypes.ConfigListOptions) ([]types.Config, error)
CreateConfig(s types.ConfigSpec) (string, error)
RemoveConfig(id string) error
GetConfig(id string) (swarm.Config, error)
UpdateConfig(idOrName string, version uint64, spec swarm.ConfigSpec) error
GetConfig(id string) (types.Config, error)
UpdateConfig(idOrName string, version uint64, spec types.ConfigSpec) error
}

View file

@ -2,26 +2,30 @@ package swarm // import "github.com/docker/docker/api/server/router/swarm"
import (
"context"
"encoding/json"
"fmt"
"io"
"net/http"
"strconv"
"github.com/containerd/log"
"github.com/docker/docker/api/server/httputils"
basictypes "github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/backend"
"github.com/docker/docker/api/types/filters"
"github.com/docker/docker/api/types/registry"
types "github.com/docker/docker/api/types/swarm"
"github.com/docker/docker/api/types/versions"
"github.com/docker/docker/errdefs"
"github.com/pkg/errors"
"github.com/sirupsen/logrus"
)
func (sr *swarmRouter) initCluster(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
var req types.InitRequest
if err := httputils.ReadJSON(r, &req); err != nil {
return err
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
if err == io.EOF {
return errdefs.InvalidParameter(errors.New("got EOF while reading request body"))
}
return errdefs.InvalidParameter(err)
}
version := httputils.VersionFromContext(ctx)
@ -36,7 +40,7 @@ func (sr *swarmRouter) initCluster(ctx context.Context, w http.ResponseWriter, r
}
nodeID, err := sr.backend.Init(req)
if err != nil {
log.G(ctx).WithContext(ctx).WithError(err).Debug("Error initializing swarm")
logrus.Errorf("Error initializing swarm: %v", err)
return err
}
return httputils.WriteJSON(w, http.StatusOK, nodeID)
@ -44,8 +48,11 @@ func (sr *swarmRouter) initCluster(ctx context.Context, w http.ResponseWriter, r
func (sr *swarmRouter) joinCluster(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
var req types.JoinRequest
if err := httputils.ReadJSON(r, &req); err != nil {
return err
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
if err == io.EOF {
return errdefs.InvalidParameter(errors.New("got EOF while reading request body"))
}
return errdefs.InvalidParameter(err)
}
return sr.backend.Join(req)
}
@ -56,13 +63,13 @@ func (sr *swarmRouter) leaveCluster(ctx context.Context, w http.ResponseWriter,
}
force := httputils.BoolValue(r, "force")
return sr.backend.Leave(ctx, force)
return sr.backend.Leave(force)
}
func (sr *swarmRouter) inspectCluster(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
swarm, err := sr.backend.Inspect()
if err != nil {
log.G(ctx).WithContext(ctx).WithError(err).Debug("Error getting swarm")
logrus.Errorf("Error getting swarm: %v", err)
return err
}
@ -71,8 +78,11 @@ func (sr *swarmRouter) inspectCluster(ctx context.Context, w http.ResponseWriter
func (sr *swarmRouter) updateCluster(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
var swarm types.Spec
if err := httputils.ReadJSON(r, &swarm); err != nil {
return err
if err := json.NewDecoder(r.Body).Decode(&swarm); err != nil {
if err == io.EOF {
return errdefs.InvalidParameter(errors.New("got EOF while reading request body"))
}
return errdefs.InvalidParameter(err)
}
rawVersion := r.URL.Query().Get("version")
@ -114,7 +124,7 @@ func (sr *swarmRouter) updateCluster(ctx context.Context, w http.ResponseWriter,
}
if err := sr.backend.Update(version, swarm, flags); err != nil {
log.G(ctx).WithContext(ctx).WithError(err).Debug("Error configuring swarm")
logrus.Errorf("Error configuring swarm: %v", err)
return err
}
return nil
@ -122,12 +132,15 @@ func (sr *swarmRouter) updateCluster(ctx context.Context, w http.ResponseWriter,
func (sr *swarmRouter) unlockCluster(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
var req types.UnlockRequest
if err := httputils.ReadJSON(r, &req); err != nil {
return err
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
if err == io.EOF {
return errdefs.InvalidParameter(errors.New("got EOF while reading request body"))
}
return errdefs.InvalidParameter(err)
}
if err := sr.backend.UnlockSwarm(req); err != nil {
log.G(ctx).WithContext(ctx).WithError(err).Debug("Error unlocking swarm")
logrus.Errorf("Error unlocking swarm: %v", err)
return err
}
return nil
@ -136,7 +149,7 @@ func (sr *swarmRouter) unlockCluster(ctx context.Context, w http.ResponseWriter,
func (sr *swarmRouter) getUnlockKey(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
unlockKey, err := sr.backend.GetUnlockKey()
if err != nil {
log.G(ctx).WithContext(ctx).WithError(err).Debug("Error retrieving swarm unlock key")
logrus.WithError(err).Errorf("Error retrieving swarm unlock key")
return err
}
@ -151,7 +164,7 @@ func (sr *swarmRouter) getServices(ctx context.Context, w http.ResponseWriter, r
}
filter, err := filters.FromJSON(r.Form.Get("filters"))
if err != nil {
return err
return errdefs.InvalidParameter(err)
}
// the status query parameter is only support in API versions >= 1.41. If
@ -168,7 +181,7 @@ func (sr *swarmRouter) getServices(ctx context.Context, w http.ResponseWriter, r
services, err := sr.backend.GetServices(basictypes.ServiceListOptions{Filters: filter, Status: status})
if err != nil {
log.G(ctx).WithContext(ctx).WithError(err).Debug("Error getting services")
logrus.Errorf("Error getting services: %v", err)
return err
}
@ -194,10 +207,7 @@ func (sr *swarmRouter) getService(ctx context.Context, w http.ResponseWriter, r
service, err := sr.backend.GetService(vars["id"], insertDefaults)
if err != nil {
log.G(ctx).WithContext(ctx).WithFields(log.Fields{
"error": err,
"service-id": vars["id"],
}).Debug("Error getting service")
logrus.Errorf("Error getting service %s: %v", vars["id"], err)
return err
}
@ -206,16 +216,15 @@ func (sr *swarmRouter) getService(ctx context.Context, w http.ResponseWriter, r
func (sr *swarmRouter) createService(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
var service types.ServiceSpec
if err := httputils.ReadJSON(r, &service); err != nil {
return err
}
// TODO(thaJeztah): remove logentries check and migration code in release v26.0.0.
if service.TaskTemplate.LogDriver != nil && service.TaskTemplate.LogDriver.Name == "logentries" {
return errdefs.InvalidParameter(errors.New("the logentries logging driver has been deprecated and removed"))
if err := json.NewDecoder(r.Body).Decode(&service); err != nil {
if err == io.EOF {
return errdefs.InvalidParameter(errors.New("got EOF while reading request body"))
}
return errdefs.InvalidParameter(err)
}
// Get returns "" if the header does not exist
encodedAuth := r.Header.Get(registry.AuthHeader)
encodedAuth := r.Header.Get("X-Registry-Auth")
queryRegistry := false
if v := httputils.VersionFromContext(ctx); v != "" {
if versions.LessThan(v, "1.30") {
@ -223,21 +232,9 @@ func (sr *swarmRouter) createService(ctx context.Context, w http.ResponseWriter,
}
adjustForAPIVersion(v, &service)
}
version := httputils.VersionFromContext(ctx)
if versions.LessThan(version, "1.44") {
if service.TaskTemplate.ContainerSpec != nil && service.TaskTemplate.ContainerSpec.Healthcheck != nil {
// StartInterval was added in API 1.44
service.TaskTemplate.ContainerSpec.Healthcheck.StartInterval = 0
}
}
resp, err := sr.backend.CreateService(service, encodedAuth, queryRegistry)
if err != nil {
log.G(ctx).WithFields(log.Fields{
"error": err,
"service-name": service.Name,
}).Debug("Error creating service")
logrus.Errorf("Error creating service %s: %v", service.Name, err)
return err
}
@ -246,12 +243,11 @@ func (sr *swarmRouter) createService(ctx context.Context, w http.ResponseWriter,
func (sr *swarmRouter) updateService(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
var service types.ServiceSpec
if err := httputils.ReadJSON(r, &service); err != nil {
return err
}
// TODO(thaJeztah): remove logentries check and migration code in release v26.0.0.
if service.TaskTemplate.LogDriver != nil && service.TaskTemplate.LogDriver.Name == "logentries" {
return errdefs.InvalidParameter(errors.New("the logentries logging driver has been deprecated and removed"))
if err := json.NewDecoder(r.Body).Decode(&service); err != nil {
if err == io.EOF {
return errdefs.InvalidParameter(errors.New("got EOF while reading request body"))
}
return errdefs.InvalidParameter(err)
}
rawVersion := r.URL.Query().Get("version")
@ -264,7 +260,7 @@ func (sr *swarmRouter) updateService(ctx context.Context, w http.ResponseWriter,
var flags basictypes.ServiceUpdateOptions
// Get returns "" if the header does not exist
flags.EncodedRegistryAuth = r.Header.Get(registry.AuthHeader)
flags.EncodedRegistryAuth = r.Header.Get("X-Registry-Auth")
flags.RegistryAuthFrom = r.URL.Query().Get("registryAuthFrom")
flags.Rollback = r.URL.Query().Get("rollback")
queryRegistry := false
@ -277,10 +273,7 @@ func (sr *swarmRouter) updateService(ctx context.Context, w http.ResponseWriter,
resp, err := sr.backend.UpdateService(vars["id"], version, service, flags, queryRegistry)
if err != nil {
log.G(ctx).WithContext(ctx).WithFields(log.Fields{
"error": err,
"service-id": vars["id"],
}).Debug("Error updating service")
logrus.Errorf("Error updating service %s: %v", vars["id"], err)
return err
}
return httputils.WriteJSON(w, http.StatusOK, resp)
@ -288,10 +281,7 @@ func (sr *swarmRouter) updateService(ctx context.Context, w http.ResponseWriter,
func (sr *swarmRouter) removeService(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
if err := sr.backend.RemoveService(vars["id"]); err != nil {
log.G(ctx).WithContext(ctx).WithFields(log.Fields{
"error": err,
"service-id": vars["id"],
}).Debug("Error removing service")
logrus.Errorf("Error removing service %s: %v", vars["id"], err)
return err
}
return nil
@ -332,7 +322,7 @@ func (sr *swarmRouter) getNodes(ctx context.Context, w http.ResponseWriter, r *h
nodes, err := sr.backend.GetNodes(basictypes.NodeListOptions{Filters: filter})
if err != nil {
log.G(ctx).WithContext(ctx).WithError(err).Debug("Error getting nodes")
logrus.Errorf("Error getting nodes: %v", err)
return err
}
@ -342,10 +332,7 @@ func (sr *swarmRouter) getNodes(ctx context.Context, w http.ResponseWriter, r *h
func (sr *swarmRouter) getNode(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
node, err := sr.backend.GetNode(vars["id"])
if err != nil {
log.G(ctx).WithContext(ctx).WithFields(log.Fields{
"error": err,
"node-id": vars["id"],
}).Debug("Error getting node")
logrus.Errorf("Error getting node %s: %v", vars["id"], err)
return err
}
@ -354,8 +341,11 @@ func (sr *swarmRouter) getNode(ctx context.Context, w http.ResponseWriter, r *ht
func (sr *swarmRouter) updateNode(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
var node types.NodeSpec
if err := httputils.ReadJSON(r, &node); err != nil {
return err
if err := json.NewDecoder(r.Body).Decode(&node); err != nil {
if err == io.EOF {
return errdefs.InvalidParameter(errors.New("got EOF while reading request body"))
}
return errdefs.InvalidParameter(err)
}
rawVersion := r.URL.Query().Get("version")
@ -366,10 +356,7 @@ func (sr *swarmRouter) updateNode(ctx context.Context, w http.ResponseWriter, r
}
if err := sr.backend.UpdateNode(vars["id"], version, node); err != nil {
log.G(ctx).WithContext(ctx).WithFields(log.Fields{
"error": err,
"node-id": vars["id"],
}).Debug("Error updating node")
logrus.Errorf("Error updating node %s: %v", vars["id"], err)
return err
}
return nil
@ -383,10 +370,7 @@ func (sr *swarmRouter) removeNode(ctx context.Context, w http.ResponseWriter, r
force := httputils.BoolValue(r, "force")
if err := sr.backend.RemoveNode(vars["id"], force); err != nil {
log.G(ctx).WithContext(ctx).WithFields(log.Fields{
"error": err,
"node-id": vars["id"],
}).Debug("Error removing node")
logrus.Errorf("Error removing node %s: %v", vars["id"], err)
return err
}
return nil
@ -403,7 +387,7 @@ func (sr *swarmRouter) getTasks(ctx context.Context, w http.ResponseWriter, r *h
tasks, err := sr.backend.GetTasks(basictypes.TaskListOptions{Filters: filter})
if err != nil {
log.G(ctx).WithContext(ctx).WithError(err).Debug("Error getting tasks")
logrus.Errorf("Error getting tasks: %v", err)
return err
}
@ -413,10 +397,7 @@ func (sr *swarmRouter) getTasks(ctx context.Context, w http.ResponseWriter, r *h
func (sr *swarmRouter) getTask(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
task, err := sr.backend.GetTask(vars["id"])
if err != nil {
log.G(ctx).WithContext(ctx).WithFields(log.Fields{
"error": err,
"task-id": vars["id"],
}).Debug("Error getting task")
logrus.Errorf("Error getting task %s: %v", vars["id"], err)
return err
}
@ -442,8 +423,11 @@ func (sr *swarmRouter) getSecrets(ctx context.Context, w http.ResponseWriter, r
func (sr *swarmRouter) createSecret(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
var secret types.SecretSpec
if err := httputils.ReadJSON(r, &secret); err != nil {
return err
if err := json.NewDecoder(r.Body).Decode(&secret); err != nil {
if err == io.EOF {
return errdefs.InvalidParameter(errors.New("got EOF while reading request body"))
}
return errdefs.InvalidParameter(err)
}
version := httputils.VersionFromContext(ctx)
if secret.Templating != nil && versions.LessThan(version, "1.37") {
@ -480,8 +464,11 @@ func (sr *swarmRouter) getSecret(ctx context.Context, w http.ResponseWriter, r *
func (sr *swarmRouter) updateSecret(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
var secret types.SecretSpec
if err := httputils.ReadJSON(r, &secret); err != nil {
return err
if err := json.NewDecoder(r.Body).Decode(&secret); err != nil {
if err == io.EOF {
return errdefs.InvalidParameter(errors.New("got EOF while reading request body"))
}
return errdefs.InvalidParameter(err)
}
rawVersion := r.URL.Query().Get("version")
@ -513,8 +500,11 @@ func (sr *swarmRouter) getConfigs(ctx context.Context, w http.ResponseWriter, r
func (sr *swarmRouter) createConfig(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
var config types.ConfigSpec
if err := httputils.ReadJSON(r, &config); err != nil {
return err
if err := json.NewDecoder(r.Body).Decode(&config); err != nil {
if err == io.EOF {
return errdefs.InvalidParameter(errors.New("got EOF while reading request body"))
}
return errdefs.InvalidParameter(err)
}
version := httputils.VersionFromContext(ctx)
@ -552,8 +542,11 @@ func (sr *swarmRouter) getConfig(ctx context.Context, w http.ResponseWriter, r *
func (sr *swarmRouter) updateConfig(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
var config types.ConfigSpec
if err := httputils.ReadJSON(r, &config); err != nil {
return err
if err := json.NewDecoder(r.Body).Decode(&config); err != nil {
if err == io.EOF {
return errdefs.InvalidParameter(errors.New("got EOF while reading request body"))
}
return errdefs.InvalidParameter(err)
}
rawVersion := r.URL.Query().Get("version")

View file

@ -3,19 +3,19 @@ package swarm // import "github.com/docker/docker/api/server/router/swarm"
import (
"context"
"fmt"
"io"
"net/http"
"github.com/docker/docker/api/server/httputils"
basictypes "github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/backend"
"github.com/docker/docker/api/types/container"
"github.com/docker/docker/api/types/swarm"
"github.com/docker/docker/api/types/versions"
)
// swarmLogs takes an http response, request, and selector, and writes the logs
// specified by the selector to the response
func (sr *swarmRouter) swarmLogs(ctx context.Context, w http.ResponseWriter, r *http.Request, selector *backend.LogSelector) error {
func (sr *swarmRouter) swarmLogs(ctx context.Context, w io.Writer, r *http.Request, selector *backend.LogSelector) error {
// Args are validated before the stream starts because when it starts we're
// sending HTTP 200 by writing an empty chunk of data to tell the client that
// daemon is going to stream. By sending this initial HTTP 200 we can't report
@ -26,9 +26,9 @@ func (sr *swarmRouter) swarmLogs(ctx context.Context, w http.ResponseWriter, r *
return fmt.Errorf("Bad parameters: you must choose at least one stream")
}
// there is probably a neater way to manufacture the LogsOptions
// there is probably a neater way to manufacture the ContainerLogsOptions
// struct, probably in the caller, to eliminate the dependency on net/http
logsConfig := &container.LogsOptions{
logsConfig := &basictypes.ContainerLogsOptions{
Follow: httputils.BoolValue(r, "follow"),
Timestamps: httputils.BoolValue(r, "timestamps"),
Since: r.Form.Get("since"),
@ -63,11 +63,6 @@ func (sr *swarmRouter) swarmLogs(ctx context.Context, w http.ResponseWriter, r *
return err
}
contentType := basictypes.MediaTypeRawStream
if !tty && versions.GreaterThanOrEqualTo(httputils.VersionFromContext(ctx), "1.42") {
contentType = basictypes.MediaTypeMultiplexedStream
}
w.Header().Set("Content-Type", contentType)
httputils.WriteLogStream(ctx, w, msgs, logsConfig, !tty)
return nil
}
@ -119,13 +114,4 @@ func adjustForAPIVersion(cliVersion string, service *swarm.ServiceSpec) {
service.Mode.ReplicatedJob = nil
service.Mode.GlobalJob = nil
}
if versions.LessThan(cliVersion, "1.44") {
// seccomp, apparmor, and no_new_privs were added in 1.44.
if service.TaskTemplate.ContainerSpec != nil && service.TaskTemplate.ContainerSpec.Privileges != nil {
service.TaskTemplate.ContainerSpec.Privileges.Seccomp = nil
service.TaskTemplate.ContainerSpec.Privileges.AppArmor = nil
service.TaskTemplate.ContainerSpec.Privileges.NoNewPrivileges = false
}
}
}

View file

@ -9,7 +9,9 @@ import (
)
func TestAdjustForAPIVersion(t *testing.T) {
expectedSysctls := map[string]string{"foo": "bar"}
var (
expectedSysctls = map[string]string{"foo": "bar"}
)
// testing the negative -- does this leave everything else alone? -- is
// prohibitively time-consuming to write, because it would need an object
// with literally every field filled in.
@ -113,4 +115,5 @@ func TestAdjustForAPIVersion(t *testing.T) {
if len(spec.TaskTemplate.ContainerSpec.Ulimits) != 0 {
t.Error("Ulimits were not stripped from spec")
}
}

View file

@ -7,41 +7,22 @@ import (
"github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/events"
"github.com/docker/docker/api/types/filters"
"github.com/docker/docker/api/types/registry"
"github.com/docker/docker/api/types/swarm"
"github.com/docker/docker/api/types/system"
)
// DiskUsageOptions holds parameters for system disk usage query.
type DiskUsageOptions struct {
// Containers controls whether container disk usage should be computed.
Containers bool
// Images controls whether image disk usage should be computed.
Images bool
// Volumes controls whether volume disk usage should be computed.
Volumes bool
}
// Backend is the methods that need to be implemented to provide
// system specific functionality.
type Backend interface {
SystemInfo(context.Context) (*system.Info, error)
SystemVersion(context.Context) (types.Version, error)
SystemDiskUsage(ctx context.Context, opts DiskUsageOptions) (*types.DiskUsage, error)
SystemInfo() *types.Info
SystemVersion() types.Version
SystemDiskUsage(ctx context.Context) (*types.DiskUsage, error)
SubscribeToEvents(since, until time.Time, ef filters.Args) ([]events.Message, chan interface{})
UnsubscribeFromEvents(chan interface{})
AuthenticateToRegistry(ctx context.Context, authConfig *registry.AuthConfig) (string, string, error)
AuthenticateToRegistry(ctx context.Context, authConfig *types.AuthConfig) (string, string, error)
}
// ClusterBackend is all the methods that need to be implemented
// to provide cluster system specific functionality.
type ClusterBackend interface {
Info(context.Context) swarm.Info
}
// StatusProvider provides methods to get the swarm status of the current node.
type StatusProvider interface {
Status() string
Info() swarm.Info
}

View file

@ -1,13 +1,8 @@
// FIXME(thaJeztah): remove once we are a module; the go:build directive prevents go from downgrading language version to go1.16:
//go:build go1.19
package system // import "github.com/docker/docker/api/server/router/system"
import (
"github.com/docker/docker/api/server/router"
"github.com/docker/docker/api/types/system"
buildkit "github.com/docker/docker/builder/builder-next"
"resenje.org/singleflight"
)
// systemRouter provides information about the Docker system overall.
@ -17,16 +12,11 @@ type systemRouter struct {
cluster ClusterBackend
routes []router.Route
builder *buildkit.Builder
features func() map[string]bool
// collectSystemInfo is a single-flight for the /info endpoint,
// unique per API version (as different API versions may return
// a different API response).
collectSystemInfo singleflight.Group[string, *system.Info]
features *map[string]bool
}
// NewRouter initializes a new system router
func NewRouter(b Backend, c ClusterBackend, builder *buildkit.Builder, features func() map[string]bool) router.Router {
func NewRouter(b Backend, c ClusterBackend, builder *buildkit.Builder, features *map[string]bool) router.Router {
r := &systemRouter{
backend: b,
cluster: c,

View file

@ -7,19 +7,17 @@ import (
"net/http"
"time"
"github.com/containerd/log"
"github.com/docker/docker/api/server/httputils"
"github.com/docker/docker/api/server/router/build"
"github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/events"
"github.com/docker/docker/api/types/filters"
"github.com/docker/docker/api/types/registry"
"github.com/docker/docker/api/types/swarm"
"github.com/docker/docker/api/types/system"
timetypes "github.com/docker/docker/api/types/time"
"github.com/docker/docker/api/types/versions"
"github.com/docker/docker/pkg/ioutils"
"github.com/pkg/errors"
pkgerrors "github.com/pkg/errors"
"github.com/sirupsen/logrus"
"golang.org/x/sync/errgroup"
)
@ -32,13 +30,10 @@ func (s *systemRouter) pingHandler(ctx context.Context, w http.ResponseWriter, r
w.Header().Add("Cache-Control", "no-cache, no-store, must-revalidate")
w.Header().Add("Pragma", "no-cache")
builderVersion := build.BuilderVersion(s.features())
builderVersion := build.BuilderVersion(*s.features)
if bv := builderVersion; bv != "" {
w.Header().Set("Builder-Version", string(bv))
}
w.Header().Set("Swarm", s.swarmStatus())
if r.Method == http.MethodHead {
w.Header().Set("Content-Type", "text/plain; charset=utf-8")
w.Header().Set("Content-Length", "0")
@ -48,167 +43,84 @@ func (s *systemRouter) pingHandler(ctx context.Context, w http.ResponseWriter, r
return err
}
func (s *systemRouter) swarmStatus() string {
func (s *systemRouter) getInfo(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
info := s.backend.SystemInfo()
if s.cluster != nil {
if p, ok := s.cluster.(StatusProvider); ok {
return p.Status()
info.Swarm = s.cluster.Info()
info.Warnings = append(info.Warnings, info.Swarm.Warnings...)
}
if versions.LessThan(httputils.VersionFromContext(ctx), "1.25") {
// TODO: handle this conversion in engine-api
type oldInfo struct {
*types.Info
ExecutionDriver string
}
old := &oldInfo{
Info: info,
ExecutionDriver: "<not supported>",
}
nameOnlySecurityOptions := []string{}
kvSecOpts, err := types.DecodeSecurityOptions(old.SecurityOptions)
if err != nil {
return err
}
for _, s := range kvSecOpts {
nameOnlySecurityOptions = append(nameOnlySecurityOptions, s.Name)
}
old.SecurityOptions = nameOnlySecurityOptions
return httputils.WriteJSON(w, http.StatusOK, old)
}
if versions.LessThan(httputils.VersionFromContext(ctx), "1.39") {
if info.KernelVersion == "" {
info.KernelVersion = "<unknown>"
}
if info.OperatingSystem == "" {
info.OperatingSystem = "<unknown>"
}
}
return string(swarm.LocalNodeStateInactive)
}
func (s *systemRouter) getInfo(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
version := httputils.VersionFromContext(ctx)
info, _, _ := s.collectSystemInfo.Do(ctx, version, func(ctx context.Context) (*system.Info, error) {
info, err := s.backend.SystemInfo(ctx)
if err != nil {
return nil, err
}
if s.cluster != nil {
info.Swarm = s.cluster.Info(ctx)
info.Warnings = append(info.Warnings, info.Swarm.Warnings...)
}
if versions.LessThan(version, "1.25") {
// TODO: handle this conversion in engine-api
kvSecOpts, err := system.DecodeSecurityOptions(info.SecurityOptions)
if err != nil {
info.Warnings = append(info.Warnings, err.Error())
}
var nameOnly []string
for _, so := range kvSecOpts {
nameOnly = append(nameOnly, so.Name)
}
info.SecurityOptions = nameOnly
info.ExecutionDriver = "<not supported>" //nolint:staticcheck // ignore SA1019 (ExecutionDriver is deprecated)
}
if versions.LessThan(version, "1.39") {
if info.KernelVersion == "" {
info.KernelVersion = "<unknown>"
}
if info.OperatingSystem == "" {
info.OperatingSystem = "<unknown>"
}
}
if versions.LessThan(version, "1.44") {
for k, rt := range info.Runtimes {
// Status field introduced in API v1.44.
info.Runtimes[k] = system.RuntimeWithStatus{Runtime: rt.Runtime}
}
}
if versions.GreaterThanOrEqualTo(version, "1.42") {
info.KernelMemory = false
}
return info, nil
})
return httputils.WriteJSON(w, http.StatusOK, info)
}
func (s *systemRouter) getVersion(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
info, err := s.backend.SystemVersion(ctx)
if err != nil {
return err
}
info := s.backend.SystemVersion()
return httputils.WriteJSON(w, http.StatusOK, info)
}
func (s *systemRouter) getDiskUsage(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
if err := httputils.ParseForm(r); err != nil {
return err
}
version := httputils.VersionFromContext(ctx)
var getContainers, getImages, getVolumes, getBuildCache bool
typeStrs, ok := r.Form["type"]
if versions.LessThan(version, "1.42") || !ok {
getContainers, getImages, getVolumes, getBuildCache = true, true, true, s.builder != nil
} else {
for _, typ := range typeStrs {
switch types.DiskUsageObject(typ) {
case types.ContainerObject:
getContainers = true
case types.ImageObject:
getImages = true
case types.VolumeObject:
getVolumes = true
case types.BuildCacheObject:
getBuildCache = true
default:
return invalidRequestError{Err: fmt.Errorf("unknown object type: %s", typ)}
}
}
}
eg, ctx := errgroup.WithContext(ctx)
var systemDiskUsage *types.DiskUsage
if getContainers || getImages || getVolumes {
eg.Go(func() error {
var err error
systemDiskUsage, err = s.backend.SystemDiskUsage(ctx, DiskUsageOptions{
Containers: getContainers,
Images: getImages,
Volumes: getVolumes,
})
return err
})
}
var du *types.DiskUsage
eg.Go(func() error {
var err error
du, err = s.backend.SystemDiskUsage(ctx)
return err
})
var buildCache []*types.BuildCache
if getBuildCache {
eg.Go(func() error {
var err error
buildCache, err = s.builder.DiskUsage(ctx)
if err != nil {
return errors.Wrap(err, "error getting build cache usage")
}
if buildCache == nil {
// Ensure empty `BuildCache` field is represented as empty JSON array(`[]`)
// instead of `null` to be consistent with `Images`, `Containers` etc.
buildCache = []*types.BuildCache{}
}
return nil
})
}
eg.Go(func() error {
var err error
buildCache, err = s.builder.DiskUsage(ctx)
if err != nil {
return pkgerrors.Wrap(err, "error getting build cache usage")
}
return nil
})
if err := eg.Wait(); err != nil {
return err
}
var builderSize int64
if versions.LessThan(version, "1.42") {
for _, b := range buildCache {
builderSize += b.Size
// Parents field was added in API 1.42 to replace the Parent field.
b.Parents = nil
}
}
if versions.GreaterThanOrEqualTo(version, "1.42") {
for _, b := range buildCache {
// Parent field is deprecated in API v1.42 and up, as it is deprecated
// in BuildKit. Empty the field to omit it in the API response.
b.Parent = "" //nolint:staticcheck // ignore SA1019 (Parent field is deprecated)
}
}
if versions.LessThan(version, "1.44") {
for _, b := range systemDiskUsage.Images {
b.VirtualSize = b.Size //nolint:staticcheck // ignore SA1019: field is deprecated, but still set on API < v1.44.
}
for _, b := range buildCache {
builderSize += b.Size
}
du := types.DiskUsage{
BuildCache: buildCache,
BuilderSize: builderSize,
}
if systemDiskUsage != nil {
du.LayersSize = systemDiskUsage.LayersSize
du.Images = systemDiskUsage.Images
du.Containers = systemDiskUsage.Containers
du.Volumes = systemDiskUsage.Volumes
}
du.BuilderSize = builderSize
du.BuildCache = buildCache
return httputils.WriteJSON(w, http.StatusOK, du)
}
@ -287,7 +199,7 @@ func (s *systemRouter) getEvents(ctx context.Context, w http.ResponseWriter, r *
case ev := <-l:
jev, ok := ev.(events.Message)
if !ok {
log.G(ctx).Warnf("unexpected event message: %q", ev)
logrus.Warnf("unexpected event message: %q", ev)
continue
}
if err := enc.Encode(jev); err != nil {
@ -296,14 +208,14 @@ func (s *systemRouter) getEvents(ctx context.Context, w http.ResponseWriter, r *
case <-timeout:
return nil
case <-ctx.Done():
log.G(ctx).Debug("Client context cancelled, stop sending events")
logrus.Debug("Client context cancelled, stop sending events")
return nil
}
}
}
func (s *systemRouter) postAuth(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
var config *registry.AuthConfig
var config *types.AuthConfig
err := json.NewDecoder(r.Body).Decode(&config)
r.Body.Close()
if err != nil {

View file

@ -7,28 +7,14 @@ import (
// TODO return types need to be refactored into pkg
"github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/filters"
"github.com/docker/docker/api/types/volume"
)
// Backend is the methods that need to be implemented to provide
// volume specific functionality
type Backend interface {
List(ctx context.Context, filter filters.Args) ([]*volume.Volume, []string, error)
Get(ctx context.Context, name string, opts ...opts.GetOption) (*volume.Volume, error)
Create(ctx context.Context, name, driverName string, opts ...opts.CreateOption) (*volume.Volume, error)
List(ctx context.Context, filter filters.Args) ([]*types.Volume, []string, error)
Get(ctx context.Context, name string, opts ...opts.GetOption) (*types.Volume, error)
Create(ctx context.Context, name, driverName string, opts ...opts.CreateOption) (*types.Volume, error)
Remove(ctx context.Context, name string, opts ...opts.RemoveOption) error
Prune(ctx context.Context, pruneFilters filters.Args) (*types.VolumesPruneReport, error)
}
// ClusterBackend is the backend used for Swarm Cluster Volumes. Regular
// volumes go through the volume service, but to avoid across-dependency
// between the cluster package and the volume package, we simply provide two
// backends here.
type ClusterBackend interface {
GetVolume(nameOrID string) (volume.Volume, error)
GetVolumes(options volume.ListOptions) ([]*volume.Volume, error)
CreateVolume(volume volume.CreateOptions) (*volume.Volume, error)
RemoveVolume(nameOrID string, force bool) error
UpdateVolume(nameOrID string, version uint64, volume volume.UpdateOptions) error
IsManager() bool
}

View file

@ -5,15 +5,13 @@ import "github.com/docker/docker/api/server/router"
// volumeRouter is a router to talk with the volumes controller
type volumeRouter struct {
backend Backend
cluster ClusterBackend
routes []router.Route
}
// NewRouter initializes a new volume router
func NewRouter(b Backend, cb ClusterBackend) router.Router {
func NewRouter(b Backend) router.Router {
r := &volumeRouter{
backend: b,
cluster: cb,
}
r.initRoutes()
return r
@ -32,8 +30,6 @@ func (r *volumeRouter) initRoutes() {
// POST
router.NewPostRoute("/volumes/create", r.postVolumesCreate),
router.NewPostRoute("/volumes/prune", r.postVolumesPrune),
// PUT
router.NewPutRoute("/volumes/{name:.*}", r.putVolumesUpdate),
// DELETE
router.NewDeleteRoute("/volumes/{name:.*}", r.deleteVolumes),
}

View file

@ -2,26 +2,18 @@ package volume // import "github.com/docker/docker/api/server/router/volume"
import (
"context"
"fmt"
"encoding/json"
"io"
"net/http"
"strconv"
"github.com/containerd/log"
"github.com/docker/docker/api/server/httputils"
"github.com/docker/docker/api/types/filters"
"github.com/docker/docker/api/types/versions"
"github.com/docker/docker/api/types/volume"
volumetypes "github.com/docker/docker/api/types/volume"
"github.com/docker/docker/errdefs"
"github.com/docker/docker/volume/service/opts"
"github.com/pkg/errors"
)
const (
// clusterVolumesVersion defines the API version that swarm cluster volume
// functionality was introduced. avoids the use of magic numbers.
clusterVolumesVersion = "1.42"
)
func (v *volumeRouter) getVolumesList(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
if err := httputils.ParseForm(r); err != nil {
return err
@ -29,62 +21,25 @@ func (v *volumeRouter) getVolumesList(ctx context.Context, w http.ResponseWriter
filters, err := filters.FromJSON(r.Form.Get("filters"))
if err != nil {
return errors.Wrap(err, "error reading volume filters")
return errdefs.InvalidParameter(errors.Wrap(err, "error reading volume filters"))
}
volumes, warnings, err := v.backend.List(ctx, filters)
if err != nil {
return err
}
version := httputils.VersionFromContext(ctx)
if versions.GreaterThanOrEqualTo(version, clusterVolumesVersion) && v.cluster.IsManager() {
clusterVolumes, swarmErr := v.cluster.GetVolumes(volume.ListOptions{Filters: filters})
if swarmErr != nil {
// if there is a swarm error, we may not want to error out right
// away. the local list probably worked. instead, let's do what we
// do if there's a bad driver while trying to list: add the error
// to the warnings. don't do this if swarm is not initialized.
warnings = append(warnings, swarmErr.Error())
}
// add the cluster volumes to the return
volumes = append(volumes, clusterVolumes...)
}
return httputils.WriteJSON(w, http.StatusOK, &volume.ListResponse{Volumes: volumes, Warnings: warnings})
return httputils.WriteJSON(w, http.StatusOK, &volumetypes.VolumeListOKBody{Volumes: volumes, Warnings: warnings})
}
func (v *volumeRouter) getVolumeByName(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
if err := httputils.ParseForm(r); err != nil {
return err
}
version := httputils.VersionFromContext(ctx)
// re: volume name duplication
//
// we prefer to get volumes locally before attempting to get them from the
// cluster. Local volumes can only be looked up by name, but cluster
// volumes can also be looked up by ID.
vol, err := v.backend.Get(ctx, vars["name"], opts.WithGetResolveStatus)
// if the volume is not found in the regular volume backend, and the client
// is using an API version greater than 1.42 (when cluster volumes were
// introduced), then check if Swarm has the volume.
if errdefs.IsNotFound(err) && versions.GreaterThanOrEqualTo(version, clusterVolumesVersion) && v.cluster.IsManager() {
swarmVol, err := v.cluster.GetVolume(vars["name"])
// if swarm returns an error and that error indicates that swarm is not
// initialized, return original NotFound error. Otherwise, we'd return
// a weird swarm unavailable error on non-swarm engines.
if err != nil {
return err
}
vol = &swarmVol
} else if err != nil {
// otherwise, if this isn't NotFound, or this isn't a high enough version,
// just return the error by itself.
volume, err := v.backend.Get(ctx, vars["name"], opts.WithGetResolveStatus)
if err != nil {
return err
}
return httputils.WriteJSON(w, http.StatusOK, vol)
return httputils.WriteJSON(w, http.StatusOK, volume)
}
func (v *volumeRouter) postVolumesCreate(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
@ -92,65 +47,23 @@ func (v *volumeRouter) postVolumesCreate(ctx context.Context, w http.ResponseWri
return err
}
var req volume.CreateOptions
if err := httputils.ReadJSON(r, &req); err != nil {
if err := httputils.CheckForJSON(r); err != nil {
return err
}
var (
vol *volume.Volume
err error
version = httputils.VersionFromContext(ctx)
)
// if the ClusterVolumeSpec is filled in, then this is a cluster volume
// and is created through the swarm cluster volume backend.
//
// re: volume name duplication
//
// As it happens, there is no good way to prevent duplication of a volume
// name between local and cluster volumes. This is because Swarm volumes
// can be created from any manager node, bypassing most of the protections
// we could put into the engine side.
//
// Instead, we will allow creating a volume with a duplicate name, which
// should not break anything.
if req.ClusterVolumeSpec != nil && versions.GreaterThanOrEqualTo(version, clusterVolumesVersion) {
log.G(ctx).Debug("using cluster volume")
vol, err = v.cluster.CreateVolume(req)
} else {
log.G(ctx).Debug("using regular volume")
vol, err = v.backend.Create(ctx, req.Name, req.Driver, opts.WithCreateOptions(req.DriverOpts), opts.WithCreateLabels(req.Labels))
}
if err != nil {
return err
}
return httputils.WriteJSON(w, http.StatusCreated, vol)
}
func (v *volumeRouter) putVolumesUpdate(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
if !v.cluster.IsManager() {
return errdefs.Unavailable(errors.New("volume update only valid for cluster volumes, but swarm is unavailable"))
}
if err := httputils.ParseForm(r); err != nil {
return err
}
rawVersion := r.URL.Query().Get("version")
version, err := strconv.ParseUint(rawVersion, 10, 64)
if err != nil {
err = fmt.Errorf("invalid swarm object version '%s': %v", rawVersion, err)
var req volumetypes.VolumeCreateBody
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
if err == io.EOF {
return errdefs.InvalidParameter(errors.New("got EOF while reading request body"))
}
return errdefs.InvalidParameter(err)
}
var req volume.UpdateOptions
if err := httputils.ReadJSON(r, &req); err != nil {
volume, err := v.backend.Create(ctx, req.Name, req.Driver, opts.WithCreateOptions(req.DriverOpts), opts.WithCreateLabels(req.Labels))
if err != nil {
return err
}
return v.cluster.UpdateVolume(vars["name"], version, req)
return httputils.WriteJSON(w, http.StatusCreated, volume)
}
func (v *volumeRouter) deleteVolumes(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
@ -158,28 +71,7 @@ func (v *volumeRouter) deleteVolumes(ctx context.Context, w http.ResponseWriter,
return err
}
force := httputils.BoolValue(r, "force")
// First we try deleting local volume. The volume may not be found as a
// local volume, but could be a cluster volume, so we ignore "not found"
// errors at this stage. Note that no "not found" error is produced if
// "force" is enabled.
err := v.backend.Remove(ctx, vars["name"], opts.WithPurgeOnError(force))
if err != nil && !errdefs.IsNotFound(err) {
return err
}
// If no volume was found, the volume may be a cluster volume. If force
// is enabled, the volume backend won't return an error for non-existing
// volumes, so we don't know if removal succeeded (or not volume existed).
// In that case we always try to delete cluster volumes as well.
if errdefs.IsNotFound(err) || force {
version := httputils.VersionFromContext(ctx)
if versions.GreaterThanOrEqualTo(version, clusterVolumesVersion) && v.cluster.IsManager() {
err = v.cluster.RemoveVolume(vars["name"], force)
}
}
if err != nil {
if err := v.backend.Remove(ctx, vars["name"], opts.WithPurgeOnError(force)); err != nil {
return err
}
w.WriteHeader(http.StatusNoContent)
@ -196,12 +88,6 @@ func (v *volumeRouter) postVolumesPrune(ctx context.Context, w http.ResponseWrit
return err
}
// API version 1.42 changes behavior where prune should only prune anonymous volumes.
// To keep older API behavior working, we need to add this filter option to consider all (local) volumes for pruning, not just anonymous ones.
if versions.LessThan(httputils.VersionFromContext(ctx), "1.42") {
pruneFilters.Add("all", "true")
}
pruneReport, err := v.backend.Prune(ctx, pruneFilters)
if err != nil {
return err

View file

@ -1,760 +0,0 @@
package volume
import (
"bytes"
"context"
"encoding/json"
"fmt"
"net/http/httptest"
"testing"
"gotest.tools/v3/assert"
"github.com/docker/docker/api/server/httputils"
"github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/filters"
"github.com/docker/docker/api/types/volume"
"github.com/docker/docker/errdefs"
"github.com/docker/docker/volume/service/opts"
)
func callGetVolume(v *volumeRouter, name string) (*httptest.ResponseRecorder, error) {
ctx := context.WithValue(context.Background(), httputils.APIVersionKey{}, clusterVolumesVersion)
vars := map[string]string{"name": name}
req := httptest.NewRequest("GET", fmt.Sprintf("/volumes/%s", name), nil)
resp := httptest.NewRecorder()
err := v.getVolumeByName(ctx, resp, req, vars)
return resp, err
}
func callListVolumes(v *volumeRouter) (*httptest.ResponseRecorder, error) {
ctx := context.WithValue(context.Background(), httputils.APIVersionKey{}, clusterVolumesVersion)
vars := map[string]string{}
req := httptest.NewRequest("GET", "/volumes", nil)
resp := httptest.NewRecorder()
err := v.getVolumesList(ctx, resp, req, vars)
return resp, err
}
func TestGetVolumeByNameNotFoundNoSwarm(t *testing.T) {
v := &volumeRouter{
backend: &fakeVolumeBackend{},
cluster: &fakeClusterBackend{},
}
_, err := callGetVolume(v, "notReal")
assert.Assert(t, err != nil)
assert.Assert(t, errdefs.IsNotFound(err))
}
func TestGetVolumeByNameNotFoundNotManager(t *testing.T) {
v := &volumeRouter{
backend: &fakeVolumeBackend{},
cluster: &fakeClusterBackend{swarm: true},
}
_, err := callGetVolume(v, "notReal")
assert.Assert(t, err != nil)
assert.Assert(t, errdefs.IsNotFound(err))
}
func TestGetVolumeByNameNotFound(t *testing.T) {
v := &volumeRouter{
backend: &fakeVolumeBackend{},
cluster: &fakeClusterBackend{swarm: true, manager: true},
}
_, err := callGetVolume(v, "notReal")
assert.Assert(t, err != nil)
assert.Assert(t, errdefs.IsNotFound(err))
}
func TestGetVolumeByNameFoundRegular(t *testing.T) {
v := &volumeRouter{
backend: &fakeVolumeBackend{
volumes: map[string]*volume.Volume{
"volume1": {
Name: "volume1",
},
},
},
cluster: &fakeClusterBackend{swarm: true, manager: true},
}
_, err := callGetVolume(v, "volume1")
assert.NilError(t, err)
}
func TestGetVolumeByNameFoundSwarm(t *testing.T) {
v := &volumeRouter{
backend: &fakeVolumeBackend{},
cluster: &fakeClusterBackend{
swarm: true,
manager: true,
volumes: map[string]*volume.Volume{
"volume1": {
Name: "volume1",
},
},
},
}
_, err := callGetVolume(v, "volume1")
assert.NilError(t, err)
}
func TestListVolumes(t *testing.T) {
v := &volumeRouter{
backend: &fakeVolumeBackend{
volumes: map[string]*volume.Volume{
"v1": {Name: "v1"},
"v2": {Name: "v2"},
},
},
cluster: &fakeClusterBackend{
swarm: true,
manager: true,
volumes: map[string]*volume.Volume{
"v3": {Name: "v3"},
"v4": {Name: "v4"},
},
},
}
resp, err := callListVolumes(v)
assert.NilError(t, err)
d := json.NewDecoder(resp.Result().Body)
respVols := volume.ListResponse{}
assert.NilError(t, d.Decode(&respVols))
assert.Assert(t, respVols.Volumes != nil)
assert.Equal(t, len(respVols.Volumes), 4, "volumes %v", respVols.Volumes)
}
func TestListVolumesNoSwarm(t *testing.T) {
v := &volumeRouter{
backend: &fakeVolumeBackend{
volumes: map[string]*volume.Volume{
"v1": {Name: "v1"},
"v2": {Name: "v2"},
},
},
cluster: &fakeClusterBackend{},
}
_, err := callListVolumes(v)
assert.NilError(t, err)
}
func TestListVolumesNoManager(t *testing.T) {
v := &volumeRouter{
backend: &fakeVolumeBackend{
volumes: map[string]*volume.Volume{
"v1": {Name: "v1"},
"v2": {Name: "v2"},
},
},
cluster: &fakeClusterBackend{swarm: true},
}
resp, err := callListVolumes(v)
assert.NilError(t, err)
d := json.NewDecoder(resp.Result().Body)
respVols := volume.ListResponse{}
assert.NilError(t, d.Decode(&respVols))
assert.Equal(t, len(respVols.Volumes), 2)
assert.Equal(t, len(respVols.Warnings), 0)
}
func TestCreateRegularVolume(t *testing.T) {
b := &fakeVolumeBackend{}
c := &fakeClusterBackend{
swarm: true,
manager: true,
}
v := &volumeRouter{
backend: b,
cluster: c,
}
volumeCreate := volume.CreateOptions{
Name: "vol1",
Driver: "foodriver",
}
buf := bytes.Buffer{}
e := json.NewEncoder(&buf)
e.Encode(volumeCreate)
ctx := context.WithValue(context.Background(), httputils.APIVersionKey{}, clusterVolumesVersion)
req := httptest.NewRequest("POST", "/volumes/create", &buf)
req.Header.Add("Content-Type", "application/json")
resp := httptest.NewRecorder()
err := v.postVolumesCreate(ctx, resp, req, nil)
assert.NilError(t, err)
respVolume := volume.Volume{}
assert.NilError(t, json.NewDecoder(resp.Result().Body).Decode(&respVolume))
assert.Equal(t, respVolume.Name, "vol1")
assert.Equal(t, respVolume.Driver, "foodriver")
assert.Equal(t, 1, len(b.volumes))
assert.Equal(t, 0, len(c.volumes))
}
func TestCreateSwarmVolumeNoSwarm(t *testing.T) {
b := &fakeVolumeBackend{}
c := &fakeClusterBackend{}
v := &volumeRouter{
backend: b,
cluster: c,
}
volumeCreate := volume.CreateOptions{
ClusterVolumeSpec: &volume.ClusterVolumeSpec{},
Name: "volCluster",
Driver: "someCSI",
}
buf := bytes.Buffer{}
json.NewEncoder(&buf).Encode(volumeCreate)
ctx := context.WithValue(context.Background(), httputils.APIVersionKey{}, clusterVolumesVersion)
req := httptest.NewRequest("POST", "/volumes/create", &buf)
req.Header.Add("Content-Type", "application/json")
resp := httptest.NewRecorder()
err := v.postVolumesCreate(ctx, resp, req, nil)
assert.Assert(t, err != nil)
assert.Assert(t, errdefs.IsUnavailable(err))
}
func TestCreateSwarmVolumeNotManager(t *testing.T) {
b := &fakeVolumeBackend{}
c := &fakeClusterBackend{swarm: true}
v := &volumeRouter{
backend: b,
cluster: c,
}
volumeCreate := volume.CreateOptions{
ClusterVolumeSpec: &volume.ClusterVolumeSpec{},
Name: "volCluster",
Driver: "someCSI",
}
buf := bytes.Buffer{}
json.NewEncoder(&buf).Encode(volumeCreate)
ctx := context.WithValue(context.Background(), httputils.APIVersionKey{}, clusterVolumesVersion)
req := httptest.NewRequest("POST", "/volumes/create", &buf)
req.Header.Add("Content-Type", "application/json")
resp := httptest.NewRecorder()
err := v.postVolumesCreate(ctx, resp, req, nil)
assert.Assert(t, err != nil)
assert.Assert(t, errdefs.IsUnavailable(err))
}
func TestCreateVolumeCluster(t *testing.T) {
b := &fakeVolumeBackend{}
c := &fakeClusterBackend{
swarm: true,
manager: true,
}
v := &volumeRouter{
backend: b,
cluster: c,
}
volumeCreate := volume.CreateOptions{
ClusterVolumeSpec: &volume.ClusterVolumeSpec{},
Name: "volCluster",
Driver: "someCSI",
}
buf := bytes.Buffer{}
json.NewEncoder(&buf).Encode(volumeCreate)
ctx := context.WithValue(context.Background(), httputils.APIVersionKey{}, clusterVolumesVersion)
req := httptest.NewRequest("POST", "/volumes/create", &buf)
req.Header.Add("Content-Type", "application/json")
resp := httptest.NewRecorder()
err := v.postVolumesCreate(ctx, resp, req, nil)
assert.NilError(t, err)
respVolume := volume.Volume{}
assert.NilError(t, json.NewDecoder(resp.Result().Body).Decode(&respVolume))
assert.Equal(t, respVolume.Name, "volCluster")
assert.Equal(t, respVolume.Driver, "someCSI")
assert.Equal(t, 0, len(b.volumes))
assert.Equal(t, 1, len(c.volumes))
}
func TestUpdateVolume(t *testing.T) {
b := &fakeVolumeBackend{}
c := &fakeClusterBackend{
swarm: true,
manager: true,
volumes: map[string]*volume.Volume{
"vol1": {
Name: "vo1",
ClusterVolume: &volume.ClusterVolume{
ID: "vol1",
},
},
},
}
v := &volumeRouter{
backend: b,
cluster: c,
}
volumeUpdate := volume.UpdateOptions{
Spec: &volume.ClusterVolumeSpec{},
}
buf := bytes.Buffer{}
json.NewEncoder(&buf).Encode(volumeUpdate)
ctx := context.WithValue(context.Background(), httputils.APIVersionKey{}, clusterVolumesVersion)
req := httptest.NewRequest("POST", "/volumes/vol1/update?version=0", &buf)
req.Header.Add("Content-Type", "application/json")
resp := httptest.NewRecorder()
err := v.putVolumesUpdate(ctx, resp, req, map[string]string{"name": "vol1"})
assert.NilError(t, err)
assert.Equal(t, c.volumes["vol1"].ClusterVolume.Meta.Version.Index, uint64(1))
}
func TestUpdateVolumeNoSwarm(t *testing.T) {
b := &fakeVolumeBackend{}
c := &fakeClusterBackend{}
v := &volumeRouter{
backend: b,
cluster: c,
}
volumeUpdate := volume.UpdateOptions{
Spec: &volume.ClusterVolumeSpec{},
}
buf := bytes.Buffer{}
json.NewEncoder(&buf).Encode(volumeUpdate)
ctx := context.WithValue(context.Background(), httputils.APIVersionKey{}, clusterVolumesVersion)
req := httptest.NewRequest("POST", "/volumes/vol1/update?version=0", &buf)
req.Header.Add("Content-Type", "application/json")
resp := httptest.NewRecorder()
err := v.putVolumesUpdate(ctx, resp, req, map[string]string{"name": "vol1"})
assert.Assert(t, err != nil)
assert.Assert(t, errdefs.IsUnavailable(err))
}
func TestUpdateVolumeNotFound(t *testing.T) {
b := &fakeVolumeBackend{}
c := &fakeClusterBackend{
swarm: true,
manager: true,
volumes: map[string]*volume.Volume{},
}
v := &volumeRouter{
backend: b,
cluster: c,
}
volumeUpdate := volume.UpdateOptions{
Spec: &volume.ClusterVolumeSpec{},
}
buf := bytes.Buffer{}
json.NewEncoder(&buf).Encode(volumeUpdate)
ctx := context.WithValue(context.Background(), httputils.APIVersionKey{}, clusterVolumesVersion)
req := httptest.NewRequest("POST", "/volumes/vol1/update?version=0", &buf)
req.Header.Add("Content-Type", "application/json")
resp := httptest.NewRecorder()
err := v.putVolumesUpdate(ctx, resp, req, map[string]string{"name": "vol1"})
assert.Assert(t, err != nil)
assert.Assert(t, errdefs.IsNotFound(err))
}
func TestVolumeRemove(t *testing.T) {
b := &fakeVolumeBackend{
volumes: map[string]*volume.Volume{
"vol1": {
Name: "vol1",
},
},
}
c := &fakeClusterBackend{swarm: true, manager: true}
v := &volumeRouter{
backend: b,
cluster: c,
}
ctx := context.WithValue(context.Background(), httputils.APIVersionKey{}, clusterVolumesVersion)
req := httptest.NewRequest("DELETE", "/volumes/vol1", nil)
resp := httptest.NewRecorder()
err := v.deleteVolumes(ctx, resp, req, map[string]string{"name": "vol1"})
assert.NilError(t, err)
assert.Equal(t, len(b.volumes), 0)
}
func TestVolumeRemoveSwarm(t *testing.T) {
b := &fakeVolumeBackend{}
c := &fakeClusterBackend{
swarm: true,
manager: true,
volumes: map[string]*volume.Volume{
"vol1": {
Name: "vol1",
ClusterVolume: &volume.ClusterVolume{},
},
},
}
v := &volumeRouter{
backend: b,
cluster: c,
}
ctx := context.WithValue(context.Background(), httputils.APIVersionKey{}, clusterVolumesVersion)
req := httptest.NewRequest("DELETE", "/volumes/vol1", nil)
resp := httptest.NewRecorder()
err := v.deleteVolumes(ctx, resp, req, map[string]string{"name": "vol1"})
assert.NilError(t, err)
assert.Equal(t, len(c.volumes), 0)
}
func TestVolumeRemoveNotFoundNoSwarm(t *testing.T) {
b := &fakeVolumeBackend{}
c := &fakeClusterBackend{}
v := &volumeRouter{
backend: b,
cluster: c,
}
ctx := context.WithValue(context.Background(), httputils.APIVersionKey{}, clusterVolumesVersion)
req := httptest.NewRequest("DELETE", "/volumes/vol1", nil)
resp := httptest.NewRecorder()
err := v.deleteVolumes(ctx, resp, req, map[string]string{"name": "vol1"})
assert.Assert(t, err != nil)
assert.Assert(t, errdefs.IsNotFound(err), err.Error())
}
func TestVolumeRemoveNotFoundNoManager(t *testing.T) {
b := &fakeVolumeBackend{}
c := &fakeClusterBackend{swarm: true}
v := &volumeRouter{
backend: b,
cluster: c,
}
ctx := context.WithValue(context.Background(), httputils.APIVersionKey{}, clusterVolumesVersion)
req := httptest.NewRequest("DELETE", "/volumes/vol1", nil)
resp := httptest.NewRecorder()
err := v.deleteVolumes(ctx, resp, req, map[string]string{"name": "vol1"})
assert.Assert(t, err != nil)
assert.Assert(t, errdefs.IsNotFound(err))
}
func TestVolumeRemoveFoundNoSwarm(t *testing.T) {
b := &fakeVolumeBackend{
volumes: map[string]*volume.Volume{
"vol1": {
Name: "vol1",
},
},
}
c := &fakeClusterBackend{}
v := &volumeRouter{
backend: b,
cluster: c,
}
ctx := context.WithValue(context.Background(), httputils.APIVersionKey{}, clusterVolumesVersion)
req := httptest.NewRequest("DELETE", "/volumes/vol1", nil)
resp := httptest.NewRecorder()
err := v.deleteVolumes(ctx, resp, req, map[string]string{"name": "vol1"})
assert.NilError(t, err)
assert.Equal(t, len(b.volumes), 0)
}
func TestVolumeRemoveNoSwarmInUse(t *testing.T) {
b := &fakeVolumeBackend{
volumes: map[string]*volume.Volume{
"inuse": {
Name: "inuse",
},
},
}
c := &fakeClusterBackend{}
v := &volumeRouter{
backend: b,
cluster: c,
}
ctx := context.WithValue(context.Background(), httputils.APIVersionKey{}, clusterVolumesVersion)
req := httptest.NewRequest("DELETE", "/volumes/inuse", nil)
resp := httptest.NewRecorder()
err := v.deleteVolumes(ctx, resp, req, map[string]string{"name": "inuse"})
assert.Assert(t, err != nil)
assert.Assert(t, errdefs.IsConflict(err))
}
func TestVolumeRemoveSwarmForce(t *testing.T) {
b := &fakeVolumeBackend{}
c := &fakeClusterBackend{
swarm: true,
manager: true,
volumes: map[string]*volume.Volume{
"vol1": {
Name: "vol1",
ClusterVolume: &volume.ClusterVolume{},
Options: map[string]string{"mustforce": "yes"},
},
},
}
v := &volumeRouter{
backend: b,
cluster: c,
}
ctx := context.WithValue(context.Background(), httputils.APIVersionKey{}, clusterVolumesVersion)
req := httptest.NewRequest("DELETE", "/volumes/vol1", nil)
resp := httptest.NewRecorder()
err := v.deleteVolumes(ctx, resp, req, map[string]string{"name": "vol1"})
assert.Assert(t, err != nil)
assert.Assert(t, errdefs.IsConflict(err))
ctx = context.WithValue(context.Background(), httputils.APIVersionKey{}, clusterVolumesVersion)
req = httptest.NewRequest("DELETE", "/volumes/vol1?force=1", nil)
resp = httptest.NewRecorder()
err = v.deleteVolumes(ctx, resp, req, map[string]string{"name": "vol1"})
assert.NilError(t, err)
assert.Equal(t, len(b.volumes), 0)
assert.Equal(t, len(c.volumes), 0)
}
type fakeVolumeBackend struct {
volumes map[string]*volume.Volume
}
func (b *fakeVolumeBackend) List(_ context.Context, _ filters.Args) ([]*volume.Volume, []string, error) {
volumes := []*volume.Volume{}
for _, v := range b.volumes {
volumes = append(volumes, v)
}
return volumes, nil, nil
}
func (b *fakeVolumeBackend) Get(_ context.Context, name string, _ ...opts.GetOption) (*volume.Volume, error) {
if v, ok := b.volumes[name]; ok {
return v, nil
}
return nil, errdefs.NotFound(fmt.Errorf("volume %s not found", name))
}
func (b *fakeVolumeBackend) Create(_ context.Context, name, driverName string, _ ...opts.CreateOption) (*volume.Volume, error) {
if _, ok := b.volumes[name]; ok {
// TODO(dperny): return appropriate error type
return nil, fmt.Errorf("already exists")
}
v := &volume.Volume{
Name: name,
Driver: driverName,
}
if b.volumes == nil {
b.volumes = map[string]*volume.Volume{
name: v,
}
} else {
b.volumes[name] = v
}
return v, nil
}
func (b *fakeVolumeBackend) Remove(_ context.Context, name string, o ...opts.RemoveOption) error {
removeOpts := &opts.RemoveConfig{}
for _, opt := range o {
opt(removeOpts)
}
if v, ok := b.volumes[name]; !ok {
if !removeOpts.PurgeOnError {
return errdefs.NotFound(fmt.Errorf("volume %s not found", name))
}
} else if v.Name == "inuse" {
return errdefs.Conflict(fmt.Errorf("volume in use"))
}
delete(b.volumes, name)
return nil
}
func (b *fakeVolumeBackend) Prune(_ context.Context, _ filters.Args) (*types.VolumesPruneReport, error) {
return nil, nil
}
type fakeClusterBackend struct {
swarm bool
manager bool
idCount int
volumes map[string]*volume.Volume
}
func (c *fakeClusterBackend) checkSwarm() error {
if !c.swarm {
return errdefs.Unavailable(fmt.Errorf("this node is not a swarm manager. Use \"docker swarm init\" or \"docker swarm join\" to connect this node to swarm and try again"))
} else if !c.manager {
return errdefs.Unavailable(fmt.Errorf("this node is not a swarm manager. Worker nodes can't be used to view or modify cluster state. Please run this command on a manager node or promote the current node to a manager"))
}
return nil
}
func (c *fakeClusterBackend) IsManager() bool {
return c.swarm && c.manager
}
func (c *fakeClusterBackend) GetVolume(nameOrID string) (volume.Volume, error) {
if err := c.checkSwarm(); err != nil {
return volume.Volume{}, err
}
if v, ok := c.volumes[nameOrID]; ok {
return *v, nil
}
return volume.Volume{}, errdefs.NotFound(fmt.Errorf("volume %s not found", nameOrID))
}
func (c *fakeClusterBackend) GetVolumes(options volume.ListOptions) ([]*volume.Volume, error) {
if err := c.checkSwarm(); err != nil {
return nil, err
}
volumes := []*volume.Volume{}
for _, v := range c.volumes {
volumes = append(volumes, v)
}
return volumes, nil
}
func (c *fakeClusterBackend) CreateVolume(volumeCreate volume.CreateOptions) (*volume.Volume, error) {
if err := c.checkSwarm(); err != nil {
return nil, err
}
if _, ok := c.volumes[volumeCreate.Name]; ok {
// TODO(dperny): return appropriate already exists error
return nil, fmt.Errorf("already exists")
}
v := &volume.Volume{
Name: volumeCreate.Name,
Driver: volumeCreate.Driver,
Labels: volumeCreate.Labels,
Options: volumeCreate.DriverOpts,
Scope: "global",
}
v.ClusterVolume = &volume.ClusterVolume{
ID: fmt.Sprintf("cluster_%d", c.idCount),
Spec: *volumeCreate.ClusterVolumeSpec,
}
c.idCount = c.idCount + 1
if c.volumes == nil {
c.volumes = map[string]*volume.Volume{
v.Name: v,
}
} else {
c.volumes[v.Name] = v
}
return v, nil
}
func (c *fakeClusterBackend) RemoveVolume(nameOrID string, force bool) error {
if err := c.checkSwarm(); err != nil {
return err
}
v, ok := c.volumes[nameOrID]
if !ok {
return errdefs.NotFound(fmt.Errorf("volume %s not found", nameOrID))
}
if _, mustforce := v.Options["mustforce"]; mustforce && !force {
return errdefs.Conflict(fmt.Errorf("volume %s must be force removed", nameOrID))
}
delete(c.volumes, nameOrID)
return nil
}
func (c *fakeClusterBackend) UpdateVolume(nameOrID string, version uint64, _ volume.UpdateOptions) error {
if err := c.checkSwarm(); err != nil {
return err
}
if v, ok := c.volumes[nameOrID]; ok {
if v.ClusterVolume.Meta.Version.Index != version {
return fmt.Errorf("wrong version")
}
v.ClusterVolume.Meta.Version.Index = v.ClusterVolume.Meta.Version.Index + 1
// for testing, we don't actually need to change anything about the
// volume object. let's just increment the version so we can see the
// call happened.
} else {
return errdefs.NotFound(fmt.Errorf("volume %q not found", nameOrID))
}
return nil
}

View file

@ -2,37 +2,125 @@ package server // import "github.com/docker/docker/api/server"
import (
"context"
"crypto/tls"
"net"
"net/http"
"strings"
"time"
"github.com/containerd/log"
"github.com/docker/docker/api/server/httpstatus"
"github.com/docker/docker/api/server/httputils"
"github.com/docker/docker/api/server/middleware"
"github.com/docker/docker/api/server/router"
"github.com/docker/docker/api/server/router/debug"
"github.com/docker/docker/api/types"
"github.com/docker/docker/dockerversion"
"github.com/gorilla/mux"
"go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp"
"github.com/sirupsen/logrus"
)
// versionMatcher defines a variable matcher to be parsed by the router
// when a request is about to be served.
const versionMatcher = "/v{version:[0-9.]+}"
// Config provides the configuration for the API server
type Config struct {
Logging bool
CorsHeaders string
Version string
SocketGroup string
TLSConfig *tls.Config
}
// Server contains instance details for the server
type Server struct {
cfg *Config
servers []*HTTPServer
routers []router.Router
middlewares []middleware.Middleware
}
// New returns a new instance of the server based on the specified configuration.
// It allocates resources which will be needed for ServeAPI(ports, unix-sockets).
func New(cfg *Config) *Server {
return &Server{
cfg: cfg,
}
}
// UseMiddleware appends a new middleware to the request chain.
// This needs to be called before the API routes are configured.
func (s *Server) UseMiddleware(m middleware.Middleware) {
s.middlewares = append(s.middlewares, m)
}
func (s *Server) makeHTTPHandler(handler httputils.APIFunc, operation string) http.HandlerFunc {
return otelhttp.NewHandler(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
// Accept sets a listener the server accepts connections into.
func (s *Server) Accept(addr string, listeners ...net.Listener) {
for _, listener := range listeners {
httpServer := &HTTPServer{
srv: &http.Server{
Addr: addr,
ReadHeaderTimeout: 5 * time.Minute, // "G112: Potential Slowloris Attack (gosec)"; not a real concern for our use, so setting a long timeout.
},
l: listener,
}
s.servers = append(s.servers, httpServer)
}
}
// Close closes servers and thus stop receiving requests
func (s *Server) Close() {
for _, srv := range s.servers {
if err := srv.Close(); err != nil {
logrus.Error(err)
}
}
}
// serveAPI loops through all initialized servers and spawns goroutine
// with Serve method for each. It sets createMux() as Handler also.
func (s *Server) serveAPI() error {
var chErrors = make(chan error, len(s.servers))
for _, srv := range s.servers {
srv.srv.Handler = s.createMux()
go func(srv *HTTPServer) {
var err error
logrus.Infof("API listen on %s", srv.l.Addr())
if err = srv.Serve(); err != nil && strings.Contains(err.Error(), "use of closed network connection") {
err = nil
}
chErrors <- err
}(srv)
}
for range s.servers {
err := <-chErrors
if err != nil {
return err
}
}
return nil
}
// HTTPServer contains an instance of http server and the listener.
// srv *http.Server, contains configuration to create an http server and a mux router with all api end points.
// l net.Listener, is a TCP or Socket listener that dispatches incoming request to the router.
type HTTPServer struct {
srv *http.Server
l net.Listener
}
// Serve starts listening for inbound requests.
func (s *HTTPServer) Serve() error {
return s.srv.Serve(s.l)
}
// Close closes the HTTPServer from listening for the inbound requests.
func (s *HTTPServer) Close() error {
return s.l.Close()
}
func (s *Server) makeHTTPHandler(handler httputils.APIFunc) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
// Define the context that we'll pass around to share info
// like the docker-request-id.
//
@ -44,7 +132,6 @@ func (s *Server) makeHTTPHandler(handler httputils.APIFunc, operation string) ht
// use intermediate variable to prevent "should not use basic type
// string as key in context.WithValue" golint errors
ctx := context.WithValue(r.Context(), dockerversion.UAStringKey{}, r.Header.Get("User-Agent"))
r = r.WithContext(ctx)
handlerFunc := s.handlerWithGlobalMiddlewares(handler)
@ -56,45 +143,65 @@ func (s *Server) makeHTTPHandler(handler httputils.APIFunc, operation string) ht
if err := handlerFunc(ctx, w, r, vars); err != nil {
statusCode := httpstatus.FromError(err)
if statusCode >= 500 {
log.G(ctx).Errorf("Handler for %s %s returned error: %v", r.Method, r.URL.Path, err)
logrus.Errorf("Handler for %s %s returned error: %v", r.Method, r.URL.Path, err)
}
_ = httputils.WriteJSON(w, statusCode, &types.ErrorResponse{
Message: err.Error(),
})
makeErrorHandler(err)(w, r)
}
}), operation).ServeHTTP
}
}
// CreateMux returns a new mux with all the routers registered.
func (s *Server) CreateMux(routers ...router.Router) *mux.Router {
// InitRouter initializes the list of routers for the server.
// This method also enables the Go profiler.
func (s *Server) InitRouter(routers ...router.Router) {
s.routers = append(s.routers, routers...)
}
type pageNotFoundError struct{}
func (pageNotFoundError) Error() string {
return "page not found"
}
func (pageNotFoundError) NotFound() {}
// createMux initializes the main router the server uses.
func (s *Server) createMux() *mux.Router {
m := mux.NewRouter()
log.G(context.TODO()).Debug("Registering routers")
for _, apiRouter := range routers {
logrus.Debug("Registering routers")
for _, apiRouter := range s.routers {
for _, r := range apiRouter.Routes() {
f := s.makeHTTPHandler(r.Handler(), r.Method()+" "+r.Path())
f := s.makeHTTPHandler(r.Handler())
log.G(context.TODO()).Debugf("Registering %s, %s", r.Method(), r.Path())
logrus.Debugf("Registering %s, %s", r.Method(), r.Path())
m.Path(versionMatcher + r.Path()).Methods(r.Method()).Handler(f)
m.Path(r.Path()).Methods(r.Method()).Handler(f)
}
}
debugRouter := debug.NewRouter()
s.routers = append(s.routers, debugRouter)
for _, r := range debugRouter.Routes() {
f := s.makeHTTPHandler(r.Handler(), r.Method()+" "+r.Path())
f := s.makeHTTPHandler(r.Handler())
m.Path("/debug" + r.Path()).Handler(f)
}
notFoundHandler := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
_ = httputils.WriteJSON(w, http.StatusNotFound, &types.ErrorResponse{
Message: "page not found",
})
})
notFoundHandler := makeErrorHandler(pageNotFoundError{})
m.HandleFunc(versionMatcher+"/{path:.*}", notFoundHandler)
m.NotFoundHandler = notFoundHandler
m.MethodNotAllowedHandler = notFoundHandler
return m
}
// Wait blocks the server goroutine until it exits.
// It sends an error message if there is any error during
// the API execution.
func (s *Server) Wait(waitChan chan error) {
if err := s.serveAPI(); err != nil {
logrus.Errorf("ServeAPI error: %v", err)
waitChan <- err
return
}
waitChan <- nil
}

View file

@ -13,13 +13,14 @@ import (
)
func TestMiddlewares(t *testing.T) {
srv := &Server{}
m, err := middleware.NewVersionMiddleware("0.1omega2", api.DefaultVersion, api.MinSupportedAPIVersion)
if err != nil {
t.Fatal(err)
cfg := &Config{
Version: "0.1omega2",
}
srv.UseMiddleware(*m)
srv := &Server{
cfg: cfg,
}
srv.UseMiddleware(middleware.NewVersionMiddleware("0.1omega2", api.DefaultVersion, api.MinVersion))
req, _ := http.NewRequest(http.MethodGet, "/containers/json", nil)
resp := httptest.NewRecorder()

File diff suppressed because it is too large Load diff

22
api/types/auth.go Normal file
View file

@ -0,0 +1,22 @@
package types // import "github.com/docker/docker/api/types"
// AuthConfig contains authorization information for connecting to a Registry
type AuthConfig struct {
Username string `json:"username,omitempty"`
Password string `json:"password,omitempty"`
Auth string `json:"auth,omitempty"`
// Email is an optional value associated with the username.
// This field is deprecated and will be removed in a later
// version of docker.
Email string `json:"email,omitempty"`
ServerAddress string `json:"serveraddress,omitempty"`
// IdentityToken is used to authenticate the user and get
// an access token for the registry.
IdentityToken string `json:"identitytoken,omitempty"`
// RegistryToken is a bearer token to be sent to a registry
RegistryToken string `json:"registrytoken,omitempty"`
}

View file

@ -5,40 +5,23 @@ import (
"io"
"time"
"github.com/distribution/reference"
"github.com/docker/docker/api/types/container"
"github.com/docker/docker/api/types/network"
ocispec "github.com/opencontainers/image-spec/specs-go/v1"
)
// ContainerCreateConfig is the parameter set to ContainerCreate()
type ContainerCreateConfig struct {
Name string
Config *container.Config
HostConfig *container.HostConfig
NetworkingConfig *network.NetworkingConfig
Platform *ocispec.Platform
DefaultReadOnlyNonRecursive bool
}
// ContainerRmConfig holds arguments for the container remove
// operation. This struct is used to tell the backend what operations
// to perform.
type ContainerRmConfig struct {
ForceRemove, RemoveVolume, RemoveLink bool
}
// ContainerAttachConfig holds the streams to use when connecting to a container to view logs.
type ContainerAttachConfig struct {
GetStreams func(multiplexed bool) (io.ReadCloser, io.Writer, io.Writer, error)
GetStreams func() (io.ReadCloser, io.Writer, io.Writer, error)
UseStdin bool
UseStdout bool
UseStderr bool
Logs bool
Stream bool
DetachKeys string
// Used to signify that streams must be multiplexed by producer as endpoint can't manage multiple streams.
// This is typically set by HTTP endpoint, while websocket can transport raw streams
// Used to signify that streams are multiplexed and therefore need a StdWriter to encode stdout/stderr messages accordingly.
// TODO @cpuguy83: This shouldn't be needed. It was only added so that http and websocket endpoints can use the same function, and the websocket function was not using a stdwriter prior to this change...
// HOWEVER, the websocket endpoint is using a single stream and SHOULD be encoded with stdout/stderr as is done for HTTP since it is still just a single stream.
// Since such a change is an API change unrelated to the current changeset we'll keep it as is here and change separately.
MuxStreams bool
}
@ -55,6 +38,8 @@ type PartialLogMetaData struct {
// LogMessage is datastructure that represents piece of output produced by some
// container. The Line member is a slice of an array whose contents can be
// changed after a log driver's Log() method returns.
// changes to this struct need to be reflect in the reset method in
// daemon/logger/logger.go
type LogMessage struct {
Line []byte
Source string
@ -90,6 +75,7 @@ type ContainerStatsConfig struct {
Stream bool
OneShot bool
OutStream io.Writer
Version string
}
// ExecInspect holds information about a running process started
@ -121,7 +107,8 @@ type ExecProcessConfig struct {
// CreateImageConfig is the configuration for creating an image from a
// container.
type CreateImageConfig struct {
Tag reference.NamedTagged
Repo string
Tag string
Pause bool
Author string
Comment string
@ -129,13 +116,6 @@ type CreateImageConfig struct {
Changes []string
}
// GetImageOpts holds parameters to retrieve image information
// from the backend.
type GetImageOpts struct {
Platform *ocispec.Platform
Details bool
}
// CommitConfig is the configuration for creating an image as part of a build.
type CommitConfig struct {
Author string
@ -147,25 +127,3 @@ type CommitConfig struct {
ContainerOS string
ParentImageID string
}
// PluginRmConfig holds arguments for plugin remove.
type PluginRmConfig struct {
ForceRemove bool
}
// PluginEnableConfig holds arguments for plugin enable
type PluginEnableConfig struct {
Timeout int
}
// PluginDisableConfig holds arguments for plugin disable.
type PluginDisableConfig struct {
ForceDisable bool
}
// NetworkListConfig stores the options available for listing networks
type NetworkListConfig struct {
// TODO(@cpuguy83): naming is hard, this is pulled from what was being used in the router before moving here
Detailed bool
Verbose bool
}

View file

@ -4,9 +4,8 @@ import (
"io"
"github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/registry"
"github.com/docker/docker/pkg/streamformatter"
ocispec "github.com/opencontainers/image-spec/specs-go/v1"
specs "github.com/opencontainers/image-spec/specs-go/v1"
)
// PullOption defines different modes for accessing images
@ -40,7 +39,7 @@ type BuildConfig struct {
// GetImageAndLayerOptions are the options supported by GetImageAndReleasableLayer
type GetImageAndLayerOptions struct {
PullOption PullOption
AuthConfig map[string]registry.AuthConfig
AuthConfig map[string]types.AuthConfig
Output io.Writer
Platform *ocispec.Platform
Platform *specs.Platform
}

View file

@ -1,7 +0,0 @@
package checkpoint
// Summary represents the details of a checkpoint when listing endpoints.
type Summary struct {
// Name is the name of the checkpoint.
Name string
}

View file

@ -1,19 +0,0 @@
package checkpoint
// CreateOptions holds parameters to create a checkpoint from a container.
type CreateOptions struct {
CheckpointID string
CheckpointDir string
Exit bool
}
// ListOptions holds parameters to list checkpoints for a container.
type ListOptions struct {
CheckpointDir string
}
// DeleteOptions holds parameters to delete a checkpoint from a container.
type DeleteOptions struct {
CheckpointID string
CheckpointDir string
}

View file

@ -7,10 +7,47 @@ import (
"github.com/docker/docker/api/types/container"
"github.com/docker/docker/api/types/filters"
"github.com/docker/docker/api/types/registry"
units "github.com/docker/go-units"
)
// CheckpointCreateOptions holds parameters to create a checkpoint from a container
type CheckpointCreateOptions struct {
CheckpointID string
CheckpointDir string
Exit bool
}
// CheckpointListOptions holds parameters to list checkpoints for a container
type CheckpointListOptions struct {
CheckpointDir string
}
// CheckpointDeleteOptions holds parameters to delete a checkpoint from a container
type CheckpointDeleteOptions struct {
CheckpointID string
CheckpointDir string
}
// ContainerAttachOptions holds parameters to attach to a container.
type ContainerAttachOptions struct {
Stream bool
Stdin bool
Stdout bool
Stderr bool
DetachKeys string
Logs bool
}
// ContainerCommitOptions holds parameters to commit changes into a container.
type ContainerCommitOptions struct {
Reference string
Comment string
Author string
Changes []string
Pause bool
Config *container.Config
}
// ContainerExecInspect holds information returned by exec inspect.
type ContainerExecInspect struct {
ExecID string `json:"ID"`
@ -20,6 +57,43 @@ type ContainerExecInspect struct {
Pid int
}
// ContainerListOptions holds parameters to list containers with.
type ContainerListOptions struct {
Quiet bool
Size bool
All bool
Latest bool
Since string
Before string
Limit int
Filters filters.Args
}
// ContainerLogsOptions holds parameters to filter logs with.
type ContainerLogsOptions struct {
ShowStdout bool
ShowStderr bool
Since string
Until string
Timestamps bool
Follow bool
Tail string
Details bool
}
// ContainerRemoveOptions holds parameters to remove containers.
type ContainerRemoveOptions struct {
RemoveVolumes bool
RemoveLinks bool
Force bool
}
// ContainerStartOptions holds parameters to start containers.
type ContainerStartOptions struct {
CheckpointID string
CheckpointDir string
}
// CopyToContainerOptions holds information
// about files to copy into a container
type CopyToContainerOptions struct {
@ -39,16 +113,10 @@ type NetworkListOptions struct {
Filters filters.Args
}
// NewHijackedResponse intializes a HijackedResponse type
func NewHijackedResponse(conn net.Conn, mediaType string) HijackedResponse {
return HijackedResponse{Conn: conn, Reader: bufio.NewReader(conn), mediaType: mediaType}
}
// HijackedResponse holds connection information for a hijacked request.
type HijackedResponse struct {
mediaType string
Conn net.Conn
Reader *bufio.Reader
Conn net.Conn
Reader *bufio.Reader
}
// Close closes the hijacked connection and reader.
@ -56,15 +124,6 @@ func (h *HijackedResponse) Close() {
h.Conn.Close()
}
// MediaType let client know if HijackedResponse hold a raw or multiplexed stream.
// returns false if HTTP Content-Type is not relevant, and container must be inspected
func (h *HijackedResponse) MediaType() (string, bool) {
if h.mediaType == "" {
return "", false
}
return h.mediaType, true
}
// CloseWriter is an interface that implements structs
// that close input streams to prevent from writing.
type CloseWriter interface {
@ -107,7 +166,7 @@ type ImageBuildOptions struct {
// at all (nil). See the parsing of buildArgs in
// api/server/router/build/build_routes.go for even more info.
BuildArgs map[string]*string
AuthConfigs map[string]registry.AuthConfig
AuthConfigs map[string]AuthConfig
Context io.Reader
Labels map[string]string
// squash the resulting image's layers to the parent
@ -157,12 +216,32 @@ type ImageBuildResponse struct {
OSType string
}
// ImageCreateOptions holds information to create images.
type ImageCreateOptions struct {
RegistryAuth string // RegistryAuth is the base64 encoded credentials for the registry.
Platform string // Platform is the target platform of the image if it needs to be pulled from the registry.
}
// ImageImportSource holds source information for ImageImport
type ImageImportSource struct {
Source io.Reader // Source is the data to send to the server to create this image from. You must set SourceName to "-" to leverage this.
SourceName string // SourceName is the name of the image to pull. Set to "-" to leverage the Source attribute.
}
// ImageImportOptions holds information to import images from the client host.
type ImageImportOptions struct {
Tag string // Tag is the name to tag this image with. This attribute is deprecated.
Message string // Message is the message to tag the image with
Changes []string // Changes are the raw changes to apply to this image
Platform string // Platform is the target platform of the image
}
// ImageListOptions holds parameters to filter the list of images with.
type ImageListOptions struct {
All bool
Filters filters.Args
}
// ImageLoadResponse returns information to the client about a load process.
type ImageLoadResponse struct {
// Body must be closed to avoid a resource leak
@ -170,6 +249,14 @@ type ImageLoadResponse struct {
JSON bool
}
// ImagePullOptions holds information to pull images.
type ImagePullOptions struct {
All bool
RegistryAuth string // RegistryAuth is the base64 encoded credentials for the registry
PrivilegeFunc RequestPrivilegeFunc
Platform string
}
// RequestPrivilegeFunc is a function interface that
// clients can supply to retry operations after
// getting an authorization error.
@ -178,6 +265,15 @@ type ImageLoadResponse struct {
// if the privilege request fails.
type RequestPrivilegeFunc func() (string, error)
// ImagePushOptions holds information to push images.
type ImagePushOptions ImagePullOptions
// ImageRemoveOptions holds parameters to remove images.
type ImageRemoveOptions struct {
Force bool
PruneChildren bool
}
// ImageSearchOptions holds parameters to search images with.
type ImageSearchOptions struct {
RegistryAuth string
@ -186,6 +282,14 @@ type ImageSearchOptions struct {
Limit int
}
// ResizeOptions holds parameters to resize a tty.
// It can be used to resize container ttys and
// exec process ttys too.
type ResizeOptions struct {
Height uint
Width uint
}
// NodeListOptions holds parameters to list nodes with.
type NodeListOptions struct {
Filters filters.Args
@ -211,6 +315,15 @@ type ServiceCreateOptions struct {
QueryRegistry bool
}
// ServiceCreateResponse contains the information returned to a client
// on the creation of a new service.
type ServiceCreateResponse struct {
// ID is the ID of the created service.
ID string
// Warnings is a set of non-fatal warning messages to pass on to the user.
Warnings []string `json:",omitempty"`
}
// Values for RegistryAuthFrom in ServiceUpdateOptions
const (
RegistryAuthFromSpec = "spec"

View file

@ -1,12 +1,38 @@
package types // import "github.com/docker/docker/api/types"
import (
"github.com/docker/docker/api/types/container"
"github.com/docker/docker/api/types/network"
specs "github.com/opencontainers/image-spec/specs-go/v1"
)
// configs holds structs used for internal communication between the
// frontend (such as an http server) and the backend (such as the
// docker daemon).
// ContainerCreateConfig is the parameter set to ContainerCreate()
type ContainerCreateConfig struct {
Name string
Config *container.Config
HostConfig *container.HostConfig
NetworkingConfig *network.NetworkingConfig
Platform *specs.Platform
AdjustCPUShares bool
}
// ContainerRmConfig holds arguments for the container remove
// operation. This struct is used to tell the backend what operations
// to perform.
type ContainerRmConfig struct {
ForceRemove, RemoveVolume, RemoveLink bool
}
// ExecConfig is a small subset of the Config struct that holds the configuration
// for the exec feature of docker.
type ExecConfig struct {
User string // User that will run the command
Privileged bool // Is the container in privileged mode
Tty bool // Attach standard streams to a tty.
ConsoleSize *[2]uint `json:",omitempty"` // Initial console size [height, width]
AttachStdin bool // Attach the standard input, makes possible user interaction
AttachStderr bool // Attach the standard error
AttachStdout bool // Attach the standard output
@ -16,3 +42,25 @@ type ExecConfig struct {
WorkingDir string // Working directory
Cmd []string // Execution commands and args
}
// PluginRmConfig holds arguments for plugin remove.
type PluginRmConfig struct {
ForceRemove bool
}
// PluginEnableConfig holds arguments for plugin enable
type PluginEnableConfig struct {
Timeout int
}
// PluginDisableConfig holds arguments for plugin disable.
type PluginDisableConfig struct {
ForceDisable bool
}
// NetworkListConfig stores the options available for listing networks
type NetworkListConfig struct {
// TODO(@cpuguy83): naming is hard, this is pulled from what was being used in the router before moving here
Detailed bool
Verbose bool
}

Some files were not shown because too many files have changed in this diff Show more