Compare commits

...

956 commits
master ... 23.0

Author SHA1 Message Date
Paweł Gronowski
ab34280ac3
Merge pull request #47699 from vvoland/v23.0-47658
[23.0 backport] Fix cases where we are wrapping a nil error
2024-04-09 13:55:11 +02:00
Brian Goff
8f67ed81aa
Fix cases where we are wrapping a nil error
This was using `errors.Wrap` when there was no error to wrap, meanwhile
we are supposed to be creating a new error.

Found this while investigating some log corruption issues and
unexpectedly getting a nil reader and a nil error from `getTailReader`.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit 0a48d26fbc)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-04-09 10:21:36 +02:00
Bjorn Neergaard
bed0abf9ca
Merge pull request #47593 from corhere/backport-23.0/libnet-resolver-nxdomain
[23.0 backport] libnet: Don't forward to upstream resolvers on internal nw
2024-03-19 12:18:12 -06:00
Albin Kerouanton
f4657eae7d libnet: Don't forward to upstream resolvers on internal nw
This commit makes sure the embedded resolver doesn't try to forward to
upstream servers when a container is only attached to an internal
network.

Co-authored-by: Albin Kerouanton <albinker@gmail.com>
Signed-off-by: Rob Murray <rob.murray@docker.com>
(cherry picked from commit 790c3039d0)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-03-19 12:50:35 -04:00
Albin Kerouanton
a379e026c9 integration: Add a new networking integration test suite
This commit introduces a new integration test suite aimed at testing
networking features like inter-container communication, network
isolation, port mapping, etc... and how they interact with daemon-level
and network-level parameters.

So far, there's pretty much no tests making sure our networks are well
configured: 1. there're a few tests for port mapping, but they don't
cover all use cases ; 2. there're a few tests that check if a specific
iptables rule exist, but that doesn't prevent that specific iptables
rule to be wrong in the first place.

As we're planning to refactor how iptables rules are written, and change
some of them to fix known security issues, we need a way to test all
combinations of parameters. So far, this was done by hand, which is
particularly painful and time consuming. As such, this new test suite is
foundational to upcoming work.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
(cherry picked from commit 409ea700c7)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-03-19 12:50:35 -04:00
Albin Kerouanton
673020119f integration: Add RunAttach helper
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
(cherry picked from commit 5bd8aa5246)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-03-18 16:38:35 -04:00
Sebastiaan van Stijn
1725cc5f92
Merge pull request #47535 from vvoland/v23.0-47530
[23.0 backport] volume: Don't decrement refcount below 0
2024-03-11 15:58:46 +01:00
Paweł Gronowski
d1aa20efb7
volume: Don't decrement refcount below 0
With both rootless and live restore enabled, there's some race condition
which causes the container to be `Unmount`ed before the refcount is
restored.

This makes sure we don't underflow the refcount (uint64) when
decrementing it.

The root cause of this race condition still needs to be investigated and
fixed, but at least this unflakies the `TestLiveRestore`.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 294fc9762e)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-08 12:56:41 +01:00
Sebastiaan van Stijn
e2ccfc19cc
Merge pull request #47529 from vvoland/v23.0-47523
[23.0 backport] builder-next: fix missing lock in ensurelayer
2024-03-07 13:24:55 +01:00
Tonis Tiigi
e35f6fbd08
builder-next: fix missing lock in ensurelayer
When this was called concurrently from the moby image
exporter there could be a data race where a layer was
written to the refs map when it was already there.

In that case the reference count got mixed up and on
release only one of these layers was actually released.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit 37545cc644)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-07 12:33:26 +01:00
Sebastiaan van Stijn
64b5835e95
Merge pull request #47515 from vvoland/v23.0-47498
[23.0 backport] daemon: overlay2: remove world writable permission from the lower file
2024-03-06 15:19:26 +01:00
Jaroslav Jindrak
f2954d7622
daemon: overlay2: remove world writable permission from the lower file
In de2447c, the creation of the 'lower' file was changed from using
os.Create to using ioutils.AtomicWriteFile, which ignores the system's
umask. This means that even though the requested permission in the
source code was always 0666, it was 0644 on systems with default
umask of 0022 prior to de2447c, so the move to AtomicFile potentially
increased the file's permissions.

This is not a security issue because the parent directory does not
allow writes into the file, but it can confuse security scanners on
Linux-based systems into giving false positives.

Signed-off-by: Jaroslav Jindrak <dzejrou@gmail.com>
(cherry picked from commit cadb124ab6)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-06 13:14:18 +01:00
Sebastiaan van Stijn
548f37a132
Merge pull request #47339 from vvoland/cache-fix-older-windows-23
[23.0 backport] image/cache: Ignore Build and Revision on Windows
2024-02-16 17:00:24 +01:00
Akihiro Suda
5cc674870c
Merge pull request #47346 from thaJeztah/23.0_backport_seccomp_updates
[23.0 backport] profiles/seccomp: add syscalls for kernel v5.17 - v6.6, match containerd's profile
2024-02-07 19:47:54 +09:00
Paweł Gronowski
d10756f356
image/cache: Require Major and Minor match for Windows OSVersion
The platform comparison was backported from the branch that vendors
containerd 1.7.

In this branch the vendored containerd version is older and doesn't have
the same comparison logic for Windows specific OSVersion.

Require both major and minor components of Windows OSVersion to match.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit b3888ed899)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-06 17:52:16 +01:00
Sebastiaan van Stijn
20b867b732
seccomp: add futex_wake syscall (kernel v6.7, libseccomp v2.5.5)
Add this syscall to match the profile in containerd

containerd: a6e52c74fa
libseccomp: 53267af3fb
kernel: 9f6c532f59

    futex: Add sys_futex_wake()

    To complement sys_futex_waitv() add sys_futex_wake(). This syscall
    implements what was previously known as FUTEX_WAKE_BITSET except it
    uses 'unsigned long' for the bitmask and takes FUTEX2 flags.

    The 'unsigned long' allows FUTEX2_SIZE_U64 on 64bit platforms.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit d69729e053)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-06 15:29:53 +01:00
Sebastiaan van Stijn
fe0619e49f
seccomp: add futex_wait syscall (kernel v6.7, libseccomp v2.5.5)
Add this syscall to match the profile in containerd

containerd: a6e52c74fa
libseccomp: 53267af3fb
kernel: cb8c4312af

    futex: Add sys_futex_wait()

    To complement sys_futex_waitv()/wake(), add sys_futex_wait(). This
    syscall implements what was previously known as FUTEX_WAIT_BITSET
    except it uses 'unsigned long' for the value and bitmask arguments,
    takes timespec and clockid_t arguments for the absolute timeout and
    uses FUTEX2 flags.

    The 'unsigned long' allows FUTEX2_SIZE_U64 on 64bit platforms.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 10d344d176)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-06 15:29:53 +01:00
Sebastiaan van Stijn
68e7b988b1
seccomp: add futex_requeue syscall (kernel v6.7, libseccomp v2.5.5)
Add this syscall to match the profile in containerd

containerd: a6e52c74fa
libseccomp: 53267af3fb
kernel: 0f4b5f9722

    futex: Add sys_futex_requeue()

    Finish off the 'simple' futex2 syscall group by adding
    sys_futex_requeue(). Unlike sys_futex_{wait,wake}() its arguments are
    too numerous to fit into a regular syscall. As such, use struct
    futex_waitv to pass the 'source' and 'destination' futexes to the
    syscall.

    This syscall implements what was previously known as FUTEX_CMP_REQUEUE
    and uses {val, uaddr, flags} for source and {uaddr, flags} for
    destination.

    This design explicitly allows requeueing between different types of
    futex by having a different flags word per uaddr.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit df57a080b6)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-06 15:29:53 +01:00
Sebastiaan van Stijn
26d766450c
seccomp: add map_shadow_stack syscall (kernel v6.6, libseccomp v2.5.5)
Add this syscall to match the profile in containerd

containerd: a6e52c74fa
libseccomp: 53267af3fb
kernel: c35559f94e

    x86/shstk: Introduce map_shadow_stack syscall

    When operating with shadow stacks enabled, the kernel will automatically
    allocate shadow stacks for new threads, however in some cases userspace
    will need additional shadow stacks. The main example of this is the
    ucontext family of functions, which require userspace allocating and
    pivoting to userspace managed stacks.

    Unlike most other user memory permissions, shadow stacks need to be
    provisioned with special data in order to be useful. They need to be setup
    with a restore token so that userspace can pivot to them via the RSTORSSP
    instruction. But, the security design of shadow stacks is that they
    should not be written to except in limited circumstances. This presents a
    problem for userspace, as to how userspace can provision this special
    data, without allowing for the shadow stack to be generally writable.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 8826f402f9)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-06 15:29:53 +01:00
Sebastiaan van Stijn
c98179d3c7
seccomp: add fchmodat2 syscall (kernel v6.6, libseccomp v2.5.5)
Add this syscall to match the profile in containerd

containerd: a6e52c74fa
libseccomp: 53267af3fb
kernel: 09da082b07

    fs: Add fchmodat2()

    On the userspace side fchmodat(3) is implemented as a wrapper
    function which implements the POSIX-specified interface. This
    interface differs from the underlying kernel system call, which does not
    have a flags argument. Most implementations require procfs [1][2].

    There doesn't appear to be a good userspace workaround for this issue
    but the implementation in the kernel is pretty straight-forward.

    The new fchmodat2() syscall allows to pass the AT_SYMLINK_NOFOLLOW flag,
    unlike existing fchmodat.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 6f242f1a28)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-06 15:29:52 +01:00
Sebastiaan van Stijn
573ebdba6e
seccomp: add cachestat syscall (kernel v6.5, libseccomp v2.5.5)
Add this syscall to match the profile in containerd

containerd: a6e52c74fa
libseccomp: 53267af3fb
kernel: cf264e1329

    NAME
        cachestat - query the page cache statistics of a file.

    SYNOPSIS
        #include <sys/mman.h>

        struct cachestat_range {
            __u64 off;
            __u64 len;
        };

        struct cachestat {
            __u64 nr_cache;
            __u64 nr_dirty;
            __u64 nr_writeback;
            __u64 nr_evicted;
            __u64 nr_recently_evicted;
        };

        int cachestat(unsigned int fd, struct cachestat_range *cstat_range,
            struct cachestat *cstat, unsigned int flags);

    DESCRIPTION
        cachestat() queries the number of cached pages, number of dirty
        pages, number of pages marked for writeback, number of evicted
        pages, number of recently evicted pages, in the bytes range given by
        `off` and `len`.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 4d0d5ee10d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-06 15:29:52 +01:00
Sebastiaan van Stijn
b9f01eea45
seccomp: add set_mempolicy_home_node syscall (kernel v5.17, libseccomp v2.5.4)
This syscall is gated by CAP_SYS_NICE, matching the profile in containerd.

containerd: a6e52c74fa
libseccomp: d83cb7ac25
kernel: c6018b4b25

    mm/mempolicy: add set_mempolicy_home_node syscall
    This syscall can be used to set a home node for the MPOL_BIND and
    MPOL_PREFERRED_MANY memory policy.  Users should use this syscall after
    setting up a memory policy for the specified range as shown below.

      mbind(p, nr_pages * page_size, MPOL_BIND, new_nodes->maskp,
            new_nodes->size + 1, 0);
      sys_set_mempolicy_home_node((unsigned long)p, nr_pages * page_size,
                    home_node, 0);

    The syscall allows specifying a home node/preferred node from which
    kernel will fulfill memory allocation requests first.
    ...

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 1251982cf7)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-06 15:29:52 +01:00
Paweł Gronowski
a51e65b334
image/cache: Use Platform from ocispec
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 2c01d53d96)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-06 14:30:18 +01:00
Paweł Gronowski
347be7f928
image/cache: Ignore Build and Revision on Windows
The compatibility depends on whether `hyperv` or `process` container
isolation is used.
This fixes cache not being used when building images based on older
Windows versions on a newer Windows host.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 91ea04089b)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-06 13:22:43 +01:00
Laura Brehm
58a26e4fda
Merge pull request #47325 from thaJeztah/23.0_backport_plugin-install-digest
[23.0 backport] plugins: Fix panic when fetching by digest
2024-02-05 13:47:13 +00:00
Laura Brehm
d5c9f26e3d
plugins: fix panic installing from repo w/ digest
Only print the tag when the received reference has a tag, if
we can't cast the received tag to a `reference.Tagged` then
skip printing the tag as it's likely a digest.

Fixes panic when trying to install a plugin from a reference
with a digest such as
`vieux/sshfs@sha256:1d3c3e42c12138da5ef7873b97f7f32cf99fb6edde75fa4f0bcf9ed277855811`

Signed-off-by: Laura Brehm <laurabrehm@hey.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-05 10:12:40 +01:00
Laura Brehm
2dc6b50863
tests: add plugin install test w/ digest
Adds a test case for installing a plugin from a remote in the form
of `plugin-content-trust@sha256:d98f2f8061...`, which is currently
causing the daemon to panic, as we found while running the CLI e2e
tests:

```
docker plugin install registry:5000/plugin-content-trust@sha256:d98f2f806144bf4ba62d4ecaf78fec2f2fe350df5a001f6e3b491c393326aedb
```

Signed-off-by: Laura Brehm <laurabrehm@hey.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-05 10:12:34 +01:00
Sebastiaan van Stijn
79de67d989
Merge pull request from GHSA-xw73-rw38-6vjc
[23.0 backport] image/cache: Restrict cache candidates to locally built images
2024-02-01 01:12:23 +01:00
Sebastiaan van Stijn
43e3116aa7
Merge pull request #47284 from thaJeztah/23.0_bump_containerd_binary_1.6.28
[23.0] update containerd binary to v1.6.28
2024-02-01 00:33:37 +01:00
Sebastiaan van Stijn
b523cec06b
Merge pull request #47271 from thaJeztah/23.0_backport_bump_runc_binary_1.1.12
[23.0 backport] update runc binary to v1.1.12
2024-02-01 00:08:16 +01:00
Sebastiaan van Stijn
045dfe9e09
update containerd binary to v1.6.28
Update the containerd binary that's used in CI

- full diff: https://github.com/containerd/containerd/compare/v1.6.27...v1.6.28
- release notes: https://github.com/containerd/containerd/releases/tag/v1.6.28

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-31 23:42:13 +01:00
Sebastiaan van Stijn
1c867c02ef
Merge pull request #47261 from thaJeztah/23.0_bump_containerd_windows_binary_1.6.27
[23.0] Dockerfile: update containerd binary for windows to 1.6.27
2024-01-31 23:41:19 +01:00
Sebastiaan van Stijn
e6ff860b96
Merge pull request #47282 from aepifanov/23.0_backport_bump_runc_1.1.12
[23.0 backport] vendor: github.com/opencontainers/runc v1.1.12
2024-01-31 23:40:13 +01:00
Andrey Epifanov
76894512ab
vendor: github.com/opencontainers/runc v1.1.12
- release notes: https://github.com/opencontainers/runc/releases/tag/v1.1.12
- full diff: https://github.com/opencontainers/runc/compare/v1.1.5...v1.1.12

Signed-off-by: Andrey Epifanov <aepifanov@mirantis.com>
2024-01-31 13:22:09 -08:00
Andrey Epifanov
e23be1cc51
vendor: github.com/cyphar/filepath-securejoin v0.2.4
full diff: https://github.com/cyphar/filepath-securejoin/compare/v0.2.3..v0.2.4

Signed-off-by: Andrey Epifanov <aepifanov@mirantis.com>
2024-01-31 13:19:19 -08:00
Sebastiaan van Stijn
46cfa2e34c
Dockerfile: update containerd binary for windows to 1.6.27
Update the containerd binary that's used in CI to align with the version used
for Linux. This was missed in d6abda0710.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-31 21:43:42 +01:00
Sebastiaan van Stijn
5fbbb68dad
Merge pull request #47267 from vvoland/ci-fix-makeps1-templatefail-23
[23.0 backport] hack/make.ps1: Fix go list pattern
2024-01-31 21:36:36 +01:00
Sebastiaan van Stijn
448e7bed7d
update runc binary to v1.1.12
Update the runc binary that's used in CI and for the static packages, which
includes a fix for [CVE-2024-21626].

- release notes: https://github.com/opencontainers/runc/releases/tag/v1.1.12
- full diff: https://github.com/opencontainers/runc/compare/v1.1.11...v1.1.12

[CVE-2024-21626]: https://github.com/opencontainers/runc/security/advisories/GHSA-xr7r-f8xq-vfvv

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 44bf407d4d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-31 21:06:43 +01:00
Paweł Gronowski
4bb779015a
hack/make.ps1: Fix go list pattern
The double quotes inside a single quoted string don't need to be
escaped.
Looks like different Powershell versions are treating this differently
and it started failing unexpectedly without any changes on our side.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit ecb217cf69)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-31 19:58:08 +01:00
Sebastiaan van Stijn
42d92e8903
Merge pull request #47226 from thaJeztah/23.0_update_containerd_binary
[23.0] Dockerfile: update containerd binary to v1.6.27
2024-01-25 21:40:50 +01:00
Paweł Gronowski
d5de9f7779
builder/windows: Don't set ArgsEscaped for RUN cache probe
Previously this was done indirectly - the `compare` function didn't
check the `ArgsEscaped`.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 96d461d27e)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 44e6f3da60)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-25 16:33:36 +01:00
Paweł Gronowski
bb03b5b86e
image/cache: Check image platform
Make sure the cache candidate platform matches the requested.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 877ebbe038)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 8a19bb7193)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-25 16:33:33 +01:00
Sebastiaan van Stijn
d6abda0710
Dockerfile: update containerd binary to v1.6.27
Update the binary version used in CI and for the static packages.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-25 16:31:10 +01:00
Paweł Gronowski
75c70b08b5
image/cache: Restrict cache candidates to locally built images
Restrict cache candidates only to images that were built locally.
This doesn't affect builds using `--cache-from`.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 96ac22768a)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 17af50f46b)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-25 16:23:08 +01:00
Paweł Gronowski
8617cd0570
daemon/imageStore: Mark images built locally
Store additional image property which makes it possible to distinguish
if image was built locally.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit c6156dc51b)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit ffb63c0bae)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-25 16:23:06 +01:00
Paweł Gronowski
e9e21b6bf1
image/cache: Compare all config fields
Add checks for some image config fields that were missing.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 537348763f)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 593b754d8f)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-25 16:23:05 +01:00
Sebastiaan van Stijn
d363536dca
Merge pull request #47223 from vvoland/pkg-pools-close-noop-23
[23.0 backport] pkg/ioutils: Make subsequent Close attempts noop
2024-01-25 16:16:18 +01:00
Paweł Gronowski
43c2952a31
pkg/ioutils: Make subsequent Close attempts noop
Turn subsequent `Close` calls into a no-op and produce a warning with an
optional stack trace (if debug mode is enabled).

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 585d74bad1)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-01-25 15:23:28 +01:00
Sebastiaan van Stijn
c0ee655462
Merge pull request #46900 from thaJeztah/23.0_update_golang_1.20.12
[23.0] update to go1.20.13
2024-01-20 14:52:48 +01:00
Sebastiaan van Stijn
751d19fd3c
update to go1.20.13
go1.20.13 (released 2024-01-09) includes fixes to the runtime and the crypto/tls
package. See the Go 1.20.13 milestone on our issue tracker for details:

- https://github.com/golang/go/issues?q=milestone%3AGo1.20.13+label%3ACherryPickApproved
- full diff: https://github.com/golang/go/compare/go1.20.12...go1.20.13

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-19 12:12:23 +01:00
Sebastiaan van Stijn
583e29949e
update to go1.20.12
go1.20.12 (released 2023-12-05) includes security fixes to the go command,
and the net/http and path/filepath packages, as well as bug fixes to the
compiler and the go command. See the Go 1.20.12 milestone on our issue
tracker for details.

- https://github.com/golang/go/issues?q=milestone%3AGo1.20.12+label%3ACherryPickApproved
- full diff: https://github.com/golang/go/compare/go1.20.11...go1.20.12

from the security mailing:

[security] Go 1.21.5 and Go 1.20.12 are released

Hello gophers,

We have just released Go versions 1.21.5 and 1.20.12, minor point releases.

These minor releases include 3 security fixes following the security policy:

- net/http: limit chunked data overhead

  A malicious HTTP sender can use chunk extensions to cause a receiver
  reading from a request or response body to read many more bytes from
  the network than are in the body.

  A malicious HTTP client can further exploit this to cause a server to
  automatically read a large amount of data (up to about 1GiB) when a
  handler fails to read the entire body of a request.

  Chunk extensions are a little-used HTTP feature which permit including
  additional metadata in a request or response body sent using the chunked
  encoding. The net/http chunked encoding reader discards this metadata.
  A sender can exploit this by inserting a large metadata segment with
  each byte transferred. The chunk reader now produces an error if the
  ratio of real body to encoded bytes grows too small.

  Thanks to Bartek Nowotarski for reporting this issue.

  This is CVE-2023-39326 and Go issue https://go.dev/issue/64433.

- cmd/go: go get may unexpectedly fallback to insecure git

  Using go get to fetch a module with the ".git" suffix may unexpectedly
  fallback to the insecure "git://" protocol if the module is unavailable
  via the secure "https://" and "git+ssh://" protocols, even if GOINSECURE
  is not set for said module. This only affects users who are not using
  the module proxy and are fetching modules directly (i.e. GOPROXY=off).

  Thanks to David Leadbeater for reporting this issue.

  This is CVE-2023-45285 and Go issue https://go.dev/issue/63845.

- path/filepath: retain trailing \ when cleaning paths like \\?\c:\

  Go 1.20.11 and Go 1.21.4 inadvertently changed the definition of the
  volume name in Windows paths starting with \\?\, resulting in
  filepath.Clean(\\?\c:\) returning \\?\c: rather than \\?\c:\ (among
  other effects). The previous behavior has been restored.

  This is an update to CVE-2023-45283 and Go issue https://go.dev/issue/64028.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-19 12:12:23 +01:00
Sebastiaan van Stijn
1dcd83acb4
update to go1.20.11
go1.20.11 (released 2023-11-07) includes security fixes to the path/filepath
package, as well as bug fixes to the linker and the net/http package. See the
Go 1.20.11 milestone on our issue tracker for details:

- https://github.com/golang/go/issues?q=milestone%3AGo1.20.11+label%3ACherryPickApproved
- full diff: https://github.com/golang/go/compare/go1.20.10...go1.20.11

from the security mailing:

[security] Go 1.21.4 and Go 1.20.11 are released

Hello gophers,

We have just released Go versions 1.21.4 and 1.20.11, minor point releases.

These minor releases include 2 security fixes following the security policy:

- path/filepath: recognize `\??\` as a Root Local Device path prefix.

  On Windows, a path beginning with `\??\` is a Root Local Device path equivalent
  to a path beginning with `\\?\`. Paths with a `\??\` prefix may be used to
  access arbitrary locations on the system. For example, the path `\??\c:\x`
  is equivalent to the more common path c:\x.

  The filepath package did not recognize paths with a `\??\` prefix as special.

  Clean could convert a rooted path such as `\a\..\??\b` into
  the root local device path `\??\b`. It will now convert this
  path into `.\??\b`.

  `IsAbs` did not report paths beginning with `\??\` as absolute.
  It now does so.

  VolumeName now reports the `\??\` prefix as a volume name.

  `Join(`\`, `??`, `b`)` could convert a seemingly innocent
  sequence of path elements into the root local device path
  `\??\b`. It will now convert this to `\.\??\b`.

  This is CVE-2023-45283 and https://go.dev/issue/63713.

- path/filepath: recognize device names with trailing spaces and superscripts

  The `IsLocal` function did not correctly detect reserved names in some cases:

  - reserved names followed by spaces, such as "COM1 ".
  - "COM" or "LPT" followed by a superscript 1, 2, or 3.

  `IsLocal` now correctly reports these names as non-local.

  This is CVE-2023-45284 and https://go.dev/issue/63713.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-19 12:12:23 +01:00
Sebastiaan van Stijn
97280b92b5
Merge pull request #46951 from thaJeztah/23.0_backport_local_logs_timezone
[23.0 backport] daemon/logger/local: always use UTC for timestamps
2024-01-19 12:11:37 +01:00
Sebastiaan van Stijn
190138d0ec
Merge pull request #47110 from corhere/backport-23.0/lock-container-when-deleting-root
[23.0 backport] Lock container when deleting its root directory
2024-01-19 12:11:16 +01:00
Cory Snider
440d3b00fe Lock container when deleting its root directory
Attempting to delete the directory while another goroutine is
concurrently executing a CheckpointTo() can fail on Windows due to file
locking. As all callers of CheckpointTo() are required to hold the
container lock, holding the lock while deleting the directory ensures
that there will be no interference.

Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit 18e322bc7c)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2024-01-18 15:23:24 -05:00
Akihiro Suda
6aeb5db52a
Merge pull request #47095 from thaJeztah/23.0_backport_bump_golangci_lint
[23.0 backport] update golangci-lint to v1.55.2
2024-01-18 18:11:37 +09:00
Sebastiaan van Stijn
5ccc9357db
update golangci-lint to v1.55.2
- full diff: https://github.com/golangci/golangci-lint/compare/v1.54.2...v1.55.2

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit d5a3fccb06)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-17 22:21:47 +01:00
Sebastiaan van Stijn
b19284a8de
daemon/logger/local: always use UTC for timestamps
When reading logs, timestamps should always be presented in UTC. Unlike
the "json-file" and other logging drivers, the "local" logging driver
was using local time.

Thanks to Roman Valov for reporting this issue, and locating the bug.

Before this change:

    echo $TZ
    Europe/Amsterdam

    docker run -d --log-driver=local nginx:alpine
    fc166c6b2c35c871a13247dddd95de94f5796459e2130553eee91cac82766af3

    docker logs --timestamps fc166c6b2c35c871a13247dddd95de94f5796459e2130553eee91cac82766af3
    2023-12-08T18:16:56.291023422+01:00 /docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
    2023-12-08T18:16:56.291056463+01:00 /docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
    2023-12-08T18:16:56.291890130+01:00 /docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
    ...

With this patch:

    echo $TZ
    Europe/Amsterdam

    docker run -d --log-driver=local nginx:alpine
    14e780cce4c827ce7861d7bc3ccf28b21f6e460b9bfde5cd39effaa73a42b4d5

    docker logs --timestamps 14e780cce4c827ce7861d7bc3ccf28b21f6e460b9bfde5cd39effaa73a42b4d5
    2023-12-08T17:18:46.635967625Z /docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
    2023-12-08T17:18:46.635989792Z /docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
    2023-12-08T17:18:46.636897417Z /docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
    ...

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit afe281964d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-04 18:33:33 +01:00
Sebastiaan van Stijn
14e65c6ff6
Merge pull request #47011 from thaJeztah/23.0_backport_bump_runc_binary
[23.0 backport] update runc binary to v1.1.11
2024-01-03 19:26:13 +01:00
Sebastiaan van Stijn
8c6145e6ba
update runc binary to v1.1.11
This is the eleventh patch release in the 1.1.z release branch of runc.
It primarily fixes a few issues with runc's handling of containers that
are configured to join existing user namespaces, as well as improvements
to cgroupv2 support.

- Fix several issues with userns path handling.
- Support memory.peak and memory.swap.peak in cgroups v2.
  Add swapOnlyUsage in MemoryStats. This field reports swap-only usage.
  For cgroupv1, Usage and Failcnt are set by subtracting memory usage
  from memory+swap usage. For cgroupv2, Usage, Limit, and MaxUsage
  are set.
- build(deps): bump github.com/cyphar/filepath-securejoin.

- release notes: https://github.com/opencontainers/runc/releases/tag/v1.1.11
- full diff: https://github.com/opencontainers/runc/compare/v1.1.10...v1.1.11

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 5fa4cfcabf)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-02 23:44:36 +01:00
Brian Goff
d67439d7b6
Merge pull request #46994 from thaJeztah/23.0_backport_46621-container_wait
[23.0 backport] Ensure that non-JSON-parsing errors are returned to the caller
2024-01-02 10:36:25 -08:00
Stefan Gehrig
69f337c285
Ensure that non-JSON-parsing errors are returned to the caller
Signed-off-by: Stefan Gehrig <stefan.gehrig.hn@googlemail.com>
Co-authored-by: Cory Snider <corhere@gmail.com>
(cherry picked from commit 0d27579fc7)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-27 14:19:46 +01:00
Sebastiaan van Stijn
03bc5f4f1c
Merge pull request #46948 from thaJeztah/23.0_backport_gc_time_filter
[23.0 backport] builder-next: fix timing filter for default policy
2023-12-17 18:01:03 +01:00
Tonis Tiigi
1fcbc7cf93
builder-next: fix timing filter for default policy
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit 49d088d9ce)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-15 19:57:57 +01:00
Sebastiaan van Stijn
cad80da86f
Merge pull request #46807 from thaJeztah/23.0_backport_bump_runc_binary_1.1.10
[23.0 backport] update runc binary to v1.1.10
2023-11-13 20:51:01 +01:00
Sebastiaan van Stijn
d3acc23a0c
update runc binary to v1.1.10
- full diff: https://github.com/opencontainers/runc/compare/v1.1.9...v1.1.10
- release notes: https://github.com/opencontainers/runc/releases/tag/v1.1.10

This is the tenth (and most likely final) patch release in the 1.1.z
release branch of runc. It mainly fixes a few issues in cgroups, and a
umask-related issue in tmpcopyup.

- Add support for `hugetlb.<pagesize>.rsvd` limiting and accounting.
  Fixes the issue of postgres failing when hugepage limits are set.
- Fixed permissions of a newly created directories to not depend on the value
  of umask in tmpcopyup feature implementation.
- libcontainer: cgroup v1 GetStats now ignores missing `kmem.limit_in_bytes`
  (fixes the compatibility with Linux kernel 6.1+).
- Fix a semi-arbitrary cgroup write bug when given a malicious hugetlb
  configuration. This issue is not a security issue because it requires a
  malicious config.json, which is outside of our threat model.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 15bcc707e6)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-11-13 16:08:52 +01:00
Sebastiaan van Stijn
48ebe353e4
Merge pull request from GHSA-jq35-85cj-fj4p
[23.0 backport] deny /sys/devices/virtual/powercap
2023-10-25 23:57:51 +02:00
Sebastiaan van Stijn
0360dbe11d
Merge pull request #46710 from thaJeztah/23.0_backport_bump_compress
[23.0 backport] vendor: github.com/klauspost/compress v1.17.2
2023-10-24 16:55:07 +02:00
Cory Snider
1a92838c41
Merge pull request #46704 from thaJeztah/23.0_backport_atomic-layer-data-write
[23.0 backport] daemon: overlay2: Write layer metadata atomically
2023-10-24 10:22:18 -04:00
Sebastiaan van Stijn
ec83f9e5e3
vendor: github.com/klauspost/compress v1.17.2
fixes data corruption with zstd output in "best"

- 1.17.2 diff: https://github.com/klauspost/compress/compare/v1.17.1...v1.17.2
- full diff: https://github.com/klauspost/compress/compare/v1.16.5...v1.17.2

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit f2c67ea82f)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-24 12:46:28 +02:00
Sebastiaan van Stijn
7bda7b5c10
vendor: github.com/klauspost/compress v1.16.5
full diff: https://github.com/klauspost/compress/compare/v1.16.3...v1.16.5

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 0354791147)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-24 12:45:38 +02:00
Sebastiaan van Stijn
e7ab9e387f
vendor: github.com/klauspost/compress v1.16.3
full diff: https://github.com/klauspost/compress/compare/v1.15.12...v1.16.3

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit b98c05dc50)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-24 12:44:12 +02:00
Mike Sul
1728815648
daemon: overlay2: Write layer metadata atomically
When the daemon process or the host running it is abruptly terminated,
the layer metadata file can become inconsistent on the file system.
Specifically, `link` and `lower` files may exist but be empty, leading
to overlay mounting errors during layer extraction, such as:
"failed to register layer: error creating overlay mount to <path>:
too many levels of symbolic links."

This commit introduces the use of `AtomicWriteFile` to ensure that the
layer metadata files contain correct data when they exist on the file system.

Signed-off-by: Mike <mike.sul@foundries.io>
(cherry picked from commit de2447c2ab)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-23 20:19:13 +02:00
Bjorn Neergaard
17f833b83d
Merge pull request #46691 from corhere/backport-23.0/update-x-net-v0.17
[23.0 backport] vendor: golang.org/x/net v0.17.0
2023-10-20 16:14:32 -06:00
Sebastiaan van Stijn
fcc2ca7cdf vendor: golang.org/x/net v0.17.0
full diff: https://github.com/golang/net/compare/v0.13.0...v0.17.0

This fixes the same CVE as go1.21.3 and go1.20.10;

- net/http: rapid stream resets can cause excessive work

  A malicious HTTP/2 client which rapidly creates requests and
  immediately resets them can cause excessive server resource consumption.
  While the total number of requests is bounded to the
  http2.Server.MaxConcurrentStreams setting, resetting an in-progress
  request allows the attacker to create a new request while the existing
  one is still executing.

  HTTP/2 servers now bound the number of simultaneously executing
  handler goroutines to the stream concurrency limit. New requests
  arriving when at the limit (which can only happen after the client
  has reset an existing, in-flight request) will be queued until a
  handler exits. If the request queue grows too large, the server
  will terminate the connection.

  This issue is also fixed in golang.org/x/net/http2 v0.17.0,
  for users manually configuring HTTP/2.

  The default stream concurrency limit is 250 streams (requests)
  per HTTP/2 connection. This value may be adjusted using the
  golang.org/x/net/http2 package; see the Server.MaxConcurrentStreams
  setting and the ConfigureServer function.

  This is CVE-2023-39325 and Go issue https://go.dev/issue/63417.
  This is also tracked by CVE-2023-44487.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 1800dd0876)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-10-20 15:38:17 -04:00
Sebastiaan van Stijn
dc1800ca7d vendor: golang.org/x/crypto v0.14.0
full diff: https://github.com/golang/crypto/compare/v0.11.0...v0.14.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 1946aead21)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-10-20 15:37:25 -04:00
Sebastiaan van Stijn
2b9d03f936 vendor: golang.org/x/text v0.13.0
full diff: https://github.com/golang/text/compare/v0.11.0...v0.13.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 0679ae984b)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-10-20 15:35:08 -04:00
Sebastiaan van Stijn
b2886424f4 vendor: golang.org/x/sys v0.13.0
full diff: https://github.com/golang/sys/compare/v0.10.0...v0.13.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit bc33c27796)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-10-20 15:34:16 -04:00
Sebastiaan van Stijn
6af8c5ba8b vendor: golang.org/x/net v0.13.0, golang.org/x/crypto v0.11.0
full diff: https://github.com/golang/net/compare/v0.10.0...v0.13.0
full diff: https://github.com/golang/crypto/compare/v0.9.0...v0.11.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 2edb858b18)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-10-20 15:33:12 -04:00
Sebastiaan van Stijn
93718ea202 vendor: golang.org/x/text v0.11.0
full diff: https://github.com/golang/text/compare/v0.9.0...v0.11.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 87de23a6cd)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-10-20 15:30:59 -04:00
Sebastiaan van Stijn
8bb9439e24 vendor: golang.org/x/crypto v0.9.0
full diff: https://github.com/golang/crypto/compare/v0.3.0...v0.9.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 9670db3904)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-10-20 15:29:47 -04:00
Sebastiaan van Stijn
b7a5724f28 vendor: golang.org/x/sys v0.10.0
full diff: https://github.com/golang/sys/compare/v0.8.0...v0.10.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit a046616249)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-10-20 15:28:58 -04:00
Sebastiaan van Stijn
ec740c4780 vendor: golang.org/x/crypto v0.3.0
full diff: https://github.com/golang/crypto/compare/v0.2.0...v0.3.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 7b66ae1531)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-10-20 15:27:17 -04:00
Cory Snider
48e5890ead vendor: golang.org/x/net v0.10.0, x/sys v0.8.0, x/text v0.9.0
full diffs:
https://github.com/golang/net/compare/v0.8.0...v0.10.0
https://github.com/golang/sys/compare/v0.6.0...v0.8.0
https://github.com/golang/text/compare/v0.8.0...v0.9.0

(cherry picked from commit 8c4dfc9e6a)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-10-20 15:24:22 -04:00
Sebastiaan van Stijn
d73e66ba3b
Merge pull request #46675 from moby/backport/46667/23.0
[23.0 backport] Add IP_NF_MANGLE to check-config.sh
2023-10-19 15:18:27 +02:00
Stephan Henningsen
d5c4a39259
Update check-config.sh
Add IP_NF_MANGLE to "Generally Required" kernel features, since it appears to be necessary for Docker Swarm to work.

Closes https://github.com/moby/moby/issues/46636

Signed-off-by: Stephan Henningsen <stephan-henningsen@users.noreply.github.com>
(cherry picked from commit cf9073397c)
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-10-18 15:59:24 -06:00
Sebastiaan van Stijn
826fc94ddd
Merge pull request #46625 from thaJeztah/23.0_update_go1.20.10
[23.0] update to go1.20.10
2023-10-12 10:59:55 +02:00
Sebastiaan van Stijn
af23b5ad8e
Merge pull request #46627 from AkihiroSuda/cherrypick-46564-23
[23.0 backport] Limit OOMScoreAdj when running in UserNS ("Rootful-in-Rootless")
2023-10-12 08:52:48 +02:00
Akihiro Suda
91227e3073
Limit OOMScoreAdj when running in UserNS ("Rootful-in-Rootless")
Fix issue 46563 "Rootful-in-Rootless dind doesn't work since systemd v250 (due to oom score adj)"

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit ad877271f3)
> Conflicts:
>	daemon/oci_linux.go
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2023-10-12 07:41:39 +09:00
Brian Goff
6b3cb4aaed
Merge pull request #46617 from xinfengliu/23.0_backport_improve_stats_collector
[23.0 backport] Make one-shot stats faster
2023-10-11 13:07:50 -07:00
Sebastiaan van Stijn
6c829a38e4
update to go1.20.10
go1.20.10 (released 2023-10-10) includes a security fix to the net/http package.
See the Go 1.20.10 milestone on our issue tracker for details:

https://github.com/golang/go/issues?q=milestone%3AGo1.20.10+label%3ACherryPickApproved

full diff: https://github.com/golang/go/compare/go1.20.9...go1.20.10

From the security mailing:

[security] Go 1.21.3 and Go 1.20.10 are released

Hello gophers,

We have just released Go versions 1.21.3 and 1.20.10, minor point releases.

These minor releases include 1 security fixes following the security policy:

- net/http: rapid stream resets can cause excessive work

  A malicious HTTP/2 client which rapidly creates requests and
  immediately resets them can cause excessive server resource consumption.
  While the total number of requests is bounded to the
  http2.Server.MaxConcurrentStreams setting, resetting an in-progress
  request allows the attacker to create a new request while the existing
  one is still executing.

  HTTP/2 servers now bound the number of simultaneously executing
  handler goroutines to the stream concurrency limit. New requests
  arriving when at the limit (which can only happen after the client
  has reset an existing, in-flight request) will be queued until a
  handler exits. If the request queue grows too large, the server
  will terminate the connection.

  This issue is also fixed in golang.org/x/net/http2 v0.17.0,
  for users manually configuring HTTP/2.

  The default stream concurrency limit is 250 streams (requests)
  per HTTP/2 connection. This value may be adjusted using the
  golang.org/x/net/http2 package; see the Server.MaxConcurrentStreams
  setting and the ConfigureServer function.

  This is CVE-2023-39325 and Go issue https://go.dev/issue/63417.
  This is also tracked by CVE-2023-44487.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-11 19:56:52 +02:00
Sebastiaan van Stijn
a577a8f29a
update to go1.20.9
go1.20.9 (released 2023-10-05) includes one security fixes to the cmd/go package,
as well as bug fixes to the go command and the linker. See the Go 1.20.9
milestone on our issue tracker for details:

https://github.com/golang/go/issues?q=milestone%3AGo1.20.9+label%3ACherryPickApproved

full diff: https://github.com/golang/go/compare/go1.20.8...go1.20.9

From the security mailing:

[security] Go 1.21.2 and Go 1.20.9 are released

Hello gophers,

We have just released Go versions 1.21.2 and 1.20.9, minor point releases.

These minor releases include 1 security fixes following the security policy:

- cmd/go: line directives allows arbitrary execution during build

  "//line" directives can be used to bypass the restrictions on "//go:cgo_"
  directives, allowing blocked linker and compiler flags to be passed during
  compliation. This can result in unexpected execution of arbitrary code when
  running "go build". The line directive requires the absolute path of the file in
  which the directive lives, which makes exploting this issue significantly more
  complex.

  This is CVE-2023-39323 and Go issue https://go.dev/issue/63211.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-11 19:56:39 +02:00
Brian Goff
a8313441d1 Make one-shot stats faster
(cherry picked from commit f6fa56194f)
Signed-off-by: Xinfeng Liu <XinfengLiu@icloud.com>
2023-10-11 16:05:00 +08:00
Bjorn Neergaard
dda228b984
profiles/apparmor: deny /sys/devices/virtual/powercap
While this is not strictly necessary as the default OCI config masks this
path, it is possible that the user disabled path masking, passed their
own list, or is using a forked (or future) daemon version that has a
modified default config/allows changing the default config.

Add some defense-in-depth by also masking out this problematic hardware
device with the AppArmor LSM.

Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
(cherry picked from commit bddd826d7a)
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-09-18 16:43:50 -06:00
Bjorn Neergaard
177ca58909
oci/defaults: deny /sys/devices/virtual/powercap
The ability to read these files may offer a power-based sidechannel
attack against any workloads running on the same kernel.

This was originally [CVE-2020-8694][1], which was fixed in
[949dd0104c496fa7c14991a23c03c62e44637e71][2] by restricting read access
to root. However, since many containers run as root, this is not
sufficient for our use case.

While untrusted code should ideally never be run, we can add some
defense in depth here by masking out the device class by default.

[Other mechanisms][3] to access this hardware exist, but they should not
be accessible to a container due to other safeguards in the
kernel/container stack (e.g. capabilities, perf paranoia).

[1]: https://nvd.nist.gov/vuln/detail/CVE-2020-8694
[2]: 949dd0104c
[3]: https://web.eece.maine.edu/~vweaver/projects/rapl/

Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
(cherry picked from commit 83cac3c3e3)
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-09-18 16:43:49 -06:00
Sebastiaan van Stijn
2ebfb1d7a0
Merge pull request #46460 from thaJeztah/23.0_backport_update_golang_1.20.8
[23.0 backport] update to go1.20.8
2023-09-12 14:39:10 +02:00
Sebastiaan van Stijn
0dae84ca64
update to go1.20.8
go1.20.8 (released 2023-09-06) includes two security fixes to the html/template
package, as well as bug fixes to the compiler, the go command, the runtime,
and the crypto/tls, go/types, net/http, and path/filepath packages. See the
Go 1.20.8 milestone on our issue tracker for details:

https://github.com/golang/go/issues?q=milestone%3AGo1.20.8+label%3ACherryPickApproved

full diff: https://github.com/golang/go/compare/go1.20.7...go1.20.8

From the security mailing:

[security] Go 1.21.1 and Go 1.20.8 are released

Hello gophers,

We have just released Go versions 1.21.1 and 1.20.8, minor point releases.

These minor releases include 4 security fixes following the security policy:

- cmd/go: go.mod toolchain directive allows arbitrary execution
  The go.mod toolchain directive, introduced in Go 1.21, could be leveraged to
  execute scripts and binaries relative to the root of the module when the "go"
  command was executed within the module. This applies to modules downloaded using
  the "go" command from the module proxy, as well as modules downloaded directly
  using VCS software.

  Thanks to Juho Nurminen of Mattermost for reporting this issue.

  This is CVE-2023-39320 and Go issue https://go.dev/issue/62198.

- html/template: improper handling of HTML-like comments within script contexts
  The html/template package did not properly handle HMTL-like "<!--" and "-->"
  comment tokens, nor hashbang "#!" comment tokens, in <script> contexts. This may
  cause the template parser to improperly interpret the contents of <script>
  contexts, causing actions to be improperly escaped. This could be leveraged to
  perform an XSS attack.

  Thanks to Takeshi Kaneko (GMO Cybersecurity by Ierae, Inc.) for reporting this
  issue.

  This is CVE-2023-39318 and Go issue https://go.dev/issue/62196.

- html/template: improper handling of special tags within script contexts
  The html/template package did not apply the proper rules for handling occurrences
  of "<script", "<!--", and "</script" within JS literals in <script> contexts.
  This may cause the template parser to improperly consider script contexts to be
  terminated early, causing actions to be improperly escaped. This could be
  leveraged to perform an XSS attack.

  Thanks to Takeshi Kaneko (GMO Cybersecurity by Ierae, Inc.) for reporting this
  issue.

  This is CVE-2023-39319 and Go issue https://go.dev/issue/62197.

- crypto/tls: panic when processing post-handshake message on QUIC connections
  Processing an incomplete post-handshake message for a QUIC connection caused a panic.

  Thanks to Marten Seemann for reporting this issue.

  This is CVE-2023-39321 and CVE-2023-39322 and Go issue https://go.dev/issue/62266.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit c41121cc48)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-12 11:23:47 +02:00
Sebastiaan van Stijn
606a7a7694
Merge pull request #46408 from ameyag/23.0-uname-backslash
[23.0 backport] Fixing dockerd-rootless-setuptools.sh when user name contains a backslash
2023-09-06 08:38:37 +02:00
Jean-Michel Rouet
6af40931b7
more robust dockerd-rootless-setuptools.sh
Fixing case where username may contain a backslash.
This case can happen for winbind/samba active directory domain users.

Signed-off-by: Jean-Michel Rouet <jean-michel.rouet@philips.com>

Use more meaningful variable name

Signed-off-by: Jean-Michel Rouet <jean-michel.rouet@philips.com>

Update contrib/dockerd-rootless-setuptool.sh

Co-authored-by: Akihiro Suda <suda.kyoto@gmail.com>
Signed-off-by: Jean-Michel Rouet <jean-michel.rouet@philips.com>

Use more meaningful variable name

Signed-off-by: Jean-Michel Rouet <jean-michel.rouet@philips.com>

Update contrib/dockerd-rootless-setuptool.sh

Co-authored-by: Akihiro Suda <suda.kyoto@gmail.com>
Signed-off-by: Jean-Michel Rouet <jean-michel.rouet@philips.com>
(cherry picked from commit 2f0ba0a7e5)
Signed-off-by: Ameya Gawde <agawde@mirantis.com>
2023-09-05 12:19:57 -07:00
Sebastiaan van Stijn
66b3a81031
Merge pull request #46332 from thaJeztah/23.0_backport_update_golangci_lint
[23.0 backport] update golangci-lint to v1.54.2
2023-08-31 00:23:28 +02:00
Cory Snider
d6cc8d1d28
Merge pull request #46216 from thaJeztah/23.0_backport_windows_fix_service_register
[23.0 backport] windows: fix --register-service when executed from within binary directory
2023-08-30 17:43:33 -04:00
Sebastiaan van Stijn
3583213144
Merge pull request #46367 from thaJeztah/23.0_backport_volume-local-restore-mounted-status
[23.0 backport] volume/local: Don't unmount, restore mounted status
2023-08-30 00:36:46 +02:00
Sebastiaan van Stijn
436bcf7b33
Merge pull request #46222 from thaJeztah/23.0_backport_capabilites
[23.0 backport] Do not drop effective&permitted set
2023-08-29 23:23:26 +02:00
Paweł Gronowski
8dac634ebd
volume/local: Don't unmount, restore mounted status
On startup all local volumes were unmounted as a cleanup mechanism for
the non-clean exit of the last engine process.

This caused live-restored volumes that used special volume opt mount
flags to be broken. While the refcount was restored, the _data directory
was just unmounted, so all new containers mounting this volume would
just have the access to the empty _data directory instead of the real
volume.

With this patch, the mountpoint isn't unmounted. Instead, if the volume
is already mounted, just mark it as mounted, so the next time Mount is
called only the ref count is incremented, but no second attempt to mount
it is performed.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 2689484402)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-29 22:55:59 +02:00
Paweł Gronowski
ecdca2b06f
integration/liveRestore: Check volume content
Make sure that the content in the live-restored volume mounted in a new
container is the same as the content in the old container.
This checks if volume's _data directory doesn't get unmounted on
startup.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit aef703fa1b)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-29 22:55:56 +02:00
Sebastiaan van Stijn
2db2638a96
Merge pull request #46288 from thaJeztah/23.0_backport_bump_containerd_1.6.22
[23.0 backport] vendor: github.com/containerd/containerd v1.6.22
2023-08-29 21:51:03 +02:00
Sebastiaan van Stijn
af3453493b
update golangci-lint to v1.54.2
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit cd49f9affd)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-25 16:25:31 +02:00
Sebastiaan van Stijn
290158c7bc
Merge pull request #46326 from thaJeztah/23.0_backport_hack_less_redirects
[23.0 backport] Dockerfile: Windows: update Golang download domains to cut down redirects
2023-08-25 14:49:00 +02:00
Sebastiaan van Stijn
f292559663
Dockerfile: Windows: update Golang download domains to cut down redirects
The `golang.org` domain moved to `go.dev`, and the download-URLs we were
using resulted in 2 redirects;

    curl -sI https://golang.org/dl/go1.20.windows-amd64.zip | grep 'location'
    location: https://go.dev/dl/go1.20.windows-amd64.zip

    curl -sI https://go.dev/dl/go1.20.windows-amd64.zip | grep 'location'
    location: https://dl.google.com/go/go1.20.windows-amd64.zip

    curl -sI https://dl.google.com/go/go1.20.windows-amd64.zip
    HTTP/2 200
    # ...

This patch cuts it down to one redirects. I decided not to use the "final"
(`dl.google.com`) URL, because that URL is not documented in the Golang docs,
and visiting the domain itself (https://dl.google.com/) redirects to a marketing
page for "Google Chrome".

Trying the `/go/` path (https://dl.google.com/go/) also does not show a landing
page that lists downloads, so I'm considering those URLs to be "unstable".

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit f6a5318f94)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-25 13:32:51 +02:00
Sebastiaan van Stijn
6cf9afe53c
hack: update link to GOPATH documentation
This documentation moved to a different page, and the Go documentation
moved to the https://go.dev/ domain.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 2aabd64477)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-25 13:32:47 +02:00
Sebastiaan van Stijn
aa95a3354d
Merge pull request #46287 from thaJeztah/23.0_backport_gha_report_timeout
[23.0 backport] gha: set 10-minute timeout on "report" actions
2023-08-22 00:03:11 +02:00
Sebastiaan van Stijn
daa07e6995
vendor: github.com/containerd/containerd v1.6.22
- full diff: https://github.com/containerd/containerd/compare/v1.6.21...v1.6.22
- release notes: https://github.com/containerd/containerd/releases/tag/v1.6.22

---

Notable Updates

- RunC: Update runc binary to v1.1.8
- CRI: Fix `additionalGids`: it should fallback to `imageConfig.User`
  when `securityContext.RunAsUser`, `RunAsUsername` are empty
- CRI: Write generated CNI config atomically
- Fix concurrent writes for `UpdateContainerStats`
- Make `checkContainerTimestamps` less strict on Windows
- Port-Forward: Correctly handle known errors
- Resolve `docker.NewResolver` race condition
- SecComp: Always allow `name_to_handle_at`
- Adding support to run hcsshim from local clone
- Pinned image support
- Runtime/V2/RunC: Handle early exits w/o big locks
- CRITool: Move up to CRI-TOOLS v1.27.0
- Fix cpu architecture detection issue on emulated ARM platform
- Task: Don't `close()` io before `cancel()`
- Fix panic when remote differ returns empty result
- Plugins: Notify readiness when registered plugins are ready
- Unwrap io errors in server connection receive error handling

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 4d674897f3)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-21 20:47:39 +02:00
Sebastiaan van Stijn
278d6d1e5d
vendor: google.golang.org/grpc v1.50.1
full diff: https://github.com/grpc/grpc-go/compare/v1.48.0...v1.50.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit c1a9e82d3e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-21 20:47:39 +02:00
Sebastiaan van Stijn
930b2a41ba
vendor: golang.org/x/net v0.8.0
full diff: https://github.com/golang/net/compare/v0.7.0...v0.8.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit f531e3cf69)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-21 20:47:38 +02:00
Sebastiaan van Stijn
de4064871f
vendor: golang.org/x/text v0.8.0
full diff: https://github.com/golang/text/compare/v0.7.0...v0.8.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 3886faced8)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-21 20:47:38 +02:00
Sebastiaan van Stijn
a6e77bda56
vendor: golang.org/x/sys v0.6.0
full diff: https://github.com/golang/sys/compare/v0.5.0...v0.6.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 9752e43644)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-21 20:47:38 +02:00
Sebastiaan van Stijn
a93e38a933
vendor: github.com/containerd/ttrpc v1.1.2
full diff: https://github.com/containerd/ttrpc/compare/v1.1.1...v1.1.2

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-21 20:47:31 +02:00
Sebastiaan van Stijn
6b1c77a975
gha: set 10-minute timeout on "report" actions
I had a CI run fail to "Upload reports":

    Exponential backoff for retry #1. Waiting for 4565 milliseconds before continuing the upload at offset 0
    Finished backoff for retry #1, continuing with upload
    Total file count: 211 ---- Processed file #160 (75.8%)
    ...
    Total file count: 211 ---- Processed file #164 (77.7%)
    Total file count: 211 ---- Processed file #164 (77.7%)
    Total file count: 211 ---- Processed file #164 (77.7%)
    A 503 status code has been received, will attempt to retry the upload
    ##### Begin Diagnostic HTTP information #####
    Status Code: 503
    Status Message: Service Unavailable
    Header Information: {
      "content-length": "592",
      "content-type": "application/json; charset=utf-8",
      "date": "Mon, 21 Aug 2023 14:08:10 GMT",
      "server": "Kestrel",
      "cache-control": "no-store,no-cache",
      "pragma": "no-cache",
      "strict-transport-security": "max-age=2592000",
      "x-tfs-processid": "b2fc902c-011a-48be-858d-c62e9c397cb6",
      "activityid": "49a48b53-0411-4ff3-86a7-4528e3f71ba2",
      "x-tfs-session": "49a48b53-0411-4ff3-86a7-4528e3f71ba2",
      "x-vss-e2eid": "49a48b53-0411-4ff3-86a7-4528e3f71ba2",
      "x-vss-senderdeploymentid": "63be6134-28d1-8c82-e969-91f4e88fcdec",
      "x-frame-options": "SAMEORIGIN"
    }
    ###### End Diagnostic HTTP information ######
    Retry limit has been reached for chunk at offset 0 to https://pipelinesghubeus5.actions.githubusercontent.com/Y2huPMnV2RyiTvKoReSyXTCrcRyxUdSDRZYoZr0ONBvpl5e9Nu/_apis/resources/Containers/8331549?itemPath=integration-reports%2Fubuntu-22.04-systemd%2Fbundles%2Ftest-integration%2FTestInfoRegistryMirrors%2Fd20ac12e48cea%2Fdocker.log
    Warning: Aborting upload for /tmp/reports/ubuntu-22.04-systemd/bundles/test-integration/TestInfoRegistryMirrors/d20ac12e48cea/docker.log due to failure
    Error: aborting artifact upload
    Total file count: 211 ---- Processed file #165 (78.1%)
    A 503 status code has been received, will attempt to retry the upload
    Exponential backoff for retry #1. Waiting for 5799 milliseconds before continuing the upload at offset 0

As a result, the "Download reports" continued retrying:

    ...
    Total file count: 1004 ---- Processed file #436 (43.4%)
    Total file count: 1004 ---- Processed file #436 (43.4%)
    Total file count: 1004 ---- Processed file #436 (43.4%)
    An error occurred while attempting to download a file
    Error: Request timeout: /Y2huPMnV2RyiTvKoReSyXTCrcRyxUdSDRZYoZr0ONBvpl5e9Nu/_apis/resources/Containers/8331549?itemPath=integration-reports%2Fubuntu-20.04%2Fbundles%2Ftest-integration%2FTestCreateWithDuplicateNetworkNames%2Fd47798cc212d1%2Fdocker.log
        at ClientRequest.<anonymous> (/home/runner/work/_actions/actions/download-artifact/v3/dist/index.js:3681:26)
        at Object.onceWrapper (node:events:627:28)
        at ClientRequest.emit (node:events:513:28)
        at TLSSocket.emitRequestTimeout (node:_http_client:839:9)
        at Object.onceWrapper (node:events:627:28)
        at TLSSocket.emit (node:events:525:35)
        at TLSSocket.Socket._onTimeout (node:net:550:8)
        at listOnTimeout (node:internal/timers:559:17)
        at processTimers (node:internal/timers:502:7)
    Exponential backoff for retry #1. Waiting for 5305 milliseconds before continuing the download
    Total file count: 1004 ---- Processed file #436 (43.4%)

And, it looks like GitHub doesn't allow cancelling the job, possibly
because it is defined with `if: always()`?

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit d6f340e784)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-21 20:21:35 +02:00
Sebastiaan van Stijn
c0ccb0010b
Merge pull request #46232 from thaJeztah/23.0_backport_remove-ibm-jenkins-jobs
[23.0 backport] Remove s390x and ppc64le Jenkins pipelines
2023-08-16 17:53:15 +02:00
Sebastiaan van Stijn
e62c37867d
Merge pull request #46229 from thaJeztah/23.0_backport_runc_binary_1.1.9
[23.0 backport] update runc binary to v1.1.9
2023-08-16 02:07:12 +02:00
Sam Thibault
55ba1841f7
remove s390x and ppc64ls pipelines
Signed-off-by: Sam Thibault <sam.thibault@docker.com>
(cherry picked from commit 59aa3dce8a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-14 18:52:22 +02:00
CrazyMax
abc61ff43b
Makefile: remove unused BUILD_APT_MIRROR
BUILD_APT_MIRROR added in https://github.com/moby/moby/pull/26375
is not used anymore.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 7c697f58f2)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-14 18:50:34 +02:00
CrazyMax
a521843442
Dockerfile: use default apt mirrors
Use default apt mirrors and also check APT_MIRROR
is set before updating mirrors.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit a1d2132bf6)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-14 18:49:22 +02:00
Akihiro Suda
d01253365a
update runc binary to v1.1.9
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit b039bbc678)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-14 17:34:47 +02:00
Luboslav Pivarc
89a731096d
Integration test for capabilities
Verify non-root containers are able to use file
capabilities.

Signed-off-by: Luboslav Pivarc <lpivarc@redhat.com>
Co-authored-by: Cory Snider <csnider@mirantis.com>
Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit 42fa7a1951)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-13 22:46:30 +02:00
Luboslav Pivarc
09b65e0082
Do not drop effective&permitted set
Currently moby drops ep sets before the entrypoint is executed.
This does mean that with combination of no-new-privileges the
file capabilities stops working with non-root containers.
This is undesired as the usability of such containers is harmed
comparing to running root containers.

This commit therefore sets the effective/permitted set in order
to allow use of file capabilities or libcap(3)/prctl(2) respectively
with combination of no-new-privileges and without respectively.

For no-new-privileges the container will be able to obtain capabilities
that are requested.

Signed-off-by: Luboslav Pivarc <lpivarc@redhat.com>
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
(cherry picked from commit 3aef732e61)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-13 22:45:39 +02:00
Cory Snider
5cc1736418
daemon: ensure OCI options play nicely together
Audit the OCI spec options used for Linux containers to ensure they are
less order-dependent. Ensure they don't assume that any pointer fields
are non-nil and that they don't unintentionally clobber mutations to the
spec applied by other options.

Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit 8a094fe609)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-13 22:45:15 +02:00
Cory Snider
4949db7f62
daemon: stop setting container resources to zero
Many of the fields in LinuxResources struct are pointers to scalars for
some reason, presumably to differentiate between set-to-zero and unset
when unmarshaling from JSON, despite zero being outside the acceptable
range for the corresponding kernel tunables. When creating the OCI spec
for a container, the daemon sets the container's OCI spec CPUShares and
BlkioWeight parameters to zero when the corresponding Docker container
configuration values are zero, signifying unset, despite the minimum
acceptable value for CPUShares being two, and BlkioWeight ten. This has
gone unnoticed as runC does not distingiush set-to-zero from unset as it
also uses zero internally to represent unset for those fields. However,
kata-containers v3.2.0-alpha.3 tries to apply the explicit-zero resource
parameters to the container, exactly as instructed, and fails loudly.
The OCI runtime-spec is silent on how the runtime should handle the case
when those parameters are explicitly set to out-of-range values and
kata's behaviour is not unreasonable, so the daemon must therefore be in
the wrong.

Translate unset values in the Docker container's resources HostConfig to
omit the corresponding fields in the container's OCI spec when starting
and updating a container in order to maximize compatibility with
runtimes.

Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit dea870f4ea)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-13 22:44:50 +02:00
Cory Snider
f178c2ab15
daemon: modernize oci_linux_test.go
Switch to using t.TempDir() instead of rolling our own.

Clean up mounts leaked by the tests as otherwise the tests fail due to
the leaked mounts because unlike the old cleanup code, t.TempDir()
cleanup does not ignore errors from os.RemoveAll.

Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit 9ff169ccf4)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-13 22:35:34 +02:00
Sebastiaan van Stijn
bd6e4ae215
windows: fix --register-service when executed from within binary directory
Go 1.15.7 contained a security fix for CVE-2021-3115, which allowed arbitrary
code to be executed at build time when using cgo on Windows.

This issue was not limited to the go command itself, and could also affect binaries
that use `os.Command`, `os.LookPath`, etc.

From the related blogpost (https://blog.golang.org/path-security):

> Are your own programs affected?
>
> If you use exec.LookPath or exec.Command in your own programs, you only need to
> be concerned if you (or your users) run your program in a directory with untrusted
> contents. If so, then a subprocess could be started using an executable from dot
> instead of from a system directory. (Again, using an executable from dot happens
> always on Windows and only with uncommon PATH settings on Unix.)
>
> If you are concerned, then we’ve published the more restricted variant of os/exec
> as golang.org/x/sys/execabs. You can use it in your program by simply replacing

At time of the go1.15 release, the Go team considered changing the behavior of
`os.LookPath()` and `exec.LookPath()` to be a breaking change, and made the
behavior "opt-in" by providing the `golang.org/x/sys/execabs` package as a
replacement.

However, for the go1.19 release, this changed, and the default behavior of
`os.LookPath()` and `exec.LookPath()` was changed. From the release notes:
https://go.dev/doc/go1.19#os-exec-path

> Command and LookPath no longer allow results from a PATH search to be found
> relative to the current directory. This removes a common source of security
> problems but may also break existing programs that depend on using, say,
> exec.Command("prog") to run a binary named prog (or, on Windows, prog.exe)
> in the current directory. See the os/exec package documentation for information
> about how best to update such programs.
>
> On Windows, Command and LookPath now respect the NoDefaultCurrentDirectoryInExePath
> environment variable, making it possible to disable the default implicit search
> of “.” in PATH lookups on Windows systems.

A result of this change was that registering the daemon as a Windows service
no longer worked when done from within the directory of the binary itself:

    C:\> cd "Program Files\Docker\Docker\resources"
    C:\Program Files\Docker\Docker\resources> dockerd --register-service
    exec: "dockerd": cannot run executable found relative to current directory

Note that using an absolute path would work around the issue:

    C:\Program Files\Docker\Docker>resources\dockerd.exe --register-service

This patch changes `registerService()` to use `os.Executable()`, instead of
depending on `os.Args[0]` and `exec.LookPath()` for resolving the absolute
path of the binary.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 3e8fda0a70)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-13 21:58:49 +02:00
Sebastiaan van Stijn
2f917915c6
Merge pull request #46178 from vvoland/hack-integrationcli-dont-failfast-23
[23.0 backport] hack/test: Don't fail-fast before integration-cli
2023-08-09 15:39:10 +02:00
Paweł Gronowski
d8bde85290
hack/test: Don't fail-fast before integration-cli
If TEST_INTEGRATION_FAIL_FAST is not set, run the integration-cli tests
even if integration tests failed.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 6841a53d17)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-08-09 14:24:04 +02:00
Sebastiaan van Stijn
ed8f0053e8
Merge pull request #46135 from thaJeztah/23.0_backport_fix_daemon_integration_test
[23.0 backport] Improve test daemon logging, and fix TestDaemonProxy integration tests
2023-08-04 18:06:04 +02:00
Akihiro Suda
87655e821b
Merge pull request #46141 from thaJeztah/23.0_backport_update_go1.20.7
[23.0 backport] update to go1.20.7
2023-08-02 14:50:01 +09:00
Sebastiaan van Stijn
3435febfc8
update to go1.20.7
Includes a fix for CVE-2023-29409

go1.20.7 (released 2023-08-01) includes a security fix to the crypto/tls
package, as well as bug fixes to the assembler and the compiler. See the
Go 1.20.7 milestone on our issue tracker for details:

- https://github.com/golang/go/issues?q=milestone%3AGo1.20.7+label%3ACherryPickApproved
- full diff: https://github.com/golang/go/compare/go1.20.6...go1.20.7

From the mailing list announcement:

[security] Go 1.20.7 and Go 1.19.12 are released

Hello gophers,

We have just released Go versions 1.20.7 and 1.19.12, minor point releases.

These minor releases include 1 security fixes following the security policy:

- crypto/tls: restrict RSA keys in certificates to <= 8192 bits

  Extremely large RSA keys in certificate chains can cause a client/server
  to expend significant CPU time verifying signatures. Limit this by
  restricting the size of RSA keys transmitted during handshakes to <=
  8192 bits.

  Based on a survey of publicly trusted RSA keys, there are currently only
  three certificates in circulation with keys larger than this, and all
  three appear to be test certificates that are not actively deployed. It
  is possible there are larger keys in use in private PKIs, but we target
  the web PKI, so causing breakage here in the interests of increasing the
  default safety of users of crypto/tls seems reasonable.

  Thanks to Mateusz Poliwczak for reporting this issue.

View the release notes for more information:
https://go.dev/doc/devel/release#go1.20.7

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit d5cb7cdeae)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-01 23:48:47 +02:00
Sebastiaan van Stijn
f11de4e4da
Merge pull request #46121 from thaJeztah/23.0_backport_bump_gotest_tools
[23.0 backport] vendor: gotest.tools/v3 v3.5.0
2023-08-01 16:27:22 +02:00
Brian Goff
9473c725a9
TestDaemonProxy: use new scanners to check logs
Also fixes up some cleanup issues.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 1a51898d2e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-01 13:24:48 +02:00
Brian Goff
063c7206b6
Fix daemon proxy test for "reload sanitized"
I noticed this was always being skipped because of race conditions
checking the logs.

This change adds a log scanner which will look through the logs line by
line rather than allocating a big buffer.
Additionally it adds a `poll.Check` which we can use to actually wait
for the desired log entry.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 476e788090)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-01 13:24:47 +02:00
Brian Goff
2434546510
TestDaemonProxy: check proxy settings early
Allows tests to report their proxy settings for easier troubleshooting
on failures.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit 8197752d68)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-01 13:24:47 +02:00
Brian Goff
b97c8a0325
Improve test daemon logging
1. On failed start tail the daemon logs
2. Exposes generic tailing functions to make test debugging simpler

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit 914888cf8b)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-01 13:24:45 +02:00
Akihiro Suda
801e90549a
Merge pull request #46104 from thaJeztah/23.0_update_containerd_binary_1.6.22
[23.0] update containerd binary to v1.6.22
2023-07-30 11:05:54 +09:00
Sebastiaan van Stijn
a8e8a4d2a0
vendor: gotest.tools/v3 v3.5.0
- go.mod: update dependencies and go version by
- Use Go1.20
- Fix couple of typos
- Added `WithStdout` and `WithStderr` helpers
- Moved `cmdOperators` handling from `RunCmd` to `StartCmd`
- Deprecate `assert.ErrorType`
- Remove outdated Dockerfile
- add godoc links

full diff: https://github.com/gotestyourself/gotest.tools/compare/v3.4.0...v3.5.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit ce053a14aa)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-29 20:56:57 +02:00
Sebastiaan van Stijn
83bf8954b7
update containerd binary to v1.6.22
- full diff: https://github.com/containerd/containerd/compare/v1.6.21...v1.6.22
- release notes: https://github.com/containerd/containerd/releases/tag/v1.6.22

---

Notable Updates

- RunC: Update runc binary to v1.1.8
- CRI: Fix `additionalGids`: it should fallback to `imageConfig.User`
  when `securityContext.RunAsUser`, `RunAsUsername` are empty
- CRI: Write generated CNI config atomically
- Fix concurrent writes for `UpdateContainerStats`
- Make `checkContainerTimestamps` less strict on Windows
- Port-Forward: Correctly handle known errors
- Resolve `docker.NewResolver` race condition
- SecComp: Always allow `name_to_handle_at`
- Adding support to run hcsshim from local clone
- Pinned image support
- Runtime/V2/RunC: Handle early exits w/o big locks
- CRITool: Move up to CRI-TOOLS v1.27.0
- Fix cpu architecture detection issue on emulated ARM platform
- Task: Don't `close()` io before `cancel()`
- Fix panic when remote differ returns empty result
- Plugins: Notify readiness when registered plugins are ready
- Unwrap io errors in server connection receive error handling

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-28 13:30:38 +02:00
Sebastiaan van Stijn
c4c2450972
Merge pull request #46062 from vvoland/integration-no-fail-fast-23
[23.0 backport] hack/integration: Add TEST_INTEGRATION_FAIL_FAST
2023-07-24 20:55:22 +02:00
Paweł Gronowski
f7782d27cc
hack/integration: Add TEST_INTEGRATION_FAIL_FAST
Before this change, integration test would fail fast and not execute all
test suites when one suite fails.
Change this behavior into opt-in enabled by TEST_INTEGRATION_FAIL_FAST
variable.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 48cc28e4ef)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-07-24 17:09:25 +02:00
Sebastiaan van Stijn
531e8a7b00
Merge pull request #46046 from neersighted/init_cleanup_23.0
[23.0 backport] Upstart & sysvinit cleanup
2023-07-21 22:08:16 +02:00
Tianon Gravi
f740890881
Remove Upstart scripts
Upstart has been EOL for 8 years and isn't used by any distributions we support any more.

Signed-off-by: Tianon Gravi <admwiggin@gmail.com>
(cherry picked from commit 0d8087fbbc)
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-07-21 12:00:40 -06:00
Tianon Gravi
0c01c0d57c
Remove Upstart and cgroups bits from Debian sysvinit script
Upstart has been EOL for 8 years and isn't used by any distributions we support any more.

Additionally, this removes the "cgroups v1" setup code because it's more reasonable now for us to expect something _else_ to have set up cgroups appropriately (especially cgroups v2).

Signed-off-by: Tianon Gravi <admwiggin@gmail.com>
(cherry picked from commit ae737656f9)
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-07-21 12:00:28 -06:00
Sebastiaan van Stijn
0cae31c7dd
Merge pull request #46030 from thaJeztah/23.0_backport_bump_runc_binary_1.1.8
[23.0 backport] Dockerfile: update runc binary to v1.1.8
2023-07-20 07:00:51 +02:00
Sebastiaan van Stijn
30da8e8b5f
Dockerfile: update runc binary to v1.1.8
release notes: https://github.com/opencontainers/runc/releases/tag/v1.1.8
full diff: https://github.com/opencontainers/runc/compare/v1.1.7...v1.1.9

This is the eighth patch release of the 1.1.z release branch of runc.
The most notable change is the addition of RISC-V support, along with a
few bug fixes.

- Support riscv64.
- init: do not print environment variable value.
- libct: fix a race with systemd removal.
- tests/int: increase num retries for oom tests.
- man/runc: fixes.
- Fix tmpfs mode opts when dir already exists.
- docs/systemd: fix a broken link.
- ci/cirrus: enable some rootless tests on cs9.
- runc delete: call systemd's reset-failed.
- libct/cg/sd/v1: do not update non-frozen cgroup after frozen failed.
- CI: bump Fedora, Vagrant, bats.
- .codespellrc: update for 2.2.5.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit df86d855f5)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-19 18:24:07 +02:00
Bjorn Neergaard
41712da47b
Merge pull request #46011 from thaJeztah/23.0_backport_buildkit_ref_replace
[23.0 backport] ci: extract buildkit version correctly with replace-d modules
2023-07-18 13:14:01 -06:00
Bjorn Neergaard
49af6c96d5
ci(buildkit): remove early-return from buildkit-ref
This doesn't really make sense now that this script returns a
$GITHUB_ENV snippet.

Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
(cherry picked from commit 7310a7cd0c)
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-07-18 12:11:40 -06:00
Bjorn Neergaard
84cb1c6705
ci(buildkit): remove misleading code from buildkit-ref
Post-f8c0d92a22bad004cb9cbb4db704495527521c42, BUILDKIT_REPO doesn't
really do what it claims to. Instead, don't allow overloading since the
import path for BuildKit is always the same, and make clear the
provenance of values when generating the final variable definitions.

We also better document the script, and follow some best practices for
both POSIX sh and Bash.

Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
(cherry picked from commit 4ecc01f3ad)
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-07-18 12:11:39 -06:00
Bjorn Neergaard
091ceed44f
hack/with-go-mod.sh: warn on stderr
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
(cherry picked from commit 48ff8a95cc)
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-07-18 12:11:38 -06:00
Justin Chadwell
e995191780
ci: extract buildkit version correctly with replace-d modules
Signed-off-by: Justin Chadwell <me@jedevc.com>
(cherry picked from commit f8c0d92a22)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-18 15:24:22 +02:00
Sebastiaan van Stijn
f860ed7c77
Merge pull request #46002 from thaJeztah/23.0_backport_notestyourself
[23.0 backport] quota: remove gotest.tools from testhelpers
2023-07-18 10:24:41 +02:00
Sebastiaan van Stijn
9f8a92473b
quota: remove gotest.tools from testhelpers
gotest.tools has an init() which registers a '-update' flag;
a80f057529/internal/source/update.go (L21-L23)

The quota helper contains a testhelpers file, which is meant for usage
in (integration) tests, but as it's in the same pacakge as production
code, would also trigger the gotest.tools init.

This patch removes the gotest.tools code from this file.

Before this patch:

    $ (exec -a libnetwork-setkey "$(which dockerd)" -help)
    Usage of libnetwork-setkey:
      -exec-root string
            docker exec root (default "/run/docker")
      -update
            update golden values

With this patch applied:

    $ (exec -a libnetwork-setkey "$(which dockerd)" -help)
    Usage of libnetwork-setkey:
      -exec-root string
            docker exec root (default "/run/docker")

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 1aa17222e7)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-17 23:13:40 +02:00
Sebastiaan van Stijn
0420d2b33c
Merge pull request #45977 from thaJeztah/23.0_backport_update_go_1.20.6
[23.0 backport] update go to go1.20.6
2023-07-15 15:46:20 +02:00
Sebastiaan van Stijn
faab6db496
update go to go1.20.6
go1.20.6 (released 2023-07-11) includes a security fix to the net/http package,
as well as bug fixes to the compiler, cgo, the cover tool, the go command,
the runtime, and the crypto/ecdsa, go/build, go/printer, net/mail, and text/template
packages. See the Go 1.20.6 milestone on our issue tracker for details.

https://github.com/golang/go/issues?q=milestone%3AGo1.20.6+label%3ACherryPickApproved

Full diff: https://github.com/golang/go/compare/go1.20.5...go1.20.6

These minor releases include 1 security fixes following the security policy:

net/http: insufficient sanitization of Host header

The HTTP/1 client did not fully validate the contents of the Host header.
A maliciously crafted Host header could inject additional headers or entire
requests. The HTTP/1 client now refuses to send requests containing an
invalid Request.Host or Request.URL.Host value.

Thanks to Bartek Nowotarski for reporting this issue.

Includes security fixes for [CVE-2023-29406 ][1] and Go issue https://go.dev/issue/60374

[1]: https://github.com/advisories/GHSA-f8f7-69v5-w4vx

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 1ead2dd35d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-15 02:11:37 +02:00
Sebastiaan van Stijn
f00e7af960
Merge pull request #45971 from thaJeztah/23.0_backport_fix_host_header
[23.0 backport] client: define a "dummy" hostname to use for local connections
2023-07-14 23:58:26 +02:00
Sebastiaan van Stijn
03c5a810ec
testutil: use dummyhost for non-tcp connections
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit e1db9e9848)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-14 22:50:27 +02:00
Sebastiaan van Stijn
5f68c5447d
pkg/plugins: use a dummy hostname for local connections
For local communications (npipe://, unix://), the hostname is not used,
but we need valid and meaningful hostname.

The current code used the socket path as hostname, which gets rejected by
go1.20.6 and go1.19.11 because of a security fix for [CVE-2023-29406 ][1],
which was implemented in  https://go.dev/issue/60374.

Prior versions go Go would clean the host header, and strip slashes in the
process, but go1.20.6 and go1.19.11 no longer do, and reject the host
header.

Before this patch, tests would fail on go1.20.6:

    === FAIL: pkg/authorization TestAuthZRequestPlugin (15.01s)
    time="2023-07-12T12:53:45Z" level=warning msg="Unable to connect to plugin: //tmp/authz2422457390/authz-test-plugin.sock/AuthZPlugin.AuthZReq: Post \"http://%2F%2Ftmp%2Fauthz2422457390%2Fauthz-test-plugin.sock/AuthZPlugin.AuthZReq\": http: invalid Host header, retrying in 1s"
    time="2023-07-12T12:53:46Z" level=warning msg="Unable to connect to plugin: //tmp/authz2422457390/authz-test-plugin.sock/AuthZPlugin.AuthZReq: Post \"http://%2F%2Ftmp%2Fauthz2422457390%2Fauthz-test-plugin.sock/AuthZPlugin.AuthZReq\": http: invalid Host header, retrying in 2s"
    time="2023-07-12T12:53:48Z" level=warning msg="Unable to connect to plugin: //tmp/authz2422457390/authz-test-plugin.sock/AuthZPlugin.AuthZReq: Post \"http://%2F%2Ftmp%2Fauthz2422457390%2Fauthz-test-plugin.sock/AuthZPlugin.AuthZReq\": http: invalid Host header, retrying in 4s"
    time="2023-07-12T12:53:52Z" level=warning msg="Unable to connect to plugin: //tmp/authz2422457390/authz-test-plugin.sock/AuthZPlugin.AuthZReq: Post \"http://%2F%2Ftmp%2Fauthz2422457390%2Fauthz-test-plugin.sock/AuthZPlugin.AuthZReq\": http: invalid Host header, retrying in 8s"
        authz_unix_test.go:82: Failed to authorize request Post "http://%2F%2Ftmp%2Fauthz2422457390%2Fauthz-test-plugin.sock/AuthZPlugin.AuthZReq": http: invalid Host header

[1]: https://github.com/advisories/GHSA-f8f7-69v5-w4vx

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 6b7705d5b2)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-14 22:50:18 +02:00
Sebastiaan van Stijn
5532d516be
client: define a "dummy" hostname to use for local connections
For local communications (npipe://, unix://), the hostname is not used,
but we need valid and meaningful hostname.

The current code used the client's `addr` as hostname in some cases, which
could contain the path for the unix-socket (`/var/run/docker.sock`), which
gets rejected by go1.20.6 and go1.19.11 because of a security fix for
[CVE-2023-29406 ][1], which was implemented in  https://go.dev/issue/60374.

Prior versions go Go would clean the host header, and strip slashes in the
process, but go1.20.6 and go1.19.11 no longer do, and reject the host
header.

This patch introduces a `DummyHost` const, and uses this dummy host for
cases where we don't need an actual hostname.

Before this patch (using go1.20.6):

    make GO_VERSION=1.20.6 TEST_FILTER=TestAttach test-integration
    === RUN   TestAttachWithTTY
        attach_test.go:46: assertion failed: error is not nil: http: invalid Host header
    --- FAIL: TestAttachWithTTY (0.11s)
    === RUN   TestAttachWithoutTTy
        attach_test.go:46: assertion failed: error is not nil: http: invalid Host header
    --- FAIL: TestAttachWithoutTTy (0.02s)
    FAIL

With this patch applied:

    make GO_VERSION=1.20.6 TEST_FILTER=TestAttach test-integration
    INFO: Testing against a local daemon
    === RUN   TestAttachWithTTY
    --- PASS: TestAttachWithTTY (0.12s)
    === RUN   TestAttachWithoutTTy
    --- PASS: TestAttachWithoutTTy (0.02s)
    PASS

[1]: https://github.com/advisories/GHSA-f8f7-69v5-w4vx

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 92975f0c11)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-14 22:49:58 +02:00
Sebastiaan van Stijn
65468a2bef
client: TestSetHostHeader: don't use un-keyed literals
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 2a59188760)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-14 22:48:30 +02:00
Sebastiaan van Stijn
d5591a4e5f
Merge pull request #45960 from vvoland/tests-fix-setuptest-23
[backport 23.0] integration: Don't env cleanup before parallel subtests
2023-07-13 18:20:28 +02:00
Paweł Gronowski
c6031ad519
integration: Don't env cleanup before parallel subtests
Calling function returned from setupTest (which calls testEnv.Clean) in
a defer block inside a test that spawns parallel subtests caused the
cleanup function to be called before any of the subtest did anything.

Change the defer expressions to use `t.Cleanup` instead to call it only
after all subtests have also finished.
This only changes tests which have parallel subtests.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit f9e2eed55d)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-07-13 15:07:08 +02:00
Brian Goff
5f99559987
Merge pull request #45912 from corhere/backport-23.0/fix-volume-npe 2023-07-07 16:31:19 -07:00
Sebastiaan van Stijn
80cb2ed333
Merge pull request #45910 from corhere/backport-23.0/improve-test-flakiness
[23.0 backport] make tests less flaky
2023-07-07 23:48:45 +02:00
Sebastiaan van Stijn
976f20547b daemon: daemon.prepareMountPoints(): fix panic if mount is not a volume
The daemon.lazyInitializeVolume() function only handles restoring Volumes
if a Driver is specified. The Container's MountPoints field may also
contain other kind of mounts (e.g., bind-mounts). Those were ignored, and
don't return an error; 1d9c8619cd/daemon/volumes.go (L243-L252C2)

However, the prepareMountPoints() assumed each MountPoint was a volume,
and logged an informational message about the volume being restored;
1d9c8619cd/daemon/mounts.go (L18-L25)

This would panic if the MountPoint was not a volume;

    github.com/docker/docker/daemon.(*Daemon).prepareMountPoints(0xc00054b7b8?, 0xc0007c2500)
            /root/rpmbuild/BUILD/src/engine/.gopath/src/github.com/docker/docker/daemon/mounts.go:24 +0x1c0
    github.com/docker/docker/daemon.(*Daemon).restore.func5(0xc0007c2500, 0x0?)
            /root/rpmbuild/BUILD/src/engine/.gopath/src/github.com/docker/docker/daemon/daemon.go:552 +0x271
    created by github.com/docker/docker/daemon.(*Daemon).restore
            /root/rpmbuild/BUILD/src/engine/.gopath/src/github.com/docker/docker/daemon/daemon.go:530 +0x8d8
    panic: runtime error: invalid memory address or nil pointer dereference
    [signal SIGSEGV: segmentation violation code=0x1 addr=0x30 pc=0x564e9be4c7c0]

This issue was introduced in 647c2a6cdd

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit a490248f4d)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-07-07 15:59:05 -04:00
Cory Snider
1dedea2fb8 integration: disable iptables in parallel tests
Multiple daemons starting/running concurrently can collide with each
other when editing iptables rules. Most integration tests which opt into
parallelism and start daemons work around this problem by starting the
daemon with the --iptables=false option. However, some of the tests
neglect to pass the option when starting or restarting the daemon,
resulting in those tests being flaky.

Audit the integration tests which call t.Parallel() and (*Daemon).Stop()
and add --iptables=false arguments where needed.

Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit cdcb7c28c5)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-07-07 15:48:05 -04:00
Cory Snider
5fa96398f1 pkg/plugins: make unit test less time sensitive
TestClientWithRequestTimeout has been observed to flake in CI. The
timing in the test is quite tight, only giving the client a 10ms window
to time out, which could potentially be missed if the host is under
load and the goroutine scheduling is unlucky. Give the client a full
five seconds of grace to time out before failing the test.

Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit 9cee34bc94)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-07-07 15:47:01 -04:00
Bjorn Neergaard
e77ff73d68
Merge pull request #45859 from neersighted/backport/45848/23.0
[23.0 backport] pkg/fileutils: GetTotalUsedFds: reduce allocations
2023-07-01 07:39:37 -06:00
Sebastiaan van Stijn
521fbb36c0
pkg/fileutils: GetTotalUsedFds(): use fast-path for Kernel 6.2 and up
Linux 6.2 and up (commit [f1f1f2569901ec5b9d425f2e91c09a0e320768f3][1])
provides a fast path for the number of open files for the process.

From the [Linux docs][2]:

> The number of open files for the process is stored in 'size' member of
> `stat()` output for /proc/<pid>/fd for fast access.

[1]: f1f1f25699
[2]: https://docs.kernel.org/filesystems/proc.html#proc-pid-fd-list-of-symlinks-to-open-files

This patch adds a fast-path for Kernels that support this, and falls back
to the slow path if the Size fields is zero.

Comparing on a Fedora 38 (kernel 6.2.9-300.fc38.x86_64):

Before/After:

    go test -bench ^BenchmarkGetTotalUsedFds$ -run ^$ ./pkg/fileutils/
    BenchmarkGetTotalUsedFds        57264     18595 ns/op     408 B/op      10 allocs/op
    BenchmarkGetTotalUsedFds       370392      3271 ns/op      40 B/op       3 allocs/op

Note that the slow path has 1 more file-descriptor, due to the open
file-handle for /proc/<pid>/fd during the calculation.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit ec79d0fc05)
Resolved conflicts:
	pkg/fileutils/fileutils_linux.go
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-06-30 19:54:13 -06:00
Sebastiaan van Stijn
6c4adc0037
pkg/fileutils: GetTotalUsedFds: reduce allocations
Use File.Readdirnames instead of os.ReadDir, as we're only interested in
the number of files, and results don't have to be sorted.

Before:

    BenchmarkGetTotalUsedFds-5   	  149272	      7896 ns/op	     945 B/op	      20 allocs/op

After:

    BenchmarkGetTotalUsedFds-5   	  153517	      7644 ns/op	     408 B/op	      10 allocs/op

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit eaa9494b71)
Resolved conflicts:
	pkg/fileutils/fileutils_linux.go
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-06-30 19:54:12 -06:00
Sebastiaan van Stijn
55c8d163d7
pkg/fileutils: add BenchmarkGetTotalUsedFds
go test -bench ^BenchmarkGetTotalUsedFds$ -run ^$ ./pkg/fileutils/
    goos: linux
    goarch: arm64
    pkg: github.com/docker/docker/pkg/fileutils
    BenchmarkGetTotalUsedFds-5   	  149272	      7896 ns/op	     945 B/op	      20 allocs/op

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 03390be5fa)
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-06-30 19:54:11 -06:00
Sebastiaan van Stijn
6d3a644c4a
pkg/fileutils: GetTotalUsedFds(): don't pretend to support FreeBSD
Commit 8d56108ffb moved this function from
the generic (no build-tags) fileutils.go to a unix file, adding "freebsd"
to the build-tags.

This likely was a wrong assumption (as other files had freebsd build-tags).
FreeBSD's procfs does not mention `/proc/<pid>/fd` in the manpage, and
we don't test FreeBSD in CI, so let's drop it, and make this a Linux-only
file.

While updating also dropping the import-tag, as we're planning to move
this file internal to the daemon.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 252e94f499)
Resolved conflicts:
	pkg/fileutils/fileutils_linux.go
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-06-30 19:54:08 -06:00
Sebastiaan van Stijn
e21df44a53
Merge pull request #45852 from thaJeztah/23.0_backport_gha_fix_missing_daemonjson
[23.0 backport] gha: don't fail if no daemon.json is present
2023-06-30 11:02:18 +02:00
Sebastiaan van Stijn
457b987170
gha: don't fail if no daemon.json is present
CI failed sometimes if no daemon.json was present:

    Run sudo rm /etc/docker/daemon.json
    sudo rm /etc/docker/daemon.json
    sudo service docker restart
    docker version
    docker info
    shell: /usr/bin/bash -e {0}
    env:
    DESTDIR: ./build
    BUILDKIT_REPO: moby/buildkit
    BUILDKIT_TEST_DISABLE_FEATURES: cache_backend_azblob,cache_backend_s3,merge_diff
    BUILDKIT_REF: 798ad6b0ce9f2fe86dfb2b0277e6770d0b545871
    rm: cannot remove '/etc/docker/daemon.json': No such file or directory
    Error: Process completed with exit code 1.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 264dbad43a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-30 01:53:11 +02:00
Sebastiaan van Stijn
5272bca50f
Merge pull request #45825 from thaJeztah/23.0_backport_fix_live_restore_local_vol_mounts
[23.0 backport] Restore active mount counts on live-restore
2023-06-29 00:38:11 +02:00
Sebastiaan van Stijn
2c1c20c04c
Merge pull request #45834 from neersighted/backport/45766/23.0
[23.0 backport] seccomp: always allow name_to_handle_at(2)
2023-06-28 18:34:35 +02:00
Bjorn Neergaard
b173b9e739
seccomp: add name_to_handle_at to allowlist
Based on the analysis on [the previous PR][1].

  [1]: https://github.com/moby/moby/pull/45766#pullrequestreview-1493908145

Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
(cherry picked from commit b335e3d305)
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-06-28 05:47:10 -06:00
Brian Goff
b83b8b2e40
Restore active mount counts on live-restore
When live-restoring a container the volume driver needs be notified that
there is an active mount for the volume.
Before this change the count is zero until the container stops and the
uint64 overflows pretty much making it so the volume can never be
removed until another daemon restart.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit 647c2a6cdd)
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-28 09:52:08 +02:00
Vitor Anjos
7f2729ff2c
remove name_to_handle_at(2) from filtered syscalls
Signed-off-by: Vitor Anjos <bartier@users.noreply.github.com>
(cherry picked from commit fdc9b7cceb)
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-06-27 13:20:44 -06:00
Bjorn Neergaard
8ef04fc32e
Merge pull request #45830 from thaJeztah/23.0_backport_sudo_tee
[23.0 backport] gha: Setup Runner: add missing sudo
2023-06-27 09:27:53 -06:00
Sebastiaan van Stijn
9ea231bb78
gha: Setup Runner: add missing sudo
I think this may be missing a sudo (as all other operations do use
sudo to access daemon.json);

    Run if [ ! -e /etc/docker/daemon.json ]; then
      if [ ! -e /etc/docker/daemon.json ]; then
       echo '{}' | tee /etc/docker/daemon.json >/dev/null
      fi
      DOCKERD_CONFIG=$(jq '.+{"experimental":true,"live-restore":true,"ipv6":true,"fixed-cidr-v6":"2001:db8:1::/64"}' /etc/docker/daemon.json)
      sudo tee /etc/docker/daemon.json <<<"$DOCKERD_CONFIG" >/dev/null
      sudo service docker restart
      shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
      env:
        GO_VERSION: 1.20.5
        GOTESTLIST_VERSION: v0.3.1
        TESTSTAT_VERSION: v0.1.3
        ITG_CLI_MATRIX_SIZE: 6
        DOCKER_EXPERIMENTAL: 1
        DOCKER_GRAPHDRIVER: overlay2
    tee: /etc/docker/daemon.json: Permission denied
    Error: Process completed with exit code 1.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit d8bc5828cd)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-27 15:41:07 +02:00
Sebastiaan van Stijn
d323999449
Merge pull request #45809 from thaJeztah/23.0_backport_fix-missing-csi-topology
[23.0 backport] Fix missing Topology in NodeCSIInfo
2023-06-26 13:04:26 +02:00
Drew Erny
dcfe991e4a
Fix missing Topology in NodeCSIInfo
Added code to correctly retrieve and convert the Topology from the gRPC
Swarm Node.

Signed-off-by: Drew Erny <derny@mirantis.com>
(cherry picked from commit cdb1293eea)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-26 11:01:54 +02:00
Bjorn Neergaard
14448e42fc
Merge pull request #45803 from corhere/backport-23.0/go_1.20.5
[23.0 backport] Update Go to go1.20.5
2023-06-24 07:36:38 -06:00
Sebastiaan van Stijn
a70bbdf673 update go to go1.20.5
go1.20.5 (released 2023-06-06) includes four security fixes to the cmd/go and
runtime packages, as well as bug fixes to the compiler, the go command, the
runtime, and the crypto/rsa, net, and os packages. See the Go 1.20.5 milestone
on our issue tracker for details:

https://github.com/golang/go/issues?q=milestone%3AGo1.20.5+label%3ACherryPickApproved

full diff: https://github.com/golang/go/compare/go1.20.4...go1.20.5

These minor releases include 3 security fixes following the security policy:

- cmd/go: cgo code injection
  The go command may generate unexpected code at build time when using cgo. This
  may result in unexpected behavior when running a go program which uses cgo.

  This may occur when running an untrusted module which contains directories with
  newline characters in their names. Modules which are retrieved using the go command,
  i.e. via "go get", are not affected (modules retrieved using GOPATH-mode, i.e.
  GO111MODULE=off, may be affected).

  Thanks to Juho Nurminen of Mattermost for reporting this issue.

  This is CVE-2023-29402 and Go issue https://go.dev/issue/60167.

- runtime: unexpected behavior of setuid/setgid binaries

  The Go runtime didn't act any differently when a binary had the setuid/setgid
  bit set. On Unix platforms, if a setuid/setgid binary was executed with standard
  I/O file descriptors closed, opening any files could result in unexpected
  content being read/written with elevated prilieges. Similarly if a setuid/setgid
  program was terminated, either via panic or signal, it could leak the contents
  of its registers.

  Thanks to Vincent Dehors from Synacktiv for reporting this issue.

  This is CVE-2023-29403 and Go issue https://go.dev/issue/60272.

- cmd/go: improper sanitization of LDFLAGS

  The go command may execute arbitrary code at build time when using cgo. This may
  occur when running "go get" on a malicious module, or when running any other
  command which builds untrusted code. This is can by triggered by linker flags,
  specified via a "#cgo LDFLAGS" directive.

  Thanks to Juho Nurminen of Mattermost for reporting this issue.

  This is CVE-2023-29404 and CVE-2023-29405 and Go issues https://go.dev/issue/60305 and https://go.dev/issue/60306.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 98a44bb18e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 24c882c3e0)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-06-23 16:55:04 -04:00
Sebastiaan van Stijn
4c57e2d55f update go to go1.20.4
go1.20.4 (released 2023-05-02) includes three security fixes to the html/template
package, as well as bug fixes to the compiler, the runtime, and the crypto/subtle,
crypto/tls, net/http, and syscall packages. See the Go 1.20.4 milestone on our
issue tracker for details:

https://github.com/golang/go/issues?q=milestone%3AGo1.20.4+label%3ACherryPickApproved

release notes: https://go.dev/doc/devel/release#go1.20.4
full diff: https://github.com/golang/go/compare/go1.20.3...go1.20.4

from the announcement:

> These minor releases include 3 security fixes following the security policy:
>
> - html/template: improper sanitization of CSS values
>
>   Angle brackets (`<>`) were not considered dangerous characters when inserted
>   into CSS contexts. Templates containing multiple actions separated by a '/'
>   character could result in unexpectedly closing the CSS context and allowing
>   for injection of unexpected HMTL, if executed with untrusted input.
>
>   Thanks to Juho Nurminen of Mattermost for reporting this issue.
>
>   This is CVE-2023-24539 and Go issue https://go.dev/issue/59720.
>
> - html/template: improper handling of JavaScript whitespace
>
>   Not all valid JavaScript whitespace characters were considered to be
>   whitespace. Templates containing whitespace characters outside of the character
>   set "\t\n\f\r\u0020\u2028\u2029" in JavaScript contexts that also contain
>   actions may not be properly sanitized during execution.
>
>   Thanks to Juho Nurminen of Mattermost for reporting this issue.
>
>   This is CVE-2023-24540 and Go issue https://go.dev/issue/59721.
>
> - html/template: improper handling of empty HTML attributes
>
>   Templates containing actions in unquoted HTML attributes (e.g. "attr={{.}}")
>   executed with empty input could result in output that would have unexpected
>   results when parsed due to HTML normalization rules. This may allow injection
>   of arbitrary attributes into tags.
>
>   Thanks to Juho Nurminen of Mattermost for reporting this issue.
>
>   This is CVE-2023-29400 and Go issue https://go.dev/issue/59722.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit b7e8868235)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-06-23 16:53:36 -04:00
Sebastiaan van Stijn
9279c41ea5 update go to go1.20.3
go1.20.3 (released 2023-04-04) includes security fixes to the go/parser,
html/template, mime/multipart, net/http, and net/textproto packages, as well
as bug fixes to the compiler, the linker, the runtime, and the time package.
See the Go 1.20.3 milestone on our issue tracker for details:

https://github.com/golang/go/issues?q=milestone%3AGo1.20.3+label%3ACherryPickApproved

full diff: https://github.com/golang/go/compare/go1.20.2...go1.20.3

Further details from the announcement on the mailing list:

We have just released Go versions 1.20.3 and 1.19.8, minor point releases.
These minor releases include 4 security fixes following the security policy:

- go/parser: infinite loop in parsing

  Calling any of the Parse functions on Go source code which contains `//line`
  directives with very large line numbers can cause an infinite loop due to
  integer overflow.
  Thanks to Philippe Antoine (Catena cyber) for reporting this issue.
  This is CVE-2023-24537 and Go issue https://go.dev/issue/59180.

- html/template: backticks not treated as string delimiters

  Templates did not properly consider backticks (`) as Javascript string
  delimiters, and as such did not escape them as expected. Backticks are
  used, since ES6, for JS template literals. If a template contained a Go
  template action within a Javascript template literal, the contents of the
  action could be used to terminate the literal, injecting arbitrary Javascript
  code into the Go template.

  As ES6 template literals are rather complex, and themselves can do string
  interpolation, we've decided to simply disallow Go template actions from being
  used inside of them (e.g. "var a = {{.}}"), since there is no obviously safe
  way to allow this behavior. This takes the same approach as
  github.com/google/safehtml. Template.Parse will now return an Error when it
  encounters templates like this, with a currently unexported ErrorCode with a
  value of 12. This ErrorCode will be exported in the next major release.

  Users who rely on this behavior can re-enable it using the GODEBUG flag
  jstmpllitinterp=1, with the caveat that backticks will now be escaped. This
  should be used with caution.

  Thanks to Sohom Datta, Manipal Institute of Technology, for reporting this issue.

  This is CVE-2023-24538 and Go issue https://go.dev/issue/59234.

- net/http, net/textproto: denial of service from excessive memory allocation

  HTTP and MIME header parsing could allocate large amounts of memory, even when
  parsing small inputs.

  Certain unusual patterns of input data could cause the common function used to
  parse HTTP and MIME headers to allocate substantially more memory than
  required to hold the parsed headers. An attacker can exploit this behavior to
  cause an HTTP server to allocate large amounts of memory from a small request,
  potentially leading to memory exhaustion and a denial of service.
  Header parsing now correctly allocates only the memory required to hold parsed
  headers.

  Thanks to Jakob Ackermann (@das7pad) for discovering this issue.

  This is CVE-2023-24534 and Go issue https://go.dev/issue/58975.

- net/http, net/textproto, mime/multipart: denial of service from excessive resource consumption

  Multipart form parsing can consume large amounts of CPU and memory when
  processing form inputs containing very large numbers of parts. This stems from
  several causes:

  mime/multipart.Reader.ReadForm limits the total memory a parsed multipart form
  can consume. ReadForm could undercount the amount of memory consumed, leading
  it to accept larger inputs than intended. Limiting total memory does not
  account for increased pressure on the garbage collector from large numbers of
  small allocations in forms with many parts. ReadForm could allocate a large
  number of short-lived buffers, further increasing pressure on the garbage
  collector. The combination of these factors can permit an attacker to cause an
  program that parses multipart forms to consume large amounts of CPU and
  memory, potentially resulting in a denial of service. This affects programs
  that use mime/multipart.Reader.ReadForm, as well as form parsing in the
  net/http package with the Request methods FormFile, FormValue,
  ParseMultipartForm, and PostFormValue.

  ReadForm now does a better job of estimating the memory consumption of parsed
  forms, and performs many fewer short-lived allocations.

  In addition, mime/multipart.Reader now imposes the following limits on the
  size of parsed forms:

  Forms parsed with ReadForm may contain no more than 1000 parts. This limit may
  be adjusted with the environment variable GODEBUG=multipartmaxparts=. Form
  parts parsed with NextPart and NextRawPart may contain no more than 10,000
  header fields. In addition, forms parsed with ReadForm may contain no more
  than 10,000 header fields across all parts. This limit may be adjusted with
  the environment variable GODEBUG=multipartmaxheaders=.

  Thanks to Jakob Ackermann for discovering this issue.

  This is CVE-2023-24536 and Go issue https://go.dev/issue/59153.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit f6cc8e3512)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-06-23 16:53:29 -04:00
Sebastiaan van Stijn
81ae2ab388 update to go1.20.2
Includes a security fix for crypto/elliptic (CVE-2023-24532).

> go1.20.2 (released 2023-03-07) includes a security fix to the crypto/elliptic package,
> as well as bug fixes to the compiler, the covdata command, the linker, the runtime, and
> the crypto/ecdh, crypto/rsa, crypto/x509, os, and syscall packages.
> See the Go 1.20.2 milestone on our issue tracker for details.

https://go.dev/doc/devel/release#go1.20.minor

From the announcement:

> We have just released Go versions 1.20.2 and 1.19.7, minor point releases.
>
> These minor releases include 1 security fixes following the security policy:
>
> - crypto/elliptic: incorrect P-256 ScalarMult and ScalarBaseMult results
>
>   The ScalarMult and ScalarBaseMult methods of the P256 Curve may return an
>   incorrect result if called with some specific unreduced scalars (a scalar larger
>   than the order of the curve).
>
>   This does not impact usages of crypto/ecdsa or crypto/ecdh.
>
> This is CVE-2023-24532 and Go issue https://go.dev/issue/58647.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 02dec48bab)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-06-23 16:53:18 -04:00
Cory Snider
a7948ebf6c Upgrade to Go 1.20.1
Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit cd779dae46)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-06-23 16:52:12 -04:00
Bjorn Neergaard
952fa4d207 Upgrade to Go 1.20
Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
Co-authored-by: Cory Snider <csnider@mirantis.com>
Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit d4e7a87b96)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-06-23 16:51:26 -04:00
Bjorn Neergaard
f00569b891
Merge pull request #45791 from crazy-max/23.0_backport_fix-host-gateway
[23.0 backport] builder: pass host-gateway IP as worker label
2023-06-23 05:55:09 -06:00
CrazyMax
40dbcd6e11
builder: pass host-gateway IP as worker label
We missed a case when parsing extra hosts from the dockerfile
frontend so the build fails.

To handle this case we need to set a dedicated worker label
that contains the host gateway IP so clients like Buildx
can just set the proper host:ip when parsing extra hosts
that contain the special string "host-gateway".

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 21e50b89c9)
2023-06-22 16:30:07 +02:00
Bjorn Neergaard
17219db1c3
Merge pull request #45775 from thaJeztah/23.0_backport_dont_cancel_stop
[23.0 backport] don't cancel container stop when cancelling context
2023-06-21 09:13:05 -06:00
Bjorn Neergaard
9c58a64fc1
Merge pull request #45783 from thaJeztah/23.0_backport_reserve_once
[23.0 backport] daemon: registerName(): don't reserve name twice
2023-06-21 09:12:37 -06:00
Sebastiaan van Stijn
dea225bfe0
daemon: registerName(): don't reserve name twice
daemon.generateNewName() already reserves the generated name, but its name
did not indicate it did. The daemon.registerName() assumed that the generated
name still had to be reserved, which could mean it would try to reserve the
same name again.

This patch renames daemon.generateNewName to daemon.generateAndReserveName
to make it clearer what it does, and updates registerName() to return early
if it successfully generated (and registered) the container name.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 3ba67ee214)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-20 23:49:09 +02:00
Sebastiaan van Stijn
e578b2510a
don't cancel container stop when cancelling context
Commit 90de570cfa passed through the request
context to daemon.ContainerStop(). As a result, cancelling the context would
cancel the "graceful" stop of the container, and would proceed with forcefully
killing the container.

This patch partially reverts the changes from 90de570cfa
and breaks the context to prevent cancelling the context from cancelling the stop.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit fc94ed0a86)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-20 23:43:43 +02:00
Akihiro Suda
a7142badb7
Merge pull request #45772 from thaJeztah/23.0_backport_fix_docker_py
[23.0 backport] testing: temporarily pin docker-py tests to use "bullseye"
2023-06-20 22:49:39 +09:00
Sebastiaan van Stijn
d38a7f99f5
testing: temporarily pin docker-py tests to use "bullseye"
The official Python images on Docker Hub switched to debian bookworm,
which is now the current stable version of Debian.

However, the location of the apt repository config file changed, which
causes the Dockerfile build to fail;

    Loaded image: emptyfs:latest
    Loaded image ID: sha256:0df1207206e5288f4a989a2f13d1f5b3c4e70467702c1d5d21dfc9f002b7bd43
    INFO: Building docker-sdk-python3:5.0.3...
    tests/Dockerfile:6
    --------------------
       5 |     ARG APT_MIRROR
       6 | >>> RUN sed -ri "s/(httpredir|deb).debian.org/${APT_MIRROR:-deb.debian.org}/g" /etc/apt/sources.list \
       7 | >>>     && sed -ri "s/(security).debian.org/${APT_MIRROR:-security.debian.org}/g" /etc/apt/sources.list
       8 |
    --------------------
    ERROR: failed to solve: process "/bin/sh -c sed -ri \"s/(httpredir|deb).debian.org/${APT_MIRROR:-deb.debian.org}/g\" /etc/apt/sources.list     && sed -ri \"s/(security).debian.org/${APT_MIRROR:-security.debian.org}/g\" /etc/apt/sources.list" did not complete successfully: exit code: 2

This needs to be fixed in docker-py, but in the meantime, we can pin to
the bullseye variant.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 19d860fa9d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-20 11:41:43 +02:00
Sebastiaan van Stijn
108d0e5765
Merge pull request #45751 from thaJeztah/23.0_update_go1.19.10
[23.0] update go to go1.19.10
2023-06-14 22:21:24 +02:00
Sebastiaan van Stijn
d2edeea758
[23.0] update go to go1.19.10
go1.19.10 (released 2023-06-06) includes four security fixes to the cmd/go and
runtime packages, as well as bug fixes to the compiler, the go command, and the
runtime. See the Go 1.19.10 milestone on our issue tracker for details:

https://github.com/golang/go/issues?q=milestone%3AGo1.19.10+label%3ACherryPickApproved

full diff: https://github.com/golang/go/compare/go1.19.9...go1.19.10

These minor releases include 3 security fixes following the security policy:

- cmd/go: cgo code injection
  The go command may generate unexpected code at build time when using cgo. This
  may result in unexpected behavior when running a go program which uses cgo.

  This may occur when running an untrusted module which contains directories with
  newline characters in their names. Modules which are retrieved using the go command,
  i.e. via "go get", are not affected (modules retrieved using GOPATH-mode, i.e.
  GO111MODULE=off, may be affected).

  Thanks to Juho Nurminen of Mattermost for reporting this issue.

  This is CVE-2023-29402 and Go issue https://go.dev/issue/60167.

- runtime: unexpected behavior of setuid/setgid binaries

  The Go runtime didn't act any differently when a binary had the setuid/setgid
  bit set. On Unix platforms, if a setuid/setgid binary was executed with standard
  I/O file descriptors closed, opening any files could result in unexpected
  content being read/written with elevated prilieges. Similarly if a setuid/setgid
  program was terminated, either via panic or signal, it could leak the contents
  of its registers.

  Thanks to Vincent Dehors from Synacktiv for reporting this issue.

  This is CVE-2023-29403 and Go issue https://go.dev/issue/60272.

- cmd/go: improper sanitization of LDFLAGS

  The go command may execute arbitrary code at build time when using cgo. This may
  occur when running "go get" on a malicious module, or when running any other
  command which builds untrusted code. This is can by triggered by linker flags,
  specified via a "#cgo LDFLAGS" directive.

  Thanks to Juho Nurminen of Mattermost for reporting this issue.

  This is CVE-2023-29404 and CVE-2023-29405 and Go issues https://go.dev/issue/60305 and https://go.dev/issue/60306.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-14 13:21:02 +02:00
Akihiro Suda
4d9cb4af16
Merge pull request #45709 from neersighted/check_config_bpf/23.0
[23.0 backport] contrib/check-config: require xt_bpf for encrypted overlay
2023-06-07 18:45:25 +09:00
Bjorn Neergaard
88049124e4
contrib/check-config: move xt_bpf check to overlay section
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
(cherry picked from commit 800ea039ec)
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-06-06 16:49:26 -06:00
Bjorn Neergaard
028a8d421a
contrib/check-config: check for xt_bpf
We omit xt_u32 as it's optional; since we will remove support for this
module in the future, it's simpler to check for xt_bpf, which will
become the new baseline.

Related issues:
* https://github.com/microsoft/WSL/issues/10029#issuecomment-1574440255
* https://github.com/docker/for-win/issues/13450#issuecomment-1574443139

Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
(cherry picked from commit 1910fdde81)
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-06-06 16:49:24 -06:00
Sebastiaan van Stijn
57cc620aa0
Merge pull request #45638 from corhere/backport-23.0/libn/fix-encrypted-overlay-nonstandard-port
[23.0 backport] libnetwork/d/overlay: support encryption on any port
2023-05-27 00:44:51 +02:00
Cory Snider
f6ff82666d libn/d/overlay: support encryption on any port
While the VXLAN interface and the iptables rules to mark outgoing VXLAN
packets for encryption are configured to use the Swarm data path port,
the XFRM policies for actually applying the encryption are hardcoded to
match packets with destination port 4789/udp. Consequently, encrypted
overlay networks do not pass traffic when the Swarm is configured with
any other data path port: encryption is not applied to the outgoing
VXLAN packets and the destination host drops the received cleartext
packets. Use the configured data path port instead of hardcoding port
4789 in the XFRM policies.

Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit 9a692a3802)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-05-26 16:43:40 -04:00
Bjorn Neergaard
91cc8ad679
Merge pull request #45614 from thaJeztah/23.0_backport_skip_criu
[23.0 backport] Dockerfile: temporarily skip CRIU stage
2023-05-25 07:03:39 -06:00
Sebastiaan van Stijn
8a8f179a6a
Dockerfile: temporarily skip CRIU stage
The package repository currently has issues;

    => ERROR https://download.opensuse.org/repositories/devel:/tools:/criu/Debian_11/Release.key

The only test currently using this binary is currently skipped, as the test
was broken;
6e98a7f2c9/integration/container/checkpoint_test.go (L32-L33)

So let's disable this stage for the time being.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit d3d2823edf)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-25 12:20:50 +02:00
Sebastiaan van Stijn
fa97b40bc2
Merge pull request #45605 from neersighted/backport/45603/23.0
[23.0 backport] hack/make/.binary: don't use "netgo" when building Windows binaries
2023-05-24 21:28:33 +02:00
Sebastiaan van Stijn
c696fdd003
hack/make/.binary: don't use "netgo" when building Windows binaries
Starting with go1.19, the Go runtime on Windows now supports the `netgo` build-
flag to use a native Go DNS resolver. Prior to that version, the build-flag
only had an effect on non-Windows platforms. When using the `netgo` build-flag,
the Windows's host resolver is not used, and as a result, custom entries in
`etc/hosts` are ignored, which is a change in behavior from binaries compiled
with older versions of the Go runtime.

From the go1.19 release notes: https://go.dev/doc/go1.19#net

> Resolver.PreferGo is now implemented on Windows and Plan 9. It previously
> only worked on Unix platforms. Combined with Dialer.Resolver and Resolver.Dial,
> it's now possible to write portable programs and be in control of all DNS name
> lookups when dialing.
>
> The net package now has initial support for the netgo build tag on Windows.
> When used, the package uses the Go DNS client (as used by Resolver.PreferGo)
> instead of asking Windows for DNS results. The upstream DNS server it discovers
> from Windows may not yet be correct with complex system network configurations,
> however.

Our Windows binaries are compiled with the "static" (`make/binary-daemon`)
script, which has the `netgo` option set by default. This patch unsets the
`netgo` option when cross-compiling for Windows.

Co-authored-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
(cherry picked from commit 53d1b12bc0)
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-05-24 10:48:57 -06:00
Bjorn Neergaard
e017245c34
Merge pull request #45559 from crazy-max/23.0_backport_fix-worker-id
[23.0 backport] build: use daemon id as worker id for the graph driver controller
2023-05-19 09:44:25 -06:00
Bjorn Neergaard
56ece96655
Merge pull request #45569 from corhere/backport-23.0/fix-empty-container-decode
[23.0 backport] api/server: allow empty body for POST /commit again
2023-05-19 14:28:38 +01:00
Sebastiaan van Stijn
532f0bafb6
Merge pull request #45572 from thaJeztah/23.0_backport_fix_insecure_registries_reload
[23.0 backport] Fix insecure registries reload
2023-05-19 00:21:48 +02:00
Nolan Miles
0a24137a19
reorder load funcs to match newServiceConfig()'s order
Signed-off-by: Nolan Miles <nolanpmiles@gmail.com>
(cherry picked from commit f3645a2aa3)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-18 23:04:06 +02:00
Nolan Miles
7f9f2c67c4
add mirror to daemon reload test for insecure registries
Signed-off-by: Nolan Miles <nolanpmiles@gmail.com>
(cherry picked from commit 3b15156e4d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-18 23:04:06 +02:00
Kevin Alvarez
276ef743cb
build: use daemon id as worker id for the graph driver controller
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 6d139e5e95)

# Conflicts:
#	builder/builder-next/controller.go
2023-05-18 22:22:23 +02:00
Cory Snider
6f7d702f9c api/server: allow empty body for POST /commit again
The error returned by DecodeConfig was changed in
b6d58d749c and caused this to regress.
Allow empty request bodies for this endpoint once again.

Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit 967c7bc5d3)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-05-18 16:11:33 -04:00
Akihiro Suda
cb3fd005d4
Merge pull request #45520 from thaJeztah/23.0_backport_vendor_distribution_v2.8.2
[23.0 backport] vendor: github.com/docker/distribution v2.8.2
2023-05-18 00:15:55 +09:00
Sebastiaan van Stijn
e94c22cc39
vendor: github.com/docker/distribution v2.8.2
CI

- Dockerfile: fix filenames of artifacts

Bugfixes

-  Fix panic in inmemory driver
-  Add code to handle pagination of parts. Fixes max layer size of 10GB bug
-  Parse http forbidden as denied
-  Revert "registry/client: set Accept: identity header when getting layers

Runtime

- Update to go1.19.9
- Dockerfile: update xx to v1.2.1 ([#3907](https://github.com/distribution/distribution/pull/3907))

Security

- Fix [CVE-2022-28391](https://www.cve.org/CVERecord?id=CVE-2022-28391) by bumping alpine from 3.14 to 3.16
- Fix [CVE-2023-2253](https://www.cve.org/CVERecord?id=CVE-2023-2253) runaway allocation on /v2/_catalog [`521ea3d9`](521ea3d973)

full diff: https://github.com/docker/distribution/compare/v2.8.1...v2.8.2

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

bump to release/2.8

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 7821d2d788)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-11 16:26:23 +02:00
Tianon Gravi
9dbdbd4b6d
Merge pull request #45371 from thaJeztah/23.0_backport_runc_binary_1.1.6
[23.0 backport] update runc binary to v1.1.7
2023-05-05 20:50:10 +00:00
Tianon Gravi
6030355735
Merge pull request #45478 from thaJeztah/23.0_containerd_binary_1.6.21
[23.0] update containerd binary to v1.6.21
2023-05-05 20:50:00 +00:00
Tianon Gravi
e1c053637b
Merge pull request #45477 from thaJeztah/23.0_backport_vendor_containerd_1.6.21
[23.0 backport] vendor: github.com/containerd/containerd v1.6.21
2023-05-05 20:43:10 +00:00
Sebastiaan van Stijn
e73310d8c9
update runc binary to v1.1.7
release notes: https://github.com/opencontainers/runc/releases/tag/v1.1.7
full diff: https://github.com/opencontainers/runc/compare/v1.1.6...v1.1.7

This is the seventh patch release in the 1.1.z release of runc, and is
the last planned release of the 1.1.z series. It contains a fix for
cgroup device rules with systemd when handling device rules for devices
that don't exist (though for devices whose drivers don't correctly
register themselves in the kernel -- such as the NVIDIA devices -- the
full fix only works with systemd v240+).

- When used with systemd v240+, systemd cgroup drivers no longer skip
  DeviceAllow rules if the device does not exist (a regression introduced
  in runc 1.1.3). This fix also reverts the workaround added in runc 1.1.5,
  removing an extra warning emitted by runc run/start.
- The source code now has a new file, runc.keyring, which contains the keys
  used to sign runc releases.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 2d0e899819)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-05 20:55:03 +02:00
Sebastiaan van Stijn
3f0402cb68
update runc binary to v1.1.6
release notes: https://github.com/opencontainers/runc/releases/tag/v1.1.6
full diff: https://github.com/opencontainers/runc/compare/v1.1.5...v1.1.6

This is the sixth patch release in the 1.1.z series of runc, which fixes
a series of cgroup-related issues.

Note that this release can no longer be built from sources using Go
1.16. Using a latest maintained Go 1.20.x or Go 1.19.x release is
recommended. Go 1.17 can still be used.

- systemd cgroup v1 and v2 drivers were deliberately ignoring UnitExist error
  from systemd while trying to create a systemd unit, which in some scenarios
  may result in a container not being added to the proper systemd unit and
  cgroup.
- systemd cgroup v2 driver was incorrectly translating cpuset range from spec's
  resources.cpu.cpus to systemd unit property (AllowedCPUs) in case of more
  than 8 CPUs, resulting in the wrong AllowedCPUs setting.
- systemd cgroup v1 driver was prefixing container's cgroup path with the path
  of PID 1 cgroup, resulting in inability to place PID 1 in a non-root cgroup.
- runc run/start may return "permission denied" error when starting a rootless
  container when the file to be executed does not have executable bit set for
  the user, not taking the CAP_DAC_OVERRIDE capability into account. This is
  a regression in runc 1.1.4, as well as in Go 1.20 and 1.20.1
- cgroup v1 drivers are now aware of misc controller.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit d0efca893b)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-05 20:54:45 +02:00
Sebastiaan van Stijn
edadebe177
update containerd binary to v1.6.21
release notes: https://github.com/containerd/containerd/releases/tag/v1.6.21

Notable Updates

- update runc binary to v1.1.7
- Remove entry for container from container store on error
- oci: partially restore comment on read-only mounts for uid/gid uses
- windows: Add ArgsEscaped support for CRI
- oci: Use WithReadonlyTempMount when adding users/groups
- archive: consistently respect value of WithSkipDockerManifest

full diff: https://github.com/containerd/containerd/compare/c0efc63d3907...v1.6.21

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-05 20:52:00 +02:00
Sebastiaan van Stijn
14661fc010
vendor: github.com/containerd/containerd v1.6.21
release notes: https://github.com/containerd/containerd/releases/tag/v1.6.21

Notable Updates

- update runc binary to v1.1.7
- Remove entry for container from container store on error
- oci: partially restore comment on read-only mounts for uid/gid uses
- windows: Add ArgsEscaped support for CRI
- oci: Use WithReadonlyTempMount when adding users/groups
- archive: consistently respect value of WithSkipDockerManifest

full diff: https://github.com/containerd/containerd/compare/c0efc63d3907...v1.6.21

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit a87313497b)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-05 20:21:51 +02:00
Djordje Lukic
e00f6efb20
Vendor containerd 1.6.x with the readonly mount fix
Uses containerd from release/1.6 commit c0efc63d39

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
(cherry picked from commit be4abf9d6e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-05 20:20:12 +02:00
Sebastiaan van Stijn
bd4620f6ee
vendor: github.com/containerd/containerd v1.6.20
full diff: https://github.com/containerd/containerd/compare/de33abf0547c...v1.6.20

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 61a0e79ad6)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-05 20:19:22 +02:00
Sebastiaan van Stijn
d06cf03729
vendor: re-vendor to add missing file
When running hack/vendor.sh, I noticed this file was added to vendor.
I suspect this should've been part of 0233029d5a,
but the vendor check doesn't appear to be catching this.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 3f09316e3b)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-05 20:16:49 +02:00
Sebastiaan van Stijn
cf7742ea7c
vendor: opencontainers/image-spec v1.1.0-rc2.0.20221005185240-3a7f492d3f1b
full diff: 02efb9a75e...3a7f492d3f

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 0233029d5a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-05 20:15:35 +02:00
Sebastiaan van Stijn
382e9ebbf6
vendor: github.com/containerd/ttrpc v1.1.1
- server: Fix connection leak when receiving ECONNRESET

full diff: https://github.com/containerd/ttrpc/compare/v1.1.0...v1.1.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit c733cf223d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-05 20:15:35 +02:00
Laura Brehm
e0843f264d
Update github.com/containerd/containerd to the HEAD of release/1.6 and vendor
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
(cherry picked from commit e85c69e1b7)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-05 20:14:51 +02:00
Sebastiaan van Stijn
24fd7c5eb8
Merge pull request #45465 from corhere/backport-23.0/copy-xattr-notsupp
[23.0 backport] d/graphdriver/copy: support src fs w/o xattr support
2023-05-05 01:14:43 +02:00
Cory Snider
94fcd2d2ef d/graphdriver/copy: support src fs w/o xattr support
Treat copying extended attributes from a source filesystem which does
not support extended attributes as a no-op, same as if the file did not
possess the extended attribute. Only fail copying extended attributes if
the source file has the attribute and the destination filesystem does
not support xattrs.

Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit 2b6761fd3e)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-05-04 15:28:04 -04:00
Sebastiaan van Stijn
c5197fe6c5
Merge pull request #45455 from thaJeztah/23.0_update_go1.19.9
[23.0] update go to go1.19.9
2023-05-04 19:49:24 +02:00
Sebastiaan van Stijn
70dbd2c870
[23.0] update go to go1.19.9
go1.19.9 (released 2023-05-02) includes three security fixes to the html/template
package, as well as bug fixes to the compiler, the runtime, and the crypto/tls
and syscall packages. See the Go 1.19.9 milestone on our issue tracker for details.

https://github.com/golang/go/issues?q=milestone%3AGo1.19.9+label%3ACherryPickApproved

release notes: https://go.dev/doc/devel/release#go1.19.9
full diff: https://github.com/golang/go/compare/go1.19.8...go1.19.9

from the announcement:

> These minor releases include 3 security fixes following the security policy:
>
>- html/template: improper sanitization of CSS values
>
>   Angle brackets (`<>`) were not considered dangerous characters when inserted
>   into CSS contexts. Templates containing multiple actions separated by a '/'
>   character could result in unexpectedly closing the CSS context and allowing
>   for injection of unexpected HMTL, if executed with untrusted input.
>
>   Thanks to Juho Nurminen of Mattermost for reporting this issue.
>
>   This is CVE-2023-24539 and Go issue https://go.dev/issue/59720.
>
> - html/template: improper handling of JavaScript whitespace
>
>   Not all valid JavaScript whitespace characters were considered to be
>   whitespace. Templates containing whitespace characters outside of the character
>   set "\t\n\f\r\u0020\u2028\u2029" in JavaScript contexts that also contain
>   actions may not be properly sanitized during execution.
>
>   Thanks to Juho Nurminen of Mattermost for reporting this issue.
>
>   This is CVE-2023-24540 and Go issue https://go.dev/issue/59721.
>
> - html/template: improper handling of empty HTML attributes
>
>   Templates containing actions in unquoted HTML attributes (e.g. "attr={{.}}")
>   executed with empty input could result in output that would have unexpected
>   results when parsed due to HTML normalization rules. This may allow injection
>   of arbitrary attributes into tags.
>
>   Thanks to Juho Nurminen of Mattermost for reporting this issue.
>
>   This is CVE-2023-29400 and Go issue https://go.dev/issue/59722.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-03 20:48:31 +02:00
Sebastiaan van Stijn
94d3ad69cc
Merge pull request #45410 from rumpl/23.0_backport-fix-buildx-tag-events
[23.0 backport] Use the image service instead of the reference store for tagging
2023-04-26 17:29:05 +02:00
Sebastiaan van Stijn
933a9f83b7
Merge pull request #45407 from thaJeztah/23.0_backport_vendor_runc_1.1.5
[23.0 backport] vendor: github.com/opencontainers/runc v1.1.5
2023-04-26 16:43:32 +02:00
Djordje Lukic
fe0a414613
Use the image service instead of the reference store for tagging
The image store sends events when a new image is created/tagged, using
it instead of the reference store makes sure we send the "tag" event
when a new image is built using buildx.

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-04-26 16:32:25 +02:00
Sebastiaan van Stijn
1b1230eaaf
Merge pull request #45375 from thaJeztah/23.0_backport_bump_swarmkit
[23.0 backport] vendor: github.com/moby/swarmkit/v2 v2.0.0-20230315203717-e28e8ba9bc83
2023-04-26 15:22:10 +02:00
Sebastiaan van Stijn
1b263035af
Merge pull request #45374 from thaJeztah/23.0_backport_stfu_grpc
[23.0 backport] Silence GRPC logs unless our log level is debug
2023-04-26 15:21:47 +02:00
Sebastiaan van Stijn
682542fd08
Merge pull request #45373 from thaJeztah/23.0_backport_assorted_test_and_packaging
[23.0 backport] assorted test- and build/packaging fixes
2023-04-26 15:21:31 +02:00
Sebastiaan van Stijn
40ec0eb9fe
vendor: github.com/opencontainers/runc v1.1.5
no changes in vendored code, just keeping scanners happy :)

release notes: https://github.com/opencontainers/runc/releases/tag/v1.1.5
diff: https://github.com/opencontainers/runc/compare/v1.1.4...v1.1.5

This is the fifth patch release in the 1.1.z series of runc, which fixes
three CVEs found in runc.

* CVE-2023-25809 is a vulnerability involving rootless containers where
  (under specific configurations), the container would have write access
  to the /sys/fs/cgroup/user.slice/... cgroup hierarchy. No other
  hierarchies on the host were affected. This vulnerability was
  discovered by Akihiro Suda.
  <https://github.com/opencontainers/runc/security/advisories/GHSA-m8cg-xc2p-r3fc>

* CVE-2023-27561 was a regression which effectively re-introduced
  CVE-2019-19921. This bug was present from v1.0.0-rc95 to v1.1.4. This
  regression was discovered by @Beuc.
  <https://github.com/advisories/GHSA-vpvm-3wq2-2wvm>

* CVE-2023-28642 is a variant of CVE-2023-27561 and was fixed by the same
  patch. This variant of the above vulnerability was reported by Lei
  Wang.
  <https://github.com/opencontainers/runc/security/advisories/GHSA-g2j6-57v7-gm8c>

In addition, the following other fixes are included in this release:

* Fix the inability to use `/dev/null` when inside a container.
* Fix changing the ownership of host's `/dev/null` caused by fd redirection
  (a regression in 1.1.1).
* Fix rare runc exec/enter unshare error on older kernels, including
  CentOS < 7.7.
* nsexec: Check for errors in `write_log()`.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit a17029ba49)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-26 14:50:34 +02:00
Sebastiaan van Stijn
9af6762d34
Merge pull request #45403 from akerouanton/23.0-cherrypick-2d31697
[23.0 backport] daemon: set docker0 subpool as the IPAM pool
2023-04-26 14:34:41 +02:00
Sebastiaan van Stijn
0b6449a477
Merge pull request #45376 from thaJeztah/23.0_backport_delete_network_more_atomically
[23.0 backport] libnetwork: clean up inDelete network atomically
2023-04-26 13:11:30 +02:00
Albin Kerouanton
227d3f39a3
daemon: set docker0 subpool as the IPAM pool
Since cc19eba (backported to v23.0.4), the PreferredPool for docker0 is
set only when the user provides the bip config parameter or when the
default bridge already exist. That means, if a user provides the
fixed-cidr parameter on a fresh install or reboot their computer/server
without bip set, dockerd throw the following error when it starts:

> failed to start daemon: Error initializing network controller: Error
> creating default "bridge" network: failed to parse pool request for
> address space "LocalDefault" pool "" subpool "100.64.0.0/26": Invalid
> Address SubPool

See #45356.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
(cherry picked from commit 2d31697)
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-04-26 11:38:55 +02:00
Sebastiaan van Stijn
6f83b8c3f4
Merge pull request #45372 from thaJeztah/23.0_backport_really_deprecate_newclient
[23.0 backport] client: deprecate NewClient properly
2023-04-25 18:31:47 +02:00
Cory Snider
c82be35412
libnetwork: clean up inDelete network atomically
The (*network).ipamRelease function nils out the network's IPAM info
fields, putting the network struct into an inconsistent state. The
network-restore startup code panics if it tries to restore a network
from a struct which has fewer IPAM config entries than IPAM info
entries. Therefore (*network).delete contains a critical section: by
persisting the network to the store after ipamRelease(), the datastore
will contain an inconsistent network until the deletion operation
completes and finishes deleting the network from the datastore. If for
any reason the deletion operation is interrupted between ipamRelease()
and deleteFromStore(), the daemon will crash on startup when it tries to
restore the network.

Updating the datastore after releasing the network's IPAM pools may have
served a purpose in the past, when a global datastore was used for
intra-cluster communication and the IPAM allocator had persistent global
state, but nowadays there is no global datastore and the IPAM allocator
has no persistent state whatsoever. Remove the vestigial datastore
update as it is no longer necessary and only serves to cause problems.
If the network deletion is interrupted before the network is deleted
from the datastore, the deletion will resume during the next daemon
startup, including releasing the IPAM pools.

Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit c957ad0067)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-21 12:47:01 +02:00
Sebastiaan van Stijn
ccccb7a170
vendor: github.com/moby/swarmkit/v2 v2.0.0-20230315203717-e28e8ba9bc83
- CSI: Manager PublishVolume verify CSI node ID is not empty

full diff: a745a8755c...e28e8ba9bc

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 72dc7a0f7b)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-21 12:42:24 +02:00
Brian Goff
8a6bec6e16
Silence GRPC logs unless our log level is debug
GRPC is logging a *lot* of garbage at info level.
This configures the GRPC logger such that it is only giving us logs when
at debug level and also adds a log field indicating where the logs are
coming from.

containerd is still currently spewing these same log messages and needs
a separate update.

Without this change `docker build` is extremely noisy in the daemon
logs.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit c7ccc68b15)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-21 12:33:33 +02:00
Aneesh Kulkarni
844fd50c67
Added timeout to validate job in github actions
Signed-off-by: Aneesh Kulkarni <askthefactorcamera@gmail.com>
(cherry picked from commit e5dbd28afd)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-21 12:29:21 +02:00
CrazyMax
8cec683574
Dockerfile: update xx to 1.2.1
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 106c694993)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-21 12:28:53 +02:00
Albin Kerouanton
7ff53c3c78
Update delve version
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
(cherry picked from commit 4f48a4ea2a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-21 12:28:27 +02:00
Paweł Gronowski
4d57ee692d
TestLogs: Increase stop check poll timeout on Windows
Stopping container on Windows can sometimes take longer than 10s which
caused this test to be flaky.
Increase the timeout to 75s when running this test on Windows.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 74dbb721aa)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-21 12:26:59 +02:00
xin.li
838d5ea058
chore: use http constants to replace numbers
Signed-off-by: xin.li <xin.li@daocloud.io>
(cherry picked from commit a770153af0)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-21 12:13:31 +02:00
Cory Snider
61d084593c
client: deprecate NewClient properly
The 'Deprecated:' line in NewClient's doc comment was not in a new
paragraph, so GoDoc, linters, and IDEs were unaware that it was
deprecated. The package documentation also continued to reference
NewClient. Update the doc comments to finish documenting that NewClient
is deprecated.

Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit 6b9968e8b1)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-21 12:05:15 +02:00
Sebastiaan van Stijn
8fdca288c5
Merge pull request #45350 from PettitWesley/awslogs-non-blocking-bug-23.0
[23.0 backport] awslogs: fix non-blocking log drop bug
2023-04-19 12:59:45 +02:00
Wesley Pettit
28b694d32d
awslogs: fix non-blocking log drop bug
Previously, the AWSLogs driver attempted to implement
non-blocking itself. Non-blocking is supposed to
implemented solely by the Docker RingBuffer that
wraps the log driver.

Please see issue and explanation here:
https://github.com/moby/moby/issues/45217

Signed-off-by: Wesley Pettit <wppttt@amazon.com>
(cherry picked from commit c8f8d11ac4)
2023-04-17 11:59:53 -07:00
Sebastiaan van Stijn
cbce331930
Merge pull request #45330 from kevingentile/buildkit-3770
[23.0] vendor: github.com/moby/buildkit v0.10.7-0.20230412161310-d52b2d584242
2023-04-14 02:59:18 +02:00
Bjorn Neergaard
5f684cb072
Merge pull request #45331 from thaJeztah/23.0_backport_rootless_script_bugs
[23.0 backport] Fix argument quoting bugs in dockerd-rootless.sh
2023-04-13 18:48:55 -06:00
kpcyrd
3731ce10d4
Fix argument quoting bugs in dockerd-rootless.sh
Signed-off-by: kpcyrd <git@rxv.cc>
(cherry picked from commit 7fe0f73838)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-14 01:26:56 +02:00
Kevin Gentile
23774ada04
vendor: github.com/moby/buildkit v0.10.7-0.20230412161310-d52b2d584242
Signed-off-by: Kevin Gentile <kevin.gentile@rockwellautomation.com>
2023-04-13 23:29:30 +02:00
Sebastiaan van Stijn
90e8a0bbf5
Merge pull request #45323 from thaJeztah/23.0_backport_vendor_sctp
[23.0 backport] vendor: github.com/ishidawataru/sctp v0.0.0-20230406120618-7ff4192f6ff2
2023-04-13 21:59:27 +02:00
Sebastiaan van Stijn
9277e64444
vendor: github.com/ishidawataru/sctp v0.0.0-20230406120618-7ff4192f6ff2
- Prevent from descriptor leak
- Fixes optlen in getsockopt() for s390x

full diff: 9a39160e90...7ff4192f6f

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 893d28469f)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-13 15:02:16 +02:00
Sebastiaan van Stijn
cdb6200887
Merge pull request #45293 from AkihiroSuda/backport-45283-23
[23.0 backport] docker-rootless-setuptools.sh: improve readability of messages
2023-04-12 20:30:21 +02:00
Akihiro Suda
09fbbd5677
docker-rootless-setuptools.sh: improve readability of messages
Before:
```console
$ docker-rootless-setuptool.sh install
...
[INFO] Use CLI context "rootless"
Current context is now "rootless"

[INFO] Make sure the following environment variables are set (or add them to ~/.bashrc):

export PATH=/usr/local/bin:$PATH
Some applications may require the following environment variable too:
export DOCKER_HOST=unix:///run/user/1001/docker.sock
```

After:
```console
$ docker-rootless-setuptool.sh install
...
[INFO] Using CLI context "rootless"
Current context is now "rootless"

[INFO] Make sure the following environment variable(s) are set (or add them to ~/.bashrc):
export PATH=/usr/local/bin:$PATH

[INFO] Some applications may require the following environment variable too:
export DOCKER_HOST=unix:///run/user/1001/docker.sock
```

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit 4aa2876c75)
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2023-04-07 10:08:16 +09:00
Sebastiaan van Stijn
4ca4705bf7
Merge pull request #45277 from thaJeztah/23.0_bump_go1.19.8
[23.0] update go to go1.19.8
2023-04-06 01:48:49 +02:00
Sebastiaan van Stijn
d3e52936c3
[23.0] update go to go1.19.8
go1.19.8 (released 2023-04-04) includes security fixes to the go/parser,
html/template, mime/multipart, net/http, and net/textproto packages, as well as
bug fixes to the linker, the runtime, and the time package. See the Go 1.19.8
milestone on our issue tracker for details:

https://github.com/golang/go/issues?q=milestone%3AGo1.19.8+label%3ACherryPickApproved

full diff: https://github.com/golang/go/compare/go1.19.7...go1.19.8

Further details from the announcement on the mailing list:

We have just released Go versions 1.20.3 and 1.19.8, minor point releases.
These minor releases include 4 security fixes following the security policy:

- go/parser: infinite loop in parsing

  Calling any of the Parse functions on Go source code which contains `//line`
  directives with very large line numbers can cause an infinite loop due to
  integer overflow.
  Thanks to Philippe Antoine (Catena cyber) for reporting this issue.
  This is CVE-2023-24537 and Go issue https://go.dev/issue/59180.

- html/template: backticks not treated as string delimiters

  Templates did not properly consider backticks (`) as Javascript string
  delimiters, and as such did not escape them as expected. Backticks are
  used, since ES6, for JS template literals. If a template contained a Go
  template action within a Javascript template literal, the contents of the
  action could be used to terminate the literal, injecting arbitrary Javascript
  code into the Go template.

  As ES6 template literals are rather complex, and themselves can do string
  interpolation, we've decided to simply disallow Go template actions from being
  used inside of them (e.g. "var a = {{.}}"), since there is no obviously safe
  way to allow this behavior. This takes the same approach as
  github.com/google/safehtml. Template.Parse will now return an Error when it
  encounters templates like this, with a currently unexported ErrorCode with a
  value of 12. This ErrorCode will be exported in the next major release.

  Users who rely on this behavior can re-enable it using the GODEBUG flag
  jstmpllitinterp=1, with the caveat that backticks will now be escaped. This
  should be used with caution.

  Thanks to Sohom Datta, Manipal Institute of Technology, for reporting this issue.

  This is CVE-2023-24538 and Go issue https://go.dev/issue/59234.

- net/http, net/textproto: denial of service from excessive memory allocation

  HTTP and MIME header parsing could allocate large amounts of memory, even when
  parsing small inputs.

  Certain unusual patterns of input data could cause the common function used to
  parse HTTP and MIME headers to allocate substantially more memory than
  required to hold the parsed headers. An attacker can exploit this behavior to
  cause an HTTP server to allocate large amounts of memory from a small request,
  potentially leading to memory exhaustion and a denial of service.
  Header parsing now correctly allocates only the memory required to hold parsed
  headers.

  Thanks to Jakob Ackermann (@das7pad) for discovering this issue.

  This is CVE-2023-24534 and Go issue https://go.dev/issue/58975.

- net/http, net/textproto, mime/multipart: denial of service from excessive resource consumption

  Multipart form parsing can consume large amounts of CPU and memory when
  processing form inputs containing very large numbers of parts. This stems from
  several causes:

  mime/multipart.Reader.ReadForm limits the total memory a parsed multipart form
  can consume. ReadForm could undercount the amount of memory consumed, leading
  it to accept larger inputs than intended. Limiting total memory does not
  account for increased pressure on the garbage collector from large numbers of
  small allocations in forms with many parts. ReadForm could allocate a large
  number of short-lived buffers, further increasing pressure on the garbage
  collector. The combination of these factors can permit an attacker to cause an
  program that parses multipart forms to consume large amounts of CPU and
  memory, potentially resulting in a denial of service. This affects programs
  that use mime/multipart.Reader.ReadForm, as well as form parsing in the
  net/http package with the Request methods FormFile, FormValue,
  ParseMultipartForm, and PostFormValue.

  ReadForm now does a better job of estimating the memory consumption of parsed
  forms, and performs many fewer short-lived allocations.

  In addition, mime/multipart.Reader now imposes the following limits on the
  size of parsed forms:

  Forms parsed with ReadForm may contain no more than 1000 parts. This limit may
  be adjusted with the environment variable GODEBUG=multipartmaxparts=. Form
  parts parsed with NextPart and NextRawPart may contain no more than 10,000
  header fields. In addition, forms parsed with ReadForm may contain no more
  than 10,000 header fields across all parts. This limit may be adjusted with
  the environment variable GODEBUG=multipartmaxheaders=.

  Thanks to Jakob Ackermann for discovering this issue.

  This is CVE-2023-24536 and Go issue https://go.dev/issue/59153.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-05 22:02:01 +02:00
Sebastiaan van Stijn
347bce43e9
Merge pull request #45279 from thaJeztah/23.0_fix_vendor
[23.0] fix vendor.mod: add hashicorp/go-multierror as direct dependency
2023-04-05 18:33:33 +02:00
Sebastiaan van Stijn
a0a597cbbe
[23.0] fix vendor.mod: add hashicorp/go-multierror as direct dependency
commit 59118bff50 made this a direct
dependency (previously it was indirect). That commit was part of an
advisory, and didn't run the vendor validation check because of that.

This patch fixes the vendor.mod to unblock CI in this branch.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-05 17:31:29 +02:00
Sebastiaan van Stijn
c9f6343c24
Merge pull request #45220 from vvoland/test-windows-execstartfails-23
[23.0 backport] integration-cli: Enable TestExecStartFails on Windows
2023-04-05 02:44:27 +02:00
Sebastiaan van Stijn
0ce1e22243
Merge pull request #45237 from thaJeztah/23.0_backport_update_runc_binary_1.1.5
[23.0 backport] update runc binary to v1.1.5
2023-04-05 02:44:15 +02:00
Sebastiaan van Stijn
3648a4915c
Merge pull request #45240 from akerouanton/cherrypick-45211
[23.0 backport] libnet/d/windows: log EnableInternalDNS val after setting it
2023-04-05 02:44:02 +02:00
Sebastiaan van Stijn
aaa102b30f
Merge pull request #45245 from thaJeztah/23.0_containerd_binary_1.6.20
[23.0] update containerd binary to v1.6.20
2023-04-05 02:43:49 +02:00
Sebastiaan van Stijn
15d6037c1e
Merge pull request #45246 from akerouanton/cherrypick-44827
[23.0 backport] daemon: let libnetwork assign default bridge IPAM
2023-04-05 02:42:50 +02:00
Sebastiaan van Stijn
59118bff50
Merge pull request from GHSA-232p-vwff-86mp
[23.0 backport] libnetwork: ensure encryption is mandatory on encrypted overlay networks
2023-04-04 20:03:51 +02:00
Cory Snider
063d3a6f1a
daemon: let libnetwork assign default bridge IPAM
The netutils.ElectInterfaceAddresses function is only used in one place
outside of tests: in the daemon, to configure the default bridge
network. The function is also messy to reason about as it references the
shared mutable state of ipamutils.PredefinedLocalScopeDefaultNetworks.
It uses the list of predefined default networks to always return an IPv4
address even if the named interface does not exist or does not have any
IPv4 addresses. This list happens to be the same as the one used to
initialize the address pool of the 'builtin' IPAM driver, though that is
far from obvious. (Start with "./libnetwork".initIPAMDrivers and trace
the dataflow of the addressPool value. Surprise! Global state is being
mutated using the value of other global mutable state.)

The daemon does not need the fallback behaviour of
ElectInterfaceAddresses. In fact, the daemon does not have to configure
an address pool for the network at all! libnetwork will acquire one of
the available address ranges from the network's IPAM driver when the
preferred-pool configuration is unset. It will do so using the same list
of address ranges and the exact same logic
(netutils.FindAvailableNetworks) as ElectInterfaceAddresses. So unless
the daemon needs to force the network to use a specific address range
because the bridge interface already exists, it can leave the details
up to libnetwork.

Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit cc19eba)
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-03-31 09:38:21 +02:00
Sebastiaan van Stijn
389e18081d
update containerd binary to v1.6.20
Notable Updates

- Disable looking up usernames and groupnames on host
- Add support for Windows ArgsEscaped images
- Update hcsshim to v0.9.8
- Fix debug flag in shim
- Add WithReadonlyTempMount to support readonly temporary mounts
- Update ttrpc to fix file descriptor leak
- Update runc binary to v1.1.5
= Update image config to support ArgsEscaped

full diff: https://github.com/containerd/containerd/compare/v1.6.19...v1.6.20

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-03-31 00:48:19 +02:00
Albin Kerouanton
348544e6e8
libnet/d/windows: log EnableInternalDNS val after setting it
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
(cherry picked from commit bae49ff)
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-03-30 14:47:55 +02:00
Sebastiaan van Stijn
3923e302a4
update runc binary to v1.1.5
release notes: https://github.com/opencontainers/runc/releases/tag/v1.1.5
diff: https://github.com/opencontainers/runc/compare/v1.1.4...v1.1.5

This is the fifth patch release in the 1.1.z series of runc, which fixes
three CVEs found in runc.

* CVE-2023-25809 is a vulnerability involving rootless containers where
  (under specific configurations), the container would have write access
  to the /sys/fs/cgroup/user.slice/... cgroup hierarchy. No other
  hierarchies on the host were affected. This vulnerability was
  discovered by Akihiro Suda.
  <https://github.com/opencontainers/runc/security/advisories/GHSA-m8cg-xc2p-r3fc>

* CVE-2023-27561 was a regression which effectively re-introduced
  CVE-2019-19921. This bug was present from v1.0.0-rc95 to v1.1.4. This
  regression was discovered by @Beuc.
  <https://github.com/advisories/GHSA-vpvm-3wq2-2wvm>

* CVE-2023-28642 is a variant of CVE-2023-27561 and was fixed by the same
  patch. This variant of the above vulnerability was reported by Lei
  Wang.
  <https://github.com/opencontainers/runc/security/advisories/GHSA-g2j6-57v7-gm8c>

In addition, the following other fixes are included in this release:

* Fix the inability to use `/dev/null` when inside a container.
* Fix changing the ownership of host's `/dev/null` caused by fd redirection
  (a regression in 1.1.1).
* Fix rare runc exec/enter unshare error on older kernels, including
  CentOS < 7.7.
* nsexec: Check for errors in `write_log()`.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 77be7b777c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-03-30 14:45:16 +02:00
Paweł Gronowski
f3946c14fe
integration-cli: Enable TestExecStartFails on Windows
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit be34e93f20)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-03-28 11:10:24 +02:00
Bjorn Neergaard
219f21bf07
Merge pull request #45196 from vvoland/integration-restart-race-23
[backport 23.0] TestDaemonRestartKillContainers: Fix races
2023-03-23 12:19:59 -06:00
Cory Snider
b87f7f18b8 libnet/d/overlay: insert the input-drop rule
FirewallD creates the root INPUT chain with a default-accept policy and
a terminal rule which rejects all packets not accepted by any prior
rule. Any subsequent rules appended to the chain are therefore inert.
The administrator would have to open the VXLAN UDP port to make overlay
networks work at all, which would result in all VXLAN traffic being
accepted and defeating our attempts to enforce encryption on encrypted
overlay networks.

Insert the rule to drop unencrypted VXLAN packets tagged for encrypted
overlay networks at the top of the INPUT chain so that enforcement of
mandatory encryption takes precedence over any accept rules configured
by the administrator. Continue to append the accept rule to the bottom
of the chain so as not to override any administrator-configured drop
rules.

Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit 965eda3b9a)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-03-22 20:54:32 -04:00
Paweł Gronowski
c6bf3071fe
StartWithLogFile: Fix d.cmd race
Use `exec.Command` created by this function instead of obtaining it from
daemon struct. This prevents a race condition where `daemon.Kill` is
called before the goroutine has the chance to call `cmd.Wait`.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 88992de283)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-03-22 15:36:44 +01:00
Paweł Gronowski
7f49ca259b
TestDaemonRestartKillContainers: Fix loop capture
TestDaemonRestartKillContainers test was always executing the last case
(`container created should not be restarted`) because the iterated
variables were not copied correctly.
Capture iterated values by value correctly and rename c to tc.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit fed1c96e10)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-03-22 15:36:41 +01:00
Cory Snider
98cbcb8003 libnet/d/overlay: add BPF-powered VNI matcher
Some newer distros such as RHEL 9 have stopped making the xt_u32 kernel
module available with the kernels they ship. They do ship the xt_bpf
kernel module, which can do everything xt_u32 can and more. Add an
alternative implementation of the iptables match rule which uses xt_bpf
to implement exactly the same logic as the u32 filter using a BPF
program. Try programming the BPF-powered rules as a fallback when
programming the u32-powered rules fails.

Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit 105b9834fb)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-03-21 11:22:16 -04:00
Cory Snider
5c5fac2374 libnet/d/overlay: extract VNI match rule builder
The iptables rule clause used to match on the VNI of VXLAN datagrams
looks like line noise to the uninitiated. It doesn't help that the
expression is repeated twice and neither copy has any commentary.
DRY out the rule builder to a common function, and document what the
rule does and how it works.

Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit 44cf27b5fc)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-03-21 11:22:16 -04:00
Cory Snider
c492a22287 libn/d/overlay: enforce encryption on sandbox init
The iptables rules which make encryption mandatory on an encrypted
overlay network are only programmed once there is a second node
participating in the network. This leaves single-node encrypted overlay
networks vulnerable to packet injection. Furthermore, failure to program
the rules is not treated as a fatal error.

Program the iptables rules to make encryption mandatory before creating
the VXLAN link to guarantee that there is no window of time where
incoming cleartext VXLAN packets for the network would be accepted, or
outgoing cleartext packets be transmitted. Only create the VXLAN link if
programming the rules succeeds to ensure that it fails closed.

Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit 142f46cac1)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-03-21 11:22:13 -04:00
Cory Snider
018edb0284 libnet/d/overlay: document some encryption code
The overlay-network encryption code is woefully under-documented, which
is especially problematic as it operates on under-documented kernel
interfaces. Document what I have puzzled out of the implementation for
the benefit of the next poor soul to touch this code.

Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit d4fd582fb2)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-03-21 11:21:05 -04:00
Bjorn Neergaard
a1fd2f22f6
Merge pull request #45157 from thaJeztah/23.0_backport_update_shfmt
[23.0 backport] Dockerfile: Update shfmt to 3.6.0
2023-03-16 12:13:15 -06:00
Bjorn Neergaard
07f9061dde
Merge pull request #45112 from tonistiigi/23.0-disable-mergeop-diffop
[23.0 backport] builder-next: temporarily disable mergeop and diffop
2023-03-16 12:12:17 -06:00
Bjorn Neergaard
1363b3e372
ci: disable testing mergeop and diffop with the dockerd worker
Co-authored-by: Tonis Tiigi <tonistiigi@gmail.com>
Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
2023-03-16 08:30:08 -06:00
Tonis Tiigi
d43958b50b
builder-next: disable mergeop and diffop
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
(cherry picked from commit 0ac3bf837b)
Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
2023-03-16 08:27:26 -06:00
Brian Goff
9740b6854d
Merge pull request #45159 from thaJeztah/23.0_backport_fix_volume_anon_from_image
[23.0 backport] Fix pruning anon volume created from image config
2023-03-14 23:06:03 +00:00
Brian Goff
fd80ca60da
Fix pruning anon volume created from image config
Volumes created from the image config were not being pruned because the
volume service did not think they were anonymous since the code to
create passes along a generated name instead of letting the volume
service generate it.

This changes the code path to have the volume service generate the name
instead of doing it ahead of time.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit 146df5fbd3)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-03-14 16:11:09 +01:00
Sebastiaan van Stijn
f09528b712
Merge pull request #45155 from thaJeztah/23.0_backport_fix_volume_error_handling
[23.0 backport] volumes: fix error-handling when removing volumes with swarm enabled
2023-03-14 16:10:22 +01:00
Paweł Gronowski
ae0cc08cab
Dockerfile: Update shfmt to 3.6.0
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 4026148ef1)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-03-14 11:58:48 +01:00
Paweł Gronowski
42751cc892
contrib/dockerize-disk.sh: Fix indentation
Reported by shfmt

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit a0741cacef)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-03-14 11:58:45 +01:00
Sebastiaan van Stijn
6c65a9a07f
volumes: fix error-handling when removing volumes with swarm enabled
Commit 3246db3755 added handling for removing
cluster volumes, but in some conditions, this resulted in errors not being
returned if the volume was in use;

    docker swarm init
    docker volume create foo
    docker create -v foo:/foo busybox top
    docker volume rm foo

This patch changes the logic for ignoring "local" volume errors if swarm
is enabled (and cluster volumes supported).

While working on this fix, I also discovered that Cluster.RemoveVolume()
did not handle the "force" option correctly; while swarm correctly handled
these, the cluster backend performs a lookup of the volume first (to obtain
its ID), which would fail if the volume didn't exist.

Before this patch:

    make TEST_FILTER=TestVolumesRemoveSwarmEnabled DOCKER_GRAPHDRIVER=vfs test-integration
    ...
    Running /go/src/github.com/docker/docker/integration/volume (arm64.integration.volume) flags=-test.v -test.timeout=10m  -test.run TestVolumesRemoveSwarmEnabled
    ...
    === RUN   TestVolumesRemoveSwarmEnabled
    === PAUSE TestVolumesRemoveSwarmEnabled
    === CONT  TestVolumesRemoveSwarmEnabled
    === RUN   TestVolumesRemoveSwarmEnabled/volume_in_use
        volume_test.go:122: assertion failed: error is nil, not errdefs.IsConflict
        volume_test.go:123: assertion failed: expected an error, got nil
    === RUN   TestVolumesRemoveSwarmEnabled/volume_not_in_use
    === RUN   TestVolumesRemoveSwarmEnabled/non-existing_volume
    === RUN   TestVolumesRemoveSwarmEnabled/non-existing_volume_force
        volume_test.go:143: assertion failed: error is not nil: Error response from daemon: volume no_such_volume not found
    --- FAIL: TestVolumesRemoveSwarmEnabled (1.57s)
        --- FAIL: TestVolumesRemoveSwarmEnabled/volume_in_use (0.00s)
        --- PASS: TestVolumesRemoveSwarmEnabled/volume_not_in_use (0.01s)
        --- PASS: TestVolumesRemoveSwarmEnabled/non-existing_volume (0.00s)
        --- FAIL: TestVolumesRemoveSwarmEnabled/non-existing_volume_force (0.00s)
    FAIL

With this patch:

    make TEST_FILTER=TestVolumesRemoveSwarmEnabled DOCKER_GRAPHDRIVER=vfs test-integration
    ...
    Running /go/src/github.com/docker/docker/integration/volume (arm64.integration.volume) flags=-test.v -test.timeout=10m  -test.run TestVolumesRemoveSwarmEnabled
    ...
    make TEST_FILTER=TestVolumesRemoveSwarmEnabled DOCKER_GRAPHDRIVER=vfs test-integration
    ...
    Running /go/src/github.com/docker/docker/integration/volume (arm64.integration.volume) flags=-test.v -test.timeout=10m  -test.run TestVolumesRemoveSwarmEnabled
    ...
    === RUN   TestVolumesRemoveSwarmEnabled
    === PAUSE TestVolumesRemoveSwarmEnabled
    === CONT  TestVolumesRemoveSwarmEnabled
    === RUN   TestVolumesRemoveSwarmEnabled/volume_in_use
    === RUN   TestVolumesRemoveSwarmEnabled/volume_not_in_use
    === RUN   TestVolumesRemoveSwarmEnabled/non-existing_volume
    === RUN   TestVolumesRemoveSwarmEnabled/non-existing_volume_force
    --- PASS: TestVolumesRemoveSwarmEnabled (1.53s)
        --- PASS: TestVolumesRemoveSwarmEnabled/volume_in_use (0.00s)
        --- PASS: TestVolumesRemoveSwarmEnabled/volume_not_in_use (0.01s)
        --- PASS: TestVolumesRemoveSwarmEnabled/non-existing_volume (0.00s)
        --- PASS: TestVolumesRemoveSwarmEnabled/non-existing_volume_force (0.00s)
    PASS

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 058a31e479)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-03-14 11:38:04 +01:00
Sebastiaan van Stijn
e3c642d1ea
integration/volumes: TestVolumesRemove: add coverage for force/no-force
Add additional test-cases for deleting non-existing volumes (with/without force).

With this patch:

    make TEST_FILTER=TestVolumesRemove DOCKER_GRAPHDRIVER=vfs test-integration

    Running /go/src/github.com/docker/docker/integration/volume (arm64.integration.volume) flags=-test.v -test.timeout=10m  -test.run TestVolumesRemove
    ...
    === RUN   TestVolumesRemove
    === RUN   TestVolumesRemove/volume_in_use
    === RUN   TestVolumesRemove/volume_not_in_use
    === RUN   TestVolumesRemove/non-existing_volume
    === RUN   TestVolumesRemove/non-existing_volume_force
    --- PASS: TestVolumesRemove (0.04s)
        --- PASS: TestVolumesRemove/volume_in_use (0.00s)
        --- PASS: TestVolumesRemove/volume_not_in_use (0.01s)
        --- PASS: TestVolumesRemove/non-existing_volume (0.00s)
        --- PASS: TestVolumesRemove/non-existing_volume_force (0.00s)
    PASS

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 7531f05c7c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-03-14 11:36:54 +01:00
Sebastiaan van Stijn
59e89b9e82
Merge pull request #45137 from thaJeztah/23.0_bump_go1.19.7
[23.0] update to go1.19.7
2023-03-10 14:14:32 +01:00
Sebastiaan van Stijn
c48f7fd12c
[23.0] update to go1.19.7
Includes a security fix for crypto/elliptic (CVE-2023-24532).

> go1.19.7 (released 2023-03-07) includes a security fix to the crypto/elliptic
> package, as well as bug fixes to the linker, the runtime, and the crypto/x509
> and syscall packages. See the Go 1.19.7 milestone on our issue tracker for
> details.

https://go.dev/doc/devel/release#go1.19.minor

From the announcement:

> We have just released Go versions 1.20.2 and 1.19.7, minor point releases.
>
> These minor releases include 1 security fixes following the security policy:
>
> - crypto/elliptic: incorrect P-256 ScalarMult and ScalarBaseMult results
    >
    >   The ScalarMult and ScalarBaseMult methods of the P256 Curve may return an
    >   incorrect result if called with some specific unreduced scalars (a scalar larger
    >   than the order of the curve).
    >
    >   This does not impact usages of crypto/ecdsa or crypto/ecdh.
>
> This is CVE-2023-24532 and Go issue https://go.dev/issue/58647.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-03-10 12:26:47 +01:00
Akihiro Suda
31a9532128
Merge pull request #45131 from thaJeztah/23.0_backport_bump_swarmkit
[23.0 backport] vendor: github.com/moby/swarmkit/v2 v2.0.0-20230309194213-a745a8755ce3
2023-03-10 19:31:00 +09:00
Sebastiaan van Stijn
f3416271a3
vendor: github.com/moby/swarmkit/v2 v2.0.0-20230309194213-a745a8755ce3
no changes in vendored code; only aligning dependencies

full diff: 80a528a868...a745a8755c

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 3bbffe96e1)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-03-09 23:22:36 +01:00
Sebastiaan van Stijn
6cb40fef42
Merge pull request #45107 from thaJeztah/23.0_backport_bump_swarmkit
[23.0 backport] vendor: github.com/moby/swarmkit/v2 v2.0.0-20230302163403-80a528a86877
2023-03-09 23:18:34 +01:00
Sebastiaan van Stijn
3bd2fc8d62
vendor: github.com/moby/swarmkit/v2 v2.0.0-20230302163403-80a528a86877
- fix docker service create doesn't work when network and generic-resource are both attached
- Fix removing tasks when a jobs service is removed
- CSI: Allow NodePublishVolume even when plugin does not support staging

full diff: 904c221ac2...80a528a868

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 088aff1620)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-03-07 12:10:12 +01:00
Brian Goff
d77ac3d8cc
Merge pull request #45110 from thaJeztah/23.0_update_buildkit
[23.0] vendor: github.com/moby/buildkit v0.10.7-0.20230306143919-70f2ad56d3e5
2023-03-06 20:54:30 +00:00
Sebastiaan van Stijn
a79be4cf31
vendor: github.com/moby/buildkit v0.10.7-0.20230306143919-70f2ad56d3e5
- buildinfo: ensure URLs are redacted before written (fixes CVE-2023-26054)

full diff: 4f0ee09c40...70f2ad56d3

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-03-06 17:25:54 +01:00
Sebastiaan van Stijn
6962a28bc8
Merge pull request #45100 from thaJeztah/23.0_backport_fix_ipam_flaky_test
[23.0 backport] libnet/ipam: fix racy, flaky unit test
2023-03-06 12:39:50 +01:00
Sebastiaan van Stijn
f8b2777872
Merge pull request #45099 from thaJeztah/23.0_backport_vendor_containerd_1.6.19
[23.0 backport] vendor: github.com/Microsoft/hcsshim v0.9.7, github.com/containerd/containerd v1.6.19
2023-03-03 18:37:04 +01:00
Cory Snider
6ac39568ee
libnet/ipam: fix racy, flaky unit test
TestRequestReleaseAddressDuplicate gets flagged by go test -race because
the same err variable inside the test is assigned to from multiple
goroutines without synchronization, which obscures whether or not there
are any data races in the code under test.

Trouble is, the test _depends on_ the data race to exit the loop if an
error occurs inside a spawned goroutine. And the test contains a logical
concurrency bug (not flagged by the Go race detector) which can result
in false-positive test failures. Because a release operation is logged
after the IP is released, the other goroutine could reacquire the
address and log that it was reacquired before the release is logged.

Fix up the test so it is no longer subject to data races or
false-positive test failures, i.e. flakes.

Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit b62445871e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-03-03 14:55:30 +01:00
Sebastiaan van Stijn
7158f76b8b
vendor: github.com/containerd/containerd v1.6.19
Update hcsshim to v0.9.7 to include fix for graceful termination and pause containers

full diff: https://github.com/containerd/containerd/compare/v1.6.18...v1.6.19

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit ef6f5367dc)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-03-03 14:50:53 +01:00
Sebastiaan van Stijn
67990f983b
vendor: github.com/Microsoft/hcsshim v0.9.7
- Retain pause.exe as entrypoint for default pause images
- wcow: support graceful termination of servercore containers

full diff: https://github.com/Microsoft/hcsshim/compare/v0.9.6...v0.9.7

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 5997ad8512)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-03-03 14:50:50 +01:00
Sebastiaan van Stijn
9401313448
Merge pull request #45084 from thaJeztah/23.0_update_containerd_vendor_1.6.19
[23.0] update containerd binary to v1.6.19
2023-03-01 12:25:12 +01:00
Sebastiaan van Stijn
3125aa0aef
[23.0] update containerd binary to v1.6.19
Update hcsshim to v0.9.7 to include fix for graceful termination and pause containers

- release notes: https://github.com/containerd/containerd/releases/tag/v1.6.19
- full diff: https://github.com/containerd/containerd/compare/v1.6.18...v1.6.19

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-02-28 22:56:02 +01:00
Cory Snider
fc59ac02ff
Merge pull request #45078 from corhere/backport-23.0/bump-exit-timeouts-for-runhcs
[23.0 backport] Wait longer for exit events on Windows
2023-02-28 12:22:34 -05:00
Cory Snider
b1d9012969 Wait longer for exit events on Windows
The latest version of containerd-shim-runhcs-v1 (v0.10.0-rc.4) pulled in
with the bump to ContainerD v1.7.0-rc.3 had several changes to make it
more robust, which had the side effect of increasing the worst-case
amount of time it takes for a container to exit in the worst case.
Notably, the total timeout for shutting down a task increased from 30
seconds to 60! Increase the timeouts hardcoded in the daemon and
integration tests so that they don't give up too soon.

Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit d634ae9b60)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-02-27 17:41:37 -05:00
Cory Snider
bfc8e1ae36
Merge pull request #45071 from corhere/backport-23.0/libnet/fix-networkdb-test-panic
[23.0 backport] libnet/networkdb: fix nil-dereference panic in test
2023-02-23 19:14:27 -05:00
Cory Snider
2337258d28 libnet/networkdb: fix nil-dereference panic in test
Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit 88f6b637a0)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-02-23 18:07:21 -05:00
Bjorn Neergaard
bb063342ca
Merge pull request #45009 from corhere/backport-23.0/runtime-eisdir
[23.0 backport] Go 1.20 enablement
2023-02-22 20:43:56 -07:00
Cory Snider
d6d48dd95d Upgrade to golangci-lint v1.51.2
Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit 3606d6a7cd)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-02-22 16:33:13 -05:00
Cory Snider
c4b655830d Migrate away from things deprecated in Go 1.20
"math/rand".Seed
  - Migrate to using local RNG instances.

"archive/tar".TypeRegA
  - The deprecated constant tar.TypeRegA is the same value as
    tar.TypeReg and so is not needed at all.

Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit dea3f2b417)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-02-22 16:32:33 -05:00
Cory Snider
c6bf777eae d/l/awslogs: fix ineffective Add in test
...flagged by golangci-lint v1.51.1 (staticcheck).

Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit e66995d840)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-02-22 16:32:32 -05:00
Cory Snider
da10937926 daemon: handle EISDIR error from runtime
Go 1.20 made a change to the behaviour of package "os/exec" which was
not mentioned in the release notes:
2b8f214094
Attempts to execute a directory now return syscall.EISDIR instead of
syscall.EACCESS. Check for EISDIR errors from the runtime and fudge the
returned error message to maintain compatibility with existing versions
of docker/cli when using a version of runc compiled with Go 1.20+.

Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit 713e02e03e)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-02-22 16:32:32 -05:00
Cory Snider
d74ef58c59
Merge pull request #45062 from corhere/backport-23.0/fix-loopclosure-test-bugs
[23.0 backport] Fix loop-closure bugs in tests
2023-02-22 16:31:13 -05:00
Sebastiaan van Stijn
0497993aba
Merge pull request #45051 from corhere/backport-23.0/containerd-fifo_v1.1
[23.0 backport] Upgrade containerd/fifo to v1.1.0
2023-02-22 19:35:29 +01:00
Cory Snider
938ed9a1ed distribution/xfer: make off-by-one error a feature
maxDownloadAttempts maps to the daemon configuration flag

    --max-download-attempts int
      Set the max download attempts for each pull (default 5)

and the daemon configuration machinery interprets a value of 0 as "apply
the default value" and not a valid user value (config validation/
normalization bugs notwithstanding). The intention is clearly that this
configuration value should be an upper limit on the number of times the
daemon should try to download a particular layer before giving up. So it
is surprising to have the configuration value interpreted as a _retry_
limit. The daemon will make up to N+1 attempts to download a layer! This
also means users cannot disable retries even if they wanted to.

As this is a longstanding bug, not a recent regression, it would not be
appropriate to backport the fix (97921915a8)
in a patch release. Update the test to assert on the buggy behaviour so
it passes again.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-02-22 12:34:59 -05:00
Cory Snider
ca712d6947 Fix loop-closure bugs in tests
...which were flagged by golangci-lint v1.51.

Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit 0c68b655f6)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-02-22 12:26:38 -05:00
Cory Snider
2c55b264f7
Upgrade containerd/fifo to v1.1.0
Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit d6c4e17411)
Signed-off-by: Cory Snider <csnider@mirantis.com>
Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
2023-02-22 10:08:15 -07:00
Bjorn Neergaard
3627bac886
Merge pull request #45058 from thaJeztah/23.0_backport_bump_bbolt_1.3.7_20230217
[23.0 backport] dependency: bump go.etcd.io/bbolt to v1.3.7
2023-02-22 10:08:04 -07:00
Benjamin Wang
8fd038fb71
dependency: bump go.etcd.io/bbolt to v1.3.7
Please refer to link below to get more detailed info on bbolt@v1.3.7,
- https://github.com/etcd-io/bbolt/blob/master/CHANGELOG/CHANGELOG-1.3.md#v1372023-01-31

Signed-off-by: Benjamin Wang <wachao@vmware.com>
(cherry picked from commit 4a886a3e8f)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-02-22 17:12:50 +01:00
Bjorn Neergaard
ab7dd5934e
Merge pull request #45056 from thaJeztah/23.0_backport_bump_golang_net
[23.0 backport] vendor: golang.org/x/net v0.7.0, golang.org/x/sys v0.5.0, golang.org/x/text v0.7.0
2023-02-22 08:35:12 -07:00
Sebastiaan van Stijn
87a1517f8f
vendor: golang.org/x/net v0.7.0
This addresses the same CVE as is patched in go1.19.6. From that announcement:

> net/http: avoid quadratic complexity in HPACK decoding
>
> A maliciously crafted HTTP/2 stream could cause excessive CPU consumption
> in the HPACK decoder, sufficient to cause a denial of service from a small
> number of small requests.
>
> This issue is also fixed in golang.org/x/net/http2 v0.7.0, for users manually
> configuring HTTP/2.
>
> This is CVE-2022-41723 and Go issue https://go.dev/issue/57855.

full diff: https://github.com/golang/net/compare/v0.5.0...v0.7.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit a36286cf89)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-02-22 13:33:56 +01:00
Sebastiaan van Stijn
d15010643c
vendor: golang.org/x/text v0.7.0
full diff: https://github.com/golang/text/compare/v0.6.0...v0.7.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit c7de76569e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-02-22 13:33:46 +01:00
Sebastiaan van Stijn
0727310950
vendor: golang.org/x/sys v0.5.0
full diff: https://github.com/golang/sys/compare/v0.4.0...v0.5.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit a53b44a266)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-02-22 13:33:32 +01:00
Bjorn Neergaard
9f496914d0
Merge pull request #45043 from neersighted/backport/44982/23.0
[23.0 backport] daemon: fully resolve `apparmor_parser` regression
2023-02-21 08:19:04 -07:00
Brian Goff
bb687c160a
Merge pull request #45022 from thaJeztah/23.0_containerd_binary_1.6.18
[23.0] update containerd binary to v1.6.18
2023-02-20 18:33:11 +00:00
Bjorn Neergaard
b8c448ef24
Revert "apparmor: Check if apparmor_parser is available"
This reverts commit ab3fa46502.

This fix was partial, and is not needed with the proper fix in
containerd.

Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
(cherry picked from commit c4c54683a9)
Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
2023-02-20 08:59:24 -07:00
Bjorn Neergaard
a3f5319563
vendor: github.com/containerd/containerd v1.6.18
Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
(cherry picked from commit 174802e15f)
Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
2023-02-20 08:59:20 -07:00
Sebastiaan van Stijn
d7d4866d70
Merge pull request #45021 from thaJeztah/23.0_backport_bump_go_1.19.6
[23.0 backport] update to go1.19.6
2023-02-17 13:51:09 +01:00
Bjorn Neergaard
a4e9b25461
Merge pull request #45020 from corhere/backport-23.0/libnetwork-test-race
[23.0 backport] libnetwork/networkdb: make go test -race ./libnetwork/networkdb pass
2023-02-16 18:15:37 -07:00
Sebastiaan van Stijn
52d667794f
[23.0] update containerd binary to v1.6.18
update containerd binary to v1.6.18, which includes fixes for CVE-2023-25153
and CVE-2023-25173.

1.6.18 release notes: https://github.com/containerd/containerd/releases/tag/v1.6.18

> - Fix OCI image importer memory exhaustion (GHSA-259w-8hf6-59c2)
> - Fix supplementary groups not being set up properly (GHSA-hmfx-3pcx-653p)
> - Revert removal of /sbin/apparmor_parser check
> - Update Go to 1.19.6

full diff: https://github.com/containerd/containerd/compare/v1.6.17...v1.6.18

1.6.17 release notes: https://github.com/containerd/containerd/releases/tag/v1.6.17

> - Add network plugin metrics
> - Update mkdir permission on /etc/cni to 0755 instead of 0700
> - Export remote snapshotter label handler
> - Add support for default hosts.toml configuration

full diff: https://github.com/containerd/containerd/compare/v1.6.16...v1.6.17

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-02-17 01:23:43 +01:00
Sebastiaan van Stijn
11715a05ca
update to go1.19.6
go1.19.6 (released 2023-02-14) includes security fixes to the crypto/tls,
mime/multipart, net/http, and path/filepath packages, as well as bug fixes to
the go command, the linker, the runtime, and the crypto/x509, net/http, and
time packages. See the Go 1.19.6 milestone on our issue tracker for details:

https://github.com/golang/go/issues?q=milestone%3AGo1.19.6+label%3ACherryPickApproved

From the announcement on the security mailing:

We have just released Go versions 1.20.1 and 1.19.6, minor point releases.

These minor releases include 4 security fixes following the security policy:

- path/filepath: path traversal in filepath.Clean on Windows

  On Windows, the filepath.Clean function could transform an invalid path such
  as a/../c:/b into the valid path c:\b. This transformation of a relative (if
  invalid) path into an absolute path could enable a directory traversal attack.
  The filepath.Clean function will now transform this path into the relative
  (but still invalid) path .\c:\b.

  This is CVE-2022-41722 and Go issue https://go.dev/issue/57274.

- net/http, mime/multipart: denial of service from excessive resource
  consumption

  Multipart form parsing with mime/multipart.Reader.ReadForm can consume largely
  unlimited amounts of memory and disk files. This also affects form parsing in
  the net/http package with the Request methods FormFile, FormValue,
  ParseMultipartForm, and PostFormValue.

  ReadForm takes a maxMemory parameter, and is documented as storing "up to
  maxMemory bytes +10MB (reserved for non-file parts) in memory". File parts
  which cannot be stored in memory are stored on disk in temporary files. The
  unconfigurable 10MB reserved for non-file parts is excessively large and can
  potentially open a denial of service vector on its own. However, ReadForm did
  not properly account for all memory consumed by a parsed form, such as map
  ntry overhead, part names, and MIME headers, permitting a maliciously crafted
  form to consume well over 10MB. In addition, ReadForm contained no limit on
  the number of disk files created, permitting a relatively small request body
  to create a large number of disk temporary files.

  ReadForm now properly accounts for various forms of memory overhead, and
  should now stay within its documented limit of 10MB + maxMemory bytes of
  memory consumption. Users should still be aware that this limit is high and
  may still be hazardous.

  ReadForm now creates at most one on-disk temporary file, combining multiple
  form parts into a single temporary file. The mime/multipart.File interface
  type's documentation states, "If stored on disk, the File's underlying
  concrete type will be an *os.File.". This is no longer the case when a form
  contains more than one file part, due to this coalescing of parts into a
  single file. The previous behavior of using distinct files for each form part
  may be reenabled with the environment variable
  GODEBUG=multipartfiles=distinct.

  Users should be aware that multipart.ReadForm and the http.Request methods
  that call it do not limit the amount of disk consumed by temporary files.
  Callers can limit the size of form data with http.MaxBytesReader.

  This is CVE-2022-41725 and Go issue https://go.dev/issue/58006.

- crypto/tls: large handshake records may cause panics

  Both clients and servers may send large TLS handshake records which cause
  servers and clients, respectively, to panic when attempting to construct
  responses.

  This affects all TLS 1.3 clients, TLS 1.2 clients which explicitly enable
  session resumption (by setting Config.ClientSessionCache to a non-nil value),
  and TLS 1.3 servers which request client certificates (by setting
  Config.ClientAuth
  > = RequestClientCert).

  This is CVE-2022-41724 and Go issue https://go.dev/issue/58001.

- net/http: avoid quadratic complexity in HPACK decoding

  A maliciously crafted HTTP/2 stream could cause excessive CPU consumption
  in the HPACK decoder, sufficient to cause a denial of service from a small
  number of small requests.

  This issue is also fixed in golang.org/x/net/http2 v0.7.0, for users manually
  configuring HTTP/2.

  This is CVE-2022-41723 and Go issue https://go.dev/issue/57855.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 94feb31516)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-02-17 01:17:42 +01:00
Cory Snider
962c238c17 libnet/networkdb: use atomics for stats counters
The per-network statistics counters are loaded and incremented without
any concurrency control. Use atomic integers to prevent data races
without having to add any synchronization.

Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit d31fa84c7c)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-02-16 19:16:36 -05:00
Tibor Vass
20d05e235e libnetwork/networkdb: make go test -race ./libnetwork/networkdb pass
Signed-off-by: Tibor Vass <teabee89@gmail.com>
Co-authored-by: Cory Snider <csnider@mirantis.com>
Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit 3539452ef0)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-02-16 19:16:36 -05:00
Bjorn Neergaard
6bf50c09a4
Merge pull request #45016 from corhere/backport-23.0/fix-spammy-cluster-api-logs
[23.0 backport] api/s/r/swarm: log backend errors at Debug level
2023-02-16 16:23:01 -07:00
Cory Snider
c2d69d06b0 api/s/r/swarm: log backend errors at Debug level
The errors are already returned to the client in the API response, so
logging them to the daemon log is redundant. Log the errors at level
Debug so as not to pollute the end-users' daemon logs with noise.

Refactor the logs to use structured fields. Add the request context to
the log entry so that logrus hooks could annotate the log entries with
contextual information about the API request in the hypothetical future.

Fixes #44997

Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit a4e3c67e44)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-02-16 11:30:06 -05:00
Cory Snider
111658c7f0
Merge pull request #45000 from corhere/backport-23.0/fix-dns-opt-host-loopback
[23.0 backport] libnetwork: check DNS loopback with user DNS opts
2023-02-14 16:16:49 -05:00
Cory Snider
7c09feb58c libnetwork: check DNS loopback with user DNS opts
DNS servers in the loopback address range should always be resolved in
the host network namespace when the servers are configured by reading
from the host's /etc/resolv.conf. The daemon mistakenly conflated the
presence of DNS options (docker run --dns-opt) with user-supplied DNS
servers, treating the list of servers loaded from the host as a user-
supplied list and attempting to resolve in the container's network
namespace. Correct this oversight so that loopback DNS servers are only
resolved in the container's network namespace when the user provides the
DNS server list, irrespective of other DNS configuration.

Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit 046cc9e776)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-02-14 14:51:12 -05:00
Cory Snider
ecd940a0c6
Merge pull request #44980 from er0k/bad-dns
[23.0] Fix nil pointer dereference when attempting to log DNS errors
2023-02-14 12:25:20 -05:00
er0k
6c2637be11
Do not log connection info before the connection exists
If the resolver encounters an error before it attempts to forward the
request to external DNS, do not try to log information about the
external connection, because at this point `extConn` is `nil`. This
makes sure `dockerd` won't panic and crash from a nil pointer
dereference when it sees an invalid DNS query.

fixes #44979

Signed-off-by: er0k <er0k@er0k.net>
2023-02-13 12:48:41 -05:00
Bjorn Neergaard
bc3805a0a0
Merge pull request #44959 from tonistiigi/23.0-buildkit-update-0208
[23.0] vendor: update buildkit to latest v0.10
2023-02-09 12:11:28 -07:00
Tonis Tiigi
4002fa877b
vendor: update buildkit to latest v0.10
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2023-02-08 07:57:57 -08:00
Bjorn Neergaard
77f3d31ed6
Merge pull request #44944 from thaJeztah/23.0_backport_worker_getremotes
[23.0 backport] builder: define GetRemotes for the worker
2023-02-08 08:44:21 -07:00
Tonis Tiigi
e697c9365f
builder: define GetRemotes for the worker
The function signature has changed since v0.10.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit 335907d187)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-02-08 01:01:18 +01:00
Sebastiaan van Stijn
1141704bc9
Merge pull request #44952 from thaJeztah/23.0_backport_ci_fix_workflows
[23.0 backport] ci: fix branch filter pattern in buildkit workflow
2023-02-08 00:59:40 +01:00
CrazyMax
4c544c3d5f
ci: fix branch filter pattern in buildkit workflow
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 80f8432b52)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-02-07 23:45:04 +01:00
Bjorn Neergaard
f7e2cbaf1b
Merge pull request #44948 from crazy-max/23.0_backport_ci-itg-cli-split
[23.0 backport] ci: enhance tests distribution
2023-02-07 12:43:42 -07:00
CrazyMax
06aca27cc1
ci: enhance tests distribution
Adds overrides with specific tests suites in our tests
matrix so we can reduce build time significantly.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 22776f8fdb)
2023-02-07 16:50:50 +01:00
Kevin Alvarez
001b2e778a
ci: update gotestlist to 0.3.1
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit e304e82b66)
2023-02-07 16:50:49 +01:00
Sebastiaan van Stijn
e664cc2239
Merge pull request #44942 from vvoland/apparmor-check-binary-23
[23.0 backport] apparmor: Check if apparmor_parser is available
2023-02-07 13:13:18 +01:00
Sebastiaan van Stijn
a25ab9224a
Merge pull request #44936 from thaJeztah/23.0_update_buildkit
[23.0] vendor: github.com/moby/buildkit v0.10.7-0.20230206124303-b8fdb4b78da0
2023-02-07 11:29:35 +01:00
Paweł Gronowski
38b70ebc3b
apparmor: Check if apparmor_parser is available
`hostSupports` doesn't check if the apparmor_parser is available.
It's possible in some environments that the apparmor will be enabled but
the tool to load the profile is not available which will cause the
ensureDefaultAppArmorProfile to fail completely.

This patch checks if the apparmor_parser is available. Otherwise the
function returns early, but still logs a warning to the daemon log.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit ab3fa46502)
2023-02-07 11:23:06 +01:00
Sebastiaan van Stijn
8a5d341d94
[23.0] vendor: github.com/moby/buildkit v0.10.7-0.20230206124303-b8fdb4b78da0
full diff: https://github.com/moby/buildkit/compare/v0.10.6...b8fdb4b78da0127260ba39b9b7de048cb3877203

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-02-07 09:55:22 +01:00
Bjorn Neergaard
fc5d6c97b3
Merge pull request #44937 from corhere/backport-23.0/fix-44925-ipvlan-panic
[23.0 backport] libnet/d/ipvlan: gracefully migrate from older dbs
2023-02-06 22:56:35 -07:00
Cory Snider
427101f656 libnet/d/ipvlan: gracefully migrate from older dbs
IPVLAN networks created on Moby v20.10 do not have the IpvlanFlag
configuration value persisted in the libnetwork database as that config
value did not exist before v23.0.0. Gracefully migrate configurations on
unmarshal to prevent type-assertion panics at daemon start after upgrade.

Fixes #44925

Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit 91725ddc92)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-02-06 18:16:21 -05:00
Sebastiaan van Stijn
b67344b81c
Merge pull request #44938 from thaJeztah/23.0_backport_dockerfile_git_dubious
[23.0 backport] Dockerfile: configure code dir as "safe" directory
2023-02-07 00:14:30 +01:00
Sebastiaan van Stijn
5b7142073f
Dockerfile: configure code dir as "safe" directory
CI is failing when bind-mounting source from the host into the dev-container;

    fatal: detected dubious ownership in repository at '/go/src/github.com/docker/docker'
    To add an exception for this directory, call:

        git config --global --add safe.directory /go/src/github.com/docker/docker

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 21677816a0)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-02-06 21:08:42 +01:00
Bjorn Neergaard
bacf32d9ab
Merge pull request #44892 from corhere/backport-23.0/fix-kata-exec-exit
[23.0 backport] Fix exit-event handling for Kata runtime
2023-02-06 12:02:50 -07:00
Brian Goff
7df264966b
Merge pull request #44922 from neersighted/backport/44916/23.0
[23.0 backport] graphdriver/overlay2: usingMetacopy ENOTSUP is non-fatal
2023-02-04 06:54:29 -08:00
Bjorn Neergaard
7e03250780
graphdriver/overlay2: usingMetacopy ENOTSUP is non-fatal
Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
(cherry picked from commit 3bcb350711)
Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
2023-02-04 00:07:08 -07:00
Cory Snider
975bdb2c96 daemon: identify container exits by ProcessID
The Pid field of an exit event cannot be relied upon to differentiate
exits of the container's task from exits of other container processes,
i.e. execs. The Pid is reported by the runtime and is implementation-
defined so there is no guarantee that a task's pid is distinct from the
pids of any other process in the same container. In particular,
kata-containers reports the pid of the hypervisor for all exit events.
Update the daemon to differentiate container exits from exec exits by
inspecting the event's ProcessID.

The local_windows libcontainerd implementation already sets the
ProcessID to InitProcessName on container exit events. Update the remote
libcontainerd implementation to match. ContainerD guarantees that the
process ID of a task (container init process) is set to the
corresponding container ID, so use that invariant to distinguish task
exits from other process exits.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-01-31 12:14:50 -05:00
Sebastiaan van Stijn
d7573ab867
Merge pull request #44885 from thaJeztah/23.0_backport_update_containerd
[23.0 backport] vendor: github.com/containerd/containerd v1.6.16
2023-01-31 15:29:44 +01:00
Sebastiaan van Stijn
9df87eaa9c
Merge pull request #44883 from thaJeztah/23.0_backport_invalid_character
[23.0 backport] client: improve error messaging on crash
2023-01-31 15:28:37 +01:00
Sebastiaan van Stijn
de3f216113
Merge pull request #44884 from thaJeztah/23.0_backport_rm_install_suffix
[23.0 backport] hack: remove -installsuffix build flag
2023-01-31 13:56:33 +01:00
Akihiro Suda
20203f0c92
Merge pull request #44881 from thaJeztah/23.0_containerd_binary_1.6.16
[23.0] update containerd binary to v1.6.16
2023-01-31 02:00:12 +09:00
Sebastiaan van Stijn
610250f06e
vendor: github.com/containerd/containerd v1.6.16
Notable Updates

- Fix push error propagation
- Fix slice append error with HugepageLimits for Linux
- Update default seccomp profile for PKU and CAP_SYS_NICE
- Fix overlayfs error when upperdirlabel option is set

full diff: https://github.com/containerd/containerd/compare/v1.6.15...v1.6.16

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit c41c8c2f86)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-01-30 15:08:36 +01:00
Sebastiaan van Stijn
63a0f1fe09
vendor: github.com/pelletier/go-toml v1.9.5
Contains a fix that prevents a panic on an invalid toml file.

full diff: https://github.com/pelletier/go-toml/compare/v1.9.4...v1.9.5

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 22098745e7)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-01-30 15:08:36 +01:00
CrazyMax
699e853be3
hack: display build cmd when DOCKER_DEBUG set
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 170a1c4709)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-01-30 15:06:33 +01:00
CrazyMax
b7e0725e98
hack: remove unnecessary vars in make.sh script
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 877baae03e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-01-30 15:06:32 +01:00
CrazyMax
96746238b2
hack: remove ORIG_BUILDFLAGS var
This var was used for the cross target but it has been removed
in 8086f40123 so not necessary anymore

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit b80f16157e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-01-30 15:06:32 +01:00
CrazyMax
df2fe70049
hack: remove -installsuffix build flag
Has been introduced in 232d59baeb to work around a bug with
"go build" but not required anymore since go 1.5: 4dab6d01f1

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 0c008cc4f8)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-01-30 15:06:32 +01:00
Nick Santos
8b83e09b9c
client: improve error messaging on crash
Repro steps:
- Run Docker Desktop
- Run `docker run busybox tail -f /dev/null`
- Run `pkill "Docker Desktop"

Expected:
An error message that indicates that Docker Desktop is shutting down.

Actual:
An error message that looks like this:

```
error waiting for container: invalid character 's' looking for beginning of value
```

here's an example:

https://github.com/docker/for-mac/issues/6575#issuecomment-1324879001

After this change, you get an error message like:

```
error waiting for container: copying response body from Docker: unexpected EOF
```

which is a bit more explicit.

Signed-off-by: Nick Santos <nick.santos@docker.com>
(cherry picked from commit 9900c7a348)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-01-30 15:03:01 +01:00
Sebastiaan van Stijn
26e9face5a
[23.0] update containerd binary to v1.6.16
Notable Updates

- Fix push error propagation
- Fix slice append error with HugepageLimits for Linux
- Update default seccomp profile for PKU and CAP_SYS_NICE
- Fix overlayfs error when upperdirlabel option is set

full diff: https://github.com/containerd/containerd/compare/v1.6.15...v1.6.16

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-01-30 09:16:22 +01:00
Sebastiaan van Stijn
9f62b37a62
Merge pull request #44863 from thaJeztah/23.0_backport_rootless_ipc_host
[23.0 backport] rootless: support `--ipc=host`
2023-01-20 18:49:22 +01:00
Sebastiaan van Stijn
3f87416881
Merge pull request #44862 from thaJeztah/23.0_backport_fix_arm
[23.0 backport] hack: check if cross-compiling before setting ARM target name
2023-01-20 17:59:14 +01:00
Akihiro Suda
6641852d51
rootless: support --ipc=host
Fix issue 44294

Co-authored-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit b3c5352386)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-01-20 17:02:23 +01:00
CrazyMax
8fc49af4ab
hack: check if cross-compiling before setting ARM target name
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 67aa86f1e0)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-01-20 16:40:33 +01:00
Bjorn Neergaard
b968cb92e6
Merge pull request #44858 from thaJeztah/23.0_backport_bump_swarmkit
[23.0 backport] vendor: github.com/moby/swarmkit/v2 v2.0.0-20230119195359-904c221ac281
2023-01-19 18:11:15 -07:00
Sebastiaan van Stijn
62c7b25605
Merge pull request #44856 from corhere/backport-23.0/fix-bytespipe-deadlock
[23.0 backport] Fix bytespipe blockThreshold deadlock bug
2023-01-20 00:33:44 +01:00
Sebastiaan van Stijn
30134ab177
vendor: github.com/moby/swarmkit/v2 v2.0.0-20230119195359-904c221ac281
full diff: 0da442b278...904c221ac2

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 25f95b0000)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-01-19 23:39:26 +01:00
ningmingxiao
3da45c0fe7 fix blockThreshold full bug
Signed-off-by: ningmingxiao <ning.mingxiao@zte.com.cn>
(cherry picked from commit dcfe23a038)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-01-19 16:37:40 -05:00
Cory Snider
941a07b339 ioutils: add regression test for bytespipe deadlock
Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit 0b83fd3217)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-01-19 16:37:40 -05:00
Cory Snider
cd5e05ee14
Merge pull request #44845 from thaJeztah/23.0_backport_reimplement_44706
[23.0 backport] libnetwork: Support IPv6 in arrangeUserFilterRule()
2023-01-19 16:22:15 -05:00
Bjorn Neergaard
f825537119
Merge pull request #44855 from thaJeztah/23.0_backport_fix_arm64_build
[23.0 backport] Dockerfile: prefer ld for building against arm64
2023-01-19 13:23:07 -07:00
CrazyMax
a21381a55a
Dockerfile: prefer ld for building against arm64
We already prefer ld for cross-building arm64 but that seems
not enough as native arm64 build also has a linker issue with lld
so we need to also prefer ld for native arm64 build.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit d2d6ef431f)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-01-19 16:39:11 +01:00
Albin Kerouanton
2616ebc5ac
libnetwork: Support IPv6 in arrangeUserFilterRule()
Fixes #44451.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
(cherry picked from commit 2d397beb00)
Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
2023-01-19 08:19:09 -07:00
Bjorn Neergaard
cd0d7c49a6
Merge pull request #44728 from thaJeztah/23.0_backport_libnetwork_cleanup
[23.0 backport] libnetwork: cleanup config package, remove old integration tests
2023-01-19 07:40:07 -07:00
Sebastiaan van Stijn
7a5f3f8053
libnetwork: fix empty-lines (revive)
libnetwork/etchosts/etchosts_test.go:167:54: empty-lines: extra empty line at the end of a block (revive)
    libnetwork/osl/route_linux.go:185:74: empty-lines: extra empty line at the start of a block (revive)
    libnetwork/osl/sandbox_linux_test.go:323:36: empty-lines: extra empty line at the start of a block (revive)
    libnetwork/bitseq/sequence.go:412:48: empty-lines: extra empty line at the start of a block (revive)
    libnetwork/datastore/datastore_test.go:67:46: empty-lines: extra empty line at the end of a block (revive)
    libnetwork/datastore/mock_store.go:34:60: empty-lines: extra empty line at the end of a block (revive)
    libnetwork/iptables/firewalld.go:202:44: empty-lines: extra empty line at the end of a block (revive)
    libnetwork/iptables/firewalld_test.go:76:36: empty-lines: extra empty line at the end of a block (revive)
    libnetwork/iptables/iptables.go:256:67: empty-lines: extra empty line at the end of a block (revive)
    libnetwork/iptables/iptables.go:303:128: empty-lines: extra empty line at the start of a block (revive)
    libnetwork/networkdb/cluster.go:183:72: empty-lines: extra empty line at the end of a block (revive)
    libnetwork/ipams/null/null_test.go:44:38: empty-lines: extra empty line at the end of a block (revive)
    libnetwork/drivers/macvlan/macvlan_store.go:45:52: empty-lines: extra empty line at the end of a block (revive)
    libnetwork/ipam/allocator_test.go:1058:39: empty-lines: extra empty line at the start of a block (revive)
    libnetwork/drivers/bridge/port_mapping.go:88:111: empty-lines: extra empty line at the end of a block (revive)
    libnetwork/drivers/bridge/link.go:26:90: empty-lines: extra empty line at the end of a block (revive)
    libnetwork/drivers/bridge/setup_ipv6_test.go:17:34: empty-lines: extra empty line at the end of a block (revive)
    libnetwork/drivers/bridge/setup_ip_tables.go:392:4: empty-lines: extra empty line at the start of a block (revive)
    libnetwork/drivers/bridge/bridge.go:804:50: empty-lines: extra empty line at the start of a block (revive)
    libnetwork/drivers/overlay/ov_serf.go:183:29: empty-lines: extra empty line at the start of a block (revive)
    libnetwork/drivers/overlay/ov_utils.go:81:64: empty-lines: extra empty line at the end of a block (revive)
    libnetwork/drivers/overlay/peerdb.go:172:67: empty-lines: extra empty line at the start of a block (revive)
    libnetwork/drivers/overlay/peerdb.go:209:67: empty-lines: extra empty line at the start of a block (revive)
    libnetwork/drivers/overlay/peerdb.go:344:89: empty-lines: extra empty line at the start of a block (revive)
    libnetwork/drivers/overlay/peerdb.go:436:63: empty-lines: extra empty line at the start of a block (revive)
    libnetwork/drivers/overlay/overlay.go:183:36: empty-lines: extra empty line at the start of a block (revive)
    libnetwork/drivers/overlay/encryption.go:69:28: empty-lines: extra empty line at the end of a block (revive)
    libnetwork/drivers/overlay/ov_network.go:563:81: empty-lines: extra empty line at the start of a block (revive)
    libnetwork/default_gateway.go:32:43: empty-lines: extra empty line at the start of a block (revive)
    libnetwork/errors_test.go:9:40: empty-lines: extra empty line at the start of a block (revive)
    libnetwork/service_common.go:184:64: empty-lines: extra empty line at the end of a block (revive)
    libnetwork/endpoint.go:161:55: empty-lines: extra empty line at the end of a block (revive)
    libnetwork/store.go:320:33: empty-lines: extra empty line at the end of a block (revive)
    libnetwork/store_linux_test.go:11:38: empty-lines: extra empty line at the end of a block (revive)
    libnetwork/sandbox.go:571:36: empty-lines: extra empty line at the start of a block (revive)
    libnetwork/service_common.go:317:246: empty-lines: extra empty line at the start of a block (revive)
    libnetwork/endpoint.go:550:17: empty-lines: extra empty line at the end of a block (revive)
    libnetwork/sandbox_dns_unix.go:213:106: empty-lines: extra empty line at the start of a block (revive)
    libnetwork/controller.go:676:85: empty-lines: extra empty line at the end of a block (revive)
    libnetwork/agent.go:876:60: empty-lines: extra empty line at the end of a block (revive)
    libnetwork/resolver.go:324:69: empty-lines: extra empty line at the end of a block (revive)
    libnetwork/network.go:1153:92: empty-lines: extra empty line at the end of a block (revive)
    libnetwork/network.go:1955:67: empty-lines: extra empty line at the start of a block (revive)
    libnetwork/network.go:2235:9: empty-lines: extra empty line at the start of a block (revive)
    libnetwork/libnetwork_internal_test.go:336:26: empty-lines: extra empty line at the start of a block (revive)
    libnetwork/resolver_test.go:76:35: empty-lines: extra empty line at the end of a block (revive)
    libnetwork/libnetwork_test.go:303:38: empty-lines: extra empty line at the end of a block (revive)
    libnetwork/libnetwork_test.go:985:46: empty-lines: extra empty line at the end of a block (revive)
    libnetwork/ipam/allocator_test.go:1263:37: empty-lines: extra empty line at the start of a block (revive)
    libnetwork/errors_test.go:9:40: empty-lines: extra empty line at the end of a block (revive)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit cd381aea56)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-01-18 21:24:32 +01:00
Sebastiaan van Stijn
7cfd4b3471
libnetwork/config: rename ParseConfigOptions() to New()
This function effectively is a constructor, so rename it to better describe
it's functionality.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 267108e113)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-01-18 21:24:32 +01:00
Sebastiaan van Stijn
1090c5fd4c
libnetwork/config: inline LoadDefaultScopes()
This method was an exported method, but only used as part of ParseConfigOptions,
so inlining it.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 09cc2f9d0e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-01-18 21:24:32 +01:00
Sebastiaan van Stijn
c93e1e9e66
libnetwork/config: merge DaemonCfg into Config
It was unclear what the distinction was between these configuration
structs, so merging them to simplify.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 528428919e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-01-18 21:24:31 +01:00
Sebastiaan van Stijn
031fb72313
libnetwork: remove old integration tests
This was used for testing purposes when libnetwork was in a separate repo, using
the dnet utility, which was removed in 7266a956a8.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 571baffd59)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-01-18 21:24:31 +01:00
Sebastiaan van Stijn
a96a97bf47
libnetwork/config: remove ParseConfig()
Libnetwork configuration files were only used as part of integration tests using
the dnet utility, which was removed in 7266a956a8

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 46f4a45769)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-01-18 21:24:31 +01:00
Sebastiaan van Stijn
9b383dbd51
libnetwork/config: inline ProcessOptions
This method was only used in a single place; inlining it makes it
easier to see what's done.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 7d574f5ac6)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-01-18 21:24:31 +01:00
Sebastiaan van Stijn
bed115e664
libnetwork/config: remove "Experimental" and "Debug" options
These were no longer used.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit a8a8bd1e42)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-01-18 21:24:28 +01:00
Sebastiaan van Stijn
9ea2300535
Merge pull request #44850 from crazy-max/23.0_backport_fix-static
[23.0 backport] Dockerfile: use clang to build dockerd/docker-proxy
2023-01-18 21:22:08 +01:00
Bjorn Neergaard
5ae2d06a1f
Merge pull request #44843 from thaJeztah/23.0_backport_bump_x_net
[23.0 backport] vendor: golang.org/x packages
2023-01-18 12:21:27 -07:00
Bjorn Neergaard
9fd854976f
Merge pull request #44844 from thaJeztah/23.0_backport_homedir_nolinux_compat
[23.0 backport] Add GetLibHome stub for non-linux OS
2023-01-18 12:14:04 -07:00
CrazyMax
64cbbaa883
Dockerfile: smoke tests for static builds
Adds smoke test stage in our Dockerfile and a
GitHub Action workflow

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 518be73a5c)
2023-01-18 19:36:07 +01:00
Kevin Alvarez
4100226e27
Dockerfile: use clang to build dockerd/docker-proxy
Static binaries for dockerd are broken on armhf and armel (32-bit).
It seems to be an issue with GCC as building using clang solves
this issue. Also adds extra instruction to prefer ld for
cross-compiling arm64 in bullseye otherwise it doesn't link.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit f676dab8dc)
2023-01-18 19:36:07 +01:00
Sebastiaan van Stijn
72f5fabd97
Merge pull request #44848 from thaJeztah/23.0_backport_skip_lookup_on_windows
[23.0 backport] Disable chrootarchive.init() on Windows
2023-01-18 19:07:23 +01:00
Gabriel Adrian Samfira
dadeec4205
Disable chrootarchive.init() on Windows
Disables user.Lookup() and net.LookupHost() in the init() function on Windows.

Any package that simply imports pkg/chrootarchive will panic on Windows
Nano Server, due to missing netapi32.dll. While docker itself is not
meant to run on Nano Server, binaries that may import this package and
run on Nano server, will fail even if they don't really use any of the
functionality in this package while running on Nano.

Conflicts:
    pkg/chrootarchive/archive_unix.go

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
(cherry picked from commit f49c88f1c4)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-01-18 17:06:11 +01:00
Roy Reznik
f328486cb3
Add GetLibHome stub for non-linux OS
Signed-off-by: Roy Reznik <roy@wiz.io>
(cherry picked from commit ff14f8ef16)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-01-18 13:04:57 +01:00
Sebastiaan van Stijn
8e83d28f31
vendor: golang.org/x/net v0.5.0
contains a fix for CVE-2022-41721, although it probably does not affect us.

full diff: https://github.com/golang/net/compare/v0.4.0...v0.5.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 65c6ba1fc4)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-01-18 12:59:57 +01:00
Sebastiaan van Stijn
06eb6ab794
vendor: golang.org/x/text v0.6.0
full diff: https://github.com/golang/text/compare/v0.5.0...v0.6.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 824dc51341)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-01-18 12:59:49 +01:00
Sebastiaan van Stijn
31cf0bf181
vendor: golang.org/x/sys v0.4.0
full diff: https://github.com/golang/sys/compare/v0.3.0...v0.4.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit e66e6bb28a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-01-18 12:59:38 +01:00
Sebastiaan van Stijn
4df8f779b5
Merge pull request #44834 from thaJeztah/23.0_backport_docker_in_overlayfs
[23.0 backport] Work around missing rename support when the backing filesystem is overlayfs
2023-01-18 10:42:34 +01:00
Sebastiaan van Stijn
52df0048f6
Merge pull request #44829 from thaJeztah/23.0_backport_update_ipvs
[23.0 backport] vendor: github.com/moby/ipvs v1.1.0
2023-01-18 10:24:37 +01:00
Bjorn Neergaard
110a9eaac1
Merge pull request #44832 from thaJeztah/23.0_backport_wide_json
[23.0 backport] daemon/config: support alternate (common) unicode encodings using a BOM
2023-01-17 22:00:54 -07:00
Sebastiaan van Stijn
d2998a4659
Merge pull request #44839 from crazy-max/23.0_backport_fix-arm-build
[23.0 backport] hack: name for target ARM architecture not specified
2023-01-17 22:40:02 +01:00
CrazyMax
0e4548bbe1
Dockerfile: fix PKG_CONFIG for xx
Current implementation in hack/make.sh overwrites PKG_CONFIG
if not defined and set it to pkg-config. When a build is invoked
using xx in our Dockerfile, it will set PKG_CONFIG to the right
value in go environments depending on the target architecture: 8015613ccc/base/xx-go (L75-L78)

Also needs to install dpkg-dev to use pkg-config when cross-building

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 71fa3b1337)
2023-01-17 20:17:18 +01:00
Kevin Alvarez
8c12a6648b
hack: name for target ARM architecture not specified
Build currently doesn't set the right name for target ARM
architecture through switches in CGO_CFLAGS and CGO_CXXFLAGS
when doing cross-compilation. This was previously fixed in https://github.com/moby/moby/pull/43474

Also removes the toolchain configuration. Following changes for
cross-compilation in https://github.com/moby/moby/pull/44546,
we forgot to remove the toolchain configuration that is
not used anymore as xx already sets correct cc/cxx envs already.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 945704208a)
2023-01-17 20:17:18 +01:00
CrazyMax
10c4ada049
hack: typo in displayed platform name
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 94639eb66f)
2023-01-17 20:17:08 +01:00
Bjorn Neergaard
10b536dd0f
Merge pull request #44831 from thaJeztah/23.0_backport_default_sigquit_handler
[23.0 backport] cmd/dockerd: use default SIGQUIT behaviour
2023-01-17 11:33:25 -07:00
Illia Antypenko
322149038a
Add additional loggig in case of error of renaming runtimes-old and removing it
Signed-off-by: Illia Antypenko <ilya@antipenko.pp.ua>
(cherry picked from commit 07ba3e35d3)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-01-17 13:52:52 +01:00
Illia Antypenko
a8926de2a1
Handle docker start inside overlayfs
Raspberry Pi allows to start system under overlayfs.
Docker is successfully fallbacks to fuse-overlay but not starting
because of the `Error starting daemon: rename /var/lib/docker/runtimes /var/lib/docker/runtimes-old: invalid cross-device link` error
It's happening because `rename` is not supported by overlayfs.

After manually removing directory `runtimes` docker starts and works successfully

Signed-off-by: Illia Antypenko <ilya@antipenko.pp.ua>
(cherry picked from commit d591710f82)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-01-17 13:52:52 +01:00
Bjorn Neergaard
524de97334
daemon/config: inform the user when the input JSON contains invalid UTF-8
This helps ensure that users are not surprised by unexpected tokens in
the JSON parser, or fallout later in the daemon.

Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
(cherry picked from commit 8dbc5df952)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-01-17 13:45:47 +01:00
Bjorn Neergaard
0b2eecdc8a
daemon/config: support alternate (common) unicode encodings using a BOM
This is a pragmatic but impure choice, in order to better support the
default tools available on Windows Server, and reduce user confusion due
to otherwise inscrutable-to-the-uninitiated errors like the following:

> invalid character 'þ' looking for beginning of value
> invalid character 'ÿ' looking for beginning of value

While meaningful to those who are familiar with and are equipped to
diagnose encoding issues, these characters will be hidden when the file
is edited with a BOM-aware text editor, and further confuse the user.

Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
(cherry picked from commit d42495033e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-01-17 13:45:47 +01:00
Bjorn Neergaard
abff66b283
daemon/config: clean up tests to use common helper
Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
(cherry picked from commit 1dcf7d5b03)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-01-17 13:45:47 +01:00
Cory Snider
d9bdb61992
cmd/dockerd: use default SIGQUIT behaviour
dockerd handles SIGQUIT by dumping all goroutine stacks to standard
error and exiting. In contrast, the Go runtime's default SIGQUIT
behaviour... dumps all goroutine stacks to standard error and exits.
The default SIGQUIT behaviour is implemented directly in the runtime's
signal handler, and so is both more robust to bugs in the Go runtime and
does not perturb the state of the process to anywhere near same degree
as dumping goroutine stacks from a user goroutine. The only notable
difference from a user's perspective is that the process exits with
status 2 instead of 128+SIGQUIT.

Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit 0867d3173c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-01-17 13:41:01 +01:00
Sebastiaan van Stijn
c627132dc8
vendor: github.com/moby/ipvs v1.1.0
full diff: https://github.com/moby/ipvs/compare/v1.0.2...v1.1.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit af6b5d55db)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-01-17 10:56:50 +01:00
Sebastiaan van Stijn
13a31b67de
vendor: github.com/vishvananda/netns v0.0.2
full diff: https://github.com/vishvananda/netns/compare/v0.0.1...v0.0.2

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit f53feeea8b)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-01-17 10:56:40 +01:00
Bjorn Neergaard
ebedb1c496
Merge pull request #44821 from thaJeztah/23.0_backport_bump_netns
[23.0 backport] vendor: github.com/vishvananda/netns v0.0.1
2023-01-14 14:58:21 -07:00
Sebastiaan van Stijn
bd5d9f3190
vendor: github.com/vishvananda/netns v0.0.1
The project started tagging releases for go modules.

full diff: https://github.com/vishvananda/netns/compare/2eb08e3e575f...v0.0.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit bb5cae2c94)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-01-14 14:59:18 +01:00
Bjorn Neergaard
7e4f58d894
Merge pull request #44811 from akerouanton/23.0-backport-44803
[23.0 backport] libnetwork: Remove iptables nat rule when hairpin is disabled
2023-01-12 10:42:01 -07:00
Albin Kerouanton
c8262e912f
libnetwork: Remove iptables nat rule when hairpin is disabled
When userland-proxy is turned off and on again, the iptables nat rule
doing hairpinning isn't properly removed. This fix makes sure this nat
rule is removed whenever the bridge is torn down or hairpinning is
disabled (through setting userland-proxy to true).

Unlike for ip masquerading and ICC, the `programChainRule()` call
setting up the "MASQ LOCAL HOST" rule has to be called unconditionally
because the hairpin parameter isn't restored from the driver store, but
always comes from the driver config.

For the "SKIP DNAT" rule, things are a bit different: this rule is
always deleted by `removeIPChains()` when the bridge driver is
initialized.

Fixes #44721.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
(cherry picked from commit 566a2e4)
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-01-12 17:28:49 +01:00
Sebastiaan van Stijn
225551ddef
Merge pull request #44785 from thaJeztah/23.0_backport_api_sync_docs
[23.0 backport] docs: api fixes and updates
2023-01-11 17:38:01 +01:00
Sebastiaan van Stijn
fb75789691
Merge pull request #44794 from thaJeztah/23.0_backport_bump_go_1.19.5
[23.0 backport] update to go1.19.5
2023-01-11 16:26:26 +01:00
Sebastiaan van Stijn
12e3398f64
update to go1.19.5
go1.19.5 (released 2023-01-10) includes fixes to the compiler, the linker,
and the crypto/x509, net/http, sync/atomic, and syscall packages. See the
Go 1.19.5 milestone on the issue tracker for details:

https://github.com/golang/go/issues?q=milestone%3AGo1.19.5+label%3ACherryPickApproved

full diff: https://github.com/golang/go/compare/go1.19.4...go1.19.5

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 155e8d7d78)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-01-11 00:18:27 +01:00
Sebastiaan van Stijn
076d57104a
docs: api: synchronise versioned API docs (v1.39 - v1.41)
synchronises some fixes between these API versions for the documentation,
including fixes from:

- 52a9f1689a
- 345346d7c6
- 18f85467e7
- 1557892c37

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 805aea501a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-01-10 16:46:25 +01:00
Sebastiaan van Stijn
8ce4ae1345
docs: api: synchronise versioned API docs (v1.42)
synchronises some fixes between these API versions for the documentation,
including fixes from:

- 18f85467e7
- 345346d7c6

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 92cbd1c69e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-01-10 16:46:25 +01:00
Niel Drummond
f0fe353ca6
api: swagger: add errorDetail to CreateImageInfo
Signed-off-by: Niel Drummond <niel@drummond.lu>
(cherry picked from commit 1557892c37)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-01-10 16:46:09 +01:00
Niel Drummond
04371160f8
api: docs: add errorDetail to CreateImageInfo (v1.42)
Signed-off-by: Niel Drummond <niel@drummond.lu>
(cherry picked from commit d983fa0ebe)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-01-10 16:46:09 +01:00
Hsing-Yu (David) Chen
e753cce70f
docs: add 401 to possible status codes of /auth
Signed-off-by: Hsing-Yu (David) Chen <davidhsingyuchen@gmail.com>
(cherry picked from commit 18f85467e7)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-01-10 16:45:52 +01:00
Bjorn Neergaard
9fd3a437a6
Merge pull request #44775 from thaJeztah/23.0_backport_44610_logs_for_DNS_failures
[23.0 backport] libnetwork: improve logs for DNS failures
2023-01-10 08:14:11 -07:00
Bjorn Neergaard
fc690c56b5
Merge pull request #44776 from thaJeztah/23.0_backport_btrfs_uapi
[23.0 backport] graphdriver/btrfs: depend on kernel UAPI instead of libbtrfs
2023-01-10 08:13:12 -07:00
Bjorn Neergaard
aefb643d79
Merge pull request #44778 from thaJeztah/23.0_backport_fix_rootless_specspaths_T43111
[23.0 backport] Use user data path for plugin discovery in rootless mode
2023-01-10 08:12:41 -07:00
Bjorn Neergaard
d29ab757fb
Merge pull request #44777 from thaJeztah/23.0_backport_ignore_bom
[23.0 backport] daemon/config: ignore UTF-8 BOM in config JSON
2023-01-10 08:09:16 -07:00
Sebastiaan van Stijn
583018846c
Merge pull request #44779 from thaJeztah/23.0_backport_update_buildkit_ref
[23.0 backport] ci: update buildkit to fix integration tests
2023-01-10 12:45:59 +01:00
CrazyMax
4d7792417e
ci: update buildkit to fix integration tests
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit dc30785947)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-01-10 11:32:18 +01:00
Jan Garcia
5802ca9e08
rootless: move ./rootless to ./pkg/rootless
Signed-off-by: Jan Garcia <github-public@n-garcia.com>
(cherry picked from commit 6ab12ec8f4)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-01-10 11:26:15 +01:00
Jan Garcia
33572e98c1
Use user paths for plugin discovery in rootless mode
Signed-off-by: Jan Garcia <github-public@n-garcia.com>
(cherry picked from commit c1bd5e9144)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-01-10 11:26:12 +01:00
Sebastiaan van Stijn
52c7a5d96b
daemon/config: New(): initialize config with platform-specific defaults
This centralizes more defaults, to be part of the config struct that's
created, instead of interweaving the defaults with other code in various
places.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit b28e66cf4f)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-01-10 11:03:04 +01:00
Bjorn Neergaard
101bd10d0e
daemon/config: group JSON preprocessing steps
Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
(cherry picked from commit cba3edbc37)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-01-10 10:51:06 +01:00
Bjorn Neergaard
def679d0e0
daemon/config: ignore UTF-8 BOM in config JSON
[RFC 8259] allows for JSON implementations to optionally ignore a BOM
when it helps with interoperability; do so in Moby as Notepad (the only
text editor available out of the box in many versions of Windows Server)
insists on writing UTF-8 with a BOM.

  [RFC 8259]: https://tools.ietf.org/html/rfc8259#section-8.1

Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
(cherry picked from commit bb19265ba8)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-01-10 10:51:06 +01:00
Akihiro Suda
cf8c728f3c
graphdriver/btrfs: needs kernel headers >= 4.12, not >= 4.7
`linux/btrfs_tree.h` was not installed to `/usr/include` until kernel 4.12
fcc8487d47

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit 89fb8b32f6)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-01-10 10:46:26 +01:00
Bjorn Neergaard
e40c2410ca
Dockerfile.*: drop libbtrfs
This is no longer necessary after the switch to the kernel UAPI.

Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
(cherry picked from commit aa80c33360)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-01-10 10:46:05 +01:00
Bjorn Neergaard
51a65e01ac
PACKAGERS: update Btrfs graphdriver dependencies
We only need suitable UAPI headers now. They are available on kernel 4.7
and newer; out of the distributions currently in support that users
might be interested in, only Enterprise Linux 7 has too old a kernel
(3.10).

Users of Enterprise Linux 7 distros can compile using a newer platform,
disable the Btrfs graphdriver as documented in this file, or use newer
kernel headers on their older distro.

Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
(cherry picked from commit c9d632e485)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-01-10 10:46:05 +01:00
Bjorn Neergaard
0cf2b14994
graphdriver/btrfs: use free wrapper consistently
While the Cgo in this entire file is quite questionable, that is a task
for another day.

Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
(cherry picked from commit d3778d65fa)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-01-10 10:46:05 +01:00
Bjorn Neergaard
21e6c282e6
graphdriver/btrfs: use kernel UAPI headers
By relying on the kernel UAPI (userspace API), we can drop a dependency
and simplify building Moby, while also ensuring that we are using a
stable/supported source of the C types and defines we need.

btrfs-progs mirrors the kernel headers, but the headers it ships with
are not the canonical source and as [we have seen before][44698], could
be subject to changes.

Depending on the canonical headers from the kernel both is more
idiomatic, and ensures we are protected by the kernel's promise to not
break userspace.

  [44698]: https://github.com/moby/moby/issues/44698

Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
(cherry picked from commit 3208dcabdc)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-01-10 10:46:05 +01:00
Bjorn Neergaard
42e7a15a63
graphdriver/btrfs: drop version information
This is actually quite meaningless as we are reporting the libbtrfs
version, but we do not use libbtrfs. We only use the kernel interface to
btrfs instead.

While we could report the version of the kernel headers in play, they're
rather all-or-nothing: they provide the structures and defines we need,
or they don't. As such, drop all version information as the host kernel
version is the only thing that matters.

Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
(cherry picked from commit 1449c82484)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-01-10 10:46:05 +01:00
Sebastiaan van Stijn
01883e1177
daemon/graphdriver: use strconv instead of fmt.Sprintf
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 7fbf321c2a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-01-10 10:45:58 +01:00
Yamazaki Masashi
25939ee45e
libnetwork: improve logs for DNS failures
Signed-off-by: Yamazaki Masashi <masi19bw@gmail.com>

libnetwork: fix function call

Signed-off-by: Yamazaki Masashi <masi19bw@gmail.com>
(cherry picked from commit 0787ea8b26)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-01-10 10:39:58 +01:00
Samuel Karp
8f5bbc24ef
Merge pull request #44769 from thaJeztah/23.0_backport_vendor_containerd_1.6.15 2023-01-09 13:24:40 -08:00
Samuel Karp
f6093fab79
Merge pull request #44766 from thaJeztah/23.0_containerd_binary_1.6.15 2023-01-09 13:23:22 -08:00
Sebastiaan van Stijn
f3761a5f38
Merge pull request #44727 from thaJeztah/23.0_backport_fix_42127
[23.0 backport] Check iptables options before looking for ip6tables binary
2023-01-09 18:10:02 +01:00
Sebastiaan van Stijn
691b44b4c3
vendor: github.com/containerd/containerd v1.6.15
no changes to vendored files

full diff: https://github.com/containerd/containerd/compare/v1.6.14...v1.6.15

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit bffe6d5670)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-01-09 13:28:10 +01:00
Sebastiaan van Stijn
5aa0c2cae2
[23.0] update containerd binary to v1.6.15
release notes: https://github.com/containerd/containerd/releases/tag/v1.6.15

> Welcome to the v1.6.15 release of containerd!
>
> The fifteenth patch release for containerd 1.6 fixes an issue with CNI in the CRI plugin
>
> Notable Updates
>
> - Fix no CNI info for pod sandbox on restart in CRI plugin

full diff: https://github.com/containerd/containerd/compare/v1.6.14...v1.6.15

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-01-09 13:12:29 +01:00
Sebastiaan van Stijn
0f2e972234
Merge pull request #44764 from vvoland/fix-test-race-logs-goroutines-23
[23.0 backport] integration-cli: Fix hanging TestLogsFollowGoroutines*
2023-01-09 13:00:41 +01:00
Paweł Gronowski
6acc1701ea
integration-cli: Fix hanging TestLogsFollowGoroutines*
cmd.Wait is called twice from different goroutines which can cause the
test to hang completely. Fix by calling Wait only once and sending its
return value over a channel.

In TestLogsFollowGoroutinesWithStdout also added additional closes and
process kills to ensure that we don't leak anything in case test returns
early because of failed test assertion.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit deb4910c5b)
2023-01-09 10:29:39 +01:00
Sebastiaan van Stijn
6a9d742f12
Merge pull request #44758 from crazy-max/23.0_fix-binary-daemon
[23.0 backport] hack: restore copy_binaries func
2023-01-06 15:24:59 +01:00
CrazyMax
8dfd907478
hack: restore copy_binaries func
This function is still useful for the dev environment.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 7e7b0622a6)
2023-01-06 08:38:33 +01:00
Sebastiaan van Stijn
ba94fff321
Merge pull request #44753 from vvoland/buildflags-nopie-whenrace-23
[23.0 backport] hack/make: Don't add -buildmode=pie with -race
2023-01-05 22:09:55 +01:00
Cory Snider
7b692e1041
Merge pull request #44752 from corhere/backport-23.0/fix-44688
[23.0 backport] Clear conntrack entries for published UDP ports
2023-01-05 15:29:27 -05:00
Paweł Gronowski
0d1aba547e
hack/make: Don't add -buildmode=pie with -race
Make it possible to add `-race` to the BUILDFLAGS without making the
build fail with error:
"-buildmode=pie not supported when -race is enabled"

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit bbe6e9e8d1)
2023-01-05 20:27:08 +01:00
Albin Kerouanton
4da19e2dca Clear conntrack entries for published UDP ports
Conntrack entries are created for UDP flows even if there's nowhere to
route these packets (ie. no listening socket and no NAT rules to
apply). Moreover, iptables NAT rules are evaluated by netfilter only
when creating a new conntrack entry.

When Docker adds NAT rules, netfilter will ignore them for any packet
matching a pre-existing conntrack entry. In such case, when
dockerd runs with userland proxy enabled, packets got routed to it and
the main symptom will be bad source IP address (as shown by #44688).

If the publishing container is run through Docker Swarm or in
"standalone" Docker but with no userland proxy, affected packets will
be dropped (eg. routed to nowhere).

As such, Docker needs to flush all conntrack entries for published UDP
ports to make sure NAT rules are correctly applied to all packets.

- Fixes #44688
- Fixes #8795
- Fixes #16720
- Fixes #7540
- Fixes moby/libnetwork#2423
- and probably more.

As a precautionary measure, those conntrack entries are also flushed
when revoking external connectivity to avoid those entries to be reused
when a new sandbox is created (although the kernel should already
prevent such case).

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
(cherry picked from commit b37d34307d)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-01-05 14:22:53 -05:00
Sebastiaan van Stijn
cb13848acd
Merge pull request #44725 from thaJeztah/23.0_backport_fix_volume_createdat
[23.0 backport] Fix volume CreatedAt being altered on initialization
2023-01-03 21:42:58 +01:00
Sebastiaan van Stijn
fe097e018c
Fix volume CreatedAt being altered on initialization
The CreatedAt date was determined from the volume's `_data`
directory (`/var/lib/docker/volumes/<volumename>/_data`).
However, when initializing a volume, this directory is updated,
causing the date to change.

Instead of using the `_data` directory, use its parent directory,
which is not updated afterwards, and should reflect the time that
the volume was created.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 01fd23b625)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-01-03 17:55:26 +01:00
Bjorn Neergaard
a166d959a0
Merge pull request #44717 from neersighted/backport/44668/23.0
[23.0 backport] api: can marshal and unmarshal when args.fields is empty
2023-01-03 08:43:30 -07:00
Sebastiaan van Stijn
0109eac486
Merge pull request #44736 from crazy-max/23.0_dockerfile-cross
[23.0 backport] Dockerfile: use TARGETPLATFORM to build Docker
2023-01-03 10:42:08 +01:00
CrazyMax
d778bbab84
docs: contributing docs update for cross comp
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 83280d44d0)
2023-01-02 18:46:04 +01:00
CrazyMax
2dd577955c
Dockerfile: add "all" stage to build binaries and extra tools
We still need a stage that build binaries and extra tools as well for
docker-ce-packaging repo: ff110508ff/static/Makefile (L41-L57)

This could be removed if we create a package for each project
like it's done in docker-packaging repo: https://github.com/docker/packaging/tree/main/pkg

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit e8a82ed24d)
2023-01-02 18:46:03 +01:00
CrazyMax
3ef71f9085
ci: add extra steps to check artifacts
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 113e95ec02)
2023-01-02 18:46:03 +01:00
CrazyMax
8fbfc2676f
Dockerfile: rename dev stages
Also remove DEV_IMAGE bake var that is not used

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit b565b496ba)
2023-01-02 18:46:03 +01:00
CrazyMax
7e286efcd3
ci: use GITHUB_REF and GITHUB_SHA to set version and commit
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 27ba1766e7)
2023-01-02 18:46:03 +01:00
CrazyMax
f29fe2daec
bake: define additional vars
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 0b3af098e1)
2023-01-02 18:46:03 +01:00
CrazyMax
3b90bb6ad3
bake: define DESTDIR
Keep the same output dir format in the bake definition
as the one used in make scripts.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 9bcf5bed05)
2023-01-02 18:46:02 +01:00
CrazyMax
171471b613
hack: update buildmode pie condition
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 491467b0b9)
2023-01-02 18:46:02 +01:00
CrazyMax
7d8c689923
hack: do not set buildmode pie for statically linked binary
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 347a5f68da)
2023-01-02 18:46:02 +01:00
CrazyMax
75596bee28
hack: use PKG_CONFIG var when checking libdevmapper
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit e37985f590)
2023-01-02 18:46:02 +01:00
CrazyMax
29758a067f
IAmStatic not used anymore
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 84ea9ee0f9)
2023-01-02 18:46:02 +01:00
CrazyMax
5d4ddce5ed
hack: remove containerutility script
this script is not used anymore. containerutility is
built in the Dockerfile.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 04c90b8cf5)
2023-01-02 18:46:01 +01:00
CrazyMax
a85bdbe1ca
Dockerfile: use TARGETPLATFORM to build Docker
Better support for cross compilation so we can fully rely
on `--platform` flag of buildx for a seamless integration.

This removes unnecessary extra cross logic in the Dockerfile,
DOCKER_CROSSPLATFORMS and CROSS vars and some hack scripts as well.

Non-sandboxed build invocation is still supported and dev stages
in the Dockerfile have been updated accordingly.

Bake definition and GitHub Actions workflows have been updated
accordingly as well.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 8086f40123)
2023-01-02 18:46:01 +01:00
Sebastiaan van Stijn
63e861c87e
Merge pull request #44726 from thaJeztah/23.0_backport_devmapper_simplify_udev_error
[23.0 backport] daemon/graphdriver/devicemapper: simplify Udev log, and update link
2023-01-02 18:09:07 +01:00
Albin Kerouanton
6a0a2c4f79
Always use iptables -C to look for rules
iptables -C flag was introduced in v1.4.11, which was released ten
years ago. Thus, there're no more Linux distributions supported by
Docker using this version. As such, this commit removes the old way of
checking if an iptables rule exists (by using substring matching).

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
(cherry picked from commit 799cc143c9)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-31 18:17:38 +01:00
Albin Kerouanton
91f2d963c6
Merge iptables.probe() into iptables.detectIptables()
The former was doing some checks and logging warnings, whereas
the latter was doing the same checks but to set some internal variables.
As both are called only once and from the same place, there're now
merged together.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
(cherry picked from commit 205e5278c6)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-31 18:17:14 +01:00
Albin Kerouanton
2bf66f725c
Check ipt options before looking for ip6t
iptables package has a function `detectIptables()` called to initialize
some local variables. Since v20.10.0, it first looks for iptables bin,
then ip6tables and finally it checks what iptables flags are available
(including -C). It early exits when ip6tables isn't available, and
doesn't execute the last check.

To remove port mappings (eg. when a container stops/dies), Docker
first checks if those NAT rules exist and then deletes them. However, in
the particular case where there's no ip6tables bin available, iptables
`-C` flag is considered unavailable and thus it looks for NAT rules by
using some substring matching. This substring matching then fails
because `iptables -t nat -S POSTROUTING` dumps rules in a slighly format
than what's expected.

For instance, here's what `iptables -t nat -S POSTROUTING` dumps:

```
-A POSTROUTING -s 172.18.0.2/32 -d 172.18.0.2/32 -p tcp -m tcp --dport 9999 -j MASQUERADE
```

And here's what Docker looks for:

```
POSTROUTING -p tcp -s 172.18.0.2 -d 172.18.0.2 --dport 9999 -j MASQUERADE
```

Because of that, those rules are considered non-existant by Docker and
thus never deleted. To fix that, this change reorders the code in
`detectIptables()`.

Fixes #42127.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
(cherry picked from commit af7236f85a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-31 18:07:42 +01:00
Chee Hau Lim
1b27ab4c73
libnetwork/iptables: Fix test panic when execute only one test
- use local variables for chains instead of sharing global variables
- make createNewChain a t.Helper

Signed-off-by: Chee Hau Lim <ch33hau@gmail.com>
(cherry picked from commit a2cea992c2)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-31 18:07:39 +01:00
Sebastiaan van Stijn
316231cced
daemon/graphdriver/devicemapper: simplify Udev log, and update link
Simplify the error message so that we don't have to distinguish between static-
and non-static builds. Also update the link to the storage-driver section to
use a "/go/" redirect in the docs, as the anchor link was no longer correct.
Using a "/go/" redirect  makes sure the link remains functional if docs is  moving
around.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit a5ebd28797)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-31 17:02:30 +01:00
Akihiro Suda
7bf0572f64
Merge pull request #44724 from thaJeztah/23.0_backport_use_the_osusergo_luke
[23.0 backport] daemon/logger/gcplogs: remove ensureHomeIfIAmStatic workaround
2023-01-01 00:12:14 +09:00
Sebastiaan van Stijn
770ace39d8
daemon/logger/gcplogs: remove ensureHomeIfIAmStatic workaround
This function was added in b86e3bee5a to
work around an issue in os/user.Current(), which SEGFAULTS when compiling
statically with cgo enabled (see golang/go#13470).

We hit similar issues in other parts, and contributed a "osusergo" build-
tag in https://go-review.googlesource.com/c/go/+/330753. The "osusergo"
build tag must be set when compiling static binaries with cgo enabled.
If that build-tag is set, the cgo implementation for user.Current() won't
be used, and a pure-go implementation is used instead;
https://github.com/golang/go/blob/go1.19.4/src/os/user/cgo_lookup_unix.go#L5

With the above in place, we no longer need this workaround, and can remove
the ensureHomeIfIAmStatic() function.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 155e39187c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-31 14:50:43 +01:00
Yamazaki Masashi
5e4464798f
api: can marshal and unmarshal when args.fields is empty
Signed-off-by: Yamazaki Masashi <masi19bw@gmail.com>
Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
(cherry picked from commit 5fed968b1d)
Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
2022-12-29 15:40:54 -07:00
Akihiro Suda
2b5ec481e7
Merge pull request #44714 from thaJeztah/23.0_backport_carry_44600_update_delve
[23.0 backport] Update delve version
2022-12-30 07:24:15 +09:00
Bjorn Neergaard
045de829e8
Merge pull request #44708 from zhsj/23.0-btrfs-progs-6.1
[23.0 backport] daemon/graphdriver/btrfs: workaround field rename in btrfs-progs 6.1
2022-12-29 14:47:13 -07:00
Sebastiaan van Stijn
8cb02d8c58
Merge pull request #44703 from thaJeztah/23.0_backport_docker_proxy_version_flag
[23.0 backport] cmd/docker-proxy: add "-v / --version" flag
2022-12-29 21:13:44 +01:00
oscar.chen
515219b716
Update delve version
Signed-off-by: oscar.chen <2972789494@qq.com>
(cherry picked from commit ad8804885c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-29 19:13:08 +01:00
Shengjing Zhu
0393897695 daemon/graphdriver/btrfs: workaround field rename in btrfs-progs 6.1
Closes: #44698

Signed-off-by: Shengjing Zhu <zhsj@debian.org>
(cherry picked from commit ffbbe3d103)
Signed-off-by: Shengjing Zhu <zhsj@debian.org>
2022-12-29 21:16:47 +08:00
Sebastiaan van Stijn
7199300239
cmd/docker-proxy: add "-v / --version" flag
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 12df6024db)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-28 08:24:47 +01:00
Sebastiaan van Stijn
4ddde6dde5
Merge pull request #44702 from thaJeztah/23.0_backport_fix_buildkit_ci
[23.0 backport] gha: update buildkit to fix integration tests
2022-12-27 21:54:53 +01:00
Sebastiaan van Stijn
e9075f70c7
gha: update buildkit to fix integration tests
full diff: 0bfcd83e6d...d77361423c

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit c42b304f62)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-27 19:23:28 +01:00
Tianon Gravi
cba986b340
Merge pull request #44679 from thaJeztah/23.0_backport_errdefs_improve_debug_logs
[23.0 backport] errdefs: FromStatusCode() don't log "FIXME" debug message
2022-12-21 09:38:50 -08:00
Sebastiaan van Stijn
b2d2f012b4
errdefs: FromStatusCode() don't log "FIXME" debug message
This utility is used by the client, which cannot do anything about errors
received from the API. In situations where no API connection was possible,
for example, if the client has no permissions to connect to the socket,
the request would have a "-1" status-code;
3e39ec60da/client/request.go (L133-L134)

In this case, a client with "debug" enabled, would print _and_ log a confusing
error message:

    DEBU[0000] FIXME: Got an status-code for which error does not match any expected type!!!  error="Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Post \"http://%2Fvar%2Frun%2Fdocker.sock/v1.24/build?buildargs=%7B%7D&cachefrom=%5B%5D&cgroupparent=&cpuperiod=0&cpuquota=0&cpusetcpus=&cpusetmems=&cpushares=0&dockerfile=Dockerfile.repro&labels=%7B%7D&memory=0&memswap=0&networkmode=default&rm=1&shmsize=0&t=repro&target=&ulimits=null&version=1\": dial unix /var/run/docker.sock: connect: permission denied" module=api status_code=-1
    Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Post "http://%2Fvar%2Frun%2Fdocker.sock/v1.24/build?buildargs=%7B%7D&cachefrom=%5B%5D&cgroupparent=&cpuperiod=0&cpuquota=0&cpusetcpus=&cpusetmems=&cpushares=0&dockerfile=Dockerfile.repro&labels=%7B%7D&memory=0&memswap=0&networkmode=default&rm=1&shmsize=0&t=repro&target=&ulimits=null&version=1": dial unix /var/run/docker.sock: connect: permission denied

In the above; `DEBU` logs the error (including the "FIXME"), and the second
line is the error message printed.

This was a mistake on my side when I added the `FromStatusCode` utility. I
implemented that to be the counterpart to `FromError`, but in doing so also
copied over the logging (see 1af30c50ca). That
log-message is only intended to be logged on the daemon side, for situations
where we return an error without a proper errdefs (which would result in an
500 "internal server error" to be returned by the API).

This patch removes the debug log, and a minor cleanup to explicitly return
"nil" if we didn't get an error in the first place.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 2e67c827bb)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-20 16:07:07 +01:00
Sebastiaan van Stijn
478f023783
Merge pull request #44675 from thaJeztah/23.0_backport_vendor_containerd_1.6.14
[23.0 backport] vendor: github.com/containerd/containerd v1.6.14
2022-12-20 10:59:36 +01:00
Sebastiaan van Stijn
52c35488a0
Merge pull request #44672 from thaJeztah/23.0__update_containerd_1.6.14
[23.0] update containerd binary to v1.6.14
2022-12-20 10:54:39 +01:00
Sebastiaan van Stijn
9980e64341
vendor: github.com/containerd/containerd v1.6.14
No changes in vendored code, other than the version bump

full diff: https://github.com/containerd/containerd/compare/v1.6.13...v1.6.14

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 83d7ae76e8)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-19 22:46:07 +01:00
Sebastiaan van Stijn
5ebd78d5f8
update containerd binary to v1.6.14
Notable Updates

- Fix `memory.memsw.limit_in_bytes: no such file or directory` error in CRI plugin

full diff: https://github.com/containerd/containerd/compare/v1.6.13...v1.6.14

See the changelog for complete list of changes:
https://github.com/containerd/containerd/releases/tag/v1.6.14

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-19 18:38:04 +01:00
Sebastiaan van Stijn
4b78386a3a
Merge pull request #44655 from thaJeztah/23.0_containerd_binary_1.6.13
[23.0] update containerd binary to v1.6.13
2022-12-16 00:19:03 +01:00
Sebastiaan van Stijn
21c1faed45
Merge pull request #44659 from thaJeztah/23.0_backport_dockerfile_targetplatform
[23.0 backport] Dockerfile: use TARGETPLATFORM
2022-12-16 00:18:27 +01:00
CrazyMax
f6c2f20cee
Dockerfile: delve cross build with TARGETPLATFORM
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 19f3448c21)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-15 22:59:27 +01:00
CrazyMax
3d6908c79b
Dockerfile: swagger cross build with TARGETPLATFORM
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit ed95e1a9ff)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-15 22:59:27 +01:00
CrazyMax
1dc34e889c
Dockerfile: containerutility cross build with TARGETPLATFORM
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 879b023706)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-15 22:59:27 +01:00
CrazyMax
effd7661df
Dockerfile: registry cross build with TARGETPLATFORM
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 8d9a357a67)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-15 22:59:27 +01:00
CrazyMax
f3e3266373
Dockerfile: dockercli cross build with TARGETPLATFORM
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit a4bc2922c2)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-15 22:59:26 +01:00
CrazyMax
fd6316e631
Dockerfile: rootlesskit cross build with TARGETPLATFORM
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 420591c4a2)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-15 22:59:26 +01:00
CrazyMax
fbe4db5be0
Dockerfile: tini cross build with TARGETPLATFORM
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit f0307892ea)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-15 22:59:26 +01:00
CrazyMax
f9a356c7e9
Dockerfile: runc cross build with TARGETPLATFORM
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 5e19d7ec54)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-15 22:59:26 +01:00
Kevin Alvarez
6e7cc3b7e3
Dockerfile: containerd cross build with TARGETPLATFORM
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 81d704d15d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-15 22:59:19 +01:00
CrazyMax
582b29f587
Dockerfile: DOCKER_STATIC arg
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit b19c11621a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-15 22:58:23 +01:00
CrazyMax
daa28fcf2d
Dockerfile: add dummy stage
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 69c9dfd6d5)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-15 22:58:23 +01:00
CrazyMax
4cab584a9c
Dockerfile: add cross compilation helper
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 6cba7d22e9)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-15 22:58:22 +01:00
Sebastiaan van Stijn
1cc21c1b81
Merge pull request #44658 from thaJeztah/23.0_backport_vendor_containerd_1.6.13
[23.0 backport] vendor: github.com/containerd/containerd v1.6.13
2022-12-15 22:36:15 +01:00
Sebastiaan van Stijn
0b12468116
vendor: github.com/containerd/containerd v1.6.13
full diff: https://github.com/containerd/containerd/compare/v1.6.12...v1.6.13

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 35cbdc9661)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-15 19:36:58 +01:00
Sebastiaan van Stijn
6570b654f4
vendor: github.com/Microsoft/hcsshim v0.9.6
full diff: https://github.com/Microsoft/hcsshim/compare/v0.9.5...v0.9.6

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 9300aca2a6)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-15 19:36:57 +01:00
Sebastiaan van Stijn
d8d11bca37
update containerd binary to v1.6.13
Notable Updates

- Update overlay snapshotter to check for tmpfs when evaluating usage of userxattr
- Update hcsschim to v0.9.6 to fix resource leak on exec
- Make swapping disabled with memory limit in CRI plugin
- Allow clients to remove created tasks with PID 0
- Fix concurrent map iteration and map write in CRI port forwarding
- Check for nil HugepageLimits to avoid panic in CRI plugin

See the changelog for complete list of changes:
https://github.com/containerd/containerd/releases/tag/v1.6.13

full diff: https://github.com/containerd/containerd/compare/v1.6.12...v1.6.13

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-15 19:26:40 +01:00
Sebastiaan van Stijn
0593c0600a
Merge pull request #44652 from thaJeztah/23.0_backport_bump_swarmkit
[23.0 backport] vendor: github.com/moby/swarmkit/v2 v2.0.0-20221215132206-0da442b2780f
2022-12-15 18:07:13 +01:00
Sebastiaan van Stijn
a72fb70eb0
vendor: github.com/moby/swarmkit/v2 v2.0.0-20221215132206-0da442b2780f
No changes in vendored code, but removes some indirect dependencies.

full diff: b17f02f0a0...0da442b278

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 0007490b21)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-15 14:29:03 +01:00
Sebastiaan van Stijn
1bb2792c76
Merge pull request #44626 from thaJeztah/23.0_backport_update_fluent_deps
[23.0 backport] vendor: update dependencies for fluentd
2022-12-15 10:55:29 +01:00
Sebastiaan van Stijn
ce27df7b3c
Merge pull request #44633 from corhere/backport-23.0/fix-44512
[23.0 Backport] Prevent containers from being included in List API before they are registered
2022-12-14 12:30:11 +01:00
Bjorn Neergaard
98cad005ca
Merge pull request #44631 from neersighted/23.0_vendor_improvements
[23.0 backport] vendor tooling improvements
2022-12-13 18:40:23 -07:00
Cory Snider
dca58c654a daemon: drop side effect from registerLinks()
(*Daemon).registerLinks() calling the WriteHostConfig() method of its
container argument is a vestigial behaviour. In the distant past,
registerLinks() would persist the container links in an SQLite database
and drop the link config from the container's persisted HostConfig. This
changed in Docker v1.10 (#16032) which migrated away from SQLite and
began using the link config in the container's HostConfig as the
persistent source of truth. registerLinks() no longer mutates the
HostConfig at all so persisting the HostConfig to disk falls outside of
its scope of responsibilities.

Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit 388fe4aea8)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-12-13 18:08:21 -05:00
Cory Snider
6149c333ff daemon: don't checkpoint container until registered
(*Container).CheckpointTo() upserts a snapshot of the container to the
daemon's in-memory ViewDB and also persists the snapshot to disk. It
does not register the live container object with the daemon's container
store, however. The ViewDB and container store are used as the source of
truth for different operations, so having a container registered in one
but not the other can result in inconsistencies. In particular, the List
Containers API uses the ViewDB as its source of truth and the Container
Inspect API uses the container store.

The (*Daemon).setHostConfig() method is called fairly early in the
process of creating a container, long before the container is registered
in the daemon's container store. Due to a rogue CheckpointTo() call
inside setHostConfig(), there is a window of time where a container can
be included in a List Containers API response but "not exist" according
to the Container Inspect API and similar endpoints which operate on a
particular container. Remove the rogue call so that the caller has full
control over when the container is checkpointed and update callers to
checkpoint explicitly. No changes to (*Daemon).create() are needed as it
checkpoints the fully-created container via (*Daemon).Register().

Fixes #44512.

Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit 0141c6db81)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-12-13 18:08:21 -05:00
Cory Snider
42bffae5ff daemon: fix GetContainer() returning (nil, nil)
GetContainer() would return (nil, nil) when looking up a container
if the container was inserted into the containersReplica ViewDB but not
the containers Store at the time of the lookup. Callers which reasonably
assume that the returned err == nil implies returned container != nil
would dereference a nil pointer and panic. Change GetContainer() so that
it always returns a container or an error.

Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit 00157a42d3)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-12-13 18:08:21 -05:00
Bjorn Neergaard
a6818fd4cb
hack: introduce validate/no-module
Moby is not a Go module; to prevent anyone from mistakenly trying to
convert it to one before we are ready, introduce a check (usable in CI
and locally) for a go.mod file.

This is preferable to trying to .gitignore the file as we can ensure
that a mistakenly created go.mod is surfaced by Git-based tooling and is
less likely to surprise a contributor.

Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
(cherry picked from commit 25c3421802)
Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
2022-12-13 14:42:07 -07:00
Bjorn Neergaard
45fe08c93c
hack: replace go-mod-prepare.sh with wrapper script
To make the local build environment more correct and consistent, we
should never leave an uncommitted go.mod in the tree; however, we need a
go.mod for certain commands to work properly. Use a wrapper script to
create and destroy the go.mod as needed instead of potentially changing
tooling behavior by leaving it.

If a go.mod already exists, this script will warn and call the wrapped
command with GO111MODULE=on.

Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
(cherry picked from commit a449f77774)
Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
2022-12-13 14:42:05 -07:00
Bjorn Neergaard
c508d13372
hack/go-mod-prepare.sh: find root robustly; make steps transparent
Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
(cherry picked from commit af8e955e8f)
Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
2022-12-13 14:41:28 -07:00
Bjorn Neergaard
036398f512
hack/validate/vendor: clean up license validation step
Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
(cherry picked from commit 9f004830a5)
Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
2022-12-13 14:41:27 -07:00
Bjorn Neergaard
a4957d2585
hack/validate/vendor: split tidy from vendor
Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
(cherry picked from commit dbd76848ca)
Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
2022-12-13 14:41:25 -07:00
Bjorn Neergaard
394eaa8495
hack/vendor.sh: allow running tidy & vendor separately
Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
(cherry picked from commit 72568286ab)
Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
2022-12-13 14:41:23 -07:00
Sebastiaan van Stijn
d80f235c47
vendor: github.com/tinylib/msgp v1.1.6
This is a dependency of github.com/fluent/fluent-logger-golang, which
currently does not provide a go.mod, but tests against the latest
versions of its dependencies.

Updating this dependency to the latest version.

Notable changes:

- all: implement omitempty
- fix: JSON encoder may produce invalid utf-8 when provided invalid utf-8 message pack string.
- added Unwrap method to errWrapped plus tests; switched travis to go 1.14
- CopyToJSON: fix bitSize for floats
- Add Reader/Writer constructors with custom buffer
- Add missing bin header functions
- msgp/unsafe: bring code in line with unsafe guidelines
- msgp/msgp: fix ReadMapKeyZC (fix "Fail to decode string encoded as bin type")

full diff: https://github.com/tinylib/msgp/compare/v1.1.0...v1.1.6

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 389dacd6e2)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-12 11:38:57 +01:00
Sebastiaan van Stijn
5ca9231b44
vendor: github.com/philhofer/fwd v1.1.2
This is an (indirect) dependency of github.com/fluent/fluent-logger-golang,
which currently does not provide a go.mod, but tests against the latest
versions of its dependencies.

Updating this dependency to the latest version.

full diff: https://github.com/philhofer/fwd/compare/v1.0.0...v1.1.2

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 24496fe097)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-12 11:36:24 +01:00
Tianon Gravi
abcb4c556c
Merge pull request #44616 from thaJeztah/23.0_backport_bump_golang_net
[23.0 backport] update dependencies
2022-12-09 13:06:42 -08:00
Sebastiaan van Stijn
47069ae18c
Merge pull request #44617 from thaJeztah/23.0_backport_images_errdefs
[23.0 backport] image/store: Use errdefs for errors
2022-12-09 21:31:25 +01:00
Paweł Gronowski
ffa2c0b517
image/store: Use errdefs for errors
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit e214503789)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-09 00:17:57 +01:00
Sebastiaan van Stijn
c0c146fc82
Remove uses of deprecated go-digest.NewDigestFromHex, go-digest.Digest.Hex
Both of these were deprecated in 55f675811a,
but the format of the GoDoc comments didn't follow the correct format, which
caused them not being picked up by tools as "deprecated".

This patch updates uses in the codebase to use the alternatives.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 0f7c9cd27e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-09 00:17:18 +01:00
Sebastiaan van Stijn
6295320d91
use consistent alias for gotest.tools/v3/assert/cmp
Make sure we use the same alias everywhere for easier finding,
and to prevent accidentally introducing duplicate imports with
different aliases for the same package.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit f6b695d2fb)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-09 00:15:26 +01:00
Sebastiaan van Stijn
1cc1395fae
vendor: golang.org/x/net v0.4.0
golang.org/x/net contains a fix for CVE-2022-41717, which was addressed
in stdlib in go1.19.4 and go1.18.9;

> net/http: limit canonical header cache by bytes, not entries
>
> An attacker can cause excessive memory growth in a Go server accepting
> HTTP/2 requests.
>
> HTTP/2 server connections contain a cache of HTTP header keys sent by
> the client. While the total number of entries in this cache is capped,
> an attacker sending very large keys can cause the server to allocate
> approximately 64 MiB per open connection.
>
> This issue is also fixed in golang.org/x/net/http2 v0.4.0,
> for users manually configuring HTTP/2.

full diff: https://github.com/golang/net/compare/v0.2.0...v0.4.0

other dependency updates (due to circular dependencies):

- golang.org/x/sys v0.3.0: https://github.com/golang/sys/compare/v0.2.0...v0.3.0
- golang.org/x/text v0.5.0: https://github.com/golang/text/compare/v0.4.0...v0.5.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 4bbc37687e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-09 00:01:27 +01:00
Sebastiaan van Stijn
0163808dbe
vendor: github.com/prometheus/client_golang v1.14.0
full diff: https://github.com/prometheus/client_golang/compare/v1.13.0...v1.14.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit dc52e08bb5)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-09 00:01:16 +01:00
Sebastiaan van Stijn
9b62b83429
vendor: github.com/go-logr/logr v1.2.3
full diff: https://github.com/go-logr/logr/compare/v1.2.2...v1.2.3

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit b2fe29ba79)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-09 00:01:07 +01:00
Sebastiaan van Stijn
d5ffdeb76c
vendor: golang.org/x/crypto v0.2.0
- all: use math/bits.RotateLeft

full diff: https://github.com/golang/crypto/compare/v0.1.0...v0.2.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 7c1b7842c6)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-09 00:00:58 +01:00
Sebastiaan van Stijn
4da8dedaab
vendor: golang.org/x/net v0.2.0
- http2/hpack: build static table with go generate

full diff: https://github.com/golang/net/compare/v0.1.0...v0.2.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit efe17dbdb4)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-09 00:00:49 +01:00
Sebastiaan van Stijn
b14ffffffd
vendor: github.com/opencontainers/runc v1.1.3
no changes in vendored code

full diff: https://github.com/opencontainers/runc/compare/v1.1.2...v1.1.3

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 0588a83380)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-09 00:00:39 +01:00
Sebastiaan van Stijn
173df30ac5
vendor: github.com/klauspost/compress v1.15.12
full diff: https://github.com/klauspost/compress/compare/v1.15.9...v1.15.12

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 38adf7a694)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-09 00:00:31 +01:00
Sebastiaan van Stijn
fe62290eed
go.mod: github.com/miekg/dns v1.1.43
This update:

- removes support for go1.11
- removes the use of "golang.org/x/crypto/ed25519", which is now part of stdlib:
  > Beginning with Go 1.13, the functionality of this package was moved to the
  > standard library as crypto/ed25519. This package only acts as a compatibility
  > wrapper.

Note that this is not the latest release; version v1.1.44 introduced a tools.go
file, which added golang.org/x/tools to the dependency tree (but only used for
"go:generate") see commit:
df84acab71

full diff: https://github.com/miekg/dns/compare/v1.1.27...v1.1.43

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit bbb1b82232)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-09 00:00:18 +01:00
Sebastiaan van Stijn
3083236871
Merge pull request #44566 from thaJeztah/23.0_backport_remove_trustkey_id_migration
[23.0 backport] use ad-hoc libtrust trustkey
2022-12-08 23:54:09 +01:00
Bjorn Neergaard
2abf4d763b
Merge pull request #44612 from thaJeztah/23.0_backport_vendor_containerd_1.6.11
[23.0 backport] vendor: github.com/containerd/containerd v1.6.12
2022-12-08 13:47:01 -07:00
Sebastiaan van Stijn
3fd34324a8
Merge pull request #44608 from thaJeztah/23.0_containerd_binary_1.6.12
[23.0] update containerd binary to v1.6.12 (addresses CVE-2022-23471)
2022-12-08 20:33:46 +01:00
Sebastiaan van Stijn
4ee3cc2531
vendor: github.com/containerd/containerd v1.6.12
no changes in vendored code, but containerd v1.6.12 is a security release,
so updating, to prevent scanners marking the dependency to have a vulnerability.

full diff: https://github.com/containerd/containerd/compare/v1.6.11...v1.6.12

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 12c76d4a15)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-08 14:11:35 +01:00
Sebastiaan van Stijn
48a0d8a8d1
[23.0] update containerd binary to v1.6.12 (addresses CVE-2022-23471)
The twelfth patch release for containerd 1.6 contains a fix for CVE-2022-23471.

Notable Updates

- Fix goroutine leak during Exec in CRI plugin (GHSA-2qjp-425j-52j9)

full diff: https://github.com/containerd/containerd/compare/v1.6.11...v1.6.12

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-08 00:51:01 +01:00
Sebastiaan van Stijn
a122104327
Merge pull request #44596 from thaJeztah/23.0_containerd_1.6.11
[23.0] update containerd binary to v1.6.11
2022-12-07 13:48:39 +01:00
Sebastiaan van Stijn
a74f0c2483
Merge pull request #44595 from thaJeztah/23.0_backport_vendor_containerd_1.6.11
[23.0 backport] vendor: github.com/containerd/containerd v1.6.11
2022-12-07 13:26:56 +01:00
Sebastiaan van Stijn
4f6786d861
Merge pull request #44592 from thaJeztah/23.0_backport_update_go_1.19.4
[23.0 backport] update to go1.19.4
2022-12-07 13:21:15 +01:00
Sebastiaan van Stijn
3d902fe719
[23.0] update containerd binary to v1.6.11
Welcome to the v1.6.11 release of containerd!

The eleventh patch release for containerd 1.6 contains a various fixes and updates.

Notable Updates

- Add pod UID annotation in CRI plugin
- Fix nil pointer deference for Windows containers in CRI plugin
- Fix lease labels unexpectedly overwriting expiration
- Fix for simultaneous diff creation using the same parent snapshot

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-06 23:33:27 +01:00
Sebastiaan van Stijn
0da8917add
vendor: github.com/containerd/containerd v1.6.11
- Fix nil pointer deference for Windows containers in CRI plugin
- Fix lease labels unexpectedly overwriting expiration
- Fix for simultaneous diff creation using the same parent snapshot

full diff: https://github.com/containerd/containerd/v1.6.10...v1.6.11

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit d331bc3b03)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-06 23:30:24 +01:00
Sebastiaan van Stijn
623f6f7702
update to go1.19.4
Includes security fixes for net/http (CVE-2022-41717, CVE-2022-41720),
and os (CVE-2022-41720).

These minor releases include 2 security fixes following the security policy:

- os, net/http: avoid escapes from os.DirFS and http.Dir on Windows

  The os.DirFS function and http.Dir type provide access to a tree of files
  rooted at a given directory. These functions permitted access to Windows
  device files under that root. For example, os.DirFS("C:/tmp").Open("COM1")
  would open the COM1 device.
  Both os.DirFS and http.Dir only provide read-only filesystem access.

  In addition, on Windows, an os.DirFS for the directory \(the root of the
  current drive) can permit a maliciously crafted path to escape from the
  drive and access any path on the system.

  The behavior of os.DirFS("") has changed. Previously, an empty root was
  treated equivalently to "/", so os.DirFS("").Open("tmp") would open the
  path "/tmp". This now returns an error.

  This is CVE-2022-41720 and Go issue https://go.dev/issue/56694.

- net/http: limit canonical header cache by bytes, not entries

  An attacker can cause excessive memory growth in a Go server accepting
  HTTP/2 requests.

  HTTP/2 server connections contain a cache of HTTP header keys sent by
  the client. While the total number of entries in this cache is capped,
  an attacker sending very large keys can cause the server to allocate
  approximately 64 MiB per open connection.

  This issue is also fixed in golang.org/x/net/http2 vX.Y.Z, for users
  manually configuring HTTP/2.

  Thanks to Josselin Costanzi for reporting this issue.

  This is CVE-2022-41717 and Go issue https://go.dev/issue/56350.

View the release notes for more information:
https://go.dev/doc/devel/release#go1.19.4

And the milestone on the issue tracker:
https://github.com/golang/go/issues?q=milestone%3AGo1.19.4+label%3ACherryPickApproved

Full diff: https://github.com/golang/go/compare/go1.19.3...go1.19.4

The golang.org/x/net fix is in 1e63c2f08a

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 52bc1ad744)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-06 22:58:19 +01:00
Sebastiaan van Stijn
e57c432eff
Merge pull request #44574 from thaJeztah/23.0_backport_builder_host_gateway
[23.0 backport] builder: handle host-gateway with extra hosts
2022-12-04 18:49:19 +01:00
Sebastiaan van Stijn
4952dc2844
distribution: remove unused RequireSchema2
It's never set, so we can remove it.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 85fddc0081)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-03 18:09:17 +01:00
Sebastiaan van Stijn
e7b26fa1b1
use ad-hoc libtrust key
This is only used for tests, and the key is not verified anymore, so
instead of creating a key and storing it, we can just use an ad-hoc
one.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 8feeaecb84)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-03 18:09:16 +01:00
Sebastiaan van Stijn
f2463e1e86
distribution: use ad-hoc trustkey for tests
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit e854b2a459)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-03 18:09:16 +01:00
Sebastiaan van Stijn
139080d093
daemon/config: remove and local trustkey utilities
Turned out that the loadOrCreateTrustKey() utility was doing exactly the
same as libtrust.LoadOrCreateTrustKey(), so making it a thin wrapped. I kept
the tests to verify the behavior, but we could remove them as we only need this
for our integration tests.

The storage location for the generated key was changed (again as we only need
this for some integration tests), so we can remove the TrustKeyPath from the
config.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 5cdd6ab7cd)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-03 18:09:16 +01:00
Sebastiaan van Stijn
46e0317bc1
[23.0] remove libtrust tests
This is a subset of 1981706196 on master,
preserving the tests for migrating the key to engine-id.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-03 18:08:29 +01:00
CrazyMax
1da1a3ea87
builder: handle host-gateway with extra hosts
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 521b8c02cc)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-03 14:39:41 +01:00
Sebastiaan van Stijn
bdf9baa207
Merge pull request #44568 from thaJeztah/23.0_backport_relax_checkSupportedMediaType
[23.0 backport] distribution: checkSupportedMediaType: allow additional media-types
2022-12-03 14:37:34 +01:00
Sebastiaan van Stijn
99d5b53b99
Merge pull request #44572 from crazy-max/23.0_backport_ci-buildkit-fix
[23.0 backport] ci(buildkit): update buildkit ref to fix issue with alpine image
2022-12-02 21:24:42 +01:00
Kevin Alvarez
7620f72fb8
ci(buildkit): update buildkit ref to fix issue with alpine image
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 381fa4afca)
2022-12-02 18:52:33 +01:00
Sebastiaan van Stijn
304c56f7f5
distribution: checkSupportedMediaType: allow additional media-types
This addresses a regression introduced in 407e3a4552,
which turned out to be "too strict", as there's old images that use, for example;

    docker pull python:3.5.1-alpine
    3.5.1-alpine: Pulling from library/python
    unsupported media type application/octet-stream

Before 407e3a4552, such mediatypes were accepted;

    docker pull python:3.5.1-alpine
    3.5.1-alpine: Pulling from library/python
    e110a4a17941: Pull complete
    30dac23631f0: Pull complete
    202fc3980a36: Pull complete
    Digest: sha256:f88925c97b9709dd6da0cb2f811726da9d724464e9be17a964c70f067d2aa64a
    Status: Downloaded newer image for python:3.5.1-alpine
    docker.io/library/python:3.5.1-alpine

This patch copies the additional media-types, using the list of types that
were added in a215e15cb1, which fixed a
similar issue.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit a6a539497a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-02 01:31:51 +01:00
Sebastiaan van Stijn
b21e8f72f2
Merge pull request #44563 from thaJeztah/23.0_backport_seccomp_block_af_vsock
[23.0 backport] seccomp: block socket calls to AF_VSOCK in default profile
2022-12-01 21:39:46 +01:00
Sebastiaan van Stijn
6875e7f1be
seccomp: block socket calls to AF_VSOCK in default profile
This syncs the seccomp-profile with the latest changes in containerd's
profile, applying the same changes as 17a9324035

Some background from the associated ticket:

> We want to use vsock for guest-host communication on KubeVirt
> (https://github.com/kubevirt/kubevirt). In KubeVirt we run VMs in pods.
>
> However since anyone can just connect from any pod to any VM with the
> default seccomp settings, we cannot limit connection attempts to our
> privileged node-agent.
>
> ### Describe the solution you'd like
> We want to deny the `socket` syscall for the `AF_VSOCK` family by default.
>
> I see in [1] and [2] that AF_VSOCK was actually already blocked for some
> time, but that got reverted since some architectures support the `socketcall`
> syscall which can't be restricted properly. However we are mostly interested
> in `arm64` and `amd64` where limiting `socket` would probably be enough.
>
> ### Additional context
> I know that in theory we could use our own seccomp profiles, but we would want
> to provide security for as many users as possible which use KubeVirt, and there
> it would be very helpful if this protection could be added by being part of the
> DefaultRuntime profile to easily ensure that it is active for all pods [3].
>
> Impact on existing workloads: It is unlikely that this will disturb any existing
> workload, becuase VSOCK is almost exclusively used for host-guest commmunication.
> However if someone would still use it: Privileged pods would still be able to
> use `socket` for `AF_VSOCK`, custom seccomp policies could be applied too.
> Further it was already blocked for quite some time and the blockade got lifted
> due to reasons not related to AF_VSOCK.
>
> The PR in KubeVirt which adds VSOCK support for additional context: [4]
>
> [1]: https://github.com/moby/moby/pull/29076#commitcomment-21831387
> [2]: dcf2632945
> [3]: https://kubernetes.io/docs/tutorials/security/seccomp/#enable-the-use-of-runtimedefault-as-the-default-seccomp-profile-for-all-workloads
> [4]: https://github.com/kubevirt/kubevirt/pull/8546

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 57b229012a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-01 14:09:46 +01:00
Sebastiaan van Stijn
bb2eab21c6
Merge pull request #44560 from vvoland/client-sharedsize-2206
[22.06 backport] client/list: Handle SharedSize
2022-12-01 09:16:40 +01:00
Sebastiaan van Stijn
cfc4677f62
Merge pull request #44557 from thaJeztah/22.06_backport_generate_authors
[22.06 backport] AUTHORS: regenerate, cleanup, refactor
2022-12-01 00:52:00 +01:00
Paweł Gronowski
978690e4f9
client/list: Handle SharedSize
This makes the `ImageList` function to add `shared-size=1` to the url
query when user caller sets the SharedSize.
SharedSize support was introduced in API version 1.42. This field was
added to the options struct, but client wasn't adjusted.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 3d97f1e22d)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2022-11-30 17:14:21 +01:00
Sebastiaan van Stijn
44eb640a1b
Merge pull request #44556 from thaJeztah/22.06_backport_ci_fixes
[22.06 backport] assorted CI and packaging fixes
2022-11-30 14:58:52 +01:00
Akihiro Suda
69ef0358c3
Merge pull request #44555 from thaJeztah/22.06_backport_remove_outdated_comment
[22.06 backport] vendor.mod: remove outdated comment about replaced module
2022-11-30 22:46:07 +09:00
Sebastiaan van Stijn
fc72ed9760
Merge pull request #44549 from thaJeztah/22.06_backport_search_remove_id
[22.06 backport] registry: session: remove unused id
2022-11-30 12:09:01 +01:00
Bjorn Neergaard
da6bb8c408
AUTHORS: regenerate
Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit e1c3305015)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-30 12:06:21 +01:00
Bjorn Neergaard
a889a17a63
.mailmap: cleanup and additions
Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
(cherry picked from commit 672383bc56)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-30 12:06:21 +01:00
Bjorn Neergaard
6f581c1808
hack/generate-authors.sh: refactor and simplify
Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit b94d1604a9)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-30 12:06:21 +01:00
CrazyMax
de3143c6b9
Jenkinsfile: Fix dev image build fox ppc64le/s390x archs
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit f7e59cbccc)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-30 11:16:51 +01:00
CrazyMax
f547f2f3c0
Makefile: always use buildx
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit bade242ddd)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-30 11:16:36 +01:00
CrazyMax
0c4b6b1742
ci: define timeout for jobs
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 426e3926ef)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-30 11:16:24 +01:00
Bjorn Neergaard
f088bcadd5
ignorefiles: cleanup
Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
(cherry picked from commit 4f17d17009)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-30 11:15:56 +01:00
Sebastiaan van Stijn
5770145433
vendor.mod: remove outdated comment about replaced module
The replace was removed in 64f9ea1cf5, but I
forgot to remove the comment.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 6326ad1729)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-30 11:12:37 +01:00
Sebastiaan van Stijn
d15fe0d782
registry: session: remove unused id
This removes the dependency on github.com/docker/docker/pkg/stringid

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit a44f547343)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-29 19:13:02 +01:00
Sebastiaan van Stijn
98040b95a7
Merge pull request #44536 from thaJeztah/22.06_backport_protobuf_extensions_fix
[22.06 backport] go.mod: golang_protobuf_extensions v1.0.4 - prevent incompatible versions
2022-11-25 17:42:20 +01:00
Sebastiaan van Stijn
546005804c
go.mod: golang_protobuf_extensions v1.0.4 - prevent incompatible versions
This module made a whoopsie, and updated to `google.golang.org/protobuf`
in a patch release, but `google.golang.org/protobuf` is not backward
compatible with `github.com/golang/protobuf`.

Updating the minimum version to v1.0.4 which corrects this, to prevent
users of containerd as a module from accidentally pulling in the wrong
version:

- v1.0.3 switched to use `google.golang.org/protobuf`; https://github.com/matttproud/golang_protobuf_extensions/compare/v1.0.2..v1.0.3
- This was reverted in v1.0.4 (which is the same as v1.0.2); https://github.com/matttproud/golang_protobuf_extensions/compare/v1.0.3..v1.0.4
- And a `v2` was created instead; https://github.com/matttproud/golang_protobuf_extensions/releases/tag/v2.0.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit e1058e6bc3)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-25 15:20:11 +01:00
Sebastiaan van Stijn
a8184baf3b
Merge pull request #44523 from crazy-max/22.06_dockerfile-vpnkit-platform
[22.06 backport] Dockerfile: remove hardcoded platforms for vpnkit stage
2022-11-25 00:47:12 +01:00
Sebastiaan van Stijn
e571db3846
Merge pull request #44522 from crazy-max/22.06_go-autogen
[22.06 backport] hack: remove obsolete sources for go-autogen
2022-11-25 00:46:53 +01:00
Sebastiaan van Stijn
a913b5ad7e
Merge pull request #44519 from thaJeztah/22.06_backport_bump_swarmkit3
[22.06 backport] vendor: github.com/moby/swarmkit/v2 v2.0.0-20221123162438-b17f02f0a054
2022-11-24 17:42:42 +01:00
Sebastiaan van Stijn
73a98393c6
vendor: github.com/moby/swarmkit/v2 v2.0.0-20221123162438-b17f02f0a054
Conflicts:
       vendor.mod

Conflict because code.cloudfoundry.org/clock moved to a direct dependency in
vendor.mod on master branch since 342b44bf20

full diff: 6341884e5f...b17f02f0a0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 64f9ea1cf5)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-24 14:47:30 +01:00
CrazyMax
acb8204a7f
Dockerfile: remove hardcoded platforms for vpnkit stage
Current Dockerfile downloads vpnkit for both linux/amd64
and linux/arm64 platforms even if target platform does not
match. This change will download vpnkit only if target
platform matches, otherwise it will just use a dummy scratch
stage.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 8a46a2a364)
2022-11-24 14:21:28 +01:00
CrazyMax
313f105443
hack: remove obsolete sources for go-autogen
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 40069797ef)
2022-11-24 14:20:48 +01:00
Sebastiaan van Stijn
2b1ba3ea6b
vendor: github.com/prometheus/client_golang v1.13.0
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit a5898e3a2d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-24 11:00:13 +01:00
Sebastiaan van Stijn
f493b770a9
vendor: github.com/aws/aws-sdk-go v1.37.0
full diff: https://github.com/aws/aws-sdk-go/compare/v1.31.6...v1.37.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 2cc6a86fd3)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-24 11:00:13 +01:00
Sebastiaan van Stijn
36430f7970
vendor: google.golang.org/grpc v1.48.0
full diff: https://github.com/grpc/grpc-go/compare/v1.47.0...v1.48.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 8f1bc3a3b7)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-24 11:00:13 +01:00
Sebastiaan van Stijn
fb24b99a2b
vendor: go.uber.org/zap v1.21.0
full diff: https://github.com/uber-go/zap/compare/v1.17.0...v1.21.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit b0e20e1b3c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-24 11:00:13 +01:00
Sebastiaan van Stijn
0c65191c49
vendor: go.uber.org/multierr v1.8.0
full diff: https://github.com/uber-go/multierr/compare/v1.6.0...v1.8.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 066fb6c69e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-24 11:00:13 +01:00
Sebastiaan van Stijn
bf78bf3458
vendor: go.uber.org/atomic v1.9.0
full diff: https://github.com/uber-go/atomic/compare/v1.7.0...v1.9.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit b41580d66e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-24 11:00:13 +01:00
Sebastiaan van Stijn
dcf06b3f5f
vendor: github.com/jmespath/go-jmespath v0.4.0
no code changes in vendored files

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 341c9e77a8)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-24 11:00:12 +01:00
Sebastiaan van Stijn
aba1d597bc
vendor: google.golang.org/genproto v0.0.0-20220706185917-7780775163c4
no changes in vendored files

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 9a8b46518b)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-24 11:00:12 +01:00
Sebastiaan van Stijn
e37ac41afb
vendor: github.com/fernet/fernet-go v0.0.0-20211208181803-9f70042a33ee
Fixes a potential panic.

full diff: 9eac43b88a...9f70042a33

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 1e48b64538)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-24 11:00:12 +01:00
Sebastiaan van Stijn
963e3ec65c
vendor: google.golang.org/protobuf v1.28.1
indirect dependency, but updating it in a separate commit

full diff: https://github.com/protocolbuffers/protobuf-go/compare/v1.28.0...v1.28.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 4113a88523)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-24 11:00:12 +01:00
Sebastiaan van Stijn
526203dd7f
vendor: github.com/spf13/cobra v1.6.1
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 26c4429f7e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-24 11:00:12 +01:00
Sebastiaan van Stijn
f528e2ab96
vendor: github.com/moby/term v0.0.0-20221120202655-abb19827d345
no significant changes in vendored code, other than updating build-tags
for go1.17, but removes some dependencies from the module, which can
help with future updates;

full diff: 3f7ff695ad...abb19827d3

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 61f266f660)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-24 10:59:17 +01:00
Sebastiaan van Stijn
3989be2f7b
vendor: github.com/bsphere/le_go v0.0.0-20200109081728-fc06dab2caa8
updates the "logentries" dependency;

- checking error when calling output
- Support Go Modules

full diff: 7a984a84b5...fc06dab2ca

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 8d5eebcc6e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-24 10:59:06 +01:00
Sebastiaan van Stijn
d406a5fd22
Merge pull request #44499 from thaJeztah/22.06_backport_update_go_radix
[22.06 backport] vendor: remove most "replace" rules and update github.com/armon/go-radix
2022-11-23 12:52:58 +01:00
Sebastiaan van Stijn
394f6c14ad
Merge pull request #44496 from crazy-max/22.06_frozen-script-variant
[22.06 backport] Dockerfile: variant support in frozen-images stage
2022-11-20 21:12:02 +01:00
Sebastiaan van Stijn
77a01aaec7
Merge pull request #44475 from thaJeztah/22.06_backport_config_fix_panic
[22.06 backport] daemon/config: use strings.Cut(), fix panic in BuilderGCFilter
2022-11-19 00:18:19 +01:00
Sebastiaan van Stijn
df2427022a
vendor.mod: add comment about replaced dependency
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 30b0cb0cd4)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-18 17:35:08 +01:00
Sebastiaan van Stijn
9e4c508b55
vendor.mod: remove replace for github.com/rexray/gocsi
While this replace was needed in swarmkit itself, it looks like
it doesn't cause issues when removed in this repository, so
let's remove it here.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 62a4a45a72)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-18 17:35:07 +01:00
Sebastiaan van Stijn
cb358e8a19
vendor: github.com/armon/go-radix v1.0.1-0.20221118154546-54df44f2176c
Previously we had to use a replace rule, as later versions of this
module resulted in a panic. This issue was fixed in:
f30034d788

Which means we can remove the replace rule, and update the dependency.
No new release was tagged yet, so sticking to a "commit" for now.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit a2d758acc9)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-18 17:35:07 +01:00
Sebastiaan van Stijn
2bc33b4c26
Merge pull request #44492 from thaJeztah/22.06_backport_update_gowinres
[22.06 backport] update github.com/tc-hib/go-winres v0.3.0 to fix schema version in manifest
2022-11-18 15:44:56 +01:00
CrazyMax
3768c71d9e
Dockerfile: variant support in frozen-images stage
using TARGETVARIANT in frozen-images stage implies changes in
`download-frozen-image-v2.sh` script to add support for variants
so we are able to build against more platforms.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 25dc760162)
2022-11-18 15:42:48 +01:00
Sebastiaan van Stijn
7b9e86f789
update github.com/tc-hib/go-winres v0.3.0 to fix schema version in manifest
- Fix xml schema version in manifest
- Provide more verbose error on failed git tag resolution

full diffs:

- https://github.com/tc-hib/go-winres/compare/v0.2.3...v0.3.0
- https://github.com/tc-hib/winres/compare/v0.1.5...v0.1.6

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit ca807edac0)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-18 00:27:16 +01:00
Sebastiaan van Stijn
214e6363b3
Merge pull request #44487 from thaJeztah/22.06_backport_update_gotestsum
[22.06 backport] update gotestsum to v1.8.2
2022-11-18 00:26:15 +01:00
Sebastiaan van Stijn
5052c38846
update gotestsum to v1.8.2
release notes: https://github.com/gotestyourself/gotestsum/releases/tag/v1.8.2

- Show shuffle seed
- Update tests, and cleanup formats
- Update dependencies
- Test against go1.19, remove go1.15
- Add project name to junit.xml output
- Adding in support for s390x and ppc64le

full diff: https://github.com/gotestyourself/gotestsum/compare/v1.8.1...v1.8.2

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 882ddf4b16)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-17 17:47:28 +01:00
Sebastiaan van Stijn
d15be0c54d
Merge pull request #44415 from thaJeztah/22.06_backport_pkg_thining
[22.06 backport] clean-up various pkg/ changes
2022-11-17 10:48:50 +01:00
Sebastiaan van Stijn
27982c186e
Merge pull request #44480 from neersighted/drop_changelog_22.06
[22.06 backport] cleanup: drop historical CHANGELOG.md
2022-11-17 10:48:08 +01:00
Sebastiaan van Stijn
9d990cbae8
Merge pull request #44471 from thaJeztah/22.06_backport_containerd_v1.6.10
[22.06 backport] update containerd v1.6.10 (binary and vendor)
2022-11-17 10:46:14 +01:00
Bjorn Neergaard
3508cfb149
hack/validate: drop changelog-related steps
Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
(cherry picked from commit be18f92bf4)
Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
2022-11-16 13:35:40 -07:00
Bjorn Neergaard
67633130c6
cleanup: drop historical CHANGELOG.md
The file will still be available in Git history; we should drop it
however as it is misleading and obsolete.

Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
(cherry picked from commit ec1bb21649)
Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
2022-11-16 13:35:38 -07:00
Sebastiaan van Stijn
2e13f771f3
Merge pull request #44477 from neersighted/drop_derek_22.06
[22.06 backport] cleanup: drop .DEREK.yml
2022-11-16 21:22:43 +01:00
Sebastiaan van Stijn
bbf3f33dc8
Merge pull request #44473 from thaJeztah/22.06_backport_rootlesskit_1.1.0
[22.06 backport] update RootlessKit to v1.1.0
2022-11-16 18:45:07 +01:00
Bjorn Neergaard
2dc7a1dc25
cleanup: drop .DEREK.yml
Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
(cherry picked from commit e8ad01594a)
Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
2022-11-16 09:10:46 -07:00
Sebastiaan van Stijn
fc657692c7
daemon/config: use strings.Cut(), fix panic in BuilderGCFilter
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit b529870558)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-16 15:23:00 +01:00
Sebastiaan van Stijn
e75fa6684c
daemon: use strconv instead of fmt.Sprintf()
Also cleaning up some errors

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 56e64270f3)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-16 15:21:05 +01:00
Akihiro Suda
abe6b3dc9b
rootlesskit.installer: s/vendor.conf/vendor.mod/
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit 7ca03c1a79)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-16 14:33:40 +01:00
Akihiro Suda
297f224a92
update RootlessKit to v1.1.0
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit d9fb730148)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-16 14:33:40 +01:00
Akihiro Suda
f48f4dde24
vendor: github.com/rootless-containers/rootlesskit v1.1.0
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit 08516af897)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-16 14:33:37 +01:00
Sebastiaan van Stijn
cd8873dd3d
Merge pull request #44468 from thaJeztah/22.06_backport_golang_x_releases_step1
[22.06 backport] vendor: golang.org/x/crypto v0.1.0 and other golang.org/x/.. dependencies
2022-11-16 14:11:02 +01:00
Jintao Zhang
2dce69e001
update containerd binary to v1.6.10
Signed-off-by: Jintao Zhang <zhangjintao9020@gmail.com>
(cherry picked from commit a5979a2106)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-16 14:03:34 +01:00
Jintao Zhang
5c4dc48995
vendor: github.com/containerd/containerd v1.6.10
Signed-off-by: Jintao Zhang <zhangjintao9020@gmail.com>
(cherry picked from commit 4e5c3b82cb)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-16 14:03:03 +01:00
Sebastiaan van Stijn
10fa0d5321
vendor: golang.org/x/oauth2 v0.1.0
The golang.org/x/ projects are now doing tagged releases.

Some notable changes:

- authhandler: Add support for PKCE
- Introduce new AuthenticationError type returned by errWrappingTokenSource.Token
- Add support to set JWT Audience in JWTConfigFromJSON()
- google/internal: Add AWS Session Token to Metadata Requests
- go.mod: update vulnerable net library
- google: add support for "impersonated_service_account" credential type.
- google/externalaccount: add support for workforce pool credentials

full diff: https://github.com/golang/oauth2/compare/2bc19b11175f...v0.1.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit a6cb8efd81)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-15 17:06:19 +01:00
Sebastiaan van Stijn
356f483038
vendor: golang.org/x/crypto v0.1.0
The golang.org/x/ projects are now doing tagged releases.

full diff: https://github.com/golang/crypto/compare/3147a52a75dd...v0.1.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 9d7bd47cb6)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-15 17:05:25 +01:00
Sebastiaan van Stijn
c0edbfd621
vendor: golang.org/x/net v0.1.0
The golang.org/x/ projects are now doing tagged releases.

full diff:

- https://github.com/golang/net/compare/f3363e06e74c...v0.1.0
- https://github.com/golang/text/compare/v0.3.7...v0.4.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 79f9ffd401)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-15 14:56:21 +01:00
Sebastiaan van Stijn
e46e43470b
vendor: golang.org/x/time v0.1.0
The golang.org/x/ projects are now doing tagged releases.

full diff: https://github.com/golang/time/compare/f0f3c7e86c11...v0.1.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 71fa64a272)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-15 14:56:12 +01:00
Sebastiaan van Stijn
d4e2341f93
vendor: golang.org/x/sync v0.1.0
The golang.org/x/ projects are now doing tagged releases.

full diff: https://github.com/golang/sync/compare/036812b2e83c...v0.1.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 4965f19626)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-15 14:56:04 +01:00
Sebastiaan van Stijn
e32bfd347c
vendor: golang.org/x/sys v0.1.0
The golang.org/x/ projects are now doing tagged releases.

full diff: https://github.com/golang/sys/compare/84dc82d7e875...v0.1.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 4bb95eef6f)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-15 14:55:56 +01:00
Sebastiaan van Stijn
65c7f3bac3
pkg/loopback: use ioctl helpers from x/sys/unix
Use the IoctlRetInt, IoctlSetInt and IoctlLoopSetStatus64 helper
functions defined in the golang.org/x/sys/unix package instead of
manually wrapping these using a locally defined function.

Inspired by 3cc3d8a560

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit c7c02eea81)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-15 14:55:17 +01:00
Sebastiaan van Stijn
5f35b157a3
vendor: golang.org/x/sys v0.0.0-20221006211917-84dc82d7e875
full diff: 3c1f35247d...84dc82d7e8

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 6742f74e0e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-15 14:54:32 +01:00
Sebastiaan van Stijn
76e132ed56
Merge pull request #44448 from neersighted/actions_deprecations_22.06
[22.06 backport] ci(actions): migrate to file-based commands
2022-11-15 14:45:56 +01:00
Sebastiaan van Stijn
6f7ee1c942
Merge pull request #44449 from thaJeztah/22.06_backport_bump_gotest_tools
[22.06 backport] vendor: gotest.tools/v3 v3.4.0, github.com/google/go-cmp v0.5.9, remove golang.org/x/xerrors
2022-11-15 11:27:11 +01:00
Akihiro Suda
f476deac40
Merge pull request #44456 from thaJeztah/22.06_backport_bump_buildkit_v0.10.6
[22.06 backport] vendor: github.com/moby/buildkit v0.10.6
2022-11-15 10:23:56 +09:00
Sebastiaan van Stijn
11973d0c0a
vendor: github.com/moby/buildkit v0.10.6
full diff: https://github.com/moby/buildkit/compare/v0.10.5...v0.10.6

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 06e4b7d1f8)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-14 20:38:51 +01:00
Sebastiaan van Stijn
251610397c
vendor: gotest.tools/v3 v3.4.0
- removes github.com/spf13/pflag dependency
- removes use of deprecated io/ioutil package
- drops support for go1.16

full diff: https://github.com/gotestyourself/gotest.tools/compare/v3.3.0...v3.4.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit d43bc26717)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-11 18:16:39 +01:00
Sebastiaan van Stijn
83f90039ef
vendor: github.com/google/go-cmp v0.5.9 to remove golang.org/x/xerrors dep
full diff: https://github.com/google/go-cmp/compare/v0.5.7...v0.5.9

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 57ba2df970)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-11 18:16:39 +01:00
Sebastiaan van Stijn
2fd846d40f
vendor: gotest.tools v3.3.0
full diff: https://github.com/gotestyourself/gotest.tools/compare/v3.2.0...v3.3.0

- golden: accept -update for updating files
- assert: golden variables

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 3e1601a980)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-11 18:16:34 +01:00
Bjorn Neergaard
f9ab209417
chore: update supported go version to 1.18+
The 1.16 `io/fs` compatibility code was being built on 1.18 and 1.19.
Drop it completely as 1.16 is long EOL, and additionally drop 1.17 as it
has been EOL for a month and 1.18 is both the minimum Go supported by
the 20.10 branch, as well as a very easy jump from 1.17.

Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
(cherry picked from commit 85fa72c599)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-11 18:01:58 +01:00
Bjorn Neergaard
bfca3185ee
ci(actions): bump outdated actions on Linux
Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
(cherry picked from commit 6a02afa56f)
Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
2022-11-10 16:48:37 -07:00
Bjorn Neergaard
7f45eb041c
ci(actions): migrate to file-based commands
Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
(cherry picked from commit 0557569947)
Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
2022-11-10 16:48:33 -07:00
Tianon Gravi
b76a60dee6
Merge pull request #44414 from thaJeztah/22.06_backport_rm_deprecated_arm_fallback
[22.06 backport] Remove long-deprecated "arm" fallback
2022-11-10 12:21:14 -08:00
Sebastiaan van Stijn
4acfbaba1e
Merge pull request #44430 from thaJeztah/22.06_swap_digestset
[22.06 backport] replace distribution/digestset with opencontainers/go-digest/digestset
2022-11-10 21:09:01 +01:00
Sebastiaan van Stijn
e749a31322
Merge pull request #44416 from thaJeztah/22.06_backport_enable_deprecated_check
[22.06 backport] Revert "validation: temporarily allows changes in integration-cli"
2022-11-10 18:02:41 +01:00
Sebastiaan van Stijn
7370bbc034
replace distribution/digestset with opencontainers/go-digest/digestset
opencontainers/go-digest is a 1:1 copy of the one in distribution. It's no
longer used in distribution itself, so may be removed there at some point.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 6174d00c03)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-09 10:22:38 +01:00
Samuel Karp
38152f4d5b
Merge pull request #44411 from thaJeztah/22.06_backport_bump_go1.19.3
fixes https://github.com/golang/go/issues/56309
2022-11-08 19:12:03 -08:00
Sebastiaan van Stijn
21feb1808d
Revert "validation: temporarily allows changes in integration-cli"
This reverts commit 7ed823ead9.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 9b71a46899)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-05 18:35:47 +01:00
Sebastiaan van Stijn
5e15ce3a4a
pkg/directory: remove unused MoveToSubdir() utility
This utility was added in 442b45628e as part of
user-namespaces, and first used in 44e1023a93 to
set up the daemon root, and move the existing content;
44e1023a93/daemon/daemon_experimental.go (L68-L71)

A later iteration no longer _moved_ the existing root directory, and removed the
use of `directory.MoveToSubdir()` e8532023f2

It looks like there's no external consumers of this utility, so we should be
save to remove it.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 26659d5eb8)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-05 18:31:05 +01:00
Sebastiaan van Stijn
92b96ac2ed
pkg/directory: minor refactor of Size()
- separate exported function from implementation, to allow for GoDoc to be
  maintained in a single location.
- don't use named return variables (no "bare" return, and potentially shadowing
  variables)
- reverse the `os.IsNotExist(err) && d != dir` condition, putting  the "lighter"
  `d != dir` first.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit bd6217bb74)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-05 18:31:05 +01:00
Sebastiaan van Stijn
e0b105623e
pkg/system: unconvert
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit ab677c41ea)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-05 18:30:44 +01:00
Sebastiaan van Stijn
9d86e1d204
pkg/system: move GetExitCode() to pkg/idtools, and un-export
This utility was only used in a single place, and had no external consumers.
Move it to where it's used.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 07b1aa822c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-05 18:30:44 +01:00
Sebastiaan van Stijn
3a946f5291
pkg/system: remove Umask() utility
It was only used in a couple of places, and in most places shouldn't be used
as those locations were in unix/linux-only files, so didn't need the wrapper.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 4347080b46)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-05 18:30:26 +01:00
Sebastiaan van Stijn
cf1e138ab1
pkg/directory: Size(): add back type-casts to account for platform differences
I noticed the comment above this code, but didn't see a corresponding type-cast.
Looking at this file's history, I found that these were removed as part of
2f5f0af3fd, which looks to have overlooked some
deliberate type-casts.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 0a861e68df)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-05 18:30:16 +01:00
Tianon Gravi
7175841ebd
Remove long-deprecated "arm" fallback
This fallback is used when we filter the manifest list by the user-provided platform and find no matches such that we match the previous Docker behavior (before it supported variant matching).  This has been deprecated long enough that I think it's time we finally stop supporting this weird fallback, especially since it makes for buggy behavior like `docker pull --platform linux/arm/v5 alpine:3.16` leading to a `linux/arm/v6` image being pulled (I specified a variant, every manifest list entry specifies a variant, so clearly the only behavior I as a user could reasonably expect is an error that `linux/arm/v5` is not supported, but instead I get an explicitly incompatible image despite doing everything I as a user can to prevent that situation).

Signed-off-by: Tianon Gravi <admwiggin@gmail.com>
(cherry picked from commit 5bc17c3e54)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-05 18:23:43 +01:00
Cory Snider
f3e180b704
Update to Go 1.19.3 to address CVE-2022-41716
On Windows, syscall.StartProcess and os/exec.Cmd did not properly
    check for invalid environment variable values. A malicious
    environment variable value could exploit this behavior to set a
    value for a different environment variable. For example, the
    environment variable string "A=B\x00C=D" set the variables "A=B" and
    "C=D".

    Thanks to RyotaK (https://twitter.com/ryotkak) for reporting this
    issue.

    This is CVE-2022-41716 and Go issue https://go.dev/issue/56284.

This Go release also fixes https://github.com/golang/go/issues/56309, a
runtime bug which can cause random memory corruption when a goroutine
exits with runtime.LockOSThread() set. This fix is necessary to unblock
work to replace certain uses of pkg/reexec with unshared OS threads.

Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit f9d4589976)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-05 17:52:37 +01:00
Sebastiaan van Stijn
afdc9a804a
Merge pull request #44404 from neersighted/swarmkit_revendor_22.06
[22.06 backport] vendor: github.com/moby/swarmkit/v2 v2.0.0-20221102165002-6341884e5fc9
2022-11-03 22:27:56 +01:00
Sebastiaan van Stijn
e24277883f
Merge pull request #44405 from vvoland/oci-artifacts-error-2206
[22.06 backport] distribution: Error when pulling OCI artifacts
2022-11-03 22:27:37 +01:00
Paweł Gronowski
07e84005ac
distribution: Error when pulling OCI artifacts
Currently an attempt to pull a reference which resolves to an OCI
artifact (Helm chart for example), results in a bit unrelated error
message `invalid rootfs in image configuration`.

This provides a more meaningful error in case a user attempts to
download a media type which isn't image related.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2022-11-03 19:41:51 +01:00
Bjorn Neergaard
39d3d3db56
vendor: github.com/moby/swarmkit/v2 v2.0.0-20221102165002-6341884e5fc9
full diff: 48dd89375d...6341884e5f

Pulls in a set of fixes to SwarmKit's nascent Cluster Volumes support
discovered during subsequent development and testing.

Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
(cherry picked from commit 57c2545cd5)
Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
2022-11-03 12:30:53 -06:00
Cory Snider
4b79d9078a
Merge pull request #44400 from corhere/backport-22.06/fix-task-delete-on-failed-start
[22.06 backport] Fix containerd task deletion after failed start
2022-11-02 18:15:19 -04:00
Cory Snider
1e0f2186a9 Fix containerd task deletion after failed start
Deleting a containerd task whose status is Created fails with a
"precondition failed" error. This is because (aside from Windows)
a process is spawned when the task is created, and deleting the task
while the process is running would leak the process if it was allowed.
libcontainerd mistakenly tries to clean up from a failed start by
deleting the created task, which will always fail with the
aforementioned error. Change it to pass the `WithProcessKill` delete
option so the cleanup has a chance to succeed.

Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit 1bef9e3fbf)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-11-02 16:59:22 -04:00
Cory Snider
4404c36460
Merge pull request #44376 from corhere/backport-22.06/gh-44363
[22.06 backport] Fix the max-concurrent-downloads and max-concurrent-uploads configs documentation
2022-10-31 13:00:46 -04:00
Cory Snider
75634f9a1e daemon: fix docs for config-default constants
Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit ad4073edc1)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-10-31 11:51:27 -04:00
Luis Henrique Mulinari
ad11d3f232 Fix the max-concurrent-downloads and max-concurrent-uploads configs documentation
This fix tries to address issues raised in #44346.
The max-concurrent-downloads and max-concurrent-uploads limits are applied for the whole engine and not for each pull/push command.

Signed-off-by: Luis Henrique Mulinari <luis.mulinari@gmail.com>
(cherry picked from commit 6c0aa5b00a)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-10-31 11:51:27 -04:00
Sebastiaan van Stijn
cbaf1808cb
Merge pull request #44360 from neersighted/backport_44224
[22.06 backport] Fix force-remove for cluster volumes
2022-10-26 10:36:39 -04:00
Drew Erny
03015fe6de
fix force remove for cluster volumes
Signed-off-by: Drew Erny <derny@mirantis.com>
(cherry picked from commit 3246db3755)
Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
2022-10-25 15:18:34 -06:00
Sebastiaan van Stijn
fa3804f8ba
Merge pull request #44357 from thaJeztah/22.06_backport_busybox_w32_img
[22.06 backport] integration: download busybox-w32 from GitHub Release
2022-10-25 07:44:47 -04:00
Sebastiaan van Stijn
4c1a3f096c
Merge pull request #44355 from thaJeztah/22.06_vendor_containerd_1.6.9
[22.06 backport] vendor: github.com/containerd/containerd v1.6.9
2022-10-24 17:58:09 -04:00
CrazyMax
09a2f7a667
integration: download busybox-w32 from GitHub Release
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 4f1d1422de)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-24 17:04:10 -04:00
Brian Goff
02e02e512f
Merge pull request #44352 from thaJeztah/22.06_update_containerd_binary
[22.06 backport] update containerd binary to v1.6.9
2022-10-24 11:57:06 -07:00
Sebastiaan van Stijn
24de1f7adc
vendor: github.com/containerd/containerd v1.6.9
release notes: https://github.com/containerd/containerd/releases/tag/v1.6.9

full diff: https://github.com/containerd/containerd/compare/v1.6.8...v1.6.9

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 04dc007c76)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-24 14:24:27 -04:00
Sebastiaan van Stijn
c4685540e4
update containerd binary to v1.6.9
release notes: https://github.com/containerd/containerd/releases/tag/v1.6.9

full diff: containerd/containerd@v1.6.8...v1.6.9

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit ac79a02ace)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-24 13:55:25 -04:00
Sebastiaan van Stijn
5aac513617
Merge pull request #44345 from thaJeztah/22.06_backport_go1.18_compat
[22.06 backport] builder/remotecontext/git: allow building on go1.18
2022-10-21 19:39:10 +02:00
Sebastiaan van Stijn
80dc5186ec
builder/remotecontext/git: allow building on go1.18
cmd.Environ() is new in go1.19, and not needed for this specific case.
Without this, trying to use this package in code that uses go1.18 will fail;

    builder/remotecontext/git/gitutils.go:216:23: cmd.Environ undefined (type *exec.Cmd has no field or method Environ)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 4fdc1bb1fb)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-21 17:44:20 +02:00
Sebastiaan van Stijn
f9cb47a052
Merge pull request #44341 from thaJeztah/22.06_backport_buildkit_skip_unit
[22.06 backport] gha: buildkit: remove "skip-integration-tests" from matrix
2022-10-21 14:21:14 +02:00
Sebastiaan van Stijn
5202b5c781
Merge pull request #44328 from thaJeztah/22.06_backport_ghsa-ambiguous-pull-by-digest
[22.06 backport] Validate digest in repo for pull by digest
2022-10-21 14:20:22 +02:00
Sebastiaan van Stijn
28c34259c7
Merge pull request #44297 from thaJeztah/22.06_backport_windows_bits
[22.06 backport] windows cleanups
2022-10-21 02:44:02 +02:00
Sebastiaan van Stijn
67ea873f61
Merge pull request #44325 from corhere/backport-22.06/fix-git-file-leak
[22.06 backport] builder: Isolate Git from local system
2022-10-21 02:11:56 +02:00
Sebastiaan van Stijn
f72c96c5c4
gha: buildkit: make checks more readable
GitHub uses these parameters to construct a name; removing the ./ prefix
to make them more readable (and add them back where it's used)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 0760c6f4e1)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-21 02:07:29 +02:00
Sebastiaan van Stijn
1bbb6f2454
gha: buildkit: remove "skip-integration-tests" from matrix
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit cfa2f9a2f2)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-21 02:07:26 +02:00
Sebastiaan van Stijn
c0be73f88d
skip TestImagePullStoredfDigestForOtherRepo() on Windows and rootless
- On Windows, we don't build and run a local  test registry (we're not running
  docker-in-docker), so we need to skip this test.
- On rootless, networking doesn't support this (currently)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 4f43cb660a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-21 01:50:09 +02:00
Brian Goff
727c4fdee3
Validate digest in repo for pull by digest
This is accomplished by storing the distribution source in the content
labels. If the distribution source is not found then we check to the
registry to see if the digest exists in the repo, if it does exist then
the puller will use it.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 27530efedb)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-21 01:50:09 +02:00
Sebastiaan van Stijn
b4c4be1f22
Revert "testutil/registry: remove unused WithStdout(), WithStErr() opts"
This reverts commit 1f21c4dd05.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 92eca900b0)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-21 01:50:09 +02:00
Sebastiaan van Stijn
7106874e39
Merge pull request #44338 from thaJeztah/22.06_backport_buildkit_testskips
[22.06 backport] gha: update buildkit to v0.10.5-6-ge27c8e24 to skip some tests
2022-10-21 01:48:00 +02:00
Sebastiaan van Stijn
4bef6f5510
gha: update buildkit to v0.10.5-6-ge27c8e24 to skip some tests
full diff: https://github.com/moby/buildkit/compare/v0.10.5...v0.10.5-6-ge27c8e24

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 201fdf67ac)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-20 23:50:16 +02:00
Cory Snider
f056df579a builder: add missing doc comment
Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-10-20 16:46:23 -04:00
Cory Snider
c062238ea4 builder: fix running git commands on Windows
Setting cmd.Env overrides the default of passing through the parent
process' environment, which works out fine most of the time, except when
it doesn't. For whatever reason, leaving out all the environment causes
git-for-windows sh.exe subprocesses to enter an infinite loop of
access violations during Cygwin initialization in certain environments
(specifically, our very own dev container image).

Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-10-20 16:46:23 -04:00
Cory Snider
20ff8a2380 builder: make git config isolation opt-in
While it is undesirable for the system or user git config to be used
when the daemon clones a Git repo, it could break workflows if it was
unconditionally applied to docker/cli as well.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-10-20 16:46:23 -04:00
Cory Snider
ca99cab891 builder: isolate git from local system
Prevent git commands we run from reading the user or system
configuration, or cloning submodules from the local filesystem.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-10-20 16:46:23 -04:00
Cory Snider
5829b244ec builder: explicitly set CWD for all git commands
Keep It Simple! Set the working directory for git commands by...setting
the git process's working directory. Git commands can be run in the
parent process's working directory by passing the empty string.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-10-20 16:46:23 -04:00
Cory Snider
3bc8fccc1b builder: modernize TestCheckoutGit
Make the test more debuggable by logging all git command output and
running each table-driven test case as a subtest.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-10-20 16:46:23 -04:00
Sebastiaan van Stijn
4a96094bf5
Merge pull request #44321 from thaJeztah/22.06_backport_bump_buildkit
[22.06 backport] vendor: github.com/moby/buildkit v0.10.5
2022-10-19 10:08:32 +02:00
Sebastiaan van Stijn
00b44caa69
vendor: github.com/moby/buildkit v0.10.5
https://github.com/moby/buildkit/releases/tag/v0.10.5

full diff: https://github.com/moby/buildkit/compare/v0.10.4...v0.10.5

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 0fc17c42af)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-18 22:39:56 +02:00
Sebastiaan van Stijn
1fcb1dd728
Merge pull request #44314 from tianon/22.06-distributable
[22.06 backport] registry: allow "allow-nondistributable-artifacts" for Docker Hub
2022-10-18 14:39:53 +02:00
Sebastiaan van Stijn
aaa8f96cc9 registry: allow "allow-nondistributable-artifacts" for Docker Hub
Previously, Docker Hub was excluded when configuring "allow-nondistributable-artifacts".
With the updated policy announced by Microsoft, we can remove this restriction;
https://techcommunity.microsoft.com/t5/containers/announcing-windows-container-base-image-redistribution-rights/ba-p/3645201

There are plans to deprecated support for foreign layers altogether in the OCI,
and we should consider to make this option the default, but as that requires
deprecating the option (and possibly keeping an "opt-out" option), we can look
at that separately.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 30e5333ce3)
2022-10-17 15:04:59 -07:00
Sebastiaan van Stijn
671bf589e2
Change restart delay for Windows service to 15s
Previously we waited for 60 seconds after the service faults to restart
it. However, there isn't much benefit to waiting this long. We expect
15 seconds to be a more reasonable delay.

Co-Authored-by: Kevin Parsons <kevpar@microsoft.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 624daf8d9e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-13 23:08:42 +02:00
Sebastiaan van Stijn
e1b240d6bd
cmd/dockerd: use golang.org/x/sys Service.SetRecoveryActions()
This is the equivalent of the local implementation.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 3c585e6567)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-13 23:08:42 +02:00
Sebastiaan van Stijn
18a54ed59c
cmd/dockerd: use golang.org/x/sys/windows.SetStdHandle()
golang.org/x/sys/windows now implements this, so we can use that
instead of a local implementation.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 6176ab5901)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-13 23:05:49 +02:00
Sebastiaan van Stijn
0c66bc948a
cmd/dockerd: replace deprecated windows.IsAnInteractiveSession()
The `IsAnInteractiveSession` was deprecated, and `IsWindowsService` is marked
as the recommended replacement.

For details, see 280f808b4a

> CL 244958 includes isWindowsService function that determines if a
> process is running as a service. The code of the function is based on
> public .Net implementation.
>
> IsAnInteractiveSession function implements similar functionality, but
> is based on an old Stackoverflow post., which is not as authoritative
> as code written by Microsoft for their official product.
>
> This change copies CL 244958 isWindowsService function into svc package
> and makes it public. The intention is that future users will prefer
> IsWindowsService to IsAnInteractiveSession.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit ffcddc908e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-13 23:05:48 +02:00
Sebastiaan van Stijn
a12d359c1a
daemon/graphdriver/windows: Remove() don't use defer() in a loop
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 75bdbf02a6)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-13 23:05:48 +02:00
Sebastiaan van Stijn
2d12e69c9f
daemon/graphdriver/windows: use go-winio.GetFileSystemType()
go-winio now defines this function, so we can consume that.

Note that there's a difference between the old implementation and the original
one (added in 1cb9e9b44e). The old implementation
had special handling for win32 error codes, which was removed in the go-winio
implementation in 0966e1ad56

As `go-winio.GetFileSystemType()` calls `filepath.VolumeName(path)` internally,
this patch also removes the `string(home[0])`, which is redundant, and could
potentially panic if an empty string would be passed.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 90431d1857)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-13 23:05:45 +02:00
Sebastiaan van Stijn
33ab36d6b3
Merge pull request #44279 from thaJeztah/22.06_backport_overlay_remove_kernel_check_override
[22.06 backport] daemon/graphdriver/overlay2: remove deprecated overrideKernelCheck
2022-10-13 22:35:45 +02:00
Sebastiaan van Stijn
fa10084a82
daemon/graphdriver/overlay2: remove deprecated overrideKernelCheck
Commit 955c1f881a (Docker v17.12.0) replaced
detection of support for multiple lowerdirs (as required by overlay2) to not
depend on the kernel version. The `overlay2.override_kernel_check` was still
used to print a warning that older kernel versions may not have full support.

After this, commit e226aea280 (Docker v20.10.0,
backported to v19.03.7) removed uses of the `overlay2.override_kernel_check`
option altogether, but we were still parsing it.

This patch changes the `parseOptions()` function to not parse the option,
printing a deprecation warning instead. We should change this to be an error,
but the  `overlay2.override_kernel_check` option was not deprecated in the
documentation, so keeping it around for one more release.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit e35700eb50)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-10 15:00:30 +02:00
Sebastiaan van Stijn
43ce8f7d24
integration/plugin: remove deprecated overlay2.override_kernel_check
It's no longer used since e226aea280

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit b43a7ac530)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-10 15:00:29 +02:00
Sebastiaan van Stijn
87d9d96ab0
Merge pull request #44262 from thaJeztah/22.06_backport_bump_selinux
[22.06 backport] vendor: github.com/opencontainers/selinux v1.10.2
2022-10-06 20:54:38 +02:00
Sebastiaan van Stijn
a5ecbf4d22
vendor: github.com/opencontainers/selinux v1.10.2
full diff: https://github.com/opencontainers/selinux/compare/v1.10.1...v1.10.2

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit a5e1baf3ab)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-06 16:48:58 +02:00
Sebastiaan van Stijn
99aa9bb766
Merge pull request #44260 from thaJeztah/22.06_backport_deprecate_pkg_fsutil
[22.06 backport] pkg/fsutils: deprecate in favor of containerd/continuity/fs
2022-10-06 02:50:07 +02:00
Brian Goff
6442025060
Merge pull request #44259 from thaJeztah/22.06_backport_volume_unnamed_label
[22.06 backport] Volume prune: only prune anonymous volumes by default
2022-10-05 17:01:24 -07:00
Sebastiaan van Stijn
ac6624773e
pkg/fsutils: deprecate in favor of containerd/continuity/fs
The pkg/fsutils package was forked in containerd, and later moved to
containerd/continuity/fs. As we're moving more bits to containerd, let's also
use the same implementation to reduce code-duplication and to prevent them from
diverging.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 5b6b42162b)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-05 23:52:55 +02:00
Brian Goff
4669418731
Volume prune: only prune anonymous volumes by default
This adds a new filter argument to the volume prune endpoint "all".
When this is not set, or it is a false-y value, then only anonymous
volumes are considered for pruning.

When `all` is set to a truth-y value, you get the old behavior.

This is an API change, but I think one that is what most people would
want.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 618f26ccbc)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-05 23:50:01 +02:00
Sebastiaan van Stijn
ff07aadeb0
Merge pull request #44245 from thaJeztah/22.06_backport_api_update_swagger_for_builder_version
[22.06 backport] docs: swagger: update description for default builder version
2022-10-05 10:56:20 +02:00
Samuel Karp
cde4767cbd
Merge pull request #44248 from thaJeztah/22.06_backport_bump_go_1.19.2 2022-10-05 00:29:49 -07:00
Sebastiaan van Stijn
1fe550cfc7
Update to go 1.19.2 to address CVE-2022-2879, CVE-2022-2880, CVE-2022-41715
From the mailing list:

We have just released Go versions 1.19.2 and 1.18.7, minor point releases.

These minor releases include 3 security fixes following the security policy:

- archive/tar: unbounded memory consumption when reading headers

  Reader.Read did not set a limit on the maximum size of file headers.
  A maliciously crafted archive could cause Read to allocate unbounded
  amounts of memory, potentially causing resource exhaustion or panics.
  Reader.Read now limits the maximum size of header blocks to 1 MiB.

  Thanks to Adam Korczynski (ADA Logics) and OSS-Fuzz for reporting this issue.

  This is CVE-2022-2879 and Go issue https://go.dev/issue/54853.

- net/http/httputil: ReverseProxy should not forward unparseable query parameters

  Requests forwarded by ReverseProxy included the raw query parameters from the
  inbound request, including unparseable parameters rejected by net/http. This
  could permit query parameter smuggling when a Go proxy forwards a parameter
  with an unparseable value.

  ReverseProxy will now sanitize the query parameters in the forwarded query
  when the outbound request's Form field is set after the ReverseProxy.Director
  function returns, indicating that the proxy has parsed the query parameters.
  Proxies which do not parse query parameters continue to forward the original
  query parameters unchanged.

  Thanks to Gal Goldstein (Security Researcher, Oxeye) and
  Daniel Abeles (Head of Research, Oxeye) for reporting this issue.

  This is CVE-2022-2880 and Go issue https://go.dev/issue/54663.

- regexp/syntax: limit memory used by parsing regexps

  The parsed regexp representation is linear in the size of the input,
  but in some cases the constant factor can be as high as 40,000,
  making relatively small regexps consume much larger amounts of memory.

  Each regexp being parsed is now limited to a 256 MB memory footprint.
  Regular expressions whose representation would use more space than that
  are now rejected. Normal use of regular expressions is unaffected.

  Thanks to Adam Korczynski (ADA Logics) and OSS-Fuzz for reporting this issue.

  This is CVE-2022-41715 and Go issue https://go.dev/issue/55949.

View the release notes for more information: https://go.dev/doc/devel/release#go1.19.2

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 7b4e4c08b5)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-04 20:42:39 +02:00
Sebastiaan van Stijn
edef49eeac
docs: swagger: update description for default builder version
Commit 7b153b9e28 updated the main
swagger file, but didn't update the v1.42 version used for the
documentation as it wasn't created yet at the time.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 271243d382)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-04 19:00:47 +02:00
Sebastiaan van Stijn
dbcd0e7aee
Merge pull request #44237 from cpuguy83/22.06_fix_restore_volumerefs
[22.06] Fix live-restore w/ restart policies + volume refs
2022-10-03 23:32:29 +02:00
Brian Goff
0a87dc9f71 Fix live-restore w/ restart policies + volume refs
Before this change restarting the daemon in live-restore with running
containers + a restart policy meant that volume refs were not restored.
This specifically happens when the container is still running *and*
there is a restart policy that would make sure the container was running
again on restart.

The bug allows volumes to be removed even though containers are
referencing them. 😱

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit 4c0e0979b4)
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2022-10-03 17:36:16 +00:00
Sebastiaan van Stijn
ed3c4e8d8e
Merge pull request #44232 from thaJeztah/22.06_backport_resolvconf_deadcode
[22.06 backport] libnetwork/resolvconf: removed unused GetIfChanged() and GetLastModified()
2022-10-03 10:59:39 +02:00
Sebastiaan van Stijn
3956644474
Merge pull request #44230 from thaJeztah/22.06_backport_migrate_filematcher
[22.06 backport] replace pkg/fileutils Matching funcs with github.com/moby/patternmatcher
2022-10-03 10:58:27 +02:00
Sebastiaan van Stijn
262ad3bb2f
libnetwork/resolvconf: removed unused GetIfChanged() and GetLastModified()
These functions were used in 63a7ccdd23, which was
part of Docker v1.5.0 and v1.6.0, but removed in Docker v1.7.0 when the network
stack was replaced with libnetwork in d18919e304.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 49de15cdcc)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-01 02:08:44 +02:00
Sebastiaan van Stijn
44d42c2b16
replace pkg/fileutils Matching funcs with github.com/moby/patternmatcher
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 3c69b9f2c5)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-01 01:09:36 +02:00
Sebastiaan van Stijn
14eb977c15
Merge pull request #44229 from thaJeztah/22.06_backport_more_linters_step1
[22.06 backport] fix (whitespace) formatting in preparation of enabling more linters
2022-10-01 01:09:05 +02:00
Sebastiaan van Stijn
f8e5145e96
runconfig, oci, image, layer, distribution: fix empty-lines (revive)
runconfig/config_test.go:23:46: empty-lines: extra empty line at the start of a block (revive)
    runconfig/config_test.go:75:55: empty-lines: extra empty line at the start of a block (revive)

    oci/devices_linux.go:57:34: empty-lines: extra empty line at the start of a block (revive)
    oci/devices_linux.go:60:69: empty-lines: extra empty line at the start of a block (revive)

    image/fs_test.go:53:38: empty-lines: extra empty line at the end of a block (revive)
    image/tarexport/save.go:88:29: empty-lines: extra empty line at the end of a block (revive)

    layer/layer_unix_test.go:21:34: empty-lines: extra empty line at the end of a block (revive)

    distribution/xfer/download.go:302:9: empty-lines: extra empty line at the end of a block (revive)
    distribution/manifest_test.go:154:99: empty-lines: extra empty line at the end of a block (revive)
    distribution/manifest_test.go:329:52: empty-lines: extra empty line at the end of a block (revive)
    distribution/manifest_test.go:354:59: empty-lines: extra empty line at the end of a block (revive)

    registry/config_test.go:323:42: empty-lines: extra empty line at the end of a block (revive)
    registry/config_test.go:350:33: empty-lines: extra empty line at the end of a block (revive)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 8a2e1245d4)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-01 00:01:14 +02:00
Sebastiaan van Stijn
24888a10f6
cmd/dockerd: fix empty-lines (revive)
cmd/dockerd/trap/trap_linux_test.go:29:29: empty-lines: extra empty line at the end of a block (revive)
    cmd/dockerd/daemon.go:327:35: empty-lines: extra empty line at the start of a block (revive)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit f63dea4337)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-01 00:01:14 +02:00
Sebastiaan van Stijn
3a1896db63
client: fix empty-lines (revive)
client/events.go:19:115: empty-lines: extra empty line at the start of a block (revive)
    client/events_test.go:60:31: empty-lines: extra empty line at the start of a block (revive)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit cd51c9fafb)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-01 00:01:13 +02:00
Sebastiaan van Stijn
47319e065d
api/server: fix empty-lines (revive)
api/server/router/build/build_routes.go:239:32: empty-lines: extra empty line at the start of a block (revive)
    api/server/middleware/version.go:45:241: empty-lines: extra empty line at the end of a block (revive)
    api/server/router/swarm/helpers_test.go:11:44: empty-lines: extra empty line at the end of a block (revive)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit f71fe8476a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-01 00:01:13 +02:00
Sebastiaan van Stijn
b9b6e68903
opts: fix empty-lines (revive)
opts/address_pools_test.go:7:39: empty-lines: extra empty line at the end of a block (revive)
    opts/opts_test.go:12:42: empty-lines: extra empty line at the end of a block (revive)
    opts/opts_test.go:60:49: empty-lines: extra empty line at the end of a block (revive)
    opts/opts_test.go:253:37: empty-lines: extra empty line at the end of a block (revive)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit b04f1416f6)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-01 00:01:13 +02:00
Sebastiaan van Stijn
c6c4d07830
daemon: fix empty-lines (revive)
daemon/network/filter_test.go:174:19: empty-lines: extra empty line at the end of a block (revive)
    daemon/restart.go:17:116: empty-lines: extra empty line at the end of a block (revive)
    daemon/daemon_linux_test.go:255:41: empty-lines: extra empty line at the end of a block (revive)
    daemon/reload_test.go:340:58: empty-lines: extra empty line at the end of a block (revive)
    daemon/oci_linux.go:495:101: empty-lines: extra empty line at the end of a block (revive)
    daemon/seccomp_linux_test.go:17:36: empty-lines: extra empty line at the start of a block (revive)
    daemon/container_operations.go:560:73: empty-lines: extra empty line at the end of a block (revive)
    daemon/daemon_unix.go:558:76: empty-lines: extra empty line at the end of a block (revive)
    daemon/daemon_unix.go:1092:64: empty-lines: extra empty line at the start of a block (revive)
    daemon/container_operations.go:587:24: empty-lines: extra empty line at the end of a block (revive)
    daemon/network.go:807:18: empty-lines: extra empty line at the end of a block (revive)
    daemon/network.go:813:42: empty-lines: extra empty line at the end of a block (revive)
    daemon/network.go:872:72: empty-lines: extra empty line at the end of a block (revive)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit ddb42f3ad2)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-01 00:01:13 +02:00
Sebastiaan van Stijn
9136c32327
daemon/images: fix empty-lines (revive)
daemon/images/image_squash.go:17:71: empty-lines: extra empty line at the start of a block (revive)
    daemon/images/store.go:128:27: empty-lines: extra empty line at the end of a block (revive)
    daemon/images/image_list.go:154:55: empty-lines: extra empty line at the start of a block (revive)
    daemon/images/image_delete.go:135:13: empty-lines: extra empty line at the end of a block (revive)
    daemon/images/image_search.go:25:64: empty-lines: extra empty line at the start of a block (revive)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 05042ce472)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-01 00:01:10 +02:00
Sebastiaan van Stijn
7cb488934b
daemon/logger: fix empty-lines (revive)
daemon/logger/loggertest/logreader.go:58:43: empty-lines: extra empty line at the end of a block (revive)
    daemon/logger/ring_test.go:119:34: empty-lines: extra empty line at the end of a block (revive)
    daemon/logger/adapter_test.go:37:12: empty-lines: extra empty line at the end of a block (revive)
    daemon/logger/adapter_test.go:41:44: empty-lines: extra empty line at the end of a block (revive)
    daemon/logger/adapter_test.go:170:9: empty-lines: extra empty line at the end of a block (revive)
    daemon/logger/loggerutils/sharedtemp_test.go:152:43: empty-lines: extra empty line at the end of a block (revive)
    daemon/logger/loggerutils/sharedtemp.go:124:117: empty-lines: extra empty line at the end of a block (revive)
    daemon/logger/syslog/syslog.go:249:87: empty-lines: extra empty line at the end of a block (revive)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 0695a910c6)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-30 23:59:31 +02:00
Sebastiaan van Stijn
aea1aa0daa
daemon/graphdriver: fix empty-lines (revive)
daemon/graphdriver/aufs/aufs.go:239:80: empty-lines: extra empty line at the start of a block (revive)
    daemon/graphdriver/graphtest/graphbench_unix.go:249:27: empty-lines: extra empty line at the start of a block (revive)
    daemon/graphdriver/graphtest/testutil.go:271:30: empty-lines: extra empty line at the end of a block (revive)
    daemon/graphdriver/graphtest/graphbench_unix.go:179:32: empty-block: this block is empty, you can remove it (revive)
    daemon/graphdriver/zfs/zfs.go:375:48: empty-lines: extra empty line at the end of a block (revive)
    daemon/graphdriver/overlay/overlay.go:248:89: empty-lines: extra empty line at the start of a block (revive)
    daemon/graphdriver/devmapper/deviceset.go:636:21: empty-lines: extra empty line at the end of a block (revive)
    daemon/graphdriver/devmapper/deviceset.go:1150:70: empty-lines: extra empty line at the start of a block (revive)
    daemon/graphdriver/devmapper/deviceset.go:1613:30: empty-lines: extra empty line at the end of a block (revive)
    daemon/graphdriver/devmapper/deviceset.go:1645:65: empty-lines: extra empty line at the start of a block (revive)
    daemon/graphdriver/btrfs/btrfs.go:53:101: empty-lines: extra empty line at the start of a block (revive)
    daemon/graphdriver/devmapper/deviceset.go:1944:89: empty-lines: extra empty line at the start of a block (revive)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 9d9cca49b4)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-30 23:59:31 +02:00
Sebastiaan van Stijn
79caa2f955
daemon/cluster: fix empty-lines (revive)
daemon/cluster/convert/service.go:96:34: empty-lines: extra empty line at the end of a block (revive)
    daemon/cluster/convert/service.go:169:44: empty-lines: extra empty line at the end of a block (revive)
    daemon/cluster/convert/service.go:470:30: empty-lines: extra empty line at the end of a block (revive)
    daemon/cluster/convert/container.go:224:23: empty-lines: extra empty line at the start of a block (revive)
    daemon/cluster/convert/network.go:109:14: empty-lines: extra empty line at the end of a block (revive)
    daemon/cluster/convert/service.go:537:27: empty-lines: extra empty line at the end of a block (revive)
    daemon/cluster/services.go:247:19: empty-lines: extra empty line at the end of a block (revive)
    daemon/cluster/services.go:252:41: empty-lines: extra empty line at the end of a block (revive)
    daemon/cluster/services.go:256:12: empty-lines: extra empty line at the end of a block (revive)
    daemon/cluster/services.go:289:80: empty-lines: extra empty line at the start of a block (revive)
    daemon/cluster/executor/container/health_test.go:18:37: empty-lines: extra empty line at the start of a block (revive)
    daemon/cluster/executor/container/adapter.go:437:68: empty-lines: extra empty line at the end of a block (revive)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 0c7b930952)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-30 23:59:30 +02:00
Sebastiaan van Stijn
87552f2e67
plugin: fix empty-lines (revive)
plugin/v2/settable_test.go:24:29: empty-lines: extra empty line at the end of a block (revive)
    plugin/manager_linux.go:96:6: empty-lines: extra empty line at the end of a block (revive)
    plugin/backend_linux.go:373:16: empty-lines: extra empty line at the start of a block (revive)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 4eb9b5f20e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-30 23:59:30 +02:00
Sebastiaan van Stijn
aad639c1fa
volume: fix empty-lines (revive)
volume/mounts/parser_test.go:42:39: empty-lines: extra empty line at the end of a block (revive)
    volume/mounts/windows_parser.go:129:24: empty-lines: extra empty line at the end of a block (revive)
    volume/local/local_test.go:16:35: empty-lines: extra empty line at the end of a block (revive)
    volume/local/local_unix.go:145:3: early-return: if c {...} else {... return } can be simplified to if !c { ... return } ... (revive)
    volume/service/service_test.go:18:38: empty-lines: extra empty line at the end of a block (revive)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 188724a597)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-30 23:59:30 +02:00
Sebastiaan van Stijn
bee5153c5b
testutil: fix empty-lines (revive)
testutil/fixtures/load/frozen.go:141:99: empty-lines: extra empty line at the end of a block (revive)
    testutil/daemon/plugin.go:56:129: empty-lines: extra empty line at the end of a block (revive)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit e9f1b83a4a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-30 23:59:30 +02:00
Sebastiaan van Stijn
31a938c73c
integration: fix empty-lines (revive)
integration/config/config_test.go:106:31: empty-lines: extra empty line at the end of a block (revive)
    integration/secret/secret_test.go:106:31: empty-lines: extra empty line at the end of a block (revive)
    integration/network/service_test.go:58:50: empty-lines: extra empty line at the end of a block (revive)
    integration/network/service_test.go:401:58: empty-lines: extra empty line at the end of a block (revive)
    integration/system/event_test.go:30:38: empty-lines: extra empty line at the end of a block (revive)
    integration/plugin/logging/read_test.go:19:41: empty-lines: extra empty line at the end of a block (revive)
    integration/service/list_test.go:30:48: empty-lines: extra empty line at the end of a block (revive)
    integration/service/create_test.go:400:46: empty-lines: extra empty line at the start of a block (revive)
    integration/container/logs_test.go:156:42: empty-lines: extra empty line at the end of a block (revive)
    integration/container/daemon_linux_test.go:135:44: empty-lines: extra empty line at the end of a block (revive)
    integration/container/restart_test.go:160:62: empty-lines: extra empty line at the end of a block (revive)
    integration/container/wait_test.go:181:47: empty-lines: extra empty line at the end of a block (revive)
    integration/container/restart_test.go:116:30: empty-lines: extra empty line at the end of a block (revive)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 786e6d80ba)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-30 23:59:30 +02:00
Sebastiaan van Stijn
9d44956d8c
builder: fix empty-lines (revive)
builder/remotecontext/detect_test.go:64:66: empty-lines: extra empty line at the end of a block (revive)
    builder/remotecontext/detect_test.go:78:46: empty-lines: extra empty line at the end of a block (revive)
    builder/remotecontext/detect_test.go:91:51: empty-lines: extra empty line at the end of a block (revive)
    builder/dockerfile/internals_test.go:95:38: empty-lines: extra empty line at the end of a block (revive)
    builder/dockerfile/copy.go:86:112: empty-lines: extra empty line at the end of a block (revive)
    builder/dockerfile/dispatchers_test.go:286:39: empty-lines: extra empty line at the start of a block (revive)
    builder/dockerfile/builder.go:280:38: empty-lines: extra empty line at the end of a block (revive)
    builder/dockerfile/dispatchers.go:66:85: empty-lines: extra empty line at the start of a block (revive)
    builder/dockerfile/dispatchers.go:559:85: empty-lines: extra empty line at the start of a block (revive)
    builder/builder-next/adapters/localinlinecache/inlinecache.go:26:183: empty-lines: extra empty line at the start of a block (revive)
    builder/builder-next/adapters/containerimage/pull.go:441:9: empty-lines: extra empty line at the start of a block (revive)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit ecb4ed172b)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-30 23:59:29 +02:00
Sebastiaan van Stijn
08d01be870
integration-cli: fix empty-lines (revive)
integration-cli/docker_cli_pull_test.go:55:69: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_exec_test.go:46:64: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_service_health_test.go:86:65: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_api_images_test.go:128:66: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_api_swarm_node_test.go:79:69: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_health_test.go:51:57: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_health_test.go:159:73: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_swarm_unix_test.go:60:67: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_api_inspect_test.go:30:33: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_api_build_test.go:429:71: empty-lines: extra empty line at the start of a block (revive)
    integration-cli/docker_cli_attach_unix_test.go:19:78: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_api_build_test.go:470:70: empty-lines: extra empty line at the start of a block (revive)
    integration-cli/docker_cli_history_test.go:29:64: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_links_test.go:93:86: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_create_test.go:33:61: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_links_test.go:145:78: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_create_test.go:114:70: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_api_attach_test.go:226:153: empty-lines: extra empty line at the start of a block (revive)
    integration-cli/docker_cli_by_digest_test.go:239:71: empty-lines: extra empty line at the start of a block (revive)
    integration-cli/docker_cli_create_test.go:135:49: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_create_test.go:143:75: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_create_test.go:181:71: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_inspect_test.go:72:65: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_api_swarm_service_test.go:98:77: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_api_swarm_service_test.go:144:69: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_rmi_test.go:63:2: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_api_swarm_service_test.go:199:79: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_rmi_test.go:69:2: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_api_swarm_service_test.go:300:75: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_prune_unix_test.go:35:25: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_events_unix_test.go:393:60: empty-lines: extra empty line at the start of a block (revive)
    integration-cli/docker_cli_events_unix_test.go:441:71: empty-lines: extra empty line at the start of a block (revive)
    integration-cli/docker_cli_ps_test.go:33:67: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_ps_test.go:559:67: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_events_test.go:117:75: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_api_containers_test.go:547:74: empty-lines: extra empty line at the start of a block (revive)
    integration-cli/docker_api_containers_test.go:1054:84: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_api_containers_test.go:1076:87: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_api_containers_test.go:1232:72: empty-lines: extra empty line at the start of a block (revive)
    integration-cli/docker_api_containers_test.go:1801:21: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_network_unix_test.go:58:95: empty-lines: extra empty line at the start of a block (revive)
    integration-cli/docker_cli_network_unix_test.go:750:75: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_network_unix_test.go:765:76: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_swarm_test.go:617:100: empty-lines: extra empty line at the start of a block (revive)
    integration-cli/docker_cli_swarm_test.go:892:72: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_daemon_test.go:119:74: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_daemon_test.go:981:68: empty-lines: extra empty line at the start of a block (revive)
    integration-cli/docker_cli_daemon_test.go:1951:87: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_run_test.go:83:66: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_run_test.go:357:72: empty-lines: extra empty line at the start of a block (revive)
    integration-cli/docker_cli_build_test.go:89:83: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:114:83: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:183:80: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:290:71: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:314:65: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:331:67: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:366:76: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:403:67: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:648:67: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:708:72: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:938:66: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:1018:72: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:1097:2: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:1182:62: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:1244:66: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:1524:69: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:1546:80: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:1716:70: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:1730:65: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:2162:74: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:2270:71: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:2288:70: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:3206:65: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:3392:66: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:3433:72: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:3678:76: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:3732:67: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:3759:69: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:3802:61: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:3898:66: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:4107:9: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:4791:74: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:4821:73: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:4854:70: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:5341:74: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:5593:81: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_api_containers_test.go:2145:11: empty-lines: extra empty line at the start of a block (revive)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit dc0c2340b8)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-30 23:59:29 +02:00
Sebastiaan van Stijn
3660ee30e3
api/types: fix empty-lines (revive)
Also renamed variables that collided with import

     api/types/strslice/strslice_test.go:36:41: empty-lines: extra empty line at the end of a block (revive)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 31441778fa)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-30 23:59:29 +02:00
Sebastiaan van Stijn
3424a7c2e3
pkg/*: fix "empty-lines" (revive)
pkg/directory/directory.go:9:49: empty-lines: extra empty line at the start of a block (revive)
    pkg/pubsub/publisher.go:8:48: empty-lines: extra empty line at the start of a block (revive)
    pkg/loopback/attach_loopback.go:96:69: empty-lines: extra empty line at the start of a block (revive)
    pkg/devicemapper/devmapper_wrapper.go:136:48: empty-lines: extra empty line at the start of a block (revive)
    pkg/devicemapper/devmapper.go:391:35: empty-lines: extra empty line at the end of a block (revive)
    pkg/devicemapper/devmapper.go:676:35: empty-lines: extra empty line at the end of a block (revive)
    pkg/archive/changes_posix_test.go:15:38: empty-lines: extra empty line at the end of a block (revive)
    pkg/devicemapper/devmapper.go:241:51: empty-lines: extra empty line at the start of a block (revive)
    pkg/fileutils/fileutils_test.go:17:47: empty-lines: extra empty line at the end of a block (revive)
    pkg/fileutils/fileutils_test.go:34:48: empty-lines: extra empty line at the end of a block (revive)
    pkg/fileutils/fileutils_test.go:318:32: empty-lines: extra empty line at the end of a block (revive)
    pkg/tailfile/tailfile.go:171:6: empty-lines: extra empty line at the end of a block (revive)
    pkg/tarsum/fileinfosums_test.go:16:41: empty-lines: extra empty line at the end of a block (revive)
    pkg/tarsum/tarsum_test.go:198:42: empty-lines: extra empty line at the start of a block (revive)
    pkg/tarsum/tarsum_test.go:294:25: empty-lines: extra empty line at the start of a block (revive)
    pkg/tarsum/tarsum_test.go:407:34: empty-lines: extra empty line at the end of a block (revive)
    pkg/ioutils/fswriters_test.go:52:45: empty-lines: extra empty line at the end of a block (revive)
    pkg/ioutils/writers_test.go:24:39: empty-lines: extra empty line at the end of a block (revive)
    pkg/ioutils/bytespipe_test.go:78:26: empty-lines: extra empty line at the end of a block (revive)
    pkg/sysinfo/sysinfo_linux_test.go:13:37: empty-lines: extra empty line at the end of a block (revive)
    pkg/archive/archive_linux_test.go:57:64: empty-lines: extra empty line at the end of a block (revive)
    pkg/archive/changes.go:248:72: empty-lines: extra empty line at the start of a block (revive)
    pkg/archive/changes_posix_test.go:15:38: empty-lines: extra empty line at the end of a block (revive)
    pkg/archive/copy.go:248:124: empty-lines: extra empty line at the end of a block (revive)
    pkg/archive/diff_test.go:198:44: empty-lines: extra empty line at the end of a block (revive)
    pkg/archive/archive.go:304:12: empty-lines: extra empty line at the end of a block (revive)
    pkg/archive/archive.go:749:37: empty-lines: extra empty line at the end of a block (revive)
    pkg/archive/archive.go:812:81: empty-lines: extra empty line at the start of a block (revive)
    pkg/archive/copy_unix_test.go:347:34: empty-lines: extra empty line at the end of a block (revive)
    pkg/system/path.go:11:39: empty-lines: extra empty line at the end of a block (revive)
    pkg/system/meminfo_linux.go:29:21: empty-lines: extra empty line at the end of a block (revive)
    pkg/plugins/plugins.go:135:32: empty-lines: extra empty line at the end of a block (revive)
    pkg/authorization/response.go:71:48: empty-lines: extra empty line at the start of a block (revive)
    pkg/authorization/api_test.go:18:51: empty-lines: extra empty line at the end of a block (revive)
    pkg/authorization/middleware_test.go:23:44: empty-lines: extra empty line at the end of a block (revive)
    pkg/authorization/middleware_unix_test.go:17:46: empty-lines: extra empty line at the end of a block (revive)
    pkg/authorization/api_test.go:57:45: empty-lines: extra empty line at the end of a block (revive)
    pkg/authorization/response.go:83:50: empty-lines: extra empty line at the start of a block (revive)
    pkg/authorization/api_test.go:66:47: empty-lines: extra empty line at the end of a block (revive)
    pkg/authorization/middleware_unix_test.go:45:48: empty-lines: extra empty line at the end of a block (revive)
    pkg/authorization/response.go:145:75: empty-lines: extra empty line at the start of a block (revive)
    pkg/authorization/middleware_unix_test.go:56:51: empty-lines: extra empty line at the end of a block (revive)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 412c650e05)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-30 23:59:25 +02:00
Sebastiaan van Stijn
36fda30565
Merge pull request #44228 from thaJeztah/22.06_backport_migrate_pubsub
[22.06 backport] migrate pkg/pubsub to github.com/moby/pubsub
2022-09-30 23:52:43 +02:00
Sebastiaan van Stijn
541fda8e90
migrate pkg/pubsub to github.com/moby/pubsub
This package was moved to a separate repository, using the steps below:

    # install filter-repo (https://github.com/newren/git-filter-repo/blob/main/INSTALL.md)
    brew install git-filter-repo

    cd ~/projects

    # create a temporary clone of docker
    git clone https://github.com/docker/docker.git moby_pubsub_temp
    cd moby_pubsub_temp

    # for reference
    git rev-parse HEAD
    # --> 572ca799db

    # remove all code, except for pkg/pubsub, license, and notice, and rename pkg/pubsub to /
    git filter-repo --path pkg/pubsub/ --path LICENSE --path NOTICE --path-rename pkg/pubsub/:

    # remove canonical imports
    git revert -s -S 585ff0ebbe6bc25b801a0e0087dd5353099cb72e

    # initialize module
    go mod init github.com/moby/pubsub
    go mod tidy

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 0249afc523)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-30 22:32:43 +02:00
Sebastiaan van Stijn
a8b1fec072
pkg/fileutils: remove gotest.tools as dependency, use t.TempDir()
In preparation of moving this package separate.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 0440ca07ba)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-30 22:32:28 +02:00
Sebastiaan van Stijn
70c3d7783f
Merge pull request #44217 from thaJeztah/22.06_backport_more_robust_rootless
[22.06 backport] contrib: make dockerd-rootless-setuptool.sh more robust
2022-09-29 16:31:36 +02:00
Sebastiaan van Stijn
fd0904805c
contrib: make dockerd-rootless-setuptool.sh more robust
The `docker` CLI currently doesn't handle situations where the current context
(as defined in `~/.docker/config.json`) is invalid or doesn't exist. As loading
(and checking) the context happens during initialization of the CLI, this
prevents `docker context` commands from being used, which makes it complicated
to fix the situation. For example, running `docker context use <correct context>`
would fail, which makes it not possible to update the `~/.docker/config.json`,
unless doing so manually.

For example, given the following `~/.docker/config.json`:

```json
{
        "currentContext": "nosuchcontext"
}
```

All of the commands below fail:

```bash
docker context inspect rootless
Current context "nosuchcontext" is not found on the file system, please check your config file at /Users/thajeztah/.docker/config.json

docker context rm --force rootless
Current context "nosuchcontext" is not found on the file system, please check your config file at /Users/thajeztah/.docker/config.json

docker context use default
Current context "nosuchcontext" is not found on the file system, please check your config file at /Users/thajeztah/.docker/config.json
```

While these things should be fixed, this patch updates the script to switch
the context using the `--context` flag; this flag is taken into account when
initializing the CLI, so that having an invalid context configured won't
block `docker context` commands from being executed. Given that all `context`
commands are local operations, "any" context can be used (it doesn't need to
make a connection with the daemon).

With this patch, those commands can now be run (and won't fail for the wrong
reason);

```bash
 docker --context=default context inspect -f "{{.Name}}" rootless
rootless

docker --context=default context inspect -f "{{.Name}}" rootless-doesnt-exist
context "rootless-doesnt-exist" does not exist
```

One other issue may also cause things to fail during uninstall; trying to remove
a context that doesn't exist will fail (even with the `-f` / `--force` option
set);

```bash
docker --context=default context rm blablabla
Error: context "blablabla": not found
```

While this is "ok" in most circumstances, it also means that (potentially) the
current context is not reset to "default", so this patch adds an explicit
`docker context use`, as well as unsetting the `DOCKER_HOST` and `DOCKER_CONTEXT`
environment variables.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit e2114731e7)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-29 10:03:30 +02:00
Sebastiaan van Stijn
3977a3c6e8
Merge pull request #44206 from thaJeztah/22.06_backport_idtools_fix_infinite_loop
[22.06 backport] pkg/idtools: mkdirAs(): fix infinite loops and repeated "chown"
2022-09-27 23:44:44 +02:00
Sebastiaan van Stijn
43cfc50bbb
Merge pull request #44207 from neersighted/createImpliedDirectories_22.06
[22.06 backport] refactor(pkg/archive): factor out createImpliedDirectories helper
2022-09-27 23:34:22 +02:00
Bjorn Neergaard
f6ebfaea19 test(pkg/archive): add TestImpliedDirectoryPermissions
Co-authored-by: Cory Snider <csnider@mirantis.com>
Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
(cherry picked from commit 5dff494b87)
2022-09-27 14:01:30 -06:00
Bjorn Neergaard
daa8708601 refactor(pkg/archive): factor out createImpliedDirectories helper
This code was duplicated in two places -- factor it out, add
documentation, and move magic numbers into a constant.

Additionally, use the same permissions (0755) in both code paths, and
ensure that the ID map is used in both code paths.

Co-authored-by: Vasiliy Ulyanov <vulyanov@suse.de>
Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
Signed-off-by: Vasiliy Ulyanov <vulyanov@suse.de>
(cherry picked from commit 4831ff9f27)
2022-09-27 14:01:30 -06:00
Sebastiaan van Stijn
7114360901
pkg/idtools: mkdirAs(): fix infinite loops and repeated "chown"
This fixes an inifinite loop in mkdirAs(), used by `MkdirAllAndChown`,
`MkdirAndChown`, and `MkdirAllAndChownNew`, as well as directories being
chown'd multiple times when relative paths are used.

The for loop in this function was incorrectly assuming that;

1. `filepath.Dir()` would always return the parent directory of any given path
2. traversing any given path to ultimately result in "/"

While this is correct for absolute and "cleaned" paths, both assumptions are
incorrect in some variations of "path";

1. for paths with a trailing path-separator ("some/path/"), or dot ("."),
   `filepath.Dir()` considers the (implicit) "." to be a location _within_ the
   directory, and returns "some/path" as ("parent") directory. This resulted
   in the path itself to be included _twice_ in the list of paths to chown.
2. for relative paths ("./some-path", "../some-path"), "traversing" the path
   would never end in "/", causing the for loop to run indefinitely:

    ```go
    // walk back to "/" looking for directories which do not exist
    // and add them to the paths array for chown after creation
    dirPath := path
    for {
        dirPath = filepath.Dir(dirPath)
        if dirPath == "/" {
            break
        }
        if _, err := os.Stat(dirPath); err != nil && os.IsNotExist(err) {
            paths = append(paths, dirPath)
        }
    }
    ```

A _partial_ mitigation for this would be to use `filepath.Clean()` before using
the path (while `filepath.Dir()` _does_ call `filepath.Clean()`, it only does so
_after_ some processing, so only cleans the result). Doing so would prevent the
double chown from happening, but would not prevent the "final" path to be "."
or ".." (in the relative path case), still causing an infinite loop, or
additional checks for "." / ".." to be needed.

| path           | filepath.Dir(path) | filepath.Dir(filepath.Clean(path)) |
|----------------|--------------------|------------------------------------|
| some-path      | .                  | .                                  |
| ./some-path    | .                  | .                                  |
| ../some-path   | ..                 | ..                                 |
| some/path/     | some/path          | some                               |
| ./some/path/   | some/path          | some                               |
| ../some/path/  | ../some/path       | ../some                            |
| some/path/.    | some/path          | some                               |
| ./some/path/.  | some/path          | some                               |
| ../some/path/. | ../some/path       | ../some                            |
| /some/path/    | /some/path         | /some                              |
| /some/path/.   | /some/path         | /some                              |

Instead, this patch adds a `filepath.Abs()` to the function, so make sure that
paths are both cleaned, and not resulting in an infinite loop.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 1e13247d6d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-27 21:59:47 +02:00
Sebastiaan van Stijn
fc6192786a
Merge pull request #44195 from thaJeztah/22.06_backport_update_golangci_lint
[22.06 backport] golangci-lint: update to v1.49.0
2022-09-27 18:24:50 +02:00
Sebastiaan van Stijn
3d6a13f072
Merge pull request #44198 from thaJeztah/22.06_backport_jenkinsfile_ubuntu_2004
[22.06 backport] Jenkinsfile: use Ubuntu 20.04 for DCO stage
2022-09-27 17:54:50 +02:00
Sebastiaan van Stijn
5ebe35cc09
Merge pull request #44201 from crazy-max/22.06_backport_api-fix-logo
[22.06 backport] swagger: update links to logo
2022-09-27 17:54:33 +02:00
CrazyMax
5dfec22079
swagger: update links to logo
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 7f3602f1c9)
2022-09-27 11:56:36 +02:00
Sebastiaan van Stijn
cee2490d84
Jenkinsfile: use Ubuntu 20.04 for DCO stage
Also switching to use arm64, as all amd64 stages have moved to GitHub actions,
so using arm64 allows the same machine to be used for tests after the DCO check
completed.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 419c47a80a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-26 19:56:46 +02:00
Sebastiaan van Stijn
3ce520ec80
golangci-lint: update to v1.49.0
Remove the "deadcode", "structcheck", and "varcheck" linters, as they are
deprecated:

    WARN [runner] The linter 'deadcode' is deprecated (since v1.49.0) due to: The owner seems to have abandoned the linter.  Replaced by unused.
    WARN [runner] The linter 'structcheck' is deprecated (since v1.49.0) due to: The owner seems to have abandoned the linter.  Replaced by unused.
    WARN [runner] The linter 'varcheck' is deprecated (since v1.49.0) due to: The owner seems to have abandoned the linter.  Replaced by unused.
    WARN [linters context] structcheck is disabled because of generics. You can track the evolution of the generics support by following the https://github.com/golangci/golangci-lint/issues/2649.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 2f1c382a6d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-26 11:58:07 +02:00
Sebastiaan van Stijn
7772535e79
Merge pull request #44175 from thaJeztah/22.06_backport_fix_g112_slowlorus
[22.06 backport] set ReadHeaderTimeout to address G112: Potential Slowloris Attack (gosec)
2022-09-26 11:54:09 +02:00
Samuel Karp
bebad9e22e
Merge pull request #44190 from thaJeztah/22.06_backport_sequential_release 2022-09-25 00:06:26 -07:00
Samuel Karp
b31d51cac6
Merge pull request #44188 from thaJeztah/22.06_backport_bump_go_systemd 2022-09-25 00:02:55 -07:00
Sebastiaan van Stijn
1d7fb64a6e
vendor: github.com/moby/sys/sequential v0.5.0
no changes, just updated to use the tagged version;

full diff: https://github.com/moby/sys/compare/b22ba8a69b30...sequential/v0.5.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 489e7b61bf)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-23 18:42:11 +02:00
Sebastiaan van Stijn
ae65811be2
vendor: github.com/github.com/coreos/go-systemd v22.4.0
- dbus: add Connected methods to check connections status
- dbus: add support for querying unit by PID
- dbus: implement support for cgroup freezer APIs
- journal: remove implicit initialization
- login1: add methods to get session/user properties
- login1: add context-aware ListSessions and ListUsers methods

full diff: https://github.com/github.com/coreos/go-systemd/compare/v22.3.2...v22.4.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 323ab8ef97)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-23 17:27:02 +02:00
Sebastiaan van Stijn
0e873d5cd8
Merge pull request #44183 from thaJeztah/22.06_backport_remove_os_check
[22.06 backport] Remove the OS check when creating a container
2022-09-22 21:37:39 +02:00
Djordje Lukic
2bc36de638
Remove the OS check when creating a container
Now that we can pass any custom containerd shim to dockerd there is need
for this check. Without this it becomes possible to use wasm shims for
example with images that have "wasi" as the OS.

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
(cherry picked from commit 1a3d8019d1)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-22 19:52:22 +02:00
Sebastiaan van Stijn
aca9143c13
Merge pull request #44126 from thaJeztah/22.06_backport_image_spec_no_literal
[22.06 backport] Update uses of Image platform fields in OCI image-spec
2022-09-22 19:01:11 +02:00
Sebastiaan van Stijn
e143eed8bc
Merge pull request #44168 from thaJeztah/22.06_backport_test_summary
[22.06 backport] ci(test): report summary output
2022-09-22 15:18:05 +02:00
Sebastiaan van Stijn
7d621608dd
Merge pull request #44173 from crazy-max/22.06_backport_ci-cache
[22.06 backport] ci: reduce cache throttling limits
2022-09-22 13:18:02 +02:00
Sebastiaan van Stijn
997ec12ec8
set ReadHeaderTimeout to address G112: Potential Slowloris Attack (gosec)
After discussing in the maintainers meeting, we concluded that Slowloris attacks
are not a real risk other than potentially having some additional goroutines
lingering around, so setting a long timeout to satisfy the linter, and to at
least have "some" timeout.

    libnetwork/diagnostic/server.go:96:10: G112: Potential Slowloris Attack because ReadHeaderTimeout is not configured in the http.Server (gosec)
        srv := &http.Server{
            Addr:    net.JoinHostPort(ip, strconv.Itoa(port)),
            Handler: s,
        }
    api/server/server.go:60:10: G112: Potential Slowloris Attack because ReadHeaderTimeout is not configured in the http.Server (gosec)
                srv: &http.Server{
                    Addr: addr,
                },
    daemon/metrics_unix.go:34:13: G114: Use of net/http serve function that has no support for setting timeouts (gosec)
            if err := http.Serve(l, mux); err != nil && !strings.Contains(err.Error(), "use of closed network connection") {
                      ^
    cmd/dockerd/metrics.go:27:13: G114: Use of net/http serve function that has no support for setting timeouts (gosec)
            if err := http.Serve(l, mux); err != nil && !strings.Contains(err.Error(), "use of closed network connection") {
                      ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 55fd77f724)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-22 12:27:32 +02:00
CrazyMax
4a8f744255
ci: reduce cache throttling limits
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 6d59619d6e)
2022-09-22 11:29:44 +02:00
Sebastiaan van Stijn
49a2f5c55c
Merge pull request #44165 from thaJeztah/22.06_backport_sysvinit_debian_restart
[22.06 backport] fix debian sysvinit script fails to restart docker daemon when stopped
2022-09-21 23:32:33 +02:00
CrazyMax
07efcaf3b2
ci(test): report summary output
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 7b9877bd8a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-21 18:08:57 +02:00
Sebastiaan van Stijn
6b04087d5f
Merge pull request #44163 from thaJeztah/22.06_backport_memberlist_transient_bumps
[22.06 backport] chore: bump transient dependencies of memberlist
2022-09-21 18:04:37 +02:00
Sebastiaan van Stijn
d752acd960
Merge pull request #44164 from thaJeztah/22.06_backport_client_kernel_memory
[22.06 backport] client: ignore kernel-memory on API >= 1.42
2022-09-21 18:03:48 +02:00
Yann Autissier
7f94f2b393
fix debian sysvinit script fails to restart docker daemon when stopped
Fixes: #44130
Signed-off-by: Yann Autissier <yann.autissier@gmail.com>
(cherry picked from commit 8ad8c6d887)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-21 16:32:57 +02:00
Sebastiaan van Stijn
970c938b56
client: ignore kernel-memory on API >= 1.42
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 2597a71623)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-21 16:27:20 +02:00
Bjorn Neergaard
d41ebd79f7
vendor: github.com/armon/go-metrics v0.4.1
Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
(cherry picked from commit c0fa14e8af)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-21 16:24:32 +02:00
Bjorn Neergaard
d0fadc859d
vendor: github.com/google/btree v1.1.2
Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
(cherry picked from commit 1d7ceb2fee)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-21 16:24:29 +02:00
Sebastiaan van Stijn
40b28dc7e1
Merge pull request #44159 from crazy-max/22.06_backport_ci-fix-filter-pattern
[22.06 backport] ci: fix branch filter pattern
2022-09-21 10:58:02 +02:00
CrazyMax
44c5f7721a
ci: fix branch filter pattern
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 5e50d002f1)
2022-09-20 21:27:42 +02:00
Sebastiaan van Stijn
a13cd44a13
Merge pull request #44148 from crazy-max/22.06_backport_cleanup-test-suite
[22.06 backport] integration-cli: remove TestDockerSuite func
2022-09-20 11:50:30 +02:00
Sebastiaan van Stijn
2e89072681
Merge pull request #44154 from crazy-max/22.06_backport_jenkins-rm-validate
[22.06 backport] Jenkinsfile: remove leftover steps and stages
2022-09-20 11:16:38 +02:00
Sebastiaan van Stijn
7b5de59256
Merge pull request #44152 from crazy-max/22.06_backport_gha-validate
[22.06 backport] ci(test): validate job matrix
2022-09-19 17:56:49 +02:00
CrazyMax
00b1722fb4
Jenkinsfile: remove unit-validate stage
Left cross step is already in GHA so we can remove
unit-validate stage.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 8596486743)
2022-09-19 16:34:07 +02:00
CrazyMax
8fdaad4018
Jenkinsfile: remove report bundles creation in unit-validate
This was missing to be removed from Jenkinsfile when we moved
to GHA for unit and integration tests.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit cd54f31984)
2022-09-19 16:34:06 +02:00
CrazyMax
fefe6290e5
Jenkinsfile: remove validation steps moved to GHA
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit a354970eaa)
2022-09-19 16:34:06 +02:00
CrazyMax
f925f295f4
ci: reusable dco workflow
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 7602edfd06)
2022-09-16 19:53:19 +02:00
CrazyMax
cc770330f8
ci(test): dynamic validate matrix
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 5a98363a92)
2022-09-16 19:53:19 +02:00
CrazyMax
e42f7db450
ci(test): validate job matrix
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit a73d78f8d5)
2022-09-16 19:53:19 +02:00
CrazyMax
9a57be4ac4
integration-cli: remove TestDockerSuite func
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 9b428a3d33)
2022-09-15 19:10:04 +02:00
Sebastiaan van Stijn
95831246a2
Update uses of Image platform fields in OCI image-spec
The OCI image spec is considering to change the Image struct and embedding the
Platform type (see opencontainers/image-spec#959) in the go implementation.
Moby currently uses some struct-literals to propagate the platform fields,
which will break once those changes in the OCI spec are merged.

Ideally (once that change arrives) we would update the code to set the Platform
information as a whole, instead of assigning related fields individually, but
in some cases in the code, image platform information is only partially set
(for example, OSVersion and OSFeatures are not preserved in all cases). This
may be on purpose, so needs to be reviewed.

This patch keeps the current behavior (assigning only specific fields), but
removes the use of struct-literals to make the code compatible with the
upcoming changes in the image-spec module.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 3cb933db9d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-10 12:19:53 +02:00
Akihiro Suda
8af2e62556
Merge pull request #44117 from thaJeztah/22.06_backport_bump_klauspost_compress
[22.06 backport] vendor: github.com/klauspost/compress v1.15.9
2022-09-09 20:37:31 +09:00
Sebastiaan van Stijn
6236ebaed5
vendor: github.com/klauspost/compress v1.15.9
various fixes in zstd compression

- https://github.com/klauspost/compress/releases/tag/v1.15.9
- https://github.com/klauspost/compress/releases/tag/v1.15.8
- https://github.com/klauspost/compress/releases/tag/v1.15.7
- https://github.com/klauspost/compress/releases/tag/v1.15.6
- https://github.com/klauspost/compress/releases/tag/v1.15.5
- https://github.com/klauspost/compress/releases/tag/v1.15.4
- https://github.com/klauspost/compress/releases/tag/v1.15.3
- https://github.com/klauspost/compress/releases/tag/v1.15.2

full diff: https://github.com/klauspost/compress/compare/v1.15.1...v1.15.9

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit f7277806c8)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-09 03:40:36 +02:00
Sebastiaan van Stijn
50d3438b26
Merge pull request #44121 from thaJeztah/22.06_backport_GHSA_rc4r_wh2q_q6c4
[22.06 backport] Updates for supplementary group permissions
2022-09-09 02:11:57 +02:00
Sebastiaan van Stijn
366d551cd2
Update some tests for supplementary group permissions
Update tests checking for groups to adjust for new policy updated in
de7af816e7, which caused those tests
to fail:

    === FAIL: amd64.integration-cli TestDockerSwarmSuite/TestSwarmServiceWithGroup (1.94s)
    docker_cli_swarm_test.go:311: assertion failed: uid=0(root) gid=0(root) groups=0(root),10(wheel),29(audio),50(staff),777 (string) != uid=0(root) gid=0(root) groups=10(wheel),29(audio),50(staff),777
(string)
    --- FAIL: TestDockerSwarmSuite/TestSwarmServiceWithGroup (1.94s)

    === FAIL: amd64.integration-cli TestDockerCLIRunSuite/TestRunGroupAdd (0.41s)
    docker_cli_run_test.go:1091: expected output uid=0(root) gid=0(root) groups=10(wheel),29(audio),50(staff),777 received uid=0(root) gid=0(root) groups=0(root),10(wheel),29(audio),50(staff),777
    --- FAIL: TestDockerCLIRunSuite/TestRunGroupAdd (0.41s)

    === FAIL: amd64.integration-cli TestDockerCLIRunSuite/TestRunUserByIDZero (0.41s)
    docker_cli_run_test.go:790: expected daemon user got uid=0(root) gid=0(root) groups=0(root),10(wheel)
    --- FAIL: TestDockerCLIRunSuite/TestRunUserByIDZero (0.41s)

    === FAIL: amd64.integration-cli TestDockerCLIRunSuite (195.70s)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit c7e77dba7f)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-09 00:07:25 +02:00
Nicolas De Loof
393027d1b1
AdditionalGids must include effective group ID
otherwise this one won't be considered for permission checks

Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
(cherry picked from commit 25345f2c04)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-09 00:07:21 +02:00
Sebastiaan van Stijn
21d818be87
Merge pull request #44115 from thaJeztah/22.06_backport_cleanup_jenkins
[22.06 backport] Jenkinsfile: remove steps moved to GitHub Actions
2022-09-08 21:17:48 +02:00
Brian Goff
6d65028804
Merge pull request #44096 from thaJeztah/22.06_backport_bump_units
[22.06 backport] vendor: github.com/docker/go-units v0.5.0
2022-09-08 12:09:35 -07:00
Brian Goff
c0e1c67c78
Merge pull request #44112 from crazy-max/22.06_backport_gha-test
[22.06 backport] ci: gha test workflow for integration and unit test
2022-09-08 11:20:42 -07:00
CrazyMax
b9b8ddc160
Jenkinsfile: remove steps moved to GitHub Actions
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 9f8bd80487)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-08 16:48:15 +02:00
CrazyMax
d96d56ff09
ci: fix .windows workflow name
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 36688496ca)
2022-09-08 13:50:19 +02:00
CrazyMax
cc7b8cc980
ci(test): send coverage to codecov
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 807c849431)
2022-09-08 13:50:19 +02:00
CrazyMax
8ca74127d9
ci(test): upload reports
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 4c8af0e2f9)
2022-09-08 13:50:19 +02:00
CrazyMax
fc2942d4e0
integration-cli: TestPluginInstallImage broken on GitHub Runner
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 60864229b8)
2022-09-08 13:50:19 +02:00
CrazyMax
874954d8bd
integration-cli: TestEventsOOM* broken on GitHub Runner
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit b8bf60c590)
2022-09-08 13:50:19 +02:00
CrazyMax
0bfb1bded3
integration: TestNetworkLoopbackNat is broken on GitHub Runner
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit df731c745a)
2022-09-08 13:50:18 +02:00
CrazyMax
4765040aa3
ci: gha test workflow for integration and unit test
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 2e04be3fb9)
2022-09-08 13:50:18 +02:00
Sebastiaan van Stijn
de0300b1c6
Merge pull request #44095 from thaJeztah/22.06_fix_linting_issues
[22.06 backport] fix various linting issues in preparation of golangci-lint update
2022-09-08 00:00:08 +02:00
Samuel Karp
4807ef2af0
Merge pull request #44085 from thaJeztah/22.06_backport_bump_go_1.19 2022-09-07 14:25:02 -07:00
Sebastiaan van Stijn
c853881610
Update to go 1.19.1 to address CVE-2022-27664, CVE-2022-32190
From the mailing list:

We have just released Go versions 1.19.1 and 1.18.6, minor point releases.
These minor releases include 2 security fixes following the security policy:

- net/http: handle server errors after sending GOAWAY
  A closing HTTP/2 server connection could hang forever waiting for a clean
  shutdown that was preempted by a subsequent fatal error. This failure mode
  could be exploited to cause a denial of service.

  Thanks to Bahruz Jabiyev, Tommaso Innocenti, Anthony Gavazzi, Steven Sprecher,
  and Kaan Onarlioglu for reporting this.

  This is CVE-2022-27664 and Go issue https://go.dev/issue/54658.

- net/url: JoinPath does not strip relative path components in all circumstances
  JoinPath and URL.JoinPath would not remove `../` path components appended to a
  relative path. For example, `JoinPath("https://go.dev", "../go")` returned the
  URL `https://go.dev/../go`, despite the JoinPath documentation stating that
  `../` path elements are cleaned from the result.

  Thanks to q0jt for reporting this issue.

  This is CVE-2022-32190 and Go issue https://go.dev/issue/54385.

Release notes:

go1.19.1 (released 2022-09-06) includes security fixes to the net/http and
net/url packages, as well as bug fixes to the compiler, the go command, the pprof
command, the linker, the runtime, and the crypto/tls and crypto/x509 packages.
See the Go 1.19.1 milestone on the issue tracker for details.

https://github.com/golang/go/issues?q=milestone%3AGo1.19.1+label%3ACherryPickApproved

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 1eadbdd9fa)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-07 22:11:46 +02:00
Sebastiaan van Stijn
2450c5a46b
update to golang 1.19
also ran gofmt with go1.19

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 58413c15cb)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-07 22:11:46 +02:00
Sebastiaan van Stijn
a490e68553
Merge pull request #44108 from rumpl/22.06-fix-local-context
[22.06 backport] fix local context
2022-09-07 22:07:15 +02:00
Djordje Lukic
5d2b3687b0 Wrap local calls to the content and lease service
The wrapper sets the default namespace in the context if none is
provided, this is needed because we are calling these services directly
and not trough GRPC that has an interceptor to set the default namespace
to all calls.

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
(cherry picked from commit 878906630b)
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2022-09-07 15:07:16 +02:00
Samuel Karp
d7e36c99fb
Merge pull request #44100 from thaJeztah/22.06_backport_bump_golang_1.18.6 2022-09-06 21:43:55 -07:00
Samuel Karp
1249d36bdd
Merge pull request #44103 from thaJeztah/22.06_backport_bump_x_net 2022-09-06 20:50:29 -07:00
Sebastiaan van Stijn
287d1656de
vendor: golang.org/x/net v0.0.0-20220906165146-f3363e06e74c
Update to the latest version that contains a fix for CVE-2022-27664;
f3363e06e7

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 518179f63e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-06 23:19:13 +02:00
Sebastiaan van Stijn
39976cd2bf
Update to go 1.18.6 to address CVE-2022-27664, CVE-2022-32190
From the mailing list:

We have just released Go versions 1.19.1 and 1.18.6, minor point releases.
These minor releases include 2 security fixes following the security policy:

- net/http: handle server errors after sending GOAWAY
  A closing HTTP/2 server connection could hang forever waiting for a clean
  shutdown that was preempted by a subsequent fatal error. This failure mode
  could be exploited to cause a denial of service.

  Thanks to Bahruz Jabiyev, Tommaso Innocenti, Anthony Gavazzi, Steven Sprecher,
  and Kaan Onarlioglu for reporting this.

  This is CVE-2022-27664 and Go issue https://go.dev/issue/54658.

- net/url: JoinPath does not strip relative path components in all circumstances
  JoinPath and URL.JoinPath would not remove `../` path components appended to a
  relative path. For example, `JoinPath("https://go.dev", "../go")` returned the
  URL `https://go.dev/../go`, despite the JoinPath documentation stating that
  `../` path elements are cleaned from the result.

  Thanks to q0jt for reporting this issue.

  This is CVE-2022-32190 and Go issue https://go.dev/issue/54385.

Release notes:

go1.18.6 (released 2022-09-06) includes security fixes to the net/http package,
as well as bug fixes to the compiler, the go command, the pprof command, the
runtime, and the crypto/tls, encoding/xml, and net packages. See the Go 1.18.6
milestone on the issue tracker for details;

https://github.com/golang/go/issues?q=milestone%3AGo1.18.6+label%3ACherryPickApproved

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit cba36a064d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-06 22:26:19 +02:00
Sebastiaan van Stijn
85f1b6ff8f
Merge pull request #44094 from thaJeztah/22.06_backport_remove_tereshkova
[22.06] cleanup namesgenerator
2022-09-06 21:58:09 +02:00
Sebastiaan van Stijn
1650fa8889
vendor: github.com/docker/go-units v0.5.0
full diff: https://github.com/docker/go-units/compare/v0.4.0...v0.5.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 13f99eb65f)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-06 16:04:56 +02:00
Sebastiaan van Stijn
e9e7491f2b
linting: host:port in url should be constructed with net.JoinHostPort
integration-cli/docker_cli_daemon_test.go:545:54: host:port in url should be constructed with net.JoinHostPort and not directly with fmt.Sprintf (nosprintfhostport)
            cmdArgs = append(cmdArgs, "--tls=false", "--host", fmt.Sprintf("tcp://%s:%s", l.daemon, l.port))
                                                               ^
    opts/hosts_test.go:35:31: host:port in url should be constructed with net.JoinHostPort and not directly with fmt.Sprintf (nosprintfhostport)
            "tcp://:5555":              fmt.Sprintf("tcp://%s:5555", DefaultHTTPHost),
                                        ^
    opts/hosts_test.go:91:30: host:port in url should be constructed with net.JoinHostPort and not directly with fmt.Sprintf (nosprintfhostport)
            ":5555":                   fmt.Sprintf("tcp://%s:5555", DefaultHTTPHost),
                                       ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 306b8c89e8)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-06 15:11:42 +02:00
Sebastiaan van Stijn
2609d4e252
linting: gosec: fix or suppress G112, G114 in test code
Updating test-code only; set ReadHeaderTimeout for some, or suppress the linter
error for others.

     contrib/httpserver/server.go:11:12: G114: Use of net/http serve function that has no support for setting timeouts (gosec)
        log.Panic(http.ListenAndServe(":80", nil))
                  ^
     integration/plugin/logging/cmd/close_on_start/main.go:42:12: G112: Potential Slowloris Attack because ReadHeaderTimeout is not configured in the http.Server (gosec)
        server := http.Server{
            Addr:    l.Addr().String(),
            Handler: mux,
        }
     integration/plugin/logging/cmd/discard/main.go:17:12: G112: Potential Slowloris Attack because ReadHeaderTimeout is not configured in the http.Server (gosec)
        server := http.Server{
            Addr:    l.Addr().String(),
            Handler: mux,
        }
     integration/plugin/logging/cmd/dummy/main.go:14:12: G112: Potential Slowloris Attack because ReadHeaderTimeout is not configured in the http.Server (gosec)
        server := http.Server{
            Addr:    l.Addr().String(),
            Handler: http.NewServeMux(),
        }
     integration/plugin/volumes/cmd/dummy/main.go:14:12: G112: Potential Slowloris Attack because ReadHeaderTimeout is not configured in the http.Server (gosec)
        server := http.Server{
            Addr:    l.Addr().String(),
            Handler: http.NewServeMux(),
        }
     testutil/fixtures/plugin/basic/basic.go:25:12: G112: Potential Slowloris Attack because ReadHeaderTimeout is not configured in the http.Server (gosec)
        server := http.Server{
            Addr:    l.Addr().String(),
            Handler: http.NewServeMux(),
        }
     volume/testutils/testutils.go:170:5: G114: Use of net/http serve function that has no support for setting timeouts (gosec)
        go http.Serve(l, mux)
           ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 31fb92c609)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-06 15:11:42 +02:00
Sebastiaan van Stijn
188c5d4a7c
linting: suppress false positive for G404 (gosec)
The linter falsely detects this as using "math/rand":

    libnetwork/networkdb/cluster.go:721:14: G404: Use of weak random number generator (math/rand instead of crypto/rand) (gosec)
       val, err := rand.Int(rand.Reader, big.NewInt(int64(n)))
                   ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 561a010161)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-06 15:11:42 +02:00
Sebastiaan van Stijn
ff4ec67b90
libnetwork/diagnostic: EnableDiagnostic(): use net.JoinHostPort
Use net.JoinHostPort to account for IPv6 addresses.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit a33d1f9a7c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-06 15:11:39 +02:00
Sebastiaan van Stijn
fee68df273
namesgenerator: remove Valentina Tereshkova
While the name generator has been frozen for new additions in 624b3cfbe8,
this person has become controversial. Our intent is for this list to be inclusive
and non-controversial.

This patch removes the name from the list.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 0f052eb4f5)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-06 13:58:28 +02:00
Sebastiaan van Stijn
b5a0d7a188
Merge pull request #44065 from thaJeztah/22.06_backport_test_updates
[22.06 backport] assorted CI changes
2022-08-31 15:15:23 +02:00
CrazyMax
f7cf9fbe48
ci: move buildkit tests to a dedicated workflow
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit a4d081cc17)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-31 11:57:25 +02:00
CrazyMax
ee87eaf9ad
ci(windows): move windows-2019 to another workflow
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 65fdd10d4e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-31 11:57:23 +02:00
Ben Langfeld
09a0b0a84a
Upgrades buildx to 0.9.1
v0.9.0 included regressions. Release notes: https://github.com/docker/buildx/releases/tag/v0.9.1

Signed-off-by: Ben Langfeld <blangfeld@powerhrg.com>
(cherry picked from commit 5dcaad0dd3)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-31 11:57:21 +02:00
Sebastiaan van Stijn
8e6ed32610
Makefile: update buildx to v0.9.0
release notes: https://github.com/docker/buildx/releases/tag/v0.9.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit f27b74f0f7)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-31 11:57:18 +02:00
Sebastiaan van Stijn
dfd2f917dc
Merge pull request #44055 from thaJeztah/22.06_backport_migrate_sequential
[22.06 backport] replace pkg/system Sequential funcs with moby/sys/sequential
2022-08-30 23:17:23 +02:00
Sebastiaan van Stijn
4f1dd92056
Merge pull request #44049 from thaJeztah/22.06_backport_validate_yaml
[22.06 backport] validate: add additional validation on YAML files
2022-08-30 19:35:22 +02:00
Sebastiaan van Stijn
f10c50958c
replace pkg/system Sequential funcs with moby/sys/sequential
Migrating these functions to allow them being shared between moby, docker/cli,
and containerd, and to allow using them without importing all of sys / system,
which (in containerd) also depends on hcsshim and more.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 509f19f611)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-30 09:34:33 +02:00
Sebastiaan van Stijn
40515da6d6
pkg/system: make IsAbs() platform-agnostic
filepath.IsAbs() will short-circuit on Linux/Unix, so having a single
implementation should not affect those platforms.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 2640aec0d7)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-29 23:19:53 +02:00
Sebastiaan van Stijn
3b9370fcf8
validate: address SC2155 (shellcheck)
see https://github.com/koalaman/shellcheck/wiki/SC2155

Looking at how these were used, I don't think we even need to
export them, so removing that.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 5cfc9c374c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-29 19:54:57 +02:00
Sebastiaan van Stijn
51bf7da729
validate: format vendor script with shfmt
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit b9fd2cf605)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-29 19:54:56 +02:00
Sebastiaan van Stijn
f1bd611d41
validate: add yamllint validation
validate other YAML files, such as the ones used in the documentation,
and GitHub actions workflows, to prevent issues such as;

- 30295c1750
- 8e8d9a3650

With this patch:

    hack/validate/yamllint
    Congratulations! yamllint config file formatted correctly
    Congratulations! YAML files are formatted correctly

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 6cef06b940)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-29 19:54:54 +02:00
Sebastiaan van Stijn
e9f7c05ae1
validate: yamllint: ignore "truthy value should be one of" warnings
Suppresses warnings like:

    LANG=C.UTF-8 yamllint -c hack/validate/yamllint.yaml -f parsable .github/workflows/*.yml
    .github/workflows/ci.yml:7:1: [warning] truthy value should be one of [false, true] (truthy)
    .github/workflows/windows.yml:7:1: [warning] truthy value should be one of [false, true] (truthy)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 91bb776bb8)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-29 19:54:52 +02:00
Sebastiaan van Stijn
72156dd7a4
validate: yamllint: set locale in config file
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit cc2134ea83)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-29 19:54:50 +02:00
Sebastiaan van Stijn
554a933944
validate: yamllint: use "parsable" output
Before:

    10030:81  error    line too long (89 > 80 characters)  (line-length)

After:

    api/swagger.yaml:10030:81: [error] line too long (89 > 80 characters) (line-length)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit f679d8c821)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-29 19:54:48 +02:00
Sebastiaan van Stijn
8d43d7fa6b
validate: yamllint rename config-file
Don't make the file hidden, and add .yaml extension, so that editors
pick up the right formatting :)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 5f114b65b4)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-29 19:54:46 +02:00
Sebastiaan van Stijn
a61b411ceb
Dockerfile: update yamllint to v1.27.1
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 1d7cd76ee9)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-29 19:54:43 +02:00
Sebastiaan van Stijn
d2590dc3cd
Merge pull request #44039 from thaJeztah/22.06_backport_update_runc_1.1.4
[22.06 backport] update runc to v1.1.4
2022-08-26 13:29:28 +02:00
Sebastiaan van Stijn
274316f89e
Merge pull request #44042 from thaJeztah/22.06_backport_containerd_binary_1.6.8
[22.06 backport] update containerd binary to v1.6.8
2022-08-26 13:07:31 +02:00
Sebastiaan van Stijn
e3e3a31989
Merge pull request #44046 from thaJeztah/22.06_backport_fix_ci_workflow
[22.06 backport] ci: fix broken workflow
2022-08-26 10:39:37 +02:00
CrazyMax
704e7a2d71
ci: fix broken workflow
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 8e8d9a3650)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-26 08:27:00 +02:00
Sebastiaan van Stijn
87b7e40a34
update containerd binary to v1.6.8
release notes: https://github.com/containerd/containerd/releases/tag/v1.6.8

full diff: https://github.com/containerd/containerd/compare/v1.6.7...v1.6.8

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit d52ffce38f)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-26 00:34:11 +02:00
Akihiro Suda
901fb577cb
update runc to v1.1.4
release notes: https://github.com/opencontainers/runc/releases/tag/v1.1.4

full diff: https://github.com/opencontainers/runc/compare/v1.1.3...v1.1.4

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit bd98bf38e9)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-26 00:26:42 +02:00
Sebastiaan van Stijn
fc8b388eac
Merge pull request #44029 from thaJeztah/22.06_backport_bump_buildkit
[22.06 backport] vendor: github.com/moby/buildkit v0.10.4
2022-08-26 00:13:00 +02:00
Sebastiaan van Stijn
9aeda305fd
vendor: github.com/moby/buildkit v0.10.4
release notes: https://github.com/moby/buildkit/releases/tag/v0.10.4

full diff: https://github.com/moby/buildkit/compare/8e2d9b9006ca...v0.10.4

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit c500d8824d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-25 21:33:53 +02:00
Sebastiaan van Stijn
48e314fbe2
gha: temporarily pin BuildKit integration test version
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 6217f8001e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-25 21:33:51 +02:00
Samuel Karp
29c636bf80
Merge pull request #44022 from thaJeztah/22.06_backport_client_remove_withdialer 2022-08-25 10:35:34 -07:00
Sebastiaan van Stijn
64b0b54fc8
Merge pull request #44027 from thaJeztah/22.06_backport_libnetwork_ipvlan_fix
[22.06 backport] libnetwork/drivers/ipvlan: fix missing IpvlanFlag field in config JSON
2022-08-24 16:56:36 +02:00
Sebastiaan van Stijn
e8d00f02aa
Merge pull request #44024 from thaJeztah/22.06_backport_vendor_containerd_1.6.8
[22.06 backport] vendor: github.com/containerd/containerd v1.6.8
2022-08-24 13:50:39 +02:00
Sebastiaan van Stijn
7b086898ee
Merge pull request #44025 from thaJeztah/22.06_backport_testfix_TestNetworkDBNodeJoinLeaveIteration
[22.06 backprot] Test: wait for network changes in TestNetworkDBNodeJoinLeaveIteration
2022-08-24 13:49:35 +02:00
Youfu Zhang
292d352ee4
libnetwork/drivers/ipvlan: fix missing IpvlanFlag field in config JSON
Fixes #42542

Signed-off-by: Youfu Zhang <zhangyoufu@gmail.com>
(cherry picked from commit 549d24b437)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-24 13:46:31 +02:00
David Wang
2293a20972
Test: wait for network changes in TestNetworkDBNodeJoinLeaveIteration
In network node change test, the expected behavior is focused on how many nodes
left in networkDB, besides timing issues, things would also go tricky for a
leave-then-join sequence, if the check (counting the nodes) happened before the
first "leave" event, then the testcase actually miss its target and report PASS
without verifying its final result; if the check happened after the 'leave' event,
but before the 'join' event, the test would report FAIL unnecessary;

This code change would check both the db changes and the node count, it would
report PASS only when networkdb has indeed changed and the node count is expected.

Signed-off-by: David Wang <00107082@163.com>
(cherry picked from commit f499c6b9ec)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-24 01:45:06 +02:00
Sebastiaan van Stijn
76fa56b62d
vendor: github.com/containerd/containerd v1.6.8
no code changes, other than a version bump

full diff: https://github.com/containerd/containerd/compare/v1.6.7...v1.6.8

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit ca52e0a244)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-24 01:42:55 +02:00
Sebastiaan van Stijn
e5958a8f08
client: remove deprecated WithDialer() option
It was deprecated in edac92409a, which
was part of 18.09 and up, so should be safe by now to remove this.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit e14924570c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-24 01:36:57 +02:00
Sebastiaan van Stijn
2dc3e510d4
Merge pull request #43994 from corhere/backport-22.06/healthcheck_timeout
[v22.06 backport] don't use canceled context to send KILL signal to healthcheck process
2022-08-24 01:35:09 +02:00
Cory Snider
e7f4963e73
daemon: kill exec process on ctx cancel
Terminating the exec process when the context is canceled has been
broken since Docker v17.11 so nobody has been able to depend upon that
behaviour in five years of releases. We are thus free from backwards-
compatibility constraints.

Co-authored-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Co-authored-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Signed-off-by: Cory Snider <csnider@mirantis.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 4b84a33217)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-23 22:09:40 +02:00
Sebastiaan van Stijn
629397f70e
Merge pull request #43992 from neersighted/22.06_builder_version
[22.06 backport] api: set default "Builder-Version" to "2" (BuildKit) on Linux
2022-08-18 23:01:12 +02:00
Sebastiaan van Stijn
1e6029e81e
Merge pull request #43993 from cpuguy83/22.06_backport_43978_default_runtime
[22.06] Allow containerd shim refs in default-runtime
2022-08-18 23:00:34 +02:00
Sebastiaan van Stijn
2a33c73574
Merge pull request #43988 from thaJeztah/22.06_backport_seccomp_bpfcap
[22.06 backport] seccomp: allow "bpf", "perf_event_open", gated by CAP_BPF, CAP_PERFMON
2022-08-18 22:57:41 +02:00
Sebastiaan van Stijn
4bf8eec265
Merge pull request #43985 from neersighted/22.06_bump_memberlist
[22.06] vendor: bump memberlist
2022-08-18 20:51:48 +02:00
Brian Goff
dfcb3e17ae Allow containerd shim refs in default-runtime
Since runtimes can now just be containerd shims, we need to check if the
reference is possibly a containerd shim.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit e6ee27a541)
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2022-08-18 18:51:23 +00:00
Sebastiaan van Stijn
8e9684c029
Merge pull request #43990 from thaJeztah/22.06_backport_deprecate_graph
[22.06 backport] daemon: complete the "--graph" / "-g" deprecation
2022-08-18 20:30:24 +02:00
Sebastiaan van Stijn
2c17e9a333 api: set default "Builder-Version" to "2" (BuildKit) on Linux
Starting with the 22.06 release, buildx is the default client for
docker build, which uses BuildKit as builder.

This patch changes the default builder version as advertised by
the daemon to "2" (BuildKit), so that pre-22.06 CLIs with BuildKit
support (but no buildx installed) also default to using BuildKit
when interacting with a 22.06 (or up) daemon.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-18 12:18:32 -06:00
Sebastiaan van Stijn
d1d9fd50c2
daemon: complete the "--graph" / "-g" deprecation
The `-g` / `--graph` options were soft deprecated in favor of `--data-root` in
261ef1fa27 (v17.05.0) and at the time considered
to not be removed. However, with the move towards containerd snapshotters, having
these options around adds additional complexity to handle fallbacks for deprecated
(and hidden) flags, so completing the deprecation.

With this patch:

    dockerd --graph=/var/lib/docker --validate
    Flag --graph has been deprecated, Use --data-root instead
    unable to configure the Docker daemon with file /etc/docker/daemon.json: merged configuration validation from file and command line flags failed: the "graph" config file option is deprecated; use "data-root" instead

    mkdir -p /etc/docker
    echo '{"graph":"/var/lib/docker"}' > /etc/docker/daemon.json

    dockerd --validate
    unable to configure the Docker daemon with file /etc/docker/daemon.json: merged configuration validation from file and command line flags failed: the "graph" config file option is deprecated; use "data-root" instead

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit b58de39ca7)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-18 18:44:17 +02:00
Sebastiaan van Stijn
8912c1fade
seccomp: allow "bpf", "perf_event_open", gated by CAP_BPF, CAP_PERFMON
Update the profile to make use of CAP_BPF and CAP_PERFMON capabilities. Prior to
kernel 5.8, bpf and perf_event_open required CAP_SYS_ADMIN. This change enables
finer control of the privilege setting, thus allowing us to run certain system
tracing tools with minimal privileges.

Based on the original patch from Henry Wang in the containerd repository.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 7b7d1132e8)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-18 18:36:49 +02:00
Bjorn Neergaard
332de3f1e3 vendor: github.com/hasicorp/memberlist v0.4.0
Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
2022-08-18 09:50:59 -06:00
Akihiro Suda
2160f0041d
Merge pull request #43923 from crazy-max/22.06_vendor-buildkit
[22.06 backport] vendor buildkit 8e2d9b9 (v0.10 branch)
2022-08-15 02:32:59 +09:00
Sebastiaan van Stijn
3254fa3b50
Merge pull request #43938 from thaJeztah/22.06_backport_bump_swarmkit3
[22.06 backport] vendor: github.com/moby/swarmkit/v2 v2.0.0-20220721174824-48dd89375d0a, change "csi" to "cluster"
2022-08-11 21:30:49 +02:00
Sebastiaan van Stijn
b73c27ef6b
vendor: github.com/moby/swarmkit/v2 v2.0.0-20220721174824-48dd89375d0a
full diff: 6068d1894d...48dd89375d

Finishes off the work to change references to cluster volumes in the API
from using "csi" as the magic word to "cluster". This reflects that the
volumes are "cluster volumes", not "csi volumes".

Notably, there is no change to the plugin definitions being "csinode"
and "csicontroller". This terminology is appropriate with regards to
plugins because it accurates reflects what the plugin is.

Signed-off-by: Drew Erny <derny@mirantis.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 9861dd069b)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-09 14:05:30 +02:00
Sebastiaan van Stijn
ec89e7cde1
Merge pull request #43932 from thaJeztah/22.06_backport_bump_containerd_1.6.7_vendor
[22.06 backport] vendor: github.com/containerd/containerd v1.6.7
2022-08-08 18:51:46 +02:00
Sebastiaan van Stijn
15f9cb5c4d
vendor: github.com/containerd/containerd v1.6.7
full diff: https://github.com/containerd/containerd/v1.6.6...v1.6.7

Welcome to the v1.6.7 release of containerd!

The seventh patch release for containerd 1.6 contains various fixes,
includes a new version of runc and adds support for ppc64le and riscv64
(requires unreleased runc 1.2) builds.

Notable Updates

- Update runc to v1.1.3
- Seccomp: Allow clock_settime64 with CAP_SYS_TIME
- Fix WWW-Authenticate parsing
- Support RISC-V 64 and ppc64le builds
- Windows: Update hcsshim to v0.9.4 to fix regression with HostProcess stats
- Windows: Fix shim logs going to panic.log file
- Allow ptrace(2) by default for kernels >= 4.8

See the changelog for complete list of changes

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 7376bf948b)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-08 11:32:26 +02:00
Tianon Gravi
ebfc35f887
Merge pull request #43915 from thaJeztah/22.06_backport_vendor_hcsshim
[22.06 backport] vendor: github.com/Microsoft/hcsshim v0.9.4
2022-08-05 15:09:34 -07:00
Tianon Gravi
f47d5ced16
Merge pull request #43919 from thaJeztah/22.06_backport_bump_containerd_1.6.7_binary
[22.06 backport] update containerd binary too v1.6.7, runc to v1.3.1
2022-08-05 15:09:16 -07:00
CrazyMax
6c78a1166e
vendor buildkit 8e2d9b9 (v0.10 branch)
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit e05f614267)
2022-08-05 14:57:32 +02:00
Sebastiaan van Stijn
8ae63006f1
update containerd binary to v1.6.7
full diff: https://github.com/containerd/containerd/v1.6.6...v1.6.7

Welcome to the v1.6.7 release of containerd!

The seventh patch release for containerd 1.6 contains various fixes,
includes a new version of runc and adds support for ppc64le and riscv64
(requires unreleased runc 1.2) builds.

Notable Updates

- Update runc to v1.1.3
- Seccomp: Allow clock_settime64 with CAP_SYS_TIME
- Fix WWW-Authenticate parsing
- Support RISC-V 64 and ppc64le builds
- Windows: Update hcsshim to v0.9.4 to fix regression with HostProcess stats
- Windows: Fix shim logs going to panic.log file
- Allow ptrace(2) by default for kernels >= 4.8

See the changelog for complete list of changes

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 4e46d9f963)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-05 00:26:02 +02:00
Sebastiaan van Stijn
aeb600bc4a
update runc binary to v1.1.3
full diff: https://github.com/opencontainers/runc/compare/v1.1.2...v1.1.3

This is the third release of the 1.1.z series of runc, and contains
various minor improvements and bugfixes.

- Our seccomp `-ENOSYS` stub now correctly handles multiplexed syscalls on
  s390 and s390x. This solves the issue where syscalls the host kernel did not
  support would return `-EPERM` despite the existence of the `-ENOSYS` stub
  code (this was due to how s390x does syscall multiplexing).
- Retry on dbus disconnect logic in libcontainer/cgroups/systemd now works as
  intended; this fix does not affect runc binary itself but is important for
  libcontainer users such as Kubernetes.
- Inability to compile with recent clang due to an issue with duplicate
  constants in libseccomp-golang.
- When using systemd cgroup driver, skip adding device paths that don't exist,
  to stop systemd from emitting warnings about those paths.
- Socket activation was failing when more than 3 sockets were used.
- Various CI fixes.
- Allow to bind mount `/proc/sys/kernel/ns_last_pid` to inside container.
- runc static binaries are now linked against libseccomp v2.5.4.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 2293de1c82)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-05 00:26:00 +02:00
Paweł Gronowski
e0d8418ddc
vendor: github.com/Microsoft/hcsshim v0.9.4
full diff: https://github.com/microsoft/hcsshim/compare/v0.9.3...v0.9.4

Changes are mostly fixes of unsafe usage of `unsafe.Pointer`

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 69f077f1aa)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-04 22:43:56 +02:00
Sebastiaan van Stijn
e6a5f44e61
Merge pull request #43914 from thaJeztah/22.06_backport_fix_api_swagger
[22.06 backport] api: swagger: fix invalid example value (API v1.39-v1.41)
2022-08-04 21:02:55 +02:00
Sebastiaan van Stijn
7130076488
Merge pull request #43890 from thaJeztah/22.06_backport_api_fix_missing_platform
[22.06 backport] api: swagger: add missing "platform" query-arg on create
2022-08-04 21:00:55 +02:00
Sebastiaan van Stijn
0133759476
api: swagger: fix invalid example value (API v1.39-v1.41)
This was introduced in 43956c1bfc

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 30295c1750)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-04 17:55:21 +02:00
Sebastiaan van Stijn
86839c826f
Merge pull request #43908 from thaJeztah/22.06_backport_deprecate_buildcache_parent
[22.06 backport] api: deprecate BuildCache.Parent, add BuildCache.Parents in API >= v1.42
2022-08-04 13:19:37 +02:00
Sebastiaan van Stijn
f93e0ef4d6
Merge pull request #43904 from thaJeztah/22.06_backport_bump_go_1.18.5
[22.06 backport] Update golang to 1.18.5
2022-08-04 11:27:00 +02:00
Sebastiaan van Stijn
572457e265
api: swagger: document BuildCache fields (API v1.39-v1.41)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 43956c1bfc)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-04 09:33:39 +02:00
Sebastiaan van Stijn
49377cdd63
api: swagger: document BuildCache fields (API v1.42)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 5371c889a8)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-04 09:33:37 +02:00
Sebastiaan van Stijn
910d5c44fc
api: add BuildCache.Parents for API >= v1.42
This field was added to replace the deprecated "Parent" field.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit e0db8207f3)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-04 09:33:35 +02:00
Sebastiaan van Stijn
0e3d20cb20
api: deprecate BuildCache.Parent in API >= v1.42
This field has been deprecated in BuildKit, so this follows the deprecation
in the Engine API.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit ebf339628a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-04 09:33:33 +02:00
Sebastiaan van Stijn
a96b75191e
api/types: add missing GoDoc for BuildCache fields.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit e0286d7f4e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-04 09:33:31 +02:00
Sebastiaan van Stijn
a285cd4d88
api: swagger: document BuildCache fields.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit dc2b34af6a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-04 09:33:28 +02:00
Sebastiaan van Stijn
4f057d8bb6
Merge pull request #43887 from thaJeztah/22.06_backport_implicit_runtime_config
[22.06 backport] daemon: support other containerd runtimes (MVP)
2022-08-03 23:55:51 +02:00
Sebastiaan van Stijn
1240460547
Update golang to 1.18.5
Update Go runtime to 1.18.5 to address CVE-2022-32189.

Full diff: https://github.com/golang/go/compare/go1.18.4...go1.18.5

--------------------------------------------------------

From the security announcement:
https://groups.google.com/g/golang-announce/c/YqYYG87xB10

We have just released Go versions 1.18.5 and 1.17.13, minor point
releases.

These minor releases include 1 security fixes following the security
policy:

encoding/gob & math/big: decoding big.Float and big.Rat can panic

Decoding big.Float and big.Rat types can panic if the encoded message is
too short.

This is CVE-2022-32189 and Go issue https://go.dev/issue/53871.

View the release notes for more information:
https://go.dev/doc/devel/release#go1.18.5

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit f1d71f7cc3)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-03 20:09:14 +02:00
Akihiro Suda
d9a6b805b3
Merge pull request #43884 from vvoland/fix-exitcode-wait-22.06
[22.06 backport] state/Wait: Fix race when reading exit status
2022-07-30 15:51:39 +09:00
Sebastiaan van Stijn
e88c28941f
docs: api: add missing "platform" query-arg on create (v1.42)
Commit 7a9cb29fb9 added a new "platform" query-
parameter to the `POST /containers/create` endpoint, but did not update the
swagger file and documentation.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 982f09f837)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-29 23:23:02 +02:00
Sebastiaan van Stijn
9c4984db6b
docs: api: add missing "platform" query-arg on create (v1.41)
Commit 7a9cb29fb9 added a new "platform" query-
parameter to the `POST /containers/create` endpoint, but did not update the
swagger file and documentation.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 1000e4ee7d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-29 23:23:00 +02:00
Sebastiaan van Stijn
af7c8ff045
api: swagger: add missing "platform" query-arg on create
Commit 7a9cb29fb9 added a new "platform" query-
parameter to the `POST /containers/create` endpoint, but did not update the
swagger file and documentation.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 3dae8e9fc2)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-29 23:22:58 +02:00
Cory Snider
6de52a29a8
daemon: support other containerd runtimes (MVP)
Contrary to popular belief, the OCI Runtime specification does not
specify the command-line API for runtimes. Looking at containerd's
architecture from the lens of the OCI Runtime spec, the _shim_ is the
OCI Runtime and runC is "just" an implementation detail of the
io.containerd.runc.v2 runtime. When one configures a non-default runtime
in Docker, what they're really doing is instructing Docker to create
containers using the io.containerd.runc.v2 runtime with a configuration
option telling the runtime that the runC binary is at some non-default
path. Consequently, only OCI runtimes which are compatible with the
io.containerd.runc.v2 shim, such as crun, can be used in this manner.
Other OCI runtimes, including kata-containers v2, come with their own
containerd shim and are not compatible with io.containerd.runc.v2.
As Docker has not historically provided a way to select a non-default
runtime which requires its own shim, runtimes such as kata-containers v2
could not be used with Docker.

Allow other containerd shims to be used with Docker; no daemon
configuration required. If the daemon is instructed to create a
container with a runtime name which does not match any of the configured
or stock runtimes, it passes the name along to containerd verbatim. A
user can start a container with the kata-containers runtime, for
example, simply by calling

    docker run --runtime io.containerd.kata.v2

Runtime names which containerd would interpret as a path to an arbitrary
binary are disallowed. While handy for development and testing it is not
strictly necessary and would allow anyone with Engine API access to
trivially execute any binary on the host as root, so we have decided it
would be safest for our users if it was not allowed.

It is not yet possible to set an alternative containerd shim as the
default runtime; it can only be configured per-container.

Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit 547da0d575)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-29 20:36:50 +02:00
Sebastiaan van Stijn
ad0ee82f0d
Merge pull request #43875 from thaJeztah/22.06_backport_42655_vfs_storage_driver
[22.06 backport] Fix file capabilities dropping in Dockerfile
2022-07-29 18:23:41 +02:00
Paweł Gronowski
85b9568d0e state/Wait: Fix race when reading exit status
Before this change there was a race condition between State.Wait reading
the exit code from State and the State being changed instantly after the
change which ended the State.Wait.

Now, each State.Wait has its own channel which is used to transmit the
desired StateStatus at the time the state transitions to the awaited
one. Wait no longer reads the status by itself so there is no race.

The issue caused the `docker run --restart=always ...' to sometimes exit
with 0 exit code, because the process was already restarted by the time
State.Wait got the chance to read the exit code.

Test run
--------
Before:
```
$ go test -count 1 -run TestCorrectStateWaitResultAfterRestart .
--- FAIL: TestCorrectStateWaitResultAfterRestart (0.00s)
    state_test.go:198: expected exit code 10, got 0
FAIL
FAIL    github.com/docker/docker/container      0.011s
FAIL

```

After:
```
$ go test -count 1 -run TestCorrectStateWaitResultAfterRestart .
ok      github.com/docker/docker/container      0.011s
```

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2022-07-29 16:49:56 +02:00
Paweł Gronowski
826003ecae integration: TestWaitRestartedContainer
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2022-07-29 16:49:56 +02:00
Paweł Gronowski
e2bd8edb0d daemon/restart: Don't mutate AutoRemove when restarting
This caused a race condition where AutoRemove could be restored before
container was considered for restart and made autoremove containers
impossible to restart.

```
$ make DOCKER_GRAPHDRIVER=vfs BIND_DIR=. TEST_FILTER='TestContainerWithAutoRemoveCanBeRestarted' TESTFLAGS='-test.count 1' test-integration
...
=== RUN   TestContainerWithAutoRemoveCanBeRestarted
=== RUN   TestContainerWithAutoRemoveCanBeRestarted/kill
=== RUN   TestContainerWithAutoRemoveCanBeRestarted/stop
--- PASS: TestContainerWithAutoRemoveCanBeRestarted (1.61s)
    --- PASS: TestContainerWithAutoRemoveCanBeRestarted/kill (0.70s)
    --- PASS: TestContainerWithAutoRemoveCanBeRestarted/stop (0.86s)
PASS

DONE 3 tests in 3.062s
```

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2022-07-29 16:49:56 +02:00
Paweł Gronowski
44fde1bdb7 integration: Add TestContainerWithAutoRemoveCanBeRestarted
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2022-07-29 16:49:56 +02:00
Sebastiaan van Stijn
d8f20bfdc1
Merge pull request #43878 from thaJeztah/22.06_backport_containerd_config_v2
[22.06 backport] libcontainerd: switch generated containerd.toml to v2 (v1 is deprecated)
2022-07-28 21:19:38 +02:00
Sebastiaan van Stijn
6ab3b50a3f
libcontainerd: switch generated containerd.toml to v2 (v1 is deprecated)
Before this patch:

    INFO[2022-07-27T14:30:06.188762628Z] Starting up
    INFO[2022-07-27T14:30:06.190750725Z] libcontainerd: started new containerd process  pid=2028
    ...
    WARN[0000] containerd config version `1` has been deprecated and will be removed in containerd v2.0, please switch to version `2`, see https://github.com/containerd/containerd/blob/main/docs/PLUGINS.md#version-header
    INFO[2022-07-27T14:30:06.220024286Z] starting containerd                           revision=10c12954828e7c7c9b6e0ea9b0c02b01407d3ae1 version=v1.6.6

With this patch:

    INFO[2022-07-27T14:28:04.025543517Z] Starting up
    INFO[2022-07-27T14:28:04.027447105Z] libcontainerd: started new containerd process  pid=1377
    ...
    INFO[2022-07-27T14:28:04.054483270Z] starting containerd                           revision=10c12954828e7c7c9b6e0ea9b0c02b01407d3ae1 version=v1.6.6

And the generated /var/run/docker/containerd/containerd.toml:

```toml
disabled_plugins = ["io.containerd.grpc.v1.cri"]
imports = []
oom_score = 0
plugin_dir = ""
required_plugins = []
root = "/var/lib/docker/containerd/daemon"
state = "/var/run/docker/containerd/daemon"
temp = ""
version = 2

[cgroup]
  path = ""

[debug]
  address = "/var/run/docker/containerd/containerd-debug.sock"
  format = ""
  gid = 0
  level = "debug"
  uid = 0

[grpc]
  address = "/var/run/docker/containerd/containerd.sock"
  gid = 0
  max_recv_message_size = 16777216
  max_send_message_size = 16777216
  tcp_address = ""
  tcp_tls_ca = ""
  tcp_tls_cert = ""
  tcp_tls_key = ""
  uid = 0

[metrics]
  address = ""
  grpc_histogram = false

[plugins]

[proxy_plugins]

[stream_processors]

[timeouts]

[ttrpc]
  address = ""
  gid = 0
  uid = 0
```

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit ba2ff69894)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-28 16:45:26 +02:00
Illo Abdulrahim
6d41219bae
Fix file capabilities droping in Dockerfile
doCopyXattrs() never reached due to copyXattrs boolean being false, as
a result file capabilities not being copied.

moved copyXattr() out of doCopyXattrs()

Signed-off-by: Illo Abdulrahim <abdulrahim.illo@nokia.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 31f654a704)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-28 09:39:21 +02:00
Sebastiaan van Stijn
dcbd68a1d4
Merge pull request #43858 from olljanat/22.06_backport_restore-custom-nat-networks
[22.06 backport] Windows: Re-create custom NAT networks after restart if missing from HNS
2022-07-25 12:23:14 +02:00
Olli Janatuinen
112fb22152 Windows: Re-create custom NAT networks after restart if missing from HNS
Signed-off-by: Olli Janatuinen <olli.janatuinen@gmail.com>
(cherry picked from commit 67c36d5)
Signed-off-by: Olli Janatuinen <olli.janatuinen@gmail.com>
2022-07-23 23:16:23 -07:00
Tianon Gravi
a60b458179
Merge pull request #43841 from thaJeztah/22.06_backport_journald_fix_break
[22.06 backport] logger/journald: fix SA4011: ineffective break statement
2022-07-20 10:13:42 -07:00
Sebastiaan van Stijn
a9081299dd
logger/journald: fix SA4011: ineffective break statement
This was introduced in 906b979b88, which changed
a `goto` to a `break`, but afaics, the intent was still to break out of the loop.
(linter didn't catch this before because it didn't have the right build-tag set)

    daemon/logger/journald/read.go:238:4: SA4011: ineffective break statement. Did you mean to break out of the outer loop? (staticcheck)
                break // won't be able to write anything anymore
                ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 75577fe7a8)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-20 16:57:22 +02:00
Brian Goff
48a144954e
Merge pull request #43812 from thaJeztah/22.06_backport_43481_support_pku
[22.06 backport] profiles: seccomp: add syscalls related to PKU in default policy
2022-07-18 15:51:39 -07:00
Sebastiaan van Stijn
c4c8a80958
Merge pull request #43813 from thaJeztah/22.06_backport_fix_43781
[22.06 backport] libnetwork: skip firewalld management for rootless
2022-07-18 09:48:12 +02:00
Sebastiaan van Stijn
1b928c1bd5
Merge pull request #43814 from thaJeztah/22.06_backport_gofmt_119_2
[22.06 backport] fix formatting of "nolint" tags for go1.19
2022-07-18 09:47:27 +02:00
Sebastiaan van Stijn
e34ab5200d
fix formatting of "nolint" tags for go1.19
The correct formatting for machine-readable comments is;

    //<some alphanumeric identifier>:<options>[,<option>...][ // comment]

Which basically means:

- MUST NOT have a space before `<identifier>` (e.g. `nolint`)
- Identified MUST be alphanumeric
- MUST be followed by a colon
- MUST be followed by at least one `<option>`
- Optionally additional `<options>` (comma-separated)
- Optionally followed by a comment

Any other format will not be considered a machine-readable comment by `gofmt`,
and thus formatted as a regular comment. Note that this also means that a
`//nolint` (without anything after it) is considered invalid, same for `//#nosec`
(starts with a `#`).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 4f08346686)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-15 13:45:13 +02:00
Akihiro Suda
863ca3f185
libnetwork: skip firewalld management for rootless
Fix issue 43781

Co-authored-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit 9464898b47)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-15 13:40:07 +02:00
zhubojun
edcc51cbee
profiles: seccomp: add syscalls related to PKU in default policy
Add pkey_alloc(2), pkey_free(2) and pkey_mprotect(2) in seccomp default profile.
pkey_alloc(2), pkey_free(2) and pkey_mprotect(2) can only configure
the calling process's own memory, so they are existing "safe for everyone" syscalls.

close issue: #43481

Signed-off-by: zhubojun <bojun.zhu@foxmail.com>
(cherry picked from commit e258d66f17)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-15 09:19:57 +02:00
Sebastiaan van Stijn
6408132d74
Merge pull request #43808 from thaJeztah/22.06_backport_client_deadcode
[22.06 backport] client: errors: remove dead code
2022-07-14 22:02:30 +02:00
Sebastiaan van Stijn
d64dd71200
Merge pull request #43809 from thaJeztah/22.06_backport_api_docs
[22.06 backport] docs: add API v1.42
2022-07-14 22:00:20 +02:00
Brian Goff
e0ba440909
Merge pull request #43806 from thaJeztah/22.06_backport_fix_import
[22.06 backport] pkg/parsers/operatingsystem: fix stray import
2022-07-14 08:19:15 -07:00
Sebastiaan van Stijn
269e10a725
Merge pull request #43807 from thaJeztah/22.06_backport_gofmt_119
[22.06 backport] format (GoDoc) comments with Go 1.19 to prepare for future updates
2022-07-14 16:48:12 +02:00
Sebastiaan van Stijn
149b7e7f03
Merge pull request #43805 from thaJeztah/22.06_backport_bump_go_1.18.4
[22.06 backport] update golang to 1.18.4
2022-07-14 16:47:51 +02:00
Sebastiaan van Stijn
c51efa8617
docs: add API v1.42
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 264b41fb9e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-13 22:49:14 +02:00
Sebastiaan van Stijn
52791b1c14
client: errors: remove dead code
- Update IsErrNotFound() to check for the current type before falling back to
  detecting the deprecated type.
- Remove unauthorizedError and notImplementedError types, which were not used.
- IsErrPluginPermissionDenied() was added in 7c36a1af03,
  but not used at the time, and still appears to be unused.
- Deprecate IsErrUnauthorized in favor of errdefs.IsUnauthorized()
- Deprecate IsErrNotImplemented in favor of errdefs,IsNotImplemented()

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit ee230d8fdd)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-13 22:44:43 +02:00
Sebastiaan van Stijn
cdbca4061b
gofmt GoDoc comments with go1.19
Older versions of Go don't format comments, so committing this as
a separate commit, so that we can already make these changes before
we upgrade to Go 1.19.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 52c1a2fae8)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-13 22:42:29 +02:00
Sebastiaan van Stijn
c52e221207
gofmt files
result of:

    gofmt -s -w $(find . -type f -name '*.go' | grep -v "/vendor/")

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 6668801d40)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-13 22:42:26 +02:00
Sebastiaan van Stijn
e417e8dfc2
pkg/parsers/operatingsystem: fix stray import
This was caught by goimports;

    goimports -w $(find . -type f -name '*.go'| grep -v "/vendor/")

CI doesn't run on these platforms, so didn't catch it.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit e4e819b49c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-13 22:40:22 +02:00
Sebastiaan van Stijn
6905fe7488
update golang to 1.18.4
go1.18.4 (released 2022-07-12) includes security fixes to the compress/gzip,
encoding/gob, encoding/xml, go/parser, io/fs, net/http, and path/filepath
packages, as well as bug fixes to the compiler, the go command, the linker,
the runtime, and the runtime/metrics package. See the Go 1.18.4 milestone on the
issue tracker for details:

https://github.com/golang/go/issues?q=milestone%3AGo1.18.4+label%3ACherryPickApproved

This update addresses:

CVE-2022-1705, CVE-2022-1962, CVE-2022-28131, CVE-2022-30630, CVE-2022-30631,
CVE-2022-30632, CVE-2022-30633, CVE-2022-30635, and CVE-2022-32148.

Full diff: https://github.com/golang/go/compare/go1.18.3...go1.18.4

From the security announcement;
https://groups.google.com/g/golang-announce/c/nqrv9fbR0zE

We have just released Go versions 1.18.4 and 1.17.12, minor point releases. These
minor releases include 9 security fixes following the security policy:

- net/http: improper sanitization of Transfer-Encoding header

  The HTTP/1 client accepted some invalid Transfer-Encoding headers as indicating
  a "chunked" encoding. This could potentially allow for request smuggling, but
  only if combined with an intermediate server that also improperly failed to
  reject the header as invalid.

  This is CVE-2022-1705 and https://go.dev/issue/53188.

- When `httputil.ReverseProxy.ServeHTTP` was called with a `Request.Header` map
  containing a nil value for the X-Forwarded-For header, ReverseProxy would set
  the client IP as the value of the X-Forwarded-For header, contrary to its
  documentation. In the more usual case where a Director function set the
  X-Forwarded-For header value to nil, ReverseProxy would leave the header
  unmodified as expected.

  This is https://go.dev/issue/53423 and CVE-2022-32148.

  Thanks to Christian Mehlmauer for reporting this issue.

- compress/gzip: stack exhaustion in Reader.Read

  Calling Reader.Read on an archive containing a large number of concatenated
  0-length compressed files can cause a panic due to stack exhaustion.

  This is CVE-2022-30631 and Go issue https://go.dev/issue/53168.

- encoding/xml: stack exhaustion in Unmarshal

  Calling Unmarshal on a XML document into a Go struct which has a nested field
  that uses the any field tag can cause a panic due to stack exhaustion.

  This is CVE-2022-30633 and Go issue https://go.dev/issue/53611.

- encoding/xml: stack exhaustion in Decoder.Skip

  Calling Decoder.Skip when parsing a deeply nested XML document can cause a
  panic due to stack exhaustion. The Go Security team discovered this issue, and
  it was independently reported by Juho Nurminen of Mattermost.

  This is CVE-2022-28131 and Go issue https://go.dev/issue/53614.

- encoding/gob: stack exhaustion in Decoder.Decode

  Calling Decoder.Decode on a message which contains deeply nested structures
  can cause a panic due to stack exhaustion.

  This is CVE-2022-30635 and Go issue https://go.dev/issue/53615.

- path/filepath: stack exhaustion in Glob

  Calling Glob on a path which contains a large number of path separators can
  cause a panic due to stack exhaustion.

  Thanks to Juho Nurminen of Mattermost for reporting this issue.

  This is CVE-2022-30632 and Go issue https://go.dev/issue/53416.

- io/fs: stack exhaustion in Glob

  Calling Glob on a path which contains a large number of path separators can
  cause a panic due to stack exhaustion.

  This is CVE-2022-30630 and Go issue https://go.dev/issue/53415.

- go/parser: stack exhaustion in all Parse* functions

  Calling any of the Parse functions on Go source code which contains deeply
  nested types or declarations can cause a panic due to stack exhaustion.

  Thanks to Juho Nurminen of Mattermost for reporting this issue.

  This is CVE-2022-1962 and Go issue https://go.dev/issue/53616.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 34b8670b1a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-13 22:37:32 +02:00
2072 changed files with 142441 additions and 48109 deletions

View file

@ -1,22 +0,0 @@
curators:
- aboch
- alexellis
- andrewhsu
- anonymuse
- arkodg
- chanwit
- ehazlett
- fntlnz
- gianarb
- kolyshkin
- mgoelzer
- olljanat
- programmerq
- rheinwein
- ripcurld0
- thajeztah
features:
- comments
- pr_description_required

View file

@ -1,7 +1,4 @@
.git
.go-pkg-cache
.gopath
bundles
bundles/
cli/winresources/**/winres.json
cli/winresources/**/*.syso
vendor/pkg

27
.github/actions/setup-runner/action.yml vendored Normal file
View file

@ -0,0 +1,27 @@
name: 'Setup Runner'
description: 'Composite action to set up the GitHub Runner for jobs in the test.yml workflow'
runs:
using: composite
steps:
- run: |
sudo modprobe ip_vs
sudo modprobe ipv6
sudo modprobe ip6table_filter
sudo modprobe -r overlay
sudo modprobe overlay redirect_dir=off
shell: bash
- run: |
if [ ! -e /etc/docker/daemon.json ]; then
echo '{}' | sudo tee /etc/docker/daemon.json >/dev/null
fi
DOCKERD_CONFIG=$(jq '.+{"experimental":true,"live-restore":true,"ipv6":true,"fixed-cidr-v6":"2001:db8:1::/64"}' /etc/docker/daemon.json)
sudo tee /etc/docker/daemon.json <<<"$DOCKERD_CONFIG" >/dev/null
sudo service docker restart
shell: bash
- run: |
./contrib/check-config.sh || true
shell: bash
- run: |
docker info
shell: bash

48
.github/workflows/.dco.yml vendored Normal file
View file

@ -0,0 +1,48 @@
# reusable workflow
name: .dco
# TODO: hide reusable workflow from the UI. Tracked in https://github.com/community/community/discussions/12025
on:
workflow_call:
env:
ALPINE_VERSION: 3.16
jobs:
run:
runs-on: ubuntu-20.04
steps:
-
name: Checkout
uses: actions/checkout@v3
with:
fetch-depth: 0
-
name: Dump context
uses: actions/github-script@v6
with:
script: |
console.log(JSON.stringify(context, null, 2));
-
name: Get base ref
id: base-ref
uses: actions/github-script@v6
with:
result-encoding: string
script: |
if (/^refs\/pull\//.test(context.ref) && context.payload?.pull_request?.base?.ref != undefined) {
return context.payload.pull_request.base.ref;
}
return context.ref.replace(/^refs\/heads\//g, '');
-
name: Validate
run: |
docker run --rm \
-v "$(pwd):/workspace" \
-e VALIDATE_REPO \
-e VALIDATE_BRANCH \
alpine:${{ env.ALPINE_VERSION }} sh -c 'apk add --no-cache -q bash git openssh-client && git config --system --add safe.directory /workspace && cd /workspace && hack/validate/dco'
env:
VALIDATE_REPO: ${{ github.server_url }}/${{ github.repository }}.git
VALIDATE_BRANCH: ${{ steps.base-ref.outputs.result }}

View file

@ -1,22 +1,22 @@
name: windows
# reusable workflow
name: .windows
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
# TODO: hide reusable workflow from the UI. Tracked in https://github.com/community/community/discussions/12025
on:
workflow_dispatch:
push:
branches:
- 'master'
- '[0-9]+.[0-9]{2}'
tags:
- 'v*'
pull_request:
workflow_call:
inputs:
os:
required: true
type: string
send_coverage:
required: false
type: boolean
default: false
env:
GO_VERSION: 1.18.3
GOTESTLIST_VERSION: v0.2.0
GO_VERSION: "1.20.13"
GOTESTLIST_VERSION: v0.3.1
TESTSTAT_VERSION: v0.1.3
WINDOWS_BASE_IMAGE: mcr.microsoft.com/windows/servercore
WINDOWS_BASE_TAG_2019: ltsc2019
@ -24,16 +24,11 @@ env:
TEST_IMAGE_NAME: moby:test
TEST_CTN_NAME: moby
DOCKER_BUILDKIT: 0
ITG_CLI_MATRIX_SIZE: 6
jobs:
build:
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
os:
- windows-2019
- windows-2022
runs-on: ${{ inputs.os }}
env:
GOPATH: ${{ github.workspace }}\go
GOBIN: ${{ github.workspace }}\go\bin
@ -56,9 +51,9 @@ jobs:
run: |
New-Item -ItemType "directory" -Path "${{ github.workspace }}\go-build"
New-Item -ItemType "directory" -Path "${{ github.workspace }}\go\pkg\mod"
If ("${{ matrix.os }}" -eq "windows-2019") {
If ("${{ inputs.os }}" -eq "windows-2019") {
echo "WINDOWS_BASE_IMAGE_TAG=${{ env.WINDOWS_BASE_TAG_2019 }}" | Out-File -FilePath $Env:GITHUB_ENV -Encoding utf-8 -Append
} ElseIf ("${{ matrix.os }}" -eq "windows-2022") {
} ElseIf ("${{ inputs.os }}" -eq "windows-2022") {
echo "WINDOWS_BASE_IMAGE_TAG=${{ env.WINDOWS_BASE_TAG_2022 }}" | Out-File -FilePath $Env:GITHUB_ENV -Encoding utf-8 -Append
}
-
@ -70,9 +65,9 @@ jobs:
~\go\pkg\mod
${{ github.workspace }}\go-build
${{ env.GOPATH }}\pkg\mod
key: ${{ matrix.os }}-${{ github.job }}-${{ hashFiles('**/vendor.sum') }}
key: ${{ inputs.os }}-${{ github.job }}-${{ hashFiles('**/vendor.sum') }}
restore-keys: |
${{ matrix.os }}-${{ github.job }}-
${{ inputs.os }}-${{ github.job }}-
-
name: Docker info
run: |
@ -103,19 +98,14 @@ jobs:
name: Upload artifacts
uses: actions/upload-artifact@v3
with:
name: build-${{ matrix.os }}
name: build-${{ inputs.os }}
path: ${{ env.BIN_OUT }}/*
if-no-files-found: error
retention-days: 2
unit-test:
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
os:
- windows-2019
- windows-2022
runs-on: ${{ inputs.os }}
timeout-minutes: 120
env:
GOPATH: ${{ github.workspace }}\go
GOBIN: ${{ github.workspace }}\go\bin
@ -138,9 +128,9 @@ jobs:
New-Item -ItemType "directory" -Path "${{ github.workspace }}\go-build"
New-Item -ItemType "directory" -Path "${{ github.workspace }}\go\pkg\mod"
New-Item -ItemType "directory" -Path "bundles"
If ("${{ matrix.os }}" -eq "windows-2019") {
If ("${{ inputs.os }}" -eq "windows-2019") {
echo "WINDOWS_BASE_IMAGE_TAG=${{ env.WINDOWS_BASE_TAG_2019 }}" | Out-File -FilePath $Env:GITHUB_ENV -Encoding utf-8 -Append
} ElseIf ("${{ matrix.os }}" -eq "windows-2022") {
} ElseIf ("${{ inputs.os }}" -eq "windows-2022") {
echo "WINDOWS_BASE_IMAGE_TAG=${{ env.WINDOWS_BASE_TAG_2022 }}" | Out-File -FilePath $Env:GITHUB_ENV -Encoding utf-8 -Append
}
-
@ -152,9 +142,9 @@ jobs:
~\go\pkg\mod
${{ github.workspace }}\go-build
${{ env.GOPATH }}\pkg\mod
key: ${{ matrix.os }}-${{ github.job }}-${{ hashFiles('**/vendor.sum') }}
key: ${{ inputs.os }}-${{ github.job }}-${{ hashFiles('**/vendor.sum') }}
restore-keys: |
${{ matrix.os }}-${{ github.job }}-
${{ inputs.os }}-${{ github.job }}-
-
name: Docker info
run: |
@ -175,7 +165,7 @@ jobs:
${{ env.TEST_IMAGE_NAME }} hack\make.ps1 -TestUnit
-
name: Send to Codecov
if: matrix.os == 'windows-2022'
if: inputs.send_coverage
uses: codecov/codecov-action@v3
with:
working-directory: ${{ env.GOPATH }}\src\github.com\docker\docker
@ -187,7 +177,7 @@ jobs:
if: always()
uses: actions/upload-artifact@v3
with:
name: ${{ matrix.os }}-unit-reports
name: ${{ inputs.os }}-unit-reports
path: ${{ env.GOPATH }}\src\github.com\docker\docker\bundles\*
unit-test-report:
@ -195,12 +185,6 @@ jobs:
if: always()
needs:
- unit-test
strategy:
fail-fast: false
matrix:
os:
- windows-2019
- windows-2022
steps:
-
name: Set up Go
@ -211,7 +195,7 @@ jobs:
name: Download artifacts
uses: actions/download-artifact@v3
with:
name: ${{ matrix.os }}-unit-reports
name: ${{ inputs.os }}-unit-reports
path: /tmp/artifacts
-
name: Install teststat
@ -244,27 +228,26 @@ jobs:
id: tests
working-directory: ./integration-cli
run: |
# Distribute integration-cli tests for the matrix in integration-test job.
# Also prepend ./... to the matrix. This is a special case to run "Test integration" step exclusively.
matrix="$(gotestlist -d 4 ./...)"
matrix="$(echo "$matrix" | jq -c '. |= ["./..."] + .')"
echo "::set-output name=matrix::$matrix"
# This step creates a matrix for integration-cli tests. Tests suites
# are distributed in integration-test job through a matrix. There is
# also an override being added to the matrix like "./..." to run
# "Test integration" step exclusively.
matrix="$(gotestlist -d ${{ env.ITG_CLI_MATRIX_SIZE }} -o "./..." ./...)"
echo "matrix=$matrix" >> $GITHUB_OUTPUT
-
name: Show matrix
run: |
echo ${{ steps.tests.outputs.matrix }}
integration-test:
runs-on: ${{ matrix.os }}
runs-on: ${{ inputs.os }}
timeout-minutes: 120
needs:
- build
- integration-test-prepare
strategy:
fail-fast: false
matrix:
os:
- windows-2019
- windows-2022
runtime:
- builtin
- containerd
@ -290,15 +273,15 @@ jobs:
name: Download artifacts
uses: actions/download-artifact@v3
with:
name: build-${{ matrix.os }}
name: build-${{ inputs.os }}
path: ${{ env.BIN_OUT }}
-
name: Init
run: |
New-Item -ItemType "directory" -Path "bundles"
If ("${{ matrix.os }}" -eq "windows-2019") {
If ("${{ inputs.os }}" -eq "windows-2019") {
echo "WINDOWS_BASE_IMAGE_TAG=${{ env.WINDOWS_BASE_TAG_2019 }}" | Out-File -FilePath $Env:GITHUB_ENV -Encoding utf-8 -Append
} ElseIf ("${{ matrix.os }}" -eq "windows-2022") {
} ElseIf ("${{ inputs.os }}" -eq "windows-2022") {
echo "WINDOWS_BASE_IMAGE_TAG=${{ env.WINDOWS_BASE_TAG_2022 }}" | Out-File -FilePath $Env:GITHUB_ENV -Encoding utf-8 -Append
}
Write-Output "${{ env.BIN_OUT }}" | Out-File -FilePath $env:GITHUB_PATH -Encoding utf8 -Append
@ -315,7 +298,7 @@ jobs:
Stop-Service -Force -Name docker
Remove-Service -Name docker
# removes event log entry. we could use "Remove-EventLog -LogName -Source docker"
# but this cmd is only available since windows-2022
# but this cmd is not available atm
$ErrorActionPreference = "SilentlyContinue"
& reg delete "HKLM\SYSTEM\CurrentControlSet\Services\EventLog\Application\docker" /f 2>&1 | Out-Null
$ErrorActionPreference = "Stop"
@ -431,7 +414,7 @@ jobs:
INTEGRATION_TESTRUN: ${{ matrix.test }}
-
name: Send to Codecov
if: matrix.os == 'windows-2022'
if: inputs.send_coverage
uses: codecov/codecov-action@v3
with:
working-directory: ${{ env.GOPATH }}\src\github.com\docker\docker
@ -480,7 +463,7 @@ jobs:
if: always()
uses: actions/upload-artifact@v3
with:
name: ${{ matrix.os }}-integration-reports-${{ matrix.runtime }}
name: ${{ inputs.os }}-integration-reports-${{ matrix.runtime }}
path: ${{ env.GOPATH }}\src\github.com\docker\docker\bundles\*
integration-test-report:
@ -491,9 +474,6 @@ jobs:
strategy:
fail-fast: false
matrix:
os:
- windows-2019
- windows-2022
runtime:
- builtin
- containerd
@ -507,7 +487,7 @@ jobs:
name: Download artifacts
uses: actions/download-artifact@v3
with:
name: ${{ matrix.os }}-integration-reports-${{ matrix.runtime }}
name: ${{ inputs.os }}-integration-reports-${{ matrix.runtime }}
path: /tmp/artifacts
-
name: Install teststat

111
.github/workflows/buildkit.yml vendored Normal file
View file

@ -0,0 +1,111 @@
name: buildkit
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
on:
workflow_dispatch:
push:
branches:
- 'master'
- '[0-9]+.[0-9]+'
pull_request:
env:
DESTDIR: ./build
jobs:
validate-dco:
uses: ./.github/workflows/.dco.yml
build:
runs-on: ubuntu-20.04
needs:
- validate-dco
steps:
-
name: Checkout
uses: actions/checkout@v3
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
-
name: Build
uses: docker/bake-action@v2
with:
targets: binary
-
name: Upload artifacts
uses: actions/upload-artifact@v3
with:
name: binary
path: ${{ env.DESTDIR }}
if-no-files-found: error
retention-days: 1
test:
runs-on: ubuntu-20.04
timeout-minutes: 120
needs:
- build
strategy:
fail-fast: false
matrix:
pkg:
- client
- cmd/buildctl
- solver
- frontend
- frontend/dockerfile
typ:
- integration
steps:
-
name: Checkout
uses: actions/checkout@v3
with:
path: moby
-
name: BuildKit ref
run: |
echo "$(./hack/buildkit-ref)" >> $GITHUB_ENV
working-directory: moby
-
name: Checkout BuildKit ${{ env.BUILDKIT_REF }}
uses: actions/checkout@v3
with:
repository: "moby/buildkit"
ref: ${{ env.BUILDKIT_REF }}
path: buildkit
-
name: Set up QEMU
uses: docker/setup-qemu-action@v2
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
-
name: Download binary artifacts
uses: actions/download-artifact@v3
with:
name: binary
path: ./buildkit/build/moby/
-
name: Update daemon.json
run: |
sudo rm -f /etc/docker/daemon.json
sudo service docker restart
docker version
docker info
-
name: Test
run: |
./hack/test ${{ matrix.typ }}
env:
CONTEXT: "."
TEST_DOCKERD: "1"
TEST_DOCKERD_BINARY: "./build/moby/dockerd"
TESTPKGS: "./${{ matrix.pkg }}"
# Diff/MergeOp tests are skipped
TESTFLAGS: "-v --parallel=1 --timeout=30m --run=/^Test([^DM]|.[^ie]|..[^fr]|...[^fg])/worker=dockerd$"
working-directory: buildkit

View file

@ -9,17 +9,22 @@ on:
push:
branches:
- 'master'
- '[0-9]+.[0-9]{2}'
- '[0-9]+.[0-9]+'
tags:
- 'v*'
pull_request:
env:
BUNDLES_OUTPUT: ./bundles
DESTDIR: ./build
jobs:
validate-dco:
uses: ./.github/workflows/.dco.yml
build:
runs-on: ubuntu-20.04
needs:
- validate-dco
strategy:
fail-fast: false
matrix:
@ -34,39 +39,59 @@ jobs:
fetch-depth: 0
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v1
uses: docker/setup-buildx-action@v2
-
name: Build
uses: docker/bake-action@v1
uses: docker/bake-action@v2
with:
targets: ${{ matrix.target }}
set: |
*.cache-from=type=gha,scope=build-${{ matrix.target }}
*.cache-to=type=gha,scope=build-${{ matrix.target }}
-
name: List artifacts
run: |
tree -nh ${{ env.DESTDIR }}
-
name: Check artifacts
run: |
find ${{ env.DESTDIR }} -type f -exec file -e ascii -- {} +
-
name: Upload artifacts
uses: actions/upload-artifact@v3
with:
name: ${{ matrix.target }}
path: ${{ env.BUNDLES_OUTPUT }}
path: ${{ env.DESTDIR }}
if-no-files-found: error
retention-days: 7
prepare-cross:
runs-on: ubuntu-latest
needs:
- validate-dco
outputs:
matrix: ${{ steps.platforms.outputs.matrix }}
steps:
-
name: Checkout
uses: actions/checkout@v3
-
name: Create matrix
id: platforms
run: |
matrix="$(docker buildx bake binary-cross --print | jq -cr '.target."binary-cross".platforms')"
echo "matrix=$matrix" >> $GITHUB_OUTPUT
-
name: Show matrix
run: |
echo ${{ steps.platforms.outputs.matrix }}
cross:
runs-on: ubuntu-20.04
needs:
- validate-dco
- prepare-cross
strategy:
fail-fast: false
matrix:
platform:
- linux/amd64
- linux/arm/v5
- linux/arm/v6
- linux/arm/v7
- linux/arm64
- linux/ppc64le
- linux/s390x
- windows/amd64
- windows/arm64
platform: ${{ fromJson(needs.prepare-cross.outputs.matrix) }}
steps:
-
name: Checkout
@ -80,93 +105,27 @@ jobs:
echo "PLATFORM_PAIR=${platform//\//-}" >> $GITHUB_ENV
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v1
uses: docker/setup-buildx-action@v2
-
name: Build
uses: docker/bake-action@v1
uses: docker/bake-action@v2
with:
targets: cross
targets: all
set: |
*.cache-from=type=gha,scope=cross-${{ env.PLATFORM_PAIR }}
*.cache-to=type=gha,scope=cross-${{ env.PLATFORM_PAIR }}
env:
DOCKER_CROSSPLATFORMS: ${{ matrix.platform }}
*.platform=${{ matrix.platform }}
-
name: List artifacts
run: |
tree -nh ${{ env.DESTDIR }}
-
name: Check artifacts
run: |
find ${{ env.DESTDIR }} -type f -exec file -e ascii -- {} +
-
name: Upload artifacts
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v3
with:
name: cross-${{ env.PLATFORM_PAIR }}
path: ${{ env.BUNDLES_OUTPUT }}
path: ${{ env.DESTDIR }}
if-no-files-found: error
retention-days: 7
test-buildkit:
needs:
- build
runs-on: ubuntu-20.04
strategy:
fail-fast: false
matrix:
pkg:
- ./client
- ./cmd/buildctl
- ./solver
- ./frontend
- ./frontend/dockerfile
typ:
- integration
include:
- pkg: ./...
skip-integration-tests: 1
steps:
-
name: Checkout
uses: actions/checkout@v3
with:
path: moby
-
name: BuildKit ref
run: |
./hack/go-mod-prepare.sh
echo "BUILDKIT_REF=0da740f7d4f782a52b416a44f564ac37504b9ee1" >> $GITHUB_ENV
# FIXME(thaJeztah) temporarily overriding version to use for tests to include https://github.com/moby/buildkit/pull/2872
# echo "BUILDKIT_REF=$(./hack/buildkit-ref)" >> $GITHUB_ENV
working-directory: moby
-
name: Checkout BuildKit ${{ env.BUILDKIT_REF }}
uses: actions/checkout@v3
with:
repository: "moby/buildkit"
ref: ${{ env.BUILDKIT_REF }}
path: buildkit
-
name: Set up QEMU
uses: docker/setup-qemu-action@v1
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v1
-
name: Download binary artifacts
uses: actions/download-artifact@v3
with:
name: binary
path: ./buildkit/build/moby/
-
name: Update daemon.json
run: |
sudo rm /etc/docker/daemon.json
sudo service docker restart
docker version
docker info
-
name: Test
run: |
./hack/test ${{ matrix.typ }}
env:
CONTEXT: "."
TEST_DOCKERD: "1"
TEST_DOCKERD_BINARY: "./build/moby/binary-daemon/dockerd"
TESTPKGS: "${{ matrix.pkg }}"
TESTFLAGS: "-v --parallel=1 --timeout=30m --run=//worker=dockerd$"
SKIP_INTEGRATION_TESTS: "${{ matrix.skip-integration-tests }}"
working-directory: buildkit

562
.github/workflows/test.yml vendored Normal file
View file

@ -0,0 +1,562 @@
name: test
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
on:
workflow_dispatch:
push:
branches:
- 'master'
- '[0-9]+.[0-9]+'
tags:
- 'v*'
pull_request:
env:
GO_VERSION: "1.20.13"
GOTESTLIST_VERSION: v0.3.1
TESTSTAT_VERSION: v0.1.3
ITG_CLI_MATRIX_SIZE: 6
DOCKER_EXPERIMENTAL: 1
DOCKER_GRAPHDRIVER: overlay2
jobs:
validate-dco:
uses: ./.github/workflows/.dco.yml
build-dev:
runs-on: ubuntu-20.04
needs:
- validate-dco
strategy:
fail-fast: false
matrix:
mode:
- ""
- systemd
steps:
-
name: Prepare
run: |
if [ "${{ matrix.mode }}" = "systemd" ]; then
echo "SYSTEMD=true" >> $GITHUB_ENV
fi
-
name: Checkout
uses: actions/checkout@v3
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
-
name: Build dev image
uses: docker/bake-action@v2
with:
targets: dev
set: |
*.cache-from=type=gha,scope=dev${{ matrix.mode }}
*.cache-to=type=gha,scope=dev${{ matrix.mode }},mode=max
*.output=type=cacheonly
validate-prepare:
runs-on: ubuntu-20.04
needs:
- validate-dco
outputs:
matrix: ${{ steps.scripts.outputs.matrix }}
steps:
-
name: Checkout
uses: actions/checkout@v3
-
name: Create matrix
id: scripts
run: |
scripts=$(jq -ncR '[inputs]' <<< "$(ls -I .validate -I all -I default -I dco -I golangci-lint.yml -I yamllint.yaml -A ./hack/validate/)")
echo "matrix=$scripts" >> $GITHUB_OUTPUT
-
name: Show matrix
run: |
echo ${{ steps.scripts.outputs.matrix }}
validate:
runs-on: ubuntu-20.04
timeout-minutes: 120
needs:
- validate-prepare
- build-dev
strategy:
fail-fast: true
matrix:
script: ${{ fromJson(needs.validate-prepare.outputs.matrix) }}
steps:
-
name: Checkout
uses: actions/checkout@v3
with:
fetch-depth: 0
-
name: Set up runner
uses: ./.github/actions/setup-runner
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
-
name: Build dev image
uses: docker/bake-action@v2
with:
targets: dev
set: |
dev.cache-from=type=gha,scope=dev
-
name: Validate
run: |
make -o build validate-${{ matrix.script }}
unit:
runs-on: ubuntu-20.04
timeout-minutes: 120
needs:
- build-dev
steps:
-
name: Checkout
uses: actions/checkout@v3
-
name: Set up runner
uses: ./.github/actions/setup-runner
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
-
name: Build dev image
uses: docker/bake-action@v2
with:
targets: dev
set: |
dev.cache-from=type=gha,scope=dev
-
name: Test
run: |
make -o build test-unit
-
name: Prepare reports
if: always()
run: |
mkdir -p bundles /tmp/reports
find bundles -path '*/root/*overlay2' -prune -o -type f \( -name '*-report.json' -o -name '*.log' -o -name '*.out' -o -name '*.prof' -o -name '*-report.xml' \) -print | xargs sudo tar -czf /tmp/reports.tar.gz
tar -xzf /tmp/reports.tar.gz -C /tmp/reports
sudo chown -R $(id -u):$(id -g) /tmp/reports
tree -nh /tmp/reports
-
name: Send to Codecov
uses: codecov/codecov-action@v3
with:
directory: ./bundles
env_vars: RUNNER_OS
flags: unit
-
name: Upload reports
if: always()
uses: actions/upload-artifact@v3
with:
name: unit-reports
path: /tmp/reports/*
unit-report:
runs-on: ubuntu-20.04
timeout-minutes: 10
if: always()
needs:
- unit
steps:
-
name: Set up Go
uses: actions/setup-go@v3
with:
go-version: ${{ env.GO_VERSION }}
-
name: Download reports
uses: actions/download-artifact@v3
with:
name: unit-reports
path: /tmp/reports
-
name: Install teststat
run: |
go install github.com/vearutop/teststat@${{ env.TESTSTAT_VERSION }}
-
name: Create summary
run: |
teststat -markdown $(find /tmp/reports -type f -name '*.json' -print0 | xargs -0) >> $GITHUB_STEP_SUMMARY
docker-py:
runs-on: ubuntu-20.04
timeout-minutes: 120
needs:
- build-dev
steps:
-
name: Checkout
uses: actions/checkout@v3
-
name: Set up runner
uses: ./.github/actions/setup-runner
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
-
name: Build dev image
uses: docker/bake-action@v2
with:
targets: dev
set: |
dev.cache-from=type=gha,scope=dev
-
name: Test
run: |
make -o build test-docker-py
-
name: Prepare reports
if: always()
run: |
mkdir -p bundles /tmp/reports
find bundles -path '*/root/*overlay2' -prune -o -type f \( -name '*-report.json' -o -name '*.log' -o -name '*.out' -o -name '*.prof' -o -name '*-report.xml' \) -print | xargs sudo tar -czf /tmp/reports.tar.gz
tar -xzf /tmp/reports.tar.gz -C /tmp/reports
sudo chown -R $(id -u):$(id -g) /tmp/reports
tree -nh /tmp/reports
-
name: Test daemon logs
if: always()
run: |
cat bundles/test-docker-py/docker.log
-
name: Upload reports
if: always()
uses: actions/upload-artifact@v3
with:
name: docker-py-reports
path: /tmp/reports/*
integration-flaky:
runs-on: ubuntu-20.04
timeout-minutes: 120
needs:
- build-dev
steps:
-
name: Checkout
uses: actions/checkout@v3
-
name: Set up runner
uses: ./.github/actions/setup-runner
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
-
name: Build dev image
uses: docker/bake-action@v2
with:
targets: dev
set: |
dev.cache-from=type=gha,scope=dev
-
name: Test
run: |
make -o build test-integration-flaky
env:
TEST_SKIP_INTEGRATION_CLI: 1
integration:
runs-on: ${{ matrix.os }}
timeout-minutes: 120
needs:
- build-dev
strategy:
fail-fast: false
matrix:
os:
- ubuntu-20.04
- ubuntu-22.04
mode:
- ""
- rootless
- systemd
#- rootless-systemd FIXME: https://github.com/moby/moby/issues/44084
steps:
-
name: Checkout
uses: actions/checkout@v3
-
name: Set up runner
uses: ./.github/actions/setup-runner
-
name: Prepare
run: |
CACHE_DEV_SCOPE=dev
if [[ "${{ matrix.mode }}" == *"rootless"* ]]; then
echo "DOCKER_ROOTLESS=1" >> $GITHUB_ENV
fi
if [[ "${{ matrix.mode }}" == *"systemd"* ]]; then
echo "SYSTEMD=true" >> $GITHUB_ENV
CACHE_DEV_SCOPE="${CACHE_DEV_SCOPE}systemd"
fi
echo "CACHE_DEV_SCOPE=${CACHE_DEV_SCOPE}" >> $GITHUB_ENV
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
-
name: Build dev image
uses: docker/bake-action@v2
with:
targets: dev
set: |
dev.cache-from=type=gha,scope=${{ env.CACHE_DEV_SCOPE }}
-
name: Test
run: |
make -o build test-integration
env:
TEST_SKIP_INTEGRATION_CLI: 1
TESTCOVERAGE: 1
-
name: Prepare reports
if: always()
run: |
reportsPath="/tmp/reports/${{ matrix.os }}"
if [ -n "${{ matrix.mode }}" ]; then
reportsPath="$reportsPath-${{ matrix.mode }}"
fi
mkdir -p bundles $reportsPath
find bundles -path '*/root/*overlay2' -prune -o -type f \( -name '*-report.json' -o -name '*.log' -o -name '*.out' -o -name '*.prof' -o -name '*-report.xml' \) -print | xargs sudo tar -czf /tmp/reports.tar.gz
tar -xzf /tmp/reports.tar.gz -C $reportsPath
sudo chown -R $(id -u):$(id -g) $reportsPath
tree -nh $reportsPath
-
name: Send to Codecov
uses: codecov/codecov-action@v3
with:
directory: ./bundles/test-integration
env_vars: RUNNER_OS
flags: integration,${{ matrix.mode }}
-
name: Test daemon logs
if: always()
run: |
cat bundles/test-integration/docker.log
-
name: Upload reports
if: always()
uses: actions/upload-artifact@v3
with:
name: integration-reports
path: /tmp/reports/*
integration-report:
runs-on: ubuntu-20.04
timeout-minutes: 10
if: always()
needs:
- integration
steps:
-
name: Set up Go
uses: actions/setup-go@v3
with:
go-version: ${{ env.GO_VERSION }}
-
name: Download reports
uses: actions/download-artifact@v3
with:
name: integration-reports
path: /tmp/reports
-
name: Install teststat
run: |
go install github.com/vearutop/teststat@${{ env.TESTSTAT_VERSION }}
-
name: Create summary
run: |
teststat -markdown $(find /tmp/reports -type f -name '*.json' -print0 | xargs -0) >> $GITHUB_STEP_SUMMARY
integration-cli-prepare:
runs-on: ubuntu-20.04
needs:
- validate-dco
outputs:
matrix: ${{ steps.tests.outputs.matrix }}
steps:
-
name: Checkout
uses: actions/checkout@v3
-
name: Set up Go
uses: actions/setup-go@v3
with:
go-version: ${{ env.GO_VERSION }}
-
name: Install gotestlist
run:
go install github.com/crazy-max/gotestlist/cmd/gotestlist@${{ env.GOTESTLIST_VERSION }}
-
name: Create matrix
id: tests
working-directory: ./integration-cli
run: |
# This step creates a matrix for integration-cli tests. Tests suites
# are distributed in integration-cli job through a matrix. There is
# also overrides being added to the matrix like "./..." to run
# "Test integration" step exclusively and specific tests suites that
# take a long time to run.
matrix="$(gotestlist -d ${{ env.ITG_CLI_MATRIX_SIZE }} -o "./..." -o "DockerSwarmSuite" -o "DockerNetworkSuite|DockerExternalVolumeSuite" ./...)"
echo "matrix=$matrix" >> $GITHUB_OUTPUT
-
name: Show matrix
run: |
echo ${{ steps.tests.outputs.matrix }}
integration-cli:
runs-on: ubuntu-20.04
timeout-minutes: 120
needs:
- build-dev
- integration-cli-prepare
strategy:
fail-fast: false
matrix:
test: ${{ fromJson(needs.integration-cli-prepare.outputs.matrix) }}
steps:
-
name: Checkout
uses: actions/checkout@v3
-
name: Set up runner
uses: ./.github/actions/setup-runner
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
-
name: Build dev image
uses: docker/bake-action@v2
with:
targets: dev
set: |
dev.cache-from=type=gha,scope=dev
-
name: Test
run: |
make -o build test-integration
env:
TEST_SKIP_INTEGRATION: 1
TESTCOVERAGE: 1
TESTFLAGS: "-test.run (${{ matrix.test }})/"
-
name: Prepare reports
if: always()
run: |
reportsPath=/tmp/reports/$(echo -n "${{ matrix.test }}" | sha256sum | cut -d " " -f 1)
mkdir -p bundles $reportsPath
echo "${{ matrix.test }}" | tr -s '|' '\n' | tee -a "$reportsPath/tests.txt"
find bundles -path '*/root/*overlay2' -prune -o -type f \( -name '*-report.json' -o -name '*.log' -o -name '*.out' -o -name '*.prof' -o -name '*-report.xml' \) -print | xargs sudo tar -czf /tmp/reports.tar.gz
tar -xzf /tmp/reports.tar.gz -C $reportsPath
sudo chown -R $(id -u):$(id -g) $reportsPath
tree -nh $reportsPath
-
name: Send to Codecov
uses: codecov/codecov-action@v3
with:
directory: ./bundles/test-integration
env_vars: RUNNER_OS
flags: integration-cli
-
name: Test daemon logs
if: always()
run: |
cat bundles/test-integration/docker.log
-
name: Upload reports
if: always()
uses: actions/upload-artifact@v3
with:
name: integration-cli-reports
path: /tmp/reports/*
integration-cli-report:
runs-on: ubuntu-20.04
timeout-minutes: 10
if: always()
needs:
- integration-cli
steps:
-
name: Set up Go
uses: actions/setup-go@v3
with:
go-version: ${{ env.GO_VERSION }}
-
name: Download reports
uses: actions/download-artifact@v3
with:
name: integration-cli-reports
path: /tmp/reports
-
name: Install teststat
run: |
go install github.com/vearutop/teststat@${{ env.TESTSTAT_VERSION }}
-
name: Create summary
run: |
teststat -markdown $(find /tmp/reports -type f -name '*.json' -print0 | xargs -0) >> $GITHUB_STEP_SUMMARY
prepare-smoke:
runs-on: ubuntu-20.04
needs:
- validate-dco
outputs:
matrix: ${{ steps.platforms.outputs.matrix }}
steps:
-
name: Checkout
uses: actions/checkout@v3
-
name: Create matrix
id: platforms
run: |
matrix="$(docker buildx bake binary-smoketest --print | jq -cr '.target."binary-smoketest".platforms')"
echo "matrix=$matrix" >> $GITHUB_OUTPUT
-
name: Show matrix
run: |
echo ${{ steps.platforms.outputs.matrix }}
smoke:
runs-on: ubuntu-20.04
needs:
- prepare-smoke
strategy:
fail-fast: false
matrix:
platform: ${{ fromJson(needs.prepare-smoke.outputs.matrix) }}
steps:
-
name: Checkout
uses: actions/checkout@v3
-
name: Prepare
run: |
platform=${{ matrix.platform }}
echo "PLATFORM_PAIR=${platform//\//-}" >> $GITHUB_ENV
-
name: Set up QEMU
uses: docker/setup-qemu-action@v2
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
-
name: Test
uses: docker/bake-action@v2
with:
targets: binary-smoketest
set: |
*.platform=${{ matrix.platform }}

22
.github/workflows/windows-2019.yml vendored Normal file
View file

@ -0,0 +1,22 @@
name: windows-2019
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
on:
schedule:
- cron: '0 10 * * *'
workflow_dispatch:
jobs:
validate-dco:
uses: ./.github/workflows/.dco.yml
run:
needs:
- validate-dco
uses: ./.github/workflows/.windows.yml
with:
os: windows-2019
send_coverage: false

25
.github/workflows/windows-2022.yml vendored Normal file
View file

@ -0,0 +1,25 @@
name: windows-2022
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
on:
workflow_dispatch:
push:
branches:
- 'master'
- '[0-9]+.[0-9]+'
pull_request:
jobs:
validate-dco:
uses: ./.github/workflows/.dco.yml
run:
needs:
- validate-dco
uses: ./.github/workflows/.windows.yml
with:
os: windows-2022
send_coverage: true

43
.gitignore vendored
View file

@ -1,27 +1,28 @@
# Docker project generated files to ignore
# if you want to ignore files created by your editor/tools,
# please consider a global .gitignore https://help.github.com/articles/ignoring-files
*.exe
*.exe~
*.gz
# If you want to ignore files created by your editor/tools, please consider a
# [global .gitignore](https://help.github.com/articles/ignoring-files).
*~
*.bak
*.orig
test.main
.*.swp
.DS_Store
# a .bashrc may be added to customize the build environment
thumbs.db
# local repository customization
.envrc
.bashrc
.editorconfig
.gopath/
.go-pkg-cache/
bundles/
cli/winresources/**/winres.json
cli/winresources/**/*.syso
cmd/dockerd/dockerd
contrib/builder/rpm/*/changelog
vendor/pkg/
go-test-report.json
profile.out
junit-report.xml
# top-level go.mod is not meant to be checked in
/go.mod
# build artifacts
bundles/
cli/winresources/*/*.syso
cli/winresources/*/winres.json
contrib/builder/rpm/*/changelog
# ci artifacts
*.exe
*.gz
go-test-report.json
junit-report.xml
profile.out
test.main

View file

@ -1,14 +1,14 @@
# Generate AUTHORS: hack/generate-authors.sh
# Tip for finding duplicates (besides scanning the output of AUTHORS for name
# duplicates that aren't also email duplicates): scan the output of:
# git log --format='%aE - %aN' | sort -uf
# This file lists the canonical name and email of contributors, and is used to
# generate AUTHORS (in hack/generate-authors.sh).
#
# For explanation on this file format: man git-shortlog
# To find new duplicates, regenerate AUTHORS and scan for name duplicates, or
# run the following to find email duplicates:
# git log --format='%aE - %aN' | sort -uf | awk -v IGNORECASE=1 '$1 in a {print a[$1]; print}; {a[$1]=$0}'
#
# For an explanation of this file format, consult gitmailmap(5).
<21551195@zju.edu.cn> <hsinko@users.noreply.github.com>
<mr.wrfly@gmail.com> <wrfly@users.noreply.github.com>
Aaron L. Xu <liker.xu@foxmail.com>
Aaron L. Xu <liker.xu@foxmail.com> <likexu@harmonycloud.cn>
Aaron Lehmann <alehmann@netflix.com>
Aaron Lehmann <alehmann@netflix.com> <aaron.lehmann@docker.com>
Abhinandan Prativadi <aprativadi@gmail.com>
@ -37,7 +37,6 @@ Aleksa Sarai <asarai@suse.de> <asarai@suse.com>
Aleksa Sarai <asarai@suse.de> <cyphar@cyphar.com>
Aleksandrs Fadins <aleks@s-ko.net>
Alessandro Boch <aboch@tetrationanalytics.com>
Alessandro Boch <aboch@tetrationanalytics.com>
Alessandro Boch <aboch@tetrationanalytics.com> <aboch@docker.com>
Alessandro Boch <aboch@tetrationanalytics.com> <aboch@socketplane.io>
Alessandro Boch <aboch@tetrationanalytics.com> <aboch@users.noreply.github.com>
@ -50,6 +49,7 @@ Alexander Larsson <alexl@redhat.com> <alexander.larsson@gmail.com>
Alexander Morozov <lk4d4math@gmail.com>
Alexander Morozov <lk4d4math@gmail.com> <lk4d4@docker.com>
Alexandre Beslic <alexandre.beslic@gmail.com> <abronan@docker.com>
Alexandre González <agonzalezro@gmail.com>
Alexis Ries <ries.alexis@gmail.com>
Alexis Ries <ries.alexis@gmail.com> <alexis.ries.ext@orange.com>
Alexis Thomas <fr.alexisthomas@gmail.com>
@ -67,6 +67,8 @@ Andrey Kolomentsev <andrey.kolomentsev@docker.com> <andrey.kolomentsev@gmail.com
André Martins <aanm90@gmail.com> <martins@noironetworks.com>
Andy Rothfusz <github@developersupport.net> <github@metaliveblog.com>
Andy Smith <github@anarkystic.com>
Andy Zhang <andy.zhangtao@hotmail.com>
Andy Zhang <andy.zhangtao@hotmail.com> <ztao@tibco-support.com>
Ankush Agarwal <ankushagarwal11@gmail.com> <ankushagarwal@users.noreply.github.com>
Antonio Murdaca <antonio.murdaca@gmail.com> <amurdaca@redhat.com>
Antonio Murdaca <antonio.murdaca@gmail.com> <me@runcom.ninja>
@ -85,6 +87,7 @@ Arnaud Porterie <icecrime@gmail.com> <arnaud.porterie@docker.com>
Arnaud Rebillout <arnaud.rebillout@collabora.com>
Arnaud Rebillout <arnaud.rebillout@collabora.com> <elboulangero@gmail.com>
Arthur Gautier <baloo@gandi.net> <superbaloo+registrations.github@superbaloo.net>
Artur Meyster <arthurfbi@yahoo.com>
Avi Miller <avi.miller@oracle.com> <avi.miller@gmail.com>
Ben Bonnefoy <frenchben@docker.com>
Ben Golub <ben.golub@dotcloud.com>
@ -101,7 +104,9 @@ Bin Liu <liubin0329@gmail.com>
Bin Liu <liubin0329@gmail.com> <liubin0329@users.noreply.github.com>
Bingshen Wang <bingshen.wbs@alibaba-inc.com>
Boaz Shuster <ripcurld.github@gmail.com>
Bojun Zhu <bojun.zhu@foxmail.com>
Boqin Qin <bobbqqin@gmail.com>
Boshi Lian <farmer1992@gmail.com>
Brandon Philips <brandon.philips@coreos.com> <brandon@ifup.co>
Brandon Philips <brandon.philips@coreos.com> <brandon@ifup.org>
Brent Salisbury <brent.salisbury@docker.com> <brent@docker.com>
@ -240,31 +245,36 @@ Gurjeet Singh <gurjeet@singh.im> <singh.gurjeet@gmail.com>
Gustav Sinder <gustav.sinder@gmail.com>
Günther Jungbluth <gunther@gameslabs.net>
Hakan Özler <hakan.ozler@kodcu.com>
Hao Shu Wei <haosw@cn.ibm.com>
Hao Shu Wei <haosw@cn.ibm.com> <haoshuwei1989@163.com>
Hao Shu Wei <haoshuwei24@gmail.com>
Hao Shu Wei <haoshuwei24@gmail.com> <haoshuwei1989@163.com>
Hao Shu Wei <haoshuwei24@gmail.com> <haosw@cn.ibm.com>
Harald Albers <github@albersweb.de> <albers@users.noreply.github.com>
Harald Niesche <harald@niesche.de>
Harold Cooper <hrldcpr@gmail.com>
Harry Zhang <harryz@hyper.sh>
Harry Zhang <harryz@hyper.sh> <harryzhang@zju.edu.cn>
Harry Zhang <harryz@hyper.sh> <resouer@163.com>
Harry Zhang <harryz@hyper.sh> <resouer@gmail.com>
Harry Zhang <resouer@163.com>
Harshal Patil <harshal.patil@in.ibm.com> <harche@users.noreply.github.com>
He Simei <hesimei@zju.edu.cn>
Helen Xie <chenjg@harmonycloud.cn>
Hiroyuki Sasagawa <hs19870702@gmail.com>
Hollie Teal <hollie@docker.com>
Hollie Teal <hollie@docker.com> <hollie.teal@docker.com>
Hollie Teal <hollie@docker.com> <hollietealok@users.noreply.github.com>
hsinko <21551195@zju.edu.cn> <hsinko@users.noreply.github.com>
Hu Keping <hukeping@huawei.com>
Hui Kang <hkang.sunysb@gmail.com>
Hui Kang <hkang.sunysb@gmail.com> <kangh@us.ibm.com>
Huu Nguyen <huu@prismskylabs.com> <whoshuu@gmail.com>
Hyeongkyu Lee <hyeongkyu.lee@navercorp.com>
Hyzhou Zhy <hyzhou.zhy@alibaba-inc.com>
Hyzhou Zhy <hyzhou.zhy@alibaba-inc.com> <1187766782@qq.com>
Ian Campbell <ian.campbell@docker.com>
Ian Campbell <ian.campbell@docker.com> <ijc@docker.com>
Ilya Khlopotov <ilya.khlopotov@gmail.com>
Iskander Sharipov <quasilyte@gmail.com>
Ivan Babrou <ibobrik@gmail.com>
Ivan Markin <sw@nogoegst.net> <twim@riseup.net>
Jack Laxson <jackjrabbit@gmail.com>
Jacob Atzen <jacob@jacobatzen.dk> <jatzen@gmail.com>
@ -276,6 +286,7 @@ Jakub Drahos <jdrahos@pulsepoint.com> <jack.drahos@gmail.com>
James Nesbitt <jnesbitt@mirantis.com>
James Nesbitt <jnesbitt@mirantis.com> <james.nesbitt@wunderkraut.com>
Jamie Hannaford <jamie@limetree.org> <jamie.hannaford@rackspace.com>
Jan Götte <jaseg@jaseg.net>
Jana Radhakrishnan <mrjana@docker.com>
Jana Radhakrishnan <mrjana@docker.com> <mrjana@socketplane.io>
Javier Bassi <javierbassi@gmail.com>
@ -315,8 +326,8 @@ John Howard <github@lowenna.com> <10522484+lowenna@users.noreply.github.com>
John Howard <github@lowenna.com> <jhoward@microsoft.com>
John Howard <github@lowenna.com> <jhoward@ntdev.microsoft.com>
John Howard <github@lowenna.com> <jhowardmsft@users.noreply.github.com>
John Howard <github@lowenna.com> <John.Howard@microsoft.com>
John Howard <github@lowenna.com> <john.howard@microsoft.com>
John Howard <github@lowenna.com> <john@lowenna.com>
John Stephens <johnstep@docker.com> <johnstep@users.noreply.github.com>
Jon Surrell <jon.surrell@gmail.com> <jon.surrell@automattic.com>
Jonathan Choy <jonathan.j.choy@gmail.com>
@ -466,6 +477,7 @@ Mohit Soni <mosoni@ebay.com> <mohitsoni1989@gmail.com>
Moorthy RS <rsmoorthy@gmail.com> <rsmoorthy@users.noreply.github.com>
Moysés Borges <moysesb@gmail.com>
Moysés Borges <moysesb@gmail.com> <moyses.furtado@wplex.com.br>
mrfly <mr.wrfly@gmail.com> <wrfly@users.noreply.github.com>
Nace Oroz <orkica@gmail.com>
Natasha Jarus <linuxmercedes@gmail.com>
Nathan LeClaire <nathan.leclaire@docker.com> <nathan.leclaire@gmail.com>
@ -507,6 +519,7 @@ Qiang Huang <h.huangqiang@huawei.com> <qhuang@10.0.2.15>
Qin TianHuan <tianhuan@bingotree.cn>
Ray Tsang <rayt@google.com> <saturnism@users.noreply.github.com>
Renaud Gaubert <rgaubert@nvidia.com> <renaud.gaubert@gmail.com>
Richard Scothern <richard.scothern@gmail.com>
Robert Terhaar <rterhaar@atlanticdynamic.com> <robbyt@users.noreply.github.com>
Roberto G. Hashioka <roberto.hashioka@docker.com> <roberto_hashioka@hotmail.com>
Roberto Muñoz Fernández <robertomf@gmail.com> <roberto.munoz.fernandez.contractor@bbva.com>
@ -529,11 +542,13 @@ Sandeep Bansal <sabansal@microsoft.com>
Sandeep Bansal <sabansal@microsoft.com> <msabansal@microsoft.com>
Santhosh Manohar <santhosh@docker.com>
Sargun Dhillon <sargun@netflix.com> <sargun@sargun.me>
Satoshi Tagomori <tagomoris@gmail.com>
Sean Lee <seanlee@tw.ibm.com> <scaleoutsean@users.noreply.github.com>
Sebastiaan van Stijn <github@gone.nl>
Sebastiaan van Stijn <github@gone.nl> <moby@example.com>
Sebastiaan van Stijn <github@gone.nl> <sebastiaan@ws-key-sebas3.dpi1.dpi>
Sebastiaan van Stijn <github@gone.nl> <thaJeztah@users.noreply.github.com>
Seongyeol Lim <seongyeol37@gmail.com>
Shaun Kaasten <shaunk@gmail.com>
Shawn Landden <shawn@churchofgit.com> <shawnlandden@gmail.com>
Shengbo Song <thomassong@tencent.com>
@ -542,8 +557,6 @@ Shih-Yuan Lee <fourdollars@gmail.com>
Shishir Mahajan <shishir.mahajan@redhat.com> <smahajan@redhat.com>
Shu-Wai Chow <shu-wai.chow@seattlechildrens.org>
Shukui Yang <yangshukui@huawei.com>
Shuwei Hao <haosw@cn.ibm.com>
Shuwei Hao <haosw@cn.ibm.com> <haoshuwei24@gmail.com>
Sidhartha Mani <sidharthamn@gmail.com>
Sjoerd Langkemper <sjoerd-github@linuxonly.nl> <sjoerd@byte.nl>
Smark Meng <smark@freecoop.net>
@ -582,6 +595,7 @@ Sylvain Baubeau <lebauce@gmail.com>
Sylvain Baubeau <lebauce@gmail.com> <sbaubeau@redhat.com>
Sylvain Bellemare <sylvain@ascribe.io>
Sylvain Bellemare <sylvain@ascribe.io> <sylvain.bellemare@ezeep.com>
Takuto Sato <tockn.jp@gmail.com>
Tangi Colin <tangicolin@gmail.com>
Tejesh Mehta <tejesh.mehta@gmail.com> <tj@init.me>
Terry Chu <zue.hterry@gmail.com>
@ -662,6 +676,7 @@ Wang Yuexiao <wang.yuexiao@zte.com.cn>
Wayne Chang <wayne@neverfear.org>
Wayne Song <wsong@docker.com> <wsong@users.noreply.github.com>
Wei Wu <wuwei4455@gmail.com> cizixs <cizixs@163.com>
Wei-Ting Kuo <waitingkuo0527@gmail.com>
Wen Cheng Ma <wenchma@cn.ibm.com>
Wenjun Tang <tangwj2@lenovo.com> <dodia@163.com>
Wewang Xiaorenfine <wang.xiaoren@zte.com.cn>
@ -696,12 +711,15 @@ Yu Changchun <yuchangchun1@huawei.com>
Yu Chengxia <yuchengxia@huawei.com>
Yu Peng <yu.peng36@zte.com.cn>
Yu Peng <yu.peng36@zte.com.cn> <yupeng36@zte.com.cn>
Yuan Sun <sunyuan3@huawei.com>
Yue Zhang <zy675793960@yeah.net>
Yufei Xiong <yufei.xiong@qq.com>
Zach Gershman <zachgersh@gmail.com>
Zach Gershman <zachgersh@gmail.com> <zachgersh@users.noreply.github.com>
Zachary Jaffee <zjaffee@us.ibm.com> <zij@case.edu>
Zachary Jaffee <zjaffee@us.ibm.com> <zjaffee@apache.org>
Zhang Kun <zkazure@gmail.com>
Zhang Wentao <zhangwentao234@huawei.com>
ZhangHang <stevezhang2014@gmail.com>
Zhenkun Bi <bi.zhenkun@zte.com.cn>
Zhou Hao <zhouhao@cn.fujitsu.com>

45
AUTHORS
View file

@ -1,5 +1,6 @@
# This file lists all individuals having contributed content to the repository.
# For how it is generated, see `hack/generate-authors.sh`.
# File @generated by hack/generate-authors.sh. DO NOT EDIT.
# This file lists all contributors to the repository.
# See hack/generate-authors.sh to make modifications.
Aanand Prasad <aanand.prasad@gmail.com>
Aaron Davidson <aaron@databricks.com>
@ -9,7 +10,6 @@ Aaron Huslage <huslage@gmail.com>
Aaron L. Xu <liker.xu@foxmail.com>
Aaron Lehmann <alehmann@netflix.com>
Aaron Welch <welch@packet.net>
Aaron.L.Xu <likexu@harmonycloud.cn>
Abel Muiño <amuino@gmail.com>
Abhijeet Kasurde <akasurde@redhat.com>
Abhinandan Prativadi <aprativadi@gmail.com>
@ -17,6 +17,7 @@ Abhinav Ajgaonkar <abhinav316@gmail.com>
Abhishek Chanda <abhishek.becs@gmail.com>
Abhishek Sharma <abhishek@asharma.me>
Abin Shahab <ashahab@altiscale.com>
Abirdcfly <fp544037857@gmail.com>
Ada Mancini <ada@docker.com>
Adam Avilla <aavilla@yp.com>
Adam Dobrawy <naczelnik@jawnosc.tk>
@ -161,7 +162,6 @@ Andrey Kolomentsev <andrey.kolomentsev@docker.com>
Andrey Petrov <andrey.petrov@shazow.net>
Andrey Stolbovsky <andrey.stolbovsky@gmail.com>
André Martins <aanm90@gmail.com>
andy <ztao@tibco-support.com>
Andy Chambers <anchambers@paypal.com>
andy diller <dillera@gmail.com>
Andy Goldstein <agoldste@redhat.com>
@ -170,6 +170,7 @@ Andy Lindeman <alindeman@salesforce.com>
Andy Rothfusz <github@developersupport.net>
Andy Smith <github@anarkystic.com>
Andy Wilson <wilson.andrew.j+github@gmail.com>
Andy Zhang <andy.zhangtao@hotmail.com>
Anes Hasicic <anes.hasicic@gmail.com>
Angel Velazquez <angelcar@amazon.com>
Anil Belur <askb23@gmail.com>
@ -209,6 +210,7 @@ Artur Meyster <arthurfbi@yahoo.com>
Arun Gupta <arun.gupta@gmail.com>
Asad Saeeduddin <masaeedu@gmail.com>
Asbjørn Enge <asbjorn@hanafjedle.net>
Austin Vazquez <macedonv@amazon.com>
averagehuman <averagehuman@users.noreply.github.com>
Avi Das <andas222@gmail.com>
Avi Kivity <avi@scylladb.com>
@ -222,6 +224,7 @@ Barnaby Gray <barnaby@pickle.me.uk>
Barry Allard <barry.allard@gmail.com>
Bartłomiej Piotrowski <b@bpiotrowski.pl>
Bastiaan Bakker <bbakker@xebia.com>
Bastien Pascard <bpascard@hotmail.com>
bdevloed <boris.de.vloed@gmail.com>
Bearice Ren <bearice@gmail.com>
Ben Bonnefoy <frenchben@docker.com>
@ -229,6 +232,7 @@ Ben Firshman <ben@firshman.co.uk>
Ben Golub <ben.golub@dotcloud.com>
Ben Gould <ben@bengould.co.uk>
Ben Hall <ben@benhall.me.uk>
Ben Langfeld <ben@langfeld.me>
Ben Sargent <ben@brokendigits.com>
Ben Severson <BenSeverson@users.noreply.github.com>
Ben Toews <mastahyeti@gmail.com>
@ -258,6 +262,7 @@ Bjorn Neergaard <bneergaard@mirantis.com>
Blake Geno <blakegeno@gmail.com>
Boaz Shuster <ripcurld.github@gmail.com>
bobby abbott <ttobbaybbob@gmail.com>
Bojun Zhu <bojun.zhu@foxmail.com>
Boqin Qin <bobbqqin@gmail.com>
Boris Pruessmann <boris@pruessmann.org>
Boshi Lian <farmer1992@gmail.com>
@ -339,6 +344,7 @@ Charlie Drage <charlie@charliedrage.com>
Charlie Lewis <charliel@lab41.org>
Chase Bolt <chase.bolt@gmail.com>
ChaYoung You <yousbe@gmail.com>
Chee Hau Lim <ch33hau@gmail.com>
Chen Chao <cc272309126@gmail.com>
Chen Chuanliang <chen.chuanliang@zte.com.cn>
Chen Hanxiao <chenhanxiao@cn.fujitsu.com>
@ -545,7 +551,6 @@ Derek Ch <denc716@gmail.com>
Derek McGowan <derek@mcg.dev>
Deric Crago <deric.crago@gmail.com>
Deshi Xiao <dxiao@redhat.com>
devmeyster <arthurfbi@yahoo.com>
Devon Estes <devon.estes@klarna.com>
Devvyn Murphy <devvyn@devvyn.com>
Dharmit Shah <shahdharmit@gmail.com>
@ -650,6 +655,7 @@ Erik Dubbelboer <erik@dubbelboer.com>
Erik Hollensbe <github@hollensbe.org>
Erik Inge Bolsø <knan@redpill-linpro.com>
Erik Kristensen <erik@erikkristensen.com>
Erik Sipsma <erik@sipsma.dev>
Erik St. Martin <alakriti@gmail.com>
Erik Weathers <erikdw@gmail.com>
Erno Hopearuoho <erno.hopearuoho@gmail.com>
@ -707,6 +713,7 @@ Fengtu Wang <wangfengtu@huawei.com>
Ferenc Szabo <pragmaticfrank@gmail.com>
Fernando <fermayo@gmail.com>
Fero Volar <alian@alian.info>
Feroz Salam <feroz.salam@sourcegraph.com>
Ferran Rodenas <frodenas@gmail.com>
Filipe Brandenburger <filbranden@google.com>
Filipe Oliveira <contato@fmoliveira.com.br>
@ -822,7 +829,7 @@ Hamish Hutchings <moredhel@aoeu.me>
Hannes Ljungberg <hannes@5monkeys.se>
Hans Kristian Flaatten <hans@starefossen.com>
Hans Rødtang <hansrodtang@gmail.com>
Hao Shu Wei <haosw@cn.ibm.com>
Hao Shu Wei <haoshuwei24@gmail.com>
Hao Zhang <21521210@zju.edu.cn>
Harald Albers <github@albersweb.de>
Harald Niesche <harald@niesche.de>
@ -861,10 +868,9 @@ Hui Kang <hkang.sunysb@gmail.com>
Hunter Blanks <hunter@twilio.com>
huqun <huqun@zju.edu.cn>
Huu Nguyen <huu@prismskylabs.com>
hyeongkyu.lee <hyeongkyu.lee@navercorp.com>
Hyeongkyu Lee <hyeongkyu.lee@navercorp.com>
Hyzhou Zhy <hyzhou.zhy@alibaba-inc.com>
Iago López Galeiras <iago@kinvolk.io>
Ian Babrou <ibobrik@gmail.com>
Ian Bishop <ianbishop@pace7.com>
Ian Bull <irbull@gmail.com>
Ian Calvert <ianjcalvert@gmail.com>
@ -881,6 +887,7 @@ Igor Dolzhikov <bluesriverz@gmail.com>
Igor Karpovich <i.karpovich@currencysolutions.com>
Iliana Weller <iweller@amazon.com>
Ilkka Laukkanen <ilkka@ilkka.io>
Illo Abdulrahim <abdulrahim.illo@nokia.com>
Ilya Dmitrichenko <errordeveloper@gmail.com>
Ilya Gusev <mail@igusev.ru>
Ilya Khlopotov <ilya.khlopotov@gmail.com>
@ -931,6 +938,7 @@ Jamie Hannaford <jamie@limetree.org>
Jamshid Afshar <jafshar@yahoo.com>
Jan Breig <git@pygos.space>
Jan Chren <dev.rindeal@gmail.com>
Jan Götte <jaseg@jaseg.net>
Jan Keromnes <janx@linux.com>
Jan Koprowski <jan.koprowski@gmail.com>
Jan Pazdziora <jpazdziora@redhat.com>
@ -943,7 +951,6 @@ Januar Wayong <januar@gmail.com>
Jared Biel <jared.biel@bolderthinking.com>
Jared Hocutt <jaredh@netapp.com>
Jaroslaw Zabiello <hipertracker@gmail.com>
jaseg <jaseg@jaseg.net>
Jasmine Hegman <jasmine@jhegman.com>
Jason A. Donenfeld <Jason@zx2c4.com>
Jason Divock <jdivock@gmail.com>
@ -1214,7 +1221,6 @@ Kris-Mikael Krister <krismikael@protonmail.com>
Kristian Haugene <kristian.haugene@capgemini.com>
Kristina Zabunova <triara.xiii@gmail.com>
Krystian Wojcicki <kwojcicki@sympatico.ca>
Kun Zhang <zkazure@gmail.com>
Kunal Kushwaha <kushwaha_kunal_v7@lab.ntt.co.jp>
Kunal Tyagi <tyagi.kunal@live.com>
Kyle Conroy <kyle.j.conroy@gmail.com>
@ -1242,7 +1248,6 @@ Leandro Siqueira <leandro.siqueira@gmail.com>
Lee Calcote <leecalcote@gmail.com>
Lee Chao <932819864@qq.com>
Lee, Meng-Han <sunrisedm4@gmail.com>
leeplay <hyeongkyu.lee@navercorp.com>
Lei Gong <lgong@alauda.io>
Lei Jitang <leijitang@huawei.com>
Leiiwang <u2takey@gmail.com>
@ -1269,7 +1274,6 @@ Lifubang <lifubang@acmcoder.com>
Lihua Tang <lhtang@alauda.io>
Lily Guo <lily.guo@docker.com>
limeidan <limeidan@loongson.cn>
limsy <seongyeol37@gmail.com>
Lin Lu <doraalin@163.com>
LingFaKe <lingfake@huawei.com>
Linus Heckemann <lheckemann@twig-world.com>
@ -1299,6 +1303,7 @@ Lucas Chi <lucas@teacherspayteachers.com>
Lucas Molas <lmolas@fundacionsadosky.org.ar>
Lucas Silvestre <lukas.silvestre@gmail.com>
Luciano Mores <leslau@gmail.com>
Luis Henrique Mulinari <luis.mulinari@gmail.com>
Luis Martínez de Bartolomé Izquierdo <lmartinez@biicode.com>
Luiz Svoboda <luizek@gmail.com>
Lukas Heeren <lukas-heeren@hotmail.com>
@ -1347,6 +1352,7 @@ Marius Gundersen <me@mariusgundersen.net>
Marius Sturm <marius@graylog.com>
Marius Voila <marius.voila@gmail.com>
Mark Allen <mrallen1@yahoo.com>
Mark Feit <mfeit@internet2.edu>
Mark Jeromin <mark.jeromin@sysfrog.net>
Mark McGranaghan <mmcgrana@gmail.com>
Mark McKinstry <mmckinst@umich.edu>
@ -1362,6 +1368,7 @@ Markus Fix <lispmeister@gmail.com>
Markus Kortlang <hyp3rdino@googlemail.com>
Martijn Dwars <ikben@martijndwars.nl>
Martijn van Oosterhout <kleptog@svana.org>
Martin Braun <braun@neuroforge.de>
Martin Dojcak <martin.dojcak@lablabs.io>
Martin Honermeyer <maze@strahlungsfrei.de>
Martin Kelly <martin@surround.io>
@ -1678,6 +1685,7 @@ Petr Švihlík <svihlik.petr@gmail.com>
Petros Angelatos <petrosagg@gmail.com>
Phil <underscorephil@gmail.com>
Phil Estes <estesp@gmail.com>
Phil Sphicas <phil.sphicas@att.com>
Phil Spitler <pspitler@gmail.com>
Philip Alexander Etling <paetling@gmail.com>
Philip Monroe <phil@philmonroe.com>
@ -1749,7 +1757,6 @@ Ricardo N Feliciano <FelicianoTech@gmail.com>
Rich Horwood <rjhorwood@apple.com>
Rich Moyse <rich@moyse.us>
Rich Seymour <rseymour@gmail.com>
Richard <richard.scothern@gmail.com>
Richard Burnison <rburnison@ebay.com>
Richard Harvey <richard@squarecows.com>
Richard Mathie <richard.mathie@amey.co.uk>
@ -1848,7 +1855,6 @@ Ryo Nakao <nakabonne@gmail.com>
Ryoga Saito <contact@proelbtn.com>
Rémy Greinhofer <remy.greinhofer@livelovely.com>
s. rannou <mxs@sbrk.org>
s00318865 <sunyuan3@huawei.com>
Sabin Basyal <sabin.basyal@gmail.com>
Sachin Joshi <sachin_jayant_joshi@hotmail.com>
Sagar Hani <sagarhani33@gmail.com>
@ -1938,7 +1944,6 @@ Shourya Sarcar <shourya.sarcar@gmail.com>
Shu-Wai Chow <shu-wai.chow@seattlechildrens.org>
shuai-z <zs.broccoli@gmail.com>
Shukui Yang <yangshukui@huawei.com>
Shuwei Hao <haosw@cn.ibm.com>
Sian Lerk Lau <kiawin@gmail.com>
Siarhei Rasiukevich <s_rasiukevich@wargaming.net>
Sidhartha Mani <sidharthamn@gmail.com>
@ -1946,7 +1951,6 @@ sidharthamani <sid@rancher.com>
Silas Sewell <silas@sewell.org>
Silvan Jegen <s.jegen@gmail.com>
Simão Reis <smnrsti@gmail.com>
Simei He <hesimei@zju.edu.cn>
Simon Barendse <simon.barendse@gmail.com>
Simon Eskildsen <sirup@sirupsen.com>
Simon Ferquel <simon.ferquel@docker.com>
@ -2022,7 +2026,7 @@ Sébastien Stormacq <sebsto@users.noreply.github.com>
Sören Tempel <soeren+git@soeren-tempel.net>
Tabakhase <mail@tabakhase.com>
Tadej Janež <tadej.j@nez.si>
TAGOMORI Satoshi <tagomoris@gmail.com>
Takuto Sato <tockn.jp@gmail.com>
tang0th <tang0th@gmx.com>
Tangi Colin <tangicolin@gmail.com>
Tatsuki Sugiura <sugi@nemui.org>
@ -2035,7 +2039,6 @@ Tejaswini Duggaraju <naduggar@microsoft.com>
Tejesh Mehta <tejesh.mehta@gmail.com>
Terry Chu <zue.hterry@gmail.com>
terryding77 <550147740@qq.com>
tgic <farmer1992@gmail.com>
Thatcher Peskens <thatcher@docker.com>
theadactyl <thea.lamkin@gmail.com>
Thell 'Bo' Fowler <thell@tbfowler.name>
@ -2059,6 +2062,7 @@ Thomas Swift <tgs242@gmail.com>
Thomas Tanaka <thomas.tanaka@oracle.com>
Thomas Texier <sharkone@en-mousse.org>
Ti Zhou <tizhou1986@gmail.com>
Tiago Seabra <tlgs@users.noreply.github.com>
Tianon Gravi <admwiggin@gmail.com>
Tianyi Wang <capkurmagati@gmail.com>
Tibor Vass <teabee89@gmail.com>
@ -2197,7 +2201,6 @@ VladimirAus <v_roudakov@yahoo.com>
Vladislav Kolesnikov <vkolesnikov@beget.ru>
Vlastimil Zeman <vlastimil.zeman@diffblue.com>
Vojtech Vitek (V-Teq) <vvitek@redhat.com>
waitingkuo <waitingkuo0527@gmail.com>
Walter Leibbrandt <github@wrl.co.za>
Walter Stanish <walter@pratyeka.org>
Wang Chao <chao.wang@ucloud.cn>
@ -2227,7 +2230,6 @@ Wendel Fleming <wfleming@usc.edu>
Wenjun Tang <tangwj2@lenovo.com>
Wenkai Yin <yinw@vmware.com>
wenlxie <wenlxie@ebay.com>
Wentao Zhang <zhangwentao234@huawei.com>
Wenxuan Zhao <viz@linux.com>
Wenyu You <21551128@zju.edu.cn>
Wenzhi Liang <wenzhi.liang@gmail.com>
@ -2286,6 +2288,7 @@ Yang Bai <hamo.by@gmail.com>
Yang Li <idealhack@gmail.com>
Yang Pengfei <yangpengfei4@huawei.com>
yangchenliang <yangchenliang@huawei.com>
Yann Autissier <yann.autissier@gmail.com>
Yanqiang Miao <miao.yanqiang@zte.com.cn>
Yao Zaiyong <yaozaiyong@hotmail.com>
Yash Murty <yashmurty@gmail.com>
@ -2305,6 +2308,7 @@ Yosef Fertel <yfertel@gmail.com>
You-Sheng Yang (楊有勝) <vicamo@gmail.com>
youcai <omegacoleman@gmail.com>
Youcef YEKHLEF <yyekhlef@gmail.com>
Youfu Zhang <zhangyoufu@gmail.com>
Yu Changchun <yuchangchun1@huawei.com>
Yu Chengxia <yuchengxia@huawei.com>
Yu Peng <yu.peng36@zte.com.cn>
@ -2357,7 +2361,6 @@ Zou Yu <zouyu7@huawei.com>
zqh <zqhxuyuan@gmail.com>
Zuhayr Elahi <zuhayr.elahi@docker.com>
Zunayed Ali <zunayed@gmail.com>
Álex González <agonzalezro@gmail.com>
Álvaro Lázaro <alvaro.lazaro.g@gmail.com>
Átila Camurça Alves <camurca.home@gmail.com>
尹吉峰 <jifeng.yin@gmail.com>

File diff suppressed because it is too large Load diff

View file

@ -1,19 +1,35 @@
# syntax=docker/dockerfile:1
ARG CROSS="false"
ARG SYSTEMD="false"
ARG GO_VERSION=1.18.3
ARG DEBIAN_FRONTEND=noninteractive
ARG VPNKIT_VERSION=0.5.0
ARG GO_VERSION=1.20.13
ARG BASE_DEBIAN_DISTRO="bullseye"
ARG GOLANG_IMAGE="golang:${GO_VERSION}-${BASE_DEBIAN_DISTRO}"
ARG XX_VERSION=1.2.1
FROM ${GOLANG_IMAGE} AS base
ARG VPNKIT_VERSION=0.5.0
ARG DOCKERCLI_VERSION=v17.06.2-ce
ARG SYSTEMD="false"
ARG DEBIAN_FRONTEND=noninteractive
ARG DOCKER_STATIC=1
# cross compilation helper
FROM --platform=$BUILDPLATFORM tonistiigi/xx:${XX_VERSION} AS xx
# dummy stage to make sure the image is built for deps that don't support some
# architectures
FROM --platform=$BUILDPLATFORM busybox AS build-dummy
RUN mkdir -p /build
FROM scratch AS binary-dummy
COPY --from=build-dummy /build /build
# base
FROM --platform=$BUILDPLATFORM ${GOLANG_IMAGE} AS base
COPY --from=xx / /
RUN echo 'Binary::apt::APT::Keep-Downloaded-Packages "true";' > /etc/apt/apt.conf.d/keep-cache
ARG APT_MIRROR
RUN sed -ri "s/(httpredir|deb).debian.org/${APT_MIRROR:-deb.debian.org}/g" /etc/apt/sources.list \
&& sed -ri "s/(security).debian.org/${APT_MIRROR:-security.debian.org}/g" /etc/apt/sources.list
RUN test -n "$APT_MIRROR" && sed -ri "s/(httpredir|deb|security).debian.org/${APT_MIRROR}/g" /etc/apt/sources.list || true
ARG DEBIAN_FRONTEND
RUN apt-get update && apt-get install --no-install-recommends -y file
ENV GO111MODULE=off
FROM base AS criu
@ -26,54 +42,71 @@ RUN --mount=type=cache,sharing=locked,id=moby-criu-aptlib,target=/var/lib/apt \
&& apt-get install -y --no-install-recommends criu \
&& install -D /usr/sbin/criu /build/criu
# registry
FROM base AS registry-src
WORKDIR /usr/src/registry
RUN git init . && git remote add origin "https://github.com/distribution/distribution.git"
FROM base AS registry
WORKDIR /go/src/github.com/docker/distribution
# REGISTRY_VERSION specifies the version of the registry to build and install
# from the https://github.com/docker/distribution repository. This version of
# the registry is used to test both schema 1 and schema 2 manifests. Generally,
# the version specified here should match a current release.
ARG REGISTRY_VERSION=v2.3.0
# REGISTRY_VERSION_SCHEMA1 specifies the version of the registry to build and
# install from the https://github.com/docker/distribution repository. This is
# an older (pre v2.3.0) version of the registry that only supports schema1
# manifests. This version of the registry is not working on arm64, so installation
# is skipped on that architecture.
ARG REGISTRY_VERSION_SCHEMA1=v2.1.0
RUN --mount=type=cache,target=/root/.cache/go-build \
ARG TARGETPLATFORM
RUN --mount=from=registry-src,src=/usr/src/registry,rw \
--mount=type=cache,target=/root/.cache/go-build,id=registry-build-$TARGETPLATFORM \
--mount=type=cache,target=/go/pkg/mod \
--mount=type=tmpfs,target=/go/src/ \
set -x \
&& git clone https://github.com/docker/distribution.git . \
&& git checkout -q "$REGISTRY_VERSION" \
&& GOPATH="/go/src/github.com/docker/distribution/Godeps/_workspace:$GOPATH" \
go build -buildmode=pie -o /build/registry-v2 github.com/docker/distribution/cmd/registry \
&& case $(dpkg --print-architecture) in \
amd64|armhf|ppc64*|s390x) \
git checkout -q "$REGISTRY_VERSION_SCHEMA1"; \
GOPATH="/go/src/github.com/docker/distribution/Godeps/_workspace:$GOPATH"; \
go build -buildmode=pie -o /build/registry-v2-schema1 github.com/docker/distribution/cmd/registry; \
;; \
esac
FROM base AS swagger
WORKDIR $GOPATH/src/github.com/go-swagger/go-swagger
--mount=type=tmpfs,target=/go/src <<EOT
set -ex
git fetch -q --depth 1 origin "${REGISTRY_VERSION}" +refs/tags/*:refs/tags/*
git checkout -q FETCH_HEAD
export GOPATH="/go/src/github.com/docker/distribution/Godeps/_workspace:$GOPATH"
CGO_ENABLED=0 xx-go build -o /build/registry-v2 -v ./cmd/registry
xx-verify /build/registry-v2
case $TARGETPLATFORM in
linux/amd64|linux/arm/v7|linux/ppc64le|linux/s390x)
git fetch -q --depth 1 origin "${REGISTRY_VERSION_SCHEMA1}" +refs/tags/*:refs/tags/*
git checkout -q FETCH_HEAD
CGO_ENABLED=0 xx-go build -o /build/registry-v2-schema1 -v ./cmd/registry
xx-verify /build/registry-v2-schema1
;;
esac
EOT
# go-swagger
FROM base AS swagger-src
WORKDIR /usr/src/swagger
# Currently uses a fork from https://github.com/kolyshkin/go-swagger/tree/golang-1.13-fix
# TODO: move to under moby/ or fix upstream go-swagger to work for us.
RUN git init . && git remote add origin "https://github.com/kolyshkin/go-swagger.git"
# GO_SWAGGER_COMMIT specifies the version of the go-swagger binary to build and
# install. Go-swagger is used in CI for validating swagger.yaml in hack/validate/swagger-gen
#
# Currently uses a fork from https://github.com/kolyshkin/go-swagger/tree/golang-1.13-fix,
# TODO: move to under moby/ or fix upstream go-swagger to work for us.
ENV GO_SWAGGER_COMMIT c56166c036004ba7a3a321e5951ba472b9ae298c
RUN --mount=type=cache,target=/root/.cache/go-build \
--mount=type=cache,target=/go/pkg/mod \
--mount=type=tmpfs,target=/go/src/ \
set -x \
&& git clone https://github.com/kolyshkin/go-swagger.git . \
&& git checkout -q "$GO_SWAGGER_COMMIT" \
&& go build -o /build/swagger github.com/go-swagger/go-swagger/cmd/swagger
ARG GO_SWAGGER_COMMIT=c56166c036004ba7a3a321e5951ba472b9ae298c
RUN git fetch -q --depth 1 origin "${GO_SWAGGER_COMMIT}" && git checkout -q FETCH_HEAD
FROM base AS swagger
WORKDIR /go/src/github.com/go-swagger/go-swagger
ARG TARGETPLATFORM
RUN --mount=from=swagger-src,src=/usr/src/swagger,rw \
--mount=type=cache,target=/root/.cache/go-build,id=swagger-build-$TARGETPLATFORM \
--mount=type=cache,target=/go/pkg/mod \
--mount=type=tmpfs,target=/go/src/ <<EOT
set -e
xx-go build -o /build/swagger ./cmd/swagger
xx-verify /build/swagger
EOT
# frozen-images
# See also frozenImages in "testutil/environment/protect.go" (which needs to
# be updated when adding images to this list)
FROM debian:${BASE_DEBIAN_DISTRO} AS frozen-images
ARG DEBIAN_FRONTEND
RUN --mount=type=cache,sharing=locked,id=moby-frozen-images-aptlib,target=/var/lib/apt \
@ -85,88 +118,46 @@ RUN --mount=type=cache,sharing=locked,id=moby-frozen-images-aptlib,target=/var/l
# Get useful and necessary Hub images so we can "docker load" locally instead of pulling
COPY contrib/download-frozen-image-v2.sh /
ARG TARGETARCH
ARG TARGETVARIANT
RUN /download-frozen-image-v2.sh /build \
busybox:latest@sha256:95cf004f559831017cdf4628aaf1bb30133677be8702a8c5f2994629f637a209 \
busybox:glibc@sha256:1f81263701cddf6402afe9f33fca0266d9fff379e59b1748f33d3072da71ee85 \
debian:bullseye-slim@sha256:dacf278785a4daa9de07596ec739dbc07131e189942772210709c5c0777e8437 \
hello-world:latest@sha256:d58e752213a51785838f9eed2b7a498ffa1cb3aa7f946dda11af39286c3db9a9 \
arm32v7/hello-world:latest@sha256:50b8560ad574c779908da71f7ce370c0a2471c098d44d1c8f6b513c5a55eeeb1
# See also frozenImages in "testutil/environment/protect.go" (which needs to be updated when adding images to this list)
FROM base AS cross-false
FROM --platform=linux/amd64 base AS cross-true
ARG DEBIAN_FRONTEND
RUN dpkg --add-architecture arm64
RUN dpkg --add-architecture armel
RUN dpkg --add-architecture armhf
RUN dpkg --add-architecture ppc64el
RUN dpkg --add-architecture s390x
RUN --mount=type=cache,sharing=locked,id=moby-cross-true-aptlib,target=/var/lib/apt \
--mount=type=cache,sharing=locked,id=moby-cross-true-aptcache,target=/var/cache/apt \
apt-get update && apt-get install -y --no-install-recommends \
crossbuild-essential-arm64 \
crossbuild-essential-armel \
crossbuild-essential-armhf \
crossbuild-essential-ppc64el \
crossbuild-essential-s390x
FROM cross-${CROSS} AS dev-base
FROM dev-base AS runtime-dev-cross-false
ARG DEBIAN_FRONTEND
RUN --mount=type=cache,sharing=locked,id=moby-cross-false-aptlib,target=/var/lib/apt \
--mount=type=cache,sharing=locked,id=moby-cross-false-aptcache,target=/var/cache/apt \
apt-get update && apt-get install -y --no-install-recommends \
binutils-mingw-w64 \
g++-mingw-w64-x86-64 \
libapparmor-dev \
libbtrfs-dev \
libdevmapper-dev \
libseccomp-dev \
libsystemd-dev \
libudev-dev
FROM --platform=linux/amd64 runtime-dev-cross-false AS runtime-dev-cross-true
ARG DEBIAN_FRONTEND
# These crossbuild packages rely on gcc-<arch>, but this doesn't want to install
# on non-amd64 systems, so other architectures cannot crossbuild amd64.
RUN --mount=type=cache,sharing=locked,id=moby-cross-true-aptlib,target=/var/lib/apt \
--mount=type=cache,sharing=locked,id=moby-cross-true-aptcache,target=/var/cache/apt \
apt-get update && apt-get install -y --no-install-recommends \
libapparmor-dev:arm64 \
libapparmor-dev:armel \
libapparmor-dev:armhf \
libapparmor-dev:ppc64el \
libapparmor-dev:s390x \
libseccomp-dev:arm64 \
libseccomp-dev:armel \
libseccomp-dev:armhf \
libseccomp-dev:ppc64el \
libseccomp-dev:s390x
FROM runtime-dev-cross-${CROSS} AS runtime-dev
FROM base AS delve
# delve
FROM base AS delve-src
WORKDIR /usr/src/delve
RUN git init . && git remote add origin "https://github.com/go-delve/delve.git"
# DELVE_VERSION specifies the version of the Delve debugger binary
# from the https://github.com/go-delve/delve repository.
# It can be used to run Docker with a possibility of
# attaching debugger to it.
#
ARG DELVE_VERSION=v1.8.1
# Delve on Linux is currently only supported on amd64 and arm64;
ARG DELVE_VERSION=v1.20.1
RUN git fetch -q --depth 1 origin "${DELVE_VERSION}" +refs/tags/*:refs/tags/* && git checkout -q FETCH_HEAD
FROM base AS delve-build
WORKDIR /usr/src/delve
ARG TARGETPLATFORM
RUN --mount=from=delve-src,src=/usr/src/delve,rw \
--mount=type=cache,target=/root/.cache/go-build,id=delve-build-$TARGETPLATFORM \
--mount=type=cache,target=/go/pkg/mod <<EOT
set -e
GO111MODULE=on xx-go build -o /build/dlv ./cmd/dlv
xx-verify /build/dlv
EOT
# delve is currently only supported on linux/amd64 and linux/arm64;
# https://github.com/go-delve/delve/blob/v1.8.1/pkg/proc/native/support_sentinel.go#L1-L6
RUN --mount=type=cache,target=/root/.cache/go-build \
--mount=type=cache,target=/go/pkg/mod \
case $(dpkg --print-architecture) in \
amd64|arm64) \
GOBIN=/build/ GO111MODULE=on go install "github.com/go-delve/delve/cmd/dlv@${DELVE_VERSION}" \
&& /build/dlv --help \
;; \
*) \
mkdir -p /build/ \
;; \
esac
FROM binary-dummy AS delve-windows
FROM binary-dummy AS delve-linux-arm
FROM binary-dummy AS delve-linux-ppc64le
FROM binary-dummy AS delve-linux-s390x
FROM delve-build AS delve-linux-amd64
FROM delve-build AS delve-linux-arm64
FROM delve-linux-${TARGETARCH} AS delve-linux
FROM delve-${TARGETOS} AS delve
FROM base AS tomll
# GOTOML_VERSION specifies the version of the tomll binary to build and install
@ -183,85 +174,205 @@ RUN --mount=type=cache,target=/root/.cache/go-build \
FROM base AS gowinres
# GOWINRES_VERSION defines go-winres tool version
ARG GOWINRES_VERSION=v0.2.3
ARG GOWINRES_VERSION=v0.3.0
RUN --mount=type=cache,target=/root/.cache/go-build \
--mount=type=cache,target=/go/pkg/mod \
GOBIN=/build/ GO111MODULE=on go install "github.com/tc-hib/go-winres@${GOWINRES_VERSION}" \
&& /build/go-winres --help
FROM dev-base AS containerd
# containerd
FROM base AS containerd-src
WORKDIR /usr/src/containerd
RUN git init . && git remote add origin "https://github.com/containerd/containerd.git"
# CONTAINERD_VERSION is used to build containerd binaries, and used for the
# integration tests. The distributed docker .deb and .rpm packages depend on a
# separate (containerd.io) package, which may be a different version as is
# specified here. The containerd golang package is also pinned in vendor.mod.
# When updating the binary version you may also need to update the vendor
# version to pick up bug fixes or new APIs, however, usually the Go packages
# are built from a commit from the master branch.
ARG CONTAINERD_VERSION=v1.6.28
RUN git fetch -q --depth 1 origin "${CONTAINERD_VERSION}" +refs/tags/*:refs/tags/* && git checkout -q FETCH_HEAD
FROM base AS containerd-build
WORKDIR /go/src/github.com/containerd/containerd
ARG DEBIAN_FRONTEND
ARG TARGETPLATFORM
RUN --mount=type=cache,sharing=locked,id=moby-containerd-aptlib,target=/var/lib/apt \
--mount=type=cache,sharing=locked,id=moby-containerd-aptcache,target=/var/cache/apt \
apt-get update && apt-get install -y --no-install-recommends \
libbtrfs-dev
ARG CONTAINERD_VERSION
COPY /hack/dockerfile/install/install.sh /hack/dockerfile/install/containerd.installer /
RUN --mount=type=cache,target=/root/.cache/go-build \
--mount=type=cache,target=/go/pkg/mod \
PREFIX=/build /install.sh containerd
apt-get update && xx-apt-get install -y --no-install-recommends \
gcc libbtrfs-dev libsecret-1-dev
ARG DOCKER_STATIC
RUN --mount=from=containerd-src,src=/usr/src/containerd,rw \
--mount=type=cache,target=/root/.cache/go-build,id=containerd-build-$TARGETPLATFORM <<EOT
set -e
export CC=$(xx-info)-gcc
export CGO_ENABLED=$([ "$DOCKER_STATIC" = "1" ] && echo "0" || echo "1")
xx-go --wrap
make $([ "$DOCKER_STATIC" = "1" ] && echo "STATIC=1") binaries
xx-verify $([ "$DOCKER_STATIC" = "1" ] && echo "--static") bin/containerd
xx-verify $([ "$DOCKER_STATIC" = "1" ] && echo "--static") bin/containerd-shim-runc-v2
xx-verify $([ "$DOCKER_STATIC" = "1" ] && echo "--static") bin/ctr
mkdir /build
mv bin/containerd bin/containerd-shim-runc-v2 bin/ctr /build
EOT
FROM containerd-build AS containerd-linux
FROM binary-dummy AS containerd-windows
FROM containerd-${TARGETOS} AS containerd
FROM base AS golangci_lint
ARG GOLANGCI_LINT_VERSION=v1.46.2
ARG GOLANGCI_LINT_VERSION=v1.55.2
RUN --mount=type=cache,target=/root/.cache/go-build \
--mount=type=cache,target=/go/pkg/mod \
GOBIN=/build/ GO111MODULE=on go install "github.com/golangci/golangci-lint/cmd/golangci-lint@${GOLANGCI_LINT_VERSION}" \
&& /build/golangci-lint --version
FROM base AS gotestsum
ARG GOTESTSUM_VERSION=v1.8.1
ARG GOTESTSUM_VERSION=v1.8.2
RUN --mount=type=cache,target=/root/.cache/go-build \
--mount=type=cache,target=/go/pkg/mod \
GOBIN=/build/ GO111MODULE=on go install "gotest.tools/gotestsum@${GOTESTSUM_VERSION}" \
&& /build/gotestsum --version
FROM base AS shfmt
ARG SHFMT_VERSION=v3.0.2
ARG SHFMT_VERSION=v3.6.0
RUN --mount=type=cache,target=/root/.cache/go-build \
--mount=type=cache,target=/go/pkg/mod \
GOBIN=/build/ GO111MODULE=on go install "mvdan.cc/sh/v3/cmd/shfmt@${SHFMT_VERSION}" \
&& /build/shfmt --version
FROM dev-base AS dockercli
ARG DOCKERCLI_CHANNEL
# dockercli
FROM base AS dockercli-src
WORKDIR /tmp/dockercli
RUN git init . && git remote add origin "https://github.com/docker/cli.git"
ARG DOCKERCLI_VERSION
COPY /hack/dockerfile/install/install.sh /hack/dockerfile/install/dockercli.installer /
RUN --mount=type=cache,target=/root/.cache/go-build \
--mount=type=cache,target=/go/pkg/mod \
PREFIX=/build /install.sh dockercli
RUN git fetch -q --depth 1 origin "${DOCKERCLI_VERSION}" +refs/tags/*:refs/tags/* && git checkout -q FETCH_HEAD
RUN [ -d ./components/cli ] && mv ./components/cli /usr/src/dockercli || mv /tmp/dockercli /usr/src/dockercli
WORKDIR /usr/src/dockercli
FROM runtime-dev AS runc
ARG RUNC_VERSION
ARG RUNC_BUILDTAGS
COPY /hack/dockerfile/install/install.sh /hack/dockerfile/install/runc.installer /
RUN --mount=type=cache,target=/root/.cache/go-build \
--mount=type=cache,target=/go/pkg/mod \
PREFIX=/build /install.sh runc
FROM base AS dockercli
WORKDIR /go/src/github.com/docker/cli
ARG DOCKERCLI_VERSION
ARG DOCKERCLI_CHANNEL=stable
ARG TARGETPLATFORM
RUN xx-apt-get install -y --no-install-recommends gcc libc6-dev
RUN --mount=from=dockercli-src,src=/usr/src/dockercli,rw \
--mount=type=cache,target=/root/.cache/go-build,id=dockercli-build-$TARGETPLATFORM <<EOT
set -e
DOWNLOAD_URL="https://download.docker.com/linux/static/${DOCKERCLI_CHANNEL}/$(xx-info march)/docker-${DOCKERCLI_VERSION#v}.tgz"
if curl --head --silent --fail "${DOWNLOAD_URL}" 1>/dev/null 2>&1; then
mkdir /build
curl -Ls "${DOWNLOAD_URL}" | tar -xz docker/docker
mv docker/docker /build/docker
else
CGO_ENABLED=0 xx-go build -o /build/docker ./cmd/docker
fi
xx-verify /build/docker
EOT
FROM dev-base AS tini
# runc
FROM base AS runc-src
WORKDIR /usr/src/runc
RUN git init . && git remote add origin "https://github.com/opencontainers/runc.git"
# RUNC_VERSION should match the version that is used by the containerd version
# that is used. If you need to update runc, open a pull request in the containerd
# project first, and update both after that is merged. When updating RUNC_VERSION,
# consider updating runc in vendor.mod accordingly.
ARG RUNC_VERSION=v1.1.12
RUN git fetch -q --depth 1 origin "${RUNC_VERSION}" +refs/tags/*:refs/tags/* && git checkout -q FETCH_HEAD
FROM base AS runc-build
WORKDIR /go/src/github.com/opencontainers/runc
ARG DEBIAN_FRONTEND
ARG TARGETPLATFORM
RUN --mount=type=cache,sharing=locked,id=moby-runc-aptlib,target=/var/lib/apt \
--mount=type=cache,sharing=locked,id=moby-runc-aptcache,target=/var/cache/apt \
apt-get update && xx-apt-get install -y --no-install-recommends \
dpkg-dev gcc libc6-dev libseccomp-dev
ARG DOCKER_STATIC
RUN --mount=from=runc-src,src=/usr/src/runc,rw \
--mount=type=cache,target=/root/.cache/go-build,id=runc-build-$TARGETPLATFORM <<EOT
set -e
xx-go --wrap
CGO_ENABLED=1 make "$([ "$DOCKER_STATIC" = "1" ] && echo "static" || echo "runc")"
xx-verify $([ "$DOCKER_STATIC" = "1" ] && echo "--static") runc
mkdir /build
mv runc /build/
EOT
FROM runc-build AS runc-linux
FROM binary-dummy AS runc-windows
FROM runc-${TARGETOS} AS runc
# tini
FROM base AS tini-src
WORKDIR /usr/src/tini
RUN git init . && git remote add origin "https://github.com/krallin/tini.git"
# TINI_VERSION specifies the version of tini (docker-init) to build. This
# binary is used when starting containers with the `--init` option.
ARG TINI_VERSION=v0.19.0
RUN git fetch -q --depth 1 origin "${TINI_VERSION}" +refs/tags/*:refs/tags/* && git checkout -q FETCH_HEAD
FROM base AS tini-build
WORKDIR /go/src/github.com/krallin/tini
ARG DEBIAN_FRONTEND
ARG TINI_VERSION
RUN --mount=type=cache,sharing=locked,id=moby-tini-aptlib,target=/var/lib/apt \
--mount=type=cache,sharing=locked,id=moby-tini-aptcache,target=/var/cache/apt \
apt-get update && apt-get install -y --no-install-recommends \
cmake \
vim-common
COPY /hack/dockerfile/install/install.sh /hack/dockerfile/install/tini.installer /
RUN --mount=type=cache,target=/root/.cache/go-build \
--mount=type=cache,target=/go/pkg/mod \
PREFIX=/build /install.sh tini
apt-get update && apt-get install -y --no-install-recommends cmake
ARG TARGETPLATFORM
RUN --mount=type=cache,sharing=locked,id=moby-tini-aptlib,target=/var/lib/apt \
--mount=type=cache,sharing=locked,id=moby-tini-aptcache,target=/var/cache/apt \
xx-apt-get install -y --no-install-recommends \
gcc libc6-dev
RUN --mount=from=tini-src,src=/usr/src/tini,rw \
--mount=type=cache,target=/root/.cache/go-build,id=tini-build-$TARGETPLATFORM <<EOT
set -e
CC=$(xx-info)-gcc cmake .
make tini-static
xx-verify --static tini-static
mkdir /build
mv tini-static /build/docker-init
EOT
FROM dev-base AS rootlesskit
ARG ROOTLESSKIT_VERSION
ARG PREFIX=/build
COPY /hack/dockerfile/install/install.sh /hack/dockerfile/install/rootlesskit.installer /
RUN --mount=type=cache,target=/root/.cache/go-build \
FROM tini-build AS tini-linux
FROM binary-dummy AS tini-windows
FROM tini-${TARGETOS} AS tini
# rootlesskit
FROM base AS rootlesskit-src
WORKDIR /usr/src/rootlesskit
RUN git init . && git remote add origin "https://github.com/rootless-containers/rootlesskit.git"
# When updating, also update rootlesskit commit in vendor.mod accordingly.
ARG ROOTLESSKIT_VERSION=v1.1.0
RUN git fetch -q --depth 1 origin "${ROOTLESSKIT_VERSION}" +refs/tags/*:refs/tags/* && git checkout -q FETCH_HEAD
FROM base AS rootlesskit-build
WORKDIR /go/src/github.com/rootless-containers/rootlesskit
ARG DEBIAN_FRONTEND
ARG TARGETPLATFORM
RUN --mount=type=cache,sharing=locked,id=moby-rootlesskit-aptlib,target=/var/lib/apt \
--mount=type=cache,sharing=locked,id=moby-rootlesskit-aptcache,target=/var/cache/apt \
apt-get update && xx-apt-get install -y --no-install-recommends \
gcc libc6-dev
ENV GO111MODULE=on
ARG DOCKER_STATIC
RUN --mount=from=rootlesskit-src,src=/usr/src/rootlesskit,rw \
--mount=type=cache,target=/go/pkg/mod \
/install.sh rootlesskit \
&& "${PREFIX}"/rootlesskit --version \
&& "${PREFIX}"/rootlesskit-docker-proxy --help
COPY ./contrib/dockerd-rootless.sh /build
COPY ./contrib/dockerd-rootless-setuptool.sh /build
--mount=type=cache,target=/root/.cache/go-build,id=rootlesskit-build-$TARGETPLATFORM <<EOT
set -e
export CGO_ENABLED=$([ "$DOCKER_STATIC" = "1" ] && echo "0" || echo "1")
xx-go build -o /build/rootlesskit -ldflags="$([ "$DOCKER_STATIC" != "1" ] && echo "-linkmode=external")" ./cmd/rootlesskit
xx-verify $([ "$DOCKER_STATIC" = "1" ] && echo "--static") /build/rootlesskit
xx-go build -o /build/rootlesskit-docker-proxy -ldflags="$([ "$DOCKER_STATIC" != "1" ] && echo "-linkmode=external")" ./cmd/rootlesskit-docker-proxy
xx-verify $([ "$DOCKER_STATIC" = "1" ] && echo "--static") /build/rootlesskit-docker-proxy
EOT
COPY ./contrib/dockerd-rootless.sh /build/
COPY ./contrib/dockerd-rootless-setuptool.sh /build/
FROM rootlesskit-build AS rootlesskit-linux
FROM binary-dummy AS rootlesskit-windows
FROM rootlesskit-${TARGETOS} AS rootlesskit
FROM base AS crun
ARG CRUN_VERSION=1.4.5
@ -288,16 +399,90 @@ RUN --mount=type=tmpfs,target=/tmp/crun-build \
./configure --bindir=/build && \
make -j install
FROM --platform=amd64 djs55/vpnkit:${VPNKIT_VERSION} AS vpnkit-amd64
# vpnkit
# use dummy scratch stage to avoid build to fail for unsupported platforms
FROM scratch AS vpnkit-windows
FROM scratch AS vpnkit-linux-386
FROM scratch AS vpnkit-linux-arm
FROM scratch AS vpnkit-linux-ppc64le
FROM scratch AS vpnkit-linux-riscv64
FROM scratch AS vpnkit-linux-s390x
FROM djs55/vpnkit:${VPNKIT_VERSION} AS vpnkit-linux-amd64
FROM djs55/vpnkit:${VPNKIT_VERSION} AS vpnkit-linux-arm64
FROM vpnkit-linux-${TARGETARCH} AS vpnkit-linux
FROM vpnkit-${TARGETOS} AS vpnkit
FROM --platform=arm64 djs55/vpnkit:${VPNKIT_VERSION} AS vpnkit-arm64
# containerutility
FROM base AS containerutil-src
WORKDIR /usr/src/containerutil
RUN git init . && git remote add origin "https://github.com/docker-archive/windows-container-utility.git"
ARG CONTAINERUTILITY_VERSION=aa1ba87e99b68e0113bd27ec26c60b88f9d4ccd9
RUN git fetch -q --depth 1 origin "${CONTAINERUTILITY_VERSION}" +refs/tags/*:refs/tags/* && git checkout -q FETCH_HEAD
FROM scratch AS vpnkit
COPY --from=vpnkit-amd64 /vpnkit /build/vpnkit.x86_64
COPY --from=vpnkit-arm64 /vpnkit /build/vpnkit.aarch64
FROM base AS containerutil-build
WORKDIR /usr/src/containerutil
ARG TARGETPLATFORM
RUN xx-apt-get install -y --no-install-recommends gcc g++ libc6-dev
RUN --mount=from=containerutil-src,src=/usr/src/containerutil,rw \
--mount=type=cache,target=/root/.cache/go-build,id=containerutil-build-$TARGETPLATFORM <<EOT
set -e
CC="$(xx-info)-gcc" CXX="$(xx-info)-g++" make
xx-verify --static containerutility.exe
mkdir /build
mv containerutility.exe /build/
EOT
# TODO: Some of this is only really needed for testing, it would be nice to split this up
FROM runtime-dev AS dev-systemd-false
FROM binary-dummy AS containerutil-linux
FROM containerutil-build AS containerutil-windows-amd64
FROM containerutil-windows-${TARGETARCH} AS containerutil-windows
FROM containerutil-${TARGETOS} AS containerutil
FROM base AS dev-systemd-false
COPY --from=dockercli /build/ /usr/local/cli
COPY --from=frozen-images /build/ /docker-frozen-images
COPY --from=swagger /build/ /usr/local/bin/
COPY --from=delve /build/ /usr/local/bin/
COPY --from=tomll /build/ /usr/local/bin/
COPY --from=gowinres /build/ /usr/local/bin/
COPY --from=tini /build/ /usr/local/bin/
COPY --from=registry /build/ /usr/local/bin/
# Skip the CRIU stage for now, as the opensuse package repository is sometimes
# unstable, and we're currently not using it in CI.
#
# FIXME(thaJeztah): re-enable this stage when https://github.com/moby/moby/issues/38963 is resolved (see https://github.com/moby/moby/pull/38984)
# COPY --from=criu /build/ /usr/local/bin/
COPY --from=gotestsum /build/ /usr/local/bin/
COPY --from=golangci_lint /build/ /usr/local/bin/
COPY --from=shfmt /build/ /usr/local/bin/
COPY --from=runc /build/ /usr/local/bin/
COPY --from=containerd /build/ /usr/local/bin/
COPY --from=rootlesskit /build/ /usr/local/bin/
COPY --from=vpnkit / /usr/local/bin/
COPY --from=containerutil /build/ /usr/local/bin/
COPY --from=crun /build/ /usr/local/bin/
COPY hack/dockerfile/etc/docker/ /etc/docker/
ENV PATH=/usr/local/cli:$PATH
WORKDIR /go/src/github.com/docker/docker
VOLUME /var/lib/docker
VOLUME /home/unprivilegeduser/.local/share/docker
# Wrap all commands in the "docker-in-docker" script to allow nested containers
ENTRYPOINT ["hack/dind"]
FROM dev-systemd-false AS dev-systemd-true
RUN --mount=type=cache,sharing=locked,id=moby-dev-aptlib,target=/var/lib/apt \
--mount=type=cache,sharing=locked,id=moby-dev-aptcache,target=/var/cache/apt \
apt-get update && apt-get install -y --no-install-recommends \
dbus \
dbus-user-session \
systemd \
systemd-sysv
RUN mkdir -p hack \
&& curl -o hack/dind-systemd https://raw.githubusercontent.com/AkihiroSuda/containerized-systemd/b70bac0daeea120456764248164c21684ade7d0d/docker-entrypoint.sh \
&& chmod +x hack/dind-systemd
ENTRYPOINT ["hack/dind-systemd"]
FROM dev-systemd-${SYSTEMD} AS dev-base
ARG DEBIAN_FRONTEND
RUN groupadd -r docker
RUN useradd --create-home --gid docker unprivilegeduser \
@ -309,6 +494,9 @@ RUN ln -sfv /go/src/github.com/docker/docker/.bashrc ~/.bashrc
RUN echo "source /usr/share/bash-completion/bash_completion" >> /etc/bash.bashrc
RUN ln -s /usr/local/completion/bash/docker /etc/bash_completion.d/docker
RUN ldconfig
# Set dev environment as safe git directory to prevent "dubious ownership" errors
# when bind-mounting the source into the dev-container. See https://github.com/moby/moby/pull/44930
RUN git config --global --add safe.directory $GOPATH/src/github.com/docker/docker
# This should only install packages that are specifically needed for the dev environment and nothing else
# Do you really need to add another package here? Can it be done in a different build stage?
RUN --mount=type=cache,sharing=locked,id=moby-dev-aptlib,target=/var/lib/apt \
@ -341,115 +529,120 @@ RUN --mount=type=cache,sharing=locked,id=moby-dev-aptlib,target=/var/lib/apt \
xz-utils \
zip \
zstd
# Switch to use iptables instead of nftables (to match the CI hosts)
# TODO use some kind of runtime auto-detection instead if/when nftables is supported (https://github.com/moby/moby/issues/26824)
RUN update-alternatives --set iptables /usr/sbin/iptables-legacy || true \
&& update-alternatives --set ip6tables /usr/sbin/ip6tables-legacy || true \
&& update-alternatives --set arptables /usr/sbin/arptables-legacy || true
RUN pip3 install yamllint==1.26.1
COPY --from=dockercli /build/ /usr/local/cli
COPY --from=frozen-images /build/ /docker-frozen-images
COPY --from=swagger /build/ /usr/local/bin/
COPY --from=delve /build/ /usr/local/bin/
COPY --from=tomll /build/ /usr/local/bin/
COPY --from=gowinres /build/ /usr/local/bin/
COPY --from=tini /build/ /usr/local/bin/
COPY --from=registry /build/ /usr/local/bin/
COPY --from=criu /build/ /usr/local/bin/
COPY --from=gotestsum /build/ /usr/local/bin/
COPY --from=golangci_lint /build/ /usr/local/bin/
COPY --from=shfmt /build/ /usr/local/bin/
COPY --from=runc /build/ /usr/local/bin/
COPY --from=containerd /build/ /usr/local/bin/
COPY --from=rootlesskit /build/ /usr/local/bin/
COPY --from=vpnkit /build/ /usr/local/bin/
COPY --from=crun /build/ /usr/local/bin/
COPY hack/dockerfile/etc/docker/ /etc/docker/
ENV PATH=/usr/local/cli:$PATH
ARG DOCKER_BUILDTAGS
ENV DOCKER_BUILDTAGS="${DOCKER_BUILDTAGS}"
WORKDIR /go/src/github.com/docker/docker
VOLUME /var/lib/docker
VOLUME /home/unprivilegeduser/.local/share/docker
# Wrap all commands in the "docker-in-docker" script to allow nested containers
ENTRYPOINT ["hack/dind"]
FROM dev-systemd-false AS dev-systemd-true
ARG YAMLLINT_VERSION=1.27.1
RUN pip3 install yamllint==${YAMLLINT_VERSION}
RUN --mount=type=cache,sharing=locked,id=moby-dev-aptlib,target=/var/lib/apt \
--mount=type=cache,sharing=locked,id=moby-dev-aptcache,target=/var/cache/apt \
apt-get update && apt-get install -y --no-install-recommends \
dbus \
dbus-user-session \
systemd \
systemd-sysv
RUN mkdir -p hack \
&& curl -o hack/dind-systemd https://raw.githubusercontent.com/AkihiroSuda/containerized-systemd/b70bac0daeea120456764248164c21684ade7d0d/docker-entrypoint.sh \
&& chmod +x hack/dind-systemd
ENTRYPOINT ["hack/dind-systemd"]
apt-get update && apt-get install --no-install-recommends -y \
gcc \
pkg-config \
dpkg-dev \
libapparmor-dev \
libdevmapper-dev \
libseccomp-dev \
libsecret-1-dev \
libsystemd-dev \
libudev-dev
FROM dev-systemd-${SYSTEMD} AS dev
FROM runtime-dev AS binary-base
ARG DOCKER_GITCOMMIT=HEAD
ENV DOCKER_GITCOMMIT=${DOCKER_GITCOMMIT}
ARG VERSION
ENV VERSION=${VERSION}
ARG PLATFORM
ENV PLATFORM=${PLATFORM}
ARG PRODUCT
ENV PRODUCT=${PRODUCT}
ARG DEFAULT_PRODUCT_LICENSE
ENV DEFAULT_PRODUCT_LICENSE=${DEFAULT_PRODUCT_LICENSE}
ARG PACKAGER_NAME
ENV PACKAGER_NAME=${PACKAGER_NAME}
ARG DOCKER_BUILDTAGS
ENV DOCKER_BUILDTAGS="${DOCKER_BUILDTAGS}"
ENV PREFIX=/build
# TODO: This is here because hack/make.sh binary copies these extras binaries
# from $PATH into the bundles dir.
# It would be nice to handle this in a different way.
COPY --from=tini /build/ /usr/local/bin/
COPY --from=runc /build/ /usr/local/bin/
COPY --from=containerd /build/ /usr/local/bin/
COPY --from=rootlesskit /build/ /usr/local/bin/
COPY --from=vpnkit /build/ /usr/local/bin/
COPY --from=gowinres /build/ /usr/local/bin/
FROM base AS build
COPY --from=gowinres /build/ /usr/local/bin/
WORKDIR /go/src/github.com/docker/docker
FROM binary-base AS build-binary
RUN --mount=type=cache,target=/root/.cache \
--mount=type=bind,target=.,ro \
ENV GO111MODULE=off
ENV CGO_ENABLED=1
ARG DEBIAN_FRONTEND
RUN --mount=type=cache,sharing=locked,id=moby-build-aptlib,target=/var/lib/apt \
--mount=type=cache,sharing=locked,id=moby-build-aptcache,target=/var/cache/apt \
apt-get update && apt-get install --no-install-recommends -y \
clang \
lld \
llvm
ARG TARGETPLATFORM
RUN --mount=type=cache,sharing=locked,id=moby-build-aptlib,target=/var/lib/apt \
--mount=type=cache,sharing=locked,id=moby-build-aptcache,target=/var/cache/apt \
xx-apt-get install --no-install-recommends -y \
dpkg-dev \
gcc \
libapparmor-dev \
libc6-dev \
libdevmapper-dev \
libseccomp-dev \
libsecret-1-dev \
libsystemd-dev \
libudev-dev
ARG DOCKER_BUILDTAGS
ARG DOCKER_DEBUG
ARG DOCKER_GITCOMMIT=HEAD
ARG DOCKER_LDFLAGS
ARG DOCKER_STATIC
ARG VERSION
ARG PLATFORM
ARG PRODUCT
ARG DEFAULT_PRODUCT_LICENSE
ARG PACKAGER_NAME
# PREFIX overrides DEST dir in make.sh script otherwise it fails because of
# read only mount in current work dir
ENV PREFIX=/tmp
RUN <<EOT
# in bullseye arm64 target does not link with lld so configure it to use ld instead
if [ "$(xx-info arch)" = "arm64" ]; then
XX_CC_PREFER_LINKER=ld xx-clang --setup-target-triple
fi
EOT
RUN --mount=type=bind,target=. \
--mount=type=tmpfs,target=cli/winresources/dockerd \
--mount=type=tmpfs,target=cli/winresources/docker-proxy \
hack/make.sh binary
FROM binary-base AS build-dynbinary
RUN --mount=type=cache,target=/root/.cache \
--mount=type=bind,target=.,ro \
--mount=type=tmpfs,target=cli/winresources/dockerd \
--mount=type=tmpfs,target=cli/winresources/docker-proxy \
hack/make.sh dynbinary
FROM binary-base AS build-cross
ARG DOCKER_CROSSPLATFORMS
RUN --mount=type=cache,target=/root/.cache \
--mount=type=bind,target=.,ro \
--mount=type=tmpfs,target=cli/winresources/dockerd \
--mount=type=tmpfs,target=cli/winresources/docker-proxy \
hack/make.sh cross
--mount=type=cache,target=/root/.cache/go-build,id=moby-build-$TARGETPLATFORM <<EOT
set -e
target=$([ "$DOCKER_STATIC" = "1" ] && echo "binary" || echo "dynbinary")
xx-go --wrap
PKG_CONFIG=$(xx-go env PKG_CONFIG) ./hack/make.sh $target
xx-verify $([ "$DOCKER_STATIC" = "1" ] && echo "--static") /tmp/bundles/${target}-daemon/dockerd$([ "$(xx-info os)" = "windows" ] && echo ".exe")
xx-verify $([ "$DOCKER_STATIC" = "1" ] && echo "--static") /tmp/bundles/${target}-daemon/docker-proxy$([ "$(xx-info os)" = "windows" ] && echo ".exe")
mkdir /build
mv /tmp/bundles/${target}-daemon/* /build/
EOT
# usage:
# > docker buildx bake binary
# > DOCKER_STATIC=0 docker buildx bake binary
# or
# > make binary
# > make dynbinary
FROM scratch AS binary
COPY --from=build-binary /build/bundles/ /
COPY --from=build /build/ /
FROM scratch AS dynbinary
COPY --from=build-dynbinary /build/bundles/ /
# usage:
# > docker buildx bake all
FROM scratch AS all
COPY --from=tini /build/ /
COPY --from=runc /build/ /
COPY --from=containerd /build/ /
COPY --from=rootlesskit /build/ /
COPY --from=containerutil /build/ /
COPY --from=vpnkit / /
COPY --from=build /build /
FROM scratch AS cross
COPY --from=build-cross /build/bundles/ /
# smoke tests
# usage:
# > docker buildx bake binary-smoketest
FROM --platform=$TARGETPLATFORM base AS smoketest
WORKDIR /usr/local/bin
COPY --from=build /build .
RUN <<EOT
set -ex
file dockerd
dockerd --version
file docker-proxy
docker-proxy --version
EOT
FROM dev AS final
COPY . /go/src/github.com/docker/docker
# usage:
# > make shell
# > SYSTEMD=true make shell
FROM dev-base AS dev
COPY . .

View file

@ -1,10 +1,9 @@
ARG GO_VERSION=1.18.3
ARG GO_VERSION=1.20.5
FROM golang:${GO_VERSION}-alpine AS base
ENV GO111MODULE=off
RUN apk --no-cache add \
bash \
btrfs-progs-dev \
build-base \
curl \
lvm2-dev \

View file

@ -5,7 +5,7 @@
# This represents the bare minimum required to build and test Docker.
ARG GO_VERSION=1.18.3
ARG GO_VERSION=1.20.13
ARG BASE_DEBIAN_DISTRO="bullseye"
ARG GOLANG_IMAGE="golang:${GO_VERSION}-${BASE_DEBIAN_DISTRO}"
@ -24,10 +24,8 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
build-essential \
curl \
cmake \
gcc \
git \
libapparmor-dev \
libbtrfs-dev \
libdevmapper-dev \
libseccomp-dev \
ca-certificates \

View file

@ -155,7 +155,7 @@
# The number of build steps below are explicitly minimised to improve performance.
# Extremely important - do not change the following line to reference a "specific" image,
# such as `mcr.microsoft.com/windows/servercore:ltsc2019`. If using this Dockerfile in process
# such as `mcr.microsoft.com/windows/servercore:ltsc2022`. If using this Dockerfile in process
# isolated containers, the kernel of the host must match the container image, and hence
# would fail between Windows Server 2016 (aka RS1) and Windows Server 2019 (aka RS5).
# It is expected that the image `microsoft/windowsservercore:latest` is present, and matches
@ -165,10 +165,10 @@ FROM microsoft/windowsservercore
# Use PowerShell as the default shell
SHELL ["powershell", "-Command", "$ErrorActionPreference = 'Stop'; $ProgressPreference = 'SilentlyContinue';"]
ARG GO_VERSION=1.18.3
ARG CONTAINERD_VERSION=v1.6.6
ARG GOTESTSUM_VERSION=v1.8.1
ARG GOWINRES_VERSION=v0.2.3
ARG GO_VERSION=1.20.13
ARG GOTESTSUM_VERSION=v1.8.2
ARG GOWINRES_VERSION=v0.3.0
ARG CONTAINERD_VERSION=v1.6.28
# Environment variable notes:
# - GO_VERSION must be consistent with 'Dockerfile' used by Linux.
@ -224,7 +224,7 @@ RUN `
`
Write-Host INFO: Downloading go...; `
$dlGoVersion=$Env:GO_VERSION -replace '\.0$',''; `
Download-File "https://golang.org/dl/go${dlGoVersion}.windows-amd64.zip" C:\go.zip; `
Download-File "https://go.dev/dl/go${dlGoVersion}.windows-amd64.zip" C:\go.zip; `
`
Write-Host INFO: Downloading compiler 1 of 3...; `
Download-File https://raw.githubusercontent.com/moby/docker-tdmgcc/master/gcc.zip C:\gcc.zip; `

910
Jenkinsfile vendored
View file

@ -8,21 +8,13 @@ pipeline {
timestamps()
}
parameters {
booleanParam(name: 'unit_validate', defaultValue: true, description: 'amd64 (x86_64) unit tests and vendor check')
booleanParam(name: 'validate_force', defaultValue: false, description: 'force validation steps to be run, even if no changes were detected')
booleanParam(name: 'amd64', defaultValue: true, description: 'amd64 (x86_64) Build/Test')
booleanParam(name: 'rootless', defaultValue: true, description: 'amd64 (x86_64) Build/Test (Rootless mode)')
booleanParam(name: 'cgroup2', defaultValue: true, description: 'amd64 (x86_64) Build/Test (cgroup v2)')
booleanParam(name: 'arm64', defaultValue: true, description: 'ARM (arm64) Build/Test')
booleanParam(name: 's390x', defaultValue: false, description: 'IBM Z (s390x) Build/Test')
booleanParam(name: 'ppc64le', defaultValue: false, description: 'PowerPC (ppc64le) Build/Test')
booleanParam(name: 'dco', defaultValue: true, description: 'Run the DCO check')
}
environment {
DOCKER_BUILDKIT = '1'
DOCKER_EXPERIMENTAL = '1'
DOCKER_GRAPHDRIVER = 'overlay2'
APT_MIRROR = 'cdn-fastly.deb.debian.org'
CHECK_CONFIG_COMMIT = '33a3680e08d1007e72c3b3f1454f823d8e9948ee'
TESTDEBUG = '0'
TIMEOUT = '120m'
@ -44,7 +36,7 @@ pipeline {
beforeAgent true
expression { params.dco }
}
agent { label 'amd64 && ubuntu-1804 && overlay2' }
agent { label 'arm64 && ubuntu-2004' }
steps {
sh '''
docker run --rm \
@ -57,904 +49,6 @@ pipeline {
}
stage('Build') {
parallel {
stage('unit-validate') {
when {
beforeAgent true
expression { params.unit_validate }
}
agent { label 'amd64 && ubuntu-1804 && overlay2' }
environment {
// On master ("non-pull-request"), force running some validation checks (vendor, swagger),
// even if no files were changed. This allows catching problems caused by pull-requests
// that were merged out-of-sequence.
TEST_FORCE_VALIDATE = sh returnStdout: true, script: 'if [ "${BRANCH_NAME%%-*}" != "PR" ] || [ "${CHANGE_TARGET:-master}" != "master" ] || [ "${validate_force}" = "true" ]; then echo "1"; fi'
}
stages {
stage("Print info") {
steps {
sh 'docker version'
sh 'docker info'
sh '''
echo "check-config.sh version: ${CHECK_CONFIG_COMMIT}"
curl -fsSL -o ${WORKSPACE}/check-config.sh "https://raw.githubusercontent.com/moby/moby/${CHECK_CONFIG_COMMIT}/contrib/check-config.sh" \
&& bash ${WORKSPACE}/check-config.sh || true
'''
}
}
stage("Build dev image") {
steps {
sh 'docker build --force-rm --build-arg APT_MIRROR --build-arg CROSS=true -t docker:${GIT_COMMIT} .'
}
}
stage("Validate") {
steps {
sh '''
docker run --rm -t --privileged \
-v "$WORKSPACE/bundles:/go/src/github.com/docker/docker/bundles" \
-v "$WORKSPACE/.git:/go/src/github.com/docker/docker/.git" \
--name docker-pr$BUILD_NUMBER \
-e DOCKER_EXPERIMENTAL \
-e DOCKER_GITCOMMIT=${GIT_COMMIT} \
-e DOCKER_GRAPHDRIVER \
-e TEST_FORCE_VALIDATE \
-e VALIDATE_REPO=${GIT_URL} \
-e VALIDATE_BRANCH=${CHANGE_TARGET} \
docker:${GIT_COMMIT} \
hack/validate/default
'''
}
}
stage("Docker-py") {
steps {
sh '''
docker run --rm -t --privileged \
-v "$WORKSPACE/bundles:/go/src/github.com/docker/docker/bundles" \
--name docker-pr$BUILD_NUMBER \
-e DOCKER_EXPERIMENTAL \
-e DOCKER_GITCOMMIT=${GIT_COMMIT} \
-e DOCKER_GRAPHDRIVER \
-e VALIDATE_REPO=${GIT_URL} \
-e VALIDATE_BRANCH=${CHANGE_TARGET} \
docker:${GIT_COMMIT} \
hack/make.sh \
dynbinary \
test-docker-py
'''
}
post {
always {
junit testResults: 'bundles/test-docker-py/junit-report.xml', allowEmptyResults: true
sh '''
echo "Ensuring container killed."
docker rm -vf docker-pr$BUILD_NUMBER || true
'''
sh '''
echo 'Chowning /workspace to jenkins user'
docker run --rm -v "$WORKSPACE:/workspace" busybox chown -R "$(id -u):$(id -g)" /workspace
'''
catchError(buildResult: 'SUCCESS', stageResult: 'FAILURE', message: 'Failed to create bundles.tar.gz') {
sh '''
bundleName=docker-py
echo "Creating ${bundleName}-bundles.tar.gz"
tar -czf ${bundleName}-bundles.tar.gz bundles/test-docker-py/*.xml bundles/test-docker-py/*.log
'''
archiveArtifacts artifacts: '*-bundles.tar.gz', allowEmptyArchive: true
}
}
}
}
stage("Static") {
steps {
sh '''
docker run --rm -t --privileged \
-v "$WORKSPACE/bundles:/go/src/github.com/docker/docker/bundles" \
--name docker-pr$BUILD_NUMBER \
-e DOCKER_GITCOMMIT=${GIT_COMMIT} \
-e DOCKER_GRAPHDRIVER \
docker:${GIT_COMMIT} \
hack/make.sh binary
'''
}
}
stage("Cross") {
steps {
sh '''
docker run --rm -t --privileged \
-v "$WORKSPACE/bundles:/go/src/github.com/docker/docker/bundles" \
--name docker-pr$BUILD_NUMBER \
-e DOCKER_GITCOMMIT=${GIT_COMMIT} \
-e DOCKER_GRAPHDRIVER \
docker:${GIT_COMMIT} \
hack/make.sh cross
'''
}
}
// needs to be last stage that calls make.sh for the junit report to work
stage("Unit tests") {
steps {
sh '''
sudo modprobe ip6table_filter
'''
sh '''
docker run --rm -t --privileged \
-v "$WORKSPACE/bundles:/go/src/github.com/docker/docker/bundles" \
--name docker-pr$BUILD_NUMBER \
-e DOCKER_EXPERIMENTAL \
-e DOCKER_GITCOMMIT=${GIT_COMMIT} \
-e DOCKER_GRAPHDRIVER \
-e VALIDATE_REPO=${GIT_URL} \
-e VALIDATE_BRANCH=${CHANGE_TARGET} \
docker:${GIT_COMMIT} \
hack/test/unit
'''
}
post {
always {
junit testResults: 'bundles/junit-report*.xml', allowEmptyResults: true
}
}
}
stage("Validate vendor") {
steps {
sh '''
docker run --rm -t --privileged \
-v "$WORKSPACE/.git:/go/src/github.com/docker/docker/.git" \
--name docker-pr$BUILD_NUMBER \
-e DOCKER_EXPERIMENTAL \
-e DOCKER_GITCOMMIT=${GIT_COMMIT} \
-e DOCKER_GRAPHDRIVER \
-e TEST_FORCE_VALIDATE \
-e VALIDATE_REPO=${GIT_URL} \
-e VALIDATE_BRANCH=${CHANGE_TARGET} \
docker:${GIT_COMMIT} \
hack/validate/vendor
'''
}
}
}
post {
always {
sh '''
echo 'Ensuring container killed.'
docker rm -vf docker-pr$BUILD_NUMBER || true
'''
sh '''
echo 'Chowning /workspace to jenkins user'
docker run --rm -v "$WORKSPACE:/workspace" busybox chown -R "$(id -u):$(id -g)" /workspace
'''
catchError(buildResult: 'SUCCESS', stageResult: 'FAILURE', message: 'Failed to create bundles.tar.gz') {
sh '''
bundleName=unit
echo "Creating ${bundleName}-bundles.tar.gz"
tar -czvf ${bundleName}-bundles.tar.gz bundles/junit-report*.xml bundles/go-test-report*.json bundles/profile*.out
'''
archiveArtifacts artifacts: '*-bundles.tar.gz', allowEmptyArchive: true
}
}
cleanup {
sh 'make clean'
deleteDir()
}
}
}
stage('amd64') {
when {
beforeAgent true
expression { params.amd64 }
}
agent { label 'amd64 && ubuntu-1804 && overlay2' }
stages {
stage("Print info") {
steps {
sh 'docker version'
sh 'docker info'
sh '''
echo "check-config.sh version: ${CHECK_CONFIG_COMMIT}"
curl -fsSL -o ${WORKSPACE}/check-config.sh "https://raw.githubusercontent.com/moby/moby/${CHECK_CONFIG_COMMIT}/contrib/check-config.sh" \
&& bash ${WORKSPACE}/check-config.sh || true
'''
}
}
stage("Build dev image") {
steps {
sh '''
# todo: include ip_vs in base image
sudo modprobe ip_vs
docker build --force-rm --build-arg APT_MIRROR -t docker:${GIT_COMMIT} .
'''
}
}
stage("Run tests") {
steps {
sh '''#!/bin/bash
# bash is needed so 'jobs -p' works properly
# it also accepts setting inline envvars for functions without explicitly exporting
set -x
run_tests() {
[ -n "$TESTDEBUG" ] && rm= || rm=--rm;
docker run $rm -t --privileged \
-v "$WORKSPACE/bundles/${TEST_INTEGRATION_DEST}:/go/src/github.com/docker/docker/bundles" \
-v "$WORKSPACE/bundles/dynbinary-daemon:/go/src/github.com/docker/docker/bundles/dynbinary-daemon" \
-v "$WORKSPACE/.git:/go/src/github.com/docker/docker/.git" \
--name "$CONTAINER_NAME" \
-e KEEPBUNDLE=1 \
-e TESTDEBUG \
-e TESTFLAGS \
-e TEST_SKIP_INTEGRATION \
-e TEST_SKIP_INTEGRATION_CLI \
-e DOCKER_GITCOMMIT=${GIT_COMMIT} \
-e DOCKER_GRAPHDRIVER \
-e TIMEOUT \
-e VALIDATE_REPO=${GIT_URL} \
-e VALIDATE_BRANCH=${CHANGE_TARGET} \
docker:${GIT_COMMIT} \
hack/make.sh \
"$1" \
test-integration
}
trap "exit" INT TERM
trap 'pids=$(jobs -p); echo "Remaining pids to kill: [$pids]"; [ -z "$pids" ] || kill $pids' EXIT
CONTAINER_NAME=docker-pr$BUILD_NUMBER
docker run --rm -t --privileged \
-v "$WORKSPACE/bundles:/go/src/github.com/docker/docker/bundles" \
-v "$WORKSPACE/.git:/go/src/github.com/docker/docker/.git" \
--name ${CONTAINER_NAME}-build \
-e DOCKER_EXPERIMENTAL \
-e DOCKER_GITCOMMIT=${GIT_COMMIT} \
-e DOCKER_GRAPHDRIVER \
docker:${GIT_COMMIT} \
hack/make.sh \
dynbinary
# flaky + integration
TEST_INTEGRATION_DEST=1 CONTAINER_NAME=${CONTAINER_NAME}-1 TEST_SKIP_INTEGRATION_CLI=1 run_tests test-integration-flaky &
# integration-cli first set
TEST_INTEGRATION_DEST=2 CONTAINER_NAME=${CONTAINER_NAME}-2 TEST_SKIP_INTEGRATION=1 TESTFLAGS="-test.run Test(DockerSuite|DockerNetworkSuite|DockerHubPullSuite|DockerRegistrySuite|DockerSchema1RegistrySuite|DockerRegistryAuthTokenSuite|DockerRegistryAuthHtpasswdSuite)/" run_tests &
# integration-cli second set
TEST_INTEGRATION_DEST=3 CONTAINER_NAME=${CONTAINER_NAME}-3 TEST_SKIP_INTEGRATION=1 TESTFLAGS="-test.run Test(DockerSwarmSuite|DockerDaemonSuite|DockerExternalVolumeSuite)/" run_tests &
c=0
for job in $(jobs -p); do
wait ${job} || c=$?
done
exit $c
'''
}
post {
always {
junit testResults: 'bundles/**/*-report.xml', allowEmptyResults: true
}
}
}
}
post {
always {
sh '''
echo "Ensuring container killed."
cids=$(docker ps -aq -f name=docker-pr${BUILD_NUMBER}-*)
[ -n "$cids" ] && docker rm -vf $cids || true
'''
sh '''
echo "Chowning /workspace to jenkins user"
docker run --rm -v "$WORKSPACE:/workspace" busybox chown -R "$(id -u):$(id -g)" /workspace
'''
catchError(buildResult: 'SUCCESS', stageResult: 'FAILURE', message: 'Failed to create bundles.tar.gz') {
sh '''
bundleName=amd64
echo "Creating ${bundleName}-bundles.tar.gz"
# exclude overlay2 directories
find bundles -path '*/root/*overlay2' -prune -o -type f \\( -name '*-report.json' -o -name '*.log' -o -name '*.prof' -o -name '*-report.xml' \\) -print | xargs tar -czf ${bundleName}-bundles.tar.gz
'''
archiveArtifacts artifacts: '*-bundles.tar.gz', allowEmptyArchive: true
}
}
cleanup {
sh 'make clean'
deleteDir()
}
}
}
stage('rootless') {
when {
beforeAgent true
expression { params.rootless }
}
agent { label 'amd64 && ubuntu-1804 && overlay2' }
stages {
stage("Print info") {
steps {
sh 'docker version'
sh 'docker info'
sh '''
echo "check-config.sh version: ${CHECK_CONFIG_COMMIT}"
curl -fsSL -o ${WORKSPACE}/check-config.sh "https://raw.githubusercontent.com/moby/moby/${CHECK_CONFIG_COMMIT}/contrib/check-config.sh" \
&& bash ${WORKSPACE}/check-config.sh || true
'''
}
}
stage("Build dev image") {
steps {
sh '''
docker build --force-rm --build-arg APT_MIRROR -t docker:${GIT_COMMIT} .
'''
}
}
stage("Integration tests") {
environment {
DOCKER_ROOTLESS = '1'
TEST_SKIP_INTEGRATION_CLI = '1'
}
steps {
sh '''
docker run --rm -t --privileged \
-v "$WORKSPACE/bundles:/go/src/github.com/docker/docker/bundles" \
--name docker-pr$BUILD_NUMBER \
-e DOCKER_GITCOMMIT=${GIT_COMMIT} \
-e DOCKER_GRAPHDRIVER \
-e DOCKER_EXPERIMENTAL \
-e DOCKER_ROOTLESS \
-e TEST_SKIP_INTEGRATION_CLI \
-e TIMEOUT \
-e VALIDATE_REPO=${GIT_URL} \
-e VALIDATE_BRANCH=${CHANGE_TARGET} \
docker:${GIT_COMMIT} \
hack/make.sh \
dynbinary \
test-integration
'''
}
post {
always {
junit testResults: 'bundles/**/*-report.xml', allowEmptyResults: true
}
}
}
}
post {
always {
sh '''
echo "Ensuring container killed."
docker rm -vf docker-pr$BUILD_NUMBER || true
'''
sh '''
echo "Chowning /workspace to jenkins user"
docker run --rm -v "$WORKSPACE:/workspace" busybox chown -R "$(id -u):$(id -g)" /workspace
'''
catchError(buildResult: 'SUCCESS', stageResult: 'FAILURE', message: 'Failed to create bundles.tar.gz') {
sh '''
bundleName=amd64-rootless
echo "Creating ${bundleName}-bundles.tar.gz"
# exclude overlay2 directories
find bundles -path '*/root/*overlay2' -prune -o -type f \\( -name '*-report.json' -o -name '*.log' -o -name '*.prof' -o -name '*-report.xml' \\) -print | xargs tar -czf ${bundleName}-bundles.tar.gz
'''
archiveArtifacts artifacts: '*-bundles.tar.gz', allowEmptyArchive: true
}
}
cleanup {
sh 'make clean'
deleteDir()
}
}
}
stage('cgroup2') {
when {
beforeAgent true
expression { params.cgroup2 }
}
agent { label 'amd64 && ubuntu-2004 && cgroup2' }
stages {
stage("Print info") {
steps {
sh 'docker version'
sh 'docker info'
}
}
stage("Build dev image") {
steps {
sh '''
docker build --force-rm --build-arg APT_MIRROR --build-arg SYSTEMD=true -t docker:${GIT_COMMIT} .
'''
}
}
stage("Integration tests") {
environment {
DOCKER_SYSTEMD = '1' // recommended cgroup driver for v2
TEST_SKIP_INTEGRATION_CLI = '1' // CLI tests do not support v2
}
steps {
sh '''
docker run --rm -t --privileged \
-v "$WORKSPACE/bundles:/go/src/github.com/docker/docker/bundles" \
--name docker-pr$BUILD_NUMBER \
-e DOCKER_GITCOMMIT=${GIT_COMMIT} \
-e DOCKER_GRAPHDRIVER \
-e DOCKER_EXPERIMENTAL \
-e DOCKER_SYSTEMD \
-e TEST_SKIP_INTEGRATION_CLI \
-e TIMEOUT \
-e VALIDATE_REPO=${GIT_URL} \
-e VALIDATE_BRANCH=${CHANGE_TARGET} \
docker:${GIT_COMMIT} \
hack/make.sh \
dynbinary \
test-integration
'''
}
post {
always {
junit testResults: 'bundles/**/*-report.xml', allowEmptyResults: true
}
}
}
}
post {
always {
sh '''
echo "Ensuring container killed."
docker rm -vf docker-pr$BUILD_NUMBER || true
'''
sh '''
echo "Chowning /workspace to jenkins user"
docker run --rm -v "$WORKSPACE:/workspace" busybox chown -R "$(id -u):$(id -g)" /workspace
'''
catchError(buildResult: 'SUCCESS', stageResult: 'FAILURE', message: 'Failed to create bundles.tar.gz') {
sh '''
bundleName=amd64-cgroup2
echo "Creating ${bundleName}-bundles.tar.gz"
# exclude overlay2 directories
find bundles -path '*/root/*overlay2' -prune -o -type f \\( -name '*-report.json' -o -name '*.log' -o -name '*.prof' -o -name '*-report.xml' \\) -print | xargs tar -czf ${bundleName}-bundles.tar.gz
'''
archiveArtifacts artifacts: '*-bundles.tar.gz', allowEmptyArchive: true
}
}
cleanup {
sh 'make clean'
deleteDir()
}
}
}
stage('s390x') {
when {
beforeAgent true
// Skip this stage on PRs unless the checkbox is selected
anyOf {
not { changeRequest() }
expression { params.s390x }
}
}
agent { label 's390x-ubuntu-2004' }
stages {
stage("Print info") {
steps {
sh 'docker version'
sh 'docker info'
sh '''
echo "check-config.sh version: ${CHECK_CONFIG_COMMIT}"
curl -fsSL -o ${WORKSPACE}/check-config.sh "https://raw.githubusercontent.com/moby/moby/${CHECK_CONFIG_COMMIT}/contrib/check-config.sh" \
&& bash ${WORKSPACE}/check-config.sh || true
'''
}
}
stage("Build dev image") {
steps {
sh '''
docker build --force-rm --build-arg APT_MIRROR -t docker:${GIT_COMMIT} .
'''
}
}
stage("Unit tests") {
steps {
sh '''
sudo modprobe ip6table_filter
'''
sh '''
docker run --rm -t --privileged \
-v "$WORKSPACE/bundles:/go/src/github.com/docker/docker/bundles" \
--name docker-pr$BUILD_NUMBER \
-e DOCKER_EXPERIMENTAL \
-e DOCKER_GITCOMMIT=${GIT_COMMIT} \
-e DOCKER_GRAPHDRIVER \
-e VALIDATE_REPO=${GIT_URL} \
-e VALIDATE_BRANCH=${CHANGE_TARGET} \
docker:${GIT_COMMIT} \
hack/test/unit
'''
}
post {
always {
junit testResults: 'bundles/junit-report*.xml', allowEmptyResults: true
}
}
}
stage("Integration tests") {
environment { TEST_SKIP_INTEGRATION_CLI = '1' }
steps {
sh '''
docker run --rm -t --privileged \
-v "$WORKSPACE/bundles:/go/src/github.com/docker/docker/bundles" \
--name docker-pr$BUILD_NUMBER \
-e DOCKER_EXPERIMENTAL \
-e DOCKER_GITCOMMIT=${GIT_COMMIT} \
-e DOCKER_GRAPHDRIVER \
-e TESTDEBUG \
-e TEST_SKIP_INTEGRATION_CLI \
-e TIMEOUT \
-e VALIDATE_REPO=${GIT_URL} \
-e VALIDATE_BRANCH=${CHANGE_TARGET} \
docker:${GIT_COMMIT} \
hack/make.sh \
dynbinary \
test-integration
'''
}
post {
always {
junit testResults: 'bundles/**/*-report.xml', allowEmptyResults: true
}
}
}
}
post {
always {
sh '''
echo "Ensuring container killed."
docker rm -vf docker-pr$BUILD_NUMBER || true
'''
sh '''
echo "Chowning /workspace to jenkins user"
docker run --rm -v "$WORKSPACE:/workspace" busybox chown -R "$(id -u):$(id -g)" /workspace
'''
catchError(buildResult: 'SUCCESS', stageResult: 'FAILURE', message: 'Failed to create bundles.tar.gz') {
sh '''
bundleName=s390x-integration
echo "Creating ${bundleName}-bundles.tar.gz"
# exclude overlay2 directories
find bundles -path '*/root/*overlay2' -prune -o -type f \\( -name '*-report.json' -o -name '*.log' -o -name '*.prof' -o -name '*-report.xml' \\) -print | xargs tar -czf ${bundleName}-bundles.tar.gz
'''
archiveArtifacts artifacts: '*-bundles.tar.gz', allowEmptyArchive: true
}
}
cleanup {
sh 'make clean'
deleteDir()
}
}
}
stage('s390x integration-cli') {
when {
beforeAgent true
// Skip this stage on PRs unless the checkbox is selected
anyOf {
not { changeRequest() }
expression { params.s390x }
}
}
agent { label 's390x-ubuntu-2004' }
stages {
stage("Print info") {
steps {
sh 'docker version'
sh 'docker info'
sh '''
echo "check-config.sh version: ${CHECK_CONFIG_COMMIT}"
curl -fsSL -o ${WORKSPACE}/check-config.sh "https://raw.githubusercontent.com/moby/moby/${CHECK_CONFIG_COMMIT}/contrib/check-config.sh" \
&& bash ${WORKSPACE}/check-config.sh || true
'''
}
}
stage("Build dev image") {
steps {
sh '''
docker build --force-rm --build-arg APT_MIRROR -t docker:${GIT_COMMIT} .
'''
}
}
stage("Integration-cli tests") {
environment { TEST_SKIP_INTEGRATION = '1' }
steps {
sh '''
docker run --rm -t --privileged \
-v "$WORKSPACE/bundles:/go/src/github.com/docker/docker/bundles" \
--name docker-pr$BUILD_NUMBER \
-e DOCKER_GITCOMMIT=${GIT_COMMIT} \
-e DOCKER_GRAPHDRIVER \
-e TEST_SKIP_INTEGRATION \
-e TIMEOUT \
-e VALIDATE_REPO=${GIT_URL} \
-e VALIDATE_BRANCH=${CHANGE_TARGET} \
docker:${GIT_COMMIT} \
hack/make.sh \
dynbinary \
test-integration
'''
}
post {
always {
junit testResults: 'bundles/**/*-report.xml', allowEmptyResults: true
}
}
}
}
post {
always {
sh '''
echo "Ensuring container killed."
docker rm -vf docker-pr$BUILD_NUMBER || true
'''
sh '''
echo "Chowning /workspace to jenkins user"
docker run --rm -v "$WORKSPACE:/workspace" busybox chown -R "$(id -u):$(id -g)" /workspace
'''
catchError(buildResult: 'SUCCESS', stageResult: 'FAILURE', message: 'Failed to create bundles.tar.gz') {
sh '''
bundleName=s390x-integration-cli
echo "Creating ${bundleName}-bundles.tar.gz"
# exclude overlay2 directories
find bundles -path '*/root/*overlay2' -prune -o -type f \\( -name '*-report.json' -o -name '*.log' -o -name '*.prof' -o -name '*-report.xml' \\) -print | xargs tar -czf ${bundleName}-bundles.tar.gz
'''
archiveArtifacts artifacts: '*-bundles.tar.gz', allowEmptyArchive: true
}
}
cleanup {
sh 'make clean'
deleteDir()
}
}
}
stage('ppc64le') {
when {
beforeAgent true
// Skip this stage on PRs unless the checkbox is selected
anyOf {
not { changeRequest() }
expression { params.ppc64le }
}
}
agent { label 'ppc64le-ubuntu-1604' }
// ppc64le machines run on Docker 18.06, and buildkit has some
// bugs on that version. Build and use buildx instead.
environment {
USE_BUILDX = '1'
DOCKER_BUILDKIT = '0'
}
stages {
stage("Print info") {
steps {
sh 'docker version'
sh 'docker info'
sh '''
echo "check-config.sh version: ${CHECK_CONFIG_COMMIT}"
curl -fsSL -o ${WORKSPACE}/check-config.sh "https://raw.githubusercontent.com/moby/moby/${CHECK_CONFIG_COMMIT}/contrib/check-config.sh" \
&& bash ${WORKSPACE}/check-config.sh || true
'''
}
}
stage("Build dev image") {
steps {
sh '''
make bundles/buildx
bundles/buildx build --load --force-rm --build-arg APT_MIRROR -t docker:${GIT_COMMIT} .
'''
}
}
stage("Unit tests") {
steps {
sh '''
sudo modprobe ip6table_filter
'''
sh '''
docker run --rm -t --privileged \
-v "$WORKSPACE/bundles:/go/src/github.com/docker/docker/bundles" \
--name docker-pr$BUILD_NUMBER \
-e DOCKER_EXPERIMENTAL \
-e DOCKER_GITCOMMIT=${GIT_COMMIT} \
-e DOCKER_GRAPHDRIVER \
-e VALIDATE_REPO=${GIT_URL} \
-e VALIDATE_BRANCH=${CHANGE_TARGET} \
docker:${GIT_COMMIT} \
hack/test/unit
'''
}
post {
always {
junit testResults: 'bundles/junit-report*.xml', allowEmptyResults: true
}
}
}
stage("Integration tests") {
environment { TEST_SKIP_INTEGRATION_CLI = '1' }
steps {
sh '''
docker run --rm -t --privileged \
-v "$WORKSPACE/bundles:/go/src/github.com/docker/docker/bundles" \
--name docker-pr$BUILD_NUMBER \
-e DOCKER_EXPERIMENTAL \
-e DOCKER_GITCOMMIT=${GIT_COMMIT} \
-e DOCKER_GRAPHDRIVER \
-e TESTDEBUG \
-e TEST_SKIP_INTEGRATION_CLI \
-e TIMEOUT \
-e VALIDATE_REPO=${GIT_URL} \
-e VALIDATE_BRANCH=${CHANGE_TARGET} \
docker:${GIT_COMMIT} \
hack/make.sh \
dynbinary \
test-integration
'''
}
post {
always {
junit testResults: 'bundles/**/*-report.xml', allowEmptyResults: true
}
}
}
}
post {
always {
sh '''
echo "Ensuring container killed."
docker rm -vf docker-pr$BUILD_NUMBER || true
'''
sh '''
echo "Chowning /workspace to jenkins user"
docker run --rm -v "$WORKSPACE:/workspace" busybox chown -R "$(id -u):$(id -g)" /workspace
'''
catchError(buildResult: 'SUCCESS', stageResult: 'FAILURE', message: 'Failed to create bundles.tar.gz') {
sh '''
bundleName=ppc64le-integration
echo "Creating ${bundleName}-bundles.tar.gz"
# exclude overlay2 directories
find bundles -path '*/root/*overlay2' -prune -o -type f \\( -name '*-report.json' -o -name '*.log' -o -name '*.prof' -o -name '*-report.xml' \\) -print | xargs tar -czf ${bundleName}-bundles.tar.gz
'''
archiveArtifacts artifacts: '*-bundles.tar.gz', allowEmptyArchive: true
}
}
cleanup {
sh 'make clean'
deleteDir()
}
}
}
stage('ppc64le integration-cli') {
when {
beforeAgent true
// Skip this stage on PRs unless the checkbox is selected
anyOf {
not { changeRequest() }
expression { params.ppc64le }
}
}
agent { label 'ppc64le-ubuntu-1604' }
// ppc64le machines run on Docker 18.06, and buildkit has some
// bugs on that version. Build and use buildx instead.
environment {
USE_BUILDX = '1'
DOCKER_BUILDKIT = '0'
}
stages {
stage("Print info") {
steps {
sh 'docker version'
sh 'docker info'
sh '''
echo "check-config.sh version: ${CHECK_CONFIG_COMMIT}"
curl -fsSL -o ${WORKSPACE}/check-config.sh "https://raw.githubusercontent.com/moby/moby/${CHECK_CONFIG_COMMIT}/contrib/check-config.sh" \
&& bash ${WORKSPACE}/check-config.sh || true
'''
}
}
stage("Build dev image") {
steps {
sh '''
make bundles/buildx
bundles/buildx build --load --force-rm --build-arg APT_MIRROR -t docker:${GIT_COMMIT} .
'''
}
}
stage("Integration-cli tests") {
environment { TEST_SKIP_INTEGRATION = '1' }
steps {
sh '''
docker run --rm -t --privileged \
-v "$WORKSPACE/bundles:/go/src/github.com/docker/docker/bundles" \
--name docker-pr$BUILD_NUMBER \
-e DOCKER_GITCOMMIT=${GIT_COMMIT} \
-e DOCKER_GRAPHDRIVER \
-e TEST_SKIP_INTEGRATION \
-e TIMEOUT \
-e VALIDATE_REPO=${GIT_URL} \
-e VALIDATE_BRANCH=${CHANGE_TARGET} \
docker:${GIT_COMMIT} \
hack/make.sh \
dynbinary \
test-integration
'''
}
post {
always {
junit testResults: 'bundles/**/*-report.xml', allowEmptyResults: true
}
}
}
}
post {
always {
sh '''
echo "Ensuring container killed."
docker rm -vf docker-pr$BUILD_NUMBER || true
'''
sh '''
echo "Chowning /workspace to jenkins user"
docker run --rm -v "$WORKSPACE:/workspace" busybox chown -R "$(id -u):$(id -g)" /workspace
'''
catchError(buildResult: 'SUCCESS', stageResult: 'FAILURE', message: 'Failed to create bundles.tar.gz') {
sh '''
bundleName=ppc64le-integration-cli
echo "Creating ${bundleName}-bundles.tar.gz"
# exclude overlay2 directories
find bundles -path '*/root/*overlay2' -prune -o -type f \\( -name '*-report.json' -o -name '*.log' -o -name '*.prof' -o -name '*-report.xml' \\) -print | xargs tar -czf ${bundleName}-bundles.tar.gz
'''
archiveArtifacts artifacts: '*-bundles.tar.gz', allowEmptyArchive: true
}
}
cleanup {
sh 'make clean'
deleteDir()
}
}
}
stage('arm64') {
when {
beforeAgent true
@ -979,7 +73,7 @@ pipeline {
}
stage("Build dev image") {
steps {
sh 'docker build --force-rm --build-arg APT_MIRROR -t docker:${GIT_COMMIT} .'
sh 'docker build --force-rm -t docker:${GIT_COMMIT} .'
}
}
stage("Unit tests") {

View file

@ -1,21 +1,7 @@
.PHONY: all binary dynbinary build cross help install manpages run shell test test-docker-py test-integration test-unit validate win
.PHONY: all binary dynbinary build cross help install manpages run shell test test-docker-py test-integration test-unit validate validate-% win
BUILDX_VERSION ?= v0.8.2
ifdef USE_BUILDX
BUILDX ?= $(shell command -v buildx)
BUILDX ?= $(shell command -v docker-buildx)
DOCKER_BUILDX_CLI_PLUGIN_PATH ?= ~/.docker/cli-plugins/docker-buildx
BUILDX ?= $(shell if [ -x "$(DOCKER_BUILDX_CLI_PLUGIN_PATH)" ]; then echo $(DOCKER_BUILDX_CLI_PLUGIN_PATH); fi)
endif
ifndef USE_BUILDX
DOCKER_BUILDKIT := 1
export DOCKER_BUILDKIT
endif
BUILDX ?= bundles/buildx
DOCKER ?= docker
BUILDX ?= $(DOCKER) buildx
# set the graph driver as the current graphdriver if not set
DOCKER_GRAPHDRIVER := $(if $(DOCKER_GRAPHDRIVER),$(DOCKER_GRAPHDRIVER),$(shell docker info 2>&1 | grep "Storage Driver" | sed 's/.*: //'))
@ -45,8 +31,6 @@ export VALIDATE_ORIGIN_BRANCH
# make DOCKER_LDFLAGS="-X github.com/docker/docker/daemon/graphdriver.priority=overlay2,devicemapper" dynbinary
#
DOCKER_ENVS := \
-e DOCKER_CROSSPLATFORMS \
-e BUILD_APT_MIRROR \
-e BUILDFLAGS \
-e KEEPBUNDLE \
-e DOCKER_BUILD_ARGS \
@ -69,10 +53,13 @@ DOCKER_ENVS := \
-e DOCKER_USERLANDPROXY \
-e DOCKERD_ARGS \
-e DELVE_PORT \
-e GITHUB_ACTIONS \
-e TEST_FORCE_VALIDATE \
-e TEST_INTEGRATION_DIR \
-e TEST_INTEGRATION_FAIL_FAST \
-e TEST_SKIP_INTEGRATION \
-e TEST_SKIP_INTEGRATION_CLI \
-e TESTCOVERAGE \
-e TESTDEBUG \
-e TESTDIRS \
-e TESTFLAGS \
@ -118,9 +105,7 @@ DOCKER_IMAGE := docker-dev
DOCKER_PORT_FORWARD := $(if $(DOCKER_PORT),-p "$(DOCKER_PORT)",)
DELVE_PORT_FORWARD := $(if $(DELVE_PORT),-p "$(DELVE_PORT)",)
DOCKER_FLAGS := $(DOCKER) run --rm -i --privileged $(DOCKER_CONTAINER_NAME) $(DOCKER_ENVS) $(DOCKER_MOUNT) $(DOCKER_PORT_FORWARD) $(DELVE_PORT_FORWARD)
BUILD_APT_MIRROR := $(if $(DOCKER_BUILD_APT_MIRROR),--build-arg APT_MIRROR=$(DOCKER_BUILD_APT_MIRROR))
export BUILD_APT_MIRROR
DOCKER_FLAGS := $(DOCKER) run --rm --privileged $(DOCKER_CONTAINER_NAME) $(DOCKER_ENVS) $(DOCKER_MOUNT) $(DOCKER_PORT_FORWARD) $(DELVE_PORT_FORWARD)
SWAGGER_DOCS_PORT ?= 9000
@ -137,6 +122,14 @@ ifeq ($(INTERACTIVE), 1)
DOCKER_FLAGS += -t
endif
# on GitHub Runners input device is not a TTY but we allocate a pseudo-one,
# otherwise keep STDIN open even if not attached if not a GitHub Runner.
ifeq ($(GITHUB_ACTIONS),true)
DOCKER_FLAGS += -t
else
DOCKER_FLAGS += -i
endif
DOCKER_RUN_DOCKER := $(DOCKER_FLAGS) "$(DOCKER_IMAGE)"
DOCKER_BUILD_ARGS += --build-arg=GO_VERSION
@ -144,39 +137,23 @@ ifdef DOCKER_SYSTEMD
DOCKER_BUILD_ARGS += --build-arg=SYSTEMD=true
endif
BUILD_OPTS := ${BUILD_APT_MIRROR} ${DOCKER_BUILD_ARGS} ${DOCKER_BUILD_OPTS} -f "$(DOCKERFILE)"
ifdef USE_BUILDX
BUILD_OPTS += $(BUILDX_BUILD_EXTRA_OPTS)
BUILD_OPTS := ${DOCKER_BUILD_ARGS} ${DOCKER_BUILD_OPTS} -f "$(DOCKERFILE)"
BUILD_CMD := $(BUILDX) build
else
BUILD_CMD := $(DOCKER) build
endif
# This is used for the legacy "build" target and anything still depending on it
BUILD_CROSS =
ifdef DOCKER_CROSS
BUILD_CROSS = --build-arg CROSS=$(DOCKER_CROSS)
endif
ifdef DOCKER_CROSSPLATFORMS
BUILD_CROSS = --build-arg CROSS=true
endif
VERSION_AUTOGEN_ARGS = --build-arg VERSION --build-arg DOCKER_GITCOMMIT --build-arg PRODUCT --build-arg PLATFORM --build-arg DEFAULT_PRODUCT_LICENSE --build-arg PACKAGER_NAME
BAKE_CMD := $(BUILDX) bake
default: binary
all: build ## validate all checks, build linux binaries, run all tests,\ncross build non-linux binaries, and generate archives
$(DOCKER_RUN_DOCKER) bash -c 'hack/validate/default && hack/make.sh'
binary: buildx ## build statically linked linux binaries
$(BUILD_CMD) $(BUILD_OPTS) --output=bundles/ --target=$@ $(VERSION_AUTOGEN_ARGS) .
binary: bundles ## build statically linked linux binaries
$(BAKE_CMD) binary
dynbinary: buildx ## build dynamically linked linux binaries
$(BUILD_CMD) $(BUILD_OPTS) --output=bundles/ --target=$@ $(VERSION_AUTOGEN_ARGS) .
dynbinary: bundles ## build dynamically linked linux binaries
$(BAKE_CMD) dynbinary
cross: BUILD_OPTS += --build-arg CROSS=true --build-arg DOCKER_CROSSPLATFORMS
cross: buildx ## cross build the binaries for darwin, freebsd and\nwindows
$(BUILD_CMD) $(BUILD_OPTS) --output=bundles/ --target=$@ $(VERSION_AUTOGEN_ARGS) .
cross: bundles ## cross build the binaries
$(BAKE_CMD) binary-cross
bundles:
mkdir bundles
@ -199,21 +176,18 @@ run: build ## run the docker daemon in a container
.PHONY: build
ifeq ($(BIND_DIR), .)
build: shell_target := --target=dev
build: shell_target := --target=dev-base
else
build: shell_target := --target=final
build: shell_target := --target=dev
endif
ifdef USE_BUILDX
build: buildx_load := --load
endif
build: buildx
$(BUILD_CMD) $(BUILD_OPTS) $(shell_target) $(buildx_load) $(BUILD_CROSS) -t "$(DOCKER_IMAGE)" .
build: bundles
$(BUILD_CMD) $(BUILD_OPTS) $(shell_target) --load -t "$(DOCKER_IMAGE)" .
shell: build ## start a shell inside the build env
$(DOCKER_RUN_DOCKER) bash
test: build test-unit ## run the unit, integration and docker-py tests
$(DOCKER_RUN_DOCKER) hack/make.sh dynbinary cross test-integration test-docker-py
$(DOCKER_RUN_DOCKER) hack/make.sh dynbinary test-integration test-docker-py
test-docker-py: build ## run the docker-py tests
$(DOCKER_RUN_DOCKER) hack/make.sh dynbinary test-docker-py
@ -237,8 +211,11 @@ test-unit: build ## run the unit tests
validate: build ## validate DCO, Seccomp profile generation, gofmt,\n./pkg/ isolation, golint, tests, tomls, go vet and vendor
$(DOCKER_RUN_DOCKER) hack/validate/all
win: build ## cross build the binary for windows
$(DOCKER_RUN_DOCKER) DOCKER_CROSSPLATFORMS=windows/amd64 hack/make.sh cross
validate-%: build ## validate specific check
$(DOCKER_RUN_DOCKER) hack/validate/$*
win: bundles ## cross build the binary for windows
$(BAKE_CMD) --set *.platform=windows/amd64 binary
.PHONY: swagger-gen
swagger-gen:
@ -255,14 +232,3 @@ swagger-docs: ## preview the API documentation
-e 'REDOC_OPTIONS=hide-hostname="true" lazy-rendering' \
-p $(SWAGGER_DOCS_PORT):80 \
bfirsh/redoc:1.14.0
.PHONY: buildx
ifdef USE_BUILDX
ifeq ($(BUILDX), bundles/buildx)
buildx: bundles/buildx ## build buildx cli tool
endif
endif
bundles/buildx: bundles ## build buildx CLI tool
curl -fsSL https://raw.githubusercontent.com/moby/buildkit/70deac12b5857a1aa4da65e90b262368e2f71500/hack/install-buildx | VERSION="$(BUILDX_VERSION)" BINDIR="$(@D)" bash
$@ version

View file

@ -92,7 +92,7 @@ func (b *Backend) Build(ctx context.Context, config backend.BuildConfig) (string
stdout := config.ProgressWriter.StdoutFormatter
fmt.Fprintf(stdout, "Successfully built %s\n", stringid.TruncateID(imageID))
}
if imageID != "" {
if imageID != "" && !useBuildKit {
err = tagger.TagImages(image.ID(imageID))
}
return imageID, err

View file

@ -33,7 +33,7 @@ func TestJsonContentType(t *testing.T) {
func TestReadJSON(t *testing.T) {
t.Run("nil body", func(t *testing.T) {
req, err := http.NewRequest("POST", "https://example.com/some/path", nil)
req, err := http.NewRequest(http.MethodPost, "https://example.com/some/path", nil)
if err != nil {
t.Error(err)
}
@ -45,7 +45,7 @@ func TestReadJSON(t *testing.T) {
})
t.Run("empty body", func(t *testing.T) {
req, err := http.NewRequest("POST", "https://example.com/some/path", strings.NewReader(""))
req, err := http.NewRequest(http.MethodPost, "https://example.com/some/path", strings.NewReader(""))
if err != nil {
t.Error(err)
}
@ -60,7 +60,7 @@ func TestReadJSON(t *testing.T) {
})
t.Run("with valid request", func(t *testing.T) {
req, err := http.NewRequest("POST", "https://example.com/some/path", strings.NewReader(`{"SomeField":"some value"}`))
req, err := http.NewRequest(http.MethodPost, "https://example.com/some/path", strings.NewReader(`{"SomeField":"some value"}`))
if err != nil {
t.Error(err)
}
@ -75,7 +75,7 @@ func TestReadJSON(t *testing.T) {
}
})
t.Run("with whitespace", func(t *testing.T) {
req, err := http.NewRequest("POST", "https://example.com/some/path", strings.NewReader(`
req, err := http.NewRequest(http.MethodPost, "https://example.com/some/path", strings.NewReader(`
{"SomeField":"some value"}
@ -95,7 +95,7 @@ func TestReadJSON(t *testing.T) {
})
t.Run("with extra content", func(t *testing.T) {
req, err := http.NewRequest("POST", "https://example.com/some/path", strings.NewReader(`{"SomeField":"some value"} and more content`))
req, err := http.NewRequest(http.MethodPost, "https://example.com/some/path", strings.NewReader(`{"SomeField":"some value"} and more content`))
if err != nil {
t.Error(err)
}
@ -112,7 +112,7 @@ func TestReadJSON(t *testing.T) {
})
t.Run("invalid JSON", func(t *testing.T) {
req, err := http.NewRequest("POST", "https://example.com/some/path", strings.NewReader(`{invalid json`))
req, err := http.NewRequest(http.MethodPost, "https://example.com/some/path", strings.NewReader(`{invalid json`))
if err != nil {
t.Error(err)
}

View file

@ -61,5 +61,4 @@ func (v VersionMiddleware) WrapHandler(handler func(ctx context.Context, w http.
ctx = context.WithValue(ctx, httputils.APIVersionKey{}, apiVersion)
return handler(ctx, w, r, vars)
}
}

View file

@ -1,6 +1,8 @@
package build // import "github.com/docker/docker/api/server/router/build"
import (
"runtime"
"github.com/docker/docker/api/server/router"
"github.com/docker/docker/api/types"
)
@ -37,17 +39,24 @@ func (r *buildRouter) initRoutes() {
}
}
// BuilderVersion derives the default docker builder version from the config
// Note: it is valid to have BuilderVersion unset which means it is up to the
// client to choose which builder to use.
// BuilderVersion derives the default docker builder version from the config.
//
// The default on Linux is version "2" (BuildKit), but the daemon can be
// configured to recommend version "1" (classic Builder). Windows does not
// yet support BuildKit for native Windows images, and uses "1" (classic builder)
// as a default.
//
// This value is only a recommendation as advertised by the daemon, and it is
// up to the client to choose which builder to use.
func BuilderVersion(features map[string]bool) types.BuilderVersion {
var bv types.BuilderVersion
if v, ok := features["buildkit"]; ok {
if v {
bv = types.BuilderBuildKit
} else {
bv = types.BuilderV1
}
// TODO(thaJeztah) move the default to daemon/config
if runtime.GOOS == "windows" {
return types.BuilderV1
}
bv := types.BuilderBuildKit
if v, ok := features["buildkit"]; ok && !v {
bv = types.BuilderV1
}
return bv
}

View file

@ -238,7 +238,6 @@ func (br *buildRouter) postBuild(ctx context.Context, w http.ResponseWriter, r *
defer func() { _ = output.Close() }()
errf := func(err error) error {
if httputils.BoolValue(r, "q") && notVerboseBuffer.Len() > 0 {
_, _ = output.Write(notVerboseBuffer.Bytes())
}

View file

@ -44,7 +44,7 @@ func (s *containerRouter) postCommit(ctx context.Context, w http.ResponseWriter,
}
config, _, _, err := s.decoder.DecodeConfig(r.Body)
if err != nil && err != io.EOF { // Do not fail if body is empty.
if err != nil && !errors.Is(err, io.EOF) { // Do not fail if body is empty.
return err
}
@ -484,6 +484,9 @@ func (s *containerRouter) postContainersCreate(ctx context.Context, w http.Respo
config, hostConfig, networkingConfig, err := s.decoder.DecodeConfig(r.Body)
if err != nil {
if errors.Is(err, io.EOF) {
return errdefs.InvalidParameter(errors.New("invalid JSON: got EOF while reading request body"))
}
return err
}
version := httputils.VersionFromContext(ctx)

View file

@ -35,7 +35,7 @@ func (sr *swarmRouter) initCluster(ctx context.Context, w http.ResponseWriter, r
}
nodeID, err := sr.backend.Init(req)
if err != nil {
logrus.Errorf("Error initializing swarm: %v", err)
logrus.WithContext(ctx).WithError(err).Debug("Error initializing swarm")
return err
}
return httputils.WriteJSON(w, http.StatusOK, nodeID)
@ -61,7 +61,7 @@ func (sr *swarmRouter) leaveCluster(ctx context.Context, w http.ResponseWriter,
func (sr *swarmRouter) inspectCluster(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
swarm, err := sr.backend.Inspect()
if err != nil {
logrus.Errorf("Error getting swarm: %v", err)
logrus.WithContext(ctx).WithError(err).Debug("Error getting swarm")
return err
}
@ -113,7 +113,7 @@ func (sr *swarmRouter) updateCluster(ctx context.Context, w http.ResponseWriter,
}
if err := sr.backend.Update(version, swarm, flags); err != nil {
logrus.Errorf("Error configuring swarm: %v", err)
logrus.WithContext(ctx).WithError(err).Debug("Error configuring swarm")
return err
}
return nil
@ -126,7 +126,7 @@ func (sr *swarmRouter) unlockCluster(ctx context.Context, w http.ResponseWriter,
}
if err := sr.backend.UnlockSwarm(req); err != nil {
logrus.Errorf("Error unlocking swarm: %v", err)
logrus.WithContext(ctx).WithError(err).Debug("Error unlocking swarm")
return err
}
return nil
@ -135,7 +135,7 @@ func (sr *swarmRouter) unlockCluster(ctx context.Context, w http.ResponseWriter,
func (sr *swarmRouter) getUnlockKey(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
unlockKey, err := sr.backend.GetUnlockKey()
if err != nil {
logrus.WithError(err).Errorf("Error retrieving swarm unlock key")
logrus.WithContext(ctx).WithError(err).Debug("Error retrieving swarm unlock key")
return err
}
@ -167,7 +167,7 @@ func (sr *swarmRouter) getServices(ctx context.Context, w http.ResponseWriter, r
services, err := sr.backend.GetServices(basictypes.ServiceListOptions{Filters: filter, Status: status})
if err != nil {
logrus.Errorf("Error getting services: %v", err)
logrus.WithContext(ctx).WithError(err).Debug("Error getting services")
return err
}
@ -193,7 +193,10 @@ func (sr *swarmRouter) getService(ctx context.Context, w http.ResponseWriter, r
service, err := sr.backend.GetService(vars["id"], insertDefaults)
if err != nil {
logrus.Errorf("Error getting service %s: %v", vars["id"], err)
logrus.WithContext(ctx).WithFields(logrus.Fields{
"error": err,
"service-id": vars["id"],
}).Debug("Error getting service")
return err
}
@ -217,7 +220,10 @@ func (sr *swarmRouter) createService(ctx context.Context, w http.ResponseWriter,
}
resp, err := sr.backend.CreateService(service, encodedAuth, queryRegistry)
if err != nil {
logrus.Errorf("Error creating service %s: %v", service.Name, err)
logrus.WithContext(ctx).WithFields(logrus.Fields{
"error": err,
"service-name": service.Name,
}).Debug("Error creating service")
return err
}
@ -253,7 +259,10 @@ func (sr *swarmRouter) updateService(ctx context.Context, w http.ResponseWriter,
resp, err := sr.backend.UpdateService(vars["id"], version, service, flags, queryRegistry)
if err != nil {
logrus.Errorf("Error updating service %s: %v", vars["id"], err)
logrus.WithContext(ctx).WithFields(logrus.Fields{
"error": err,
"service-id": vars["id"],
}).Debug("Error updating service")
return err
}
return httputils.WriteJSON(w, http.StatusOK, resp)
@ -261,7 +270,10 @@ func (sr *swarmRouter) updateService(ctx context.Context, w http.ResponseWriter,
func (sr *swarmRouter) removeService(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
if err := sr.backend.RemoveService(vars["id"]); err != nil {
logrus.Errorf("Error removing service %s: %v", vars["id"], err)
logrus.WithContext(ctx).WithFields(logrus.Fields{
"error": err,
"service-id": vars["id"],
}).Debug("Error removing service")
return err
}
return nil
@ -302,7 +314,7 @@ func (sr *swarmRouter) getNodes(ctx context.Context, w http.ResponseWriter, r *h
nodes, err := sr.backend.GetNodes(basictypes.NodeListOptions{Filters: filter})
if err != nil {
logrus.Errorf("Error getting nodes: %v", err)
logrus.WithContext(ctx).WithError(err).Debug("Error getting nodes")
return err
}
@ -312,7 +324,10 @@ func (sr *swarmRouter) getNodes(ctx context.Context, w http.ResponseWriter, r *h
func (sr *swarmRouter) getNode(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
node, err := sr.backend.GetNode(vars["id"])
if err != nil {
logrus.Errorf("Error getting node %s: %v", vars["id"], err)
logrus.WithContext(ctx).WithFields(logrus.Fields{
"error": err,
"node-id": vars["id"],
}).Debug("Error getting node")
return err
}
@ -333,7 +348,10 @@ func (sr *swarmRouter) updateNode(ctx context.Context, w http.ResponseWriter, r
}
if err := sr.backend.UpdateNode(vars["id"], version, node); err != nil {
logrus.Errorf("Error updating node %s: %v", vars["id"], err)
logrus.WithContext(ctx).WithFields(logrus.Fields{
"error": err,
"node-id": vars["id"],
}).Debug("Error updating node")
return err
}
return nil
@ -347,7 +365,10 @@ func (sr *swarmRouter) removeNode(ctx context.Context, w http.ResponseWriter, r
force := httputils.BoolValue(r, "force")
if err := sr.backend.RemoveNode(vars["id"], force); err != nil {
logrus.Errorf("Error removing node %s: %v", vars["id"], err)
logrus.WithContext(ctx).WithFields(logrus.Fields{
"error": err,
"node-id": vars["id"],
}).Debug("Error removing node")
return err
}
return nil
@ -364,7 +385,7 @@ func (sr *swarmRouter) getTasks(ctx context.Context, w http.ResponseWriter, r *h
tasks, err := sr.backend.GetTasks(basictypes.TaskListOptions{Filters: filter})
if err != nil {
logrus.Errorf("Error getting tasks: %v", err)
logrus.WithContext(ctx).WithError(err).Debug("Error getting tasks")
return err
}
@ -374,7 +395,10 @@ func (sr *swarmRouter) getTasks(ctx context.Context, w http.ResponseWriter, r *h
func (sr *swarmRouter) getTask(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
task, err := sr.backend.GetTask(vars["id"])
if err != nil {
logrus.Errorf("Error getting task %s: %v", vars["id"], err)
logrus.WithContext(ctx).WithFields(logrus.Fields{
"error": err,
"task-id": vars["id"],
}).Debug("Error getting task")
return err
}

View file

@ -115,5 +115,4 @@ func TestAdjustForAPIVersion(t *testing.T) {
if len(spec.TaskTemplate.ContainerSpec.Ulimits) != 0 {
t.Error("Ulimits were not stripped from spec")
}
}

View file

@ -174,6 +174,15 @@ func (s *systemRouter) getDiskUsage(ctx context.Context, w http.ResponseWriter,
if versions.LessThan(version, "1.42") {
for _, b := range buildCache {
builderSize += b.Size
// Parents field was added in API 1.42 to replace the Parent field.
b.Parents = nil
}
}
if versions.GreaterThanOrEqualTo(version, "1.42") {
for _, b := range buildCache {
// Parent field is deprecated in API v1.42 and up, as it is deprecated
// in BuildKit. Empty the field to omit it in the API response.
b.Parent = "" //nolint:staticcheck // ignore SA1019 (Parent field is deprecated)
}
}

View file

@ -159,20 +159,29 @@ func (v *volumeRouter) deleteVolumes(ctx context.Context, w http.ResponseWriter,
}
force := httputils.BoolValue(r, "force")
version := httputils.VersionFromContext(ctx)
// First we try deleting local volume. The volume may not be found as a
// local volume, but could be a cluster volume, so we ignore "not found"
// errors at this stage. Note that no "not found" error is produced if
// "force" is enabled.
err := v.backend.Remove(ctx, vars["name"], opts.WithPurgeOnError(force))
if err != nil {
if errdefs.IsNotFound(err) && versions.GreaterThanOrEqualTo(version, clusterVolumesVersion) && v.cluster.IsManager() {
err := v.cluster.RemoveVolume(vars["name"], force)
if err != nil {
return err
}
} else {
return err
if err != nil && !errdefs.IsNotFound(err) {
return err
}
// If no volume was found, the volume may be a cluster volume. If force
// is enabled, the volume backend won't return an error for non-existing
// volumes, so we don't know if removal succeeded (or not volume existed).
// In that case we always try to delete cluster volumes as well.
if errdefs.IsNotFound(err) || force {
version := httputils.VersionFromContext(ctx)
if versions.GreaterThanOrEqualTo(version, clusterVolumesVersion) && v.cluster.IsManager() {
err = v.cluster.RemoveVolume(vars["name"], force)
}
}
if err != nil {
return err
}
w.WriteHeader(http.StatusNoContent)
return nil
}
@ -187,6 +196,12 @@ func (v *volumeRouter) postVolumesPrune(ctx context.Context, w http.ResponseWrit
return err
}
// API version 1.42 changes behavior where prune should only prune anonymous volumes.
// To keep older API behavior working, we need to add this filter option to consider all (local) volumes for pruning, not just anonymous ones.
if versions.LessThan(httputils.VersionFromContext(ctx), "1.42") {
pruneFilters.Add("all", "true")
}
pruneReport, err := v.backend.Prune(ctx, pruneFilters)
if err != nil {
return err

View file

@ -79,7 +79,7 @@ func TestGetVolumeByNameFoundRegular(t *testing.T) {
backend: &fakeVolumeBackend{
volumes: map[string]*volume.Volume{
"volume1": &volume.Volume{
"volume1": {
Name: "volume1",
},
},
@ -98,7 +98,7 @@ func TestGetVolumeByNameFoundSwarm(t *testing.T) {
swarm: true,
manager: true,
volumes: map[string]*volume.Volume{
"volume1": &volume.Volume{
"volume1": {
Name: "volume1",
},
},
@ -112,16 +112,16 @@ func TestListVolumes(t *testing.T) {
v := &volumeRouter{
backend: &fakeVolumeBackend{
volumes: map[string]*volume.Volume{
"v1": &volume.Volume{Name: "v1"},
"v2": &volume.Volume{Name: "v2"},
"v1": {Name: "v1"},
"v2": {Name: "v2"},
},
},
cluster: &fakeClusterBackend{
swarm: true,
manager: true,
volumes: map[string]*volume.Volume{
"v3": &volume.Volume{Name: "v3"},
"v4": &volume.Volume{Name: "v4"},
"v3": {Name: "v3"},
"v4": {Name: "v4"},
},
},
}
@ -140,8 +140,8 @@ func TestListVolumesNoSwarm(t *testing.T) {
v := &volumeRouter{
backend: &fakeVolumeBackend{
volumes: map[string]*volume.Volume{
"v1": &volume.Volume{Name: "v1"},
"v2": &volume.Volume{Name: "v2"},
"v1": {Name: "v1"},
"v2": {Name: "v2"},
},
},
cluster: &fakeClusterBackend{},
@ -155,8 +155,8 @@ func TestListVolumesNoManager(t *testing.T) {
v := &volumeRouter{
backend: &fakeVolumeBackend{
volumes: map[string]*volume.Volume{
"v1": &volume.Volume{Name: "v1"},
"v2": &volume.Volume{Name: "v2"},
"v1": {Name: "v1"},
"v2": {Name: "v2"},
},
},
cluster: &fakeClusterBackend{swarm: true},
@ -318,7 +318,7 @@ func TestUpdateVolume(t *testing.T) {
swarm: true,
manager: true,
volumes: map[string]*volume.Volume{
"vol1": &volume.Volume{
"vol1": {
Name: "vo1",
ClusterVolume: &volume.ClusterVolume{
ID: "vol1",
@ -409,7 +409,7 @@ func TestUpdateVolumeNotFound(t *testing.T) {
func TestVolumeRemove(t *testing.T) {
b := &fakeVolumeBackend{
volumes: map[string]*volume.Volume{
"vol1": &volume.Volume{
"vol1": {
Name: "vol1",
},
},
@ -436,7 +436,7 @@ func TestVolumeRemoveSwarm(t *testing.T) {
swarm: true,
manager: true,
volumes: map[string]*volume.Volume{
"vol1": &volume.Volume{
"vol1": {
Name: "vol1",
ClusterVolume: &volume.ClusterVolume{},
},
@ -494,7 +494,7 @@ func TestVolumeRemoveNotFoundNoManager(t *testing.T) {
func TestVolumeRemoveFoundNoSwarm(t *testing.T) {
b := &fakeVolumeBackend{
volumes: map[string]*volume.Volume{
"vol1": &volume.Volume{
"vol1": {
Name: "vol1",
},
},
@ -518,7 +518,7 @@ func TestVolumeRemoveFoundNoSwarm(t *testing.T) {
func TestVolumeRemoveNoSwarmInUse(t *testing.T) {
b := &fakeVolumeBackend{
volumes: map[string]*volume.Volume{
"inuse": &volume.Volume{
"inuse": {
Name: "inuse",
},
},
@ -544,7 +544,7 @@ func TestVolumeRemoveSwarmForce(t *testing.T) {
swarm: true,
manager: true,
volumes: map[string]*volume.Volume{
"vol1": &volume.Volume{
"vol1": {
Name: "vol1",
ClusterVolume: &volume.ClusterVolume{},
Options: map[string]string{"mustforce": "yes"},
@ -574,6 +574,7 @@ func TestVolumeRemoveSwarmForce(t *testing.T) {
assert.NilError(t, err)
assert.Equal(t, len(b.volumes), 0)
assert.Equal(t, len(c.volumes), 0)
}
type fakeVolumeBackend struct {
@ -616,9 +617,16 @@ func (b *fakeVolumeBackend) Create(_ context.Context, name, driverName string, _
return v, nil
}
func (b *fakeVolumeBackend) Remove(_ context.Context, name string, _ ...opts.RemoveOption) error {
func (b *fakeVolumeBackend) Remove(_ context.Context, name string, o ...opts.RemoveOption) error {
removeOpts := &opts.RemoveConfig{}
for _, opt := range o {
opt(removeOpts)
}
if v, ok := b.volumes[name]; !ok {
return errdefs.NotFound(fmt.Errorf("volume %s not found", name))
if !removeOpts.PurgeOnError {
return errdefs.NotFound(fmt.Errorf("volume %s not found", name))
}
} else if v.Name == "inuse" {
return errdefs.Conflict(fmt.Errorf("volume in use"))
}

View file

@ -6,6 +6,7 @@ import (
"net"
"net/http"
"strings"
"time"
"github.com/docker/docker/api/server/httpstatus"
"github.com/docker/docker/api/server/httputils"
@ -58,7 +59,8 @@ func (s *Server) Accept(addr string, listeners ...net.Listener) {
for _, listener := range listeners {
httpServer := &HTTPServer{
srv: &http.Server{
Addr: addr,
Addr: addr,
ReadHeaderTimeout: 5 * time.Minute, // "G112: Potential Slowloris Attack (gosec)"; not a real concern for our use, so setting a long timeout.
},
l: listener,
}

View file

@ -24,7 +24,7 @@ info:
title: "Docker Engine API"
version: "1.42"
x-logo:
url: "https://docs.docker.com/images/logo-docker-main.png"
url: "https://docs.docker.com/assets/images/logo-docker-main.png"
description: |
The Engine API is an HTTP API served by Docker Engine. It is the API the
Docker client uses to communicate with the Engine, so everything the Docker
@ -214,12 +214,14 @@ definitions:
- `volume` a docker volume with the given `Name`.
- `tmpfs` a `tmpfs`.
- `npipe` a named pipe from the host into the container.
- `cluster` a Swarm cluster volume
type: "string"
enum:
- "bind"
- "volume"
- "tmpfs"
- "npipe"
- "cluster"
example: "volume"
Name:
description: |
@ -350,12 +352,14 @@ definitions:
- `volume` Creates a volume with the given name and options (or uses a pre-existing volume with the same name and options). These are **not** removed when the container is removed.
- `tmpfs` Create a tmpfs with the given options. The mount source cannot be specified for tmpfs.
- `npipe` Mounts a named pipe from the host into the container. Must exist prior to creating the container.
- `cluster` a Swarm cluster volume
type: "string"
enum:
- "bind"
- "volume"
- "tmpfs"
- "npipe"
- "cluster"
ReadOnly:
description: "Whether the mount should be read-only."
type: "boolean"
@ -2247,23 +2251,63 @@ definitions:
BuildCache:
type: "object"
description: |
BuildCache contains information about a build cache record.
properties:
ID:
type: "string"
description: |
Unique ID of the build cache record.
example: "ndlpt0hhvkqcdfkputsk4cq9c"
Parent:
description: |
ID of the parent build cache record.
> **Deprecated**: This field is deprecated, and omitted if empty.
type: "string"
x-nullable: true
example: ""
Parents:
description: |
List of parent build cache record IDs.
type: "array"
items:
type: "string"
x-nullable: true
example: ["hw53o5aio51xtltp5xjp8v7fx"]
Type:
type: "string"
description: |
Cache record type.
example: "regular"
# see https://github.com/moby/buildkit/blob/fce4a32258dc9d9664f71a4831d5de10f0670677/client/diskusage.go#L75-L84
enum:
- "internal"
- "frontend"
- "source.local"
- "source.git.checkout"
- "exec.cachemount"
- "regular"
Description:
type: "string"
description: |
Description of the build-step that produced the build cache.
example: "mount / from exec /bin/sh -c echo 'Binary::apt::APT::Keep-Downloaded-Packages \"true\";' > /etc/apt/apt.conf.d/keep-cache"
InUse:
type: "boolean"
description: |
Indicates if the build cache is in use.
example: false
Shared:
type: "boolean"
description: |
Indicates if the build cache is shared.
example: true
Size:
description: |
Amount of disk space used by the build cache (in bytes).
type: "integer"
example: 51
CreatedAt:
description: |
Date and time at which the build cache was created in
@ -2281,6 +2325,7 @@ definitions:
example: "2017-08-09T07:09:37.632105588Z"
UsageCount:
type: "integer"
example: 26
ImageID:
type: "object"
@ -2298,6 +2343,8 @@ definitions:
type: "string"
error:
type: "string"
errorDetail:
$ref: "#/definitions/ErrorDetail"
status:
type: "string"
progress:
@ -6210,6 +6257,28 @@ paths:
`/?[a-zA-Z0-9][a-zA-Z0-9_.-]+`.
type: "string"
pattern: "^/?[a-zA-Z0-9][a-zA-Z0-9_.-]+$"
- name: "platform"
in: "query"
description: |
Platform in the format `os[/arch[/variant]]` used for image lookup.
When specified, the daemon checks if the requested image is present
in the local image cache with the given OS and Architecture, and
otherwise returns a `404` status.
If the option is not set, the host's native OS and Architecture are
used to look up the image in the image cache. However, if no platform
is passed and the given image does exist in the local image cache,
but its OS or architecture does not match, the container is created
with the available image, and a warning is added to the `Warnings`
field in the response, for example;
WARNING: The requested image's platform (linux/arm64/v8) does not
match the detected host platform (linux/amd64) and no
specific platform was requested
type: "string"
default: ""
- name: "body"
in: "body"
description: "Container to create"
@ -8658,6 +8727,10 @@ paths:
IdentityToken: "9cbaf023786cd7..."
204:
description: "No error"
401:
description: "Auth error"
schema:
$ref: "#/definitions/ErrorResponse"
500:
description: "Server error"
schema:
@ -8719,7 +8792,17 @@ paths:
description: "Max API Version the server supports"
Builder-Version:
type: "string"
description: "Default version of docker image builder"
description: |
Default version of docker image builder
The default on Linux is version "2" (BuildKit), but the daemon
can be configured to recommend version "1" (classic Builder).
Windows does not yet support BuildKit for native Windows images,
and uses "1" (classic builder) as a default.
This value is a recommendation as advertised by the daemon, and
it is up to the client to choose which builder to use.
default: "2"
Docker-Experimental:
type: "boolean"
description: "If the server is running with experimental mode enabled"
@ -9013,7 +9096,7 @@ paths:
BuildCache:
-
ID: "hw53o5aio51xtltp5xjp8v7fx"
Parent: ""
Parents: []
Type: "regular"
Description: "pulled from docker.io/library/debian@sha256:234cb88d3020898631af0ccbbcca9a66ae7306ecd30c9720690858c1b007d2a0"
InUse: false
@ -9024,7 +9107,7 @@ paths:
UsageCount: 26
-
ID: "ndlpt0hhvkqcdfkputsk4cq9c"
Parent: "hw53o5aio51xtltp5xjp8v7fx"
Parents: ["ndlpt0hhvkqcdfkputsk4cq9c"]
Type: "regular"
Description: "mount / from exec /bin/sh -c echo 'Binary::apt::APT::Keep-Downloaded-Packages \"true\";' > /etc/apt/apt.conf.d/keep-cache"
InUse: false
@ -9622,6 +9705,7 @@ paths:
Available filters:
- `label` (`label=<key>`, `label=<key>=<value>`, `label!=<key>`, or `label!=<key>=<value>`) Prune volumes with (or without, in case `label!=...` is used) the specified labels.
- `all` (`all=true`) - Consider all (local) volumes for pruning and not just anonymous volumes.
type: "string"
responses:
200:

View file

@ -1,4 +1,5 @@
/*Package filters provides tools for encoding a mapping of keys to a set of
/*
Package filters provides tools for encoding a mapping of keys to a set of
multiple values.
*/
package filters // import "github.com/docker/docker/api/types/filters"
@ -49,7 +50,7 @@ func (args Args) Keys() []string {
// MarshalJSON returns a JSON byte representation of the Args
func (args Args) MarshalJSON() ([]byte, error) {
if len(args.fields) == 0 {
return []byte{}, nil
return []byte("{}"), nil
}
return json.Marshal(args.fields)
}
@ -107,9 +108,6 @@ func FromJSON(p string) (Args, error) {
// UnmarshalJSON populates the Args from JSON encode bytes
func (args Args) UnmarshalJSON(raw []byte) error {
if len(raw) == 0 {
return nil
}
return json.Unmarshal(raw, &args.fields)
}

View file

@ -1,6 +1,7 @@
package filters // import "github.com/docker/docker/api/types/filters"
import (
"encoding/json"
"errors"
"testing"
@ -8,6 +9,26 @@ import (
is "gotest.tools/v3/assert/cmp"
)
func TestMarshalJSON(t *testing.T) {
fields := map[string]map[string]bool{
"created": {"today": true},
"image.name": {"ubuntu*": true, "*untu": true},
}
a := Args{fields: fields}
_, err := a.MarshalJSON()
if err != nil {
t.Errorf("failed to marshal the filters: %s", err)
}
}
func TestMarshalJSONWithEmpty(t *testing.T) {
_, err := json.Marshal(NewArgs())
if err != nil {
t.Errorf("failed to marshal the filters: %s", err)
}
}
func TestToJSON(t *testing.T) {
fields := map[string]map[string]bool{
"created": {"today": true},

View file

@ -18,7 +18,7 @@ const (
// TypeNamedPipe is the type for mounting Windows named pipes
TypeNamedPipe Type = "npipe"
// TypeCluster is the type for Swarm Cluster Volumes.
TypeCluster = "csi"
TypeCluster Type = "cluster"
)
// Mount represents a mount (volume).

View file

@ -45,31 +45,32 @@ func (ipnet *NetIPNet) UnmarshalJSON(b []byte) (err error) {
// IndexInfo contains information about a registry
//
// RepositoryInfo Examples:
// {
// "Index" : {
// "Name" : "docker.io",
// "Mirrors" : ["https://registry-2.docker.io/v1/", "https://registry-3.docker.io/v1/"],
// "Secure" : true,
// "Official" : true,
// },
// "RemoteName" : "library/debian",
// "LocalName" : "debian",
// "CanonicalName" : "docker.io/debian"
// "Official" : true,
// }
//
// {
// "Index" : {
// "Name" : "127.0.0.1:5000",
// "Mirrors" : [],
// "Secure" : false,
// "Official" : false,
// },
// "RemoteName" : "user/repo",
// "LocalName" : "127.0.0.1:5000/user/repo",
// "CanonicalName" : "127.0.0.1:5000/user/repo",
// "Official" : false,
// }
// {
// "Index" : {
// "Name" : "docker.io",
// "Mirrors" : ["https://registry-2.docker.io/v1/", "https://registry-3.docker.io/v1/"],
// "Secure" : true,
// "Official" : true,
// },
// "RemoteName" : "library/debian",
// "LocalName" : "debian",
// "CanonicalName" : "docker.io/debian"
// "Official" : true,
// }
//
// {
// "Index" : {
// "Name" : "127.0.0.1:5000",
// "Mirrors" : [],
// "Secure" : false,
// "Official" : false,
// },
// "RemoteName" : "user/repo",
// "LocalName" : "127.0.0.1:5000/user/repo",
// "CanonicalName" : "127.0.0.1:5000/user/repo",
// "Official" : false,
// }
type IndexInfo struct {
// Name is the name of the registry, such as "docker.io"
Name string

View file

@ -33,17 +33,16 @@ func TestStrSliceUnmarshalJSON(t *testing.T) {
"[]": {},
`["/bin/sh","-c","echo"]`: {"/bin/sh", "-c", "echo"},
}
for json, expectedParts := range parts {
for input, expected := range parts {
strs := StrSlice{"default", "values"}
if err := strs.UnmarshalJSON([]byte(json)); err != nil {
if err := strs.UnmarshalJSON([]byte(input)); err != nil {
t.Fatal(err)
}
actualParts := []string(strs)
if !reflect.DeepEqual(actualParts, expectedParts) {
t.Fatalf("%#v: expected %v, got %v", json, expectedParts, actualParts)
if !reflect.DeepEqual(actualParts, expected) {
t.Fatalf("%#v: expected %v, got %v", input, expected, actualParts)
}
}
}

View file

@ -100,8 +100,10 @@ func GetTimestamp(value string, reference time.Time) (string, error) {
// if the incoming nanosecond portion is longer or shorter than 9 digits it is
// converted to nanoseconds. The expectation is that the seconds and
// seconds will be used to create a time variable. For example:
// seconds, nanoseconds, err := ParseTimestamp("1136073600.000000001",0)
// if err == nil since := time.Unix(seconds, nanoseconds)
//
// seconds, nanoseconds, err := ParseTimestamp("1136073600.000000001",0)
// if err == nil since := time.Unix(seconds, nanoseconds)
//
// returns seconds as def(aultSeconds) if value == ""
func ParseTimestamps(value string, def int64) (int64, int64, error) {
if value == "" {

View file

@ -774,18 +774,31 @@ type BuildResult struct {
ID string
}
// BuildCache contains information about a build cache record
// BuildCache contains information about a build cache record.
type BuildCache struct {
ID string
Parent string
Type string
// ID is the unique ID of the build cache record.
ID string
// Parent is the ID of the parent build cache record.
//
// Deprecated: deprecated in API v1.42 and up, as it was deprecated in BuildKit; use Parents instead.
Parent string `json:"Parent,omitempty"`
// Parents is the list of parent build cache record IDs.
Parents []string `json:" Parents,omitempty"`
// Type is the cache record type.
Type string
// Description is a description of the build-step that produced the build cache.
Description string
InUse bool
Shared bool
Size int64
CreatedAt time.Time
LastUsedAt *time.Time
UsageCount int
// InUse indicates if the build cache is in use.
InUse bool
// Shared indicates if the build cache is shared.
Shared bool
// Size is the amount of disk space used by the build cache (in bytes).
Size int64
// CreatedAt is the date and time at which the build cache was created.
CreatedAt time.Time
// LastUsedAt is the date and time at which the build cache was last used.
LastUsedAt *time.Time
UsageCount int
}
// BuildCachePruneOptions hold parameters to prune the build cache

View file

@ -104,7 +104,7 @@ type AccessMode struct {
BlockVolume *TypeBlock `json:",omitempty"`
}
// Scope defines the Scope of a CSI Volume. This is how many nodes a
// Scope defines the Scope of a Cluster Volume. This is how many nodes a
// Volume can be accessed simultaneously on.
type Scope string
@ -118,7 +118,7 @@ const (
ScopeMultiNode Scope = "multi"
)
// SharingMode defines the Sharing of a CSI Volume. This is how Tasks using a
// SharingMode defines the Sharing of a Cluster Volume. This is how Tasks using a
// Volume at the same time can use it.
type SharingMode string

View file

@ -177,7 +177,7 @@ func (is *Source) Resolve(ctx context.Context, id source.Identifier, sm *session
p := &puller{
src: imageIdentifier,
is: is,
//resolver: is.getResolver(is.RegistryHosts, imageIdentifier.Reference.String(), sm, g),
// resolver: is.getResolver(is.RegistryHosts, imageIdentifier.Reference.String(), sm, g),
platform: platform,
sm: sm,
}
@ -439,7 +439,6 @@ func (p *puller) Snapshot(ctx context.Context, g session.Group) (cache.Immutable
// TODO: Optimize to do dispatch and integrate pulling with download manager,
// leverage existing blob mapping and layer storage
} else {
// TODO: need a wrapper snapshot interface that combines content
// and snapshots as 1) buildkit shouldn't have a dependency on contentstore
// or 2) cachemanager should manage the contentstore

View file

@ -24,7 +24,6 @@ import (
// ResolveCacheImporterFunc returns a resolver function for local inline cache
func ResolveCacheImporterFunc(sm *session.Manager, resolverFunc docker.RegistryHosts, cs content.Store, rs reference.Store, is imagestore.Store) remotecache.ResolveCacheImporterFunc {
upstream := registryremotecache.ResolveCacheImporterFunc(sm, cs, resolverFunc)
return func(ctx context.Context, group session.Group, attrs map[string]string) (remotecache.Importer, specs.Descriptor, error) {

View file

@ -22,6 +22,9 @@ func (s *snapshotter) GetDiffIDs(ctx context.Context, key string) ([]layer.DiffI
}
func (s *snapshotter) EnsureLayer(ctx context.Context, key string) ([]layer.DiffID, error) {
s.layerCreateLocker.Lock(key)
defer s.layerCreateLocker.Unlock(key)
diffIDs, err := s.GetDiffIDs(ctx, key)
if err != nil {
return nil, err

View file

@ -16,6 +16,7 @@ import (
"github.com/docker/docker/pkg/idtools"
"github.com/moby/buildkit/identity"
"github.com/moby/buildkit/snapshot"
"github.com/moby/locker"
"github.com/opencontainers/go-digest"
"github.com/pkg/errors"
bolt "go.etcd.io/bbolt"
@ -48,10 +49,11 @@ type checksumCalculator interface {
type snapshotter struct {
opt Opt
refs map[string]layer.Layer
db *bolt.DB
mu sync.Mutex
reg graphIDRegistrar
refs map[string]layer.Layer
db *bolt.DB
mu sync.Mutex
reg graphIDRegistrar
layerCreateLocker *locker.Locker
}
// NewSnapshotter creates a new snapshotter
@ -68,10 +70,11 @@ func NewSnapshotter(opt Opt, prevLM leases.Manager) (snapshot.Snapshotter, lease
}
s := &snapshotter{
opt: opt,
db: db,
refs: map[string]layer.Layer{},
reg: reg,
opt: opt,
db: db,
refs: map[string]layer.Layer{},
reg: reg,
layerCreateLocker: locker.New(),
}
lm := newLeaseManager(s, prevLM)

View file

@ -15,9 +15,11 @@ import (
"github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/backend"
"github.com/docker/docker/builder"
containerimageexp "github.com/docker/docker/builder/builder-next/exporter"
"github.com/docker/docker/daemon/config"
"github.com/docker/docker/daemon/images"
"github.com/docker/docker/libnetwork"
"github.com/docker/docker/opts"
"github.com/docker/docker/pkg/idtools"
"github.com/docker/docker/pkg/streamformatter"
"github.com/docker/go-units"
@ -67,8 +69,10 @@ var cacheFields = map[string]bool{
type Opt struct {
SessionManager *session.Manager
Root string
EngineID string
Dist images.DistributionServices
NetworkController libnetwork.NetworkController
ImageTagger containerimageexp.ImageTagger
DefaultCgroupParent string
RegistryHosts docker.RegistryHosts
BuilderConfig config.BuilderConfig
@ -81,6 +85,7 @@ type Opt struct {
// Builder can build using BuildKit backend
type Builder struct {
controller *control.Controller
dnsconfig config.DNSConfig
reqBodyHandler *reqBodyHandler
mu sync.Mutex
@ -97,6 +102,7 @@ func New(opt Opt) (*Builder, error) {
}
b := &Builder{
controller: c,
dnsconfig: opt.DNSConfig,
reqBodyHandler: reqHandler,
jobs: map[string]*buildJob{},
}
@ -129,7 +135,8 @@ func (b *Builder) DiskUsage(ctx context.Context) ([]*types.BuildCache, error) {
for _, r := range duResp.Record {
items = append(items, &types.BuildCache{
ID: r.ID,
Parent: r.Parent,
Parent: r.Parent, //nolint:staticcheck // ignore SA1019 (Parent field is deprecated)
Parents: r.Parents,
Type: r.RecordType,
Description: r.Description,
InUse: r.InUse,
@ -310,7 +317,7 @@ func (b *Builder) Build(ctx context.Context, opt backend.BuildConfig) (*builder.
return nil, errors.Errorf("network mode %q not supported by buildkit", opt.Options.NetworkMode)
}
extraHosts, err := toBuildkitExtraHosts(opt.Options.ExtraHosts)
extraHosts, err := toBuildkitExtraHosts(opt.Options.ExtraHosts, b.dnsconfig.HostGatewayIP)
if err != nil {
return nil, err
}
@ -550,18 +557,28 @@ func (j *buildJob) SetUpload(ctx context.Context, rc io.ReadCloser) error {
}
// toBuildkitExtraHosts converts hosts from docker key:value format to buildkit's csv format
func toBuildkitExtraHosts(inp []string) (string, error) {
func toBuildkitExtraHosts(inp []string, hostGatewayIP net.IP) (string, error) {
if len(inp) == 0 {
return "", nil
}
hosts := make([]string, 0, len(inp))
for _, h := range inp {
parts := strings.Split(h, ":")
if len(parts) != 2 || parts[0] == "" || net.ParseIP(parts[1]) == nil {
host, ip, ok := strings.Cut(h, ":")
if !ok || host == "" || ip == "" {
return "", errors.Errorf("invalid host %s", h)
}
hosts = append(hosts, parts[0]+"="+parts[1])
// If the IP Address is a "host-gateway", replace this value with the
// IP address stored in the daemon level HostGatewayIP config variable.
if ip == opts.HostGatewayName {
gateway := hostGatewayIP.String()
if gateway == "" {
return "", fmt.Errorf("unable to derive the IP value for host-gateway")
}
ip = gateway
} else if net.ParseIP(ip) == nil {
return "", fmt.Errorf("invalid host %s", h)
}
hosts = append(hosts, host+"="+ip)
}
return strings.Join(hosts, ","), nil
}

View file

@ -17,6 +17,7 @@ import (
containerimageexp "github.com/docker/docker/builder/builder-next/exporter"
"github.com/docker/docker/builder/builder-next/imagerefchecker"
mobyworker "github.com/docker/docker/builder/builder-next/worker"
wlabel "github.com/docker/docker/builder/builder-next/worker/label"
"github.com/docker/docker/daemon/config"
"github.com/docker/docker/daemon/graphdriver"
units "github.com/docker/go-units"
@ -39,6 +40,9 @@ import (
"github.com/moby/buildkit/worker"
"github.com/pkg/errors"
bolt "go.etcd.io/bbolt"
"github.com/moby/buildkit/solver/pb"
"github.com/moby/buildkit/util/apicaps"
)
func newController(rt http.RoundTripper, opt Opt) (*control.Controller, error) {
@ -49,6 +53,18 @@ func newController(rt http.RoundTripper, opt Opt) (*control.Controller, error) {
dist := opt.Dist
root := opt.Root
pb.Caps.Init(apicaps.Cap{
ID: pb.CapMergeOp,
Enabled: false,
DisabledReasonMsg: "only enabled with containerd image store backend",
})
pb.Caps.Init(apicaps.Cap{
ID: pb.CapDiffOp,
Enabled: false,
DisabledReasonMsg: "only enabled with containerd image store backend",
})
var driver graphdriver.Driver
if ls, ok := dist.LayerStore.(interface {
Driver() graphdriver.Driver
@ -144,9 +160,9 @@ func newController(rt http.RoundTripper, opt Opt) (*control.Controller, error) {
}
exp, err := containerimageexp.New(containerimageexp.Opt{
ImageStore: dist.ImageStore,
ReferenceStore: dist.ReferenceStore,
Differ: differ,
ImageStore: dist.ImageStore,
Differ: differ,
ImageTagger: opt.ImageTagger,
})
if err != nil {
return nil, err
@ -176,7 +192,7 @@ func newController(rt http.RoundTripper, opt Opt) (*control.Controller, error) {
}
wopt := mobyworker.Opt{
ID: "moby",
ID: opt.EngineID,
ContentStore: store,
CacheManager: cm,
GCPolicy: gcPolicy,
@ -189,6 +205,7 @@ func newController(rt http.RoundTripper, opt Opt) (*control.Controller, error) {
Transport: rt,
Layers: layers,
Platforms: archutil.SupportedPlatforms(true),
Labels: getLabels(opt, nil),
}
wc := &worker.Controller{}
@ -271,3 +288,11 @@ func getEntitlements(conf config.BuilderConfig) []string {
}
return ents
}
func getLabels(opt Opt, labels map[string]string) map[string]string {
if labels == nil {
labels = make(map[string]string)
}
labels[wlabel.HostGatewayIP] = opt.DNSConfig.HostGatewayIP.String()
return labels
}

View file

@ -128,7 +128,7 @@ func (iface *lnInterface) Set(s *specs.Spec) error {
s.Hooks = &specs.Hooks{
Prestart: []specs.Hook{{
Path: filepath.Join("/proc", strconv.Itoa(os.Getpid()), "exe"),
Args: []string{"libnetwork-setkey", "-exec-root=" + iface.provider.Config().Daemon.ExecRoot, iface.sbx.ContainerID(), shortNetCtlrID},
Args: []string{"libnetwork-setkey", "-exec-root=" + iface.provider.Config().ExecRoot, iface.sbx.ContainerID(), shortNetCtlrID},
}},
}
return nil

View file

@ -7,10 +7,10 @@ import (
"strconv"
"strings"
"github.com/docker/distribution/reference"
distref "github.com/docker/distribution/reference"
"github.com/docker/docker/image"
"github.com/docker/docker/layer"
"github.com/docker/docker/reference"
"github.com/moby/buildkit/exporter"
"github.com/moby/buildkit/exporter/containerimage/exptypes"
"github.com/moby/buildkit/util/compression"
@ -29,11 +29,15 @@ type Differ interface {
EnsureLayer(ctx context.Context, key string) ([]layer.DiffID, error)
}
type ImageTagger interface {
TagImageWithReference(imageID image.ID, newTag reference.Named) error
}
// Opt defines a struct for creating new exporter
type Opt struct {
ImageStore image.Store
ReferenceStore reference.Store
Differ Differ
ImageStore image.Store
Differ Differ
ImageTagger ImageTagger
}
type imageExporter struct {
@ -206,10 +210,10 @@ func (e *imageExporterInstance) Export(ctx context.Context, inp exporter.Source,
}
_ = configDone(nil)
if e.opt.ReferenceStore != nil {
if e.opt.ImageTagger != nil {
for _, targetName := range e.targetNames {
tagDone := oneOffProgress(ctx, "naming to "+targetName.String())
if err := e.opt.ReferenceStore.AddTag(targetName, digest.Digest(id), true); err != nil {
if err := e.opt.ImageTagger.TagImageWithReference(image.ID(digest.Digest(id)), targetName); err != nil {
return nil, tagDone(err)
}
_ = tagDone(nil)

View file

@ -22,11 +22,10 @@ import (
func emptyImageConfig() ([]byte, error) {
pl := platforms.Normalize(platforms.DefaultSpec())
img := ocispec.Image{
Architecture: pl.Architecture,
OS: pl.OS,
Variant: pl.Variant,
}
img := ocispec.Image{}
img.Architecture = pl.Architecture
img.OS = pl.OS
img.Variant = pl.Variant
img.RootFS.Type = "layers"
img.Config.WorkingDir = "/"
img.Config.Env = []string{"PATH=" + system.DefaultPathEnv(pl.OS)}

View file

@ -2,6 +2,7 @@ package worker
import (
"math"
"time"
"github.com/moby/buildkit/client"
)
@ -30,12 +31,12 @@ func DefaultGCPolicy(p string, defaultKeepBytes int64) []client.PruneInfo {
// if build cache uses more than 512MB delete the most easily reproducible data after it has not been used for 2 days
{
Filter: []string{"type==source.local,type==exec.cachemount,type==source.git.checkout"},
KeepDuration: 48 * 3600, // 48h
KeepDuration: 48 * time.Hour,
KeepBytes: tempCacheKeepBytes,
},
// remove any data not used for 60 days
{
KeepDuration: 60 * 24 * 3600, // 60d
KeepDuration: 60 * 24 * time.Hour,
KeepBytes: keep,
},
// keep the unshared build cache under cap

View file

@ -0,0 +1,9 @@
package label
// Pre-defined label keys similar to BuildKit ones
// https://github.com/moby/buildkit/blob/v0.11.6/worker/label/label.go#L3-L16
const (
prefix = "org.mobyproject.buildkit.worker.moby."
HostGatewayIP = prefix + "host-gateway-ip"
)

View file

@ -18,6 +18,7 @@ import (
"github.com/docker/docker/layer"
pkgprogress "github.com/docker/docker/pkg/progress"
"github.com/moby/buildkit/cache"
cacheconfig "github.com/moby/buildkit/cache/config"
"github.com/moby/buildkit/client"
"github.com/moby/buildkit/client/llb"
"github.com/moby/buildkit/executor"
@ -36,7 +37,6 @@ import (
"github.com/moby/buildkit/source/http"
"github.com/moby/buildkit/source/local"
"github.com/moby/buildkit/util/archutil"
"github.com/moby/buildkit/util/compression"
"github.com/moby/buildkit/util/contentutil"
"github.com/moby/buildkit/util/progress"
"github.com/opencontainers/go-digest"
@ -79,6 +79,10 @@ type Worker struct {
SourceManager *source.Manager
}
var _ interface {
GetRemotes(context.Context, cache.ImmutableRef, bool, cacheconfig.RefConfig, bool, session.Group) ([]*solver.Remote, error)
} = &Worker{}
// NewWorker instantiates a local worker
func NewWorker(opt Opt) (*Worker, error) {
sm, err := source.NewManager()
@ -227,8 +231,11 @@ func (w *Worker) Exporter(name string, sm *session.Manager) (exporter.Exporter,
}
}
// GetRemote returns a remote snapshot reference for a local one
func (w *Worker) GetRemote(ctx context.Context, ref cache.ImmutableRef, createIfNeeded bool, _ compression.Type, s session.Group) (*solver.Remote, error) {
// GetRemotes returns the remote snapshot references given a local reference
func (w *Worker) GetRemotes(ctx context.Context, ref cache.ImmutableRef, createIfNeeded bool, _ cacheconfig.RefConfig, all bool, s session.Group) ([]*solver.Remote, error) {
if ref == nil {
return nil, nil
}
var diffIDs []layer.DiffID
var err error
if !createIfNeeded {
@ -258,10 +265,10 @@ func (w *Worker) GetRemote(ctx context.Context, ref cache.ImmutableRef, createIf
}
}
return &solver.Remote{
return []*solver.Remote{{
Descriptors: descriptors,
Provider: &emptyProvider{},
}, nil
}}, nil
}
// PruneCacheMounts removes the current cache snapshots for specified IDs

View file

@ -15,6 +15,7 @@ import (
"github.com/docker/docker/image"
"github.com/docker/docker/layer"
"github.com/docker/docker/pkg/containerfs"
ocispec "github.com/opencontainers/image-spec/specs-go/v1"
)
const (
@ -89,7 +90,7 @@ type ImageCacheBuilder interface {
type ImageCache interface {
// GetCache returns a reference to a cached image whose parent equals `parent`
// and runconfig equals `cfg`. A cache miss is expected to return an empty ID and a nil error.
GetCache(parentID string, cfg *container.Config) (imageID string, err error)
GetCache(parentID string, cfg *container.Config, platform ocispec.Platform) (imageID string, err error)
}
// Image represents a Docker image used by the builder.

View file

@ -295,7 +295,6 @@ func (b *Builder) dispatchDockerfileWithCancellation(parseResult []instructions.
}
dispatchRequest.state.updateRunConfig()
fmt.Fprintf(b.Stdout, " ---> %s\n", stringid.TruncateID(dispatchRequest.state.imageID))
}
if err := emitImageID(b.Aux, dispatchRequest.state); err != nil {
return nil, err

View file

@ -9,7 +9,6 @@ import (
"net/url"
"os"
"path/filepath"
"runtime"
"sort"
"strings"
"time"
@ -25,7 +24,7 @@ import (
"github.com/docker/docker/pkg/streamformatter"
"github.com/docker/docker/pkg/system"
"github.com/moby/buildkit/frontend/dockerfile/instructions"
specs "github.com/opencontainers/image-spec/specs-go/v1"
ocispec "github.com/opencontainers/image-spec/specs-go/v1"
"github.com/pkg/errors"
)
@ -75,7 +74,7 @@ type copier struct {
source builder.Source
pathCache pathCache
download sourceDownloader
platform *specs.Platform
platform ocispec.Platform
// for cleanup. TODO: having copier.cleanup() is error prone and hard to
// follow. Code calling performCopy should manage the lifecycle of its params.
// Copier should take override source as input, not imageMount.
@ -84,19 +83,7 @@ type copier struct {
}
func copierFromDispatchRequest(req dispatchRequest, download sourceDownloader, imageSource *imageMount) copier {
platform := req.builder.platform
if platform == nil {
// May be nil if not explicitly set in API/dockerfile
platform = &specs.Platform{}
}
if platform.OS == "" {
// Default to the dispatch requests operating system if not explicit in API/dockerfile
platform.OS = req.state.operatingSystem
}
if platform.OS == "" {
// This is a failsafe just in case. Shouldn't be hit.
platform.OS = runtime.GOOS
}
platform := req.builder.getPlatform(req.state)
return copier{
source: req.source,
@ -105,7 +92,6 @@ func copierFromDispatchRequest(req dispatchRequest, download sourceDownloader, i
imageSource: imageSource,
platform: platform,
}
}
func (o *copier) createCopyInstruction(sourcesAndDest instructions.SourcesAndDest, cmdName string) (copyInstruction, error) {

View file

@ -35,7 +35,6 @@ import (
//
// Sets the environment variable foo to bar, also makes interpolation
// in the dockerfile available from the next statement on via ${foo}.
//
func dispatchEnv(d dispatchRequest, c *instructions.EnvCommand) error {
runConfig := d.state.runConfig
commitMessage := bytes.NewBufferString("ENV")
@ -65,7 +64,6 @@ func dispatchEnv(d dispatchRequest, c *instructions.EnvCommand) error {
//
// Sets the maintainer metadata.
func dispatchMaintainer(d dispatchRequest, c *instructions.MaintainerCommand) error {
d.state.maintainer = c.Maintainer
return d.builder.commit(d.state, "MAINTAINER "+c.Maintainer)
}
@ -73,7 +71,6 @@ func dispatchMaintainer(d dispatchRequest, c *instructions.MaintainerCommand) er
// LABEL some json data describing the image
//
// Sets the Label variable foo to bar,
//
func dispatchLabel(d dispatchRequest, c *instructions.LabelCommand) error {
if d.state.runConfig.Labels == nil {
d.state.runConfig.Labels = make(map[string]string)
@ -90,7 +87,6 @@ func dispatchLabel(d dispatchRequest, c *instructions.LabelCommand) error {
//
// Add the file 'foo' to '/path'. Tarball and Remote URL (http, https) handling
// exist here. If you do not wish to have this automatic handling, use COPY.
//
func dispatchAdd(d dispatchRequest, c *instructions.AddCommand) error {
if c.Chmod != "" {
return errors.New("the --chmod option requires BuildKit. Refer to https://docs.docker.com/go/buildkit/ to learn how to build images with BuildKit enabled")
@ -112,7 +108,6 @@ func dispatchAdd(d dispatchRequest, c *instructions.AddCommand) error {
// COPY foo /path
//
// Same as 'ADD' but without the tar and remote url handling.
//
func dispatchCopy(d dispatchRequest, c *instructions.CopyCommand) error {
if c.Chmod != "" {
return errors.New("the --chmod option requires BuildKit. Refer to https://docs.docker.com/go/buildkit/ to learn how to build images with BuildKit enabled")
@ -157,7 +152,6 @@ func (d *dispatchRequest) getImageMount(imageRefOrID string) (*imageMount, error
}
// FROM [--platform=platform] imagename[:tag | @digest] [AS build-stage-name]
//
func initializeStage(d dispatchRequest, cmd *instructions.Stage) error {
d.builder.imageProber.Reset()
@ -290,7 +284,6 @@ func dispatchOnbuild(d dispatchRequest, c *instructions.OnbuildCommand) error {
// WORKDIR /tmp
//
// Set the working directory for future RUN/CMD/etc statements.
//
func dispatchWorkdir(d dispatchRequest, c *instructions.WorkdirCommand) error {
runConfig := d.state.runConfig
var err error
@ -333,7 +326,6 @@ func dispatchWorkdir(d dispatchRequest, c *instructions.WorkdirCommand) error {
// RUN echo hi # sh -c echo hi (Linux and LCOW)
// RUN echo hi # cmd /S /C echo hi (Windows)
// RUN [ "echo", "hi" ] # echo hi
//
func dispatchRun(d dispatchRequest, c *instructions.RunCommand) error {
if !system.IsOSSupported(d.state.operatingSystem) {
return system.ErrNotSupportedOperatingSystem
@ -353,9 +345,16 @@ func dispatchRun(d dispatchRequest, c *instructions.RunCommand) error {
saveCmd = prependEnvOnCmd(d.state.buildArgs, buildArgs, cmdFromArgs)
}
cacheArgsEscaped := argsEscaped
// ArgsEscaped is not persisted in the committed image on Windows.
// Use the original from previous build steps for cache probing.
if d.state.operatingSystem == "windows" {
cacheArgsEscaped = stateRunConfig.ArgsEscaped
}
runConfigForCacheProbe := copyRunConfig(stateRunConfig,
withCmd(saveCmd),
withArgsEscaped(argsEscaped),
withArgsEscaped(cacheArgsEscaped),
withEntrypointOverride(saveCmd, nil))
if hit, err := d.builder.probeCache(d.state, runConfigForCacheProbe); err != nil || hit {
return err
@ -428,7 +427,6 @@ func prependEnvOnCmd(buildArgs *BuildArgs, buildArgVars []string, cmd strslice.S
//
// Set the default command to run in the container (which may be empty).
// Argument handling is the same as RUN.
//
func dispatchCmd(d dispatchRequest, c *instructions.CmdCommand) error {
runConfig := d.state.runConfig
cmd, argsEscaped := resolveCmdLine(c.ShellDependantCmdLine, runConfig, d.state.operatingSystem, c.Name(), c.String())
@ -459,7 +457,6 @@ func dispatchCmd(d dispatchRequest, c *instructions.CmdCommand) error {
//
// Set the default healthcheck command to run in the container (which may be empty).
// Argument handling is the same as RUN.
//
func dispatchHealthcheck(d dispatchRequest, c *instructions.HealthCheckCommand) error {
runConfig := d.state.runConfig
if runConfig.Healthcheck != nil {
@ -479,7 +476,6 @@ func dispatchHealthcheck(d dispatchRequest, c *instructions.HealthCheckCommand)
//
// Handles command processing similar to CMD and RUN, only req.runConfig.Entrypoint
// is initialized at newBuilder time instead of through argument parsing.
//
func dispatchEntrypoint(d dispatchRequest, c *instructions.EntrypointCommand) error {
runConfig := d.state.runConfig
cmd, argsEscaped := resolveCmdLine(c.ShellDependantCmdLine, runConfig, d.state.operatingSystem, c.Name(), c.String())
@ -509,7 +505,6 @@ func dispatchEntrypoint(d dispatchRequest, c *instructions.EntrypointCommand) er
//
// Expose ports for links and port mappings. This all ends up in
// req.runConfig.ExposedPorts for runconfig.
//
func dispatchExpose(d dispatchRequest, c *instructions.ExposeCommand, envs []string) error {
// custom multi word expansion
// expose $FOO with FOO="80 443" is expanded as EXPOSE [80,443]. This is the only command supporting word to words expansion
@ -543,7 +538,6 @@ func dispatchExpose(d dispatchRequest, c *instructions.ExposeCommand, envs []str
//
// Set the user to 'foo' for future commands and when running the
// ENTRYPOINT/CMD at container run time.
//
func dispatchUser(d dispatchRequest, c *instructions.UserCommand) error {
d.state.runConfig.User = c.User
return d.builder.commit(d.state, fmt.Sprintf("USER %v", c.User))
@ -552,7 +546,6 @@ func dispatchUser(d dispatchRequest, c *instructions.UserCommand) error {
// VOLUME /foo
//
// Expose the volume /foo for use. Will also accept the JSON array form.
//
func dispatchVolume(d dispatchRequest, c *instructions.VolumeCommand) error {
if d.state.runConfig.Volumes == nil {
d.state.runConfig.Volumes = map[string]struct{}{}
@ -570,7 +563,6 @@ func dispatchVolume(d dispatchRequest, c *instructions.VolumeCommand) error {
//
// Set the signal that will be used to kill the container.
func dispatchStopSignal(d dispatchRequest, c *instructions.StopSignalCommand) error {
_, err := signal.ParseSignal(c.Signal)
if err != nil {
return errdefs.InvalidParameter(err)

View file

@ -284,7 +284,6 @@ func TestHealthcheckNone(t *testing.T) {
}
func TestHealthcheckCmd(t *testing.T) {
b := newBuilderWithMockBackend()
sb := newDispatchRequest(b, '`', nil, NewBuildArgs(make(map[string]*string)), newStagesBuildResults())
expectedTest := []string{"CMD-SHELL", "curl -f http://localhost/ || exit 1"}

View file

@ -3,6 +3,7 @@ package dockerfile // import "github.com/docker/docker/builder/dockerfile"
import (
"github.com/docker/docker/api/types/container"
"github.com/docker/docker/builder"
ocispec "github.com/opencontainers/image-spec/specs-go/v1"
"github.com/sirupsen/logrus"
)
@ -10,7 +11,7 @@ import (
// cache.
type ImageProber interface {
Reset()
Probe(parentID string, runConfig *container.Config) (string, error)
Probe(parentID string, runConfig *container.Config, platform ocispec.Platform) (string, error)
}
type imageProber struct {
@ -37,11 +38,11 @@ func (c *imageProber) Reset() {
// Probe checks if cache match can be found for current build instruction.
// It returns the cachedID if there is a hit, and the empty string on miss
func (c *imageProber) Probe(parentID string, runConfig *container.Config) (string, error) {
func (c *imageProber) Probe(parentID string, runConfig *container.Config, platform ocispec.Platform) (string, error) {
if c.cacheBusted {
return "", nil
}
cacheID, err := c.cache.GetCache(parentID, runConfig)
cacheID, err := c.cache.GetCache(parentID, runConfig, platform)
if err != nil {
return "", err
}
@ -58,6 +59,6 @@ type nopProber struct{}
func (c *nopProber) Reset() {}
func (c *nopProber) Probe(_ string, _ *container.Config) (string, error) {
func (c *nopProber) Probe(_ string, _ *container.Config, _ ocispec.Platform) (string, error) {
return "", nil
}

View file

@ -10,6 +10,7 @@ import (
"io"
"strings"
"github.com/containerd/containerd/platforms"
"github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/backend"
"github.com/docker/docker/api/types/container"
@ -364,7 +365,7 @@ func getShell(c *container.Config, os string) []string {
}
func (b *Builder) probeCache(dispatchState *dispatchState, runConfig *container.Config) (bool, error) {
cachedID, err := b.imageProber.Probe(dispatchState.imageID, runConfig)
cachedID, err := b.imageProber.Probe(dispatchState.imageID, runConfig, b.getPlatform(dispatchState))
if cachedID == "" || err != nil {
return false, err
}
@ -424,3 +425,17 @@ func hostConfigFromOptions(options *types.ImageBuildOptions) *container.HostConf
}
return hc
}
func (b *Builder) getPlatform(state *dispatchState) specs.Platform {
// May be nil if not explicitly set in API/dockerfile
out := platforms.DefaultSpec()
if b.platform != nil {
out = *b.platform
}
if state.operatingSystem != "" {
out.OS = state.operatingSystem
}
return out
}

View file

@ -140,7 +140,6 @@ func TestCopyRunConfig(t *testing.T) {
// Assert the original was not modified
assert.Check(t, runConfig != runConfigCopy, testcase.doc)
}
}
func fullMutableRunConfig() *container.Config {

View file

@ -14,6 +14,7 @@ import (
"github.com/docker/docker/image"
"github.com/docker/docker/layer"
"github.com/docker/docker/pkg/containerfs"
ocispec "github.com/opencontainers/image-spec/specs-go/v1"
)
// MockBackend implements the builder.Backend interface for unit testing
@ -111,7 +112,7 @@ type mockImageCache struct {
getCacheFunc func(parentID string, cfg *container.Config) (string, error)
}
func (mic *mockImageCache) GetCache(parentID string, cfg *container.Config) (string, error) {
func (mic *mockImageCache) GetCache(parentID string, cfg *container.Config, _ ocispec.Platform) (string, error) {
if mic.getCacheFunc != nil {
return mic.getCacheFunc(parentID, cfg)
}

View file

@ -13,9 +13,9 @@ import (
"github.com/docker/docker/builder"
"github.com/docker/docker/builder/remotecontext/urlutil"
"github.com/docker/docker/errdefs"
"github.com/docker/docker/pkg/fileutils"
"github.com/moby/buildkit/frontend/dockerfile/dockerignore"
"github.com/moby/buildkit/frontend/dockerfile/parser"
"github.com/moby/patternmatcher"
"github.com/pkg/errors"
"github.com/sirupsen/logrus"
)
@ -130,7 +130,7 @@ func removeDockerfile(c modifiableContext, filesToRemove ...string) error {
f.Close()
filesToRemove = append([]string{".dockerignore"}, filesToRemove...)
for _, fileToRemove := range filesToRemove {
if rm, _ := fileutils.MatchesOrParentMatches(fileToRemove, excludes); rm {
if rm, _ := patternmatcher.MatchesOrParentMatches(fileToRemove, excludes); rm {
if err := c.Remove(fileToRemove); err != nil {
logrus.Errorf("failed to remove %s: %v", fileToRemove, err)
}

View file

@ -72,7 +72,6 @@ func TestProcessShouldRemoveDockerfileDockerignore(t *testing.T) {
executeProcess(t, contextDir)
checkDirectory(t, contextDir, []string{shouldStayFilename})
}
func TestProcessNoDockerignore(t *testing.T) {
@ -85,7 +84,6 @@ func TestProcessNoDockerignore(t *testing.T) {
executeProcess(t, contextDir)
checkDirectory(t, contextDir, []string{shouldStayFilename, builder.DefaultDockerfileName})
}
func TestProcessShouldLeaveAllFiles(t *testing.T) {
@ -99,7 +97,6 @@ func TestProcessShouldLeaveAllFiles(t *testing.T) {
executeProcess(t, contextDir)
checkDirectory(t, contextDir, []string{shouldStayFilename, builder.DefaultDockerfileName, dockerignoreFilename})
}
// TODO: remove after moving to a separate pkg

View file

@ -11,7 +11,7 @@ import (
// MakeGitContext returns a Context from gitURL that is cloned in a temporary directory.
func MakeGitContext(gitURL string) (builder.Source, error) {
root, err := git.Clone(gitURL)
root, err := git.Clone(gitURL, git.WithIsolatedConfig(true))
if err != nil {
return nil, err
}

View file

@ -16,21 +16,38 @@ type gitRepo struct {
remote string
ref string
subdir string
isolateConfig bool
}
// CloneOption changes the behaviour of Clone().
type CloneOption func(*gitRepo)
// WithIsolatedConfig disables reading the user or system gitconfig files when
// performing Git operations.
func WithIsolatedConfig(v bool) CloneOption {
return func(gr *gitRepo) {
gr.isolateConfig = v
}
}
// Clone clones a repository into a newly created directory which
// will be under "docker-build-git"
func Clone(remoteURL string) (string, error) {
func Clone(remoteURL string, opts ...CloneOption) (string, error) {
repo, err := parseRemoteURL(remoteURL)
if err != nil {
return "", err
}
return cloneGitRepo(repo)
for _, opt := range opts {
opt(&repo)
}
return repo.clone()
}
func cloneGitRepo(repo gitRepo) (checkoutDir string, err error) {
func (repo gitRepo) clone() (checkoutDir string, err error) {
fetch := fetchArgs(repo.remote, repo.ref)
root, err := os.MkdirTemp("", "docker-build-git")
@ -44,21 +61,21 @@ func cloneGitRepo(repo gitRepo) (checkoutDir string, err error) {
}
}()
if out, err := gitWithinDir(root, "init"); err != nil {
if out, err := repo.gitWithinDir(root, "init"); err != nil {
return "", errors.Wrapf(err, "failed to init repo at %s: %s", root, out)
}
// Add origin remote for compatibility with previous implementation that
// used "git clone" and also to make sure local refs are created for branches
if out, err := gitWithinDir(root, "remote", "add", "origin", repo.remote); err != nil {
if out, err := repo.gitWithinDir(root, "remote", "add", "origin", repo.remote); err != nil {
return "", errors.Wrapf(err, "failed add origin repo at %s: %s", repo.remote, out)
}
if output, err := gitWithinDir(root, fetch...); err != nil {
if output, err := repo.gitWithinDir(root, fetch...); err != nil {
return "", errors.Wrapf(err, "error fetching: %s", output)
}
checkoutDir, err = checkoutGit(root, repo.ref, repo.subdir)
checkoutDir, err = repo.checkout(root)
if err != nil {
return "", err
}
@ -162,20 +179,20 @@ func supportsShallowClone(remoteURL string) bool {
return true
}
func checkoutGit(root, ref, subdir string) (string, error) {
func (repo gitRepo) checkout(root string) (string, error) {
// Try checking out by ref name first. This will work on branches and sets
// .git/HEAD to the current branch name
if output, err := gitWithinDir(root, "checkout", ref); err != nil {
if output, err := repo.gitWithinDir(root, "checkout", repo.ref); err != nil {
// If checking out by branch name fails check out the last fetched ref
if _, err2 := gitWithinDir(root, "checkout", "FETCH_HEAD"); err2 != nil {
return "", errors.Wrapf(err, "error checking out %s: %s", ref, output)
if _, err2 := repo.gitWithinDir(root, "checkout", "FETCH_HEAD"); err2 != nil {
return "", errors.Wrapf(err, "error checking out %s: %s", repo.ref, output)
}
}
if subdir != "" {
newCtx, err := symlink.FollowSymlinkInScope(filepath.Join(root, subdir), root)
if repo.subdir != "" {
newCtx, err := symlink.FollowSymlinkInScope(filepath.Join(root, repo.subdir), root)
if err != nil {
return "", errors.Wrapf(err, "error setting git context, %q not within git root", subdir)
return "", errors.Wrapf(err, "error setting git context, %q not within git root", repo.subdir)
}
fi, err := os.Stat(newCtx)
@ -191,13 +208,21 @@ func checkoutGit(root, ref, subdir string) (string, error) {
return root, nil
}
func gitWithinDir(dir string, args ...string) ([]byte, error) {
a := []string{"--work-tree", dir, "--git-dir", filepath.Join(dir, ".git")}
return git(append(a, args...)...)
}
func (repo gitRepo) gitWithinDir(dir string, args ...string) ([]byte, error) {
args = append([]string{"-c", "protocol.file.allow=never"}, args...) // Block sneaky repositories from using repos from the filesystem as submodules.
cmd := exec.Command("git", args...)
cmd.Dir = dir
// Disable unsafe remote protocols.
cmd.Env = append(os.Environ(), "GIT_PROTOCOL_FROM_USER=0")
func git(args ...string) ([]byte, error) {
return exec.Command("git", args...).CombinedOutput()
if repo.isolateConfig {
cmd.Env = append(cmd.Env,
"GIT_CONFIG_NOSYSTEM=1", // Disable reading from system gitconfig.
"HOME=/dev/null", // Disable reading from user gitconfig.
)
}
return cmd.CombinedOutput()
}
// isGitTransport returns true if the provided str is a git transport by inspecting

View file

@ -1,8 +1,10 @@
package git // import "github.com/docker/docker/builder/remotecontext/git"
import (
"bytes"
"fmt"
"net/http"
"net/http/cgi"
"net/http/httptest"
"net/url"
"os"
@ -160,7 +162,7 @@ func TestCloneArgsGit(t *testing.T) {
}
func gitGetConfig(name string) string {
b, err := git([]string{"config", "--get", name}...)
b, err := gitRepo{}.gitWithinDir("", "config", "--get", name)
if err != nil {
// since we are interested in empty or non empty string,
// we can safely ignore the err here.
@ -170,9 +172,50 @@ func gitGetConfig(name string) string {
}
func TestCheckoutGit(t *testing.T) {
root, err := os.MkdirTemp("", "docker-build-git-checkout")
root := t.TempDir()
gitpath, err := exec.LookPath("git")
assert.NilError(t, err)
defer os.RemoveAll(root)
gitversion, _ := exec.Command(gitpath, "version").CombinedOutput()
t.Logf("%s", gitversion) // E.g. "git version 2.30.2"
// Serve all repositories under root using the Smart HTTP protocol so
// they can be cloned. The Dumb HTTP protocol is incompatible with
// shallow cloning but we unconditionally shallow-clone submodules, and
// we explicitly disable the file protocol.
// (Another option would be to use `git daemon` and the Git protocol,
// but that listens on a fixed port number which is a recipe for
// disaster in CI. Funnily enough, `git daemon --port=0` works but there
// is no easy way to discover which port got picked!)
// Associate git-http-backend logs with the current (sub)test.
// Incompatible with parallel subtests.
currentSubtest := t
githttp := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
var logs bytes.Buffer
(&cgi.Handler{
Path: gitpath,
Args: []string{"http-backend"},
Dir: root,
Env: []string{
"GIT_PROJECT_ROOT=" + root,
"GIT_HTTP_EXPORT_ALL=1",
},
Stderr: &logs,
}).ServeHTTP(w, r)
if logs.Len() == 0 {
return
}
for {
line, err := logs.ReadString('\n')
currentSubtest.Log("git-http-backend: " + line)
if err != nil {
break
}
}
})
server := httptest.NewServer(&githttp)
defer server.Close()
autocrlf := gitGetConfig("core.autocrlf")
if !(autocrlf == "true" || autocrlf == "false" ||
@ -184,88 +227,54 @@ func TestCheckoutGit(t *testing.T) {
eol = "\r\n"
}
must := func(out []byte, err error) {
t.Helper()
if len(out) > 0 {
t.Logf("%s", out)
}
assert.NilError(t, err)
}
gitDir := filepath.Join(root, "repo")
_, err = git("init", gitDir)
assert.NilError(t, err)
_, err = gitWithinDir(gitDir, "config", "user.email", "test@docker.com")
assert.NilError(t, err)
_, err = gitWithinDir(gitDir, "config", "user.name", "Docker test")
assert.NilError(t, err)
err = os.WriteFile(filepath.Join(gitDir, "Dockerfile"), []byte("FROM scratch"), 0644)
assert.NilError(t, err)
must(gitRepo{}.gitWithinDir(root, "-c", "init.defaultBranch=master", "init", gitDir))
must(gitRepo{}.gitWithinDir(gitDir, "config", "user.email", "test@docker.com"))
must(gitRepo{}.gitWithinDir(gitDir, "config", "user.name", "Docker test"))
assert.NilError(t, os.WriteFile(filepath.Join(gitDir, "Dockerfile"), []byte("FROM scratch"), 0644))
subDir := filepath.Join(gitDir, "subdir")
assert.NilError(t, os.Mkdir(subDir, 0755))
err = os.WriteFile(filepath.Join(subDir, "Dockerfile"), []byte("FROM scratch\nEXPOSE 5000"), 0644)
assert.NilError(t, err)
assert.NilError(t, os.WriteFile(filepath.Join(subDir, "Dockerfile"), []byte("FROM scratch\nEXPOSE 5000"), 0644))
if runtime.GOOS != "windows" {
if err = os.Symlink("../subdir", filepath.Join(gitDir, "parentlink")); err != nil {
t.Fatal(err)
}
if err = os.Symlink("/subdir", filepath.Join(gitDir, "absolutelink")); err != nil {
t.Fatal(err)
}
assert.NilError(t, os.Symlink("../subdir", filepath.Join(gitDir, "parentlink")))
assert.NilError(t, os.Symlink("/subdir", filepath.Join(gitDir, "absolutelink")))
}
_, err = gitWithinDir(gitDir, "add", "-A")
assert.NilError(t, err)
must(gitRepo{}.gitWithinDir(gitDir, "add", "-A"))
must(gitRepo{}.gitWithinDir(gitDir, "commit", "-am", "First commit"))
must(gitRepo{}.gitWithinDir(gitDir, "checkout", "-b", "test"))
_, err = gitWithinDir(gitDir, "commit", "-am", "First commit")
assert.NilError(t, err)
assert.NilError(t, os.WriteFile(filepath.Join(gitDir, "Dockerfile"), []byte("FROM scratch\nEXPOSE 3000"), 0644))
assert.NilError(t, os.WriteFile(filepath.Join(subDir, "Dockerfile"), []byte("FROM busybox\nEXPOSE 5000"), 0644))
_, err = gitWithinDir(gitDir, "checkout", "-b", "test")
assert.NilError(t, err)
err = os.WriteFile(filepath.Join(gitDir, "Dockerfile"), []byte("FROM scratch\nEXPOSE 3000"), 0644)
assert.NilError(t, err)
err = os.WriteFile(filepath.Join(subDir, "Dockerfile"), []byte("FROM busybox\nEXPOSE 5000"), 0644)
assert.NilError(t, err)
_, err = gitWithinDir(gitDir, "add", "-A")
assert.NilError(t, err)
_, err = gitWithinDir(gitDir, "commit", "-am", "Branch commit")
assert.NilError(t, err)
_, err = gitWithinDir(gitDir, "checkout", "master")
assert.NilError(t, err)
must(gitRepo{}.gitWithinDir(gitDir, "add", "-A"))
must(gitRepo{}.gitWithinDir(gitDir, "commit", "-am", "Branch commit"))
must(gitRepo{}.gitWithinDir(gitDir, "checkout", "master"))
// set up submodule
subrepoDir := filepath.Join(root, "subrepo")
_, err = git("init", subrepoDir)
assert.NilError(t, err)
must(gitRepo{}.gitWithinDir(root, "-c", "init.defaultBranch=master", "init", subrepoDir))
must(gitRepo{}.gitWithinDir(subrepoDir, "config", "user.email", "test@docker.com"))
must(gitRepo{}.gitWithinDir(subrepoDir, "config", "user.name", "Docker test"))
_, err = gitWithinDir(subrepoDir, "config", "user.email", "test@docker.com")
assert.NilError(t, err)
assert.NilError(t, os.WriteFile(filepath.Join(subrepoDir, "subfile"), []byte("subcontents"), 0644))
_, err = gitWithinDir(subrepoDir, "config", "user.name", "Docker test")
assert.NilError(t, err)
must(gitRepo{}.gitWithinDir(subrepoDir, "add", "-A"))
must(gitRepo{}.gitWithinDir(subrepoDir, "commit", "-am", "Subrepo initial"))
err = os.WriteFile(filepath.Join(subrepoDir, "subfile"), []byte("subcontents"), 0644)
assert.NilError(t, err)
_, err = gitWithinDir(subrepoDir, "add", "-A")
assert.NilError(t, err)
_, err = gitWithinDir(subrepoDir, "commit", "-am", "Subrepo initial")
assert.NilError(t, err)
cmd := exec.Command("git", "submodule", "add", subrepoDir, "sub") // this command doesn't work with --work-tree
cmd.Dir = gitDir
assert.NilError(t, cmd.Run())
_, err = gitWithinDir(gitDir, "add", "-A")
assert.NilError(t, err)
_, err = gitWithinDir(gitDir, "commit", "-am", "With submodule")
assert.NilError(t, err)
must(gitRepo{}.gitWithinDir(gitDir, "submodule", "add", server.URL+"/subrepo", "sub"))
must(gitRepo{}.gitWithinDir(gitDir, "add", "-A"))
must(gitRepo{}.gitWithinDir(gitDir, "commit", "-am", "With submodule"))
type singleCase struct {
frag string
@ -299,28 +308,31 @@ func TestCheckoutGit(t *testing.T) {
}
for _, c := range cases {
ref, subdir := getRefAndSubdir(c.frag)
r, err := cloneGitRepo(gitRepo{remote: gitDir, ref: ref, subdir: subdir})
t.Run(c.frag, func(t *testing.T) {
currentSubtest = t
ref, subdir := getRefAndSubdir(c.frag)
r, err := gitRepo{remote: server.URL + "/repo", ref: ref, subdir: subdir}.clone()
if c.fail {
assert.Check(t, is.ErrorContains(err, ""))
continue
}
assert.NilError(t, err)
defer os.RemoveAll(r)
if c.submodule {
b, err := os.ReadFile(filepath.Join(r, "sub/subfile"))
if c.fail {
assert.Check(t, is.ErrorContains(err, ""))
return
}
assert.NilError(t, err)
assert.Check(t, is.Equal("subcontents", string(b)))
} else {
_, err := os.Stat(filepath.Join(r, "sub/subfile"))
assert.Assert(t, is.ErrorContains(err, ""))
assert.Assert(t, os.IsNotExist(err))
}
defer os.RemoveAll(r)
if c.submodule {
b, err := os.ReadFile(filepath.Join(r, "sub/subfile"))
assert.NilError(t, err)
assert.Check(t, is.Equal("subcontents", string(b)))
} else {
_, err := os.Stat(filepath.Join(r, "sub/subfile"))
assert.Assert(t, is.ErrorContains(err, ""))
assert.Assert(t, os.IsNotExist(err))
}
b, err := os.ReadFile(filepath.Join(r, "Dockerfile"))
assert.NilError(t, err)
assert.Check(t, is.Equal(c.exp, string(b)))
b, err := os.ReadFile(filepath.Join(r, "Dockerfile"))
assert.NilError(t, err)
assert.Check(t, is.Equal(c.exp, string(b)))
})
}
}

View file

@ -80,10 +80,10 @@ func GetWithStatusError(address string) (resp *http.Response, err error) {
// inspectResponse looks into the http response data at r to determine whether its
// content-type is on the list of acceptable content types for remote build contexts.
// This function returns:
// - a string representation of the detected content-type
// - an io.Reader for the response body
// - an error value which will be non-nil either when something goes wrong while
// reading bytes from r or when the detected content-type is not acceptable.
// - a string representation of the detected content-type
// - an io.Reader for the response body
// - an error value which will be non-nil either when something goes wrong while
// reading bytes from r or when the detected content-type is not acceptable.
func inspectResponse(ct string, r io.Reader, clen int64) (string, io.Reader, error) {
plen := clen
if plen <= 0 || plen > maxPreambleLength {

View file

@ -109,7 +109,7 @@ func (cs *CachableSource) HandleChange(kind fsutil.ChangeKind, p string, fi os.F
}
hfi := &fileInfo{
sum: h.Digest().Hex(),
sum: h.Digest().Encoded(),
}
cs.txn.Insert([]byte(p), hfi)
cs.mu.Unlock()

View file

@ -30,12 +30,11 @@ func IsURL(str string) bool {
//
// The following patterns are considered to be a Git URL:
//
// - https://(.*).git(?:#.+)?$ git repository URL with optional fragment, as
// known to be used by GitHub and GitLab.
// - http://(.*).git(?:#.+)?$ same, but non-TLS
// - git://(.*) URLs using git:// scheme
// - git@(.*)
// - github.com/ see description below
// - https://(.*).git(?:#.+)?$ git repository URL with optional fragment, as known to be used by GitHub and GitLab.
// - http://(.*).git(?:#.+)?$ same, but non-TLS
// - git://(.*) URLs using git:// scheme
// - git@(.*)
// - github.com/ see description below
//
// The github.com/ prefix is a special case used to treat context-paths
// starting with "github.com/" as a git URL if the given path does not
@ -49,7 +48,7 @@ func IsURL(str string) bool {
// path. Code using this function should check if the path exists locally before
// using it as a URL.
//
// Fragments
// # Fragments
//
// Git URLs accept context configuration in their fragment section, separated by
// a colon (`:`). The first part represents the reference to check out, and can
@ -74,7 +73,6 @@ func IsURL(str string) bool {
// | my-repo.git#master:directory | refs/heads/master | /directory |
// | my-repo.git#mytag:directory | refs/tags/my-tag | /directory |
// | my-repo.git#mybranch:directory | refs/heads/my-branch | /directory |
//
func IsGitURL(str string) bool {
if IsURL(str) && urlPathWithFragmentSuffix.MatchString(str) {
return true

View file

@ -4,11 +4,12 @@ Package client is a Go client for the Docker Engine API.
For more information about the Engine API, see the documentation:
https://docs.docker.com/engine/api/
Usage
# Usage
You use the library by creating a client object and calling methods on it. The
client can be created either from environment variables with NewClientWithOpts(client.FromEnv),
or configured manually with NewClient().
You use the library by constructing a client object using [NewClientWithOpts]
and calling methods on it. The client can be configured from environment
variables by passing the [FromEnv] option, or configured manually by passing any
of the other available [Opts].
For example, to list running containers (the equivalent of "docker ps"):
@ -37,7 +38,6 @@ For example, to list running containers (the equivalent of "docker ps"):
fmt.Printf("%s %s\n", container.ID[:10], container.Image)
}
}
*/
package client // import "github.com/docker/docker/client"
@ -56,6 +56,36 @@ import (
"github.com/pkg/errors"
)
// DummyHost is a hostname used for local communication.
//
// It acts as a valid formatted hostname for local connections (such as "unix://"
// or "npipe://") which do not require a hostname. It should never be resolved,
// but uses the special-purpose ".localhost" TLD (as defined in [RFC 2606, Section 2]
// and [RFC 6761, Section 6.3]).
//
// [RFC 7230, Section 5.4] defines that an empty header must be used for such
// cases:
//
// If the authority component is missing or undefined for the target URI,
// then a client MUST send a Host header field with an empty field-value.
//
// However, [Go stdlib] enforces the semantics of HTTP(S) over TCP, does not
// allow an empty header to be used, and requires req.URL.Scheme to be either
// "http" or "https".
//
// For further details, refer to:
//
// - https://github.com/docker/engine-api/issues/189
// - https://github.com/golang/go/issues/13624
// - https://github.com/golang/go/issues/61076
// - https://github.com/moby/moby/issues/45935
//
// [RFC 2606, Section 2]: https://www.rfc-editor.org/rfc/rfc2606.html#section-2
// [RFC 6761, Section 6.3]: https://www.rfc-editor.org/rfc/rfc6761#section-6.3
// [RFC 7230, Section 5.4]: https://datatracker.ietf.org/doc/html/rfc7230#section-5.4
// [Go stdlib]: https://github.com/golang/go/blob/6244b1946bc2101b01955468f1be502dbadd6807/src/net/http/transport.go#L558-L569
const DummyHost = "api.moby.localhost"
// ErrRedirect is the error returned by checkRedirect when the request is non-GET.
var ErrRedirect = errors.New("unexpected redirect in response")
@ -121,12 +151,10 @@ func CheckRedirect(req *http.Request, via []*http.Request) error {
// itself with values from environment variables (client.FromEnv), and has
// automatic API version negotiation enabled (client.WithAPIVersionNegotiation()).
//
//
// cli, err := client.NewClientWithOpts(
// client.FromEnv,
// client.WithAPIVersionNegotiation(),
// )
//
func NewClientWithOpts(ops ...Opt) (*Client, error) {
client, err := defaultHTTPClient(DefaultDockerHost)
if err != nil {

View file

@ -9,7 +9,11 @@ import "net/http"
// It won't send any version information if the version number is empty. It is
// highly recommended that you set a version or your client may break if the
// server is upgraded.
// Deprecated: use NewClientWithOpts
//
// Deprecated: use [NewClientWithOpts] passing the [WithHost], [WithVersion],
// [WithHTTPClient] and [WithHTTPHeaders] options. We recommend enabling API
// version negotiation by passing the [WithAPIVersionNegotiation] option instead
// of WithVersion.
func NewClient(host string, version string, client *http.Client, httpHeaders map[string]string) (*Client, error) {
return NewClientWithOpts(WithHost(host), WithVersion(version), WithHTTPClient(client), WithHTTPHeaders(httpHeaders))
}
@ -17,7 +21,7 @@ func NewClient(host string, version string, client *http.Client, httpHeaders map
// NewEnvClient initializes a new API client based on environment variables.
// See FromEnv for a list of support environment variables.
//
// Deprecated: use NewClientWithOpts(FromEnv)
// Deprecated: use [NewClientWithOpts] passing the [FromEnv] option.
func NewEnvClient() (*Client, error) {
return NewClientWithOpts(FromEnv)
}

View file

@ -22,7 +22,7 @@ import (
// multiplexed.
// The format of the multiplexed stream is as follows:
//
// [8]byte{STREAM_TYPE, 0, 0, 0, SIZE1, SIZE2, SIZE3, SIZE4}[]byte{OUTPUT}
// [8]byte{STREAM_TYPE, 0, 0, 0, SIZE1, SIZE2, SIZE3, SIZE4}[]byte{OUTPUT}
//
// STREAM_TYPE can be 1 for stdout and 2 for stderr
//

View file

@ -26,23 +26,25 @@ func (cli *Client) ContainerCreate(ctx context.Context, config *container.Config
if err := cli.NewVersionError("1.25", "stop timeout"); config != nil && config.StopTimeout != nil && err != nil {
return response, err
}
clientVersion := cli.ClientVersion()
// When using API 1.24 and under, the client is responsible for removing the container
if hostConfig != nil && versions.LessThan(clientVersion, "1.25") {
hostConfig.AutoRemove = false
}
// When using API under 1.42, the Linux daemon doesn't respect the ConsoleSize
if hostConfig != nil && platform != nil && platform.OS == "linux" && versions.LessThan(clientVersion, "1.42") {
hostConfig.ConsoleSize = [2]uint{0, 0}
}
if err := cli.NewVersionError("1.41", "specify container image platform"); platform != nil && err != nil {
return response, err
}
if hostConfig != nil {
if versions.LessThan(cli.ClientVersion(), "1.25") {
// When using API 1.24 and under, the client is responsible for removing the container
hostConfig.AutoRemove = false
}
if versions.GreaterThanOrEqualTo(cli.ClientVersion(), "1.42") || versions.LessThan(cli.ClientVersion(), "1.40") {
// KernelMemory was added in API 1.40, and deprecated in API 1.42
hostConfig.KernelMemory = 0
}
if platform != nil && platform.OS == "linux" && versions.LessThan(cli.ClientVersion(), "1.42") {
// When using API under 1.42, the Linux daemon doesn't respect the ConsoleSize
hostConfig.ConsoleSize = [2]uint{0, 0}
}
}
query := url.Values{}
if p := formatPlatform(platform); p != "" {
query.Set("platform", p)

View file

@ -24,7 +24,7 @@ import (
// multiplexed.
// The format of the multiplexed stream is as follows:
//
// [8]byte{STREAM_TYPE, 0, 0, 0, SIZE1, SIZE2, SIZE3, SIZE4}[]byte{OUTPUT}
// [8]byte{STREAM_TYPE, 0, 0, 0, SIZE1, SIZE2, SIZE3, SIZE4}[]byte{OUTPUT}
//
// STREAM_TYPE can be 1 for stdout and 2 for stderr
//

View file

@ -1,14 +1,19 @@
package client // import "github.com/docker/docker/client"
import (
"bytes"
"context"
"encoding/json"
"errors"
"io"
"net/url"
"github.com/docker/docker/api/types/container"
"github.com/docker/docker/api/types/versions"
)
const containerWaitErrorMsgLimit = 2 * 1024 /* Max: 2KiB */
// ContainerWait waits until the specified container is in a certain state
// indicated by the given condition, either "not-running" (default),
// "next-exit", or "removed".
@ -46,9 +51,27 @@ func (cli *Client) ContainerWait(ctx context.Context, containerID string, condit
go func() {
defer ensureReaderClosed(resp)
body := resp.body
responseText := bytes.NewBuffer(nil)
stream := io.TeeReader(body, responseText)
var res container.WaitResponse
if err := json.NewDecoder(resp.body).Decode(&res); err != nil {
errC <- err
if err := json.NewDecoder(stream).Decode(&res); err != nil {
// NOTE(nicks): The /wait API does not work well with HTTP proxies.
// At any time, the proxy could cut off the response stream.
//
// But because the HTTP status has already been written, the proxy's
// only option is to write a plaintext error message.
//
// If there's a JSON parsing error, read the real error message
// off the body and send it to the client.
if errors.As(err, new(*json.SyntaxError)) {
_, _ = io.ReadAll(io.LimitReader(stream, containerWaitErrorMsgLimit))
errC <- errors.New(responseText.String())
} else {
errC <- err
}
return
}

View file

@ -9,11 +9,14 @@ import (
"log"
"net/http"
"strings"
"syscall"
"testing"
"testing/iotest"
"time"
"github.com/docker/docker/api/types/container"
"github.com/docker/docker/errdefs"
"github.com/pkg/errors"
)
func TestContainerWaitError(t *testing.T) {
@ -62,6 +65,101 @@ func TestContainerWait(t *testing.T) {
}
}
func TestContainerWaitProxyInterrupt(t *testing.T) {
expectedURL := "/v1.30/containers/container_id/wait"
msg := "copying response body from Docker: unexpected EOF"
client := &Client{
version: "1.30",
client: newMockClient(func(req *http.Request) (*http.Response, error) {
if !strings.HasPrefix(req.URL.Path, expectedURL) {
return nil, fmt.Errorf("Expected URL '%s', got '%s'", expectedURL, req.URL)
}
return &http.Response{
StatusCode: http.StatusOK,
Body: io.NopCloser(strings.NewReader(msg)),
}, nil
}),
}
resultC, errC := client.ContainerWait(context.Background(), "container_id", "")
select {
case err := <-errC:
if !strings.Contains(err.Error(), msg) {
t.Fatalf("Expected: %s, Actual: %s", msg, err.Error())
}
case result := <-resultC:
t.Fatalf("Unexpected result: %v", result)
}
}
func TestContainerWaitProxyInterruptLong(t *testing.T) {
expectedURL := "/v1.30/containers/container_id/wait"
msg := strings.Repeat("x", containerWaitErrorMsgLimit*5)
client := &Client{
version: "1.30",
client: newMockClient(func(req *http.Request) (*http.Response, error) {
if !strings.HasPrefix(req.URL.Path, expectedURL) {
return nil, fmt.Errorf("Expected URL '%s', got '%s'", expectedURL, req.URL)
}
return &http.Response{
StatusCode: http.StatusOK,
Body: io.NopCloser(strings.NewReader(msg)),
}, nil
}),
}
resultC, errC := client.ContainerWait(context.Background(), "container_id", "")
select {
case err := <-errC:
// LimitReader limiting isn't exact, because of how the Readers do chunking.
if len(err.Error()) > containerWaitErrorMsgLimit*2 {
t.Fatalf("Expected error to be limited around %d, actual length: %d", containerWaitErrorMsgLimit, len(err.Error()))
}
case result := <-resultC:
t.Fatalf("Unexpected result: %v", result)
}
}
func TestContainerWaitErrorHandling(t *testing.T) {
for _, test := range []struct {
name string
rdr io.Reader
exp error
}{
{name: "invalid json", rdr: strings.NewReader(`{]`), exp: errors.New("{]")},
{name: "context canceled", rdr: iotest.ErrReader(context.Canceled), exp: context.Canceled},
{name: "context deadline exceeded", rdr: iotest.ErrReader(context.DeadlineExceeded), exp: context.DeadlineExceeded},
{name: "connection reset", rdr: iotest.ErrReader(syscall.ECONNRESET), exp: syscall.ECONNRESET},
} {
t.Run(test.name, func(t *testing.T) {
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
client := &Client{
version: "1.30",
client: newMockClient(func(req *http.Request) (*http.Response, error) {
return &http.Response{
StatusCode: http.StatusOK,
Body: io.NopCloser(test.rdr),
}, nil
}),
}
resultC, errC := client.ContainerWait(ctx, "container_id", "")
select {
case err := <-errC:
if err.Error() != test.exp.Error() {
t.Fatalf("ContainerWait() errC = %v; want %v", err, test.exp)
}
return
case result := <-resultC:
t.Fatalf("expected to not get a wait result, got %d", result.StatusCode)
return
}
// Unexpected - we should not reach this line
})
}
}
func ExampleClient_ContainerWait_withTimeout() {
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()

View file

@ -40,11 +40,11 @@ type notFound interface {
// IsErrNotFound returns true if the error is a NotFound error, which is returned
// by the API when some object is not found.
func IsErrNotFound(err error) bool {
var e notFound
if errors.As(err, &e) {
if errdefs.IsNotFound(err) {
return true
}
return errdefs.IsNotFound(err)
var e notFound
return errors.As(err, &e)
}
type objectNotFoundError struct {
@ -58,22 +58,11 @@ func (e objectNotFoundError) Error() string {
return fmt.Sprintf("Error: No such %s: %s", e.object, e.id)
}
// unauthorizedError represents an authorization error in a remote registry.
type unauthorizedError struct {
cause error
}
// Error returns a string representation of an unauthorizedError
func (u unauthorizedError) Error() string {
return u.cause.Error()
}
// IsErrUnauthorized returns true if the error is caused
// when a remote registry authentication fails
//
// Deprecated: use errdefs.IsUnauthorized
func IsErrUnauthorized(err error) bool {
if _, ok := err.(unauthorizedError); ok {
return ok
}
return errdefs.IsUnauthorized(err)
}
@ -85,32 +74,12 @@ func (e pluginPermissionDenied) Error() string {
return "Permission denied while installing plugin " + e.name
}
// IsErrPluginPermissionDenied returns true if the error is caused
// when a user denies a plugin's permissions
func IsErrPluginPermissionDenied(err error) bool {
_, ok := err.(pluginPermissionDenied)
return ok
}
type notImplementedError struct {
message string
}
func (e notImplementedError) Error() string {
return e.message
}
func (e notImplementedError) NotImplemented() bool {
return true
}
// IsErrNotImplemented returns true if the error is a NotImplemented error.
// This is returned by the API when a requested feature has not been
// implemented.
//
// Deprecated: use errdefs.IsNotImplemented
func IsErrNotImplemented(err error) bool {
if _, ok := err.(notImplementedError); ok {
return ok
}
return errdefs.IsNotImplemented(err)
}

View file

@ -17,7 +17,6 @@ import (
// be sent over the error channel. If an error is sent all processing will be stopped. It's up
// to the caller to reopen the stream in the event of an error by reinvoking this method.
func (cli *Client) Events(ctx context.Context, options types.EventsOptions) (<-chan events.Message, <-chan error) {
messages := make(chan events.Message)
errs := make(chan error, 1)

View file

@ -58,7 +58,6 @@ func TestEventsErrorFromServer(t *testing.T) {
}
func TestEvents(t *testing.T) {
expectedURL := "/events"
filters := filters.NewArgs()

View file

@ -64,7 +64,11 @@ func fallbackDial(proto, addr string, tlsConfig *tls.Config) (net.Conn, error) {
}
func (cli *Client) setupHijackConn(ctx context.Context, req *http.Request, proto string) (net.Conn, string, error) {
req.Host = cli.addr
req.URL.Host = cli.addr
if cli.proto == "unix" || cli.proto == "npipe" {
// Override host header for non-tcp connections.
req.Host = DummyHost
}
req.Header.Set("Connection", "Upgrade")
req.Header.Set("Upgrade", proto)

View file

@ -34,6 +34,9 @@ func (cli *Client) ImageList(ctx context.Context, options types.ImageListOptions
if options.All {
query.Set("all", "1")
}
if options.SharedSize && versions.GreaterThanOrEqualTo(cli.version, "1.42") {
query.Set("shared-size", "1")
}
serverResp, err := cli.get(ctx, "/images/json", query, nil)
defer ensureReaderClosed(serverResp)

View file

@ -7,12 +7,15 @@ import (
"fmt"
"io"
"net/http"
"net/url"
"strings"
"testing"
"github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/filters"
"github.com/docker/docker/errdefs"
"gotest.tools/v3/assert"
is "gotest.tools/v3/assert/cmp"
)
func TestImageListError(t *testing.T) {
@ -158,3 +161,41 @@ func TestImageListApiBefore125(t *testing.T) {
t.Fatalf("expected 2 images, got %v", images)
}
}
// Checks if shared-size query parameter is set/not being set correctly
// for /images/json.
func TestImageListWithSharedSize(t *testing.T) {
t.Parallel()
const sharedSize = "shared-size"
for _, tc := range []struct {
name string
version string
options types.ImageListOptions
sharedSize string // expected value for the shared-size query param, or empty if it should not be set.
}{
{name: "unset after 1.42, no options set", version: "1.42"},
{name: "set after 1.42, if requested", version: "1.42", options: types.ImageListOptions{SharedSize: true}, sharedSize: "1"},
{name: "unset before 1.42, even if requested", version: "1.41", options: types.ImageListOptions{SharedSize: true}},
} {
tc := tc
t.Run(tc.name, func(t *testing.T) {
t.Parallel()
var query url.Values
client := &Client{
client: newMockClient(func(req *http.Request) (*http.Response, error) {
query = req.URL.Query()
return &http.Response{
StatusCode: http.StatusOK,
Body: io.NopCloser(strings.NewReader("[]")),
}, nil
}),
version: tc.version,
}
_, err := client.ImageList(context.Background(), tc.options)
assert.Check(t, err)
expectedSet := tc.sharedSize != ""
assert.Check(t, is.Equal(query.Has(sharedSize), expectedSet))
assert.Check(t, is.Equal(query.Get(sharedSize), tc.sharedSize))
})
}
}

View file

@ -44,13 +44,6 @@ func FromEnv(c *Client) error {
return nil
}
// WithDialer applies the dialer.DialContext to the client transport. This can be
// used to set the Timeout and KeepAlive settings of the client.
// Deprecated: use WithDialContext
func WithDialer(dialer *net.Dialer) Opt {
return WithDialContext(dialer.DialContext)
}
// WithDialContext applies the dialer to the client transport. This can be
// used to set the Timeout and KeepAlive settings of the client.
func WithDialContext(dialContext func(ctx context.Context, network, addr string) (net.Conn, error)) Opt {

View file

@ -96,16 +96,14 @@ func (cli *Client) buildRequest(method, path string, body io.Reader, headers hea
return nil, err
}
req = cli.addHeaders(req, headers)
req.URL.Scheme = cli.scheme
req.URL.Host = cli.addr
if cli.proto == "unix" || cli.proto == "npipe" {
// For local communications, it doesn't matter what the host is. We just
// need a valid and meaningful host name. (See #189)
req.Host = "docker"
// Override host header for non-tcp connections.
req.Host = DummyHost
}
req.URL.Host = cli.addr
req.URL.Scheme = cli.scheme
if expectedPayload && req.Header.Get("Content-Type") == "" {
req.Header.Set("Content-Type", "text/plain")
}

View file

@ -28,24 +28,24 @@ func TestSetHostHeader(t *testing.T) {
expectedURLHost string
}{
{
"unix:///var/run/docker.sock",
"docker",
"/var/run/docker.sock",
host: "unix:///var/run/docker.sock",
expectedHost: DummyHost,
expectedURLHost: "/var/run/docker.sock",
},
{
"npipe:////./pipe/docker_engine",
"docker",
"//./pipe/docker_engine",
host: "npipe:////./pipe/docker_engine",
expectedHost: DummyHost,
expectedURLHost: "//./pipe/docker_engine",
},
{
"tcp://0.0.0.0:4243",
"",
"0.0.0.0:4243",
host: "tcp://0.0.0.0:4243",
expectedHost: "",
expectedURLHost: "0.0.0.0:4243",
},
{
"tcp://localhost:4243",
"",
"localhost:4243",
host: "tcp://localhost:4243",
expectedHost: "",
expectedURLHost: "localhost:4243",
},
}

View file

@ -9,12 +9,13 @@ import (
"os/signal"
"syscall"
"github.com/docker/docker/dockerversion"
"github.com/ishidawataru/sctp"
)
func main() {
f := os.NewFile(3, "signal-parent")
host, container := parseHostContainerAddrs()
host, container := parseFlags()
p, err := NewProxy(host, container)
if err != nil {
@ -30,19 +31,26 @@ func main() {
p.Run()
}
// parseHostContainerAddrs parses the flags passed on reexec to create the TCP/UDP/SCTP
// net.Addrs to map the host and container ports
func parseHostContainerAddrs() (host net.Addr, container net.Addr) {
// parseFlags parses the flags passed on reexec to create the TCP/UDP/SCTP
// net.Addrs to map the host and container ports.
func parseFlags() (host net.Addr, container net.Addr) {
var (
proto = flag.String("proto", "tcp", "proxy protocol")
hostIP = flag.String("host-ip", "", "host ip")
hostPort = flag.Int("host-port", -1, "host port")
containerIP = flag.String("container-ip", "", "container ip")
containerPort = flag.Int("container-port", -1, "container port")
printVer = flag.Bool("v", false, "print version information and quit")
printVersion = flag.Bool("version", false, "print version information and quit")
)
flag.Parse()
if *printVer || *printVersion {
fmt.Printf("docker-proxy (commit %s) version %s\n", dockerversion.GitCommit, dockerversion.Version)
os.Exit(0)
}
switch *proto {
case "tcp":
host = &net.TCPAddr{IP: net.ParseIP(*hostIP), Port: *hostPort}

View file

@ -12,20 +12,6 @@ const defaultTrustKeyFile = "key.json"
// installCommonConfigFlags adds flags to the pflag.FlagSet to configure the daemon
func installCommonConfigFlags(conf *config.Config, flags *pflag.FlagSet) error {
var err error
conf.Pidfile, err = getDefaultPidFile()
if err != nil {
return err
}
conf.Root, err = getDefaultDataRoot()
if err != nil {
return err
}
conf.ExecRoot, err = getDefaultExecRoot()
if err != nil {
return err
}
var (
allowNonDistributable = opts.NewNamedListOptsRef("allow-nondistributable-artifacts", &conf.AllowNondistributableArtifacts, registry.ValidateIndexName)
registryMirrors = opts.NewNamedListOptsRef("registry-mirrors", &conf.Mirrors, registry.ValidateMirror)
@ -59,8 +45,8 @@ func installCommonConfigFlags(conf *config.Config, flags *pflag.FlagSet) error {
flags.Var(opts.NewNamedMapOpts("log-opts", conf.LogConfig.Config, nil), "log-opt", "Default log driver options for containers")
flags.StringVar(&conf.CorsHeaders, "api-cors-header", "", "Set CORS headers in the Engine API")
flags.IntVar(&conf.MaxConcurrentDownloads, "max-concurrent-downloads", conf.MaxConcurrentDownloads, "Set the max concurrent downloads for each pull")
flags.IntVar(&conf.MaxConcurrentUploads, "max-concurrent-uploads", conf.MaxConcurrentUploads, "Set the max concurrent uploads for each push")
flags.IntVar(&conf.MaxConcurrentDownloads, "max-concurrent-downloads", conf.MaxConcurrentDownloads, "Set the max concurrent downloads")
flags.IntVar(&conf.MaxConcurrentUploads, "max-concurrent-uploads", conf.MaxConcurrentUploads, "Set the max concurrent uploads")
flags.IntVar(&conf.MaxDownloadAttempts, "max-download-attempts", conf.MaxDownloadAttempts, "Set the max download attempts for each pull")
flags.IntVar(&conf.ShutdownTimeout, "shutdown-timeout", conf.ShutdownTimeout, "Set the default shutdown timeout")
@ -79,10 +65,9 @@ func installCommonConfigFlags(conf *config.Config, flags *pflag.FlagSet) error {
// Deprecated flags / options
// "--graph" is "soft-deprecated" in favor of "data-root". This flag was added
// before Docker 1.0, so won't be removed, only hidden, to discourage its usage.
flags.StringVarP(&conf.Root, "graph", "g", conf.Root, "Root of the Docker runtime")
_ = flags.MarkHidden("graph")
//nolint:staticcheck // TODO(thaJeztah): remove in next release.
flags.StringVarP(&conf.RootDeprecated, "graph", "g", conf.RootDeprecated, "Root of the Docker runtime")
_ = flags.MarkDeprecated("graph", "Use --data-root instead")
flags.BoolVarP(&conf.AutoRestart, "restart", "r", true, "--restart on the daemon has been deprecated in favor of --restart policies on docker run")
_ = flags.MarkDeprecated("restart", "Please use a restart policy on docker run")
flags.StringVar(&conf.ClusterAdvertise, "cluster-advertise", "", "Address or interface name to advertise")

View file

@ -5,18 +5,13 @@ package main
import (
"net"
"os/exec"
"path/filepath"
"github.com/containerd/cgroups"
"github.com/docker/docker/api/types"
"github.com/docker/docker/daemon/config"
"github.com/docker/docker/opts"
"github.com/docker/docker/pkg/homedir"
"github.com/docker/docker/pkg/rootless"
"github.com/docker/docker/registry"
"github.com/docker/docker/rootless"
units "github.com/docker/go-units"
"github.com/pkg/errors"
"github.com/spf13/pflag"
)
@ -27,12 +22,6 @@ func installConfigFlags(conf *config.Config, flags *pflag.FlagSet) error {
return err
}
conf.Ulimits = make(map[string]*units.Ulimit)
// Set default value for `--default-shm-size`
conf.ShmSize = opts.MemBytes(config.DefaultShmSize)
conf.Runtimes = make(map[string]types.Runtime)
// Then platform-specific install flags
flags.Var(opts.NewNamedRuntimeOpt("runtimes", &conf.Runtimes, config.StockRuntimeName), "add-runtime", "Register an additional OCI compatible runtime")
flags.StringVarP(&conf.SocketGroup, "group", "G", "docker", "Group for the unix socket")
@ -53,16 +42,7 @@ func installConfigFlags(conf *config.Config, flags *pflag.FlagSet) error {
flags.BoolVar(&conf.BridgeConfig.InterContainerCommunication, "icc", true, "Enable inter-container communication")
flags.IPVar(&conf.BridgeConfig.DefaultIP, "ip", net.IPv4zero, "Default IP when binding container ports")
flags.BoolVar(&conf.BridgeConfig.EnableUserlandProxy, "userland-proxy", true, "Use userland proxy for loopback traffic")
defaultUserlandProxyPath := ""
if rootless.RunningWithRootlessKit() {
var err error
// use rootlesskit-docker-proxy for exposing the ports in RootlessKit netns to the initial namespace.
defaultUserlandProxyPath, err = exec.LookPath(rootless.RootlessKitDockerProxyBinary)
if err != nil {
return errors.Wrapf(err, "running with RootlessKit, but %s not installed", rootless.RootlessKitDockerProxyBinary)
}
}
flags.StringVar(&conf.BridgeConfig.UserlandProxyPath, "userland-proxy-path", defaultUserlandProxyPath, "Path to the userland proxy binary")
flags.StringVar(&conf.BridgeConfig.UserlandProxyPath, "userland-proxy-path", conf.BridgeConfig.UserlandProxyPath, "Path to the userland proxy binary")
flags.StringVar(&conf.CgroupParent, "cgroup-parent", "", "Set parent cgroup for all containers")
flags.StringVar(&conf.RemappedRoot, "userns-remap", "", "User/Group setting for user namespaces")
flags.BoolVar(&conf.LiveRestoreEnabled, "live-restore", false, "Enable live restore of docker when containers are still running")
@ -71,19 +51,15 @@ func installConfigFlags(conf *config.Config, flags *pflag.FlagSet) error {
flags.StringVar(&conf.InitPath, "init-path", "", "Path to the docker-init binary")
flags.Int64Var(&conf.CPURealtimePeriod, "cpu-rt-period", 0, "Limit the CPU real-time period in microseconds for the parent cgroup for all containers (not supported with cgroups v2)")
flags.Int64Var(&conf.CPURealtimeRuntime, "cpu-rt-runtime", 0, "Limit the CPU real-time runtime in microseconds for the parent cgroup for all containers (not supported with cgroups v2)")
flags.StringVar(&conf.SeccompProfile, "seccomp-profile", config.SeccompProfileDefault, `Path to seccomp profile. Use "unconfined" to disable the default seccomp profile`)
flags.StringVar(&conf.SeccompProfile, "seccomp-profile", conf.SeccompProfile, `Path to seccomp profile. Use "unconfined" to disable the default seccomp profile`)
flags.Var(&conf.ShmSize, "default-shm-size", "Default shm size for containers")
flags.BoolVar(&conf.NoNewPrivileges, "no-new-privileges", false, "Set no-new-privileges by default for new containers")
flags.StringVar(&conf.IpcMode, "default-ipc-mode", string(config.DefaultIpcMode), `Default mode for containers ipc ("shareable" | "private")`)
flags.StringVar(&conf.IpcMode, "default-ipc-mode", conf.IpcMode, `Default mode for containers ipc ("shareable" | "private")`)
flags.Var(&conf.NetworkConfig.DefaultAddressPools, "default-address-pool", "Default address pools for node specific local networks")
// rootless needs to be explicitly specified for running "rootful" dockerd in rootless dockerd (#38702)
// Note that defaultUserlandProxyPath and honorXDG are configured according to the value of rootless.RunningWithRootlessKit, not the value of --rootless.
flags.BoolVar(&conf.Rootless, "rootless", rootless.RunningWithRootlessKit(), "Enable rootless mode; typically used with RootlessKit")
defaultCgroupNamespaceMode := config.DefaultCgroupNamespaceMode
if cgroups.Mode() != cgroups.Unified {
defaultCgroupNamespaceMode = config.DefaultCgroupV1NamespaceMode
}
flags.StringVar(&conf.CgroupNamespaceMode, "default-cgroupns-mode", string(defaultCgroupNamespaceMode), `Default mode for containers cgroup namespace ("host" | "private")`)
// Note that conf.BridgeConfig.UserlandProxyPath and honorXDG are configured according to the value of rootless.RunningWithRootlessKit, not the value of --rootless.
flags.BoolVar(&conf.Rootless, "rootless", conf.Rootless, "Enable rootless mode; typically used with RootlessKit")
flags.StringVar(&conf.CgroupNamespaceMode, "default-cgroupns-mode", conf.CgroupNamespaceMode, `Default mode for containers cgroup namespace ("host" | "private")`)
return nil
}
@ -97,36 +73,3 @@ func configureCertsDir() {
}
}
}
func getDefaultPidFile() (string, error) {
if !honorXDG {
return "/var/run/docker.pid", nil
}
runtimeDir, err := homedir.GetRuntimeDir()
if err != nil {
return "", err
}
return filepath.Join(runtimeDir, "docker.pid"), nil
}
func getDefaultDataRoot() (string, error) {
if !honorXDG {
return "/var/lib/docker", nil
}
dataHome, err := homedir.GetDataHome()
if err != nil {
return "", err
}
return filepath.Join(dataHome, "docker"), nil
}
func getDefaultExecRoot() (string, error) {
if !honorXDG {
return "/var/run/docker", nil
}
runtimeDir, err := homedir.GetRuntimeDir()
if err != nil {
return "", err
}
return filepath.Join(runtimeDir, "docker"), nil
}

View file

@ -15,8 +15,9 @@ import (
func TestDaemonParseShmSize(t *testing.T) {
flags := pflag.NewFlagSet("test", pflag.ContinueOnError)
conf := &config.Config{}
err := installConfigFlags(conf, flags)
conf, err := config.New()
assert.NilError(t, err)
err = installConfigFlags(conf, flags)
assert.NilError(t, err)
// By default `--default-shm-size=64M`
assert.Check(t, is.Equal(int64(64*1024*1024), conf.ShmSize.Value()))

View file

@ -1,25 +1,10 @@
package main
import (
"os"
"path/filepath"
"github.com/docker/docker/daemon/config"
"github.com/spf13/pflag"
)
func getDefaultPidFile() (string, error) {
return "", nil
}
func getDefaultDataRoot() (string, error) {
return filepath.Join(os.Getenv("programdata"), "docker"), nil
}
func getDefaultExecRoot() (string, error) {
return filepath.Join(os.Getenv("programdata"), "docker", "exec-root"), nil
}
// installConfigFlags adds flags to the pflag.FlagSet to configure the daemon
func installConfigFlags(conf *config.Config, flags *pflag.FlagSet) error {
// First handle install flags which are consistent cross-platform

View file

@ -46,10 +46,10 @@ import (
"github.com/docker/docker/pkg/jsonmessage"
"github.com/docker/docker/pkg/pidfile"
"github.com/docker/docker/pkg/plugingetter"
"github.com/docker/docker/pkg/rootless"
"github.com/docker/docker/pkg/sysinfo"
"github.com/docker/docker/pkg/system"
"github.com/docker/docker/plugin"
"github.com/docker/docker/rootless"
"github.com/docker/docker/runconfig"
"github.com/docker/go-connections/tlsconfig"
"github.com/moby/buildkit/session"
@ -294,7 +294,9 @@ func newRouterOptions(config *config.Config, d *daemon.Daemon) (routerOptions, e
bk, err := buildkit.New(buildkit.Opt{
SessionManager: sm,
Root: filepath.Join(config.Root, "buildkit"),
EngineID: d.ID(),
Dist: d.DistributionServices(),
ImageTagger: d.ImageService(),
NetworkController: d.NetworkController(),
DefaultCgroupParent: cgroupParent,
RegistryHosts: d.RegistryHosts(),
@ -323,7 +325,6 @@ func newRouterOptions(config *config.Config, d *daemon.Daemon) (routerOptions, e
func (cli *DaemonCli) reloadConfig() {
reload := func(c *config.Config) {
// Revalidate and reload the authorization plugins
if err := validateAuthzPlugins(c.AuthorizationPlugins, cli.d.PluginStore); err != nil {
logrus.Fatalf("Error validating authorization plugin: %v", err)
@ -395,9 +396,6 @@ func loadDaemonCliConfig(opts *daemonOptions) (*config.Config, error) {
conf.Hosts = opts.Hosts
conf.LogLevel = opts.LogLevel
if flags.Changed("graph") && flags.Changed("data-root") {
return nil, errors.New(`cannot specify both "--graph" and "--data-root" option`)
}
if flags.Changed(FlagTLS) {
conf.TLS = &opts.TLS
}
@ -448,10 +446,6 @@ func loadDaemonCliConfig(opts *daemonOptions) (*config.Config, error) {
return nil, err
}
if flags.Changed("graph") {
logrus.Warnf(`The "-g / --graph" flag is deprecated. Please use "--data-root" instead`)
}
// Check if duplicate label-keys with different values are found
newLabels, err := config.GetConflictFreeLabels(conf.Labels)
if err != nil {
@ -608,7 +602,7 @@ func (cli *DaemonCli) getContainerdDaemonOpts() ([]supervisor.DaemonOpt, error)
}
if !cli.Config.CriContainerd {
opts = append(opts, supervisor.WithPlugin("cri", nil))
opts = append(opts, supervisor.WithPlugin("io.containerd.grpc.v1.cri", nil))
}
return opts, nil
@ -648,7 +642,7 @@ func newAPIServerConfig(config *config.Config) (*apiserver.Config, error) {
// checkTLSAuthOK checks basically for an explicitly disabled TLS/TLSVerify
// Going forward we do not want to support a scenario where dockerd listens
// on TCP without either TLS client auth (or an explicit opt-in to disable it)
// on TCP without either TLS client auth (or an explicit opt-in to disable it)
func checkTLSAuthOK(c *config.Config) bool {
if c.TLS == nil {
// Either TLS is enabled by default, in which case TLS verification should be enabled by default, or explicitly disabled

View file

@ -24,7 +24,7 @@ func setDefaultUmask() error {
}
func getDaemonConfDir(root string) (string, error) {
return filepath.Join(root, `\config`), nil
return filepath.Join(root, "config"), nil
}
// preNotifyReady sends a message to the host when the API is active, but before the daemon is

View file

@ -9,7 +9,7 @@ import (
"github.com/docker/docker/dockerversion"
"github.com/docker/docker/pkg/jsonmessage"
"github.com/docker/docker/pkg/reexec"
"github.com/docker/docker/rootless"
"github.com/docker/docker/pkg/rootless"
"github.com/moby/buildkit/util/apicaps"
"github.com/moby/term"
"github.com/sirupsen/logrus"
@ -21,7 +21,11 @@ var (
)
func newDaemonCommand() (*cobra.Command, error) {
opts := newDaemonOptions(config.New())
cfg, err := config.New()
if err != nil {
return nil, err
}
opts := newDaemonOptions(cfg)
cmd := &cobra.Command{
Use: "dockerd [OPTIONS]",
@ -84,6 +88,7 @@ func main() {
_, stdout, stderr := term.StdStreams()
initLogging(stdout, stderr)
configureGRPCLog()
onError := func(err error) {
fmt.Fprintf(stderr, "%s\n", err)

View file

@ -24,7 +24,7 @@ func runDaemon(opts *daemonOptions) error {
// Windows specific settings as these are not defaulted.
if opts.configFile == "" {
opts.configFile = filepath.Join(opts.daemonConfig.Root, `config\daemon.json`)
opts.configFile = filepath.Join(opts.daemonConfig.Root, "config", "daemon.json")
}
if runAsService {
// If Windows SCM manages the service - no need for PID files

17
cmd/dockerd/grpclog.go Normal file
View file

@ -0,0 +1,17 @@
package main
import (
"github.com/sirupsen/logrus"
"google.golang.org/grpc/grpclog"
)
// grpc's default logger is *very* noisy and uses "info" and even "warn" level logging for mostly useless messages.
// This function configures the grpc logger to step down the severity of all messages.
//
// info => trace
// warn => debug
// error => warn
func configureGRPCLog() {
l := logrus.WithField("library", "grpc")
grpclog.SetLoggerV2(grpclog.NewLoggerV2(l.WriterLevel(logrus.TraceLevel), l.WriterLevel(logrus.DebugLevel), l.WriterLevel(logrus.WarnLevel)))
}

View file

@ -4,6 +4,7 @@ import (
"net"
"net/http"
"strings"
"time"
metrics "github.com/docker/go-metrics"
"github.com/sirupsen/logrus"
@ -24,7 +25,11 @@ func startMetricsServer(addr string) error {
mux.Handle("/metrics", metrics.Handler())
go func() {
logrus.Infof("metrics API listening on %s", l.Addr())
if err := http.Serve(l, mux); err != nil && !strings.Contains(err.Error(), "use of closed network connection") {
srv := &http.Server{
Handler: mux,
ReadHeaderTimeout: 5 * time.Minute, // "G112: Potential Slowloris Attack (gosec)"; not a real concern for our use, so setting a long timeout.
}
if err := srv.Serve(l); err != nil && !strings.Contains(err.Error(), "use of closed network connection") {
logrus.WithError(err).Error("error serving metrics API")
}
}()

View file

@ -65,8 +65,6 @@ func (o *daemonOptions) installFlags(flags *pflag.FlagSet) {
flags.BoolVar(&o.TLS, FlagTLS, DefaultTLSValue, "Use TLS; implied by --tlsverify")
flags.BoolVar(&o.TLSVerify, FlagTLSVerify, dockerTLSVerify || DefaultTLSValue, "Use TLS and verify the remote")
// TODO use flag flags.String("identity"}, "i", "", "Path to libtrust key file")
o.TLSOptions = &tlsconfig.Options{}
tlsOptions := o.TLSOptions
flags.StringVar(&tlsOptions.CAFile, "tlscacert", filepath.Join(dockerCertPath, DefaultCaFile), "Trust certs signed only by this CA")

View file

@ -7,10 +7,8 @@ import (
"io"
"log"
"os"
"os/exec"
"path/filepath"
"time"
"unsafe"
"github.com/sirupsen/logrus"
"github.com/spf13/pflag"
@ -27,9 +25,8 @@ var (
flUnregisterService *bool
flRunService *bool
setStdHandle = windows.NewLazySystemDLL("kernel32.dll").NewProc("SetStdHandle")
oldStderr windows.Handle
panicFile *os.File
oldStderr windows.Handle
panicFile *os.File
service *handler
)
@ -147,16 +144,8 @@ func (h *etwHook) Fire(e *logrus.Entry) error {
return windows.ReportEvent(h.log.Handle, etype, 0, eid, 0, count, 0, &ss[0], nil)
}
func getServicePath() (string, error) {
p, err := exec.LookPath(os.Args[0])
if err != nil {
return "", err
}
return filepath.Abs(p)
}
func registerService() error {
p, err := getServicePath()
p, err := os.Executable()
if err != nil {
return err
}
@ -188,35 +177,14 @@ func registerService() error {
}
defer s.Close()
// See http://stackoverflow.com/questions/35151052/how-do-i-configure-failure-actions-of-a-windows-service-written-in-go
const (
scActionNone = 0
scActionRestart = 1
scActionReboot = 2
scActionRunCommand = 3
serviceConfigFailureActions = 2
err = s.SetRecoveryActions(
[]mgr.RecoveryAction{
{Type: mgr.ServiceRestart, Delay: 15 * time.Second},
{Type: mgr.ServiceRestart, Delay: 15 * time.Second},
{Type: mgr.NoAction},
},
uint32(24*time.Hour/time.Second),
)
type serviceFailureActions struct {
ResetPeriod uint32
RebootMsg *uint16
Command *uint16
ActionsCount uint32
Actions uintptr
}
type scAction struct {
Type uint32
Delay uint32
}
t := []scAction{
{Type: scActionRestart, Delay: uint32(60 * time.Second / time.Millisecond)},
{Type: scActionRestart, Delay: uint32(60 * time.Second / time.Millisecond)},
{Type: scActionNone},
}
lpInfo := serviceFailureActions{ResetPeriod: uint32(24 * time.Hour / time.Second), ActionsCount: uint32(3), Actions: uintptr(unsafe.Pointer(&t[0]))}
err = windows.ChangeServiceConfig2(s.Handle, serviceConfigFailureActions, (*byte)(unsafe.Pointer(&lpInfo)))
if err != nil {
return err
}
@ -264,7 +232,8 @@ func initService(daemonCli *DaemonCli) (bool, bool, error) {
return false, false, nil
}
interactive, err := svc.IsAnInteractiveSession()
// Check if we're running as a Windows service or interactively.
isService, err := svc.IsWindowsService()
if err != nil {
return false, false, err
}
@ -276,7 +245,7 @@ func initService(daemonCli *DaemonCli) (bool, bool, error) {
}
var log *eventlog.Log
if !interactive {
if isService {
log, err = eventlog.Open(*flServiceName)
if err != nil {
return false, false, err
@ -288,10 +257,10 @@ func initService(daemonCli *DaemonCli) (bool, bool, error) {
service = h
go func() {
if interactive {
err = debug.Run(*flServiceName, h)
} else {
if isService {
err = svc.Run(*flServiceName, h)
} else {
err = debug.Run(*flServiceName, h)
}
h.fromsvc <- err
@ -387,21 +356,19 @@ func initPanicFile(path string) error {
// Update STD_ERROR_HANDLE to point to the panic file so that Go writes to
// it when it panics. Remember the old stderr to restore it before removing
// the panic file.
sh := uint32(windows.STD_ERROR_HANDLE)
h, err := windows.GetStdHandle(sh)
h, err := windows.GetStdHandle(windows.STD_ERROR_HANDLE)
if err != nil {
return err
}
oldStderr = h
err = windows.SetStdHandle(windows.STD_ERROR_HANDLE, windows.Handle(panicFile.Fd()))
if err != nil {
return err
}
oldStderr = h
r, _, err := setStdHandle.Call(uintptr(sh), uintptr(panicFile.Fd()))
if r == 0 && err != nil {
return err
}
// Reset os.Stderr to the panic file (so fmt.Fprintf(os.Stderr,...) actually gets redirected)
os.Stderr = os.NewFile(uintptr(panicFile.Fd()), "/dev/stderr")
os.Stderr = os.NewFile(panicFile.Fd(), "/dev/stderr")
// Force threads that panic to write to stderr (the panicFile handle now), otherwise it will go into the ether
log.SetOutput(os.Stderr)
@ -412,8 +379,7 @@ func initPanicFile(path string) error {
func removePanicFile() {
if st, err := panicFile.Stat(); err == nil {
if st.Size() == 0 {
sh := uint32(windows.STD_ERROR_HANDLE)
setStdHandle.Call(uintptr(sh), uintptr(oldStderr))
windows.SetStdHandle(windows.STD_ERROR_HANDLE, oldStderr)
panicFile.Close()
os.Remove(panicFile.Name())
}

View file

@ -3,32 +3,27 @@ package trap // import "github.com/docker/docker/cmd/dockerd/trap"
import (
"fmt"
"os"
gosignal "os/signal"
"os/signal"
"sync/atomic"
"syscall"
"github.com/docker/docker/pkg/stack"
)
// Trap sets up a simplified signal "trap", appropriate for common
// behavior expected from a vanilla unix command-line tool in general
// (and the Docker engine in particular).
//
// * If SIGINT or SIGTERM are received, `cleanup` is called, then the process is terminated.
// * If SIGINT or SIGTERM are received 3 times before cleanup is complete, then cleanup is
// skipped and the process is terminated immediately (allows force quit of stuck daemon)
// * A SIGQUIT always causes an exit without cleanup, with a goroutine dump preceding exit.
// * Ignore SIGPIPE events. These are generated by systemd when journald is restarted while
// the docker daemon is not restarted and also running under systemd.
// Fixes https://github.com/docker/docker/issues/19728
//
// - If SIGINT or SIGTERM are received, `cleanup` is called, then the process is terminated.
// - If SIGINT or SIGTERM are received 3 times before cleanup is complete, then cleanup is
// skipped and the process is terminated immediately (allows force quit of stuck daemon)
// - Ignore SIGPIPE events. These are generated by systemd when journald is restarted while
// the docker daemon is not restarted and also running under systemd.
// Fixes https://github.com/docker/docker/issues/19728
func Trap(cleanup func(), logger interface {
Info(args ...interface{})
}) {
c := make(chan os.Signal, 1)
// we will handle INT, TERM, QUIT, SIGPIPE here
signals := []os.Signal{os.Interrupt, syscall.SIGTERM, syscall.SIGQUIT, syscall.SIGPIPE}
gosignal.Notify(c, signals...)
// we will handle INT, TERM, SIGPIPE here
signal.Notify(c, os.Interrupt, syscall.SIGTERM, syscall.SIGPIPE)
go func() {
interruptCount := uint32(0)
for sig := range c {
@ -53,11 +48,8 @@ func Trap(cleanup func(), logger interface {
// 3 SIGTERM/INT signals received; force exit without cleanup
logger.Info("Forcing docker daemon shutdown without cleanup; 3 interrupts received")
}
case syscall.SIGQUIT:
stack.Dump()
logger.Info("Forcing docker daemon shutdown without cleanup on SIGQUIT")
}
// for the SIGINT/TERM, and SIGQUIT non-clean shutdown case, exit with 128 + signal #
// for the SIGINT/TERM non-clean shutdown case, exit with 128 + signal #
os.Exit(128 + int(sig.(syscall.Signal)))
}(sig)
}

Some files were not shown because too many files have changed in this diff Show more