Commit graph

2991 commits

Author SHA1 Message Date
Paweł Gronowski
f8275cb303
c8d/docker-py: Skip test_build_squash
build --squash is an experimental feature that is not implemented in the
containerd image store.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-12-14 09:27:31 +01:00
Sebastiaan van Stijn
86cd6da76e
Merge pull request #46916 from thaJeztah/bump_docker_py
testing: update docker-py to 7.0.0
2023-12-11 12:02:23 +01:00
Sebastiaan van Stijn
b524ed2dda
testing: update docker-py to 7.0.0
full diff: https://github.com/docker/docker-py/compare/7.0.0b1...7.0.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-09 01:36:38 +01:00
Sebastiaan van Stijn
33d2ec08ba
update containerd binary to 1.7.11
- full diff: https://github.com/containerd/containerd/compare/v1.7.10...v1.7.11
- release notes: https://github.com/containerd/containerd/releases/tag/v1.7.11

Welcome to the v1.7.11 release of containerd!

The eleventh patch release for containerd 1.7 contains various fixes and
updates including one security issue.

Notable Updates

- Fix Windows default path overwrite issue
- Update push to always inherit distribution sources from parent
- Update shim to use net dial for gRPC shim sockets
- Fix otel version incompatibility
- Fix Windows snapshotter blocking snapshot GC on remove failure
- Mask /sys/devices/virtual/powercap path in runtime spec and deny in
  default apparmor profile [GHSA-7ww5-4wqc-m92c]

Deprecation Warnings

- Emit deprecation warning for AUFS snapshotter
- Emit deprecation warning for v1 runtime
- Emit deprecation warning for deprecated CRI configs
- Emit deprecation warning for CRI v1alpha1 usage
- Emit deprecation warning for CRIU config in CRI

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-09 01:26:44 +01:00
Sebastiaan van Stijn
c51a262e34
update containerd binary to 1.7.10
- full diff: https://github.com/containerd/containerd/compare/v1.7.9...v1.7.10
- release notes: https://github.com/containerd/containerd/releases/tag/v1.7.10

Welcome to the v1.7.10 release of containerd!

The tenth patch release for containerd 1.7 contains various fixes and
updates.

Notable Updates

- Enhance container image unpack client logs
- cri: fix using the pinned label to pin image
- fix: ImagePull should close http connection if there is no available data to read.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-09 01:25:57 +01:00
Sebastiaan van Stijn
82a318db5f
Merge pull request #46894 from vvoland/hack-load-specialimages
hack: Load special images on demand
2023-12-07 11:28:38 +01:00
Paweł Gronowski
bc94dfc7d2
hack: Load special images on demand
Rewrite `.build-empty-images` shell script that produced special images
(emptyfs with no layers, and empty danglign image) to a Go functions
that construct the same archives in a temporary directory.

Use them to load these images on demand only in the tests that need
them.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-12-06 17:16:37 +01:00
Sebastiaan van Stijn
4046ae5e2f
Merge pull request #46787 from thaJeztah/update_golang_1.21.4
update to go1.21.5
2023-12-06 01:03:19 +01:00
Sebastiaan van Stijn
862caf826c
update to go1.21.5
go1.21.5 (released 2023-12-05) includes security fixes to the go command,
and the net/http and path/filepath packages, as well as bug fixes to the
compiler, the go command, the runtime, and the crypto/rand, net, os, and
syscall packages. See the Go 1.21.5 milestone on our issue tracker for
details:

- https://github.com/golang/go/issues?q=milestone%3AGo1.21.5+label%3ACherryPickApproved
- full diff: https://github.com/golang/go/compare/go1.21.4...go1.21.5

from the security mailing:

[security] Go 1.21.5 and Go 1.20.12 are released

Hello gophers,

We have just released Go versions 1.21.5 and 1.20.12, minor point releases.

These minor releases include 3 security fixes following the security policy:

- net/http: limit chunked data overhead

  A malicious HTTP sender can use chunk extensions to cause a receiver
  reading from a request or response body to read many more bytes from
  the network than are in the body.

  A malicious HTTP client can further exploit this to cause a server to
  automatically read a large amount of data (up to about 1GiB) when a
  handler fails to read the entire body of a request.

  Chunk extensions are a little-used HTTP feature which permit including
  additional metadata in a request or response body sent using the chunked
  encoding. The net/http chunked encoding reader discards this metadata.
  A sender can exploit this by inserting a large metadata segment with
  each byte transferred. The chunk reader now produces an error if the
  ratio of real body to encoded bytes grows too small.

  Thanks to Bartek Nowotarski for reporting this issue.

  This is CVE-2023-39326 and Go issue https://go.dev/issue/64433.

- cmd/go: go get may unexpectedly fallback to insecure git

  Using go get to fetch a module with the ".git" suffix may unexpectedly
  fallback to the insecure "git://" protocol if the module is unavailable
  via the secure "https://" and "git+ssh://" protocols, even if GOINSECURE
  is not set for said module. This only affects users who are not using
  the module proxy and are fetching modules directly (i.e. GOPROXY=off).

  Thanks to David Leadbeater for reporting this issue.

  This is CVE-2023-45285 and Go issue https://go.dev/issue/63845.

- path/filepath: retain trailing \ when cleaning paths like \\?\c:\

  Go 1.20.11 and Go 1.21.4 inadvertently changed the definition of the
  volume name in Windows paths starting with \\?\, resulting in
  filepath.Clean(\\?\c:\) returning \\?\c: rather than \\?\c:\ (among
  other effects). The previous behavior has been restored.

  This is an update to CVE-2023-45283 and Go issue https://go.dev/issue/64028.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-05 23:27:58 +01:00
Sebastiaan van Stijn
0bf6ffba43
update to go1.21.4
go1.21.4 (released 2023-11-07) includes security fixes to the path/filepath
package, as well as bug fixes to the linker, the runtime, the compiler, and
the go/types, net/http, and runtime/cgo packages. See the Go 1.21.4 milestone
on our issue tracker for details:

- https://github.com/golang/go/issues?q=milestone%3AGo1.21.4+label%3ACherryPickApproved
- full diff: https://github.com/golang/go/compare/go1.21.3...go1.21.4

from the security mailing:

[security] Go 1.21.4 and Go 1.20.11 are released

Hello gophers,

We have just released Go versions 1.21.4 and 1.20.11, minor point releases.

These minor releases include 2 security fixes following the security policy:

- path/filepath: recognize `\??\` as a Root Local Device path prefix.

  On Windows, a path beginning with `\??\` is a Root Local Device path equivalent
  to a path beginning with `\\?\`. Paths with a `\??\` prefix may be used to
  access arbitrary locations on the system. For example, the path `\??\c:\x`
  is equivalent to the more common path c:\x.

  The filepath package did not recognize paths with a `\??\` prefix as special.

  Clean could convert a rooted path such as `\a\..\??\b` into
  the root local device path `\??\b`. It will now convert this
  path into `.\??\b`.

  `IsAbs` did not report paths beginning with `\??\` as absolute.
  It now does so.

  VolumeName now reports the `\??\` prefix as a volume name.

  `Join(`\`, `??`, `b`)` could convert a seemingly innocent
  sequence of path elements into the root local device path
  `\??\b`. It will now convert this to `\.\??\b`.

  This is CVE-2023-45283 and https://go.dev/issue/63713.

- path/filepath: recognize device names with trailing spaces and superscripts

  The `IsLocal` function did not correctly detect reserved names in some cases:

  - reserved names followed by spaces, such as "COM1 ".
  - "COM" or "LPT" followed by a superscript 1, 2, or 3.

  `IsLocal` now correctly reports these names as non-local.

  This is CVE-2023-45284 and https://go.dev/issue/63713.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-05 23:27:08 +01:00
Sebastiaan van Stijn
08e4e88482
daemon: raise default minimum API version to v1.24
The daemon currently provides support for API versions all the way back
to v1.12, which is the version of the API that shipped with docker 1.0. On
Windows, the minimum supported version is v1.24.

Such old versions of the client are rare, and supporting older API versions
has accumulated significant amounts of code to remain backward-compatible
(which is largely untested, and a "best-effort" at most).

This patch updates the minimum API version to v1.24, which is the fallback
API version used when API-version negotiation fails. The intent is to start
deprecating older API versions, but no code is removed yet as part of this
patch, and a DOCKER_MIN_API_VERSION environment variable is added, which
allows overriding the minimum version (to allow restoring the behavior from
before this patch).

With this patch the daemon defaults to API v1.24 as minimum:

    docker version
    Client:
     Version:           24.0.2
     API version:       1.43
     Go version:        go1.20.4
     Git commit:        cb74dfc
     Built:             Thu May 25 21:50:49 2023
     OS/Arch:           linux/arm64
     Context:           default

    Server:
     Engine:
      Version:          dev
      API version:      1.44 (minimum version 1.24)
      Go version:       go1.21.3
      Git commit:       0322a29b9ef8806aaa4b45dc9d9a2ebcf0244bf4
      Built:            Mon Dec  4 15:22:17 2023
      OS/Arch:          linux/arm64
      Experimental:     false
     containerd:
      Version:          v1.7.9
      GitCommit:        4f03e100cb967922bec7459a78d16ccbac9bb81d
     runc:
      Version:          1.1.10
      GitCommit:        v1.1.10-0-g18a0cb0
     docker-init:
      Version:          0.19.0
      GitCommit:        de40ad0

Trying to use an older version of the API produces an error:

    DOCKER_API_VERSION=1.23 docker version
    Client:
     Version:           24.0.2
     API version:       1.23 (downgraded from 1.43)
     Go version:        go1.20.4
     Git commit:        cb74dfc
     Built:             Thu May 25 21:50:49 2023
     OS/Arch:           linux/arm64
     Context:           default
    Error response from daemon: client version 1.23 is too old. Minimum supported API version is 1.24, please upgrade your client to a newer version

To restore the previous minimum, users can start the daemon with the
DOCKER_MIN_API_VERSION environment variable set:

    DOCKER_MIN_API_VERSION=1.12 dockerd

API 1.12 is the oldest supported API version on Linux;

    docker version
    Client:
     Version:           24.0.2
     API version:       1.43
     Go version:        go1.20.4
     Git commit:        cb74dfc
     Built:             Thu May 25 21:50:49 2023
     OS/Arch:           linux/arm64
     Context:           default

    Server:
     Engine:
      Version:          dev
      API version:      1.44 (minimum version 1.12)
      Go version:       go1.21.3
      Git commit:       0322a29b9ef8806aaa4b45dc9d9a2ebcf0244bf4
      Built:            Mon Dec  4 15:22:17 2023
      OS/Arch:          linux/arm64
      Experimental:     false
     containerd:
      Version:          v1.7.9
      GitCommit:        4f03e100cb967922bec7459a78d16ccbac9bb81d
     runc:
      Version:          1.1.10
      GitCommit:        v1.1.10-0-g18a0cb0
     docker-init:
      Version:          0.19.0
      GitCommit:        de40ad0

When using the `DOCKER_MIN_API_VERSION` with a version of the API that
is not supported, an error is produced when starting the daemon;

    DOCKER_MIN_API_VERSION=1.11 dockerd --validate
    invalid DOCKER_MIN_API_VERSION: minimum supported API version is 1.12: 1.11

    DOCKER_MIN_API_VERSION=1.45 dockerd --validate
    invalid DOCKER_MIN_API_VERSION: maximum supported API version is 1.44: 1.45

Specifying a malformed API version also produces the same error;

    DOCKER_MIN_API_VERSION=hello dockerd --validate
    invalid DOCKER_MIN_API_VERSION: minimum supported API version is 1.12: hello

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-05 23:11:02 +01:00
Sebastiaan van Stijn
e72c4818c4
Dockerfile: update to Debian "bookworm" (current stable)
Also switch yamllint to be installed from debian's packages, which are
currently at v1.29.0.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-11-30 11:50:34 +01:00
Sebastiaan van Stijn
65cfcc28ab
hack/dind: update comments around AppArmor
Provide more context to the steps we're doing.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-11-27 14:48:51 +01:00
Sebastiaan van Stijn
cfb8ca520a
hack/dind-systemd: make AppArmor work with systemd enabled
On bookworm, AppArmor failed to start inside the container, which can be
seen at startup of the dev-container:

    Created symlink /etc/systemd/system/systemd-firstboot.service → /dev/null.
    Created symlink /etc/systemd/system/systemd-udevd.service → /dev/null.
    Created symlink /etc/systemd/system/multi-user.target.wants/docker-entrypoint.service → /etc/systemd/system/docker-entrypoint.service.
    hack/dind-systemd: starting /lib/systemd/systemd --show-status=false --unit=docker-entrypoint.target
    systemd 252.17-1~deb12u1 running in system mode (+PAM +AUDIT +SELINUX +APPARMOR +IMA +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 -IDN +IPTC +KMOD +LIBCRYPTSETUP +LIBFDISK +PCRE2 -PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD -BPF_FRAMEWORK -XKBCOMMON +UTMP +SYSVINIT default-hierarchy=unified)
    Detected virtualization docker.
    Detected architecture x86-64.
    modprobe@configfs.service: Deactivated successfully.
    modprobe@dm_mod.service: Deactivated successfully.
    modprobe@drm.service: Deactivated successfully.
    modprobe@efi_pstore.service: Deactivated successfully.
    modprobe@fuse.service: Deactivated successfully.
    modprobe@loop.service: Deactivated successfully.
    apparmor.service: Starting requested but asserts failed.
    proc-sys-fs-binfmt_misc.automount: Got automount request for /proc/sys/fs/binfmt_misc, triggered by 49 (systemd-binfmt)
    + source /etc/docker-entrypoint-cmd
    ++ hack/make.sh dynbinary test-integration

When checking "aa-status", an error was printed that the filesystem was
not mounted:

    aa-status
    apparmor filesystem is not mounted.
    apparmor module is loaded.

Checking if "local-fs.target" was loaded, that seemed to be the case;

    systemctl status local-fs.target
    ● local-fs.target - Local File Systems
         Loaded: loaded (/lib/systemd/system/local-fs.target; static)
         Active: active since Mon 2023-11-27 10:48:38 UTC; 18s ago
           Docs: man:systemd.special(7)

However, **on the host**, "/sys/kernel/security" has a mount, which was not
present inside the container:

    mount | grep securityfs
    securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)

Interestingly, on `debian:bullseye`, this was not the case either; no
`securityfs` mount was present inside the container, and apparmor actually
failed to start, but succeeded silently:

    mount | grep securityfs
    systemctl start apparmor
    systemctl status apparmor
    ● apparmor.service - Load AppArmor profiles
         Loaded: loaded (/lib/systemd/system/apparmor.service; enabled; vendor preset: enabled)
         Active: active (exited) since Mon 2023-11-27 11:59:09 UTC; 44s ago
           Docs: man:apparmor(7)
                 https://gitlab.com/apparmor/apparmor/wikis/home/
        Process: 43 ExecStart=/lib/apparmor/apparmor.systemd reload (code=exited, status=0/SUCCESS)
       Main PID: 43 (code=exited, status=0/SUCCESS)
            CPU: 10ms

    Nov 27 11:59:09 9519f89cade1 apparmor.systemd[43]: Not starting AppArmor in container

Same, using the `/etc/init.d/apparmor` script:

    /etc/init.d/apparmor start
    Starting apparmor (via systemctl): apparmor.service.
    echo $?
    0

And apparmor was not actually active:

    aa-status
    apparmor module is loaded.
    apparmor filesystem is not mounted.

    aa-enabled
    Maybe - policy interface not available.

After further investigating, I found that the non-systemd dind script
had a mount for AppArmor, which was added in 31638ab2ad

The systemd variant was missing this mount, which may have gone unnoticed
because `debian:bullseye` was silently ignoring this when starting the
apparmor service.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-11-27 14:47:59 +01:00
Sebastiaan van Stijn
7786f8512b
Revert "testing: temporarily pin docker-py tests to use "bullseye""
This reverts commit 19d860fa9d.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-11-21 23:53:45 +01:00
Sebastiaan van Stijn
4394c61e6c
testing: update docker-py to 7.0.0b1
https://github.com/docker/docker-py/compare/6.1.3...7.0.0b1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-11-21 23:53:13 +01:00
Sebastiaan van Stijn
68e73ceb67
update containerd binary to v1.7.9
full diff: https://github.com/containerd/containerd/compare/v1.7.8...v1.7.9
release notes: https://github.com/containerd/containerd/releases/tag/v1.7.9

Notable Updates

- update runc binary to v1.1.10
- vendor: upgrade OpenTelemetry to v1.19.0 / v0.45.0
- Expose usage of cri-api v1alpha2
- integration: deflake TestIssue9103
- fix: shimv1 leak issue
- cri: add deprecation warnings for mirrors, auths, and configs
- Update hcsshim tag to v0.11.4
- Expose usage of deprecated features

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-11-19 13:34:04 +01:00
Sebastiaan van Stijn
15bcc707e6
update runc binary to v1.1.10
- full diff: https://github.com/opencontainers/runc/compare/v1.1.9...v1.1.10
- release notes: https://github.com/opencontainers/runc/releases/tag/v1.1.10

This is the tenth (and most likely final) patch release in the 1.1.z
release branch of runc. It mainly fixes a few issues in cgroups, and a
umask-related issue in tmpcopyup.

- Add support for `hugetlb.<pagesize>.rsvd` limiting and accounting.
  Fixes the issue of postgres failing when hugepage limits are set.
- Fixed permissions of a newly created directories to not depend on the value
  of umask in tmpcopyup feature implementation.
- libcontainer: cgroup v1 GetStats now ignores missing `kmem.limit_in_bytes`
  (fixes the compatibility with Linux kernel 6.1+).
- Fix a semi-arbitrary cgroup write bug when given a malicious hugetlb
  configuration. This issue is not a security issue because it requires a
  malicious config.json, which is outside of our threat model.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-11-13 16:01:02 +01:00
Djordje Lukic
d60de4cafd
Move the golangci-lint configuration file to the root dir
This will help other tools and IDEs find our lint configuration

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-10-30 13:48:58 +01:00
Sebastiaan van Stijn
4356962c69
update containerd binary to v1.7.8
release notes: https://github.com/containerd/containerd/releases/tag/v1.7.8
full diff: https://github.com/containerd/containerd/compare/v1.7.6...v1.7.8

Notable Updates

- Fix ambiguous TLS fallback
- Update Go to 1.20.10
- Add a new image label on converted schema 1 images
- Fix handling for missing basic auth credentials
- Fix potential deadlock in create handler for containerd-shim-runc-v2

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-27 00:48:26 +02:00
Cory Snider
52da88201c hack/validate: stop suppressing Xattrs deprecation
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-10-23 16:21:41 -04:00
Sebastiaan van Stijn
ec69501e94
api/types: move ServiceCreateResponse, and generate from swagger
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-12 11:29:23 +02:00
Sebastiaan van Stijn
5b3e6555a3
api/types: move ServiceUpdateResponse
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-12 11:29:22 +02:00
Sebastiaan van Stijn
48cacbca24
api/types: move image-types to api/types/image
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-12 11:29:20 +02:00
Sebastiaan van Stijn
056be8b7b0
Merge pull request #46473 from thaJeztah/c8d_logs
migrate to github.com/containerd/log v0.1.0
2023-10-12 10:35:02 +02:00
Sebastiaan van Stijn
bb8bc1ffc8
update to go1.21.3
go1.21.3 (released 2023-10-10) includes a security fix to the net/http package.
See the Go 1.21.3 milestone on our issue tracker for details:

https://github.com/golang/go/issues?q=milestone%3AGo1.21.3+label%3ACherryPickApproved

full diff: https://github.com/golang/go/compare/go1.21.2...go1.21.3

From the security mailing:

[security] Go 1.21.3 and Go 1.20.10 are released

Hello gophers,

We have just released Go versions 1.21.3 and 1.20.10, minor point releases.

These minor releases include 1 security fixes following the security policy:

- net/http: rapid stream resets can cause excessive work

  A malicious HTTP/2 client which rapidly creates requests and
  immediately resets them can cause excessive server resource consumption.
  While the total number of requests is bounded to the
  http2.Server.MaxConcurrentStreams setting, resetting an in-progress
  request allows the attacker to create a new request while the existing
  one is still executing.

  HTTP/2 servers now bound the number of simultaneously executing
  handler goroutines to the stream concurrency limit. New requests
  arriving when at the limit (which can only happen after the client
  has reset an existing, in-flight request) will be queued until a
  handler exits. If the request queue grows too large, the server
  will terminate the connection.

  This issue is also fixed in golang.org/x/net/http2 v0.17.0,
  for users manually configuring HTTP/2.

  The default stream concurrency limit is 250 streams (requests)
  per HTTP/2 connection. This value may be adjusted using the
  golang.org/x/net/http2 package; see the Server.MaxConcurrentStreams
  setting and the ConfigureServer function.

  This is CVE-2023-39325 and Go issue https://go.dev/issue/63417.
  This is also tracked by CVE-2023-44487.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-11 20:01:17 +02:00
Sebastiaan van Stijn
3bc45d78c9
update to go1.21.2
go1.21.2 (released 2023-10-05) includes one security fixes to the cmd/go package,
as well as bug fixes to the compiler, the go command, the linker, the runtime,
and the runtime/metrics package. See the Go 1.21.2 milestone on our issue
tracker for details:

https://github.com/golang/go/issues?q=milestone%3AGo1.21.2+label%3ACherryPickApproved

full diff: https://github.com/golang/go/compare/go1.21.1...go1.21.2

From the security mailing:

[security] Go 1.21.2 and Go 1.20.9 are released

Hello gophers,

We have just released Go versions 1.21.2 and 1.20.9, minor point releases.

These minor releases include 1 security fixes following the security policy:

- cmd/go: line directives allows arbitrary execution during build

  "//line" directives can be used to bypass the restrictions on "//go:cgo_"
  directives, allowing blocked linker and compiler flags to be passed during
  compliation. This can result in unexpected execution of arbitrary code when
  running "go build". The line directive requires the absolute path of the file in
  which the directive lives, which makes exploting this issue significantly more
  complex.

  This is CVE-2023-39323 and Go issue https://go.dev/issue/63211.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-11 20:01:05 +02:00
Sebastiaan van Stijn
cff4f20c44
migrate to github.com/containerd/log v0.1.0
The github.com/containerd/containerd/log package was moved to a separate
module, which will also be used by upcoming (patch) releases of containerd.

This patch moves our own uses of the package to use the new module.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-11 17:52:23 +02:00
Sebastiaan van Stijn
a293cd6ab9
vendor: github.com/containerd/containerd v1.7.7
full diff: https://github.com/containerd/containerd/compare/v1.7.6...v1.7.7

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-10 14:47:44 +02:00
Sebastiaan van Stijn
e2c23ed57d
Merge pull request #46069 from thaJeztah/update_to_go1.21
update to go1.21.1, default to GOTOOLCHAIN=local
2023-09-28 20:24:21 +02:00
Bjorn Neergaard
16063c7456
vendor.mod: bump go line to 1.20
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-09-28 10:01:25 -06:00
Sebastiaan van Stijn
e465ebf2f3
update to go1.21.1, and fix download-URL
This required changes to the download-URL, as downloads are now provided
using the full version (including the `.0` patch version);

    curl -sI https://go.dev/dl/go1.21.windows-amd64.zip | grep 'location'
    location: https://dl.google.com/go/go1.21.windows-amd64.zip

    curl -sI https://dl.google.com/go/go1.21.windows-amd64.zip
    HTTP/2 404
    # ...

    curl -sI https://dl.google.com/go/go1.21.0.windows-amd64.zip
    HTTP/2 200
    # ...

Unfortunately this also means that the GO_VERSION can no longer be set to
versions lower than 1.21.0 (without additional changes), because older
versions do NOT provide the `.0` version, and Go 1.21.0 and up, no longer
provides URLs _without_ the `.0` version.

Co-authored-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-28 00:51:34 +02:00
Sebastiaan van Stijn
aa282973d4
Dockerfile: use GOTOOLCHAIN=local
Related discussion in https://github.com/docker-library/golang/issues/472

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-28 00:51:18 +02:00
Albin Kerouanton
41c186c344
hack: Run integration tests through Delve
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-09-26 15:42:11 +02:00
Sebastiaan van Stijn
24102aa8ca
update containerd binary to v1.7.6
Update the version used in testing;

full diff: https://github.com/containerd/containerd/compare/v1.7.3...v1.7.6

v1.7.6 release notes:

full diff: https://github.com/containerd/containerd/compare/v1.7.5...v1.7.6

The sixth patch release for containerd 1.7 contains various fixes and updates.

- Fix log package for clients overwriting the global logger
- Fix blockfile snapshotter copy on Darwin
- Add support for Linux usernames on non-Linux platforms
- Update Windows platform matcher to invoke stable ABI compability function
- Update Golang to 1.20.8
- Update push to inherit distribution sources from parent

v1.7.5 release notes:

full diff: https://github.com/containerd/containerd/compare/v1.7.4...v1.7.5

The fifth patch release for containerd 1.7 fixes a versioning issue from
the previous release and includes some internal logging API changes.

v1.7.4 release notes:

full diff: https://github.com/containerd/containerd/compare/v1.7.3...v1.7.4

The fourth patch release for containerd 1.7 contains remote differ plugin support,
a new block file based snapshotter, and various fixes and updates.

Notable Updates

- Add blockfile snapshotter
- Add remote/proxy differ
- Update runc binary to v1.1.9
- Cri: Don't use rel path for image volumes
- Allow attaching to any combination of stdin/out/err
- Fix ro mount option being passed
- Fix leaked shim caused by high IO pressure
- Add configurable mount options to overlay snapshotter

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-15 12:21:07 +02:00
Sebastiaan van Stijn
3b04fd10e8
Merge pull request #46251 from akerouanton/libnet-forbid-duplicated-network-names
libnet: Make sure network names are unique
2023-09-12 16:53:14 +02:00
Albin Kerouanton
78479b1915
libnet: Make sure network names are unique
Fixes #18864, #20648, #33561, #40901.

[This GH comment][1] makes clear network name uniqueness has never been
enforced due to the eventually consistent nature of Classic Swarm
datastores:

> there is no guaranteed way to check for duplicates across a cluster of
> docker hosts.

And this is further confirmed by other comments made by @mrjana in that
same issue, eg. [this one][2]:

> we want to adopt a schema which can pave the way in the future for a
> completely decentralized cluster of docker hosts (if scalability is
> needed).

This decentralized model is what Classic Swarm was trying to be. It's
been superseded since then by Docker Swarm, which has a centralized
control plane.

To circumvent this drawback, the `NetworkCreate` endpoint accepts a
`CheckDuplicate` flag. However it's not perfectly reliable as it won't
catch concurrent requests.

Due to this design decision, API clients like Compose have to implement
workarounds to make sure names are really unique (eg.
docker/compose#9585). And the daemon itself has seen a string of issues
due to that decision, including some that aren't fixed to this day (for
instance moby/moby#40901):

> The problem is, that if you specify a network for a container using
> the ID, it will add that network to the container but it will then
> change it to reference the network by using the name.

To summarize, this "feature" is broken, has no practical use and is a
source of pain for Docker users and API consumers. So let's just remove
it for _all_ API versions.

[1]: https://github.com/moby/moby/issues/18864#issuecomment-167201414
[2]: https://github.com/moby/moby/issues/18864#issuecomment-167202589

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-09-12 10:40:13 +02:00
Sebastiaan van Stijn
c41121cc48
update to go1.20.8
go1.20.8 (released 2023-09-06) includes two security fixes to the html/template
package, as well as bug fixes to the compiler, the go command, the runtime,
and the crypto/tls, go/types, net/http, and path/filepath packages. See the
Go 1.20.8 milestone on our issue tracker for details:

https://github.com/golang/go/issues?q=milestone%3AGo1.20.8+label%3ACherryPickApproved

full diff: https://github.com/golang/go/compare/go1.20.7...go1.20.8

From the security mailing:

[security] Go 1.21.1 and Go 1.20.8 are released

Hello gophers,

We have just released Go versions 1.21.1 and 1.20.8, minor point releases.

These minor releases include 4 security fixes following the security policy:

- cmd/go: go.mod toolchain directive allows arbitrary execution
  The go.mod toolchain directive, introduced in Go 1.21, could be leveraged to
  execute scripts and binaries relative to the root of the module when the "go"
  command was executed within the module. This applies to modules downloaded using
  the "go" command from the module proxy, as well as modules downloaded directly
  using VCS software.

  Thanks to Juho Nurminen of Mattermost for reporting this issue.

  This is CVE-2023-39320 and Go issue https://go.dev/issue/62198.

- html/template: improper handling of HTML-like comments within script contexts
  The html/template package did not properly handle HMTL-like "<!--" and "-->"
  comment tokens, nor hashbang "#!" comment tokens, in <script> contexts. This may
  cause the template parser to improperly interpret the contents of <script>
  contexts, causing actions to be improperly escaped. This could be leveraged to
  perform an XSS attack.

  Thanks to Takeshi Kaneko (GMO Cybersecurity by Ierae, Inc.) for reporting this
  issue.

  This is CVE-2023-39318 and Go issue https://go.dev/issue/62196.

- html/template: improper handling of special tags within script contexts
  The html/template package did not apply the proper rules for handling occurrences
  of "<script", "<!--", and "</script" within JS literals in <script> contexts.
  This may cause the template parser to improperly consider script contexts to be
  terminated early, causing actions to be improperly escaped. This could be
  leveraged to perform an XSS attack.

  Thanks to Takeshi Kaneko (GMO Cybersecurity by Ierae, Inc.) for reporting this
  issue.

  This is CVE-2023-39319 and Go issue https://go.dev/issue/62197.

- crypto/tls: panic when processing post-handshake message on QUIC connections
  Processing an incomplete post-handshake message for a QUIC connection caused a panic.

  Thanks to Marten Seemann for reporting this issue.

  This is CVE-2023-39321 and CVE-2023-39322 and Go issue https://go.dev/issue/62266.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-11 15:47:00 +02:00
Sebastiaan van Stijn
6ce5aa1cd5
Merge pull request #46434 from vvoland/test-ignore-cgroup-check
hack: Add TEST_IGNORE_CGROUP_CHECK
2023-09-08 23:23:18 +02:00
Sebastiaan van Stijn
d582a26107
Merge pull request #46439 from akerouanton/no-optimizations
hack: compile w/o optimizations & inlining when debugging
2023-09-08 23:09:33 +02:00
Albin Kerouanton
820f37b1a6
hack: compile w/o optimizations & inlining when debugging
Without these compile flags, Delve is unable to report the value of some
variables and it's not possible to jump into inlined code.

As the contributing docs already mention that `DOCKER_DEBUG` should
disable "build optimizations", the env var is reused here instead of
introducing a new one.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-09-08 17:16:58 +02:00
Sebastiaan van Stijn
6dcefa3739
Merge pull request #46431 from rumpl/fix-ci-timeout
Use process substitution to redirect to tee
2023-09-08 15:28:09 +02:00
Paweł Gronowski
c2004fb8c4
hack: Add TEST_IGNORE_CGROUP_CHECK
Add env variable that allows to run integration-cli tests with cgroup v2 enabled.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-09-08 13:41:03 +02:00
Djordje Lukic
3d8b8dc09a
Use process substitution to redirect to tee
In some cases, when the daemon launched by a test panics and quits, the
cleanup code would end with an error when trying to kill it by its pid.
In those cases the whole suite will end up waiting for the daemon that
we start in .integration-daemon-start to finish and we end up waiting 2
hours for the CI to cancel after a timeout.

Using process substitution makes the integration tests quit.

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-09-08 10:56:21 +02:00
Brian Goff
e8dc902781 Wire up tests to support otel tracing
Integration tests will now configure clients to propagate traces as well
as create spans for all tests.

Some extra changes were needed (or desired for trace propagation) in the
test helpers to pass through tracing spans via context.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2023-09-07 18:38:22 +00:00
Djordje Lukic
75097cf09a
test: remove platform related timeout values
These were dependent on the DOCKER_ENGINE_GOARCH environment variable
but this var was no longer set. There was also some weird check to see
if the architecture is "windows" which doesn't make sense. Seeing how
nothing failed ever since the TIMEOUT was no longer platform-dependent
we can safely remove this check.

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-08-31 10:39:28 +02:00
Sebastiaan van Stijn
a9a0ffaf51
testing: update docker-py 6.1.3
full diff: https://github.com/docker/docker-py/compare/6.0.1...6.1.3

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-29 17:31:33 +02:00
Sebastiaan van Stijn
f6959bc597
test-docker-py: skip AttachContainerTest::test_run_container_reading_socket_ws
Tests are failing with this error:

    E   ValueError: scheme http+docker is invalid

Which is reported in docker-py in https://github.com/docker/docker-py/issues/1478.
Not sure what changed in the tests, but could be due to updated Python
version or dependencies, but let's skip it for now.

Test failure:

    ___________ AttachContainerTest.test_run_container_reading_socket_ws ___________
    tests/integration/api_container_test.py:1245: in test_run_container_reading_socket_ws
        pty_stdout = self.client.attach_socket(container, opts, ws=True)
    docker/utils/decorators.py:19: in wrapped
        return f(self, resource_id, *args, **kwargs)
    docker/api/container.py:98: in attach_socket
        return self._attach_websocket(container, params)
    docker/utils/decorators.py:19: in wrapped
        return f(self, resource_id, *args, **kwargs)
    docker/api/client.py:312: in _attach_websocket
        return self._create_websocket_connection(full_url)
    docker/api/client.py:315: in _create_websocket_connection
        return websocket.create_connection(url)
    /usr/local/lib/python3.7/site-packages/websocket/_core.py:601: in create_connection
        websock.connect(url, **options)
    /usr/local/lib/python3.7/site-packages/websocket/_core.py:245: in connect
        options.pop('socket', None))
    /usr/local/lib/python3.7/site-packages/websocket/_http.py:117: in connect
        hostname, port, resource, is_secure = parse_url(url)
    /usr/local/lib/python3.7/site-packages/websocket/_url.py:62: in parse_url
        raise ValueError("scheme %s is invalid" % scheme)
    E   ValueError: scheme http+docker is invalid
    ------- generated xml file: /src/bundles/test-docker-py/junit-report.xml -------

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-29 17:31:09 +02:00
Sebastiaan van Stijn
e0f171f342
testing: update docker-py 6.0.1
release notes: https://github.com/docker/docker-py/releases/tag/6.0.1

full diff: https://github.com/docker/docker-py/compare/5.0.3...6.0.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-29 17:31:07 +02:00
Sebastiaan van Stijn
79a0f82ca1
test-docker-py: remove comment about docker 17.06 limitations
Our dev-container now has buildx installed, so we're no longer
limited to 17.06.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-29 17:29:49 +02:00