Commit graph

48284 commits

Author SHA1 Message Date
Paweł Gronowski
68eead6628
Merge pull request #46901 from thaJeztah/move_more_backend_types
api/types: move Plugin-types and NetworkListConfig to api/types/backend
2023-12-06 14:51:49 +01:00
Sebastiaan van Stijn
debcb76939
Merge pull request #46895 from vvoland/integration-build-cleanenv
integration/build: Use setupTest
2023-12-06 02:26:43 +01:00
Sebastiaan van Stijn
7cb1efebec
api/types: move NetworkListConfig to api/types/backend
This struct is intended for internal use only for the backend, and is
not intended to be used externally.

This moves the plugin-related `NetworkListConfig` types to the backend
package to prevent it being imported in the client, and to make it more
clear that this is part of internal APIs, and not public-facing.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-06 02:21:21 +01:00
Sebastiaan van Stijn
a58b0a3d9c
api/types: move Plugin-types to api/types/backend
These structs are intended for internal use only for the backend, and are
not intended to be used externally.

This moves the plugin-related `PluginRmConfig`, `PluginEnableConfig`, and
`PluginDisableConfig` types to the backend package to prevent them being
imported in the client, and to make it more clear that this is part of
internal APIs, and not public-facing.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-06 02:16:02 +01:00
Sebastiaan van Stijn
4046ae5e2f
Merge pull request #46787 from thaJeztah/update_golang_1.21.4
update to go1.21.5
2023-12-06 01:03:19 +01:00
Sebastiaan van Stijn
6af9204938
Merge pull request #46887 from thaJeztah/minimum_api_version
daemon: raise default minimum API version to v1.24
2023-12-06 00:15:03 +01:00
Sebastiaan van Stijn
862caf826c
update to go1.21.5
go1.21.5 (released 2023-12-05) includes security fixes to the go command,
and the net/http and path/filepath packages, as well as bug fixes to the
compiler, the go command, the runtime, and the crypto/rand, net, os, and
syscall packages. See the Go 1.21.5 milestone on our issue tracker for
details:

- https://github.com/golang/go/issues?q=milestone%3AGo1.21.5+label%3ACherryPickApproved
- full diff: https://github.com/golang/go/compare/go1.21.4...go1.21.5

from the security mailing:

[security] Go 1.21.5 and Go 1.20.12 are released

Hello gophers,

We have just released Go versions 1.21.5 and 1.20.12, minor point releases.

These minor releases include 3 security fixes following the security policy:

- net/http: limit chunked data overhead

  A malicious HTTP sender can use chunk extensions to cause a receiver
  reading from a request or response body to read many more bytes from
  the network than are in the body.

  A malicious HTTP client can further exploit this to cause a server to
  automatically read a large amount of data (up to about 1GiB) when a
  handler fails to read the entire body of a request.

  Chunk extensions are a little-used HTTP feature which permit including
  additional metadata in a request or response body sent using the chunked
  encoding. The net/http chunked encoding reader discards this metadata.
  A sender can exploit this by inserting a large metadata segment with
  each byte transferred. The chunk reader now produces an error if the
  ratio of real body to encoded bytes grows too small.

  Thanks to Bartek Nowotarski for reporting this issue.

  This is CVE-2023-39326 and Go issue https://go.dev/issue/64433.

- cmd/go: go get may unexpectedly fallback to insecure git

  Using go get to fetch a module with the ".git" suffix may unexpectedly
  fallback to the insecure "git://" protocol if the module is unavailable
  via the secure "https://" and "git+ssh://" protocols, even if GOINSECURE
  is not set for said module. This only affects users who are not using
  the module proxy and are fetching modules directly (i.e. GOPROXY=off).

  Thanks to David Leadbeater for reporting this issue.

  This is CVE-2023-45285 and Go issue https://go.dev/issue/63845.

- path/filepath: retain trailing \ when cleaning paths like \\?\c:\

  Go 1.20.11 and Go 1.21.4 inadvertently changed the definition of the
  volume name in Windows paths starting with \\?\, resulting in
  filepath.Clean(\\?\c:\) returning \\?\c: rather than \\?\c:\ (among
  other effects). The previous behavior has been restored.

  This is an update to CVE-2023-45283 and Go issue https://go.dev/issue/64028.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-05 23:27:58 +01:00
Sebastiaan van Stijn
0bf6ffba43
update to go1.21.4
go1.21.4 (released 2023-11-07) includes security fixes to the path/filepath
package, as well as bug fixes to the linker, the runtime, the compiler, and
the go/types, net/http, and runtime/cgo packages. See the Go 1.21.4 milestone
on our issue tracker for details:

- https://github.com/golang/go/issues?q=milestone%3AGo1.21.4+label%3ACherryPickApproved
- full diff: https://github.com/golang/go/compare/go1.21.3...go1.21.4

from the security mailing:

[security] Go 1.21.4 and Go 1.20.11 are released

Hello gophers,

We have just released Go versions 1.21.4 and 1.20.11, minor point releases.

These minor releases include 2 security fixes following the security policy:

- path/filepath: recognize `\??\` as a Root Local Device path prefix.

  On Windows, a path beginning with `\??\` is a Root Local Device path equivalent
  to a path beginning with `\\?\`. Paths with a `\??\` prefix may be used to
  access arbitrary locations on the system. For example, the path `\??\c:\x`
  is equivalent to the more common path c:\x.

  The filepath package did not recognize paths with a `\??\` prefix as special.

  Clean could convert a rooted path such as `\a\..\??\b` into
  the root local device path `\??\b`. It will now convert this
  path into `.\??\b`.

  `IsAbs` did not report paths beginning with `\??\` as absolute.
  It now does so.

  VolumeName now reports the `\??\` prefix as a volume name.

  `Join(`\`, `??`, `b`)` could convert a seemingly innocent
  sequence of path elements into the root local device path
  `\??\b`. It will now convert this to `\.\??\b`.

  This is CVE-2023-45283 and https://go.dev/issue/63713.

- path/filepath: recognize device names with trailing spaces and superscripts

  The `IsLocal` function did not correctly detect reserved names in some cases:

  - reserved names followed by spaces, such as "COM1 ".
  - "COM" or "LPT" followed by a superscript 1, 2, or 3.

  `IsLocal` now correctly reports these names as non-local.

  This is CVE-2023-45284 and https://go.dev/issue/63713.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-05 23:27:08 +01:00
Sebastiaan van Stijn
08e4e88482
daemon: raise default minimum API version to v1.24
The daemon currently provides support for API versions all the way back
to v1.12, which is the version of the API that shipped with docker 1.0. On
Windows, the minimum supported version is v1.24.

Such old versions of the client are rare, and supporting older API versions
has accumulated significant amounts of code to remain backward-compatible
(which is largely untested, and a "best-effort" at most).

This patch updates the minimum API version to v1.24, which is the fallback
API version used when API-version negotiation fails. The intent is to start
deprecating older API versions, but no code is removed yet as part of this
patch, and a DOCKER_MIN_API_VERSION environment variable is added, which
allows overriding the minimum version (to allow restoring the behavior from
before this patch).

With this patch the daemon defaults to API v1.24 as minimum:

    docker version
    Client:
     Version:           24.0.2
     API version:       1.43
     Go version:        go1.20.4
     Git commit:        cb74dfc
     Built:             Thu May 25 21:50:49 2023
     OS/Arch:           linux/arm64
     Context:           default

    Server:
     Engine:
      Version:          dev
      API version:      1.44 (minimum version 1.24)
      Go version:       go1.21.3
      Git commit:       0322a29b9ef8806aaa4b45dc9d9a2ebcf0244bf4
      Built:            Mon Dec  4 15:22:17 2023
      OS/Arch:          linux/arm64
      Experimental:     false
     containerd:
      Version:          v1.7.9
      GitCommit:        4f03e100cb967922bec7459a78d16ccbac9bb81d
     runc:
      Version:          1.1.10
      GitCommit:        v1.1.10-0-g18a0cb0
     docker-init:
      Version:          0.19.0
      GitCommit:        de40ad0

Trying to use an older version of the API produces an error:

    DOCKER_API_VERSION=1.23 docker version
    Client:
     Version:           24.0.2
     API version:       1.23 (downgraded from 1.43)
     Go version:        go1.20.4
     Git commit:        cb74dfc
     Built:             Thu May 25 21:50:49 2023
     OS/Arch:           linux/arm64
     Context:           default
    Error response from daemon: client version 1.23 is too old. Minimum supported API version is 1.24, please upgrade your client to a newer version

To restore the previous minimum, users can start the daemon with the
DOCKER_MIN_API_VERSION environment variable set:

    DOCKER_MIN_API_VERSION=1.12 dockerd

API 1.12 is the oldest supported API version on Linux;

    docker version
    Client:
     Version:           24.0.2
     API version:       1.43
     Go version:        go1.20.4
     Git commit:        cb74dfc
     Built:             Thu May 25 21:50:49 2023
     OS/Arch:           linux/arm64
     Context:           default

    Server:
     Engine:
      Version:          dev
      API version:      1.44 (minimum version 1.12)
      Go version:       go1.21.3
      Git commit:       0322a29b9ef8806aaa4b45dc9d9a2ebcf0244bf4
      Built:            Mon Dec  4 15:22:17 2023
      OS/Arch:          linux/arm64
      Experimental:     false
     containerd:
      Version:          v1.7.9
      GitCommit:        4f03e100cb967922bec7459a78d16ccbac9bb81d
     runc:
      Version:          1.1.10
      GitCommit:        v1.1.10-0-g18a0cb0
     docker-init:
      Version:          0.19.0
      GitCommit:        de40ad0

When using the `DOCKER_MIN_API_VERSION` with a version of the API that
is not supported, an error is produced when starting the daemon;

    DOCKER_MIN_API_VERSION=1.11 dockerd --validate
    invalid DOCKER_MIN_API_VERSION: minimum supported API version is 1.12: 1.11

    DOCKER_MIN_API_VERSION=1.45 dockerd --validate
    invalid DOCKER_MIN_API_VERSION: maximum supported API version is 1.44: 1.45

Specifying a malformed API version also produces the same error;

    DOCKER_MIN_API_VERSION=hello dockerd --validate
    invalid DOCKER_MIN_API_VERSION: minimum supported API version is 1.12: hello

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-05 23:11:02 +01:00
Sebastiaan van Stijn
4fa5a79833
Merge pull request #46898 from thaJeztah/backend_types
api/types: move ContainerCreateConfig, ContainerRmConfig to api/types/backend
2023-12-05 19:44:22 +01:00
Sebastiaan van Stijn
484e6b784c
api/types: move ContainerCreateConfig, ContainerRmConfig to api/types/backend
The `ContainerCreateConfig` and `ContainerRmConfig` structs are used for
options to be passed to the backend, and are not used in client code.

Thess struct currently is intended for internal use only (for example, the
`AdjustCPUShares` is an internal implementation details to adjust the container's
config when older API versions are used).

Somewhat ironically, the signature of the Backend has a nicer UX than that
of the client's `ContainerCreate` signature (which expects all options to
be passed as separate arguments), so we may want to update that signature
to be closer to what the backend is using, but that can be left as a future
exercise.

This patch moves the `ContainerCreateConfig` and `ContainerRmConfig` structs
to the backend package to prevent it being imported in the client, and to make
it more clear that this is part of internal APIs, and not public-facing.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-05 16:41:36 +01:00
Sebastiaan van Stijn
6a810b4259
Merge pull request #46897 from thaJeztah/bump_criu_debian
Dockerfile: update criu stage to Debian 12 (Bookworm)
2023-12-05 15:40:30 +01:00
Sebastiaan van Stijn
c786c3a974
Dockerfile: update criu stage to Debian 12 (Bookworm)
Follow-up to e72c4818c4, which updated the
Dockerfile to use Debian 12 "bookworm", but forgot to update the package
repository to use for the CRIU packages. Note that the criu stage is currently
not built by default (see d3d2823edf), so to
verify the stage, it needs to be built manually;

    docker build --target=criu .

This patch adds an extra `criu --version` to the build, so that it's verified
to be "functional".

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-05 14:09:57 +01:00
Paweł Gronowski
d5eba1bfe5
integration/build: Use setupTest
These tests build new images, setupTest sets up the test cleanup
function that clears the test environment from created images,
containers, etc.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-12-05 12:49:30 +01:00
Sebastiaan van Stijn
2a38569337
Merge pull request #46891 from thaJeztah/cleanup_api_versions
integration / integration-cil: cleanup and fix API versions used in tests
2023-12-05 10:41:56 +01:00
Sebastiaan van Stijn
713c7d49a1
integration(-cli): remove skips for old daemon versions (<20.10)
This removes various skips that accounted for running the integration tests
against older versions of the daemon before 20.10 (API version v1.41). Those
versions are EOL, and we don't run tests against them.

This reverts most of e440831802, and similar
PRs.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-05 01:03:50 +01:00
Sebastiaan van Stijn
7f68e3107e
integration-cli: TestAPIClientVersionOldNotSupported: use daemon API version
Use the minimum API version as advertised by the test-daemon, instead of the
hard-coded API version from code.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-04 22:58:47 +01:00
Sebastiaan van Stijn
97549d923c
integration/container: TestInfoAPIVersioned: adjust API version
This test was using API version 1.20 to test old behavior, but the actual change
in behavior was API v1.25; see commit 6d98e344c7
and 63b5a37203.
This updates the test to use API v1.24 to test the old behavior.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-04 22:38:55 +01:00
Sebastiaan van Stijn
410feab962
integration/container: TestKillDifferentUserContainer: adjust API version
TestKillDifferentUserContainer was migrated from integration-cli in
commit 0855922cd3. Before migration, it
was not using a specific API version, so we can assume "current"
API version.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-04 22:38:52 +01:00
Cory Snider
757a004a90 libnetwork/diagnostic: drop Init method
Fold it into the constructor, because that's what the constructor is
supposed to do.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-12-04 15:13:17 -05:00
Cory Snider
f270057e0c libnetwork/diagnostic: un-embed sync.Mutex field
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-12-04 15:13:17 -05:00
Sebastiaan van Stijn
7dc5f9d0b1
Merge pull request #46884 from thaJeztah/remove_apt_mirror
Dockerfile: remove APT_MIRROR build-arg
2023-12-04 21:02:04 +01:00
Sebastiaan van Stijn
aa2fa2caa3
un-containerize TestDaemonNoSpaceLeftOnDeviceError
Commit 59b83d8aae containerized these steps,
as they didn't work well on Debian Jessie:

> Because the `mount` here will sometimes fail when run in `debian:jessie`,
> which is what the environrment hosting the test suite is running if run
> from the `Makefile`.
> Also, why the heck not containerize it, all the things.

Follow-up commits, such as 228d74842f, and
1c5806cf57 updated the Debian distro, but
also updated this comment, losing the original context (the issue was
(originally) related to Debian Jessie).

This patch changes the test back to not use containers, which seems to
work fine (at least "it worked on my machine").

    make TEST_IGNORE_CGROUP_CHECK=1 TEST_FILTER=TestDaemonNoSpaceLeftOnDeviceError DOCKER_GRAPHDRIVER=overlay2 test-integration

    === RUN   TestDockerDaemonSuite/TestDaemonNoSpaceLeftOnDeviceError
        check_test.go:589: [df36ad96a412b] daemon is not started
    --- PASS: TestDockerDaemonSuite (5.12s)
        --- PASS: TestDockerDaemonSuite/TestDaemonNoSpaceLeftOnDeviceError (5.12s)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-04 18:51:15 +01:00
Sebastiaan van Stijn
49f3e7d233
Dockerfile: remove APT_MIRROR build-arg
This option was originally added in 8ec8564691,
at which time the upstream debian package repositories were not always
reliable, so using a mirror helped with CI stability and performance.

Debian's package repositories are a lot more reliable now, so there's no
longer a need to use a mirror.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-04 13:52:58 +01:00
Sebastiaan van Stijn
df59a357ec
Merge pull request #46877 from robmry/early_error_for_cifs_url_with_port
Don't allow port in CIFS URL
2023-12-04 12:44:02 +01:00
Sebastiaan van Stijn
bcc0315003
Merge pull request #46882 from vvoland/vendor-buildkit-0.12.4
vendor: github.com/moby/buildkit v0.12.4
2023-12-04 12:42:05 +01:00
Rob Murray
baca6cb417 Don't allow port in CIFS URL
When creating a CIFS volume, generate an error if the device URL
includes a port number, for example:
   --opt device="//some.server.com:2345/thepath"

The port must be specified in the port option instead, for example:
    --opt o=username=USERNAME,password=PASSWORD,vers=3,sec=ntlmsspi,port=1234

Signed-off-by: Rob Murray <rob.murray@docker.com>
2023-12-04 10:08:09 +00:00
Paweł Gronowski
c8134fa046
vendor: github.com/moby/buildkit v0.12.4
full diff: https://github.com/moby/buildkit/compare/v0.12.3...v0.12.4

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-12-04 10:56:20 +01:00
Martin Jirku
a189832454
add TaskStatus to swagger docs
Add the TaskStatus, PortStatus and ContainerStatus to api docs. TaskStatus was moved to the swagger definitions root from anonymous type definition, and PortStatus and Container Status are its dependencies.

Signed-off-by: Martin Jirku <martin@jirku.sk>
2023-12-03 16:12:32 +01:00
Sebastiaan van Stijn
029519a149
Merge pull request #46876 from thaJeztah/bump_mux
vendor: github.com/gorilla/mux v1.8.1
2023-12-01 16:26:10 +01:00
Sebastiaan van Stijn
faa843fcef
vendor: github.com/gorilla/mux v1.8.1
full diff: https://github.com/gorilla/mux/compare/v1.8.0...v1.8.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-01 12:48:09 +01:00
Sebastiaan van Stijn
f179243876
Merge pull request #46875 from thaJeztah/bump_cobra
vendor: github.com/spf13/cobra v1.8.0
2023-12-01 12:09:44 +01:00
Sebastiaan van Stijn
028ed1bb22
vendor: github.com/spf13/cobra v1.8.0
full diff: https://github.com/spf13/cobra/compare/v1.7.0...v1.8.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-01 11:01:59 +01:00
Sebastiaan van Stijn
bb99b3f02e
Merge pull request #46871 from thaJeztah/less_cobra_customisation
cmd/dockerd: remove various cobra templating
2023-12-01 10:58:07 +01:00
Sebastiaan van Stijn
d3533eee68
Merge pull request #46849 from robmry/37937_explicit_bridge_mtu
Explicitly set MTU on bridge devices.
2023-11-30 22:30:11 +01:00
Tianon Gravi
488fe76eb8
Merge pull request #45789 from thaJeztah/update_debian
Dockerfile: update to Debian "bookworm", and update frozen images to debian:bookworm-slim
2023-11-30 12:33:48 -08:00
Sebastiaan van Stijn
75546e1e46
Merge pull request #46865 from vvoland/c8d-fix-DockerRegistryAuthTokenSuite
c8d/integration-cli: Adjust DockerRegistryAuthTokenSuite
2023-11-30 17:36:45 +01:00
Sebastiaan van Stijn
ec7c2b784a
Merge pull request #46268 from thaJeztah/detachcontext
container: internalize InitAttachContext
2023-11-30 17:03:34 +01:00
Sebastiaan van Stijn
08035dc805
Merge pull request #46868 from vvoland/c8d-skip-TestListDanglingImagesWithDigests
c8d/integration-cli: Skip part of TestListDanglingImagesWithDigests
2023-11-30 16:16:02 +01:00
Sebastiaan van Stijn
e18f5a5304
container: internalize InitAttachContext
Move the initialization logic to the attachContext itself, so that
the container doesn't have to be aware about mutexes and other logic.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-11-30 15:26:53 +01:00
Sebastiaan van Stijn
c8b9dfb25e
Merge pull request #46867 from thaJeztah/log_info
daemon: kill: use log level to "warn" if container doesn't exit in time
2023-11-30 14:41:32 +01:00
Paweł Gronowski
fcb89da9c2
c8d/integration-cli: Skip TestListDanglingImagesWithDigests
Skip TestListDanglingImagesWithDigests which tests graphdriver
implementation specific behavior of `docker images --filter
dangling=true`.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-11-30 14:01:22 +01:00
Sebastiaan van Stijn
61d94850a7
cmd/dockerd: make cobra templates a const
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-11-30 13:40:15 +01:00
Sebastiaan van Stijn
0f163170a7
cmd/dockerd: remove various cobra templating
This removes various templating functions that were added for the
docker CLI. These are not needed for the dockerd binary, which does
not have subcommands or management commands.

Revert "Only hide commands if the env variable is set."

This reverts commit a7c8bcac2b.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-11-30 13:40:10 +01:00
Sebastiaan van Stijn
faecc65cd3
Merge pull request #46863 from michaelkebe/fix/cifs-volume-resolve-hostname
cifs volume resolves hostname correctly
2023-11-30 12:27:46 +01:00
Paweł Gronowski
ae6960cf91
Merge pull request #46857 from vvoland/liverestore-fix-46308
liverestore: Don't remove `--rm` containers on restart
2023-11-30 12:08:13 +01:00
Sebastiaan van Stijn
3bfb6a9420
frozen images: update to debian:bookworm-slim
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-11-30 12:01:06 +01:00
Sebastiaan van Stijn
fd40dfaf58
Dockerfile: remove uses of DEBIAN_FRONTEND
We used DEBIAN_FRONTEND in some places to prevent installation of packages
from being blocked. However, debian bookworm now [includes a fix][1] for
situations like this (it was specifically reported for Docker situations <3),
so we can get rid of these.

Thanks to Tianon for noticing this, and for linking to the Debian ticket!

[1]: https://bugs.debian.org/929417

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-11-30 11:51:18 +01:00
Sebastiaan van Stijn
e72c4818c4
Dockerfile: update to Debian "bookworm" (current stable)
Also switch yamllint to be installed from debian's packages, which are
currently at v1.29.0.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-11-30 11:50:34 +01:00
Michael Kebe
8ae94cafa5 cifs volume resolves hostname correctly.
Co-authored-by: Paweł Gronowski <me@woland.xyz>
Signed-off-by: Michael Kebe <michael.kebe@gmail.com>
2023-11-30 11:32:04 +01:00