Commit graph

46560 commits

Author SHA1 Message Date
Sebastiaan van Stijn
4f0747b0df
Merge pull request #46014 from vvoland/c8d-inspect-fix-duplicate-digested-24
[backport 24.0] c8d/inspect: Don't duplicate digested ref
2023-07-18 20:19:57 +02:00
Bjorn Neergaard
ff0144de3b
ci(buildkit): remove early-return from buildkit-ref
This doesn't really make sense now that this script returns a
$GITHUB_ENV snippet.

Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
(cherry picked from commit 7310a7cd0c)
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-07-18 12:11:56 -06:00
Bjorn Neergaard
a936ae7e98
ci(buildkit): remove misleading code from buildkit-ref
Post-f8c0d92a22bad004cb9cbb4db704495527521c42, BUILDKIT_REPO doesn't
really do what it claims to. Instead, don't allow overloading since the
import path for BuildKit is always the same, and make clear the
provenance of values when generating the final variable definitions.

We also better document the script, and follow some best practices for
both POSIX sh and Bash.

Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
(cherry picked from commit 4ecc01f3ad)
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-07-18 12:11:55 -06:00
Bjorn Neergaard
4c29864b02
hack/with-go-mod.sh: warn on stderr
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
(cherry picked from commit 48ff8a95cc)
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-07-18 12:11:54 -06:00
Paweł Gronowski
3c5c192baf
c8d/resolveImage: Fix Digested and Named reference
When resolving a reference that is both a Named and Digested, it could
be resolved to an image that has the same digest, but completely
different repository name.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 48fc306764fc5c39d4284021520a0337ef7e0cb0)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-07-18 18:02:36 +02:00
Paweł Gronowski
572de8764e
c8d/inspect: Don't duplicate digested ref
If image name is already an untagged digested reference, don't produce
additional digested ref.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 028eab9ebb)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-07-18 17:48:35 +02:00
Justin Chadwell
5dded3340c
ci: extract buildkit version correctly with replace-d modules
Signed-off-by: Justin Chadwell <me@jedevc.com>
(cherry picked from commit f8c0d92a22)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-18 15:23:57 +02:00
Sebastiaan van Stijn
8b24eea65e
Merge pull request #45990 from thaJeztah/24.0_backport_hijack_share_request_builder
[24.0 backport] client: Client.postHijacked: use Client.buildRequest
2023-07-18 15:05:20 +02:00
Sebastiaan van Stijn
8ab6d025f6
Merge pull request #46001 from thaJeztah/24.0_backport_notestyourself
[24.0 backport] quota: remove gotest.tools from testhelpers
2023-07-18 10:24:21 +02:00
Sebastiaan van Stijn
bd1ae65aab
quota: remove gotest.tools from testhelpers
gotest.tools has an init() which registers a '-update' flag;
a80f057529/internal/source/update.go (L21-L23)

The quota helper contains a testhelpers file, which is meant for usage
in (integration) tests, but as it's in the same pacakge as production
code, would also trigger the gotest.tools init.

This patch removes the gotest.tools code from this file.

Before this patch:

    $ (exec -a libnetwork-setkey "$(which dockerd)" -help)
    Usage of libnetwork-setkey:
      -exec-root string
            docker exec root (default "/run/docker")
      -update
            update golden values

With this patch applied:

    $ (exec -a libnetwork-setkey "$(which dockerd)" -help)
    Usage of libnetwork-setkey:
      -exec-root string
            docker exec root (default "/run/docker")

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 1aa17222e7)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-17 23:12:28 +02:00
Sebastiaan van Stijn
2e3f3fd1e0
Merge pull request #45998 from neersighted/backport/45997/24.0
[24.0 backport] Build system grab-bag
2023-07-17 20:19:50 +02:00
Bjorn Neergaard
544032f7a4
hack/d/cli.sh: properly handle errors in curl
Add `-f` to output nothing to tar if the curl fails, and `-S` to report
errors if they happen.

Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
(cherry picked from commit 780e8b2332)
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-07-17 11:23:13 -06:00
Bjorn Neergaard
0df2e1bdd8
Dockerfile: improve CLI/rootlesskit caching
Use bind-mounts instead of a `COPY` for cli.sh, and use `COPY --link`
for rootlesskit's build stage.

Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
(cherry picked from commit 12a19dcd84)
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-07-17 11:23:12 -06:00
Bjorn Neergaard
05f82fdd00
Dockerfile(.simple): align APT_MIRROR support
Use a non-slash escape sequence to support mirrors with a path
component, and do not unconditionally replace the mirror in
Dockerfile.simple.

Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
(cherry picked from commit 235cd6c6b2)
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-07-17 11:23:11 -06:00
Bjorn Neergaard
151686a5c8
Makefile: pass through APT_MIRROR
This aligns `docker build` as invoked by the Makefile with both `docker
buildx bake` as invoked by the Makefile and directly by the user.

Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
(cherry picked from commit bcea83ab9b)
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-07-17 11:23:10 -06:00
CrazyMax
31567e0973
Dockerfile: use default apt mirrors
Use default apt mirrors and also check APT_MIRROR
is set before updating mirrors.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit a1d2132bf6)
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-07-17 11:23:05 -06:00
Sebastiaan van Stijn
d94f2dcab2
client: Client.postHijacked: use Client.buildRequest
Use Client.buildRequest instead of a local copy of the same logic so
that we're using the same logic, and there's less chance of diverging.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit c219b09d4a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-17 09:41:41 +02:00
Sebastiaan van Stijn
bff68bf2cc
client: Client.setupHijackConn: explicitly ignore errors
Just making my IDE and some linters slightly happier.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit e11555218b)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-17 09:41:41 +02:00
Sebastiaan van Stijn
8443a06149
Merge pull request #45974 from thaJeztah/24.0_backport_create-overlay-on-non-manager
[24.0 backport] libnet: Return proper error when overlay network can't be created
2023-07-17 09:20:55 +02:00
Sebastiaan van Stijn
36e9e796c6
Merge pull request #45976 from thaJeztah/24.0_backport_moby-bin-tags
[24.0 backport] gha: assorted fixes for bin-image
2023-07-15 01:57:25 +02:00
Sebastiaan van Stijn
e916ec1584
Merge pull request #45975 from thaJeztah/24.0_backport_hack-make-run-disable-tls
[24.0 backport] Disable tls when launching dockerd through hack/make.sh
2023-07-15 01:54:01 +02:00
Bjorn Neergaard
eb34c0b6d2
Merge pull request #45970 from thaJeztah/24.0_backport_update_go_1.20.6
[24.0 backport] update go to go1.20.6
2023-07-14 17:32:07 -06:00
Bjorn Neergaard
8bdf6d1baf
ci(bin-image): add SHA-based tags
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
(cherry picked from commit ecfa4f5866)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-15 01:05:28 +02:00
Kevin Alvarez
26a457e7a3
ci(bin-image): fix meta step
We can't upload the same file in a matrix so generate
metadata in prepare job instead. Also fixes wrong bake meta
file in merge job.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 0a126a85a4)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-15 01:04:43 +02:00
CrazyMax
b9904ba319
ci(bin-image): fix typo
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 749d7449f9)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-15 01:04:33 +02:00
CrazyMax
e7c333cb6e
ci(bin-image): don't set tags when pushing by digest
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 1686540594)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-15 01:03:17 +02:00
CrazyMax
fcb87e8ae1
ci: push bin image to Docker Hub
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 41261ea4ec)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-15 01:02:55 +02:00
Albin Kerouanton
68c0cec772
Disable tls when launching dockerd through hack/make.sh
The daemon sleeps for 15 seconds at start up when the API binds to a TCP
socket with no TLS certificate set. That's what the hack/make/run script
does, but it doesn't explicitly disable tls, thus we're experiencing
this annoying delay every time we use this script.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
(cherry picked from commit 6b1b71ced4)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-15 00:58:32 +02:00
Albin Kerouanton
738d8417e0
libnet: Return a 403 when overlay network isn't allowed
With this change, the API will now return a 403 instead of a 500 when
trying to create an overlay network on a non-manager node.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
(cherry picked from commit d29240d9eb)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-15 00:51:58 +02:00
Albin Kerouanton
a5c0fda157
libnet: Return proper error when overlay network can't be created
The commit befff0e13f inadvertendly
disabled the error returned when trying to create an overlay network on
a node which is not part of a Swarm cluster.

Since commit e3708a89cc the overlay
netdriver returns the error: `no VNI provided`.

This commit reinstate the original error message by checking if the node
is a manager before calling libnetwork's `controller.NewNetwork()`.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
(cherry picked from commit 21dcbada2d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-15 00:51:50 +02:00
Sebastiaan van Stijn
deea880581
pkg/jsonmessage: use string-literals for easier grep'ing
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit ec11aea880)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-15 00:51:38 +02:00
Sebastiaan van Stijn
962a4f434f
pkg/ioutils: use string-literals for easier grep'ing
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit fded42c3bd)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-15 00:51:28 +02:00
Sebastiaan van Stijn
cea5829402
pkg/idtools: use string-literals for easier grep'ing
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 1da079f211)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-15 00:51:19 +02:00
Sebastiaan van Stijn
69d77bc150
opts: use string-literals for easier grep'ing
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 84000190d3)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-15 00:51:09 +02:00
Sebastiaan van Stijn
ff667ed932
integration: use string-literals for easier grep'ing
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit ddec605aef)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-15 00:50:57 +02:00
Sebastiaan van Stijn
efe9e90ef5
libnetwork: use string-literals for easier grep'ing
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 96a1c444cc)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-15 00:50:47 +02:00
Sebastiaan van Stijn
2d2df4376b
daemon/cluster: use string-literals for easier grep'ing
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 0db4a32b9c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-15 00:50:37 +02:00
Sebastiaan van Stijn
ae8e3294dd
client: use string-literals for easier grep'ing
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 4e69e16fde)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-15 00:50:25 +02:00
Sebastiaan van Stijn
892857179a
cli/debug: use string-literals for easier grep'ing
Also removed some newlines from t.Fatal() as they shouldn't be needed.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit c3d533f37f)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-15 00:49:55 +02:00
Sebastiaan van Stijn
147b87a03e
daemon: use string-literals for easier grep'ing
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 02815416bb)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-15 00:49:46 +02:00
Sebastiaan van Stijn
a3f1f4eeb0
integration-cli: use string-literals for easier grep'ing
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 6331a3a346)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-15 00:49:35 +02:00
Sebastiaan van Stijn
5bba60b1bb
builder/builder-next: use string-literals for easier grep'ing
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 2f61620339)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-15 00:49:08 +02:00
Sebastiaan van Stijn
632fc235d6
builder/dockerfile: use string-literals for easier grep'ing
Use string-literal for reduce escaped quotes, which makes for easier grepping.
While at it, also changed http -> https to keep some linters at bay.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 202907b14c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-15 00:48:58 +02:00
Sebastiaan van Stijn
75a90f85ad
gha: add note about buildkit using older go version
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 41f235a2f8)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-14 22:46:03 +02:00
Sebastiaan van Stijn
fa909dfaf4
update go to go1.20.6
go1.20.6 (released 2023-07-11) includes a security fix to the net/http package,
as well as bug fixes to the compiler, cgo, the cover tool, the go command,
the runtime, and the crypto/ecdsa, go/build, go/printer, net/mail, and text/template
packages. See the Go 1.20.6 milestone on our issue tracker for details.

https://github.com/golang/go/issues?q=milestone%3AGo1.20.6+label%3ACherryPickApproved

Full diff: https://github.com/golang/go/compare/go1.20.5...go1.20.6

These minor releases include 1 security fixes following the security policy:

net/http: insufficient sanitization of Host header

The HTTP/1 client did not fully validate the contents of the Host header.
A maliciously crafted Host header could inject additional headers or entire
requests. The HTTP/1 client now refuses to send requests containing an
invalid Request.Host or Request.URL.Host value.

Thanks to Bartek Nowotarski for reporting this issue.

Includes security fixes for [CVE-2023-29406 ][1] and Go issue https://go.dev/issue/60374

[1]: https://github.com/advisories/GHSA-f8f7-69v5-w4vx

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 1ead2dd35d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-14 22:45:41 +02:00
Sebastiaan van Stijn
d09fe00d36
Merge pull request #45962 from thaJeztah/24.0_backport_fix_host_header
[24.0 backport] client: define a "dummy" hostname to use for local connections
2023-07-14 22:43:28 +02:00
Sebastiaan van Stijn
bdaadec788
testutil: use dummyhost for non-tcp connections
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit e1db9e9848)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-14 20:36:52 +02:00
Sebastiaan van Stijn
547ea18fbb
pkg/plugins: use a dummy hostname for local connections
For local communications (npipe://, unix://), the hostname is not used,
but we need valid and meaningful hostname.

The current code used the socket path as hostname, which gets rejected by
go1.20.6 and go1.19.11 because of a security fix for [CVE-2023-29406 ][1],
which was implemented in  https://go.dev/issue/60374.

Prior versions go Go would clean the host header, and strip slashes in the
process, but go1.20.6 and go1.19.11 no longer do, and reject the host
header.

Before this patch, tests would fail on go1.20.6:

    === FAIL: pkg/authorization TestAuthZRequestPlugin (15.01s)
    time="2023-07-12T12:53:45Z" level=warning msg="Unable to connect to plugin: //tmp/authz2422457390/authz-test-plugin.sock/AuthZPlugin.AuthZReq: Post \"http://%2F%2Ftmp%2Fauthz2422457390%2Fauthz-test-plugin.sock/AuthZPlugin.AuthZReq\": http: invalid Host header, retrying in 1s"
    time="2023-07-12T12:53:46Z" level=warning msg="Unable to connect to plugin: //tmp/authz2422457390/authz-test-plugin.sock/AuthZPlugin.AuthZReq: Post \"http://%2F%2Ftmp%2Fauthz2422457390%2Fauthz-test-plugin.sock/AuthZPlugin.AuthZReq\": http: invalid Host header, retrying in 2s"
    time="2023-07-12T12:53:48Z" level=warning msg="Unable to connect to plugin: //tmp/authz2422457390/authz-test-plugin.sock/AuthZPlugin.AuthZReq: Post \"http://%2F%2Ftmp%2Fauthz2422457390%2Fauthz-test-plugin.sock/AuthZPlugin.AuthZReq\": http: invalid Host header, retrying in 4s"
    time="2023-07-12T12:53:52Z" level=warning msg="Unable to connect to plugin: //tmp/authz2422457390/authz-test-plugin.sock/AuthZPlugin.AuthZReq: Post \"http://%2F%2Ftmp%2Fauthz2422457390%2Fauthz-test-plugin.sock/AuthZPlugin.AuthZReq\": http: invalid Host header, retrying in 8s"
        authz_unix_test.go:82: Failed to authorize request Post "http://%2F%2Ftmp%2Fauthz2422457390%2Fauthz-test-plugin.sock/AuthZPlugin.AuthZReq": http: invalid Host header

[1]: https://github.com/advisories/GHSA-f8f7-69v5-w4vx

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 6b7705d5b2)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-14 20:36:44 +02:00
Sebastiaan van Stijn
597a5f9794
client: define a "dummy" hostname to use for local connections
For local communications (npipe://, unix://), the hostname is not used,
but we need valid and meaningful hostname.

The current code used the client's `addr` as hostname in some cases, which
could contain the path for the unix-socket (`/var/run/docker.sock`), which
gets rejected by go1.20.6 and go1.19.11 because of a security fix for
[CVE-2023-29406 ][1], which was implemented in  https://go.dev/issue/60374.

Prior versions go Go would clean the host header, and strip slashes in the
process, but go1.20.6 and go1.19.11 no longer do, and reject the host
header.

This patch introduces a `DummyHost` const, and uses this dummy host for
cases where we don't need an actual hostname.

Before this patch (using go1.20.6):

    make GO_VERSION=1.20.6 TEST_FILTER=TestAttach test-integration
    === RUN   TestAttachWithTTY
        attach_test.go:46: assertion failed: error is not nil: http: invalid Host header
    --- FAIL: TestAttachWithTTY (0.11s)
    === RUN   TestAttachWithoutTTy
        attach_test.go:46: assertion failed: error is not nil: http: invalid Host header
    --- FAIL: TestAttachWithoutTTy (0.02s)
    FAIL

With this patch applied:

    make GO_VERSION=1.20.6 TEST_FILTER=TestAttach test-integration
    INFO: Testing against a local daemon
    === RUN   TestAttachWithTTY
    --- PASS: TestAttachWithTTY (0.12s)
    === RUN   TestAttachWithoutTTy
    --- PASS: TestAttachWithoutTTy (0.02s)
    PASS

[1]: https://github.com/advisories/GHSA-f8f7-69v5-w4vx

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 92975f0c11)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-14 20:36:33 +02:00
Sebastiaan van Stijn
fee4db80a0
client: TestSetHostHeader: don't use un-keyed literals
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 2a59188760)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-14 20:35:01 +02:00