Commit graph

46867 commits

Author SHA1 Message Date
Sebastiaan van Stijn
e57479dc30
Merge pull request #45888 from thaJeztah/cleanup_iptables
libnetwork/iptables: some cleanups and refactoring
2023-07-12 15:14:47 +02:00
Bjorn Neergaard
ecfa4f5866
ci(bin-image): add SHA-based tags
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-07-12 07:04:41 -06:00
Sebastiaan van Stijn
93e65a2a6c
Merge pull request #45918 from akerouanton/hack-make-run-disable-tls
Disable tls when launching dockerd through hack/make.sh
2023-07-12 14:37:56 +02:00
Bjorn Neergaard
daa3166cb6
Merge pull request #45932 from crazy-max/fix-moby-bin2
ci(bin-image): fix meta step
2023-07-12 06:11:56 -06:00
Evan Lezar
ce2e6b5cda Bump container-device-interface dependency to v0.6.0
Signed-off-by: Evan Lezar <elezar@nvidia.com>
2023-07-11 23:21:12 +02:00
Kevin Alvarez
0a126a85a4
ci(bin-image): fix meta step
We can't upload the same file in a matrix so generate
metadata in prepare job instead. Also fixes wrong bake meta
file in merge job.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-07-11 22:54:27 +02:00
Bjorn Neergaard
ebcb230cff
Merge pull request #45923 from thaJeztah/client_header
client: remove custom "headers" type (use http.Header), and omit "version" header on API >= 1.30
2023-07-11 14:30:20 -06:00
CrazyMax
749d7449f9
ci(bin-image): fix typo
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-07-11 22:01:55 +02:00
CrazyMax
8b7ec57cb4
Merge pull request #45930 from crazy-max/fix-moby-bin
ci(bin-image): don't set tags when pushing by digest
2023-07-11 21:31:32 +02:00
Sebastiaan van Stijn
6fee7c8f8b
Merge pull request #45861 from thaJeztah/fileutils_deprecate_GetTotalUsedFds
pkg/fileutils: GetTotalUsedFds(): slight optimization for macOS
2023-07-11 20:06:24 +02:00
CrazyMax
1686540594
ci(bin-image): don't set tags when pushing by digest
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-07-11 19:57:53 +02:00
Bjorn Neergaard
080844b9b3
Merge pull request #44740 from crazy-max/moby-bin
ci: push bin image to Docker Hub
2023-07-11 11:44:36 -06:00
Bjorn Neergaard
8c301fa933
Merge pull request #45915 from akerouanton/create-overlay-on-non-manager
libnet: Return proper error when overlay network can't be created
2023-07-11 10:13:08 -06:00
Sebastiaan van Stijn
43007108e6
Merge pull request #45917 from josegonzalez/patch-2
Sort unconsumed build arguments before usage
2023-07-11 13:25:58 +02:00
Sebastiaan van Stijn
e3f20f5088
Merge pull request #45924 from thaJeztah/registry_minor_fixes
registry: minor fixes and cleanups in search code
2023-07-11 13:19:31 +02:00
Sebastiaan van Stijn
83477ce8d0
client: remove custom "headers" type, and use "http.Header" instead
Use http.Header, which is more descriptive on intent, and we're already
importing the package in the client. Removing the "header" type also fixes
various locations where the type was shadowed by local variables named
"headers".

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-11 13:14:28 +02:00
Sebastiaan van Stijn
e4ce676d3d
client: ServiceCreate, ServiceUpdate: omit "version" header on API >= 1.30
The "version" header was added in c0afd9c873,
but used the wrong information to get the API version.

This issue was fixed in a9d20916c3, which switched
the API handler code to get the API version from the context. That change is part
of Docker Engine 20.10 (API v1.30 and up)

This patch updates the code to only set the header on APi v1.29 and older, as it's
not used by newer API versions.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-11 13:00:00 +02:00
Albin Kerouanton
d29240d9eb
libnet: Return a 403 when overlay network isn't allowed
With this change, the API will now return a 403 instead of a 500 when
trying to create an overlay network on a non-manager node.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-07-11 12:41:24 +02:00
Albin Kerouanton
21dcbada2d
libnet: Return proper error when overlay network can't be created
The commit befff0e13f inadvertendly
disabled the error returned when trying to create an overlay network on
a node which is not part of a Swarm cluster.

Since commit e3708a89cc the overlay
netdriver returns the error: `no VNI provided`.

This commit reinstate the original error message by checking if the node
is a manager before calling libnetwork's `controller.NewNetwork()`.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-07-11 12:40:55 +02:00
Sebastiaan van Stijn
93853eca94
pkg/fileutils: GetTotalUsedFds(): slight optimization for macOS
This patch contains some optimizations I still had stashed when working
on eaa9494b71.

- Use the bytes package for handling the output of "lsof", instead of
  converting to a string.
- Count the number of newlines in the output, instead of splitting the
  output into a slice of strings. We're only interested in the number
  of lines in the output.
- Use lsof's -F option to only print the file-descriptor for each line,
  as we don't need other information.
- Use the -l, -n, and -P options to omit converting usernames, host names,
  and port numbers.

From the [LSOF(8)][1] man-page:

   -l    This option inhibits the conversion of user ID numbers to
         login names. It is also useful when login name lookup is
         working improperly or slowly.

   -n    This option inhibits the conversion of network numbers to host
         names for network files. Inhibiting conversion can make lsof run faster.
         It is also useful when host name lookup is not working properly.

   -P    This option inhibits the conversion of port numbers to port names for network files.
         Inhibiting the conversion can make lsof run a little faster.
         It is also useful when host name lookup is not working properly.

Output looks something like;

    lsof -lnP -Ff -p 39849
    p39849
    fcwd
    ftxt
    ftxt
    f0
    f1
    f2
    f3
    f4
    f5
    f6
    f7
    f8
    f9
    f10
    f11

Before/After:

    BenchmarkGetTotalUsedFds-10  122  9479384 ns/op   10816 B/op  63 allocs/op
    BenchmarkGetTotalUsedFds-10  154  7814697 ns/op    7257 B/op  60 allocs/op

[1]: https://opensource.apple.com/source/lsof/lsof-49/lsof/lsof.man.auto.html

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-11 10:50:31 +02:00
Sebastiaan van Stijn
c5c977855d
registry: session.searchRepositories(): return typed error, and small cleanup
- return a errdefs.System if we fail to decode the registry's response
- use strconv.Itoa instead of fmt.Sprintf

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-11 09:47:02 +02:00
Sebastiaan van Stijn
68ebfa2f18
registry: session.searchRepositories(): log actual search URL
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-11 09:41:52 +02:00
Sebastiaan van Stijn
27698f1d13
registry: ParseSearchIndexInfo: remove redundant error-handling
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-10 17:48:10 +02:00
Sebastiaan van Stijn
462d6ef826
Merge pull request #45901 from thaJeztah/api_move_system_info
api/types: move system info types to api/types/system
2023-07-10 17:45:13 +02:00
Sebastiaan van Stijn
e477a57cc5
Merge pull request #45920 from thaJeztah/fix_expose_npe
daemon/containerd: fix assignment to entry in nil map during commit
2023-07-10 13:53:16 +02:00
Sebastiaan van Stijn
a0e1155b28
daemon/containerd: fix assignment to entry in nil map during commit
A panic would happen when converting an config that had ports exposed, because
the ExposedPorts map in the OCI-spec was not initialized. This could happen
when committing a container, or when using the classic builder and the
parent image had ports exposed, for example

    FROM busybox AS stage0
    EXPOSE 80

    FROM stage0 AS stage1
    RUN echo hello

Example of the panic:

    2023/07/07 15:13:02 http: panic serving @: assignment to entry in nil map
    goroutine 1944 [running]:
    net/http.(*conn).serve.func1()
    	/usr/local/go/src/net/http/server.go:1854 +0xbf
    panic({0x45f660, 0xb6a8d0})
    	/usr/local/go/src/runtime/panic.go:890 +0x263
    github.com/docker/docker/daemon/containerd.containerConfigToOciImageConfig(...)
    	/go/src/github.com/docker/docker/daemon/containerd/image_import.go:397
    github.com/docker/docker/daemon/containerd.generateCommitImageConfig({0xc001470498, {0x0, 0x0}, {0xc000c437d8, 0x5}, {0x0, 0x0}, {0xc000c43b27, 0x5}, {0x0, ...}, ...}, ...)
    	/go/src/github.com/docker/docker/daemon/containerd/image_commit.go:138 +0x40e
    github.com/docker/docker/daemon/containerd.(*ImageService).CommitImage(0xc0008853e0, {0xb8f660, 0xc000c4f7c0}, {{0x0, 0x0}, {0x0, 0x0}, 0xc00104b900, 0xc00104b180, {0xc0011a7640, ...}, ...})
    	/go/src/github.com/docker/docker/daemon/containerd/image_commit.go:82 +0x73b
    github.com/docker/docker/daemon/containerd.(*ImageService).CommitBuildStep(0xc0008853e0, {0xb8f660, 0xc000c4f7c0}, {{0x0, 0x0}, {0x0, 0x0}, 0xc00104b900, 0xc00104b180, {0xc0011a7640, ...}, ...})
    	/go/src/github.com/docker/docker/daemon/containerd/image_commit.go:308 +0x110
    github.com/docker/docker/builder/dockerfile.(*Builder).commitContainer(0xc0012b8cc0, {0xb8f660, 0xc000c4f7c0}, 0xc0010b2b60, {0xc0011a7640, 0x40}, 0xc00104b180)
    	/go/src/github.com/docker/docker/builder/dockerfile/internals.go:61 +0x168
    github.com/docker/docker/builder/dockerfile.(*Builder).commit(0xc0012b8cc0, {0xb8f660, 0xc000c4f7c0}, 0xc0010b2b60, {0xc0012a7d80?, 0xc001340060?})
    	/go/src/github.com/docker/docker/builder/dockerfile/internals.go:45 +0x1aa
    github.com/docker/docker/builder/dockerfile.dispatchLabel({0xb8f660, 0xc000c4f7c0}, {0xc0010b2b60, 0xc000c6b628, 0xc0012b8cc0, {0xb80f60, 0xc0011a46c0}, 0xc000bc2560}, 0x1e24a85?)
    	/go/src/github.com/docker/docker/builder/dockerfile/dispatchers.go:83 +0x258
    github.com/docker/docker/builder/dockerfile.dispatch({0xb8f660, 0xc000c4f7c0}, {0xc0010b2b60, 0xc000c6b628, 0xc0012b8cc0, {0xb80f60, 0xc0011a46c0}, 0xc000bc2560}, {0xb7be40, 0xc00111cde0})
    	/go/src/github.com/docker/docker/builder/dockerfile/evaluator.go:74 +0x529
    github.com/docker/docker/builder/dockerfile.(*Builder).dispatchDockerfileWithCancellation(0xc0012b8cc0, {0xb8f660, 0xc000c4f7c0}, {0xc000b1d380, 0x1, 0xc0011a4660?}, {0x0, 0x0, 0x0?}, 0x5c, ...)
    	/go/src/github.com/docker/docker/builder/dockerfile/builder.go:296 +0x8f2
    github.com/docker/docker/builder/dockerfile.(*Builder).build(0xc0012b8cc0, {0xb8f660, 0xc000c4f7c0}, {0xb80f60, 0xc0011a46c0}, 0xc0011a49f0)
    	/go/src/github.com/docker/docker/builder/dockerfile/builder.go:211 +0x2e5
    github.com/docker/docker/builder/dockerfile.(*BuildManager).Build(0xc0008868c0, {0xb8f708, 0xc0011a44b0}, {{0xb789c0, 0xc0011a4540}, {{0xb6b940, 0xc000c22a50}, {0xb6c5e0, 0xc000c22a68}, {0xb6c5e0, ...}, ...}, ...})
    	/go/src/github.com/docker/docker/builder/dockerfile/builder.go:98 +0x358
    github.com/docker/docker/api/server/backend/build.(*Backend).Build(0xc0007d0870, {0xb8f708, 0xc0011a44b0}, {{0xb789c0, 0xc0011a4540}, {{0xb6b940, 0xc000c22a50}, {0xb6c5e0, 0xc000c22a68}, {0xb6c5e0, ...}, ...}, ...})
    	/go/src/github.com/docker/docker/api/server/backend/build/backend.go:69 +0x186
    github.com/docker/docker/api/server/router/build.(*buildRouter).postBuild(0xc0008333c0, {0xb8f708, 0xc0011a44b0}, {0xb8e130, 0xc0000ed500}, 0xc0010d4800, 0xc0012df760?)
    	/go/src/github.com/docker/docker/api/server/router/build/build_routes.go:280 +0x7a6
    github.com/docker/docker/api/server/middleware.ExperimentalMiddleware.WrapHandler.func1({0xb8f708, 0xc0011a44b0}, {0xb8e130?, 0xc0000ed500?}, 0x36cf80?, 0xc0010ab550?)
    	/go/src/github.com/docker/docker/api/server/middleware/experimental.go:26 +0x15b
    github.com/docker/docker/api/server/middleware.VersionMiddleware.WrapHandler.func1({0xb8f708, 0xc0011a4480}, {0xb8e130, 0xc0000ed500}, 0xc000d787e8?, 0xc000d787a0?)
    	/go/src/github.com/docker/docker/api/server/middleware/version.go:62 +0x4d7
    github.com/docker/docker/pkg/authorization.(*Middleware).WrapHandler.func1({0xb8f708, 0xc0011a4480}, {0xb8e130?, 0xc0000ed500?}, 0xc0010d4800, 0xc0010ab500?)
    	/go/src/github.com/docker/docker/pkg/authorization/middleware.go:59 +0x649
    github.com/docker/docker/api/server.(*Server).makeHTTPHandler.func1({0xb8e130, 0xc0000ed500}, 0xc0010d4700)
    	/go/src/github.com/docker/docker/api/server/server.go:53 +0x1ce
    net/http.HandlerFunc.ServeHTTP(0xc0010d4600?, {0xb8e130?, 0xc0000ed500?}, 0xc000d789e8?)
    	/usr/local/go/src/net/http/server.go:2122 +0x2f
    github.com/docker/docker/vendor/github.com/gorilla/mux.(*Router).ServeHTTP(0xc0001a7e00, {0xb8e130, 0xc0000ed500}, 0xc000d37600)
    	/go/src/github.com/docker/docker/vendor/github.com/gorilla/mux/mux.go:210 +0x1cf
    net/http.serverHandler.ServeHTTP({0xb7ec58?}, {0xb8e130, 0xc0000ed500}, 0xc000d37600)
    	/usr/local/go/src/net/http/server.go:2936 +0x316
    net/http.(*conn).serve(0xc0012661b0, {0xb8f708, 0xc000fd0360})
    	/usr/local/go/src/net/http/server.go:1995 +0x612
    created by net/http.(*Server).Serve
    	/usr/local/go/src/net/http/server.go:3089 +0x5ed

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-10 10:58:06 +02:00
Albin Kerouanton
6b1b71ced4
Disable tls when launching dockerd through hack/make.sh
The daemon sleeps for 15 seconds at start up when the API binds to a TCP
socket with no TLS certificate set. That's what the hack/make/run script
does, but it doesn't explicitly disable tls, thus we're experiencing
this annoying delay every time we use this script.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-07-09 22:26:07 +02:00
Sebastiaan van Stijn
49b55b220e
Merge pull request #42138 from cpuguy83/test_daemon_logging
Improve test daemon logging
2023-07-09 10:02:43 +02:00
Jose Diaz-Gonzalez
079a9d4562 Sort unconsumed build arguments before usage
Golang map iteration order is not guaranteed, so in some cases the built slice has it's output of order as well. This means that testing for exact warning messages in docker build output would result in random test failures, making it more annoying for end-users to test against this functionality.

Signed-off-by: Jose Diaz-Gonzalez <email@josediazgonzalez.com>
2023-07-09 03:23:06 -04:00
CrazyMax
41261ea4ec
ci: push bin image to Docker Hub
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-07-09 02:59:03 +02:00
Bjorn Neergaard
c57097bcd4
Merge pull request #45908 from corhere/libn/drop-swarmkit-cruft
libnetwork: drop cruft formerly needed by Swarmkit
2023-07-07 15:26:00 -06:00
Cory Snider
5b3acc15d1 libnetwork: drop legacy driver cruft
...that Swarmkit no longer needs now that it has been migrated to use
the new-style driver registration APIs.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-07-07 15:02:58 -04:00
Cory Snider
f9a83daebb libnetwork: delete package bitseq
Package idm was the last remaining user of bitseq.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-07-07 14:53:44 -04:00
Cory Snider
8b167535db libnetwork: delete package idm
The only remaining user is Swarmkit, which now has its own private copy
of the package tailored to its needs.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-07-07 14:52:08 -04:00
Cory Snider
7366e44019 vendor: github.com/moby/swarmkit/v2 v2.0.0-20230707182847-6f78b8199b05
36334ed187...6f78b8199b

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-07-07 14:48:12 -04:00
Brian Goff
7d548c6c00
Merge pull request #45902 from thaJeztah/fix_volume_npe 2023-07-07 07:38:21 -07:00
Sebastiaan van Stijn
a490248f4d
daemon: daemon.prepareMountPoints(): fix panic if mount is not a volume
The daemon.lazyInitializeVolume() function only handles restoring Volumes
if a Driver is specified. The Container's MountPoints field may also
contain other kind of mounts (e.g., bind-mounts). Those were ignored, and
don't return an error; 1d9c8619cd/daemon/volumes.go (L243-L252C2)

However, the prepareMountPoints() assumed each MountPoint was a volume,
and logged an informational message about the volume being restored;
1d9c8619cd/daemon/mounts.go (L18-L25)

This would panic if the MountPoint was not a volume;

    github.com/docker/docker/daemon.(*Daemon).prepareMountPoints(0xc00054b7b8?, 0xc0007c2500)
            /root/rpmbuild/BUILD/src/engine/.gopath/src/github.com/docker/docker/daemon/mounts.go:24 +0x1c0
    github.com/docker/docker/daemon.(*Daemon).restore.func5(0xc0007c2500, 0x0?)
            /root/rpmbuild/BUILD/src/engine/.gopath/src/github.com/docker/docker/daemon/daemon.go:552 +0x271
    created by github.com/docker/docker/daemon.(*Daemon).restore
            /root/rpmbuild/BUILD/src/engine/.gopath/src/github.com/docker/docker/daemon/daemon.go:530 +0x8d8
    panic: runtime error: invalid memory address or nil pointer dereference
    [signal SIGSEGV: segmentation violation code=0x1 addr=0x30 pc=0x564e9be4c7c0]

This issue was introduced in 647c2a6cdd

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-07 15:31:00 +02:00
Sebastiaan van Stijn
c90229ed9a
api/types: move system info types to api/types/system
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-07 13:01:36 +02:00
Bjorn Neergaard
98d3da79ef
Merge pull request #45881 from corhere/libn/register-configless-drivers
libnetwork: stop passing config to drivers which ignore the config argument
2023-07-06 12:17:17 -06:00
Cory Snider
d519bde5da libnet/drivers: stop passing config to drivers...
...which ignore the config argument. Notably, none of the network
drivers referenced by Swarmkit use config, which is good as Swarmkit
unconditionally passes nil for the config when registering drivers.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-07-06 12:57:00 -04:00
Cory Snider
1980deffae libn: refactor platform driver registration
Hide knowledge of the network driver initializer functions from
controller.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-07-06 12:56:09 -04:00
Bjorn Neergaard
b60c02b065
Merge pull request #45887 from thaJeztah/move_mtu
daemon/config: move MTU to BridgeConfig, and warn when using on Windows
2023-07-06 09:41:06 -06:00
Sebastiaan van Stijn
a2e9ae0055
Merge pull request #45895 from thaJeztah/albin_maintainer 2023-07-06 17:14:01 +02:00
Sebastiaan van Stijn
065b00b787
Merge pull request #45894 from thaJeztah/laura_maintainer 2023-07-06 17:12:33 +02:00
Sebastiaan van Stijn
bf42ccfbec
Merge pull request #45893 from thaJeztah/kevin_maintainer 2023-07-06 17:08:50 +02:00
Bjorn Neergaard
c9397ec8d4
Merge pull request #45892 from corhere/libn/overlay-cut-vni-list
libnetwork/drivers/overlay: parse VNI list option in O(1) memory
2023-07-06 08:38:19 -06:00
Sebastiaan van Stijn
c87c4b5bbf
Add Albin Kerouanton (@akerouanton) as maintainer
Albin is currently a curator, has been contributing for various years prior
to that, and has taken on the daunting task to work on Moby's networking stack.

Albin would be a great addition to our list of maintainers and to allow him
to perform his work in these areas in a more official capacity.

I nominated Albin as maintainer, and votes passed, so opening a PR to
make it official.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-06 14:47:20 +02:00
Sebastiaan van Stijn
0066138379
Add Kevin Alvarez (@crazy-max) as maintainer
Kevin is a maintainer for BuildKit, Buildx, and Docker's official GitHub
actions (among others), has been our "in-house GitHub actions expert"
for a long time, and has made significant contributions to the integration
with BuildKit, and to improve our build pipeline(s).

Kevin would be a great addition to our list of maintainers and to allow him
to perform his work in these areas in a more official capacity.

I nominated Kevin as maintainer, and votes passed, so opening a PR to
make it official.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-06 14:43:27 +02:00
Sebastiaan van Stijn
d8056ddecd
Add Laura Brehm (@laurazard) as maintainer
Laura has done significant work on the containerd integration, helping
triage and fixing bugs, both in this repository, containerd, and the
docker CLI, and would make a great addition to our list of maintainers.

I nominated Laura as maintainer, and votes passed, so opening a PR to
make it official.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-06 14:42:39 +02:00
Bjorn Neergaard
e4c866f387
Merge pull request #40894 from cpuguy83/health_start_interval
Add health start interval
2023-07-05 19:47:41 -06:00