Commit graph

48607 commits

Author SHA1 Message Date
Sebastiaan van Stijn
5d15da8290
Merge pull request #46206 from thaJeztah/libnetwork_no_walk
libnetwork: implement Controller.GetSandbox, remove Controller.WalkSandboxes and related code
2023-08-21 21:21:16 +02:00
Sebastiaan van Stijn
3ffa00909d
Merge pull request #46285 from thaJeztah/gha_report_timeout
gha: set 10-minute timeout on "report" actions
2023-08-21 20:19:39 +02:00
Bjorn Neergaard
764419ed4e
Merge pull request #45534 from polarathene/fix/config-init-limits
fix: Normalize `RLIMIT_NOFILE` to sensible defaults
2023-08-21 12:09:38 -06:00
Brian Goff
fc15e08a6c
Merge pull request #46137 from thaJeztah/v1_deprecation 2023-08-21 10:51:23 -07:00
Sebastiaan van Stijn
dff53a02ef
libnetwork: remove Controller.Sandboxes as it's no longer used
The Controller.Sandboxes method was used by some SandboxWalkers. Now
that those have been removed, there are no longer any consumers of this
method, so let's remove it for now.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-21 19:08:43 +02:00
Sebastiaan van Stijn
d6f340e784
gha: set 10-minute timeout on "report" actions
I had a CI run fail to "Upload reports":

    Exponential backoff for retry #1. Waiting for 4565 milliseconds before continuing the upload at offset 0
    Finished backoff for retry #1, continuing with upload
    Total file count: 211 ---- Processed file #160 (75.8%)
    ...
    Total file count: 211 ---- Processed file #164 (77.7%)
    Total file count: 211 ---- Processed file #164 (77.7%)
    Total file count: 211 ---- Processed file #164 (77.7%)
    A 503 status code has been received, will attempt to retry the upload
    ##### Begin Diagnostic HTTP information #####
    Status Code: 503
    Status Message: Service Unavailable
    Header Information: {
      "content-length": "592",
      "content-type": "application/json; charset=utf-8",
      "date": "Mon, 21 Aug 2023 14:08:10 GMT",
      "server": "Kestrel",
      "cache-control": "no-store,no-cache",
      "pragma": "no-cache",
      "strict-transport-security": "max-age=2592000",
      "x-tfs-processid": "b2fc902c-011a-48be-858d-c62e9c397cb6",
      "activityid": "49a48b53-0411-4ff3-86a7-4528e3f71ba2",
      "x-tfs-session": "49a48b53-0411-4ff3-86a7-4528e3f71ba2",
      "x-vss-e2eid": "49a48b53-0411-4ff3-86a7-4528e3f71ba2",
      "x-vss-senderdeploymentid": "63be6134-28d1-8c82-e969-91f4e88fcdec",
      "x-frame-options": "SAMEORIGIN"
    }
    ###### End Diagnostic HTTP information ######
    Retry limit has been reached for chunk at offset 0 to https://pipelinesghubeus5.actions.githubusercontent.com/Y2huPMnV2RyiTvKoReSyXTCrcRyxUdSDRZYoZr0ONBvpl5e9Nu/_apis/resources/Containers/8331549?itemPath=integration-reports%2Fubuntu-22.04-systemd%2Fbundles%2Ftest-integration%2FTestInfoRegistryMirrors%2Fd20ac12e48cea%2Fdocker.log
    Warning: Aborting upload for /tmp/reports/ubuntu-22.04-systemd/bundles/test-integration/TestInfoRegistryMirrors/d20ac12e48cea/docker.log due to failure
    Error: aborting artifact upload
    Total file count: 211 ---- Processed file #165 (78.1%)
    A 503 status code has been received, will attempt to retry the upload
    Exponential backoff for retry #1. Waiting for 5799 milliseconds before continuing the upload at offset 0

As a result, the "Download reports" continued retrying:

    ...
    Total file count: 1004 ---- Processed file #436 (43.4%)
    Total file count: 1004 ---- Processed file #436 (43.4%)
    Total file count: 1004 ---- Processed file #436 (43.4%)
    An error occurred while attempting to download a file
    Error: Request timeout: /Y2huPMnV2RyiTvKoReSyXTCrcRyxUdSDRZYoZr0ONBvpl5e9Nu/_apis/resources/Containers/8331549?itemPath=integration-reports%2Fubuntu-20.04%2Fbundles%2Ftest-integration%2FTestCreateWithDuplicateNetworkNames%2Fd47798cc212d1%2Fdocker.log
        at ClientRequest.<anonymous> (/home/runner/work/_actions/actions/download-artifact/v3/dist/index.js:3681:26)
        at Object.onceWrapper (node:events:627:28)
        at ClientRequest.emit (node:events:513:28)
        at TLSSocket.emitRequestTimeout (node:_http_client:839:9)
        at Object.onceWrapper (node:events:627:28)
        at TLSSocket.emit (node:events:525:35)
        at TLSSocket.Socket._onTimeout (node:net:550:8)
        at listOnTimeout (node:internal/timers:559:17)
        at processTimers (node:internal/timers:502:7)
    Exponential backoff for retry #1. Waiting for 5305 milliseconds before continuing the download
    Total file count: 1004 ---- Processed file #436 (43.4%)

And, it looks like GitHub doesn't allow cancelling the job, possibly
because it is defined with `if: always()`?

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-21 18:52:52 +02:00
Sebastiaan van Stijn
f44fbab3ec
libnetwork: remove Controller.WalkSandboxes and related code
This functionality has been replaced with Controller.GetSandbox, and is
no longer used anywhere.

This patch removes:

- the Controller.WalkSandboxes method
- the SandboxContainerWalker SandboxWalker
- the SandboxWalker type

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-21 15:06:28 +02:00
Sebastiaan van Stijn
6dba98cf38
libnetwork: implement Controller.GetSandbox(containerID)
Various parts of the code were using "walkers" to iterate over the
controller's sandboxes, and the only condition for all of them was
to find the sandbox for a given container-ID. Iterating over all
sandboxes was also sub-optimal, because on Windows, the ContainerID
is used as Sandbox-ID, which can be used to lookup the sandbox from
the "sandboxes" map on the controller.

This patch implements a GetSandbox method on the controller that
looks up the sandbox for a given container-ID, using the most optimal
approach (depending on the platform).

The new method can return errors for invalid (empty) container-IDs, and
a "not found" error to allow consumers to detect non-existing sandboxes,
or potentially invalid IDs.

This new method replaces the (non-exported) Daemon.getNetworkSandbox(),
which was only used internally, in favor of directly accessing the
controller's method.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-21 15:06:26 +02:00
Sebastiaan van Stijn
6c4153f348
libnetwork: parallelTester: move vars closer to where they're used
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-21 15:03:14 +02:00
Sebastiaan van Stijn
e690cdb6a3
libnetwork: remove redundant sandboxTable type
It was not exported so let's remove the abstraction to not make it look
like something more than it is.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-21 15:03:14 +02:00
Sebastiaan van Stijn
3c10db669e
libnetwork: remove unused SandboxKeyWalker
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-21 15:03:08 +02:00
Sebastiaan van Stijn
a01bcf9767
Merge pull request #46270 from thaJeztah/classic_build_remove_logs
classic builder: align "removing intermediate container" output
2023-08-21 12:30:55 +02:00
Sebastiaan van Stijn
cc414a2012
libnetwork/osl: remove Sandbox.Info()
"Pay no attention to the implementation behind the curtain!"

There's only one implementation of the Sandbox interface, and only one implementation
of the Info interface, and they both happens to be implemented by the same type:
networkNamespace. Let's merge these interfaces.

And now that we know that there's one, and only one Info, we can drop the charade,
and relieve the Sandbox from its dual personality.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-20 19:26:39 +02:00
Sebastiaan van Stijn
3b9f4395cf
libnetwork: remove InterfaceInfo interface
Use the only implementation (EndpointInterface) instead.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-20 19:08:21 +02:00
Albin Kerouanton
d146e592d8
api/t/network: Move IPAM types to their own file
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-08-19 11:56:28 +02:00
Sebastiaan van Stijn
cf15460a3b
Merge pull request #46273 from crazy-max/ci-bin-image-repo-origin
ci(bin-image): check repo origin
2023-08-19 11:19:13 +02:00
CrazyMax
219d4d9db9
ci(bin-image): check repo origin
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-08-19 07:19:10 +02:00
Bjorn Neergaard
d125823d3f
hack: use long SHA for DOCKER_GITCOMMIT
This better aligns to GHA/CI settings, and is in general a better
practice in the year 2023.

We also drop the 'unsupported' fallback for `git rev-parse` in the
Makefile; we have a better fallback behavior for an empty
DOCKER_GITCOMMIT in `hack/make.sh`.

Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-08-18 14:42:51 -06:00
Bjorn Neergaard
2010f4338e
ci(bin-image): clean up metadata
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-08-18 14:32:59 -06:00
Bjorn Neergaard
ad91fc1b00
ci(bin-image): clean up env var handling
There are still messy special cases (e.g. DOCKER_GITCOMMIT vs VERSION),
but this makes things a little easier to follow, as we keep
GHA-specifics in the GHA files.

Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-08-18 14:30:20 -06:00
Sebastiaan van Stijn
5dc44f03f0
classic builder: align "removing intermediate container" output
This is something that stood out to me: removing the intermediate
container is part of a build step, but unlike the other output from
the build, wasn't indented (and prefixed with `--->`) to be shown
as part of the build.

This patch adds the `--->` prefix, to make it clearer what step the
removal was part of.

While at it, I also updated the message itself: this output is printed
_after_ the intermediate container has been removed, so we may as well
make it match reality, so I changed "removing" to "removed".

Before:

    echo -e 'FROM busybox\nRUN echo hello > /dev/null\nRUN echo world > /dev/null\n' | DOCKER_BUILDKIT=0 docker build --no-cache -
    Sending build context to Docker daemon  2.048kB
    Step 1/3 : FROM busybox
     ---> a416a98b71e2
    Step 2/3 : RUN echo hello > /dev/null
     ---> Running in a1a65b9365ac
    Removing intermediate container a1a65b9365ac
     ---> 8c6b57ebebdd
    Step 3/3 : RUN echo world > /dev/null
     ---> Running in 9fa977b763a5
    Removing intermediate container 9fa977b763a5
     ---> 795c1f2fc7b9
    Successfully built 795c1f2fc7b9

After:

    echo -e 'FROM busybox\nRUN echo hello > /dev/null\nRUN echo world > /dev/null\n' | DOCKER_BUILDKIT=0 docker build --no-cache -
    Sending build context to Docker daemon  2.048kB
    Step 1/3 : FROM busybox
     ---> fc9db2894f4e
    Step 2/3 : RUN echo hello > /dev/null
     ---> Running in 38d7c34c2178
     ---> Removed intermediate container 38d7c34c2178
     ---> 7c0dbc45111d
    Step 3/3 : RUN echo world > /dev/null
     ---> Running in 629620285d4c
     ---> Removed intermediate container 629620285d4c
     ---> b92f70f2e57d
    Successfully built b92f70f2e57d

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-18 21:27:56 +02:00
Sebastiaan van Stijn
fc302d7b7b
Merge pull request #46146 from akerouanton/libnet-errors-into-http-status-code
api: Convert libnet's errors into HTTP status code
2023-08-18 17:21:03 +02:00
Sebastiaan van Stijn
a866e50e6e
Merge pull request #46238 from vvoland/c8d-inspect-handle-missing-config
c8d/inspect: Ignore manifest with missing config
2023-08-18 15:35:44 +02:00
Sebastiaan van Stijn
1c937c5816
Merge pull request #46189 from vvoland/c8d-more-mount-refcount
c8d integration: Use refcount mounter for diff and export
2023-08-18 15:29:13 +02:00
Sebastiaan van Stijn
6cef3c2b77
Merge pull request #46256 from crazy-max/ci-win-baseimg
windows: update default base image for dev container
2023-08-18 14:15:00 +02:00
Sebastiaan van Stijn
06e540c271
libnetwork: move all SandboxOptions to a separate file
Just a minor reorganisation; move options together into a dedicated file.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-18 13:12:25 +02:00
Sebastiaan van Stijn
2afa4eba43
libnetwork: resolver: Resolver.dialExtDNS use joinHostPort and cleanup
Slightly refactor Resolver.dialExtDNS:

- use net.JoinHostPort to properly format IPv6 addresses
- define a const for the default port, and avoid int ->  string
  conversion if no custom port is defined
- slightly simplify logic if the HostLoopback is used (at the cost of
  duplicating one line); in that case we don't need to define the closure

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-18 13:06:42 +02:00
Sebastiaan van Stijn
3218e26a22
libnetwork: resolver: remove some intermediate variables
Use struct-literals where possible for slightly more readable code.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-18 13:06:42 +02:00
Sebastiaan van Stijn
986de11464
libnetwork: resolver: remove setCommonFlags, use createRespMsg
This function was added in 36fd9d02be
(libnetwork: ce6c6e8c35),
because there were multiple places where a DNS response was created,
which had to use the same options. However, new "common" options were
added since, and having it in a function separate from the other (also
common) options was just hiding logic, so let's remove it.

What the above probably _should_ have done was to create a common utility
to create a DNS response (as all other options are shared as well). This
was actually done in 0c22e1bd07 (libnetwork:
be3531759b),
which added a `createRespMsg` utility, but missed that it could be used
for both cases.

This patch:

- removes the setCommonFlags function
- uses createRespMsg instead to share common options

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-18 13:06:39 +02:00
Sebastiaan van Stijn
9efe6501bb
Merge pull request #46261 from vvoland/volume-decrement-refcount-typo
volume/local: Fix debug log typo
2023-08-18 12:50:38 +02:00
Paweł Gronowski
7f965d55c7
volume/local: Fix debug log typo
Active count is incremented, but message claimed the opposite.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-08-18 11:38:50 +02:00
Kevin Alvarez
678ce73907
windows: update default base image for dev container
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-08-18 10:19:10 +02:00
Sebastiaan van Stijn
01ac4892e0
Merge pull request #46243 from thaJeztah/remove_datascope_consts
libnetwork/datastore: remove deprecated scope consts
2023-08-17 23:49:56 +02:00
Sebastiaan van Stijn
f7ce59f7dc
Merge pull request #46259 from neersighted/bin_image_sha
ci(bin-image): populate DOCKER_GITCOMMIT, take 2
2023-08-17 22:50:57 +02:00
Sebastiaan van Stijn
0c522c6bbf
libnetwork/datastore: remove deprecated scope consts
Removes the deprecated consts, which moved to a separate "scope" package
in commit 6ec03d6745, and are no longer used;

- datastore.LocalScope
- datastore.GlobalScope
- datastore.SwarmScope

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-17 22:42:15 +02:00
Sebastiaan van Stijn
16f152b1ec
Merge pull request #46242 from thaJeztah/bump_swarmkit
vendor: github.com/moby/swarmkit/v2 v2.0.0-20230815220644-3f2e40b3ed51
2023-08-17 21:35:45 +02:00
Bjorn Neergaard
73ffb48bfb
ci(bin-image): populate DOCKER_GITCOMMIT, take 2
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-08-17 13:17:33 -06:00
Bjorn Neergaard
73e47ee39c
Merge pull request #46257 from neersighted/bin_image_sha
ci(bin-image): populate DOCKER_GITCOMMIT
2023-08-17 13:12:22 -06:00
Bjorn Neergaard
9aed6308d4
ci(bin-image): populate DOCKER_GITCOMMIT
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-08-17 11:53:27 -06:00
Albin Kerouanton
c22ec82477
libnet: Fix error capitalization
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-08-17 16:48:09 +02:00
Albin Kerouanton
bd0111c1f4
libnet: Replace NoServiceError with UnavailableError
UnavailableError is now compatible with errdefs.UnavailableError. These
errors will now return a 503 instead of a 500.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-08-17 16:46:53 +02:00
Albin Kerouanton
42d34e40f9
libnet: Replace BadRequest with InvalidParameter
InvalidParameter is now compatible with errdefs.InvalidParameter. Thus,
these errors will now return a 400 status code instead of a 500.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-08-17 16:45:04 +02:00
Paweł Gronowski
a64adda4e7
c8d/inspect: Ignore manifest with missing config
Fix a failure to inspect image if any of its present manifest references
an image config which isn't present locally.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-08-17 09:48:29 +02:00
Sebastiaan van Stijn
c85a33d74b
libnetwork/iptables: un-export ZoneSettings, and slight refactor
- un-export ZoneSettings, because it's only used internally
- make conversion to a "interface" slice a method on the struct
- remove the getDockerZoneSettings() function, and move the type-definition
  close to where it's used, as it was only used in a single location

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-16 17:41:23 +02:00
Sebastiaan van Stijn
d979d2af45
libnetwork/iptables: fix TestFirewalldInit
This test didn't make a lot of sense, because `checkRunning()` depends on
the `connection` package-var being set, which is done by `firewalldInit()`,
so would never be true on its own.

Add a small utility that opens its own D-Bus connection to verify if
firewalld is running, and otherwise skips the tests (preserving any
error in the process).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-16 17:28:07 +02:00
Sebastiaan van Stijn
513063bcf9
libnetwork/iptables: ProgramChain: don't fail if interface not found
DelInterfaceFirewalld returns an error if the interface to delete was
not found. Let's ignore cases where we were successfully able to get
the list of interfaces in the zone, but the interface was not part of
the zone.

This patch changes the error for these cases to an errdefs.ErrNotFound,
and updates IPTable.ProgramChain to ignore those errors.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-16 17:25:10 +02:00
Sebastiaan van Stijn
d83ead8434
Merge pull request #46188 from akerouanton/custom-multierror
Add a temporary drop-in replacement for errors.Join
2023-08-16 17:16:09 +02:00
Albin Kerouanton
64de635626
Add a temporary drop-in replacement for errors.Join
As we have a hard time figuring out what moby/moby#46099 should look
like, this drop-in replacement will solve the initial formatting problem
we have. It's made internal such that we can remove it whenever we want
and unlike moby/moby#46099 doesn't require thoughtful API changes.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-16 16:18:41 +02:00
Sebastiaan van Stijn
cff341e5e6
vendor: github.com/moby/swarmkit/v2 v2.0.0-20230815220644-3f2e40b3ed51
Remove uses of deprecated datastore.LocalScope const

full diff: 60421a63a7...3f2e40b3ed

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-16 14:49:13 +02:00
Sebastiaan van Stijn
56b62640a2
libnetwork: Controller.NewSandbox: use named error-return
It's used in various defers, but was using `err` as name, which can be
confusing, and increases the risk of accidentally shadowing the error.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-16 13:25:10 +02:00