Commit graph

48538 commits

Author SHA1 Message Date
Sebastiaan van Stijn
e2fc596de4
Merge pull request #46293 from rumpl/fix-platform-check
Don't return lease not found errors on platform mismatch
2023-08-23 15:55:04 +02:00
Sebastiaan van Stijn
c0568a95d8
integration/container: check some error-types in tests
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-23 15:50:46 +02:00
Sebastiaan van Stijn
2b583c0923
daemon: cleanupContainer: slightly cleanup error messages
Also remove integration-cli: `DockerAPISuite.TestContainerAPIDeleteConflict`,
which was testing the same conditions as `TestRemoveContainerRunning` in
integration/container.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-23 15:50:43 +02:00
Sebastiaan van Stijn
69cf2ad6a5
daemon: cleanupContainer: don't fail if container is already stopped
Saw this failure in a flaky test, and I wondered why we consider this an
error condition;

    === RUN   TestKillWithStopSignalAndRestartPolicies
        main_test.go:32: assertion failed: error is not nil: Error response from daemon: Could not kill running container 668f62511f4aa62357269cd405cff1fbe295b7f6d5011e7cfed434e3072330b7, cannot remove - Container 668f62511f4aa62357269cd405cff1fbe295b7f6d5011e7cfed434e3072330b7 is not running: failed to remove 668f62511f4aa62357269cd405cff1fbe295b7f6d5011e7cfed434e3072330b7
    --- FAIL: TestKillWithStopSignalAndRestartPolicies (0.84s)
    === RUN   TestKillWithStopSignalAndRestartPolicies/same-signal-disables-restart-policy
        --- PASS: TestKillWithStopSignalAndRestartPolicies/same-signal-disables-restart-policy (0.42s)
    === RUN   TestKillWithStopSignalAndRestartPolicies/different-signal-keep-restart-policy
        --- PASS: TestKillWithStopSignalAndRestartPolicies/different-signal-keep-restart-policy (0.23s)

In the above;

1. `Error response from daemon: Could not kill running container 668f62511f4aa62357269cd405cff1fbe295b7f6d5011e7cfed434e3072330b7`
2. `cannot remove - Container 668f62511f4aa62357269cd405cff1fbe295b7f6d5011e7cfed434e3072330b7 is not running`
3. `failed to remove 668f62511f4aa62357269cd405cff1fbe295b7f6d5011e7cfed434e3072330b7`

So it looks like the removal fails because we couldn't kill the container
because it was already stopped, which may be a race condition where the first
check shows the container to be running (but may already be in process to be
removed or killed. In either case, we probably shouldn't fail the removal if
the container is already stopped.

This patch adds a `isNotRunning()` utility, so that we can ignore this case,
and proceed with the removal.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-23 15:47:43 +02:00
Sebastiaan van Stijn
20b770237c
daemon/network: Settings: move sandbox fields together
Just a minor nit; let's move these fields together.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-23 14:26:58 +02:00
Sebastiaan van Stijn
405f571ae8
daemon: updateSandboxNetworkSettings: remove unused error-return
This function never returns an error, so let's remove the error-return,
and give it a slightly more to-the-point name.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-23 14:26:58 +02:00
Sebastiaan van Stijn
2eccf0e8d1
libnetwork: Controller.NewSandbox: don't generate ID if not used
Windows uses the container-iD as ID for sandboxes, so it's not needed to
generate an ID  when running on Windows.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-23 14:26:53 +02:00
Sebastiaan van Stijn
389b21a341
Merge pull request #46262 from thaJeztah/libnetwork_resolv_cleanups
libnetwork: resolve: assorted cleanups
2023-08-23 14:22:33 +02:00
Sebastiaan van Stijn
13c4eaea92
Merge pull request #46205 from thaJeztah/libnetwork_noexecroot
libnetwork: cleanup SetBasePath, un-export SetExternalKey and other cleanups
2023-08-23 14:21:30 +02:00
Sebastiaan van Stijn
8f0b62e4d9
Merge pull request #46226 from thaJeztah/conntrack_reduce_noise
libnetwork/iptables: reduce DeleteConntrackEntriesXX debug logs
2023-08-23 14:19:53 +02:00
Sebastiaan van Stijn
bb5813a93d
Merge pull request #46240 from thaJeztah/libnetwork_named_returns
libnetwork: Controller.NewNetwork, Controller.NewSandbox: use named error-return
2023-08-23 14:19:27 +02:00
Sebastiaan van Stijn
cec49ba976
Merge pull request #46247 from thaJeztah/libnetwork_unexport_ZoneSettings
libnetwork/iptables: un-export ZoneSettings, and slight refactor
2023-08-23 14:17:38 +02:00
Sebastiaan van Stijn
0e7186af95
Merge pull request #46278 from thaJeztah/libnetwork_remove_InterfaceInfo_interface
libnetwork: remove InterfaceInfo interface
2023-08-23 14:16:23 +02:00
Sebastiaan van Stijn
8c3b14ee85
Merge pull request #46264 from thaJeztah/libnetwork_sandbox_split_options
libnetwork: move all SandboxOptions to a separate file
2023-08-23 14:13:52 +02:00
Sebastiaan van Stijn
77539e7bc7
Merge pull request #46279 from thaJeztah/libnetwork_remove_sandbox_info
libnetwork/osl: remove Sandbox.Info()
2023-08-23 14:13:26 +02:00
Sebastiaan van Stijn
79fa17e181
Merge pull request #46298 from rumpl/c8d-test-skip-graphdriver
test: Skip graph driver tests when using containerd snapshotters
2023-08-23 14:00:58 +02:00
Sebastiaan van Stijn
18c5d863d6
Merge pull request #46296 from thaJeztah/replace_dockerignore
replace dockerfile/dockerignore with patternmatcher/ignorefile
2023-08-23 10:55:14 +02:00
Djordje Lukic
6cbe06ff3f
test: Skip graph driver tests when using containerd snapshotters
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-08-23 09:48:27 +02:00
Sebastiaan van Stijn
3553b4c684
replace dockerfile/dockerignore with patternmatcher/ignorefile
The BuildKit dockerignore package was integrated in the patternmatcher
repository / module. This patch updates our uses of the BuildKit package
with its new location.

A small local change was made to keep the format of the existing error message,
because the "ignorefile" package is slightly more agnostic in that respect
and doesn't include ".dockerignore" in the error message.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-23 00:53:18 +02:00
Sebastiaan van Stijn
a479b287c7
vendor: github.com/moby/patternmatcher v0.6.0
- integrate frontend/dockerfile/dockerignore from buildkit

full diff: https://github.com/moby/patternmatcher/compare/v0.5.0...v0.6.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-23 00:50:05 +02:00
Sebastiaan van Stijn
70ae5c13ea
Merge pull request #44704 from thaJeztah/api_hostconfig_RestartPolicyMode_enum
api/types/container: add RestartPolicyMode type and enum
2023-08-22 22:31:51 +02:00
Bjorn Neergaard
8383430946
Merge pull request #45759 from akerouanton/validate-ipam-config
Validate IPAM config before handing it over to libnetwork
2023-08-22 13:58:28 -06:00
Paweł Gronowski
bedcc94de4
daemon: Handle NotFound when deleting container lease
If the lease doesn't exit (for example when creating the container
failed), just ignore the not found error.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-08-22 17:32:22 +02:00
Paweł Gronowski
dfaff9598c
c8d/run: Allow running container without image
This allows the legacy builder to apply changes to the `FROM scratch`
layer.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-08-22 17:32:20 +02:00
Paweł Gronowski
eb56493f4e
c8d/commit: Don't produce an empty layer
If the diff is empty and don't produce an empty layer.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-08-22 17:32:18 +02:00
Albin Kerouanton
3e8af0817a
ipam: Replace ChildSubnet with parent Subnet when its mask is bigger
Prior to moby/moby#44968, libnetwork would happily accept a ChildSubnet
with a bigger mask than its parent subnet. In such case, it was
producing IP addresses based on the parent subnet, and the child subnet
was not allocated from the address pool.

This commit automatically fixes invalid ChildSubnet for networks stored
in libnetwork's datastore.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-08-22 17:12:06 +02:00
Albin Kerouanton
4f47013feb
api: Validate IPAM config before creating a network
Currently, IPAM config is never validated by the API. Some checks
are done by the CLI, but they're not exhaustive. And some of these
misconfigurations might be caught early by libnetwork (ie. when the
network is created), and others only surface when connecting a container
to a misconfigured network. In both cases, the API would return a 500.

Although the `NetworkCreate` endpoint might already return warnings,
these are never displayed by the CLI. As such, it was decided during a
maintainer's call to return validation errors _for all API versions_.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-22 17:11:54 +02:00
Sebastiaan van Stijn
2be118379e
api/types/container: add RestartPolicyMode type and enum
Also move the validation function to live with the type definition,
which allows it to be used outside of the daemon as well.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-22 16:40:57 +02:00
Djordje Lukic
b8ff8ea58e
Don't return an error if the lease is not found
If the image for the wanted platform doesn't exist then the lease
doesn't exist either. Returning this error hides the real error, so
let's not return it.

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-08-22 14:52:41 +02:00
Sebastiaan van Stijn
331854a126
Merge pull request #46239 from thaJeztah/vars_collide
libnetwork:  rename vars that collided
2023-08-22 10:50:35 +02:00
Sebastiaan van Stijn
fa517bb420
Merge pull request #46271 from neersighted/bin_image_meta
bin-image: metadata cleanup, take two
2023-08-21 21:43:04 +02:00
Sebastiaan van Stijn
5d15da8290
Merge pull request #46206 from thaJeztah/libnetwork_no_walk
libnetwork: implement Controller.GetSandbox, remove Controller.WalkSandboxes and related code
2023-08-21 21:21:16 +02:00
Sebastiaan van Stijn
3ffa00909d
Merge pull request #46285 from thaJeztah/gha_report_timeout
gha: set 10-minute timeout on "report" actions
2023-08-21 20:19:39 +02:00
Bjorn Neergaard
764419ed4e
Merge pull request #45534 from polarathene/fix/config-init-limits
fix: Normalize `RLIMIT_NOFILE` to sensible defaults
2023-08-21 12:09:38 -06:00
Brian Goff
fc15e08a6c
Merge pull request #46137 from thaJeztah/v1_deprecation 2023-08-21 10:51:23 -07:00
Sebastiaan van Stijn
dff53a02ef
libnetwork: remove Controller.Sandboxes as it's no longer used
The Controller.Sandboxes method was used by some SandboxWalkers. Now
that those have been removed, there are no longer any consumers of this
method, so let's remove it for now.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-21 19:08:43 +02:00
Sebastiaan van Stijn
d6f340e784
gha: set 10-minute timeout on "report" actions
I had a CI run fail to "Upload reports":

    Exponential backoff for retry #1. Waiting for 4565 milliseconds before continuing the upload at offset 0
    Finished backoff for retry #1, continuing with upload
    Total file count: 211 ---- Processed file #160 (75.8%)
    ...
    Total file count: 211 ---- Processed file #164 (77.7%)
    Total file count: 211 ---- Processed file #164 (77.7%)
    Total file count: 211 ---- Processed file #164 (77.7%)
    A 503 status code has been received, will attempt to retry the upload
    ##### Begin Diagnostic HTTP information #####
    Status Code: 503
    Status Message: Service Unavailable
    Header Information: {
      "content-length": "592",
      "content-type": "application/json; charset=utf-8",
      "date": "Mon, 21 Aug 2023 14:08:10 GMT",
      "server": "Kestrel",
      "cache-control": "no-store,no-cache",
      "pragma": "no-cache",
      "strict-transport-security": "max-age=2592000",
      "x-tfs-processid": "b2fc902c-011a-48be-858d-c62e9c397cb6",
      "activityid": "49a48b53-0411-4ff3-86a7-4528e3f71ba2",
      "x-tfs-session": "49a48b53-0411-4ff3-86a7-4528e3f71ba2",
      "x-vss-e2eid": "49a48b53-0411-4ff3-86a7-4528e3f71ba2",
      "x-vss-senderdeploymentid": "63be6134-28d1-8c82-e969-91f4e88fcdec",
      "x-frame-options": "SAMEORIGIN"
    }
    ###### End Diagnostic HTTP information ######
    Retry limit has been reached for chunk at offset 0 to https://pipelinesghubeus5.actions.githubusercontent.com/Y2huPMnV2RyiTvKoReSyXTCrcRyxUdSDRZYoZr0ONBvpl5e9Nu/_apis/resources/Containers/8331549?itemPath=integration-reports%2Fubuntu-22.04-systemd%2Fbundles%2Ftest-integration%2FTestInfoRegistryMirrors%2Fd20ac12e48cea%2Fdocker.log
    Warning: Aborting upload for /tmp/reports/ubuntu-22.04-systemd/bundles/test-integration/TestInfoRegistryMirrors/d20ac12e48cea/docker.log due to failure
    Error: aborting artifact upload
    Total file count: 211 ---- Processed file #165 (78.1%)
    A 503 status code has been received, will attempt to retry the upload
    Exponential backoff for retry #1. Waiting for 5799 milliseconds before continuing the upload at offset 0

As a result, the "Download reports" continued retrying:

    ...
    Total file count: 1004 ---- Processed file #436 (43.4%)
    Total file count: 1004 ---- Processed file #436 (43.4%)
    Total file count: 1004 ---- Processed file #436 (43.4%)
    An error occurred while attempting to download a file
    Error: Request timeout: /Y2huPMnV2RyiTvKoReSyXTCrcRyxUdSDRZYoZr0ONBvpl5e9Nu/_apis/resources/Containers/8331549?itemPath=integration-reports%2Fubuntu-20.04%2Fbundles%2Ftest-integration%2FTestCreateWithDuplicateNetworkNames%2Fd47798cc212d1%2Fdocker.log
        at ClientRequest.<anonymous> (/home/runner/work/_actions/actions/download-artifact/v3/dist/index.js:3681:26)
        at Object.onceWrapper (node:events:627:28)
        at ClientRequest.emit (node:events:513:28)
        at TLSSocket.emitRequestTimeout (node:_http_client:839:9)
        at Object.onceWrapper (node:events:627:28)
        at TLSSocket.emit (node:events:525:35)
        at TLSSocket.Socket._onTimeout (node:net:550:8)
        at listOnTimeout (node:internal/timers:559:17)
        at processTimers (node:internal/timers:502:7)
    Exponential backoff for retry #1. Waiting for 5305 milliseconds before continuing the download
    Total file count: 1004 ---- Processed file #436 (43.4%)

And, it looks like GitHub doesn't allow cancelling the job, possibly
because it is defined with `if: always()`?

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-21 18:52:52 +02:00
Sebastiaan van Stijn
f44fbab3ec
libnetwork: remove Controller.WalkSandboxes and related code
This functionality has been replaced with Controller.GetSandbox, and is
no longer used anywhere.

This patch removes:

- the Controller.WalkSandboxes method
- the SandboxContainerWalker SandboxWalker
- the SandboxWalker type

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-21 15:06:28 +02:00
Sebastiaan van Stijn
6dba98cf38
libnetwork: implement Controller.GetSandbox(containerID)
Various parts of the code were using "walkers" to iterate over the
controller's sandboxes, and the only condition for all of them was
to find the sandbox for a given container-ID. Iterating over all
sandboxes was also sub-optimal, because on Windows, the ContainerID
is used as Sandbox-ID, which can be used to lookup the sandbox from
the "sandboxes" map on the controller.

This patch implements a GetSandbox method on the controller that
looks up the sandbox for a given container-ID, using the most optimal
approach (depending on the platform).

The new method can return errors for invalid (empty) container-IDs, and
a "not found" error to allow consumers to detect non-existing sandboxes,
or potentially invalid IDs.

This new method replaces the (non-exported) Daemon.getNetworkSandbox(),
which was only used internally, in favor of directly accessing the
controller's method.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-21 15:06:26 +02:00
Sebastiaan van Stijn
6c4153f348
libnetwork: parallelTester: move vars closer to where they're used
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-21 15:03:14 +02:00
Sebastiaan van Stijn
e690cdb6a3
libnetwork: remove redundant sandboxTable type
It was not exported so let's remove the abstraction to not make it look
like something more than it is.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-21 15:03:14 +02:00
Sebastiaan van Stijn
3c10db669e
libnetwork: remove unused SandboxKeyWalker
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-21 15:03:08 +02:00
Sebastiaan van Stijn
a01bcf9767
Merge pull request #46270 from thaJeztah/classic_build_remove_logs
classic builder: align "removing intermediate container" output
2023-08-21 12:30:55 +02:00
Sebastiaan van Stijn
cc414a2012
libnetwork/osl: remove Sandbox.Info()
"Pay no attention to the implementation behind the curtain!"

There's only one implementation of the Sandbox interface, and only one implementation
of the Info interface, and they both happens to be implemented by the same type:
networkNamespace. Let's merge these interfaces.

And now that we know that there's one, and only one Info, we can drop the charade,
and relieve the Sandbox from its dual personality.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-20 19:26:39 +02:00
Sebastiaan van Stijn
3b9f4395cf
libnetwork: remove InterfaceInfo interface
Use the only implementation (EndpointInterface) instead.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-20 19:08:21 +02:00
Albin Kerouanton
d146e592d8
api/t/network: Move IPAM types to their own file
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-08-19 11:56:28 +02:00
Sebastiaan van Stijn
cf15460a3b
Merge pull request #46273 from crazy-max/ci-bin-image-repo-origin
ci(bin-image): check repo origin
2023-08-19 11:19:13 +02:00
CrazyMax
219d4d9db9
ci(bin-image): check repo origin
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-08-19 07:19:10 +02:00
Bjorn Neergaard
d125823d3f
hack: use long SHA for DOCKER_GITCOMMIT
This better aligns to GHA/CI settings, and is in general a better
practice in the year 2023.

We also drop the 'unsupported' fallback for `git rev-parse` in the
Makefile; we have a better fallback behavior for an empty
DOCKER_GITCOMMIT in `hack/make.sh`.

Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-08-18 14:42:51 -06:00
Bjorn Neergaard
2010f4338e
ci(bin-image): clean up metadata
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-08-18 14:32:59 -06:00