moby/integration-cli
Sebastiaan van Stijn 4f9db655ed
portmapper: move userland-proxy lookup to daemon config
When mapping a port with the userland-proxy enabled, the daemon would
perform an "exec.LookPath" for every mapped port (which, in case of
a range of ports, would be for every port in the range).

This was both inefficient (looking up the binary for each port), inconsistent
(when running in rootless-mode, the binary was looked-up once), as well as
inconvenient, because a missing binary, or a mis-configureed userland-proxy-path
would not be detected daeemon startup, and not produce an error until starting
the container;

    docker run -d -P nginx:alpine
    4f7b6589a1680f883d98d03db12203973387f9061e7a963331776170e4414194
    docker: Error response from daemon: driver failed programming external connectivity on endpoint romantic_wiles (7cfdc361821f75cbc665564cf49856cf216a5b09046d3c22d5b9988836ee088d): fork/exec docker-proxy: no such file or directory.

However, the container would still be created (but invalid);

    docker ps -a
    CONTAINER ID   IMAGE          COMMAND                  CREATED          STATUS    PORTS     NAMES
    869f41d7e94f   nginx:alpine   "/docker-entrypoint.…"   10 seconds ago   Created             romantic_wiles

This patch changes how the userland-proxy is configured;

- The path of the userland-proxy is now looked up / configured at daemon
  startup; this is similar to how the proxy is configured in rootless-mode.
- A warning is logged when failing to lookup the binary.
- If the daemon is configured with "userland-proxy" enabled, an error is
  produced, and the daemon will refuse to start.
- The "proxyPath" argument for newProxyCommand() (in libnetwork/portmapper)
  is now required to be set. It no longer looks up the executable, and
  produces an error if no path was provided. While this change was not
  required, it makes the daemon config the canonical source of truth, instead
  of logic spread accross multiplee locations.

Some of this logic is a change of behavior, but these changes were made with
the assumption that we don't want to support;

- installing the userland proxy _after_ the daemon was started
- moving the userland proxy (or installing a proxy with a higher
  preference in PATH)

With this patch:

Validating the config produces an error if the binary is not found:

    dockerd --validate
    WARN[2023-12-29T11:36:39.748699591Z] failed to lookup default userland-proxy binary       error="exec: \"docker-proxy\": executable file not found in $PATH"
    userland-proxy is enabled, but userland-proxy-path is not set

Disabling userland-proxy prints a warning, but validates as "OK":

    dockerd --userland-proxy=false --validate
    WARN[2023-12-29T11:38:30.752523879Z] ffailed to lookup default userland-proxy binary       error="exec: \"docker-proxy\": executable file not found in $PATH"
    configuration OK

Speficying a non-absolute path produces an error:

    dockerd --userland-proxy-path=docker-proxy --validate
    invalid userland-proxy-path: must be an absolute path: docker-proxy

Befor this patch, we would not validate this path, which would allow the daemon
to start, but fail to map a port;

    docker run -d -P nginx:alpine
    4f7b6589a1680f883d98d03db12203973387f9061e7a963331776170e4414194
    docker: Error response from daemon: driver failed programming external connectivity on endpoint romantic_wiles (7cfdc361821f75cbc665564cf49856cf216a5b09046d3c22d5b9988836ee088d): fork/exec docker-proxy: no such file or directory.

Specifying an invalid userland-proxy-path produces an error as well:

    dockerd --userland-proxy-path=/usr/local/bin/no-such-binary --validate
    userland-proxy-path is invalid: stat /usr/local/bin/no-such-binary: no such file or directory

    mkdir -p /usr/local/bin/not-a-file
    dockerd --userland-proxy-path=/usr/local/bin/not-a-file --validate
    userland-proxy-path is invalid: exec: "/usr/local/bin/not-a-file": is a directory

    touch /usr/local/bin/not-an-executable
    dockerd --userland-proxy-path=/usr/local/bin/not-an-executable --validate
    userland-proxy-path is invalid: exec: "/usr/local/bin/not-an-executable": permission denied

Same when using the daemon.json config-file;

    echo '{"userland-proxy-path":"no-such-binary"}' > /etc/docker/daemon.json
    dockerd --validate
    unable to configure the Docker daemon with file /etc/docker/daemon.json: merged configuration validation from file and command line flags failed: invalid userland-proxy-path: must be an absolute path: no-such-binary

    dockerd --userland-proxy-path=hello --validate
    unable to configure the Docker daemon with file /etc/docker/daemon.json: the following directives are specified both as a flag and in the configuration file: userland-proxy-path: (from flag: hello, from file: /usr/local/bin/docker-proxy)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-29 16:23:18 +01:00
..
checker bump gotest.tools v3.0.1 for compatibility with Go 1.14 2020-02-11 00:06:42 +01:00
cli Add t.Helper() to the cli test helper functions 2023-07-13 13:37:26 +02:00
daemon Wire up tests to support otel tracing 2023-09-07 18:38:22 +00:00
environment Wire up tests to support otel tracing 2023-09-07 18:38:22 +00:00
fixtures integration-cli: fix test rogue certs 2022-05-19 10:54:31 +02:00
requirement integration-cli: make testRequires() a Helper 2019-10-09 21:44:22 +02:00
testdata Clean some integration-cli/fixtures package/files 2018-04-16 10:48:58 +02:00
benchmark_test.go integration-cli: DockerBenchmarkSuite: replace dockerCmd and waitRun 2023-10-19 17:38:54 +02:00
check_test.go c8d: disable schema1 registry integration tests 2023-09-11 15:30:44 +02:00
daemon_swarm_hack_test.go rm-gocheck: run goimports to compile successfully 2019-09-09 21:06:12 +00:00
docker_api_attach_test.go integration-cli: DockerAPISuite: replace dockerCmd and waitRun 2023-10-19 17:34:55 +02:00
docker_api_build_test.go integration(-cli): remove skips for old daemon versions (<20.10) 2023-12-05 01:03:50 +01:00
docker_api_build_windows_test.go Wire up tests to support otel tracing 2023-09-07 18:38:22 +00:00
docker_api_containers_test.go integration(-cli): remove skips for old daemon versions (<20.10) 2023-12-05 01:03:50 +01:00
docker_api_containers_unix_test.go remove pre-go1.17 build-tags 2023-05-19 20:38:51 +02:00
docker_api_containers_windows_test.go api/types: move ContainerStartOptions to api/types/container 2023-10-12 11:29:24 +02:00
docker_api_exec_resize_test.go integration(-cli): remove skips for old daemon versions (<20.10) 2023-12-05 01:03:50 +01:00
docker_api_exec_test.go integration(-cli): remove skips for old daemon versions (<20.10) 2023-12-05 01:03:50 +01:00
docker_api_images_test.go integration-cli: DockerAPISuite: replace dockerCmd and waitRun 2023-10-19 17:34:55 +02:00
docker_api_inspect_test.go integration-cli: DockerAPISuite: replace dockerCmd and waitRun 2023-10-19 17:34:55 +02:00
docker_api_logs_test.go integration(-cli): remove skips for old daemon versions (<20.10) 2023-12-05 01:03:50 +01:00
docker_api_network_test.go integration(-cli): remove skips for old daemon versions (<20.10) 2023-12-05 01:03:50 +01:00
docker_api_stats_test.go integration-cli: DockerAPISuite: replace dockerCmd and waitRun 2023-10-19 17:34:55 +02:00
docker_api_swarm_node_test.go Wire up tests to support otel tracing 2023-09-07 18:38:22 +00:00
docker_api_swarm_service_test.go Wire up tests to support otel tracing 2023-09-07 18:38:22 +00:00
docker_api_swarm_test.go libnet: Make sure network names are unique 2023-09-12 10:40:13 +02:00
docker_api_test.go integration(-cli): remove skips for old daemon versions (<20.10) 2023-12-05 01:03:50 +01:00
docker_cli_attach_test.go integration-cli: DockerCLIAttachSuite: replace dockerCmd and waitRun 2023-10-19 17:35:46 +02:00
docker_cli_attach_unix_test.go integration-cli: DockerCLIAttachSuite: replace dockerCmd and waitRun 2023-10-19 17:35:46 +02:00
docker_cli_build_test.go c8d/integration-cli: Adjust TestBuildClearCmd 2023-12-07 14:17:35 +01:00
docker_cli_build_unix_test.go integration-cli: DockerCLIBuildSuite: replace dockerCmd 2023-10-19 17:52:39 +02:00
docker_cli_by_digest_test.go c8d/integration-cli: Skip TestListDanglingImagesWithDigests 2023-11-30 14:01:22 +01:00
docker_cli_commit_test.go integration(-cli): remove skips for old daemon versions (<20.10) 2023-12-05 01:03:50 +01:00
docker_cli_cp_from_container_test.go integration-cli: format code with gofumpt 2023-06-29 00:08:27 +02:00
docker_cli_cp_test.go integration-cli: DockerCLICpSuite: replace dockerCmd 2023-10-19 17:52:44 +02:00
docker_cli_cp_to_container_test.go integration-cli: remove isCpCannotCopyReadOnly utility 2022-09-27 22:15:19 +02:00
docker_cli_cp_to_container_unix_test.go integration-cli: DockerCLICpSuite: replace dockerCmd 2023-10-19 17:52:44 +02:00
docker_cli_cp_utils_test.go integration-cli: DockerCLICpSuite: replace dockerCmd 2023-10-19 17:52:44 +02:00
docker_cli_create_test.go integration-cli: DockerCLICreateSuite: replace dockerCmd and waitRun 2023-10-19 18:00:27 +02:00
docker_cli_daemon_plugins_test.go remove pre-go1.17 build-tags 2023-05-19 20:38:51 +02:00
docker_cli_daemon_test.go portmapper: move userland-proxy lookup to daemon config 2023-12-29 16:23:18 +01:00
docker_cli_events_test.go integration-cli: DockerCLIEventSuite: replace dockerCmd and waitRun 2023-10-19 17:36:41 +02:00
docker_cli_events_unix_test.go integration-cli: DockerCLIEventSuite: replace dockerCmd and waitRun 2023-10-19 17:36:41 +02:00
docker_cli_exec_test.go integration-cli: DockerCLIExecSuite: replace dockerCmd and waitRun 2023-10-19 17:36:46 +02:00
docker_cli_exec_unix_test.go integration-cli: DockerCLIExecSuite: replace dockerCmd and waitRun 2023-10-19 17:36:46 +02:00
docker_cli_external_volume_driver_test.go integration-cli: DockerExternalVolumeSuite: replace dockerCmd 2023-10-19 18:01:37 +02:00
docker_cli_health_test.go integration-cli: DockerCLIHealthSuite: replace dockerCmd and waitRun 2023-10-19 17:39:00 +02:00
docker_cli_history_test.go integration-cli: DockerCLIHistorySuite: replace dockerCmd 2023-10-19 18:00:56 +02:00
docker_cli_images_test.go integration-cli: DockerCLIImagesSuite: replace dockerCmd 2023-10-19 17:58:12 +02:00
docker_cli_import_test.go integration-cli: DockerCLIImportSuite: replace dockerCmd 2023-10-19 18:01:24 +02:00
docker_cli_info_test.go integration-cli: DockerCLIInfoSuite: replace dockerCmd and waitRun 2023-10-19 17:38:33 +02:00
docker_cli_info_unix_test.go Wire up tests to support otel tracing 2023-09-07 18:38:22 +00:00
docker_cli_inspect_test.go hack: Load special images on demand 2023-12-06 17:16:37 +01:00
docker_cli_links_test.go integration-cli: DockerCLILinksSuite: replace dockerCmd and waitRun 2023-10-19 17:39:17 +02:00
docker_cli_login_test.go integration-cli: DockerRegistryAuthHtpasswdSuite: replace dockerCmd 2023-10-19 17:39:53 +02:00
docker_cli_logout_test.go integration-cli: DockerRegistryAuthHtpasswdSuite: replace dockerCmd 2023-10-19 17:39:53 +02:00
docker_cli_logs_test.go integration-cli: DockerCLILogsSuite: replace dockerCmd and waitRun 2023-10-19 17:40:15 +02:00
docker_cli_netmode_test.go integration-cli: mark dockerCmdWithFail as an helper function 2023-10-25 22:55:59 +02:00
docker_cli_network_test.go Wire up tests to support otel tracing 2023-09-07 18:38:22 +00:00
docker_cli_network_unix_test.go frozen images: update to debian:bookworm-slim 2023-11-30 12:01:06 +01:00
docker_cli_plugins_logdriver_test.go integration-cli: DockerCLIPluginLogDriverSuite: replace dockerCmd 2023-10-19 18:01:46 +02:00
docker_cli_plugins_test.go integration-cli: DockerCLIPluginsSuite: replace dockerCmd 2023-10-19 18:02:04 +02:00
docker_cli_port_test.go integration-cli: DockerCLIPortSuite: replace dockerCmd and waitRun 2023-10-19 17:57:41 +02:00
docker_cli_proxy_test.go Wire up tests to support otel tracing 2023-09-07 18:38:22 +00:00
docker_cli_prune_test.go integration-cli: split DockerSuite into subsequent build suites 2022-06-17 10:59:04 +02:00
docker_cli_prune_unix_test.go integration-cli: DockerCLIPruneSuite: replace dockerCmd 2023-10-19 18:02:16 +02:00
docker_cli_ps_test.go integration(-cli): remove skips for old daemon versions (<20.10) 2023-12-05 01:03:50 +01:00
docker_cli_pull_local_test.go Skip TestPullManifestList when using containerd 2023-11-02 16:09:35 +01:00
docker_cli_pull_test.go Merge pull request #46517 from rumpl/c8d-windows-pull-linux 2023-11-03 09:28:16 +01:00
docker_cli_push_test.go c8d/integration-cli: Adjust DockerRegistryAuthTokenSuite 2023-11-29 14:28:18 +01:00
docker_cli_registry_user_agent_test.go integration-cli: DockerRegistrySuite: replace dockerCmd 2023-10-19 17:53:43 +02:00
docker_cli_restart_test.go integration-cli: DockerCLIRestartSuite: replace dockerCmd and waitRun 2023-10-19 18:00:39 +02:00
docker_cli_rmi_test.go Add dangling image reference on delete when last image has children 2023-12-19 05:56:43 -08:00
docker_cli_run_test.go hack: Load special images on demand 2023-12-06 17:16:37 +01:00
docker_cli_run_unix_test.go frozen images: update to debian:bookworm-slim 2023-11-30 12:01:06 +01:00
docker_cli_save_load_test.go hack: Load special images on demand 2023-12-06 17:16:37 +01:00
docker_cli_save_load_unix_test.go integration-cli: DockerCLISaveLoadSuite: replace dockerCmd 2023-10-19 17:42:48 +02:00
docker_cli_search_test.go integration-cli: DockerCLISearchSuite: replace dockerCmd 2023-10-19 17:54:11 +02:00
docker_cli_service_create_test.go Wire up tests to support otel tracing 2023-09-07 18:38:22 +00:00
docker_cli_service_health_test.go Wire up tests to support otel tracing 2023-09-07 18:38:22 +00:00
docker_cli_service_logs_test.go Wire up tests to support otel tracing 2023-09-07 18:38:22 +00:00
docker_cli_service_scale_test.go Wire up tests to support otel tracing 2023-09-07 18:38:22 +00:00
docker_cli_sni_test.go Wire up tests to support otel tracing 2023-09-07 18:38:22 +00:00
docker_cli_start_test.go integration-cli: DockerCLIStartSuite: replace dockerCmd 2023-10-19 17:59:34 +02:00
docker_cli_stats_test.go integration-cli: DockerCLIStatsSuite: replace dockerCmd and waitRun 2023-10-19 18:00:00 +02:00
docker_cli_swarm_test.go libnet: Make sure network names are unique 2023-09-12 10:40:13 +02:00
docker_cli_swarm_unix_test.go Wire up tests to support otel tracing 2023-09-07 18:38:22 +00:00
docker_cli_top_test.go integration-cli: DockerCLITopSuite: replace dockerCmd 2023-10-19 18:01:09 +02:00
docker_cli_update_test.go integration-cli: split DockerSuite into subsequent build suites 2022-06-17 10:59:04 +02:00
docker_cli_update_unix_test.go integration-cli: DockerCLIUpdateSuite: replace dockerCmd and waitRun 2023-10-19 17:59:24 +02:00
docker_cli_userns_test.go c8d: Handle userns properly 2023-09-11 16:39:29 +02:00
docker_cli_v2_only_test.go refactor: move from io/ioutil to io and os package 2021-08-27 14:56:57 +08:00
docker_cli_volume_test.go integration-cli: DockerCLIVolumeSuite: replace dockerCmd 2023-10-19 17:58:57 +02:00
docker_deprecated_api_v124_test.go integration(-cli): remove skips for old daemon versions (<20.10) 2023-12-05 01:03:50 +01:00
docker_deprecated_api_v124_unix_test.go integration-cli: DockerNetworkSuite: replace dockerCmd and waitRun 2023-10-19 17:20:56 +02:00
docker_hub_pull_suite_test.go Wire up tests to support otel tracing 2023-09-07 18:38:22 +00:00
docker_utils_test.go hack: Load special images on demand 2023-12-06 17:16:37 +01:00
events_utils_test.go integration-cli: remove deprecated dockerCmd and waitRun utilities 2023-10-19 18:02:30 +02:00
fixtures_linux_daemon_test.go frozen images: update to debian:bookworm-slim 2023-11-30 12:01:06 +01:00
requirements_test.go integration(-cli): remove skips for old daemon versions (<20.10) 2023-12-05 01:03:50 +01:00
requirements_unix_test.go c8d: fix TestRemoveContainerAfterLiveRestore 2023-09-18 15:34:20 +02:00
requirements_windows_test.go Wire up tests to support otel tracing 2023-09-07 18:38:22 +00:00
test_vars_test.go testutil/environment: remove Execution.OSType field 2023-06-26 11:18:09 +02:00
test_vars_unix_test.go remove pre-go1.17 build-tags 2023-05-19 20:38:51 +02:00
test_vars_windows_test.go remove pre-go1.17 build-tags 2023-05-19 20:38:51 +02:00
utils_test.go integration-cli: remove deprecated dockerCmd and waitRun utilities 2023-10-19 18:02:30 +02:00
utils_unix_test.go remove pre-go1.17 build-tags 2023-05-19 20:38:51 +02:00
utils_windows_test.go pkg/system: move GetLongPathName to integration-cli 2022-11-29 17:02:53 +01:00