Commit graph

387 commits

Author SHA1 Message Date
Sebastiaan van Stijn
4adc40ac40
fix duplicate words (dupwords)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-03-07 10:57:03 +01:00
Sebastiaan van Stijn
20bd690844
integration-cli: simplify test-file creation
Also fixes some potentially unclosed file-handles,
inlines some variables, and use consts for fixed
values.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-22 21:43:30 +01:00
Sebastiaan van Stijn
4f9db655ed
portmapper: move userland-proxy lookup to daemon config
When mapping a port with the userland-proxy enabled, the daemon would
perform an "exec.LookPath" for every mapped port (which, in case of
a range of ports, would be for every port in the range).

This was both inefficient (looking up the binary for each port), inconsistent
(when running in rootless-mode, the binary was looked-up once), as well as
inconvenient, because a missing binary, or a mis-configureed userland-proxy-path
would not be detected daeemon startup, and not produce an error until starting
the container;

    docker run -d -P nginx:alpine
    4f7b6589a1680f883d98d03db12203973387f9061e7a963331776170e4414194
    docker: Error response from daemon: driver failed programming external connectivity on endpoint romantic_wiles (7cfdc361821f75cbc665564cf49856cf216a5b09046d3c22d5b9988836ee088d): fork/exec docker-proxy: no such file or directory.

However, the container would still be created (but invalid);

    docker ps -a
    CONTAINER ID   IMAGE          COMMAND                  CREATED          STATUS    PORTS     NAMES
    869f41d7e94f   nginx:alpine   "/docker-entrypoint.…"   10 seconds ago   Created             romantic_wiles

This patch changes how the userland-proxy is configured;

- The path of the userland-proxy is now looked up / configured at daemon
  startup; this is similar to how the proxy is configured in rootless-mode.
- A warning is logged when failing to lookup the binary.
- If the daemon is configured with "userland-proxy" enabled, an error is
  produced, and the daemon will refuse to start.
- The "proxyPath" argument for newProxyCommand() (in libnetwork/portmapper)
  is now required to be set. It no longer looks up the executable, and
  produces an error if no path was provided. While this change was not
  required, it makes the daemon config the canonical source of truth, instead
  of logic spread accross multiplee locations.

Some of this logic is a change of behavior, but these changes were made with
the assumption that we don't want to support;

- installing the userland proxy _after_ the daemon was started
- moving the userland proxy (or installing a proxy with a higher
  preference in PATH)

With this patch:

Validating the config produces an error if the binary is not found:

    dockerd --validate
    WARN[2023-12-29T11:36:39.748699591Z] failed to lookup default userland-proxy binary       error="exec: \"docker-proxy\": executable file not found in $PATH"
    userland-proxy is enabled, but userland-proxy-path is not set

Disabling userland-proxy prints a warning, but validates as "OK":

    dockerd --userland-proxy=false --validate
    WARN[2023-12-29T11:38:30.752523879Z] ffailed to lookup default userland-proxy binary       error="exec: \"docker-proxy\": executable file not found in $PATH"
    configuration OK

Speficying a non-absolute path produces an error:

    dockerd --userland-proxy-path=docker-proxy --validate
    invalid userland-proxy-path: must be an absolute path: docker-proxy

Befor this patch, we would not validate this path, which would allow the daemon
to start, but fail to map a port;

    docker run -d -P nginx:alpine
    4f7b6589a1680f883d98d03db12203973387f9061e7a963331776170e4414194
    docker: Error response from daemon: driver failed programming external connectivity on endpoint romantic_wiles (7cfdc361821f75cbc665564cf49856cf216a5b09046d3c22d5b9988836ee088d): fork/exec docker-proxy: no such file or directory.

Specifying an invalid userland-proxy-path produces an error as well:

    dockerd --userland-proxy-path=/usr/local/bin/no-such-binary --validate
    userland-proxy-path is invalid: stat /usr/local/bin/no-such-binary: no such file or directory

    mkdir -p /usr/local/bin/not-a-file
    dockerd --userland-proxy-path=/usr/local/bin/not-a-file --validate
    userland-proxy-path is invalid: exec: "/usr/local/bin/not-a-file": is a directory

    touch /usr/local/bin/not-an-executable
    dockerd --userland-proxy-path=/usr/local/bin/not-an-executable --validate
    userland-proxy-path is invalid: exec: "/usr/local/bin/not-an-executable": permission denied

Same when using the daemon.json config-file;

    echo '{"userland-proxy-path":"no-such-binary"}' > /etc/docker/daemon.json
    dockerd --validate
    unable to configure the Docker daemon with file /etc/docker/daemon.json: merged configuration validation from file and command line flags failed: invalid userland-proxy-path: must be an absolute path: no-such-binary

    dockerd --userland-proxy-path=hello --validate
    unable to configure the Docker daemon with file /etc/docker/daemon.json: the following directives are specified both as a flag and in the configuration file: userland-proxy-path: (from flag: hello, from file: /usr/local/bin/docker-proxy)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-29 16:23:18 +01:00
Sebastiaan van Stijn
aa2fa2caa3
un-containerize TestDaemonNoSpaceLeftOnDeviceError
Commit 59b83d8aae containerized these steps,
as they didn't work well on Debian Jessie:

> Because the `mount` here will sometimes fail when run in `debian:jessie`,
> which is what the environrment hosting the test suite is running if run
> from the `Makefile`.
> Also, why the heck not containerize it, all the things.

Follow-up commits, such as 228d74842f, and
1c5806cf57 updated the Debian distro, but
also updated this comment, losing the original context (the issue was
(originally) related to Debian Jessie).

This patch changes the test back to not use containers, which seems to
work fine (at least "it worked on my machine").

    make TEST_IGNORE_CGROUP_CHECK=1 TEST_FILTER=TestDaemonNoSpaceLeftOnDeviceError DOCKER_GRAPHDRIVER=overlay2 test-integration

    === RUN   TestDockerDaemonSuite/TestDaemonNoSpaceLeftOnDeviceError
        check_test.go:589: [df36ad96a412b] daemon is not started
    --- PASS: TestDockerDaemonSuite (5.12s)
        --- PASS: TestDockerDaemonSuite/TestDaemonNoSpaceLeftOnDeviceError (5.12s)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-04 18:51:15 +01:00
Sebastiaan van Stijn
3bfb6a9420
frozen images: update to debian:bookworm-slim
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-11-30 12:01:06 +01:00
Sebastiaan van Stijn
e72c4818c4
Dockerfile: update to Debian "bookworm" (current stable)
Also switch yamllint to be installed from debian's packages, which are
currently at v1.29.0.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-11-30 11:50:34 +01:00
Sebastiaan van Stijn
c3eed9fa3e
integration-cli: fix TestDaemonICC tests for newer iptables versions
Debian Woodworm ships with a newer version of iptables, which caused two
tests to fail:

    === FAIL: amd64.integration-cli TestDockerDaemonSuite/TestDaemonICCLinkExpose (1.18s)
    docker_cli_daemon_test.go:841: assertion failed: false (matched bool) != true (true bool): iptables output should have contained "DROP.*all.*ext-bridge6.*ext-bridge6", but was "Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)\n pkts bytes target prot opt in out source destination \n 0 0 DOCKER-USER 0 -- * * 0.0.0.0/0 0.0.0.0/0 \n 0 0 DOCKER-ISOLATION-STAGE-1 0 -- * * 0.0.0.0/0 0.0.0.0/0 \n 0 0 ACCEPT 0 -- * ext-bridge6 0.0.0.0/0 0.0.0.0/0 ctstate RELATED,ESTABLISHED\n 0 0 DOCKER 0 -- * ext-bridge6 0.0.0.0/0 0.0.0.0/0 \n 0 0 ACCEPT 0 -- ext-bridge6 !ext-bridge6 0.0.0.0/0 0.0.0.0/0 \n 0 0 DROP 0 -- ext-bridge6 ext-bridge6 0.0.0.0/0 0.0.0.0/0 \n"
    --- FAIL: TestDockerDaemonSuite/TestDaemonICCLinkExpose (1.18s)

    === FAIL: amd64.integration-cli TestDockerDaemonSuite/TestDaemonICCPing (1.19s)
    docker_cli_daemon_test.go:803: assertion failed: false (matched bool) != true (true bool): iptables output should have contained "DROP.*all.*ext-bridge5.*ext-bridge5", but was "Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)\n pkts bytes target prot opt in out source destination \n 0 0 DOCKER-USER 0 -- * * 0.0.0.0/0 0.0.0.0/0 \n 0 0 DOCKER-ISOLATION-STAGE-1 0 -- * * 0.0.0.0/0 0.0.0.0/0 \n 0 0 ACCEPT 0 -- * ext-bridge5 0.0.0.0/0 0.0.0.0/0 ctstate RELATED,ESTABLISHED\n 0 0 DOCKER 0 -- * ext-bridge5 0.0.0.0/0 0.0.0.0/0 \n 0 0 ACCEPT 0 -- ext-bridge5 !ext-bridge5 0.0.0.0/0 0.0.0.0/0 \n 0 0 DROP 0 -- ext-bridge5 ext-bridge5 0.0.0.0/0 0.0.0.0/0 \n"
    --- FAIL: TestDockerDaemonSuite/TestDaemonICCPing (1.19s)

Both the `TestDaemonICCPing`, and `TestDaemonICCLinkExpose` test were introduced
in dd0666e64f. These tests called `iptables` with
the `-n` (`--numeric`) option, which prevents it from doing a reverse-DNS lookup
as an optimization.

However, the `-n` option did not have an effect to the `prot` column before
commit [da8ecc62dd765b15df84c3aa6b83dcb7a81d4ffa] (iptables < v1.8.9 or v1.8.8).
Newer versions, such as the iptables version shipping with Debian Woodworm do,
so we need to update the expected output for this version.

This patch removes the `-n` option, to keep the test more portable, also when
run non-containerized, and removes the use of regular expressions to check the
result, as these regular expressions were quite permissive (using `.*` wild-
card matching). Instead, we're getting the

With this change;

make DOCKER_GRAPHDRIVER=vfs TEST_FILTER=TestDaemonICC TEST_IGNORE_CGROUP_CHECK=1 test-integration
...
--- PASS: TestDockerDaemonSuite (139.11s)
--- PASS: TestDockerDaemonSuite/TestDaemonICCLinkExpose (54.62s)
--- PASS: TestDockerDaemonSuite/TestDaemonICCPing (84.48s)

[da8ecc62dd765b15df84c3aa6b83dcb7a81d4ffa]: https://git.netfilter.org/iptables/commit/?id=da8ecc62dd765b15df84c3aa6b83dcb7a81d4ffa

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-11-28 18:58:03 +01:00
Sebastiaan van Stijn
981a878cf5
vendor: github.com/docker/go-connections fa09c952e3ea (v0.5.0-dev)
0b8c1f4e07...fa09c952e3

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-11-10 23:23:37 +01:00
Sebastiaan van Stijn
d1108b8a06
integration-cli: DockerDaemonSuite: replace dockerCmd
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-19 17:59:10 +02:00
Djordje Lukic
23f2e0fea7
Skip tests that check max concurrency
These are not yet implemented with containerd snapshotters. We skip them
now because implementing this is not trivial with containerd.

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-10-10 11:36:36 +02:00
Paweł Gronowski
1f6bacc486
TestDaemonNoSpaceLeftOnDeviceError: Adjust to snapshotters
- Pass empty containerd socket which forces the daemon to create a new
  supervised containerd. Otherwise a global containerd daemon will be
  used and the pulled image data will be stored in its data directory,
  instead of the the newly specified `data-root` that has a limited
  storage capacity.

- Don't try to use `vfs` snapshotter, instead use `native` which is
  containerd's equivalent for `vfs`.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-10-04 15:59:30 +02:00
Djordje Lukic
6c563cda82
c8d: fix TestRemoveContainerAfterLiveRestore
There is no need to pass the storage driver to the daemon the test
starts

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-09-18 15:34:20 +02:00
Brian Goff
e8dc902781 Wire up tests to support otel tracing
Integration tests will now configure clients to propagate traces as well
as create spans for all tests.

Some extra changes were needed (or desired for trace propagation) in the
test helpers to pass through tracing spans via context.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2023-09-07 18:38:22 +00:00
Sebastiaan van Stijn
225e2562c9
integration-cli: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:08:27 +02:00
Cory Snider
0f6eeecac0 daemon: consolidate runtimes config validation
The daemon has made a habit of mutating the DefaultRuntime and Runtimes
values in the Config struct to merge defaults. This would be fine if it
was a part of the regular configuration loading and merging process,
as is done with other config options. The trouble is it does so in
surprising places, such as in functions with 'verify' or 'validate' in
their name. It has been necessary in order to validate that the user has
not defined a custom runtime named "runc" which would shadow the
built-in runtime of the same name. Other daemon code depends on the
runtime named "runc" always being defined in the config, but merging it
with the user config at the same time as the other defaults are merged
would trip the validation. The root of the issue is that the daemon has
used the same config values for both validating the daemon runtime
configuration as supplied by the user and for keeping track of which
runtimes have been set up by the daemon. Now that a completely separate
value is used for the latter purpose, surprising contortions are no
longer required to make the validation work as intended.

Consolidate the validation of the runtimes config and merging of the
built-in runtimes into the daemon.setupRuntimes() function. Set the
result of merging the built-in runtimes config and default default
runtime on the returned runtimes struct, without back-propagating it
onto the config.Config argument.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-06-01 14:45:25 -04:00
Sebastiaan van Stijn
ab35df454d
remove pre-go1.17 build-tags
Removed pre-go1.17 build-tags with go fix;

    go mod init
    go fix -mod=readonly ./...
    rm go.mod

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-19 20:38:51 +02:00
Sebastiaan van Stijn
dc11d2a2d8
remove deprecated devicemapper storage-driver
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-20 23:51:54 +02:00
Sebastiaan van Stijn
018347f802
integration-cli: remove waitInspectWithArgs()
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-29 18:26:12 +01:00
Sebastiaan van Stijn
61c434c030
integration-cli: remove inspectMountPointJSON()
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-29 18:26:00 +01:00
Sebastiaan van Stijn
1981706196
daemon: remove migrateTrustKeyID()
The migration code is in the 22.06 branch, and if we don't migrate
the only side-effect is the daemon's ID being regenerated (as a
UUID).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-28 20:28:55 +01:00
Sebastiaan van Stijn
dc0c2340b8
integration-cli: fix empty-lines (revive)
integration-cli/docker_cli_pull_test.go:55:69: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_exec_test.go:46:64: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_service_health_test.go:86:65: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_api_images_test.go:128:66: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_api_swarm_node_test.go:79:69: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_health_test.go:51:57: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_health_test.go:159:73: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_swarm_unix_test.go:60:67: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_api_inspect_test.go:30:33: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_api_build_test.go:429:71: empty-lines: extra empty line at the start of a block (revive)
    integration-cli/docker_cli_attach_unix_test.go:19:78: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_api_build_test.go:470:70: empty-lines: extra empty line at the start of a block (revive)
    integration-cli/docker_cli_history_test.go:29:64: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_links_test.go:93:86: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_create_test.go:33:61: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_links_test.go:145:78: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_create_test.go:114:70: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_api_attach_test.go:226:153: empty-lines: extra empty line at the start of a block (revive)
    integration-cli/docker_cli_by_digest_test.go:239:71: empty-lines: extra empty line at the start of a block (revive)
    integration-cli/docker_cli_create_test.go:135:49: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_create_test.go:143:75: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_create_test.go:181:71: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_inspect_test.go:72:65: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_api_swarm_service_test.go:98:77: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_api_swarm_service_test.go:144:69: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_rmi_test.go:63:2: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_api_swarm_service_test.go:199:79: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_rmi_test.go:69:2: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_api_swarm_service_test.go:300:75: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_prune_unix_test.go:35:25: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_events_unix_test.go:393:60: empty-lines: extra empty line at the start of a block (revive)
    integration-cli/docker_cli_events_unix_test.go:441:71: empty-lines: extra empty line at the start of a block (revive)
    integration-cli/docker_cli_ps_test.go:33:67: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_ps_test.go:559:67: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_events_test.go:117:75: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_api_containers_test.go:547:74: empty-lines: extra empty line at the start of a block (revive)
    integration-cli/docker_api_containers_test.go:1054:84: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_api_containers_test.go:1076:87: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_api_containers_test.go:1232:72: empty-lines: extra empty line at the start of a block (revive)
    integration-cli/docker_api_containers_test.go:1801:21: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_network_unix_test.go:58:95: empty-lines: extra empty line at the start of a block (revive)
    integration-cli/docker_cli_network_unix_test.go:750:75: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_network_unix_test.go:765:76: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_swarm_test.go:617:100: empty-lines: extra empty line at the start of a block (revive)
    integration-cli/docker_cli_swarm_test.go:892:72: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_daemon_test.go:119:74: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_daemon_test.go:981:68: empty-lines: extra empty line at the start of a block (revive)
    integration-cli/docker_cli_daemon_test.go:1951:87: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_run_test.go:83:66: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_run_test.go:357:72: empty-lines: extra empty line at the start of a block (revive)
    integration-cli/docker_cli_build_test.go:89:83: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:114:83: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:183:80: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:290:71: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:314:65: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:331:67: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:366:76: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:403:67: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:648:67: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:708:72: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:938:66: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:1018:72: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:1097:2: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:1182:62: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:1244:66: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:1524:69: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:1546:80: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:1716:70: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:1730:65: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:2162:74: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:2270:71: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:2288:70: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:3206:65: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:3392:66: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:3433:72: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:3678:76: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:3732:67: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:3759:69: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:3802:61: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:3898:66: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:4107:9: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:4791:74: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:4821:73: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:4854:70: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:5341:74: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_cli_build_test.go:5593:81: empty-lines: extra empty line at the end of a block (revive)
    integration-cli/docker_api_containers_test.go:2145:11: empty-lines: extra empty line at the start of a block (revive)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-28 01:58:49 +02:00
Sebastiaan van Stijn
306b8c89e8
linting: host:port in url should be constructed with net.JoinHostPort
integration-cli/docker_cli_daemon_test.go:545:54: host:port in url should be constructed with net.JoinHostPort and not directly with fmt.Sprintf (nosprintfhostport)
            cmdArgs = append(cmdArgs, "--tls=false", "--host", fmt.Sprintf("tcp://%s:%s", l.daemon, l.port))
                                                               ^
    opts/hosts_test.go:35:31: host:port in url should be constructed with net.JoinHostPort and not directly with fmt.Sprintf (nosprintfhostport)
            "tcp://:5555":              fmt.Sprintf("tcp://%s:5555", DefaultHTTPHost),
                                        ^
    opts/hosts_test.go:91:30: host:port in url should be constructed with net.JoinHostPort and not directly with fmt.Sprintf (nosprintfhostport)
            ":5555":                   fmt.Sprintf("tcp://%s:5555", DefaultHTTPHost),
                                       ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-04 15:36:53 +02:00
Cory Snider
547da0d575 daemon: support other containerd runtimes (MVP)
Contrary to popular belief, the OCI Runtime specification does not
specify the command-line API for runtimes. Looking at containerd's
architecture from the lens of the OCI Runtime spec, the _shim_ is the
OCI Runtime and runC is "just" an implementation detail of the
io.containerd.runc.v2 runtime. When one configures a non-default runtime
in Docker, what they're really doing is instructing Docker to create
containers using the io.containerd.runc.v2 runtime with a configuration
option telling the runtime that the runC binary is at some non-default
path. Consequently, only OCI runtimes which are compatible with the
io.containerd.runc.v2 shim, such as crun, can be used in this manner.
Other OCI runtimes, including kata-containers v2, come with their own
containerd shim and are not compatible with io.containerd.runc.v2.
As Docker has not historically provided a way to select a non-default
runtime which requires its own shim, runtimes such as kata-containers v2
could not be used with Docker.

Allow other containerd shims to be used with Docker; no daemon
configuration required. If the daemon is instructed to create a
container with a runtime name which does not match any of the configured
or stock runtimes, it passes the name along to containerd verbatim. A
user can start a container with the kata-containers runtime, for
example, simply by calling

    docker run --runtime io.containerd.kata.v2

Runtime names which containerd would interpret as a path to an arbitrary
binary are disallowed. While handy for development and testing it is not
strictly necessary and would allow anyone with Engine API access to
trivially execute any binary on the host as root, so we have decided it
would be safest for our users if it was not allowed.

It is not yet possible to set an alternative containerd shim as the
default runtime; it can only be configured per-container.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-07-27 14:22:49 -04:00
Sebastiaan van Stijn
968ff5ab44
fix some minor linting issues
libnetwork/firewall_linux.go:11:21: var-declaration: should drop = nil from declaration of var ctrl; it is the zero value (revive)
        ctrl *controller = nil
                           ^
    distribution/pull_v2_test.go:213:4: S1038: should use t.Fatalf(...) instead of t.Fatal(fmt.Sprintf(...)) (gosimple)
                t.Fatal(fmt.Sprintf("expected formatPlatform to show windows platform with a version, but got '%s'", result))
                ^
    integration-cli/docker_cli_build_test.go:5951:3: S1038: should use c.Skipf(...) instead of c.Skip(fmt.Sprintf(...)) (gosimple)
            c.Skip(fmt.Sprintf("Bug fixed in 18.06 or higher.Skipping it for %s", testEnv.DaemonInfo.ServerVersion))
            ^
    integration-cli/docker_cli_daemon_test.go:240:3: S1038: should use c.Skipf(...) instead of c.Skip(fmt.Sprintf(...)) (gosimple)
            c.Skip(fmt.Sprintf("New base device size (%v) must be greater than (%s)", units.HumanSize(float64(newBasesizeBytes)), units.HumanSize(float64(oldBasesizeBytes))))
            ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-04 10:15:28 +02:00
Sebastiaan van Stijn
18529568d3
integration-cli: TestRemoveContainerAfterLiveRestore use overlay2
the overlay storage driver is deprecated, so we might as well use overlay2
for this test.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-05-24 17:28:14 +02:00
Sebastiaan van Stijn
070da63310
daemon: only create trust-key if DOCKER_ALLOW_SCHEMA1_PUSH_DONOTUSE is set
The libtrust trust-key is only used for pushing legacy image manifests;
pushing these images has been deprecated, and we only need to be able
to push them in our CI.

This patch disables generating the trust-key (and related paths) unless
the DOCKER_ALLOW_SCHEMA1_PUSH_DONOTUSE env-var is set (which we do in
our CI).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-05-04 20:18:08 +02:00
Eng Zer Jun
36049a04d2
test: use T.Setenv to set env vars in tests
This commit replaces `os.Setenv` with `t.Setenv` in tests. The
environment variable is automatically restored to its original value
when the test and all its subtests complete.

Reference: https://pkg.go.dev/testing#T.Setenv
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2022-04-23 17:44:16 +08:00
Sebastiaan van Stijn
40182954fa
daemon/logger/fluentd: add coverage for ValidateLogOpt(), parseAddress()
This exposed a bug where host is ignored on some valid cases (to be fixed).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-11 17:55:31 +02:00
Sebastiaan van Stijn
c2ca3e1118
daemon/logger/syslog: remove uses of pkg/urlutil.IsTransportURL()
pkg/urlutil (despite its poorly chosen name) is not really intended as a generic
utility to handle URLs, and should only be used by the builder to handle (remote)
build contexts.

This patch:

- removes a redundant use of urlutil.IsTransportURL(); instead adding some code
  to check if the given scheme (protocol) is supported.
- define a `defaultPort` const for the default port.
- use `net.JoinHostPort()` instead of string concatenating, to account for possible
  issues with IPv6 addresses.
- renames a variable that collided with an imported package.
- improves test coverage, and moves an integration test.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-11 17:48:40 +02:00
Sebastiaan van Stijn
d2c6602cac
integration-cli: use unique names for custom bridges
Use unique names to prevent tests from interfering, using a shorter
name, as there's a maximum length for these.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-06 18:28:19 +01:00
Sebastiaan van Stijn
65b92a730a
integration(-cli): remove discovery related tests
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-06 18:28:13 +01:00
Sebastiaan van Stijn
f415028976
integration-cli: delete existing docker0 bridge to prevent it interfering
A number of tests in the TestDockerDaemonSuite create a custom bridge as part
of the test. In some cases, an existing `docker0` bridge could interfere with
those tests. For example, the `TestDaemonICCLinkExpose` and `TestDaemonICCPing`
verify that no "icc" communication is possible, and for this create a new
bridge with a custom IP-range.

However, depending on which tests ran before the test, a default `docker0` bridge
may exist (e.g., if the`TestDefaultGatewayIPv4Implicit`) with the same IP-range,
in which iptables rules may have been set up that allow communication, and thus
make the "icc" tests fail.

This patch removes the `docker0` interface at the start of tests that create
their own bridge to prevent it from interfering.

Note that alternatively, we could update those tests to use an IP-range that's
less likely to overlap, but this may be more brittle (but could still be done
in addition to this change as a follow-up).

To verify these changes;

    make DOCKER_GRAPHDRIVER=vfs TEST_SKIP_INTEGRATION=1 TESTFLAGS='-test.run TestDockerDaemonSuite/TestDaemon(DefaultGatewayIPv4|ICC)' test-integration-cli

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-12-21 16:12:24 +01:00
Eng Zer Jun
c55a4ac779
refactor: move from io/ioutil to io and os package
The io/ioutil package has been deprecated in Go 1.16. This commit
replaces the existing io/ioutil functions with their new definitions in
io and os packages.

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2021-08-27 14:56:57 +08:00
Sebastiaan van Stijn
686be57d0a
Update to Go 1.17.0, and gofmt with Go 1.17
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-08-24 23:33:27 +02:00
Sebastiaan van Stijn
f68260ba22
Dockerfile: frozen images: update to bullseye, remove buildpack-dep
Update the frozen images to also be based on Debian bullseye. Using the "slim"
variant (which looks to have all we're currently using),  and remove the
buildpack-dep frozen image.

The buildpack-dep image is quite large, and it looks like we only use it to
compile some C binaries, which should work fine on a regular debian image;

    docker build -t debian:bullseye-slim-gcc -<<EOF
    FROM debian:bullseye-slim
    RUN apt-get update && apt-get install -y gcc libc6-dev --no-install-recommends
    EOF

    docker image ls

    REPOSITORY        TAG                 IMAGE ID       CREATED          SIZE
    debian            bullseye-slim-gcc   1851750242af   About a minute ago   255MB
    buildpack-deps    bullseye            fe8fece98de2   2 days ago           834MB

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-08-20 18:23:06 +02:00
Brian Goff
4b981436fe Fixup libnetwork lint errors
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2021-06-01 23:48:32 +00:00
Brian Goff
a0a473125b Fix libnetwork imports
After moving libnetwork to this repo, we need to update all the import
paths for libnetwork to point to docker/docker/libnetwork instead of
docker/libnetwork.
This change implements that.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2021-06-01 21:51:23 +00:00
Eric Mountain
1c5806cf57
Move use of debian:buster frozen image to debian:bullseye
Signed-off-by: Eric Mountain <eric.mountain@datadoghq.com>
2020-12-19 17:59:36 +01:00
Arko Dasgupta
dfc2d770e4 Update iptable.Exists API in integration-cli
Signed-off-by: Arko Dasgupta <arko.dasgupta@docker.com>
2020-11-03 15:03:41 -08:00
Brian Goff
5f5285a6e2 Sterner warnings for unathenticated tcp
People keep doing this and getting pwned because they accidentally left
it exposed to the internet.

The warning about doing this has been there forever.
This introduces a sleep after warning.
To disable the extra sleep users must explicitly specify `--tls=false`
or `--tlsverify=false`

Warning also specifies this sleep will be removed in the next release
where the flag will be required if running unauthenticated.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2020-09-25 00:21:54 +00:00
wanghuaiqing
0f0c1c04c8 fix TestDaemonNoSpaceLeftOnDeviceError for mips64le
Signed-off-by: wanghuaiqing <wanghuaiqing@loongson.cn>
2020-08-05 11:31:51 +08:00
wanghuaiqing
ad30ef9113 Supplement the omission of pr#41145
Signed-off-by: wanghuaiqing <wanghuaiqing@loongson.cn>
2020-08-03 14:12:55 +08:00
wanghuaiqing
228d74842f Upgrading the versions of images in Dockerfile.
In order to run tests at mips64el device.
Now official-images has supported the following images for mips64el.
buildpack-deps:stretch
buildpack-deps:buster
debian:stretch
debian:buster

But official-images does not support the following images for mips64el.
debian:jessie
buildpack-deps:jessie

Signed-off-by: wanghuaiqing <wanghuaiqing@loongson.cn>
2020-06-30 12:24:06 +08:00
Sebastiaan van Stijn
0fc914de5c
Merge pull request #40595 from sfzhu93/GL_2_25
[integration] add buffer to prevent goroutine leak
2020-05-08 16:00:52 +02:00
Kir Kolyshkin
39048cf656 Really switch to moby/sys/mount*
Switch to moby/sys/mount and mountinfo. Keep the pkg/mount for potential
outside users.

This commit was generated by the following bash script:

```
set -e -u -o pipefail

for file in $(git grep -l 'docker/docker/pkg/mount"' | grep -v ^pkg/mount); do
	sed -i -e 's#/docker/docker/pkg/mount"#/moby/sys/mount"#' \
		-e 's#mount\.\(GetMounts\|Mounted\|Info\|[A-Za-z]*Filter\)#mountinfo.\1#g' \
		$file
	goimports -w $file
done
```

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2020-03-20 09:46:25 -07:00
Ziheng Liu
c322af8019 test: add buffer to prevent goroutine leak
Signed-off-by: Ziheng Liu <lzhfromustc@gmail.com>
2020-02-27 21:37:26 -05:00
Sebastiaan van Stijn
9f0b3f5609
bump gotest.tools v3.0.1 for compatibility with Go 1.14
full diff: https://github.com/gotestyourself/gotest.tools/compare/v2.3.0...v3.0.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-02-11 00:06:42 +01:00
Sebastiaan van Stijn
580d3677c8
integration-cli: normalize comment formatting
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-11-27 15:36:45 +01:00
Sebastiaan van Stijn
22662cac57
testutil/daemon: wrap errors
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-11 00:38:00 +02:00
Kir Kolyshkin
a7942baf76
Use io.SeekStart instead of os.SEEK_SET
Fixes linter warnings like this one:

> distribution/pull_v2.go:229:39: SA1019: os.SEEK_SET is deprecated: Use io.SeekStart, io.SeekCurrent, and io.SeekEnd.  (staticcheck)

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2019-09-18 12:57:31 +02:00