No more concept of "anonymous endpoints". The equivalent is now an
endpoint with no DNSNames set.
Some of the code removed by this commit was mutating user-supplied
endpoint's Aliases to add container's short ID to that list. In order to
preserve backward compatibility for the ContainerInspect endpoint, this
commit also takes care of adding that short ID (and the container
hostname) to `EndpointSettings.Aliases` before returning the response.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
They fail because exporting an image which targets a manifest list when
only one platform is available exports only the platform-specific
manifest so the ID of the loaded image is different (ID of the platform
manifest, not manifest list).
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
build --squash is an experimental feature that is not implemented in the
containerd image store.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Fixes#18864, #20648, #33561, #40901.
[This GH comment][1] makes clear network name uniqueness has never been
enforced due to the eventually consistent nature of Classic Swarm
datastores:
> there is no guaranteed way to check for duplicates across a cluster of
> docker hosts.
And this is further confirmed by other comments made by @mrjana in that
same issue, eg. [this one][2]:
> we want to adopt a schema which can pave the way in the future for a
> completely decentralized cluster of docker hosts (if scalability is
> needed).
This decentralized model is what Classic Swarm was trying to be. It's
been superseded since then by Docker Swarm, which has a centralized
control plane.
To circumvent this drawback, the `NetworkCreate` endpoint accepts a
`CheckDuplicate` flag. However it's not perfectly reliable as it won't
catch concurrent requests.
Due to this design decision, API clients like Compose have to implement
workarounds to make sure names are really unique (eg.
docker/compose#9585). And the daemon itself has seen a string of issues
due to that decision, including some that aren't fixed to this day (for
instance moby/moby#40901):
> The problem is, that if you specify a network for a container using
> the ID, it will add that network to the container but it will then
> change it to reference the network by using the name.
To summarize, this "feature" is broken, has no practical use and is a
source of pain for Docker users and API consumers. So let's just remove
it for _all_ API versions.
[1]: https://github.com/moby/moby/issues/18864#issuecomment-167201414
[2]: https://github.com/moby/moby/issues/18864#issuecomment-167202589
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
In some cases, when the daemon launched by a test panics and quits, the
cleanup code would end with an error when trying to kill it by its pid.
In those cases the whole suite will end up waiting for the daemon that
we start in .integration-daemon-start to finish and we end up waiting 2
hours for the CI to cancel after a timeout.
Using process substitution makes the integration tests quit.
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
Tests are failing with this error:
E ValueError: scheme http+docker is invalid
Which is reported in docker-py in https://github.com/docker/docker-py/issues/1478.
Not sure what changed in the tests, but could be due to updated Python
version or dependencies, but let's skip it for now.
Test failure:
___________ AttachContainerTest.test_run_container_reading_socket_ws ___________
tests/integration/api_container_test.py:1245: in test_run_container_reading_socket_ws
pty_stdout = self.client.attach_socket(container, opts, ws=True)
docker/utils/decorators.py:19: in wrapped
return f(self, resource_id, *args, **kwargs)
docker/api/container.py:98: in attach_socket
return self._attach_websocket(container, params)
docker/utils/decorators.py:19: in wrapped
return f(self, resource_id, *args, **kwargs)
docker/api/client.py:312: in _attach_websocket
return self._create_websocket_connection(full_url)
docker/api/client.py:315: in _create_websocket_connection
return websocket.create_connection(url)
/usr/local/lib/python3.7/site-packages/websocket/_core.py:601: in create_connection
websock.connect(url, **options)
/usr/local/lib/python3.7/site-packages/websocket/_core.py:245: in connect
options.pop('socket', None))
/usr/local/lib/python3.7/site-packages/websocket/_http.py:117: in connect
hostname, port, resource, is_secure = parse_url(url)
/usr/local/lib/python3.7/site-packages/websocket/_url.py:62: in parse_url
raise ValueError("scheme %s is invalid" % scheme)
E ValueError: scheme http+docker is invalid
------- generated xml file: /src/bundles/test-docker-py/junit-report.xml -------
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Also remove integration-cli: `DockerAPISuite.TestContainerAPIDeleteConflict`,
which was testing the same conditions as `TestRemoveContainerRunning` in
integration/container.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Currently, IPAM config is never validated by the API. Some checks
are done by the CLI, but they're not exhaustive. And some of these
misconfigurations might be caught early by libnetwork (ie. when the
network is created), and others only surface when connecting a container
to a misconfigured network. In both cases, the API would return a 500.
Although the `NetworkCreate` endpoint might already return warnings,
these are never displayed by the CLI. As such, it was decided during a
maintainer's call to return validation errors _for all API versions_.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The official Python images on Docker Hub switched to debian bookworm,
which is now the current stable version of Debian.
However, the location of the apt repository config file changed, which
causes the Dockerfile build to fail;
Loaded image: emptyfs:latest
Loaded image ID: sha256:0df1207206e5288f4a989a2f13d1f5b3c4e70467702c1d5d21dfc9f002b7bd43
INFO: Building docker-sdk-python3:5.0.3...
tests/Dockerfile:6
--------------------
5 | ARG APT_MIRROR
6 | >>> RUN sed -ri "s/(httpredir|deb).debian.org/${APT_MIRROR:-deb.debian.org}/g" /etc/apt/sources.list \
7 | >>> && sed -ri "s/(security).debian.org/${APT_MIRROR:-security.debian.org}/g" /etc/apt/sources.list
8 |
--------------------
ERROR: failed to solve: process "/bin/sh -c sed -ri \"s/(httpredir|deb).debian.org/${APT_MIRROR:-deb.debian.org}/g\" /etc/apt/sources.list && sed -ri \"s/(security).debian.org/${APT_MIRROR:-security.debian.org}/g\" /etc/apt/sources.list" did not complete successfully: exit code: 2
This needs to be fixed in docker-py, but in the meantime, we can pin to
the bullseye variant.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This adds a new filter argument to the volume prune endpoint "all".
When this is not set, or it is a false-y value, then only anonymous
volumes are considered for pruning.
When `all` is set to a truth-y value, you get the old behavior.
This is an API change, but I think one that is what most people would
want.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This test is verifying that the container has the right options set (through
`docker inspect`), but also checks if the cgroup-rules are set within the container
by reading `/sys/fs/cgroup/devices/devices`
Unlike cgroups v1, on cgroups v2, there is no file interface, and rules are handled
through ebpf, which means that the test will fail because this file is not present.
From the Linux documentation for cgroups v2: https://github.com/torvalds/linux/blob/v5.16/Documentation/admin-guide/cgroup-v2.rst#device-controller
> (...)
> Device controller manages access to device files. It includes both creation of
> new device files (using mknod), and access to the existing device files.
>
> Cgroup v2 device controller has no interface files and is implemented on top of
> cgroup BPF. To control access to device files, a user may create bpf programs
> of type BPF_PROG_TYPE_CGROUP_DEVICE and att>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This test was disabled in the past, but re-enabled when we upgraded
docker-py to 4.2.0.
The test looks to be still flaky though, so skipping it again:
```
[2020-02-10T23:40:44.429Z] =================================== FAILURES ===================================
[2020-02-10T23:40:44.429Z] __________________ AttachContainerTest.test_attach_no_stream ___________________
[2020-02-10T23:40:44.429Z] tests/integration/api_container_test.py:1250: in test_attach_no_stream
[2020-02-10T23:40:44.429Z] assert output == 'hello\n'.encode(encoding='ascii')
[2020-02-10T23:40:44.429Z] E AssertionError: assert b'' == b'hello\n'
[2020-02-10T23:40:44.429Z] E Right contains more items, first extra item: 104
[2020-02-10T23:40:44.429Z] E Use -v to get the full diff
[2020-02-10T23:40:44.429Z] ------- generated xml file: /src/bundles/test-docker-py/junit-report.xml -------
````
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Otherwise some tests are skipped with the default API version
used:
SKIPPED [1] tests/integration/api_service_test.py:882: API version is too low (< 1.38)
SKIPPED [1] tests/integration/api_swarm_test.py:59: API version is too low (< 1.39)
SKIPPED [1] tests/integration/api_swarm_test.py:38: API version is too low (< 1.39)
SKIPPED [1] tests/integration/api_swarm_test.py:45: API version is too low (< 1.39)
SKIPPED [1] tests/integration/api_swarm_test.py:52: API version is too low (< 1.39)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The ImageCollectionTest.test_pull_multiple test performs a `docker pull` without
a `:tag` specified) to pull all tags of the given repository (image).
After pulling the image, the image(s) pulled are checked to verify if the list
of images contains the `:latest` tag.
However, the test assumes that all tags of the image are tags for the same
version of the image (same digest), and thus a *single* image is returned, which
is not always the case.
Currently, the `hello-world:latest` and `hello-world:linux` tags point to a
different digest, therefore the `client.images.pull()` returns multiple images:
one image for digest, making the test fail:
=================================== FAILURES ===================================
____________________ ImageCollectionTest.test_pull_multiple ____________________
tests/integration/models_images_test.py:90: in test_pull_multiple
assert len(images) == 1
E AssertionError: assert 2 == 1
E + where 2 = len([<Image: 'hello-world:linux'>, <Image: 'hello-world:latest'>])
This patch temporarily skips the broken test until it is fixed upstream.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The docker-py tests were broken, because the version of
py-test that was used, used a dependency that had a new
major release with a breaking change.
Unfortunately, it was not pinned to a specific version,
so when the dependency did the release, py-test broke;
```
22:16:47 Traceback (most recent call last):
22:16:47 File "/usr/local/bin/pytest", line 10, in <module>
22:16:47 sys.exit(main())
22:16:47 File "/usr/local/lib/python3.6/site-packages/_pytest/config/__init__.py", line 61, in main
22:16:47 config = _prepareconfig(args, plugins)
22:16:47 File "/usr/local/lib/python3.6/site-packages/_pytest/config/__init__.py", line 182, in _prepareconfig
22:16:47 config = get_config()
22:16:47 File "/usr/local/lib/python3.6/site-packages/_pytest/config/__init__.py", line 156, in get_config
22:16:47 pluginmanager.import_plugin(spec)
22:16:47 File "/usr/local/lib/python3.6/site-packages/_pytest/config/__init__.py", line 530, in import_plugin
22:16:47 __import__(importspec)
22:16:47 File "/usr/local/lib/python3.6/site-packages/_pytest/tmpdir.py", line 25, in <module>
22:16:47 class TempPathFactory(object):
22:16:47 File "/usr/local/lib/python3.6/site-packages/_pytest/tmpdir.py", line 35, in TempPathFactory
22:16:47 lambda p: Path(os.path.abspath(six.text_type(p)))
22:16:47 TypeError: attrib() got an unexpected keyword argument 'convert'
```
docker-py master has a fix for this (bumping the version of
`py-test`), but it's not in a release yet, and the docker cli that's used
in our CI is pinned to 17.06, which doesn't support building from a remote
git repository from a specific git commit.
To fix the immediate situation, this patch switches the docker-py
tests to run from the master branch.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Seen failing a couple of times:
```
[2019-09-02T08:40:15.796Z] =================================== FAILURES ===================================
[2019-09-02T08:40:15.796Z] __________________ AttachContainerTest.test_attach_no_stream ___________________
[2019-09-02T08:40:15.796Z] tests/integration/api_container_test.py:1250: in test_attach_no_stream
[2019-09-02T08:40:15.796Z] assert output == 'hello\n'.encode(encoding='ascii')
[2019-09-02T08:40:15.796Z] E AssertionError: assert b'' == b'hello\n'
[2019-09-02T08:40:15.796Z] E Right contains more items, first extra item: 104
[2019-09-02T08:40:15.796Z] E Use -v to get the full diff
```
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- SC2006: use $(...) notation instead of legacy backticked `...`
- SC2086: double quote to prevent globbing and word splitting
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
and remove `PullImageTest::test_build_invalid_platform` from the list,
which was a copy/paste error in f8cde0b32d
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
When building this image docker-in-docker, the DNS in the environment
may not be usable for the build-container, causing resolution to fail:
```
02:35:31 W: Failed to fetch http://deb.debian.org/debian/dists/jessie/Release.gpg Temporary failure resolving 'deb.debian.org'
```
This patch detects if we're building from within a container, and if
so, skips creating a networking namespace for the build by using
`--network=host`.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This removes all the installation steps for docker-py from the
Dockerfile, and instead builds the upstream Dockerfile, and runs
docker-py tests in a container.
To test;
```
make test-docker-py
...
Removing bundles/
---> Making bundle: dynbinary (in bundles/dynbinary)
Building: bundles/dynbinary-daemon/dockerd-dev
Created binary: bundles/dynbinary-daemon/dockerd-dev
---> Making bundle: test-docker-py (in bundles/test-docker-py)
---> Making bundle: .integration-daemon-start (in bundles/test-docker-py)
Using test binary docker
Starting dockerd
INFO: Waiting for daemon to start...
.
INFO: Building docker-sdk-python3:3.7.0...
sha256:686428ae28479e9b5c8fdad1cadc9b7a39b462e66bd13a7e35bd79c6a152a402
INFO: Starting docker-py tests...
============================= test session starts ==============================
platform linux -- Python 3.6.8, pytest-4.1.0, py-1.8.0, pluggy-0.9.0
rootdir: /src, inifile: pytest.ini
plugins: timeout-1.3.3, cov-2.6.1
collected 359 items
tests/integration/api_build_test.py .......s....
....
```
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
These scripts explicitly use Bash, so we should be able to use
`[[` instead of `[` (which seems to be recommended).
Also added curly brackets to some bare variables, and quoted some paths.
This makes my IDE a bit more silent :-)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
