This hopefully makes the test less flakey (or removes any flake that
would be caused by the test itself).
1. Adds tail of cluster daemon logs when there is a test failure so we
can more easily see what may be happening
2. Scans the daemon logs to check if the key is rotated before
restarting the daemon. This is a little hacky but a little better
than assuming it is done after a hard-coded 3 seconds.
3. Cleans up the `node ls` check such that it uses a poll function
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit fbdc02534a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
All underlying jobs inherit from the status of all parent jobs
in the tree, not just the very parent. We need to apply the same
kind of special condition.
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
(cherry picked from commit 0252a6f475)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Starting with [6e0ed3d19c54603f0f7d628ea04b550151d8a262], the minimum
allowed size is now 300MB. Given that this is a sparse image, and
the size of the image is irrelevant to the test (we check for
limits defined through project-quotas, not the size of the
device itself), we can raise the size of this image.
[6e0ed3d19c54603f0f7d628ea04b550151d8a262]: https://git.kernel.org/pub/scm/fs/xfs/xfsprogs-dev.git/commit/?id=6e0ed3d19c54603f0f7d628ea04b550151d8a262
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 9709b7e458)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
In some cases, when the daemon launched by a test panics and quits, the
cleanup code would end with an error when trying to kill it by its pid.
In those cases the whole suite will end up waiting for the daemon that
we start in .integration-daemon-start to finish and we end up waiting 2
hours for the CI to cancel after a timeout.
Using process substitution makes the integration tests quit.
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
(cherry picked from commit 3d8b8dc09a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The test considered `Foo/bar` to be an invalid name, with the assumption
that it was `[docker.io]/Foo/bar`. However, this was incorrect, and the
test passed because the reference parsing had a bug; if the first element
(`Foo`) is not lowercase (so not a valid namespace / "path element"), then
it *should* be considered a domain (as uppercase domain names are valid).
The reference parser did not account for this, and running the test with
a version of the parser with a fix caused the test to fail:
=== Failed
=== FAIL: client TestImageTagInvalidSourceImageName/invalidRepo/FOO/bar (0.00s)
image_tag_test.go:54: assertion failed: expected error to contain "not a valid repository/tag", got "Error response from daemon: client should not have made an API call"
Error response from daemon: client should not have made an API call
=== FAIL: client TestImageTagInvalidSourceImageName (0.00s)
This patch removes the faulty test-case.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit c243efb0cd)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This test was testing the client-side validation, so might as well
move it there, and validate that the client invalidates before
trying to make an API call.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 3d3ce9812f)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 71da8c13e1)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
We no longer have any arm (not 64) CI.
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
(cherry picked from commit 159b168eea)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
full diff: d3e6c1360f...435cb77e36
The 0.11 branch of buildkit defaults to go1.19 (EOL), and
Alpine 3.17 (EOL).
We already set GO_VERSION to override the go version to
use go1.20, but the Dockerfile also has a ALPINE_VERSION
build-arg, so let's override that as well to prevent the
build from failing:
Dockerfile:39
--------------------
37 |
38 | # go base image
39 | >>> FROM --platform=$BUILDPLATFORM golang:${GO_VERSION}-alpine${ALPINE_VERSION} AS golatest
40 |
41 | # git stage is used for checking out remote repository sources
--------------------
ERROR: failed to solve: golang:1.20.13-alpine3.17: docker.io/library/golang:1.20.13-alpine3.17: not found
Error: Process completed with exit code 1.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This reverts commit 19d860fa9d.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 7786f8512b)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Tests are failing with this error:
E ValueError: scheme http+docker is invalid
Which is reported in docker-py in https://github.com/docker/docker-py/issues/1478.
Not sure what changed in the tests, but could be due to updated Python
version or dependencies, but let's skip it for now.
Test failure:
___________ AttachContainerTest.test_run_container_reading_socket_ws ___________
tests/integration/api_container_test.py:1245: in test_run_container_reading_socket_ws
pty_stdout = self.client.attach_socket(container, opts, ws=True)
docker/utils/decorators.py:19: in wrapped
return f(self, resource_id, *args, **kwargs)
docker/api/container.py:98: in attach_socket
return self._attach_websocket(container, params)
docker/utils/decorators.py:19: in wrapped
return f(self, resource_id, *args, **kwargs)
docker/api/client.py:312: in _attach_websocket
return self._create_websocket_connection(full_url)
docker/api/client.py:315: in _create_websocket_connection
return websocket.create_connection(url)
/usr/local/lib/python3.7/site-packages/websocket/_core.py:601: in create_connection
websock.connect(url, **options)
/usr/local/lib/python3.7/site-packages/websocket/_core.py:245: in connect
options.pop('socket', None))
/usr/local/lib/python3.7/site-packages/websocket/_http.py:117: in connect
hostname, port, resource, is_secure = parse_url(url)
/usr/local/lib/python3.7/site-packages/websocket/_url.py:62: in parse_url
raise ValueError("scheme %s is invalid" % scheme)
E ValueError: scheme http+docker is invalid
------- generated xml file: /src/bundles/test-docker-py/junit-report.xml -------
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit f6959bc597)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Our dev-container now has buildx installed, so we're no longer
limited to 17.06.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 79a0f82ca1)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Provide more context to the steps we're doing.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 65cfcc28ab)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
On bookworm, AppArmor failed to start inside the container, which can be
seen at startup of the dev-container:
Created symlink /etc/systemd/system/systemd-firstboot.service → /dev/null.
Created symlink /etc/systemd/system/systemd-udevd.service → /dev/null.
Created symlink /etc/systemd/system/multi-user.target.wants/docker-entrypoint.service → /etc/systemd/system/docker-entrypoint.service.
hack/dind-systemd: starting /lib/systemd/systemd --show-status=false --unit=docker-entrypoint.target
systemd 252.17-1~deb12u1 running in system mode (+PAM +AUDIT +SELINUX +APPARMOR +IMA +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 -IDN +IPTC +KMOD +LIBCRYPTSETUP +LIBFDISK +PCRE2 -PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD -BPF_FRAMEWORK -XKBCOMMON +UTMP +SYSVINIT default-hierarchy=unified)
Detected virtualization docker.
Detected architecture x86-64.
modprobe@configfs.service: Deactivated successfully.
modprobe@dm_mod.service: Deactivated successfully.
modprobe@drm.service: Deactivated successfully.
modprobe@efi_pstore.service: Deactivated successfully.
modprobe@fuse.service: Deactivated successfully.
modprobe@loop.service: Deactivated successfully.
apparmor.service: Starting requested but asserts failed.
proc-sys-fs-binfmt_misc.automount: Got automount request for /proc/sys/fs/binfmt_misc, triggered by 49 (systemd-binfmt)
+ source /etc/docker-entrypoint-cmd
++ hack/make.sh dynbinary test-integration
When checking "aa-status", an error was printed that the filesystem was
not mounted:
aa-status
apparmor filesystem is not mounted.
apparmor module is loaded.
Checking if "local-fs.target" was loaded, that seemed to be the case;
systemctl status local-fs.target
● local-fs.target - Local File Systems
Loaded: loaded (/lib/systemd/system/local-fs.target; static)
Active: active since Mon 2023-11-27 10:48:38 UTC; 18s ago
Docs: man:systemd.special(7)
However, **on the host**, "/sys/kernel/security" has a mount, which was not
present inside the container:
mount | grep securityfs
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
Interestingly, on `debian:bullseye`, this was not the case either; no
`securityfs` mount was present inside the container, and apparmor actually
failed to start, but succeeded silently:
mount | grep securityfs
systemctl start apparmor
systemctl status apparmor
● apparmor.service - Load AppArmor profiles
Loaded: loaded (/lib/systemd/system/apparmor.service; enabled; vendor preset: enabled)
Active: active (exited) since Mon 2023-11-27 11:59:09 UTC; 44s ago
Docs: man:apparmor(7)
https://gitlab.com/apparmor/apparmor/wikis/home/
Process: 43 ExecStart=/lib/apparmor/apparmor.systemd reload (code=exited, status=0/SUCCESS)
Main PID: 43 (code=exited, status=0/SUCCESS)
CPU: 10ms
Nov 27 11:59:09 9519f89cade1 apparmor.systemd[43]: Not starting AppArmor in container
Same, using the `/etc/init.d/apparmor` script:
/etc/init.d/apparmor start
Starting apparmor (via systemctl): apparmor.service.
echo $?
0
And apparmor was not actually active:
aa-status
apparmor module is loaded.
apparmor filesystem is not mounted.
aa-enabled
Maybe - policy interface not available.
After further investigating, I found that the non-systemd dind script
had a mount for AppArmor, which was added in 31638ab2ad
The systemd variant was missing this mount, which may have gone unnoticed
because `debian:bullseye` was silently ignoring this when starting the
apparmor service.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit cfb8ca520a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- full diff: https://github.com/containerd/containerd/compare/v1.7.11...v1.7.12
- release notes: https://github.com/containerd/containerd/releases/tag/v1.7.12
Welcome to the v1.7.12 release of containerd!
The twelfth patch release for containerd 1.7 contains various fixes and updates.
Notable Updates
- Fix on dialer function for Windows
- Improve `/etc/group` handling when appending groups
- Update shim pidfile permissions to 0644
- Update runc binary to v1.1.11
- Allow import and export to reference missing content
- Remove runc import
- Update Go version to 1.20.13
Deprecation Warnings
- Emit deprecation warning for `containerd.io/restart.logpath` label usage
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit cd1709b0d4)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Commit 0b1c1877c5 updated the version in
hack/dockerfile/install/rootlesskit.installer, but forgot to update the
version in Dockerfile.
Also updating both to use a tag, instead of commit. While it's good to pin by
an immutable reference, I think it's reasonably safe to use the tag, which is
easier to use, and what we do for other binaries, such as runc as well.
Full diff: https://github.com/rootless-containers/rootlesskit/compare/v1.1.0...v1.1.1
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit e27ffdab0f)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This is the eleventh patch release in the 1.1.z release branch of runc.
It primarily fixes a few issues with runc's handling of containers that
are configured to join existing user namespaces, as well as improvements
to cgroupv2 support.
- Fix several issues with userns path handling.
- Support memory.peak and memory.swap.peak in cgroups v2.
Add swapOnlyUsage in MemoryStats. This field reports swap-only usage.
For cgroupv1, Usage and Failcnt are set by subtracting memory usage
from memory+swap usage. For cgroupv2, Usage, Limit, and MaxUsage
are set.
- build(deps): bump github.com/cyphar/filepath-securejoin.
- release notes: https://github.com/opencontainers/runc/releases/tag/v1.1.11
- full diff: https://github.com/opencontainers/runc/compare/v1.1.10...v1.1.11
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 5fa4cfcabf)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- full diff: https://github.com/opencontainers/runc/compare/v1.1.9...v1.1.10
- release notes: https://github.com/opencontainers/runc/releases/tag/v1.1.10
This is the tenth (and most likely final) patch release in the 1.1.z
release branch of runc. It mainly fixes a few issues in cgroups, and a
umask-related issue in tmpcopyup.
- Add support for `hugetlb.<pagesize>.rsvd` limiting and accounting.
Fixes the issue of postgres failing when hugepage limits are set.
- Fixed permissions of a newly created directories to not depend on the value
of umask in tmpcopyup feature implementation.
- libcontainer: cgroup v1 GetStats now ignores missing `kmem.limit_in_bytes`
(fixes the compatibility with Linux kernel 6.1+).
- Fix a semi-arbitrary cgroup write bug when given a malicious hugetlb
configuration. This issue is not a security issue because it requires a
malicious config.json, which is outside of our threat model.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit fb53da508f)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit 79b467808e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
update the dependency to v0.2.4 to prevent scanners from flagging the
vulnerability (GHSA-6xv5-86q9-7xr8 / GO-2023-2048). Note that that vulnerability
only affects Windows, and is currently only used in runc/libcontainer, so should
not impact our use (as that code is Linux-only).
full diff: https://github.com/cyphar/filepath-securejoin/compare/v0.2.3...v0.2.4
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 291eaf0a77)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
When reading logs, timestamps should always be presented in UTC. Unlike
the "json-file" and other logging drivers, the "local" logging driver
was using local time.
Thanks to Roman Valov for reporting this issue, and locating the bug.
Before this change:
echo $TZ
Europe/Amsterdam
docker run -d --log-driver=local nginx:alpine
fc166c6b2c35c871a13247dddd95de94f5796459e2130553eee91cac82766af3
docker logs --timestamps fc166c6b2c35c871a13247dddd95de94f5796459e2130553eee91cac82766af3
2023-12-08T18:16:56.291023422+01:00 /docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
2023-12-08T18:16:56.291056463+01:00 /docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
2023-12-08T18:16:56.291890130+01:00 /docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
...
With this patch:
echo $TZ
Europe/Amsterdam
docker run -d --log-driver=local nginx:alpine
14e780cce4c827ce7861d7bc3ccf28b21f6e460b9bfde5cd39effaa73a42b4d5
docker logs --timestamps 14e780cce4c827ce7861d7bc3ccf28b21f6e460b9bfde5cd39effaa73a42b4d5
2023-12-08T17:18:46.635967625Z /docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
2023-12-08T17:18:46.635989792Z /docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
2023-12-08T17:18:46.636897417Z /docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
...
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit afe281964d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
docs.docker.com switched from Jekyll to Hugo, which uses "aliases"
instead of "redirect_from".
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit c1a289e05a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
