The WASM spec tests caused a stack overflow when generated with wat2wasm
version 1.0.23, which ships with homebrew. To give feature parity,
manually download the same version from GitHub packages for Ubuntu.
Document the dependencies of the WASM spec tests option, as well.
This should speed it up quite a bit and give us more consistent
performance. (So this workflow could eventually be used for perf
regression testing as well)
It turns out that ccache caches are highly compressible (total size is
reduced by about 65%), so we should be able to increase the cache limit
for some free speedups. :^)
`wasm-as` will do some semantic analysis on the modules, which is not
something we're looking for here.
Instead, use `wat2wasm` to generate the exact module.
While these look nice, they require the discord workflow to sit around
and wait for the build-and-lint workflow to finish in order to get its
status which means we waste an extra workflow runner that does nothing
for each build-and-lint run.
With the increased volume of PRs being opened and merged lately,
multiple people have complained that the IRC is absolutely flooded with
SerenityBot posts. Remove the IRC notifications from the CI scripts, and
the Meta script that handles parsing the github actions context into
an IRC message.
Github Actions added clang-12 to their ubuntu 20.04 images, so let's
take full advantage of that. Stop relying on the llvm upstream
repository for clang-12, since it often causes jobs to fail when
their mirrors are syncing.
clang-tidy-11, libstdc++-10-dev and npm 6.14.x are also all already
pre-installed, so we don't need to waste time fetching them in the
dependency fetch step.
Note that until UBSAN is made deadly by default in LibSanitizer, UBSAN
warnings will not fail the build.
Also remove BUILD_LAGOM=ON from the NORMAL_DEBUG build as it's
unnecessary and extends the build time for no benefit when building with
sanitizers
This option replaces the use of ENABLE_ALL_THE_DEBUG_MACROS in CI runs,
and enables all debug options that might be broken by developers
unintentionally that are only used in specific debugging situations.
Options shamelessly stolen from this article on systemd's website:
https://systemd.io/TESTING_WITH_SANITIZERS/
We make ASAN more strict and tell UBSAN to print more verbose output on
failure. One of the more interesting ASAN options,
detect_stack_use_after_return, sadly causes both UBSAN and ASAN failures
in test-js.
Change run-tests-and-shutdown.sh to output a dead simple results file
that just records how many tests failed.
In the CI script, mount the _disk_image after running tests and verify
that the number of failed tests is 0. Otherwise, fail the build :^)
While we're here, bump the timeout for the tests up to 30 minutes, to
make sure that less powerful runners don't fail the job unecessarily.
Ubuntu 20.04 only ships QEMU 4.2.1, which is quite an older release.
The BuildQemu.sh script uses version 6.0.0, while the server-backports
PPA is currently shipping 5.2.1. If it turns out the server-backports
PPA is not right, then we can switch to manually building and caching
the source build.
Uncomment the tests that were disabled due to frequent freezes when
running without KVM. This also adds a new github actions group for
every single test, which makes it easier to browse test boundaries
during test runs.
Move catting the serial output log back to its own step, so that it
has higher visibility. The previous solution was also shown to not
actually cat the log in the case of a failed boot and timeout :^(.
We don't need it. A recent change to the ubuntu-20.04 image has made it
the default, causing builds to fail - we're installing and want to use
clang-12 anyway, so let's just get rid of the other installed versions.
Index page:
- Change links from "Man 1" to "Section 1"
Section index pages:
- Change title from "1" to "Section 1 - SerenityOS man pages"
- Change links from "foo" to "foo(1)"
Man pages:
- Change title from "foo" to "foo(1) - SerenityOS man pages"
This new check will ensure that all commit message lines are at most 72
characters long, as well as ensure the commit title conforms to the
"Category: Brief description of what's being changed" format.
Since some builds can take even longer than 1 hour
(for example: f033416893)
this commit just increases the timeout to be github's
own workflow timeout (effectively disabling it) and just
lets github handle it instead.
The previous filter would filter out queued checks as well, which would
result in erroneous build success notifications going out if github
started the discord notifications workflow before all other workflows.
Since our tests usually take at least 10 minutes theres no point in
checking every 10 seconds, and github was starting to complain about
the very high API usage.
It looks like some particularly long builds (After a toolchain cache
reset and on a slow worker) can take much longer than the current set
timeout of 20 minutes.
While we did allow the notify_discord job to run inside the context,
we didnt ask github to run in that context. This commit also uses the
"action-wait-for-check" sub-action to ensure the posted build results
are accurate for pull requests (since the build workflow is done in a
separate context for PRs)
If the job fails early (e.g. during linting), the 'cat debug.log' step would *also* fail.
This would confuse GA into thinking that this is the crucial thing and highlights it.
This misleads the user into looking at the wrong thing.
As documented in #5541, there are some Kernel issues that can
sporadically cause the test run to fail. Add continue on error with a
loud comment to let readers know what the issue(s) might be.
Build a new version of Serenity in CI that doesn't have all the debug
symbols on, or we'd be waiting a very long time to boot.
Insert a TestRunner entry into SystemServer.ini that will run a shell
script that runs tests in /bin and /usr/Tests and shuts down the system
in the new self-test boot mode. Also make sure enough basic services are
started in self-test such that the tests will actually run properly.
This will make it easier to keep macos tests and non-mac tests in
lockstep. Also, make sure flake8 and python are installed. This also
makes it easier to add other OS targets if we want.
* Add SERENITY_ARCH option to CMake for selecting the target toolchain
* Port all build scripts but continue to use i686
* Update GitHub Actions cache to include BuildIt.sh
A good number of contributors use macOS. However, we have a bit of
a tendency of breaking the macOS build without realising it.
Luckily, GitHub Actions does actually supply macOS environments,
so let's use it.
CMake tests usually takes ~40 seconds. However, sometimes it deadlocks
and is only timed out after the 6 hour time limit.
Let's set a 2 minute timeout to make it fail sooner. 2 minutes instead
of 1 for good measure.
There are cases where Lagom will build with GCC but not Clang.
This often goes unnoticed for a while as we don't often build with
Clang.
However, this is now important to test in CI because of the
OSS-Fuzz integration.
Note that this only tests the build, it does not run any tests.
Note that it also only builds LagomCore, Lagom and the fuzzers.
It does not build the other programs that use Lagom.
CodeQL is a static analysis technology that was purchased by GitHub
and has been tightly integrated into the platform. It's different
from most other static analysis solutions because it's based on a
database built from your codebase, and then language specific rules
can be executed against that database. The rules are fully user
extensible, and are written in a datalog/query language.
The default cpp language rules coming from CodeQL will probably find
some issues, the ability to easily write custom rules/queries will
lend it self nicely to allowing us to validate Serenity specific
semantics are followed throughout the code.
References:
- https://www.youtube.com/watch?v=AMzGorD28Ks
- https://securitylab.github.com/tools/codeql
