By using RefPtrs to handle interfaces, the IDL parser could store cyclic
references to interfaces that import each other. One main example is the
"EventTarget.idl" and the "AbortSignal.idl" files, which both reference
each other. This caused huge amounts of memory not to be freed on exit.
To fix this, the parsed IDL interfaces are now stored in a HashTable of
NonnullOwnPtr<Interface>, which serves as the sole reference for every
parsed interface. All other usages of the Interface are changed to use
references instead of RefPtrs, or occasionally as raw pointers where
references don't fit inside the data structures.
This new HashTable is static, and as such will automatically be freed
prior to exiting the generator. This ensures that the code generator
properly cleans up after itself.
With this change, The IDL code generators can properly run on Lagom when
compiled with the -DENABLE_ADDRESS_SANITIZER=ON flag, and gets compiled
properly on the CI :^)
This commit bumps the required QEMU version to 6.2 and updates the
version checking logic in Meta/run.sh to support checking against
major and minor version numbers instead of checking against the major
version only
This commit upgrades Github Actions workers to ubuntu-22.04
As part of that change, we (currently) no longer need the backports
nor toolchain-r/test PPAs, because ubuntu-22.04 include
recent-enough version of QEMU and gcc
By default we enable the Kernel Undefined Behavior Sanitizer, which
checks for undefined behavior at runtime. However, sometimes a developer
might want to turn that off, so now there is a easy way to do that.
This moves all code comprehension-related code to a new library,
LibCodeComprehension.
This also moves some types related to code comprehension tasks (such as
autocomplete, find declaration) out of LibGUI and into
LibCodeComprehension.
This keeps users from leaking their host environment variables (CFLAGS,
etc.) into Ports, and it keeps us from leaking Port-specific settings
into their dependencies.
This shouldn't cause any breaking changes, so a toolchain rebuild is not
required.
As per Hendiadyoin's request, math errno is disabled by default, which
should enable some extra compiler optimizations in LibGL and LibSoftGPU
code that uses math functions heavily.
Co-Authored-By: Ali Mohammad Pur <mpfard@serenityos.org>
Previously, `serenity.sh rebuild-toolchain x86_64 clang` would simply
start building GCC instead and call it a day. With this change, if a
toolchain is passed as an argument, it is validated if it is either
"GNU" or "Clang" and exits with an error otherwise.
The `TOOLCHAIN` value defaults to the `SERENITY_TOOLCHAIN` environment
variable if not provided explicitly.
Instead of first doubling the required size for the determined inode
count and then _also_ tripling the sum of that and the determined disk
size, let's be a bit more reasonable and just double the sum of inode
count * size and disk size.
This results in a 1.4GB _disk_image, instead of the 2GB from before
(for < 800MB worth of files).
By providing SERENITY_DISK_SIZE_BYTES as an environment variable, the
calculation of default value considered suitable for the size of files
and number of inodes that will be included can be sidestepped.
Also removes mrsh from the list of ports missing descriptions. I tried
to be descriptive about the patches, but as I picked this port up from
someone else, I'm not 100% sure how to best explain the patches.
This let us test the VMWare SVGA adapter easily. We already use the std
vga (which is compatible with bochs-display that only lacks VGA support)
on the i440FX QEMU machine so we keep testing it there too, and on the
Q35 machine we use a bochs-display device as secondary display.
Add a job to the Azure pipelines to run tests with coverage enabled, and
aggregate the test results in a folder of html pages showing the
coverage results overall, and per-file.
Future work is needed to take the published pipeline artifact for the
coverage results and display them somewhere interesting.
The analyze-qemu-coverage.sh script cracks open the _disk_image for the
given SERENITY_ARCH and SERENITY_TOOLCHAIN and extracts llvm profile
data into a local directory owned by the current user. It then calls a
coverage artifact script from llvm to create a nice html report for all
the source files referenced by the profile data files.
We currently grab a script from llvm via wget. In the future a custom
script to call llvm-cov and llvm-profdata should probably be used.
This option sets -fprofile-instr-generate -fcoverage-mapping for Clang
builds only on almost all of Userland. Loader and LibTimeZone are
exempt. This can be used for generating code coverage reports, or even
PGO in the future.
Before, we wouldn't enable virtualization on Windows anymore unless
SERENITY_VIRTUALIZATION_SUPPORT was set explicitly. As far as we know,
there's no automatic way of detecting whether WHPX is enabled or not. So
we'll just enable virtualization on Windows by default, and if that
doesn't work the user can still disable it manually with
SERENITY_VIRTUALIZATION_SUPPORT=0.
Various Clang binaries are now considered when choosing the compiler for
Lagom.
The selection precedence is as follows:
1. Use the compiler set via CC/CXX if it's a supported version
2. Use newest available GCC if it's supported
3. Use newest available Clang if it's supported
Note that Apple Clang is still not supported, as its versioning scheme
and the fact that it masquerades as both GCC and Clang would complicate
this logic even more.
Fixes#12253
The LLVM patch has been broken up into smaller commits and moved to a
separate directory. CI should look at this new location to determine if
the toolchain needs to be rebuilt.
Besides a version bump, the following changes have been made to our
toolchain infrastructure:
- LLVM/Clang is now built with -march=native if the host compiler
supports it. An exception to this is CI, as the toolchain cache is
shared among many different machines there.
- The LLVM tarball is not re-extracted if the hash of the applied
patches doesn't differ.
- The patches have been split up into atomic chunks.
- Port-specific patches have been integrated into the main patches,
which will aid in the work towards self-hosting.
- <sysroot>/usr/local/lib is now appended to the linker's search path by
default.
- --pack-dyn-relocs=relr is appended to the linker command line by
default, meaning ports take advantage of RELR relocations without any
patches or additional compiler flags.
The formatting of LLVM port's package.sh has been bothering me, so I
also indented the arguments to the CMake invocation.
Previously, we were sending Buffers to the server whenever we had new
audio data for it. This meant that for every audio enqueue action, we
needed to create a new shared memory anonymous buffer, send that
buffer's file descriptor over IPC (+recfd on the other side) and then
map the buffer into the audio server's memory to be able to play it.
This was fine for sending large chunks of audio data, like when playing
existing audio files. However, in the future we want to move to
real-time audio in some applications like Piano. This means that the
size of buffers that are sent need to be very small, as just the size of
a buffer itself is part of the audio latency. If we were to try
real-time audio with the existing system, we would run into problems
really quickly. Dealing with a continuous stream of new anonymous files
like the current audio system is rather expensive, as we need Kernel
help in multiple places. Additionally, every enqueue incurs an IPC call,
which are not optimized for >1000 calls/second (which would be needed
for real-time audio with buffer sizes of ~40 samples). So a fundamental
change in how we handle audio sending in userspace is necessary.
This commit moves the audio sending system onto a shared single producer
circular queue (SSPCQ) (introduced with one of the previous commits).
This queue is intended to live in shared memory and be accessed by
multiple processes at the same time. It was specifically written to
support the audio sending case, so e.g. it only supports a single
producer (the audio client). Now, audio sending follows these general
steps:
- The audio client connects to the audio server.
- The audio client creates a SSPCQ in shared memory.
- The audio client sends the SSPCQ's file descriptor to the audio server
with the set_buffer() IPC call.
- The audio server receives the SSPCQ and maps it.
- The audio client signals start of playback with start_playback().
- At the same time:
- The audio client writes its audio data into the shared-memory queue.
- The audio server reads audio data from the shared-memory queue(s).
Both sides have additional before-queue/after-queue buffers, depending
on the exact application.
- Pausing playback is just an IPC call, nothing happens to the buffer
except that the server stops reading from it until playback is
resumed.
- Muting has nothing to do with whether audio data is read or not.
- When the connection closes, the queues are unmapped on both sides.
This should already improve audio playback performance in a bunch of
places.
Implementation & commit notes:
- Audio loaders don't create LegacyBuffers anymore. LegacyBuffer is kept
for WavLoader, see previous commit message.
- Most intra-process audio data passing is done with FixedArray<Sample>
or Vector<Sample>.
- Improvements to most audio-enqueuing applications. (If necessary I can
try to extract some of the aplay improvements.)
- New APIs on LibAudio/ClientConnection which allows non-realtime
applications to enqueue audio in big chunks like before.
- Removal of status APIs from the audio server connection for
information that can be directly obtained from the shared queue.
- Split the pause playback API into two APIs with more intuitive names.
I know this is a large commit, and you can kinda tell from the commit
message. It's basically impossible to break this up without hacks, so
please forgive me. These are some of the best changes to the audio
subsystem and I hope that that makes up for this :yaktangle: commit.
:yakring:
This new class with an admittedly long OOP-y name provides a circular
queue in shared memory. The queue is a lock-free synchronous queue
implemented with atomics, and its implementation is significantly
simplified by only accounting for one producer (and multiple consumers).
It is intended to be used as a producer-consumer communication
datastructure across processes. The original motivation behind this
class is efficient short-period transfer of audio data in userspace.
This class includes formal proofs of several correctness properties of
the main queue operations `enqueue` and `dequeue`. These proofs are not
100% complete in their existing form as the invariants they depend on
are "handwaved". This seems fine to me right now, as any proof is better
than no proof :^). Anyways, the proofs should build confidence that the
implemented algorithms, which are only roughly based on existing work,
operate correctly in even the worst-case concurrency scenarios.
Each LibGL test can now be tested against a reference QOI image.
Initially, these images can be generated by setting `SAVE_OUTPUT` to
`true`, which will save a bunch of QOI images to `/home/anon`.
Similar reasoning to making Core::Stream::read() return Bytes, except
that every user of read_line() creates a StringView from the result, so
let's just return one right away.
A mistake I've repeatedly made is along these lines:
```c++
auto nread = TRY(source_file->read(buffer));
TRY(destination_file->write(buffer));
```
It's a little clunky to have to create a Bytes or StringView from the
buffer's data pointer and the nread, and easy to forget and just use
the buffer. So, this patch changes the read() function to return a
Bytes of the data that were just read.
The other read_foo() methods will be modified in the same way in
subsequent commits.
Fixes#13687
