* Matches how the loader is organized
* `compress_VP8L_image_data()` will grow longer when we add actual
compression
* Maybe someone wants to write a lossy compressor one day
No behavior change.
The new baked image is a Prekernel and a Kernel baked together now, so
essentially we no longer need to pass the Prekernel as -kernel and the
actual kernel image as -initrd to QEMU, leaving the option to pass an
actual initrd or initramfs module later on with multiboot.
Nobody uses this functionality. I used this code on my old 2007 ICH7
test machine about a year ago, but bare metal is a small aspect of the
project, so it's safe to assume that nobody really tests this piece of
code.
Therefore, let's drop this for good and focus on more modern hardware.
Now both /bin/zcat and /bin/gunzip are symlinks to /bin/gzip, and we
essentially running it in decompression mode through these symlinks.
This ensures we don't maintain 2 versions of code to decompress Gzipped
data anymore, and handle the use case of gzipped-streaming input only
once in the codebase.
This is a more general and robust replacement of the LibJSGCVerifier.
We want to add more generic static analysis, and this new plugin will
be built in a way that integrates into the rest of the system.
🤽 - U+1F93D PERSON PLAYING WATER POLO
🤽♂️ - U+1F93D U+200D U+2642 MAN PLAYING WATER POLO
🤽♀️ - U+1F93D U+200D U+2640 WOMAN PLAYING WATER POLO
🦨 - U+1F9A8 SKUNK
The high-level design is that we have a static method on WebPWriter that
returns an AnimationWriter object. AnimationWriter has a virtual method
for writing individual frames. This allows streaming animations to disk,
without having to buffer up the entire animation in memory first.
The semantics of this function, add_frame(), are that data is flushed
to disk every time the function is called, so that no explicit `close()`
method is needed.
For some formats that store animation length at the start of the file,
including WebP, this means that this needs to write to a SeekableStream,
so that add_frame() can seek to the start and update the size when a
frame is written.
This design should work for GIF and APNG writing as well. We can move
AnimationWriter to a new header if we add writers for these.
Currently, `animation` can read any animated image format we can read
(apng, gif, webp) and convert it to an animated webp file.
The written animated webp file is not compressed whatsoever, so this
creates large output files at the moment.
Xcode clang doesn't understand the -std=c++23 spelling yet, and this
is what CMake's `set(CMAKE_CXX_STANDARD 23)` translates to too.
Unbreaks building with Xcode clang on macOS.
An AudioNode is the fundamental building block used in 'Audio
Contexts'. In our immediate case, the audio node we are working towards
implementing is an oscillating source node.
Previously the GML compiler did not support object properties such as
`content_widget: @GUI::Widget{}` for GUI::ScrollableContainerWidget;
this commit adds support for such properties by simply calling
`set_<key>(<TProperty>&)` on the object.
This commit also removes the previous hack where
ScrollableContainerWidget was special-cased to have its singular child
used as the content widget; the only GML file using this behaviour was
also changed to be in line with 'proper' GML as handled by the GML
Playground.
This makes it possible to use externally defined toplevel widgets that
have no C++ header defining them.
Note that this only allows widget-native properties on the object, as
the actual original definition is not available.
We already have required this version for quite a while for Lagom,
Ladybird and Serenity. Now that we require it in all of our CMakeLists,
let's scrub for better ways of writing things.
This allows main UI processes created while there is a currently
running one to request a new tab or a new window with the initial urls
provided on the command line. This matches (almost) the behavior of
Chromium and Firefox.
Add a new IPC protocol between two UI processes. The main UI process
will create an IPC server socket, while secondary UI processes will
connect to that socket and send over the URLs and action it wants the
main process to take.
This was resulting in a whole lot of rebuilding whenever a new IDL
interface was added.
Instead, just directly include the prototype in every C++ file which
needs it. While we only really need a forward declaration in each cpp
file; including the full prototype header (which itself only includes
LibJS/Object.h, which is already transitively brought in by
PlatformObject) - it seems like a small price to pay compared to what
feels like a full rebuild of LibWeb whenever a new IDL file is added.
Given all of these includes are only needed for the ::initialize
method, there is probably a smart way of avoiding this problem
altogether. I've considered both using some macro trickery or generating
these functions somehow instead.
The prototype header generation was getting a bit long.
This is also a step towards generating code for IDL files only
containing an enum definition without any interface. In that case we
can't put the enum definitions alongside the prototype - there is no
prototype to speak of.
We should never hit this case - so don't generate code for it, and
instead put in a VERIFY_NOT_REACHED.
Also improve the formatting of the generated code to closer match the
serenity code style.
Instead of a cryptic error that occurs due to an interface with no name,
fail early on by explicitly checking that an interface was parsed with a
name.
This change moves WebAssembly related data that was previously globally
accessible into the `WebAssemblyCache` object and creates one of these
per global object. This ensures that WebAssembly data cannot be
accessed across realms.
The following command was used to clang-format these files:
clang-format-18 -i $(find . \
-not \( -path "./\.*" -prune \) \
-not \( -path "./Base/*" -prune \) \
-not \( -path "./Build/*" -prune \) \
-not \( -path "./Toolchain/*" -prune \) \
-not \( -path "./Ports/*" -prune \) \
-type f -name "*.cpp" -o -name "*.mm" -o -name "*.h")
There are a couple of weird cases where clang-format now thinks that a
pointer access in an initializer list, e.g. `m_member(ptr->foo)`, is a
lambda return statement, and it puts spaces around the `->`.
Currently, if the prebuilt toolchain cache gets used, we will not try to
build the toolchain. Thus, the toolchain's ccache does not get used, and
is then pruned entirely at the end of the run.
So for now, let's just not prune the toolchain ccache. After a few years
it only reached 0.8 GB in size. And now that we are starting from empty
again, it would likely be a few more years before we reach 0.8 GB again.
If there is a cache miss while downloading the ccache from GitHub/Azure,
the .ccache directory won't exist when we try to update the modification
time of its contents. Configure the ccache size first, which will create
the .ccache directory if it doesn't exist.
Clang builds of ASAN+UBSAN on Linux take significantly less time on the
Azure CI runners. Measured times were 82 minutes for Clang 18 and
112 minutes for GCC 13, with no cache.
To keep our coverage of Ladybird builds + testing with GCC, add a
nightly job to run through the full test cycle on GCC 13.
It previously resided in LibWebView to hide the details of launching a
singleton process. That functionality now lives in LibCore. By moving
this to Ladybird, we will be able to register the process with the task
manager.
This just moves the code to launch a single process such as SQLServer to
LibCore. This will allow re-using this feature for other processes, and
will allow moving the launching of SQLServer to Ladybird.
This will be needed to collect statistics from processes that do not
have anything to do with LibWebView. The ProcessInfo structure must be
virtual to allow callers to add application-specific information.
This refactor eliminates the need for a second "fd passing socket" on
Lagom, as it uses SCM_RIGHTS in the expected fashion, to send fds along
with the data of our Unix socket message.
When launched with the new --enable-idl-tracing option, we now log
every call to web platform APIs declared via IDL, along with the
arguments passed.
This can be very helpful when trying to figure out what a site is
doing, especially if it's not doing what you'd expect.
On macOS, it's not trivial to get a Mach task port for your children.
This implementation registers the chrome process as a well-known
service with launchd based on its pid, and lets each child process
send over a reference to its mach_task_self() back to the chrome.
We'll need this Mach task port right to get process statistics.
C++ classes that inherit from JS::Cell and are leaf classes should have
their own type-specific allocator. We also do this for non-leaf classes
that are constructable from JS.
To do this, JSON messages are passed to communicate information about
each class the Clang tool comes across. This is the only message we have
to worry about for now, but in the future if we want to transmit
different kinds of information, we can make this message format more
generic.
This allows each Clang process to send JSON messages to the
orchestrating Python process, which aggregates the message and can do
something with them all at the end. This is required because we run
Clang multithreaded to speed up the tool execution.
I did try to add a second frontend tool that accepts all the files at
once, but it was _extremely_ slow, so this is the next best thing.
For example, consider the following code snippet:
Vector<Function<void()>> m_callbacks;
void add_callback(Function<void()> callback)
{
m_callbacks.append(move(callback));
}
// Somewhere else...
void do_something()
{
int a = 10;
add_callback([&a] {
dbgln("a is {}", a);
});
} // Oops, "a" is now destroyed, but the callback in m_callbacks
// has a reference to it!
We now statically detect the capture of "a" in the lambda above and flag
it as incorrect. Note that capturing the value implicitly with a capture
list of `[&]` would also be detected.
Of course, many functions that accept Function<...> don't store them
anywhere, instead immediately invoking them inside of the function. To
avoid a warning in this case, the parameter can be annotated with
NOESCAPE to indicate that capturing stack variables is fine:
void do_something_now(NOESCAPE Function<...> callback)
{
callback(...)
}
Lastly, there are situations where the callback does generally escape,
but where the caller knows that it won't escape long enough to cause any
issues. For example, consider this fake example from LibWeb:
void do_something()
{
bool is_done = false;
HTML::queue_global_task([&] {
do_some_work();
is_done = true;
});
HTML::main_thread_event_loop().spin_until([&] {
return is_done;
});
}
In this case, we know that the lambda passed to queue_global_task will
be executed before the function returns, and will not persist
afterwards. To avoid this warning, annotate the type of the capture
with IGNORE_USE_IN_ESCAPING_LAMBDA:
void do_something()
{
IGNORE_USE_IN_ESCAPING_LAMBDA bool is_done = false;
// ...
}
As defined in: https://w3c.github.io/pointerevents
With the exception of the getCoalescedEvents and getPredictedEvents
APIs.
There are still many other parts of that spec (such as the event
handlers) left to implement, but this does get us at least some of the
way.
Previously, parsing would continue if a parameter wasn't given a name
and malformed code would be generated, leading to hard to diagnose
compiler errors.
These allow us to binary search the code point compositions based on
the first code point being combined, which makes the search close to
O(log N) instead of O(N).
Previously we would only warn about missing calls to visit inside
visit_edges implementations, now we warn as well when there's no
visit_edges implementation at all.
This lets us avoid false positives when a GCPtr-wrapped member is only
a weak reference which is automatically updated by the GC when the
member's gc state is updated.
clang doesn't make all `Base::visit_edges()` calls CXXMemberCallExprs
This would lead to false positives like in HeapFunction,
where the matcher would fail to match and report a warning.
Also previously the matcher would succeed
if the visited class is missing the call to `Base::visit_edges()`
but an included class has a correct method.
The new matcher checks the current class for `visit_edges`-overrides
and matches all `visit_edges`-memberExprs inside,
checking those for starting with `Base::`.
This seems to get rid of the false positives
and should be more correct detecting missing calls.
When building, clang would throw errors about dangling references.
Extracting `template_args` to a variable before the loop and
indexing into that seems to fix the errors.
Since we're parsing segment headers for random-access jbig2 inputs
already, just always do that and get the image dimensions from the
PageInformation segment data. Not all that much more code, and it
makes this script much more pleasant to use.
jbig2 data in PDFs is in the embedded organization, which is like the
sequential organization with the file header removed.
That means jbig2 files using the random-access organization need to
be transformed to be supported. A random-access jbig2 has all segment
headers at the start, followed by the data of all segments. Decode
all headers and rewrite them to the sequential organization, where
each segment header is followed by that segment's data.
The motivation is that almost all of the jbig2 files in
ghostpdl/test/jbig2 use the random-access organization.
On Serenity, it's not trivial to extract the peer pid from a socket that
is created by SystemServer and then passed to a forked service process.
This patch adds an API to let the WebContent process notify the UI
directly, which makes the WebContent process show up in the Serenity
port's TaskManagerWidget. It seems that we will need to do something of
this sort in order to properly gather metrics on macOS as well, due to
the way that self mach ports work.
To the 'convert to int' AO. Nothing actually makes use of the [Clamp]
attribute yet in our implementation, but we may as well add support for
it now since it is trivial to do do.
This partially reverts d1e2d2a4, which made us explicitly specify the
library type for lagom libraries. This broke the fuzzer build, which
relies on the BUILD_SHARED_LIBS cmake variable to enable static builds.
We were able to keep LibCoreMinimal a bit smaller as an object library,
but that is causing ODR violations in the fuzzer build (realistically,
should be an issue in all builds, but only the fuzzer actively complains
for some reason).
To make it a shared library, we have to add a couple more symbols to it,
and make LibCore publicly depend on it.
Let's not re-invoke the "page did start loading" IPC when the history
state is pushed/replaced. It's a bit misleading (the change does not
actually load the new URL), but also the chromes may do more work than
we want when we change the URL.
Instead, add a new IPC for the history object to invoke.
Most browsers have some indicator when audio is playing in a tab, which
makes it easier to find that tab and mute unwanted audio. This adds an
IPC to allow the Ladybird chromes to do something similar.
We were off-by-one when returning the result of parsing a quoted string
in Web::Fetch::Infrastructure::collect_an_http_quoted_string. Instead of
backtracking the lexer and consuming the backtracked string, do a simple
substring operation.
We have been dancing around circular dependencies between LibCore and
generated sources. For example, LibURL currently cannot depend on
LibUnicode because the LibUnicode generators depend on LibCore, and
LibCore depends on LibURL. LibTimeZone is in a similar situation.
To alleviate this, we can define the minimal sources that the code
generators need as an object library. This will allow the generators to
depend on this library, rather than the full LibCore.
This is achieved by moving ClauseHeader::{AbstractOperation,Accessor,
Method} to Function.h itself and storing them in FunctionDeclaration.
This commit also introduces QualifiedName class that is used to store
function's name split by '.' (just like it appear in the spec).
JPEG2000 is the last image format used in PDF filters that we
don't have a loader for. Let's change that.
This adds all the scaffolding, but no actual implementation yet.
This is a fetching AO and is only used by LibWeb in the context of fetch
tasks. Move it to LibWeb with other fetch methods.
The main reason for this is that it requires the use of other LibWeb AOs
such as the forgiving Base64 decoder and MIME sniffing. These AOs aren't
available within LibURL.
And add a verification step to the emoji data generator to ensure all
emoji are listed in this file. This file will be used as a sources list
in both the CMake and GN build systems.
It is probably possible to generate this list. But in a first attempt,
the CMake code to set the file as a dependency of a pseudo target, which
would then parse the file and install the listed emoji was getting quite
verbose and complicated. So for now, let's just maintain this list.
Previously, the invalid value default wasn't taken into account when
determining the value that should be returned from the getter of an
enumerated attribute. This caused a crash when an enumerated attribute
of type DOMString? was set to an invalid value.
This URL library ends up being a relatively fundamental base library of
the system, as LibCore depends on LibURL.
This change has two main benefits:
* Moving AK back more towards being an agnostic library that can
be used between the kernel and userspace. URL has never really fit
that description - and is not used in the kernel.
* URL _should_ depend on LibUnicode, as it needs punnycode support.
However, it's not really possible to do this inside of AK as it can't
depend on any external library. This change brings us a little closer
to being able to do that, but unfortunately we aren't there quite
yet, as the code generators depend on LibCore.
Most image viewers can't display JBIG2 files.
All PDF viewers can display JBIG2 files.
This is useful for checking that PDF viewers render JBIG2 files the
same way we do.
These are standalone applications meant to be run by the user directly,
as opposed to other libexec processes which are programmatically forked
by the browser. To do this, we simply remove these processes from the
`ladybird_helper_processes` list. We must also explicitly list the
dependencies for these processes.
Text can be rendered in various ways in PDFs: Filled, stroked,
both filled and stroked, set as clipping path, hidden, or
some combinations thereof.
We don't implement any of this at the moment except "filled".
Hidden text is used in scanned documents: The image of the scan is
drawn in the background, and then OCRd text is "drawn" as hidden
on top of the scanned bitmap. That way, the (hidden) text can be
selected and copied, and it looks like you're selecting text from
the scanned bitmap. Find-in-page also works similarly. (We currently
have neither text selection nor find-in-page, but one day we will.)
Now that we have pretty good support for CCITT and are growing some
support for JBIG2, we now draw both the scanned background image
as well as the foreground text. They're not always perfectly aligned.
This change makes it so that we don't render text that's marked as
hidden. (We still do most of the coordinate math, which will probably
come in handy at some point when we implement text selection.)
This makes these scanned documents appear as they're supposed to
appear (at least in documents where we manage to decode the background
bitmap).
This also adds a debug option to force rendering of hidden text.
This SpecParser.cpp had an ever increasing number of lines and contained
an implementation of 8 different classes. So I figured out it's about
the time to split it.
No behavior change.
For Ninja Multi-Config, Xcode and Visual Studio, the way we set up our
output directories would result in exectuables that can't run from the
build directory. Add the same sauce that we added to Jakt to insert
``$<CONFIG>`` where appropriate.
PDFViewer has this, and it's useful for PDFs that have the same
text both as a scanned bitmap in the background as well as using
vector text in the foreground.
xib changes: Added a new menu entry connected to `toggleShowImages:`,
and also toggled the initial state of two menu entries. (The latter
part has no effect when the program runs since we dynamically update
this state, but it makes the menu entries show their initial state
in Xcode's menu editor.)
This does not implement any of the IDL methods, but GitHub requires the
interface exists to upload files via an <input type="file"> element.
Their JS handles uploads via this element and via drag-and-drop in one
function, and check if the uploaded file is `instanceof DataTransfer` to
decide how to handle it.
This patch implements and tests window.crypto.sublte.generateKey with
an RSA-OAEP algorithm. In order for the types to be happy, the
KeyAlgorithms objects are moved to their own .h/.cpp pair, and the new
KeyAlgorithms for RSA are added there.
This patch throws away some of the spec suggestions for how to implement
the normalize_algorithm AO and uses a new pattern that we can actually
extend in our C++.
Also update CryptoKey to store the key data.
We are recently seeing SEGV crashes during the build (while running code
generators) from within libasan itself. Turns out this is libasan bug
seen with the Linux 6.5 kernel on Ubuntu.
Now that all input events are handled by LibWebView, replace the IPCs
which send the fields of Web::KeyEvent / Web::MouseEvent individually
with one IPC per event type (key or mouse).
We can also replace the ad-hoc queued input structure with a smaller
struct that simply holds the tranferred Web::KeyEvent / Web::MouseEvent.
In the future, we can also adapt Web::EventHandler to use these structs.
The Serenity chrome is the only chrome thus far that sends all input key
and mouse events to WebContent, including shortcut activations. This is
necessary for all chromes - we must give web pages a chance to intercept
input events before handling them ourselves.
To make this easier for other chromes, this patch moves Serenity's input
event handling to LibWebView. To do so, we add the Web::InputEvent type,
which models the event data we need within LibWeb. Chromes will then be
responsible for converting between this type and their native events.
This class lives in LibWeb (rather than LibWebView) because the plan is
to use it wholesale throughout the Page's event handler and across IPC.
Right now, we still send the individual fields of the event over IPC,
but it will be an easy refactor to send the event itself. We just can't
do this until all chromes have been ported to this event queueing.
Also note that we now only handle key input events back in the chrome.
WebContent handles all mouse events that it possibly can. If it was not
able to handle a mouse event, there's nothing for the chrome to do (i.e.
there is no clicking, scrolling, etc. the chrome is able to do if the
WebContent couldn't).
This will be used to transfer information about the parent context to
DedicatedWorkers and future out-of-process Worker/Worklet
implementations for fetching purposes. In order to properly check
same-origin and other policies, we need to know more about the outside
settings than we were previously passing to the WebWorker process.
We currently use 0.6GB for a clean build with sanitizers enabled. This
will allow us to have ~3 clean builds cached and save 3GB of disk space
on CI. (The effect is actually double, because Azure archives the cache
in a tar file before uploading).
It aligns better with the Filesystem Heirarchy Standard[1] to put our
program-specific helper programs that are not intended to be executed by
the user of the application in $prefix/libexec or in whatever the
packager sets as the CMake equivalent. Namely, on Debian systems this
should be /usr/lib/Ladybird or similar.
[1] https://refspecs.linuxfoundation.org/FHS_3.0/fhs-3.0.html#usrlibexec
We had previous implemented some plumbing for file input elements in
commit 636602a54e.
This implements the return path for chromes to inform WebContent of the
file(s) the user selected. This patch includes a dummy implementation
for headless-browser to enable testing.
This commit fixes a regression introduced in
1528e9109c.
Turns out that the type of `this_value` in the property setter of the
Window object depends on how the variable is accessed. If the property
is accessed as a global variable, then this_value is of type `Window`.
For example:
```js
performance = null
```
However, when it is accessed as a property of the window object,
`this_value` is of type `WindowProxy`. For example:
```js
window.performance = null
```
This commit updates the window property setters generator to handle
both scenarios.
With this change https://discord.com/login works again.
Along with putting functions in the URL namespace into a DOMURL
namespace.
This is done as LibWeb is in an awkward situation where it needs
two URL classes. AK::URL is the general purpose URL class which
is all that is needed in 95% of cases. URL in the Web namespace
is needed predominantly for interfacing with the javascript
interfaces.
Because of two URLs in the same namespace, AK::URL has had to be
used throughout LibWeb. If we move AK::URL into a URL namespace,
this becomes more painful - where ::URL::URL is required to
specify the constructor (and something like
::URL::create_with_url_or_path in other places).
To fix this problem - rename the class in LibWeb implementing the
URL IDL interface to DOMURL, along with moving the other Web URL
related classes into this DOMURL folder.
One could argue that this name also makes the situation a little
more clear in LibWeb for why these two URL classes need be used
in the first place.
Add support for the extended attribute "ImplementedAs" for IDL
interfaces too. This allows a class which implements an IDL interface to
have a different class name than the interface itself.
"Meta/serenity.sh gdb" command opens tmux and
creates windows for GDB and emulator's logs.
The problem is that while it uses "trap" to close
just opened tmux session once debugging
is finished, it will close previously
opened session as well due to "trap" is
setup twice.
This commit tries to prevent touching
other tmux session.
This adds a basic `mkfs.fat` utility, which can format FAT12, FAT16
and FAT32 partitions.
This does have a few limitations, namely in that FAT12 formatting is
limited to a set known floppy disk sizes, and we can only generate
512-byte sectors.
In a nutshell, when we understand that the expression is a function
call (this happens at '(' after an expression), stop parsing expression
and start parsing function arguments. This makes
`FunctionCallCanonicalizationPass` and the workaround for zero argument
function calls obsolete.
