Now that the lambda capture plugin isn't full of false-positives, we can
make the jump and start halting builds for these errors. It also allows
these plugins to be useful in CI.
Instead of being opt-out with NOESCAPE, it is now opt-in with ESCAPING.
Opt-out is ideal, but unfortunately this was extremely noisy when
compiling the entire codebase. Escaping functions are rarer than non-
escaping ones, so let's just go with that for now.
This also allows us to gradually add heuristics for detecting missing
ESCAPING annotations and emitting them as errors. It also nicely matches
the spelling that Swift uses (@escaping), which is where this idea
originally came from.
This allows readonly attributes and functions to have a 'FIXME' extended
attribute added to the IDL definition to stub out the function. This
makes debugging web compatibility issues on live sites much easier as a
FIXME message is logged whenever one of these functions or attributes
are called.
Support still needs to be extended to non-readonly attributes (and some
other special cases), but this should allow us to set a big percentage
of the commented out attributes/functions in IDL files to instead use
this extended attribute.
Now both /bin/zcat and /bin/gunzip are symlinks to /bin/gzip, and we
essentially running it in decompression mode through these symlinks.
This ensures we don't maintain 2 versions of code to decompress Gzipped
data anymore, and handle the use case of gzipped-streaming input only
once in the codebase.
This is a more general and robust replacement of the LibJSGCVerifier.
We want to add more generic static analysis, and this new plugin will
be built in a way that integrates into the rest of the system.
The high-level design is that we have a static method on WebPWriter that
returns an AnimationWriter object. AnimationWriter has a virtual method
for writing individual frames. This allows streaming animations to disk,
without having to buffer up the entire animation in memory first.
The semantics of this function, add_frame(), are that data is flushed
to disk every time the function is called, so that no explicit `close()`
method is needed.
For some formats that store animation length at the start of the file,
including WebP, this means that this needs to write to a SeekableStream,
so that add_frame() can seek to the start and update the size when a
frame is written.
This design should work for GIF and APNG writing as well. We can move
AnimationWriter to a new header if we add writers for these.
Currently, `animation` can read any animated image format we can read
(apng, gif, webp) and convert it to an animated webp file.
The written animated webp file is not compressed whatsoever, so this
creates large output files at the moment.
An AudioNode is the fundamental building block used in 'Audio
Contexts'. In our immediate case, the audio node we are working towards
implementing is an oscillating source node.
Previously the GML compiler did not support object properties such as
`content_widget: @GUI::Widget{}` for GUI::ScrollableContainerWidget;
this commit adds support for such properties by simply calling
`set_<key>(<TProperty>&)` on the object.
This commit also removes the previous hack where
ScrollableContainerWidget was special-cased to have its singular child
used as the content widget; the only GML file using this behaviour was
also changed to be in line with 'proper' GML as handled by the GML
Playground.
This makes it possible to use externally defined toplevel widgets that
have no C++ header defining them.
Note that this only allows widget-native properties on the object, as
the actual original definition is not available.
This was resulting in a whole lot of rebuilding whenever a new IDL
interface was added.
Instead, just directly include the prototype in every C++ file which
needs it. While we only really need a forward declaration in each cpp
file; including the full prototype header (which itself only includes
LibJS/Object.h, which is already transitively brought in by
PlatformObject) - it seems like a small price to pay compared to what
feels like a full rebuild of LibWeb whenever a new IDL file is added.
Given all of these includes are only needed for the ::initialize
method, there is probably a smart way of avoiding this problem
altogether. I've considered both using some macro trickery or generating
these functions somehow instead.
The prototype header generation was getting a bit long.
This is also a step towards generating code for IDL files only
containing an enum definition without any interface. In that case we
can't put the enum definitions alongside the prototype - there is no
prototype to speak of.
We should never hit this case - so don't generate code for it, and
instead put in a VERIFY_NOT_REACHED.
Also improve the formatting of the generated code to closer match the
serenity code style.
Instead of a cryptic error that occurs due to an interface with no name,
fail early on by explicitly checking that an interface was parsed with a
name.
This change moves WebAssembly related data that was previously globally
accessible into the `WebAssemblyCache` object and creates one of these
per global object. This ensures that WebAssembly data cannot be
accessed across realms.
The following command was used to clang-format these files:
clang-format-18 -i $(find . \
-not \( -path "./\.*" -prune \) \
-not \( -path "./Base/*" -prune \) \
-not \( -path "./Build/*" -prune \) \
-not \( -path "./Toolchain/*" -prune \) \
-not \( -path "./Ports/*" -prune \) \
-type f -name "*.cpp" -o -name "*.mm" -o -name "*.h")
There are a couple of weird cases where clang-format now thinks that a
pointer access in an initializer list, e.g. `m_member(ptr->foo)`, is a
lambda return statement, and it puts spaces around the `->`.
This refactor eliminates the need for a second "fd passing socket" on
Lagom, as it uses SCM_RIGHTS in the expected fashion, to send fds along
with the data of our Unix socket message.
When launched with the new --enable-idl-tracing option, we now log
every call to web platform APIs declared via IDL, along with the
arguments passed.
This can be very helpful when trying to figure out what a site is
doing, especially if it's not doing what you'd expect.
C++ classes that inherit from JS::Cell and are leaf classes should have
their own type-specific allocator. We also do this for non-leaf classes
that are constructable from JS.
To do this, JSON messages are passed to communicate information about
each class the Clang tool comes across. This is the only message we have
to worry about for now, but in the future if we want to transmit
different kinds of information, we can make this message format more
generic.
This allows each Clang process to send JSON messages to the
orchestrating Python process, which aggregates the message and can do
something with them all at the end. This is required because we run
Clang multithreaded to speed up the tool execution.
I did try to add a second frontend tool that accepts all the files at
once, but it was _extremely_ slow, so this is the next best thing.
For example, consider the following code snippet:
Vector<Function<void()>> m_callbacks;
void add_callback(Function<void()> callback)
{
m_callbacks.append(move(callback));
}
// Somewhere else...
void do_something()
{
int a = 10;
add_callback([&a] {
dbgln("a is {}", a);
});
} // Oops, "a" is now destroyed, but the callback in m_callbacks
// has a reference to it!
We now statically detect the capture of "a" in the lambda above and flag
it as incorrect. Note that capturing the value implicitly with a capture
list of `[&]` would also be detected.
Of course, many functions that accept Function<...> don't store them
anywhere, instead immediately invoking them inside of the function. To
avoid a warning in this case, the parameter can be annotated with
NOESCAPE to indicate that capturing stack variables is fine:
void do_something_now(NOESCAPE Function<...> callback)
{
callback(...)
}
Lastly, there are situations where the callback does generally escape,
but where the caller knows that it won't escape long enough to cause any
issues. For example, consider this fake example from LibWeb:
void do_something()
{
bool is_done = false;
HTML::queue_global_task([&] {
do_some_work();
is_done = true;
});
HTML::main_thread_event_loop().spin_until([&] {
return is_done;
});
}
In this case, we know that the lambda passed to queue_global_task will
be executed before the function returns, and will not persist
afterwards. To avoid this warning, annotate the type of the capture
with IGNORE_USE_IN_ESCAPING_LAMBDA:
void do_something()
{
IGNORE_USE_IN_ESCAPING_LAMBDA bool is_done = false;
// ...
}
As defined in: https://w3c.github.io/pointerevents
With the exception of the getCoalescedEvents and getPredictedEvents
APIs.
There are still many other parts of that spec (such as the event
handlers) left to implement, but this does get us at least some of the
way.
Previously, parsing would continue if a parameter wasn't given a name
and malformed code would be generated, leading to hard to diagnose
compiler errors.
These allow us to binary search the code point compositions based on
the first code point being combined, which makes the search close to
O(log N) instead of O(N).
Previously we would only warn about missing calls to visit inside
visit_edges implementations, now we warn as well when there's no
visit_edges implementation at all.
This lets us avoid false positives when a GCPtr-wrapped member is only
a weak reference which is automatically updated by the GC when the
member's gc state is updated.
clang doesn't make all `Base::visit_edges()` calls CXXMemberCallExprs
This would lead to false positives like in HeapFunction,
where the matcher would fail to match and report a warning.
Also previously the matcher would succeed
if the visited class is missing the call to `Base::visit_edges()`
but an included class has a correct method.
The new matcher checks the current class for `visit_edges`-overrides
and matches all `visit_edges`-memberExprs inside,
checking those for starting with `Base::`.
This seems to get rid of the false positives
and should be more correct detecting missing calls.
When building, clang would throw errors about dangling references.
Extracting `template_args` to a variable before the loop and
indexing into that seems to fix the errors.
To the 'convert to int' AO. Nothing actually makes use of the [Clamp]
attribute yet in our implementation, but we may as well add support for
it now since it is trivial to do do.
This partially reverts d1e2d2a4, which made us explicitly specify the
library type for lagom libraries. This broke the fuzzer build, which
relies on the BUILD_SHARED_LIBS cmake variable to enable static builds.
Let's not re-invoke the "page did start loading" IPC when the history
state is pushed/replaced. It's a bit misleading (the change does not
actually load the new URL), but also the chromes may do more work than
we want when we change the URL.
Instead, add a new IPC for the history object to invoke.
Most browsers have some indicator when audio is playing in a tab, which
makes it easier to find that tab and mute unwanted audio. This adds an
IPC to allow the Ladybird chromes to do something similar.
This is achieved by moving ClauseHeader::{AbstractOperation,Accessor,
Method} to Function.h itself and storing them in FunctionDeclaration.
This commit also introduces QualifiedName class that is used to store
function's name split by '.' (just like it appear in the spec).
And add a verification step to the emoji data generator to ensure all
emoji are listed in this file. This file will be used as a sources list
in both the CMake and GN build systems.
It is probably possible to generate this list. But in a first attempt,
the CMake code to set the file as a dependency of a pseudo target, which
would then parse the file and install the listed emoji was getting quite
verbose and complicated. So for now, let's just maintain this list.
Previously, the invalid value default wasn't taken into account when
determining the value that should be returned from the getter of an
enumerated attribute. This caused a crash when an enumerated attribute
of type DOMString? was set to an invalid value.
This URL library ends up being a relatively fundamental base library of
the system, as LibCore depends on LibURL.
This change has two main benefits:
* Moving AK back more towards being an agnostic library that can
be used between the kernel and userspace. URL has never really fit
that description - and is not used in the kernel.
* URL _should_ depend on LibUnicode, as it needs punnycode support.
However, it's not really possible to do this inside of AK as it can't
depend on any external library. This change brings us a little closer
to being able to do that, but unfortunately we aren't there quite
yet, as the code generators depend on LibCore.
Text can be rendered in various ways in PDFs: Filled, stroked,
both filled and stroked, set as clipping path, hidden, or
some combinations thereof.
We don't implement any of this at the moment except "filled".
Hidden text is used in scanned documents: The image of the scan is
drawn in the background, and then OCRd text is "drawn" as hidden
on top of the scanned bitmap. That way, the (hidden) text can be
selected and copied, and it looks like you're selecting text from
the scanned bitmap. Find-in-page also works similarly. (We currently
have neither text selection nor find-in-page, but one day we will.)
Now that we have pretty good support for CCITT and are growing some
support for JBIG2, we now draw both the scanned background image
as well as the foreground text. They're not always perfectly aligned.
This change makes it so that we don't render text that's marked as
hidden. (We still do most of the coordinate math, which will probably
come in handy at some point when we implement text selection.)
This makes these scanned documents appear as they're supposed to
appear (at least in documents where we manage to decode the background
bitmap).
This also adds a debug option to force rendering of hidden text.
This SpecParser.cpp had an ever increasing number of lines and contained
an implementation of 8 different classes. So I figured out it's about
the time to split it.
No behavior change.
PDFViewer has this, and it's useful for PDFs that have the same
text both as a scanned bitmap in the background as well as using
vector text in the foreground.
xib changes: Added a new menu entry connected to `toggleShowImages:`,
and also toggled the initial state of two menu entries. (The latter
part has no effect when the program runs since we dynamically update
this state, but it makes the menu entries show their initial state
in Xcode's menu editor.)
We had previous implemented some plumbing for file input elements in
commit 636602a54e.
This implements the return path for chromes to inform WebContent of the
file(s) the user selected. This patch includes a dummy implementation
for headless-browser to enable testing.
This commit fixes a regression introduced in
1528e9109c.
Turns out that the type of `this_value` in the property setter of the
Window object depends on how the variable is accessed. If the property
is accessed as a global variable, then this_value is of type `Window`.
For example:
```js
performance = null
```
However, when it is accessed as a property of the window object,
`this_value` is of type `WindowProxy`. For example:
```js
window.performance = null
```
This commit updates the window property setters generator to handle
both scenarios.
With this change https://discord.com/login works again.
Along with putting functions in the URL namespace into a DOMURL
namespace.
This is done as LibWeb is in an awkward situation where it needs
two URL classes. AK::URL is the general purpose URL class which
is all that is needed in 95% of cases. URL in the Web namespace
is needed predominantly for interfacing with the javascript
interfaces.
Because of two URLs in the same namespace, AK::URL has had to be
used throughout LibWeb. If we move AK::URL into a URL namespace,
this becomes more painful - where ::URL::URL is required to
specify the constructor (and something like
::URL::create_with_url_or_path in other places).
To fix this problem - rename the class in LibWeb implementing the
URL IDL interface to DOMURL, along with moving the other Web URL
related classes into this DOMURL folder.
One could argue that this name also makes the situation a little
more clear in LibWeb for why these two URL classes need be used
in the first place.
Add support for the extended attribute "ImplementedAs" for IDL
interfaces too. This allows a class which implements an IDL interface to
have a different class name than the interface itself.
This adds a basic `mkfs.fat` utility, which can format FAT12, FAT16
and FAT32 partitions.
This does have a few limitations, namely in that FAT12 formatting is
limited to a set known floppy disk sizes, and we can only generate
512-byte sectors.
In a nutshell, when we understand that the expression is a function
call (this happens at '(' after an expression), stop parsing expression
and start parsing function arguments. This makes
`FunctionCallCanonicalizationPass` and the workaround for zero argument
function calls obsolete.
Setters for Window object should consider WindowProxy wrapper by:
- Verifying `this_value` is `WindowProxy` (not `HTML::Window`)
- Defining properties on the underlying Window object instead of on
the WindowProxy itself.
The former automatically adapts the prefix to binary and octal
output, and is what we already use in the majority of cases.
Patch generated by:
rg -l '0x\{' | xargs sed -i '' -e 's/0x{:/{:#/'
I ran it 4 times (until it stopped changing things) since each
invocation only converted one instance per line.
No behavior change.
We currently support optional integral types, but not nullable types. So
if an IDL contains e.g. "long?", passing null will be coerced to 0.
This will be used by the Inspector, but will also eventually be used by
real IDL interfaces (e.g. HTMLInputElement's selectionStart).
This can apply to optional, nullable parameters of platform types. This
will cause the generator to generate the type as
Optional<JS::GCPtr<T>> rather than JS::GCPtr<T>, allowing the
implementation to differentiate between the case where the argument was
not passed, and the case where null or undefined was passed.
Needing to differentiate these two cases is quite niche, hence why it is
an opt-in behavior.
This is a simple extension of GenericLexer, and is used in more than
just LibXML, so let's move it into AK.
The move also resolves a FIXME, which is removed in this commit.
Instead of limiting ourselves to the key of the property in the JSON,
allow overriding the function name so we can generate completely
different function signatures with very similar names to other existing
API methods (e.g. `glUniform*` vs `glUniform*v`).
On argument swapping to put positional ones toward the end,
m_arg_index was pointing at "last arg index" + "skipped args" +
"consumed args" and thus was pointing ahead of the skipped ones.
m_arg_index now points after the current parsed option arguments.
CMYK data describes which inks a printer should use to print a color.
If a screen should display a color that's supposed to look similar
to what the printer produces, it results in a color very different
to what Color::from_cmyk() produces. (It's also printer-dependent.)
There are many ICC profiles describing printing processes. It doesn't
matter too much which one we use -- most of them look somewhat
similar, and they all look dramatically better than Color::from_cmyk().
This patch adds a function to download a zip file that Adobe offers
on their web site. They even have a page for redistribution:
https://www.adobe.com/support/downloads/iccprofiles/icc_eula_win_dist.html
(That one leads to a broken download though, so this downloads the
end-user version.)
In case we have to move off this download at some point, there are also
a whole bunch of profiles at https://www.color.org/registry/index.xalter
that "may be used, embedded, exchanged, and shared without restriction".
The adobe zip contains a whole bunch of other useful and fun profiles,
so I went with it.
For now, this only unzips the USWebCoatedSWOP.icc file though, and
installs it in ${CMAKE_BINARY_DIR}/Root/res/icc/Adobe/CMYK/. In
Serenity builds, this will make it to /res/icc/Adobe/CMYK in the
disk image. And in lagom build, after #23016 this is the
lagom res staging directory that tools can install via
Core::ResourceImplementation. `pdf` and `MacPDF` already do that,
`TestPDF` now does it too.
The final piece is that LibPDF then loads the profile from there
and uses it for DeviceCMYK color conversions.
(Doing file access from the bowels of a library is a bit weird,
especially in a system that has sandboxing built in. But LibGfx does
that in FontDatabase too already, and LibPDF uses that, so it's not a
new problem.)
This way, build files can install things into
`${CMAKE_BINARY_DIR}/Root/res/` and it'll work in both serenity and
lagom builds.
It allows using a single `Core::ResourceImplementation::install()`
call to install both checked-in and generated files (as long as both
get copied into this new build-time staging dir).
No behavior change.
Many widget classes need to run substantial initialization code after
they have been setup from GML. With this change, an
initialize_fallibles() function is called if available, allowing the
initialization to be invoked from the GML setup automatically. This
means that the GML-generated creation function can now be used directly
for many more cases, and reduces code duplication.
We currently bundle AK with LibCore on Lagom. This means that to use AK,
all libraries must also depend on LibCore. This will create circular
dependencies when we create LibURL, as LibURL will depend on LibUnicode,
which will depend on LibCore, which will depend on LibURL.
This helper can automatically link against libraries needed by all
utilities, such as LibCore. This is to make extracting AK into its own
library a bit easier.
Due to the way expression parser is written, we need to resolve the
ambiguity between member access operators and dots used for punctuation
during lexing. The lexer uses a (totally bulletproof) heuristic to do
that: whenever '.' is followed by ' ' or '\n', it is considered a dot
and member access otherwise. While it works fine for prettified test
cases, non-prettified files often lack enter after a trailing dot
character. Since MemberAccess will always be invalid at that position,
explicitly treat trailing dot as a part of punctuation.
RecursiveASTVisitor was recursing into the subtrees of an old root if it
was changed in on_entry callback. Fix that by querying root pointer just
after on_entry callback returns. While on it, also use
`AK::TemporaryChange` instead of setting `m_current_subtree_pointer`
manually.
As it turns out, `FunctionCallCanonicalizationPass` was relying on being
able to replace tree on entry, and the bug in RecursiveASTVisitor made
the pass to not fully canonicalize nested function calls.
The changes to GenericASTPass.cpp alone are enough to fix the problem
but it is canonical (for some definition of canonicity) to only change
trees in on_leave. Therefore, the commit also switches
FunctionCallCanonicalizationPass to on_leave callback.
A test for this fix and one from the previous commit is also included.
We cannot handle them normally since we need text between parenthesis to
be a valid expression. As a workaround, we now push an artificial value
to stack to act as an argument (it'll be later removed during function
call canonicalization).
I got fed up with looking at error messages that tell me "VERIFICATION
FAILED: !is_error()". So this commit introduces DiagnosticEngine class
whose purpose is to accumulate and print more user-friendly errors.
For some reason I was afraid to add trivial accessors to classes
in earlier PRs, so we now have dozens of classes with public fields. I'm
not exactly looking forward to refactoring them all at once but I'll
do so gradually.
FontDatabase.h with its includes add up to quite a lot of code. In the
next commit, compiled GML files are going to need to access the
FontWeight enum, so let's allow them to do that without pulling in lots
of other things.
Also, change users to include FontWeight.h instead of FontDatabase.h
where appropriate.
This splits the RIFFTypes header/TU into the WAV specific parts, which
move to WavTypes.h, as well as the general RIFF parts which move to the
new LibRIFF.
Sidenote for the spec comments: even though they are linked from a site
that explains the WAV format, the document is the (an) overall RIFF spec
from Microsoft. A better source may be used later; the changes to the
header are as minimal as possible.
...and hook it up.
I opened MainMenu.xib in Xcode, added a new "Submenu Menu Item"
from the Library (cmd-shift-l), added a User Defined
"toggleShowClippingPaths:" action on First Responder and connected
the menu item's action to that action.
(I first tried duplicating the existing Window menu and editing that,
but the Window menu is marked as `systemMenu="window"` in the xib and
I couldn't find a way to undo that in Xcode. So the Debug menu first
acted as a second Window menu.)
I made "Debug" a toplevel menu to make it consistent with Ladybird.app
for now, but I'll probably make it a submenu of "View" in the future.
This creates a bitmap filled with a fixed color, then (in memory)
saves it as jpeg and loads it again, and repeats that until the
color of the bitmap no longer changes. It then reports how many
iterations that took, and what the final color was.
It does this for a couple of colors.
This is for quality assessment of the jpeg codec. Ideally, it should
converge quickly (in one iteration), and on a color not very far from
the original input color.
`JsonValue::to_byte_string` has peculiar type-erasure semantics which is
not usually intended. Unfortunately, it also has a very stereotypical
name which does not warn about unexpected behavior. So let's prefix it
with `deprecated_` to make new code use `as_string` if it just wants to
get string value or `serialized<StringBuilder>` if it needs to do proper
serialization.
A bunch of users used consume_specific with a constant ByteString
literal, which can be replaced by an allocation-free StringView literal.
The generic consume_while overload gains a requires clause so that
consume_specific("abc") causes a more understandable and actionable
error.
We have two known PlatformObjects that need to implement some of the
behavior of LegacyPlatformObjects to date: Window, and HTMLFormElement.
To make this not require double (or virtual) inheritance of
PlatformObject, move the behavior of LegacyPlatformObject into
PlatformObject. The selection of LegacyPlatformObject behavior is done
with a new bitfield of feature flags instead of a dozen virtual
functions that return bool. This change simplifies every class involved
in the diff with the notable exception of Window, which now needs some
ugly const casts to implement named property access.
If we know that the peer disconnected while receiving a message in the
generated code, let's shutdown the connection from here instead of
forcing each client to do so.
This commit introduces NamedVariableDeclaration and
SSAVariableDeclaration and allows storing both of them in Variable node.
Also, it adds additional structures in FunctionDefinition and
BasicBlock, which will be used to store SSA form related information.
We assume that variable shadowing is impossible, so then there is no
reason to keep distinct Declaration and Assignment operators after
ReferenceResolvingPass.
We would previously always generate string parameters to pass through
to functions as a `String`. This works fine if the argument is a
`FlyString const&`, but falls apart for optional types where we need to
accept an `Optional<FlyString> const&`.
Support this by implementing a [FlyString] extended attribute which
if present results in the parameter for the function being generated
as a FlyString.
This is how the spec tells us we should be converting to these integer
types.
Also leave around a FIXME to pass through information about the [Clamp]
and [EnforceRange] extended attributes, and port over these instances to
the new WebIDL integer typedefs.
Per:
https://dom.spec.whatwg.org/#concept-reflect
We should be calling `get_attribute_value` for reflected IDL strings.
No functional change as nowhere is performing a reflect on a nullable
type, and just ends up simplifying the code.
In a bunch of cases, this actually ends up simplifying the code as
to_number will handle something such as:
```
Optional<I> opt;
if constexpr (IsSigned<I>)
opt = view.to_int<I>();
else
opt = view.to_uint<I>();
```
For us.
The main goal here however is to have a single generic number conversion
API between all of the String classes.
This commit un-deprecates DeprecatedString, and repurposes it as a byte
string.
As the null state has already been removed, there are no other
particularly hairy blockers in repurposing this type as a byte string
(what it _really_ is).
This commit is auto-generated:
$ xs=$(ack -l \bDeprecatedString\b\|deprecated_string AK Userland \
Meta Ports Ladybird Tests Kernel)
$ perl -pie 's/\bDeprecatedString\b/ByteString/g;
s/deprecated_string/byte_string/g' $xs
$ clang-format --style=file -i \
$(git diff --name-only | grep \.cpp\|\.h)
$ gn format $(git ls-files '*.gn' '*.gni')
If we don't do this, and there a class in a namespace with the same
name, type resolution gets confused between `<namespace>::<class>` and
`<class>::<constructor>`.
My system's python3 is not in /bin/.
The README did not indicate that a clang-toolchain build of Serenity is
required, so this patch adds that explicit instruction.
