The headless-browser source is getting a bit unwieldy. The ordering of
class and method definitions is fragile; e.g. the application and web
view classes each require full definitions of each other. So it has
reached the point where it makes sense to give headless-browser some
better file structure.
To prepare for that, this patch simply moves its source to live along-
side the other browser chromes. This location is a bit better prepared
for creating more files, as the Utilities folder doesn't even have its
own CMakeLists.txt.
We have a bit of forgiveness around allowing tests to pass with varying
trailing newlines. Only write a rebaselined test to disk if it would not
have passed under those conditions.
We currently create a single WebView and run all 1400+ LibWeb tests in
serial over that WebView. Instead, let's create as many WebViews as
there are processes on the system, and run LibWeb tests concurrently
over those views.
To do this performantly requires that we never block the main thread of
the headless-browser process once the tests are running. Doing so will
effectively pause execution of all other tests. So test execution is now
Promise-based.
On my machine (with a hardware concurrency of 32), this reduces the run
time of LibWeb tests from 31.382s to 3.640s. CPU utilization increases
from 5% to 67%.
Instead, just log the view's current URL when the WebView crashes. It
won't make any sense to track the executing test this way once there are
many tests running concurrently.
We currently run all tests in a single WebView instance. That instance
owns the process-wide RequestClient / ImageDecoderClient, so if we were
to create a second instance, we'd run into trouble.
This migrates ownership of these services to the Application class, and
makes the Application own the WebView. In the future, this will let the
Application own a list of views.
We currently pass around the individual fields of the Application class
to a bunch of free functions. This makes adding a new field, and passing
it all the way to e.g. run_dump_test pretty annoying, as we have to go
through about 5 function calls.
This will get much worse in an upcoming patch to run LibWeb tests
concurrently. There, we will have to further pass these flags around as
async lambda value captures.
To make this nicer, just access the flags from Application::the(), which
is how the "real" UIs access their application objects as well.
The existing rebaseline script is a bit limiting in that it can only
rebaseline a single test at a time. When making sweeping changes, this
patch will let us rebaseline any number of tests at once.
For example, in the following abbreviated test HTML:
<span>some text</span>
<script>println("whf")</script>
We would have to craft the expectation file to include the "some text"
segment, usually with some leading whitespace. This is a bit annoying,
and makes it difficult to manually craft expectation files.
So instead of comparing the expectation against the entire DOM inner
text, we now send the inner text of just the <pre> element containing
the test output when we invoke `internals.signalTextTestIsDone`.
Previously, if you ran with a relative path, then everything would run
fine except that the test name would be blank in the output, eg:
1/1234:
instead of:
1/1234: Text/input/canvas/export.html
The IPCs to request a page's text, layout tree, etc. are currently all
synchronous. This can result in a deadlock when WebContent also makes
a synchronous IPC call, as both ends will be waiting on each other.
This replaces the page info IPCs with a single, asynchronous IPC. This
new IPC is promise-based, much like our screenshot IPC.
The main motivator here was noticing that --disable-sql-database did not
work with AppKit. Rather than re-implementing this there, move ownership
of these classes to WebView::Application, so that each UI does not need
to individually worry about it.
Prior to funcref, a partial chunk of an invalid module was never needed,
but funcref allows a partially instantiated module to modify imported
tables with references to its own functions, which means we need to keep
the second module alive while that function reference is present within
the imported table.
This was tested by the spectests, but very rarely caught as our GC does
not behave particularly predictably, making it so the offending module
remains in memory just long enough to let the tests pass.
This commit makes it so all function references keep their respective
modules alive.
The identifier "Protocol" is claimed by Objective-C and Swift for use
by the language's built-in protocol conformance feature, which is
similar to Rust traits or Java interfaces.
Rename LibProtocol -> LibRequests, and its namespace from Protocol to
Requests to accomodate this.
Web specs do not return through javascript percent decoded URL path
components - but we were doing this in a number of places due to the
default behaviour of URL::serialize_path.
Since percent encoded URL paths may not contain valid UTF-8 - this was
resulting in us crashing in these places.
For example - on an HTMLAnchorElement when retrieving the pathname for
the URL of:
http://ladybird.org/foo%C2%91%91
To fix this make the URL class only return the percent encoded
serialized path, matching the URL spec. When the decoded path is
required instead explicitly call URL::percent_decode.
This fixes a crash running WPT URL tests for the anchor element on:
https://wpt.live/url/a-element.html
Remove `for_each_section_of_type` in favor of making the module's
sections defined as distinct fields. This means it is no longer possible
to have two of the same section (which is invalid in WebAssembly, for
anything other than custom sections).
Currently, if we want to add a new e.g. WebContent command line option,
we have to add it to all of Qt, AppKit, and headless-browser. (Or worse,
we only add it to one of these, and we have feature disparity).
To prevent this, this moves command line flags to WebView::Application.
The flags are assigned to ChromeOptions and WebContentOptions structs.
Each chrome can still add its platform-specific options; for example,
the Qt chrome has a flag to enable Qt networking.
There should be no behavior change here, other than that AppKit will now
support command line flags that were previously only supported by Qt.
Skia painter is visibly faster than LibGfx painter and has more complete
CSS transforms support. With this change:
- On Linux, it will try to use Vulkan-backend with fallback to
CPU-backend
- On macOS it will try to use Metal-backend with fallback to
CPU-backend
- headless-browser always runs with CPU-backend in layout mode
LibGfx's output is consistent across different platforms, which allows
us to have one set of expectations for screenshot tests. This
consistency will not hold for Skia, where features like antialiasing and
gradient color interpolation vary slightly depending on the platform. In
upcoming changes, we are going to switch to using Skia as the default
painter, which leaves us with the following options:
- Have per-platform screenshot test expectations.
- Limit screenshot tests to run only on one platform and maintain a
single set of expectation files.
For now, I have decided to choose the latter option, using Linux as it
seems to be the most popular platform among developers.
This change will make it easier to disable screenshot comparison tests
on a specific platform or have per-platform expectations.
Additionally, it's nice to be able to tell if a ref-test uses a
screenshot as an expectation by looking at the test path.
This large commit also refactors LibWebView's process handling to use
a top-level Application class that uses a new WebView::Process class to
encapsulate the IPC-centric nature of each helper process.
This is the same behavior as RequestServer, with the added benefit that
we know how to gracefully reconnect ImageDecoder to all WebContent
processes on restart.
LibLocale was split off from LibUnicode a couple years ago to reduce the
number of applications on SerenityOS that depend on CLDR data. Now that
we use ICU, both LibUnicode and LibLocale are actually linking in this
data. And since vcpkg gives us static libraries, both libraries are over
30MB in size.
This patch reverts the separation and merges LibLocale into LibUnicode
again. We now have just one library that includes the ICU data.
Further, this will let LibUnicode share the locale cache that previously
would only exist in LibLocale.
