The Replacement Character (U+FFFD) is most commonly used to signal a
text encoding error, i.e. when a stream of bytes couldn't be converted
to a sequence of code points. For glyphs that don't exist in a
particular font, our rendering logic already does the right thing by
drawing empty boxes (`.notdef`); let's not forcibly turn these into
U+FFFD during rendering.
Skia now uses GPU-accelerated painting on Linux if Vulkan is available.
Most of the performance gain is currently negated by reading the GPU
backend back into RAM to pass it to the Browser process. In the future,
this could be improved by sharing GPU-allocated memory across the
Browser and WebContent processes.
This will reduce log noise when visiting sites
that have a lot of filtered content.
Previously, red error text would be displayed in
the logs for each filtered URL.
Fetch requests from web workers fail CORS checks because the origin is
not inherited from the outside settings. Ensure web worker origin is
correctly inherited from outside settings
GPU painter that uses AccelGfx is slower and way less complete compared
to both default Gfx::Painter and Skia painter. It does not make much
sense to keep it, considering Skia painter already uses Metal backend on
macOS by default and there is an option to enable GPU-accelerated
backend on linux.
This allows selection to work within shadow roots and prevents the
selection being cleared if the mouse moves outside of the current
document or shadow root.
This call is used to inform the chrome that it should display a tooltip
now and avoid any hovering timers. This is used by <video> tags to
display the volume percentage when it is changed.
Now instead of sending the position in which the user entered the
tooltip area, send just the text, and let the chrome figure out how to
display it.
In the case of Qt, wait for 600 milliseconds of no mouse movement, then
display it under the mouse cursor.
Previously, unnecessary boundary checks were being done when
constructing the range objects used to represent find in page matches.
These checks are no longer performed leading to a significant speedup
when performing find in page queries on pages containing a lot of text.
The first step of the find in page algorithm is to walk the layout tree
of each document on the page and construct a list of strings against
which to search for matches.
Previously, this was being done for each new query, even when the
page content hadn't been updated. The output of this process is now
cached in the viewport node of the associated document. This ensures
that this process is no longer repeated unnceessarily.
Unit tests on macOS deadlock because the WebContent process is waiting
for the next opportunity to render before a screenshot is taken. For
some reason unknown to myself, this opportunity never arrives. In
order to not deadlock, screenshot requests are now also processed
separately from rendering.
Since "Character" tokens do not have an "end position", viewing source
drops the source contents following the final non-"Character" token.
By handling the EOF token and breaking out of the loop, we avoid this
issue.
This is an attempt to fix the hanging CI on macOS caused by some
screenshot requests being stuck unprocessed. With this change, we at
least make sure that the HTML event loop processing, which triggers
repainting, will happen as long as there are navigables that need to be
repainted.
This avoids an unecessary lossy conversion for the current time from
double to i32. And avoids an UBSAN failure on macOS that's dependent
on the current uptime.
From SkPaint.h:
"Sets the thickness of the pen used by the paint to outline the shape.
A stroke-width of zero is treated as "hairline" width. Hairlines are
always exactly one pixel wide in device space (their thickness does not
change as the canvas is scaled)."
While we expect stroke-width=0 to simply not be painted.
This ensures that removing the last view from a WebContentClient will
close its associated process, assuming the WebContent process is not
hung. A more drastic measure will be needed to trigger forcefully
killing the process when it doesn't respond to this request.
This large commit also refactors LibWebView's process handling to use
a top-level Application class that uses a new WebView::Process class to
encapsulate the IPC-centric nature of each helper process.
Doing so results in incorrect values being created, ultimately leading
to traps or errors.
(cherry picked from commit f6c3b333334f7bb5314a844804cb259cf277005e)
The ChunkIterator now limits a chunk to using only one font (before, it
was possible to have a chunk with >1 font, when `unicode-range` CSS
property is used).
This change allows us to reduce some complexity in the text shaping and
painting code and makes us compatible with the APIs in Skia and
HarfBuzz.
Typeface is a more widely used name for the data represented by
class previously named VectorFont.
Now:
- Typeface represents decoded font that is not ready for rendering
- ScaledFont represents the combination of typeface and size for
rendering
Implements the same optimization we already have for DrawGlyphRun by
saving unscaled glyph run and scale factor in a painting command, which
allows to avoid copying of glyphs vector to apply scaling during
recording.
By moving this up to ConnectionBase, we have less custom code for each
templated subclass, and it gets a little easier to edit the code since
you don't have to rebuild as much when making changes.
If Metal context and IOSurface are available, Skia painter will use
Ganesh GPU backend on macOS, which is noticeably faster than the default
CPU backend.
Painting pipeline:
1. (WebContent) Allocate IOSurface for backing store
2. (WebContent) Allocate MTLTexture that wraps IOSurface
3. (WebContent) Paint into MTLTexture using Skia
4. (Browser) Wrap IOSurface into Gfx::Painter and use
QPainter/CoreGraphics to blit backing store into viewport.
Things we should improve in the future:
1. Upload textures for images in advance instead of doing that before
every repaint.
2. Teach AppKit client to read directly from IOSurface instead of
copying.
This is a non-standard API that other browsers implement, which
highlights matching text in the current window.
This is just a thin wrapper around our find in page functionality, the
main motivation for adding this API is that it allows us to write tests
for our find in page implementation.
This is the same behavior as RequestServer, with the added benefit that
we know how to gracefully reconnect ImageDecoder to all WebContent
processes on restart.
This is mostly useful when some application-level logic needs to
iterate over all child processes. A more robust Process abstraction
would make this easier.
They are now blocked on pages which:
- Don't have an opaque origin (should be only user-initiated or about:)
- Aren't other file: pages
- Aren't other resource: pages
The `deepEquals` algorithm used for testing was naive, and incorrectly
evaluated equality of objects in some cases. The new algorithm considers
that the second object could have more keys than the first, and compares
the prototypes of the objects.
An input event is now fired when the step up or step down button of an
input element of type number is clicked.
This ensures that any associated <output> element is updated when these
buttons are clicked.
Input elements without a defined user-interaction behavior need to fire
an input event when the user changes the element's value in some way.
This change moves the code to do this into its own function and adds
some spec text to explain what is being done.
The changes to tests are due to LibTimeZone incorrectly interpreting
time stamps in the TZDB. The TZDB will list zone transitions in either
UTC or the zone's local time (which is then subject to DST offsets).
LibTimeZone did not handle the latter at all.
For example:
The following rule is in effect until November 18, 6PM UTC.
America/Chicago -5:50:36 - LMT 1883 Nov 18 18:00u
The following rule is in effect until March 1, 2AM in Chicago time. But
at that time, a DST transition occurs, so the local time is actually
3AM.
America/Chicago -6:00 Chicago C%sT 1936 Mar 1 2:00
This required updating some LibJS spec steps to their latest versions,
as the data expected by the old steps does not quite match the APIs that
are available with the ICU. The new spec steps are much more aligned.
Multiple APIs have moved from the DOM Parsing and Serialization spec to
HTML.
Updates spec URLs and comments.
Delete InnerHTML file:
- Make parse_fragment a member of Element, matching serialize_fragment
on Node.
- Move inner_html_setter inline into Element and ShadowRoot as per the
spec.
Add FIXME to Range.idl for Trusted Types createContextualFragment
This stuff has moved from a mixin defined by the DOM Parsing spec, over
to the HTML spec, where they are now defined as partial interfaces for
Element and ShadowRoot.
There's also some new functionality that we don't implement yet, so
patch marks them as FIXME properties.
And let the old shadow_root(), which was only supposed to be used by
bindings, be called shadow_root_for_bindings() instead.
This makes it much easier to read DOM code, and we don't have to worry
about when to use shadow_root_internal() or why.
Implement setLineDash() and getLineDash() in CanvasPathDrawingStyles,
which write/read from the CanvasState object.
This doesn't implement the actual drawing of a dashed line, but at least
sites using the Chart.js library no longer fail with an exception.
Unfortunately the Painter classes don't support dashed/dotted lines
based on segments yet.
Any data that sticks around in a decoder, especially frames that
haven't been retrieved, may cause issues for playback.
This is especially the case with H.264, since its arbitrary frame
ordering to allow reference frames to precede B-frames causes it to
hold onto frames, and causes the playback manager to get back a frame
at a completely wrong timestamp after seeking.
The timestamp offset of a block was being converted from i16 to u64, so
negative values would overflow and cause timestamps that fall before
the cluster's timestamp from being close to the minimum representable
i64.
The math is also now done using saturating operations to prevent any
other similar issues from occurring.
H.264 in Matroska can have blocks with unordered timestamps. Without
passing these as the presentation timestamp into the FFmpeg decoder,
the frames will not be returned in chronological order.
VideoFrame will now include a timestamp that is used by the
PlaybackManager, rather than assuming that it is the same timestamp
returned by the demuxer.
VP9 continues to function, but this also allows AV1 to be decoded. With
this commit, H.264 is still non-functional, as the decoder requires
some extra initial data from the track definition in the Matroska file.
These aren't particularly small objects, but we were still copying them
around all over the place. When TrackEntry contains data buffers, they
won't need to be copied as well.
This should halve the size of frames in memory for frames with 8-bit
color components, which is the majority of videos.
Calculation of the size of subsampled planes has also been consolidated
into a struct. There are likely some places that will still need to
change over to this, but it should prevent issues due to differing
handling of rounding/ceiling.
BT.2020 will mainly be used with bit depths greater than 8, so having
this specialization is mostly pointless until we use fixed-point math
for higher bit depths.
It is not needed by code generators anymore, so it is not needed in the
Lagom tools build. And it is not needed as an object library anymore; it
was created this way so it could be included in Serenity's LibC.
Prior to this change, our find in page function always highlighted the
first match whenever the query was updated. After this change the
current match index is set such that it is the first match to occur
after the end of the current selection.
This means the current match position is not lost if the user modifies
their existing query.
This matches the behavior of find in page in other browsers.
Using mmap-allocated memory for backing stores does not allow us to
benefit from using GPU-accelerated painting, because all the performance
increase we get is mostly negated by reading the GPU-allocated texture
back into RAM, so it can be shared with the browser process.
With IOSurface, we get a framebuffer that is both shareable between
processes and can be used as underlying memory for an OpenGL/Metal
texture.
This change does not yet benefit from using IOSurface and merely wraps
them into Gfx::Bitmap to be used by the CPU painter.
Allows WebContentClient to get pid of WebContent process right after
creation, so there is no window between forking and
notify_process_information() IPC response, when client doesn't know the
pid.
In upcoming changes, bitmap is going to be used to wrap the memory of
the IOSurface, and we will want to release the corresponding IOSurface
along with the bitmap.
In the upcoming changes, we are going to switch macOS to using an
IOSurface for the backing store. This change will simplify the process
of sharing an IOSurface between processes because we already have the
MachPortServer running in the browser, and WebContent knows how to
locate the corresponding server.
This change ensures that if a screenshot is requested between teardown
of paintable tree and relayout, we will wait for layout to complete
before taking a screenshot.
LibLocale was split off from LibUnicode a couple years ago to reduce the
number of applications on SerenityOS that depend on CLDR data. Now that
we use ICU, both LibUnicode and LibLocale are actually linking in this
data. And since vcpkg gives us static libraries, both libraries are over
30MB in size.
This patch reverts the separation and merges LibLocale into LibUnicode
again. We now have just one library that includes the ICU data.
Further, this will let LibUnicode share the locale cache that previously
would only exist in LibLocale.
We only need LibCoreMinimal for the lagom-tools build. In particular, by
removing LibUnicode, we remove the lagom-tools dependence on the system
ICU package, as we do not have vcpkg hooked into this build. (We could
probably add vcpkg here, but since this libraries aren't even needed, we
don't need to bother).
When any LibCore file (or any of its dependents) changes, we have to
regenerate all IDL bindings. By depending on LibCoreMinimal, the number
of impacting files greatly reduces.
As an example, changing a .cpp file in LibUnicode would previously cause
about 1800 ninja targets to rebuild. This is now reduced to about 200.
If a flex item has a preferred aspect ratio and the flex basis is not
definite, we were falling back to using stretch-fit for the main size,
since that appeared to match other browsers.
However, we missed the case where we actually have a definite cross size
through which the preferred aspect ratio can be naturally resolved.
This patch adds a simple in-memory HTTP cache to each WebContent
process.
It's currently off by default (turn it on with --enable-http-cache)
since the validation logic is lacking and incomplete.
This implements most of the CloseWatcher API from the html spec.
AbortSignal support is unimplemented.
Integration with dialogs and popovers is also unimplemented.
For SerenityOS, we parse emoji metadata from the UCD to learn emoji
groups, subgroups, names, etc. We used this information only in the
emoji picker dialog. It is entirely unused within Ladybird.
This removes our dependence on the UCD emoji file, as we no longer
need any of its information. All we need to know is the file path to
our custom emoji, which we get from Meta/emoji-file-list.txt.
To get away from the ancient (and buggy) text layout code in
Gfx::Painter, we want to remove all the uses of Painter::draw_text().
As a first step towards this, we implement the draw_text() display list
command in terms of the draw_text_run() command by performing the very
simple necessary layout in draw_text() beforehand.
Our current segmenter implementation lives in LibUnicode, and is not
locale-aware. We will need such awareness for ECMA-402, and so LibLocale
will be the new home for text segmentation.
The tests here are ported directly from LibUnicode/TestSegmentation.cpp.
There are a couple of differences here due to using ICU:
1. Titlecasing behaves slightly differently. We previously transformed
"123dollars" to "123Dollars", as we would use word segmentation to
split a string into words, then transform the first cased character
to titlecase. ICU doesn't go quite that far, and leaves the string
as "123dollars". While this is a behavior change, the only user of
this API is the `text-transform: capitalize;` CSS rule, and we now
match the behavior of other browsers.
2. There isn't an API to compare strings with case insensitivity without
allocating case-folded strings for both the left- and right-hand-side
strings. Our implementation was previously allocation-free; however,
in a benchmark, ICU is still ~1.4x faster.
Find in page will no longer match text that spans across block elements.
Previously, given the markup `WH<div>F</div>`, the query `WHF` would
find a match. We would now match `WH` and `F` separately, but not `WHF`.
When resuming execution of a suspended function, we must not overwrite
any cached `this` value with something from the ExecutionContext.
This was causing an empty JS::Value to leak into the VM when resuming
an async arrow function, since the "this mode" for such functions is
lexical and thus ExecutionContext will have an empty `this`.
It became a problem due to the bytecode optimization where we allow
ourselves to assume that `this` remains cached after we've executed a
ResolveThisBinding in this (or the first) basic block of the executable.
Fixes https://github.com/LadybirdBrowser/ladybird/issues/138
Apple Clang doesn't like this, rather than waiting for their version of
random-clang-commit-to-call-a-release to catch up with llvm trunk, just
work around the issue.
Fixes#186.
This just allows using the ICU header within LibUnicode, without having
to link against LibLocale.
Eventually, I think it will make sense to combine LibUnicode & LibLocale
back into a single library. They were separated to remove the large CLDR
data library from LibUnicode since most users did not need it. But that
is not much of a concern now.
There have been a number of changes to the locale resolution AOs that
we've fallen behind on. Mostly editorial, but includes one normative
change to canonicalize Unicode extension keywords in the Intl.Locale
constructor.
Instead of taking an out-parameter, return the canonicalization result.
This allows the API to be used where specs want to store the result and
the original values in separate variables.
This change introduces Skia painter available under a flag. It's not
yet match capabilities of Gfx::Painter and is not ready to replace it.
Motivation:
- The current CPU painter is a performance bottleneck on most websites.
Our own GPU painter implementation is very immature and has received
relatively little attention.
- There is ongoing effort on building painter that supports affine
transforms (AffineCommandExecutorCPU) but it is far from being on par
with the default CPU painter. Skia will allow us to immediately get
full transformation support for all painting commands.
GPU painting:
I experimented with Ganesh GPU-backend, however profiling revealed that
without sharing viewport texture between WebContent and Browser
processes, it won't bring much performance improvement compared to
CPU-backend. Therefore, I decided to keep this PR focused on
implementing most of painting commands and switch to GPU-backend in
future changes.
Text rendring:
Skia painter uses glyph bitmaps produced by LibGfx. Using Skia for text
rendering will require large refactoring of the font rendering
subsystem. Currently, it's impossible to construct SkFont right before
rendering because Gfx::VectorFont can't be serialized back into sequence
of bytes.
There is a problem with ugly include paths like:
`#include <core/SkBitmap.h>`.
I would prefer to have skia prefix in the path. There was an attempt to
fix that but PR was rejected https://github.com/microsoft/vcpkg/pull/32660
Regressions compared to Gfx::Painter:
- DrawText is not implemented
- PaintTextShadow is not implemented
- PaintRadialGradient and PaintLinearGradient do not support "transition
hints" and repeat length
- PaintConicGradient is not implemented
- DrawTriangleWave is not implemented
- DrawLine does not account for line style property
- DrawScaledBitmap and DrawScaledImmutableBitmap do not account for
scaling mode property
This change is a preparation before introducing Skia painter in an
upcoming change. It's needed because Skia does not have an API to
implement ClearClipRect command. It only allows to return previous
clip rect by popping from its internal state stack.
A bit more context: initially we had save and restore commands, but
their frequent use led to many reallocations of vector during painting
commands recording. To resolve this, we switched to SegmentedVector to
store commands list, which allows fast appends. Now, having many save
and restore commands no longer causes noticeable performance issue.
This adds a motion preference to the browser UI similar to the existing
ones for color scheme and contrast.
Both AppKit UI and Qt UI has this new preference.
The auto value is currently the same as NoPreference, follow-ups can
address wiring that up to the actual preference for the OS.
ListFormat was the first formatter I ported to ICU. This patch makes it
match the style of subsequently ported formatters, where we create the
formatter once per Intl object, rather than once per prototype
invocation.
We were previously treating undefined and null as the same (an empty
Optional). However, there are edge cases in ECMA-402 where we must treat
them differently. Namely, the hour cycle (hc) keyword. An undefined hc
value has no effect on the resolved locale, whereas a null hc value can
actively override any hc specified in the locale string. For example:
new Intl.DateTimeFormat("en-u-hc-h11", { hour12: false });
In that object, the hour12 option does not match the u-hc-h11 value. So
the spec dictates we remove the hc value by setting it to null.
Previously, the "Find Next Match" and "Find Previous Match" actions
simply updated the match index of the last query to be performed. This
led to incorrect results if the page had been modified after the last
query had been run.
`Page::find_in_page_next_match()` and
`Page::find_in_page_previous_match()` both now rerun the last query to
ensure the results are up to date before updating the match index.
The match index is also reset if the URL of the active document has
changed since the last query. The current match index is maintained if
only the URL fragment changes.
The values aren't that complex, so it doesn't make much sense to have a
dedicated generator for them. Parsing them manually also allows us to
have much more control over the produced values, so as a result of this
change, EasingStyleValue becomes much more ergonomic.
This was useful for code generation as reducing the size of the enums
had a notable impact on the size of the generated code. This is not the
case any longer.
This uses ICU for all of the Intl.PluralRules prototypes, which lets us
remove all data from our plural rules generator.
Plural rules depend directly on internal data from the number formatter,
so rather than creating a separate Locale::PluralRules class (which will
make accessing that data awkward), this adds plural rules APIs to the
existing Locale::NumberFormat.
Previously, the find in page function would fail to find text which was
split across multiple text nodes. For example, given the following
markup: `WH<span>F` the query `WHF` would previously fail to be
matched.
This is done by traversing all of the document's text nodes -
constructing a complete string to query against and keeping track of
the locations where that string is split across multiple nodes.
