Truly independent web browser
38b7c7f068
There are two changes happening here: a correctness fix, and an optimization. In theory they are unrelated, but the optimization actually paves the way for the correctness fix. Before this commit, the HTML tokenizer would attempt to look for named character references by checking from after the `&` until the end of m_decoded_input, which meant that it was unable to recognize things like named character references that are inserted via `document.write` one byte at a time. For example, if `∉` was written one-byte-at-a-time with `document.write`, then the tokenizer would only check against `n` since that's all that would exist at the time of the check and therefore erroneously conclude that it was an invalid named character reference. This commit modifies the approach taken for named character reference matching by using a trie-like structure (specifically, a deterministic acyclic finite state automaton or DAFSA), which allows for efficiently matching one-character-at-a-time and therefore it is able to pick up matching where it left off after each code point is consumed. Note: Because it's possible for a partial match to not actually develop into a full match (e.g. `¬indo` which could lead to `⋵̸`), some backtracking is performed after-the-fact in order to only consume the code points within the longest match found (e.g. `¬indo` would backtrack back to `¬`). With this new approach, `document.write` being called one-byte-at-a-time is handled correctly, which allows for passing more WPT tests, with the most directly relevant tests being `/html/syntax/parsing/html5lib_entities01.html` and `/html/syntax/parsing/html5lib_entities02.html` when run with `?run_type=write_single`. Additionally, the implementation now better conforms to the language of the spec (and resolves a FIXME) because exactly the matched characters are consumed and nothing more, so SWITCH_TO is able to be used as the spec says instead of RECONSUME_IN. The new approach is also an optimization: - Instead of a linear search using `starts_with`, the usage of a DAFSA means that it is always aware of which characters can lead to a match at any given point, and will bail out whenever a match is no longer possible. - The DAFSA is able to take advantage of the note in the section `13.5 Named character references` that says "This list is static and will not be expanded or changed in the future." and tailor its Node struct accordingly to tightly pack each node's data into 32-bits. Together with the inherent DAFSA property of redundant node deduplication, the amount of data stored for named character reference matching is minimized. In my testing: - A benchmark tokenizing an arbitrary set of HTML test files was about 1.23x faster (2070ms to 1682ms). - A benchmark tokenizing a file with tens of thousands of named character references mixed in with truncated named character references and arbitrary ASCII characters/ampersands runs about 8x faster (758ms to 93ms). - The size of `liblagom-web.so` was reduced by 94.96KiB. Some technical details: A DAFSA (deterministic acyclic finite state automaton) is essentially a trie flattened into an array, but it also uses techniques to minimize redundant nodes. This provides fast lookups while minimizing the required data size, but normally does not allow for associating data related to each word. However, by adding a count of the number of possible words from each node, it becomes possible to also use it to achieve minimal perfect hashing for the set of words (which allows going from word -> unique index as well as unique index -> word). This allows us to store a second array of data so that the DAFSA can be used as a lookup for e.g. the associated code points. For the Swift implementation, the new NamedCharacterReferenceMatcher was used to satisfy the previous API and the tokenizer was left alone otherwise. In the future, the Swift implementation should be updated to use the same implementation for its NamedCharacterReference state as the updated C++ implementation. |
||
|---|---|---|
| .devcontainer | ||
| .github | ||
| AK | ||
| Base/res | ||
| Documentation | ||
| Libraries | ||
| Meta | ||
| Services | ||
| Tests | ||
| Toolchain | ||
| UI | ||
| Utilities | ||
| .clang-format | ||
| .clang-tidy | ||
| .clangd | ||
| .editorconfig | ||
| .gitattributes | ||
| .gitignore | ||
| .gn | ||
| .mailmap | ||
| .pre-commit-config.yaml | ||
| .prettierignore | ||
| .prettierrc | ||
| .swift-format | ||
| .ycm_extra_conf.py | ||
| CMakeLists.txt | ||
| CMakePresets.json | ||
| CODE_OF_CONDUCT.md | ||
| CONTRIBUTING.md | ||
| flake.lock | ||
| flake.nix | ||
| ISSUES.md | ||
| LICENSE | ||
| README.md | ||
| SECURITY.md | ||
| shell.nix | ||
| vcpkg-configuration.json | ||
| vcpkg.json | ||
Ladybird
Ladybird is a truly independent web browser, using a novel engine based on web standards.
Important
Ladybird is in a pre-alpha state, and only suitable for use by developers
Features
We aim to build a complete, usable browser for the modern web.
Ladybird uses a multi-process architecture with a main UI process, several WebContent renderer processes, an ImageDecoder process, and a RequestServer process.
Image decoding and network connections are done out of process to be more robust against malicious content. Each tab has its own renderer process, which is sandboxed from the rest of the system.
At the moment, many core library support components are inherited from SerenityOS:
- LibWeb: Web rendering engine
- LibJS: JavaScript engine
- LibWasm: WebAssembly implementation
- LibCrypto/LibTLS: Cryptography primitives and Transport Layer Security
- LibHTTP: HTTP/1.1 client
- LibGfx: 2D Graphics Library, Image Decoding and Rendering
- LibUnicode: Unicode and locale support
- LibMedia: Audio and video playback
- LibCore: Event loop, OS abstraction layer
- LibIPC: Inter-process communication
How do I build and run this?
See build instructions for information on how to build Ladybird.
Ladybird runs on Linux, macOS, Windows (with WSL2), and many other *Nixes.
How do I read the documentation?
Code-related documentation can be found in the documentation folder.
Get in touch and participate!
Join our Discord server to participate in development discussion.
Please read Getting started contributing if you plan to contribute to Ladybird for the first time.
Before opening an issue, please see the issue policy and the detailed issue-reporting guidelines.
The full contribution guidelines can be found in CONTRIBUTING.md.
License
Ladybird is licensed under a 2-clause BSD license.