This adds support for the [Unscopable] extended attribute to attributes
and functions.
I believe it should be applicable to all interface members, but I
haven't done that here.
This currently only supports pair iterables (i.e. iterable<key, value>)
support for value iterables (i.e. iterable<value>) is left as TODO().
Since currently our cmake setup calls the WrapperGenerator separately
and unconditionally for each (hard-coded) output file iterable wrappers
have to be explicitly marked so in the CMakeLists.txt declaration, we
could likely improve this in the future by querying WrapperGenerator
for the outputs based on the IDL.
This patch essentially just splits the non return-specific logic from
generate_return_statement (i.e. the wrapping of the cpp value into
a javascript one) into a separate function generate_wrap_statement that
can be used to wrap any cpp value during wrapper generation.
This custom attribute will be used for objects that hold onto arbitrary
JS::Value's. This is needed as JS::Handle can only be constructed for
objects that implement JS::Cell, which JS::Value doesn't.
This works by overriding the `visit_edges` function in the wrapper.
This overridden function calls the base `visit_edges` and then forwards
it to the underlying implementation.
This will be used for CustomEvent, which must hold onto an arbitrary
JS::Value for it's entire lifespan.
A legacy platform object is a non-global platform object that
implements a special operation. A special operation is a getter, setter
and/or deleter. This is particularly used for old collection types,
such as HTMLCollection, NodeList, etc.
This will be used to make these spec-compliant and remove their custom
wrappers. Additionally, it will be used to implement collections that
we don't have yet, such as DOMStringMap.
This does a few things, that are hard to separate. For a while now, it's
been confuzing what `StyleValue::is_foo()` actually means. It sometimes
was used to check the type, and sometimes to see if it could return a
certain value type. The new naming scheme is:
- `is_length()` - is it a LengthStyleValue?
- `as_length()` - casts it to LengthStyleValue
- `has_length()` - can it return a Length?
- `to_length()` - gets the internal value out (eg, Length)
This also means, no more `static_cast<LengthStyleValue const&>(*this)`
stuff when dealing with StyleValues. :^)
Hopefully this will be a bit clearer going forward. There are lots of
places using the original methods, so I'll be going through them to
hopefully catch any issues.
These now crash as VM::call() uses ThrowExceptionOr<T>, which refuses to
hold an empty JS::Value as its non-exception result.
We only need to return an empty value when should_return_empty() says
so for the return value of throw_dom_exception_if_needed().
Co-authored-by: Luke Wilde <lukew@serenityos.org>
For `number` and `integer` types, you can add a range afterwards to add
a range check, using similar syntax to that used in the CSS specs. For
example:
```json
"font-weight": {
...
"valid-types": [
"number [1,1000]"
],
...
}
```
This limits any numbers to the range `1 <= n <= 1000`.
Previously, we have not been validating the values for CSS declarations
inside the Parser. This causes issues, since we should be discarding
invalid style declarations, so that previous ones are used instead. For
example, in this code:
```css
.foo {
width: 2em;
width: orange;
}
```
... the `width: orange` declaration overwrites the `width: 2em` one,
even though it is invalid. According to the spec, `width: orange` should
be rejected at parse time, and discarded, leaving `width: 2em` as the
resulting value.
Many properties (mostly shorthands) are parsed specially, and so they
are already rejected if they are invalid. But for simple properties, we
currently accept any value. With `property_accepts_value()`, we can
check if the value is valid in `parse_css_value()`, and reject it if it
is not.
We already expand shorthands in the cascade, so there's no need to
preserve them in the output.
This patch reorganizes the CSS::PropertyID enum values so that we can
easily iterate over all shorthand or longhand properties.
This patch adds a basic initial implementation of these API's.
Since LibWeb currently doesn't support workers, this implementation of
messaging doesn't bother with serializing and deserializing messages.
When parsing shorthand values, we'd like to use
`property_initial_value()` to get their longhand property values,
instead of hard-coding them as we currently do. That involves
recursively calling that function while the `initial_values` map is
being initialized, which causes problems because the shorthands appear
alphabetically before their longhand components, so the longhands aren't
initialized yet!
The solution here is to perform 2 passes when generating the code,
outputting properties without "longhands" first, and the rest after.
This could potentially cause issues when shorthands have multiple
levels, in particular `border` -> `border-color` -> `border-left-color`.
But, we do not currently define a default value for `border`, and
`border-color` takes only a single value, so it's fine for now. :^)
Multi-lib distros like Gentoo and Fedora install lagom-core.so into
lagom-install/lib64 rather than lib. Set the install RPATH based on
CMAKE_INSTALL_LIBDIR to avoid the wrong path being set in the binaries.
Also apply macOS specific RPATH rules to fix the build on that platform.
Replace the old logic where we would start with a host build, and swap
all the CMake compiler and target variables underneath it to trick
CMake into building for Serenity after we configured and built the Lagom
code generators.
The SuperBuild creates two ExternalProjects, one for Lagom and one for
Serenity. The Serenity project depends on the install stage for the
Lagom build. The SuperBuild also generates a CMakeToolchain file for the
Serenity build to use that replaces the old toolchain file that was only
used for Ports.
To ensure that code generators are rebuilt when core libraries such as
AK and LibCore are modified, developers will need to direct their manual
`ninja` invocations to the SuperBuild's binary directory instead of the
Serenity binary directory.
This commit includes warning coalescing and option style cleanup for the
affected CMakeLists in the Kernel, top level, and runtime support
libraries. A large part of the cleanup is replacing USE_CLANG_TOOLCHAIN
with the proper CMAKE_CXX_COMPILER_ID variable, which will no longer be
confused by a host clang compiler.
This common strategy of having a serenity_option() macro defined in
either the Lagom or top level CMakeLists.txt allows us to do two things:
First, we can more clearly see which options are Serenity-specific,
Lagom-specific, or common between the target and host builds.
Second, it enables the upcoming SuperBuild changes to set() the options
in the SuperBuild's CMake cache and forward each target's options to the
corresponding ExternalProject.
This makes it so we don't need to specify the full path to all the
helper scripts we include() from different places in the codebase and
feels a lot cleaner.
We'll use this to prevent repeating common tool dependencies. They all
depend on LibCore and AK only. We also want to encapsulate common
install rules for them.
This namespace will be used for all interfaces defined in the URL
specification, like URL and URLSearchParams.
This has the unfortunate side-effect of requiring us to use the fully
qualified AK::URL name whenever we want to refer to the AK class, so
this commit also fixes all such references.
This lets you query if a given Quirk applies to a given PropertyID.
Currently this applies only to the "Hashless hex color" and "Unitless
length" quirks.
This removes the awkward String::replace API which was the only String
API which mutated the String and replaces it with a new immutable
version that returns a new String with the replacements applied. This
also fixes a couple of UAFs that were caused by the use of this API.
As an optimization an equivalent StringView::replace API was also added
to remove an unnecessary String allocations in the format of:
`String { view }.replace(...);`
There's only a couple of cases like this, but there are some locale
paths in the CLDR that contain variants. For example, there isn't a
en-US path, but there is a en-US-POSIX path. This interferes with the
operation to search for locales by name. The algorithm is such that
searching for en-US will not result in en-US-POSIX being found. To
resolve this, we should remove variants from the locale name.
This data informs consumers how to join lists of values. For example,
in en-US, the list ["a", "b", "c"] formatted to a string should become
"a, b, and c".
This is to simply the Default Case Conversion implementation. Otherwise,
the implementation would need to determine which special casing rule to
apply, instead of just picking the first match.
The amount of aliases in the likely-subtags dataset is quite large, so
this also needed to change the way the data is generated. Otherwise, the
compiler would complain about the size of the generated code.
Previously, a static method was generated that would effectively parse
the dataset into a HashMap of Unicode::LanguageID at runtime. We now
perform that parsing at generation-time, and instead generate an Array
of a structure similar to Unicode::LanguageID (we cannot use the same
structure because it contains String and Optional, which cannot be used
at compile-time).
The DOM specification says that the primary use case for these is to
give Promises abort semantics. It is also a prerequisite for Fetch,
as it is used to make Fetch abortable.
a
CLDR contains a set of likely subtag data where, given a locale, you can
resolve what is the most likely language, script, or territory of that
locale. This data is needed for resolving territory aliases. These
aliases might contain multiple territories, and we need to resolve which
of those territories is most likely correct for a locale.
Note that the likely subtag data is quite huge (a few thousand entries).
As an optimization encouraged by the spec, we only generate the smallest
subset of this data that we actually need (about 150 entries).
Most alias substitutions are "simple", meaning that alias matching is
done by examining a single locale subtag. However, there are a handful
of "complex" aliases where matching is done by examining multiple
subtags. For example, the variant subtag "lojban" causes the locale
"art-lojban" to be canonicalized to "jbo", but only when the language
subtag is "art" (i.e. this should not occur for the locale "en-lojban").
This generates a method to perform complex alias matching.
CLDR contains a set of aliases for languages, territories, etc. that no
longer are meant to be used (e.g. due to deprecation). For example, the
language "aam" is deprecated and should be canonicalized as "aas".
This is needed so all headers and files exist on disk, so that
the sonar cloud analyzer can find them when executing the compilation
commands contained in compile_commands.json, without actually building.
Co-authored-by: Andrew Kaster <akaster@serenityos.org>
This allows us to remove all the add_subdirectory calls from the top
level CMakeLists.txt that referred to targets linking LagomCore.
Segregating the host tools and Serenity targets helps us get to a place
where the main Serenity build can simply use a CMake toolchain file
rather than swapping all the compiler/sysroot variables after building
host libraries and tools.
Moving this helper CMake file to the centralized Meta/CMake folder helps
to get a better grasp on what extra files are required for the build,
and what files are generated.
While we're at it, don't use add_compile_definitions for
ENABLE_UNICODE_DATA, which only needs to be seen by LibUnicode sources.
The top-level CMakeLists.txt already automatically detects ccache, but
CI will invoke CMake with Lagom's CMakeLists.txt. Add an option to Lagom
to do the same detection.
This standard CMake option controls whether add_library() calls will
use STATIC or SHARED by default. The flag is set to on by default
since that's what we want for normal CI jobs and local builds and the
test262 runner, but disabled for oss-fuzz builds.
This should finally fix the oss-fuzz build after it was broken in #9017
oss-fuzz un-breakage was verified by running the following commands in
the oss-fuzz repo:
python infra/helper.py build_image serenity
python infra/helper.py build_fuzzers --sanitizer address --engine afl \
--architecture x86_64 serenity /path/to/local/checkout/Meta/Lagom
python infra/helper.py check_build --sanitizer address --engine afl \
--architecture x86_64 serenity
This supports some binary property matching. It does not support any
properties not yet parsed by LibUnicode, nor does it support value
matching (such as Script_Extensions=Latin).
Split the Lagom build into shared libraries to match the Serenity build.
This reduces the cognitive load when trying to edit the Lagom CMakeLists
significantly. It also reduces the amount of source files that must be
compiled to run each test or host program significantly.
Also re-organize all the build rules into sections. And reorganize the
CMakeLists file in general.
LibTTF has a concrete dependency on LibGfx for things like Gfx::Bitmap,
and LibGfx has a concrete dependency in the TTF::Font class in
Gfx::FontDatabase. This circular dependency works fine for Serenity and
Lagom Linux builds of the two libraries. It also works fine for static
library builds on Lagom macOS builds.
However, future changes will make Lagom use shared libraries, and
circular library dependencies are not tolerated in macOS.
This is primarily to allow using LibUnicode within LibJS and its REPL.
Note: this seems to be the first time that a Lagom dependency requires
generated source files. For this to work, some of Lagom's CMakeLists.txt
commands needed to be re-organized to include the CMake files that fetch
and parse UnicodeData.txt. The paths required to invoke the generator
also differ depending on what is currently building (SerenityOS vs.
Lagom as part of the Serenity build vs. a standalone Lagom build).
These are usually incorrect, and people sometimes forget to add the
correct values as a result of them being optional, so they should just
be specified explicitly.
This removes all usages of the non-standard define_property helper
method and replaces all it's usages with the specification required
alternative or with define_direct_property where appropriate.
All GUI applications currently load all TTF fonts on startup
(to populate the Gfx::FontDatabase. This could probably be smarter.)
Before this patch, everyone would open the files and read them into
heap-allocated storage. Now we simply mmap() them instead. :^)
This commit adds a bunch of passes, the most interesting of which is a
pass that merges blocks together, and a pass that places blocks that
flow into each other next to each other, and a very simply pass that
removes duplicate basic blocks.
Note that this does not remove the jump at the end of each block in that
pass to avoid scope creep in the passes.
Previously, AK::Function would accept _any_ callable type, and try to
call it when called, first with the given set of arguments, then with
zero arguments, and if all of those failed, it would simply not call the
function and **return a value-constructed Out type**.
This lead to many, many, many hard to debug situations when someone
forgot a `const` in their lambda argument types, and many cases of
people taking zero arguments in their lambdas to ignore them.
This commit reworks the Function interface to not include any such
surprising behaviour, if your function instance is not callable with
the declared argument set of the Function, it can simply not be
assigned to that Function instance, end of story.
Previously ByteBuffer::grow() behaved like Vector<T>::resize().
However the function name was somewhat ambiguous - and so this patch
updates ByteBuffer to behave more like Vector<T> by replacing grow()
with resize() and adding an ensure_capacity() method.
This also lets the user change the buffer's capacity without affecting
the size which was not previously possible.
Additionally this patch makes the capacity() method public (again).
This only tests "can it be parsed", but the goal of this commit is to
provide a test framework that can be built upon :)
The conformance tests are downloaded, compiled* and installed only if
the INCLUDE_WASM_SPEC_TESTS cmake option is enabled.
(*) Since we do not yet have a wast parser, the compilation is delegated
to an external tool from binaryen, `wasm-as`, which is required for the
test suite download/install to succeed.
This *does* run the tests in CI, but it currently does not include the
spec conformance tests.
Previously the CMake options for -fsanitize=address, thread and
undefined were gated behind clang, which was unecessary. Only
-fsanitize=fuzzer is clang-only.
Since the operations are already complicated and will become even more
so soon, let's split them into their own files. We can also integrate
the NumberTheory operations that would better fit there into this class
as well.
This commit doesn't change behaviors, but moves the allocation of some
variables into caller classes.
Previously the directions omitted that you have to specify
`-CMAKE_CXX_COMPILER` when building the Fuzzers. This
would cause all kinds of weird problems at compilation and
link time. You can't specify one or the other, they must
both be pointing at clang in order for things to work as
experted. Fix this by updating the documentation to specify
that the user should specify both the C and CXX compiler explicitly
to be safe, as well as forcing the cmake clang argument handling
to modify the CXX compiler variable instead of the C version.
Clang's default constexpr-steps limit is 1048576, which is not enough
for LibGfx's generation of the unicode bidirectional class lookup table
while GCC doesn't have any limit at all, so this patch increases the
limit to an arbitrarily larger value.
As many macros as possible are moved to Macros.h, while the
macros to create a test case are moved to TestCase.h. TestCase is now
the only user-facing header for creating a test case. TestSuite and its
helpers have moved into a .cpp file. Instead of requiring a TEST_MAIN
macro to be instantiated into the test file, a TestMain.cpp file is
provided instead that will be linked against each test. This has the
side effect that, if we wanted to have test cases split across multiple
files, it's as simple as adding them all to the same executable.
The test main should be portable to kernel mode as well, so if
there's a set of tests that should be run in self-test mode in kernel
space, we can accomodate that.
A new serenity_test CMake function streamlines adding a new test with
arguments for the test source file, subdirectory under /usr/Tests to
install the test application and an optional list of libraries to link
against the test application. To accomodate future test where the
provided TestMain.cpp is not suitable (e.g. test-js), a CUSTOM_MAIN
parameter can be passed to the function to not link against the
boilerplate main function.
SPDX License Identifiers are a more compact / standardized
way of representing file license information.
See: https://spdx.dev/resources/use/#identifiers
This was done with the `ambr` search and replace tool.
ambr --no-parent-ignore --key-from-file --rep-from-file key.txt rep.txt *
Move LibCompress unit tests to LibCompress/Tests directory and register
them with CMake's add_test. This allows us to run these tests with
ninja test instead of running a separate executable.
Also split the existing tests in 3 test files that better follow the
source code structure (inspired by AK tests).
This allows us to remove the FAIL_REGEX logic from the CTest invocation
of AK and LibRegex tests, as they will return a non-zero exit code on
failure :^).
Also means that running a failing TestSuite-enabled test with the
run-test-and-shutdown script will actually print that the test failed.
These tests were never built for the serenity target. Move their Lagom
build steps to the Lagom CMakeLists.txt, and add serenity build steps
for them. Also, fix the build errors when building them with the
serenity cross-compiler :^)
This is already set in the root CMakeLists.txt as well as here for Clang
(-Wno-user-defined-literals), but was forgotten for GCC which made an
Lagom-only build (cmake ../Meta/Lagom [...]) fail.
This is basically just for consistency, it's quite strange to see
multiple AK container types next to each other, some with and some
without the namespace prefix - we're 'using AK::Foo;' a lot and should
leverage that. :^)
A new operator, operator""sv was added as of C++17 to support
string_view literals. This allows string_views to be constructed
from string literals and with no runtime cost to find the string
length.
See: https://en.cppreference.com/w/cpp/string/basic_string_view/operator%22%22sv
This change implements that functionality in AK::StringView.
We do have to suppress some warnings about implementing reserved
operators as we are essentially implementing STL functions in AK
as we have no STL :).
(...and ASSERT_NOT_REACHED => VERIFY_NOT_REACHED)
Since all of these checks are done in release builds as well,
let's rename them to VERIFY to prevent confusion, as everyone is
used to assertions being compiled out in release.
We can introduce a new ASSERT macro that is specifically for debug
checks, but I'm doing this wholesale conversion first since we've
accumulated thousands of these already, and it's not immediately
obvious which ones are suitable for ASSERT.
Arbitrarily split up to make git bisect easier.
These unnecessary #include's were found by combining an automated tool (which
determined likely candidates) and some brain power (which decided whether
the #include is also semantically superfluous).
This caused some confusion: Apparently, clang has no trouble overriding Shell's
main, and this issue only surfaced when I tried to build the fuzzers with
wrong configuration (i.e., without the clang-injected 'main').
The diff is suggested by, and work of, @alimpfard.
This was done with the help of several scripts, I dump them here to
easily find them later:
awk '/#ifdef/ { print "#cmakedefine01 "$2 }' AK/Debug.h.in
for debug_macro in $(awk '/#ifdef/ { print $2 }' AK/Debug.h.in)
do
find . \( -name '*.cpp' -o -name '*.h' -o -name '*.in' \) -not -path './Toolchain/*' -not -path './Build/*' -exec sed -i -E 's/#ifdef '$debug_macro'/#if '$debug_macro'/' {} \;
done
# Remember to remove WRAPPER_GERNERATOR_DEBUG from the list.
awk '/#cmake/ { print "set("$2" ON)" }' AK/Debug.h.in
-fsanitize=fuzzer was being added to LINKER_FLAGS from Lagom/CMakeLists,
which we don't want with FuzzilliJs as we want to define the functions
it provides ourselves.
There's no guarantee that the last executed command will have a zero
exit code, and so the shell exit code may or may not be zero, even if
all the tests pass.
Also changes the `test || echo fail && exit` to
`if not test { echo fail && exit }`, since that's nicer-looking.
These changes are arbitrarily divided into multiple commits to make it
easier to find potentially introduced bugs with git bisect.Everything:
The modifications in this commit were automatically made using the
following command:
find . -name '*.cpp' -exec sed -i -E 's/dbg\(\) << ("[^"{]*");/dbgln\(\1\);/' {} \;
