Commit graph

310 commits

Author SHA1 Message Date
Andrew Kaster
6e7cc40b18 Meta: Add Meta/CMake to the CMAKE_MODULE_PATH for Serenity and Lagom
This makes it so we don't need to specify the full path to all the
helper scripts we include() from different places in the codebase and
feels a lot cleaner.
2021-09-15 19:04:52 +04:30
Ali Mohammad Pur
741886a4c4 LibRegex: Make the optimizer understand references and capture groups
Otherwise the fork in patterns like `(1+)\1` would be (incorrectly)
optimized away.
2021-09-15 15:52:28 +04:30
Andreas Kling
0a09eaf3a1 LibJS+LibTest: Use JS::Script and JS::SourceTextModule in test-js
Instead of creating a Parser and Lexer manually in test-js, we now
use either JS::Script::parse() or JS::SourceTextModule::parse()
to load tests.
2021-09-14 21:41:51 +02:00
Ali Mohammad Pur
ccb53c64e9 AK: Add an abstraction over multiple disjoint buffers
DisjointChunks<T> provides a nice interface over multiple sequential
Vector<T>'s, allowing the user to iterate over/index into/slice from
said buffers as if they were a single contiguous buffer.
To work with views on such objects, DisjointSpans<T> is provided, which
has the same behaviour but does not own the underlying objects.
2021-09-14 21:33:15 +04:30
Idan Horowitz
d6cfa34667 AK: Make URL::m_port an Optional<u16>, Expose raw port getter
Our current way of signalling a missing port with m_port == 0 was
lacking, as 0 is a valid port number in URLs.
2021-09-14 00:14:45 +02:00
Ali Mohammad Pur
246ab432ff LibRegex: Add a basic optimization pass
This currently tries to convert forking loops to atomic groups, and
unify the left side of alternations.
2021-09-13 14:38:53 +04:30
Timothy Flynn
470262c8ab LibJS: Use ErrorType::NotAnObjectOfType instead of NotA 2021-09-12 00:16:39 +02:00
Timothy Flynn
c59b97043e LibWeb: Use ErrorType::NotAnObjectOfType instead of NotA 2021-09-12 00:16:39 +02:00
Idan Horowitz
6704961c82 AK: Replace the mutable String::replace API with an immutable version
This removes the awkward String::replace API which was the only String
API which mutated the String and replaces it with a new immutable
version that returns a new String with the replacements applied. This
also fixes a couple of UAFs that were caused by the use of this API.

As an optimization an equivalent StringView::replace API was also added
to remove an unnecessary String allocations in the format of:
`String { view }.replace(...);`
2021-09-11 20:36:43 +03:00
Ben Wiederhake
2e4ec891da Everywhere: Fix format-vulnerabilities
Command used:
grep -Pirn '(out|warn)ln\((?!["\)]|format,|stderr,|stdout,|output, ")' \
     AK Kernel/ Tests/ Userland/
(Plus some manual reviewing.)

Let's pick ArgsParser as an example:
    outln(file, m_general_help);
This will fail at runtime if the general help happens to contain braces.

Even if this transformation turns out to be unnecessary in a place or
two, this way the code is "more obviously" correct.
2021-09-11 15:16:26 +01:00
Brian Gianforcaro
a4efaa7b47 Tests/Kernel: Fix test after off-by-one fix in Memory::is_user_range()
Commit 890c647e0f fixed an off-by-one bug, so the mapping of the page
at the very end of the user address space now works correctly.

This change adjusts the test so cover the corner cases the original
version was designed too.validate.
2021-09-11 04:15:16 +00:00
Ali Mohammad Pur
14c8373eb0 AK+Kernel: Reduce the number of template parameters of IntrusiveRBTree
This makes the user-facing type only take the node member pointer, and
lets the compiler figure out the other needed types from that.
2021-09-10 18:05:46 +03:00
Ali Mohammad Pur
5a0cdb15b0 AK+Everywhere: Reduce the number of template parameters of IntrusiveList
This makes the user-facing type only take the node member pointer, and
lets the compiler figure out the other needed types from that.
2021-09-10 18:05:46 +03:00
Timothy Flynn
4f2bcebe74 LibUnicode+LibJS: Store locale keyword values as a single string
Previously, LibUnicode would store the values of a keyword as a Vector.
For example, the locale "en-u-ca-abc-def" would have its keyword "ca"
stored as {"abc, "def"}. Then, canonicalization would occur on each of
the elements in that Vector.

This is incorrect because, for example, the keyword value "true" should
only be dropped if that is the entire value. That is, the canonical form
of "en-u-kb-true" is "en-u-kb", but "en-u-kb-abc-true" does not change
for canonicalization. However, we would canonicalize that locale as
"en-u-kb-abc".
2021-09-08 21:08:48 +01:00
Idan Horowitz
cb9720baab AK: Set IntrusiveRBTree Node key on insertion instead of construction
This makes the API look much nicer.
2021-09-08 19:17:07 +03:00
Idan Horowitz
1db9250766 AK: Make IntrusiveRedBlackTree capable of holding non-raw pointers
This is completely based on e4412f1f59
and will allow us to convert some AK::HashMap users in the kernel.
2021-09-08 19:17:07 +03:00
davidot
43b17f27a3 test-js: Add a mark_as_garbage method to force GC to collect that object
This should fix the flaky tests of test-js.
It also fixes the tests when running with the -g flag since the values
will not be garbage collected too soon.
2021-09-08 08:53:02 +01:00
Ali Mohammad Pur
7fefb8148b LibRegex: Use the correct capture group index in ERE bytecode generation
Otherwise the left and right capture instructions wouldn't point to the
same capture group if there was another nested group there.
2021-09-07 20:01:58 +02:00
Andreas Kling
6ad427993a Everywhere: Behaviour => Behavior 2021-09-07 13:53:14 +02:00
Timothy Flynn
50158abaf1 LibUnicode: Implement locale-aware BEFORE_DOT special casing
Note that the algorithm in the Unicode spec is for checking that a code
point precedes U+0307, but the special casing condition NotBeforeDot is
interested in the inverse of this rule.
2021-09-06 15:24:27 +01:00
Timothy Flynn
436faf9fd9 LibUnicode: Implement locale-aware MORE_ABOVE special casing 2021-09-06 15:24:27 +01:00
Timothy Flynn
1427ebc622 LibUnicode: Implement locale-aware AFTER_SOFT_DOTTED special casing 2021-09-06 15:24:27 +01:00
Timothy Flynn
0053d48c41 LibUnicode: Implement locale-aware AFTER_I special casing 2021-09-06 15:24:27 +01:00
Ali Mohammad Pur
63523d3836 Tests/LibRegex: Decrease the size of the fork chain test 2021-09-06 13:51:30 +04:30
Ali Mohammad Pur
abbe9da255 LibRegex: Make infinite repetitions short-circuit on empty matches
This makes (addmittedly weird) patterns like `(a*)*` work correctly
without going into an infinite fork loop.
2021-09-06 13:51:30 +04:30
Ali Mohammad Pur
97e97bccab Everywhere: Make ByteBuffer::{create_*,copy}() OOM-safe 2021-09-06 01:53:26 +02:00
Ali Mohammad Pur
3a9f00c59b Everywhere: Use OOM-safe ByteBuffer APIs where possible
If we can easily communicate failure, let's avoid asserting and report
failure instead.
2021-09-06 01:53:26 +02:00
Brian Gianforcaro
782e7834c3 Tests: Reduce the execution time of the LibC TestQSort test
Executing 100 times vs 10 times doesn't increase test coverage
substantial, this API is stable and more iterations is just a
waste of time.

Without KVM this test is a clear outlier in runtime during CI:

```
START  LibC/TestQsort (106/172)
PASS   LibC/TestQsort (41.233692s)
```
2021-09-05 22:17:28 +02:00
Andreas Kling
7dda773426 AK: Add rvalue-ref qualifiers for Optional's value() and value_or()
This avoids a value copy when calling value() or value_or() on a
temporary Optional. This is very common when using the HashMap::get()
API like this:

    auto value = hash_map.get(key).value_or(fallback_value);
2021-09-04 03:02:08 +02:00
Daniel Bertalan
d7b6cc6421 Everywhere: Prevent risky implicit casts of (Nonnull)RefPtr
Our existing implementation did not check the element type of the other
pointer in the constructors and move assignment operators. This meant
that some operations that would require explicit casting on raw pointers
were done implicitly, such as:
- downcasting a base class to a derived class (e.g. `Kernel::Inode` =>
  `Kernel::ProcFSDirectoryInode` in Kernel/ProcFS.cpp),
- casting to an unrelated type (e.g. `Promise<bool>` => `Promise<Empty>`
  in LibIMAP/Client.cpp)

This, of course, allows gross violations of the type system, and makes
the need to type-check less obvious before downcasting. Luckily, while
adding the `static_ptr_cast`s, only two truly incorrect usages were
found; in the other instances, our casts just needed to be made
explicit.
2021-09-03 23:20:23 +02:00
Timothy Flynn
a05419db55 LibUnicode: Add lexer to test if a string matches the "type" production 2021-09-02 17:56:42 +01:00
Andrew Kaster
58797a1289 Tests: Remove all file(GLOB) from CMakeLists in Tests
Using a file(GLOB) to find all the test files in a directory is an easy
hack to get things started, but has some drawbacks. Namely, if you add
a test, it won't be found again without re-running CMake. `ninja` seems
to do this automatically, but it would be nice to one day stop seeing it
rechecking our globbed directories.
2021-09-02 09:08:23 +02:00
sin-ack
b5a145b466 Tests: Add tests for Core::deferred_invoke 2021-09-02 03:47:47 +04:30
Timothy Flynn
fd0011989a LibUnicode: Resolve the most likely territory alias when there are many 2021-09-01 14:14:47 +01:00
Timothy Flynn
72f49e42b4 LibUnicode: Perform complex Unicode locale alias substitution 2021-09-01 14:14:47 +01:00
Timothy Flynn
da89cf9afb LibUnicode: Canonicalize calendar subtags
Calendar subtags are a bit of an odd-man-out in that we must match the
variants "ethiopic-amete-alem" in that order, without any other variant
in the locale. So a separate method is needed for this, and we now defer
sorting the variant list until after other canonicalization is done.
2021-09-01 14:14:47 +01:00
Timothy Flynn
8458f477a4 LibUnicode: Canonicalize timezone subtags 2021-09-01 14:14:47 +01:00
Timothy Flynn
335f985b31 LibUnicode: Canonicalize the subtag "imperial" to "uksystem" 2021-09-01 14:14:47 +01:00
Timothy Flynn
2d90144888 LibUnicode: Canonicalize the subtag "primary" and "tertiary" to "levelN" 2021-09-01 14:14:47 +01:00
Timothy Flynn
409f39b336 LibUnicode: Canonicalize the subtag "names" to "prprname" 2021-09-01 14:14:47 +01:00
Timothy Flynn
f907a7dc38 LibUnicode: Canonicalize the subtag "yes" to "true" 2021-09-01 14:14:47 +01:00
Timothy Flynn
556374a904 LibUnicode: Substitute Unicode locale aliases during canonicalization
Unicode TR35 defines how locale subtag aliases should be emplaced when
converting a locale to canonical form. For most subtags, it is a simple
substitution. Language subtags depend on context; for example, the
language "sh" should become "sr-Latn", but if the original locale has a
script subtag already ("sh-Cyrl"), then only the language subtag of the
alias should be taken ("sr-Latn").

To facilitate this, we now make two passes when canonicalizing a locale.
In the first pass, we convert the LocaleID structure to canonical syntax
(where the conversions all happen in-place). In the second pass, we form
the canonical string based on the canonical syntax.
2021-09-01 14:14:47 +01:00
Timothy Flynn
d13142f015 LibJS+LibUnicode: Store parsed Unicode locale data as full strings
Originally, it was convenient to store the parsed Unicode locale data as
views into the original string being parsed. But to implement locale
aliases will require mutating the data that was parsed. To prepare for
that, store the parsed data as proper strings.
2021-09-01 14:14:47 +01:00
Andrew Kaster
b3e3e4d45d Tests: Convert remaining LibC tests to LibTest
Convert them to using outln instead of printf at the same time.
2021-09-01 13:44:24 +02:00
Peter Elliott
33d7fdca28 Everywhere: Use my cool new @serenityos.org email address 2021-09-01 11:37:25 +04:30
Peter Elliott
8d2c04821f Tests: Test LibMarkdown against commonmark test suite
TestCommonmark runs the CommonMark test suite
(https://spec.commonmark.org/0.30/spec.json) against LibMarkdown.
Currently 44/652 tests pass.
2021-08-31 16:53:51 +02:00
Hediadyoin1
fdef6e5f76 AK: Add FixedPoint arithmetic helper
Co-authored-by: Hendiadyoin1 <leon2002.la@gmail.com>
Co-authored-by: kleines Filmröllchen <malu.bertsch@gmail.com>
2021-08-31 17:03:55 +04:30
Ali Mohammad Pur
30a1a25daa Tests/LibWasm: Handle all stream errors in parse_webassembly_module 2021-08-30 22:47:02 +02:00
Ali Mohammad Pur
99199b9bfd Tests/LibWasm: Add support for javascript bigint values
Some i64 values will not fit in normal doubles, and these values _are_
tested by the test suite, this makes the test runtime capable of
handling them correctly.
2021-08-30 22:47:02 +02:00
Timothy Flynn
f897c2edb3 LibUnicode: Canonicalize locale private use extensions 2021-08-30 19:42:40 +01:00