This implements Optional<T&> as a T*, whose presence has been missing
since the early days of Optional.
As a lot of find_foo() APIs return an Optional<T> which imposes a
pointless copy on the underlying value, and can sometimes be very
misleading, with this change, those APIs can return Optional<T&>.
This is in Tests/LibTTF instead of Tests/LibGfx because Tests/LibGfx
depends on serenity's file system layout and can't run in lagom,
but this new test runs just fine in lagom.
This caused a system-wide crash because of a previous bug relating to
non-trivial types in HashTable. Therefore, check that such types
actually work under various workloads.
Thrashing is what I call the situations where a table is mostly filled
with deleted markers, causing an increase in size (at least temporarily)
when a simple re-hash would be enough to get rid of those. This happens
when a hash table (especially with many elements) has a lot of deletes
and re-inserts done to it, which is what this benchmark does.
Security handlers manage encryption and decription of PDF files. The
standard security handler uses RC4/MD5 to perform its crypto (AES as
well, but that is not yet implemented).
This is an enum-like type that works with arbitrary sized storage > u64,
which is the limit for a regular enum class - which limits it to 64
members when needing bit field behavior.
Co-authored-by: Ali Mohammad Pur <mpfard@serenityos.org>
The SHA384 and SHA512 hashes would produce incorrect results for data
where the length % 128 was in the range 112-119. This was because the
total number of bits in the hashed values was added at the end as a
64-bit number instead of a 128-bit number. In most cases this would not
cause any issues, as this space was padded with zeroes, however in the
case that the length % 128 was 112-119, some incorrect data ended up
where this 128-bit length value was expected.
This change fixes the problems in LibTLS where some websites would
result in a DecryptError on handshake.
This converts the return value of File::read_link() from String to
ErrorOr<String>.
The rest of the change is to support the potential of an Error being
returned and subsequent release of the value when no Error is returned.
Unfortunately at this stage none of the places affected can utililize
our TRY() macro.
Add the required methods to SECP256r1 to conform to the EllipticCurve
virtual base class. Using this updated version of SECP256r1, support in
LibTLS is implemented.
These changes generalize the interface with an elliptic curve
implementation. This allows LibTLS to support elliptic curves generally
without needing the specifics of elliptic curve implementations.
This should allow for easier addition of other elliptic curves.
Previously, case-insensitively searching the haystack "Go Go Back" for
the needle "Go Back" would return false:
1. Match the first three characters. "Go ".
2. Notice that 'G' and 'B' don't match.
3. Skip ahead 3 characters, plus 1 for the outer for-loop.
4. Now, the haystack is effectively "o Back", so the match fails.
Reducing the skip by 1 fixes this issue. I'm not 100% convinced this
fixes all cases, but I haven't been able to find any cases where it
doesn't work now. :^)
This implementation of the secp256r1 elliptic curve uses two techniques
to improve the performance of the operations.
1. All coordinates are stored in Jacobian form, (X/Z^2, Y/Z^3, Z), which
removes the need for division operations during point addition or
doubling. The points are converted at the start of the computation,
and converted back at the end.
2. All values are transformed to Montgomery form, to allow for faster
modular multiplication using the Montgomery modular multiplication
method. This means that all coordinates have to be converted into
this form, and back out of this form before returning them.
It makes no sense to require passing a global object and doing a stack
space check in some cases where running out of stack is highly unlikely,
we can't recover from errors, and currently ignore the result anyway.
This is most commonly in constructors and when setting things up, rather
than regular function calls.
This is the IPv6 counter part to the IPv4Address class and implements
parsing strings into a in6_addr and formatting one as a string. It
supports the address compression scheme as well as IPv4 mapped
addresses.
This necessitated making HTMLParser ref-counted, and having it register
itself with Document when created. That makes it possible for scripts to
add new input at the current parser insertion point.
There is now a reference cycle between Document and HTMLParser. This
cycle is explicitly broken by calling Document::detach_parser() at the
end of HTMLParser::run().
This is a huge progression on ACID3, from 31% to 49%! :^)
Also, update the expected hash in the LibWeb TestHTMLTokenizer
regression test.
This is due to the "This comment has a few too many dashes." comment
token being updated.
While quantifying assertions is very much meaningless, the specification
allows them with annex B's extended grammar for browsers, so read and
apply the quantifiers.
Fixes#12373.
I've attempted to handle the errors gracefully where it was clear how to
do so, and simple, but a lot of this was just adding
`release_value_but_fixme_should_propagate_errors()` in places.
Parse JSON floating point literals properly,
No longer throwing a SyntaxError when the decimal portion
of the number exceeds the capacity of u32.
Added tests to AK/TestJSON and LibJS/builtins/JSON/JSON.parse
Commit b193351a99 caused the HTML comments to flash when changing
the text cursor. Also, when double-clicking on a comment, the selection
started from the beginning of the file instead.
The following message was displaying when `TOKENIZER_TRACE_DEBUG`
was enabled:
(Tokenizer::nth_last_position) Invalid position requested: 4th-last
of 4. Returning (0-0).
Changing the `nth_last_position` to 3 fixes this. I'm guessing that's
because the parser is at that moment on the second hyphen of the `<!--`
string, so it has to go back only by three characters.
Now that expression evaluation can use TRY, we can allow binary operator
methods to fail as well. This also fixes a few instances of converting a
Value to a double when we meant to convert to an integer.
Putting everything in the global scope will lead to mayhem and failing
tests with an actually correct implementation of scoping :^)
Also adds in a tiny debug log of the exception, otherwise we'd be
staring at failing tests with no info on what failed.
Instead of crashing on the spot, return a descriptive error that will
eventually continue its days as a javascript "InternalError" exception.
This should make random crashes with BC less likely.
The result of a SQL statement execution is either:
1. An error.
2. The list of rows inserted, deleted, selected, etc.
(2) is currently represented by a combination of the Result class and
the ResultSet list it holds. This worked okay, but issues start to
arise when trying to use Result in non-statement contexts (for example,
when introducing Result to SQL expression execution).
What we really need is for Result to be a thin wrapper that represents
both (1) and (2), and to not have any explicit members like a ResultSet.
So this commit removes ResultSet from Result, and introduces ResultOr,
which is just an alias for AK::ErrorOrr. Statement execution now returns
ResultOr<ResultSet> instead of Result. This further opens the door for
expression execution to return ResultOr<Value> in the future.
Lastly, this moves some other context held by Result over to ResultSet.
This includes the row count (which is really just the size of ResultSet)
and the command for which the result is for.
Before this was incorrectly assuming that if the current node `n` was at
least the key and the left child of `n` was below the key that `n` was
always correct.
However, the right child(ren) of the left child of `n` could still be
at least the key.
Also added some tests which produced the wrong results before this.
The environment settings object is effectively the context a piece of
script is running under, for example, it contains the origin,
responsible document, realm, global object and event loop for the
current context. This effectively replaces ScriptExecutionContext, but
it cannot be removed in this commit as EventTarget still depends on it.
https://html.spec.whatwg.org/multipage/webappapis.html#environment-settings-object
This commit removes all exception related code:
Remove VM::exception(), VM::throw_exception() etc. Any leftover
throw_exception calls are moved to throw_completion.
The one method left is clear_exception() which is now a no-op. Most of
these calls are just to clear whatever exception might have been thrown
when handling a Completion. So to have a cleaner commit this will be
removed in a next commit.
It also removes the actual Exception and TemporaryClearException classes
since these are no longer used.
In any spot where the exception was actually used an attempt was made to
preserve that behavior. However since it is no longer tracked by the VM
we cannot access exceptions which were thrown in previous calls.
There are two such cases which might have different behavior:
- In Web::DOM::Document::interpreter() the on_call_stack_emptied hook
used to print any uncaught exception but this is now no longer
possible as the VM does not store uncaught exceptions.
- In js the code used to be interruptable by throwing an exception on
the VM. This is no longer possible but was already somewhat fragile
before as you could happen to throw an exception just before a VERIFY.
If a big integer were to become negative zero, set the sign to instead
be positive. This prevents odd scenarios where users of signed big ints
would falsely think the result of some big int arithmetic is negative.
This commit converts TLS::TLSv12 to a Core::Stream object, and in the
process allows TLS to now wrap other Core::Stream::Socket objects.
As a large part of LibHTTP and LibGemini depend on LibTLS's interface,
this also converts those to support Core::Stream, which leads to a
simplification of LibHTTP (as there's no need to care about the
underlying socket type anymore).
Note that RequestServer now controls the TLS socket options, which is a
better place anyway, as RS is the first receiver of the user-requested
options (though this is currently not particularly useful).
ECMA-262 defines \s as:
Return the CharSet containing all characters corresponding to a code
point on the right-hand side of the WhiteSpace or LineTerminator
productions.
The LineTerminator production is simply: U+000A, U+000D, U+2028, or
U+2029. Unfortunately there isn't a Unicode property that covers just
those code points.
The WhiteSpace production is: U+0009, U+000B, U+000C, U+FEFF, or any
code point with the Space_Separator general category.
If the Unicode generators are disabled, this will fall back to ASCII
space code points.
LibRegex already implements this loop in a more performant way, so all
LibJS has to do here is to return things in the right shape, and not
loop over the input string.
Previously this was a quadratic operation on string length, which lead
to crazy execution times on failing regexps - now it's nice and fast :^)
Note that a Regex test has to be updated to remove the stateful flag as
it repeats matching on multiple strings.
Using an Optional was extremely wasteful for function objects that don't
even have a bytecode executable.
This allows ECMAScriptFunctionObject to fit in a smaller size class.
These tests are not meant as a replacement to test-js with the -b option
but are meant to test simple cases until that works.
Before this it was very easy to accidentally break bytecode since no
tests were run in bytecode mode. This hopefully makes it easier to spot
such regressions :^).
As ECMA262 regex allows `[^]` and literal newlines to match newlines in
the input string, we shouldn't split the input string into lines, rather
simply make boundaries and catchall patterns capable of checking for
these conditions specifically.
Apologies for the enormous commit, but I don't see a way to split this
up nicely. In the vast majority of cases it's a simple change. A few
extra places can use TRY instead of manual error checking though. :^)
We weren't properly iterating the extension blocks and thought we
encountered an unexpected extension map block, when we really should
have just skipped over it.
Reverts recent change introduced to support implicit symbolic permission
which broke the parser when multiple classes are specified.
The state machine must assume it's dealing with classes until an
operation character is consumed.
This library can be used (for the most part) by kernel drivers as well
as user mode. For this reason FixedPoint is used rather than floating
point, but kept to a minimum.
Rather than casting the FixedPoint to double, format the FixedPoint
directly. This avoids using floating point instruction, which in
turn enables this to be used even in the kernel.
The implementation of LIKE uses regexes under the hood, and this
implementation of REGEXP takes the same approach. It employs
PosixExtended from LibRegex with case insensitive and Unicode flags
set. The implementation of LIKE is based on SQLlite specs, but SQLlite
does not offer directions for a built-in regex functionality, so this
one uses LibRegex.
The event loop system was previously very singletony to the point that
there's only a single event loop stack per process and only one event
loop (the topmost) can run at a time. This commit simply makes the event
loop stack and related structures thread-local so that each thread has
an isolated event loop system.
Some things are kept at a global level and synchronized with the new
MutexProtected: The main event loop needs to still be obtainable from
anywhere, as it closes down the application when it exits. The ID
allocator is global as IDs should not be shared even between threads.
And for the inspector server connection, the same as for the main loop
holds.
Note that currently, the wake pipe is only created by the main thread,
so notifications don't work on other threads.
This removes the temporary mutex fix for notifiers, introduced in
0631d3fed5 .
This is no longer needed now that LibTimeZone is included within LibC.
Remove the direct linkage so that others do not mistakenly copy-paste
the CMakeLists text elsewhere.
Instead of leaking all capture groups and selectively clearing some,
simply avoid leaking things and only "define" the ones that need to
exist.
This *actually* implements the capture groups ECMA262 quirk.
Also adds the test removed in the previous commit (to avoid messing up
test runs across bisects).
This partially reverts commit c11be92e23.
That commit fixes one thing and breaks many more, a next commit will
implement this quirk in a more sane way.
Previously we were jumping to the new end of the previous block (created
by the newly inserted ForkStay), correct the offset to jump to the
correct block as shown in the comments.
Fixes#12033.
This test makes sure that Socket classes such as TCPSocket properly
return an error when connection fails rather than crashing or creating
an invalid object.
Accidentally regressed this test during the Core::LocalServer refactor,
and didn't catch it since TestLibCoreStream is disabled in the CI right
now. We have to wait for some data to become available, as pending_bytes
will immediately return 0 and a 0-sized read immediately returns.
We went through some trouble to make & and | work right. Reimplement ^
in terms of & and | to make ^ work right as well.
This is less fast than a direct implementation, but let's get things
working first.
Similar to the bitwise_and change, but we have to be careful to
sign-extend two's complement numbers only up to the highest set bit
in the positive number.
Bitwise and is defined in terms of two's complement, so some converting
needs to happen for SignedBigInteger's sign/magnitude representation to
work out.
UnsignedBigInteger::bitwise_not() is repurposed to convert all
high-order zero bits to ones up to a limit, for the two's complement
conversion to work.
Fixes test262/test/language/expressions/bitwise-and/bigint.js.
Bitwise operators are defined on two's complement, but SignedBitInteger
uses sign-magnitude. Correctly convert between the two.
Let LibJS delegate to SignedBitInteger for bitwise_not, like it does
for all other bitwise_ operations on bigints.
No behavior change (LibJS is now the only client of
SignedBitInteger::bitwise_not()).
Ordering is done by replacing the straight Vector holding the query
result in the SQLResult object with a dedicated Vector subclass that
inserts result rows according to their sort key using a binary search.
This is done in the ResultSet class.
There are limitations:
- "SELECT ... ORDER BY 1" (or 2 or 3 etc) is supposed to sort by the
n-th result column. This doesn't work yet
- "SELECT ... column-expression alias ... ORDER BY alias" is supposed to
sort by the column with the given alias. This doesn't work yet
What does work however is something like
```SELECT foo FROM bar SORT BY quux```
i.e. sorted by a column not in the result set. Once functions are
supported it should be possible to sort by random functions.
This change unfortunately cannot be atomically made without a single
commit changing everything.
Most of the important changes are in LibIPC/Connection.cpp,
LibIPC/ServerConnection.cpp and LibCore/LocalServer.cpp.
The notable changes are:
- IPCCompiler now generates the decode and decode_message functions such
that they take a Core::Stream::LocalSocket instead of the socket fd.
- IPC::Decoder now uses the receive_fd method of LocalSocket instead of
doing system calls directly on the fd.
- IPC::ConnectionBase and related classes now use the Stream API
functions.
- IPC::ServerConnection no longer constructs the socket itself; instead,
a convenience macro, IPC_CLIENT_CONNECTION, is used in place of
C_OBJECT and will generate a static try_create factory function for
the ServerConnection subclass. The subclass is now responsible for
passing the socket constructed in this function to its
ServerConnection base; the socket is passed as the first argument to
the constructor (as a NonnullOwnPtr<Core::Stream::LocalServer>) before
any other arguments.
- The functionality regarding taking over sockets from SystemServer has
been moved to LibIPC/SystemServerTakeover.cpp. The Core::LocalSocket
implementation of this functionality hasn't been deleted due to my
intention of removing this class in the near future and to reduce
noise on this (already quite noisy) PR.
This makes the following code behave as expected:
Variant<int, String> x { some_string() };
x.visit(
[](String const&) {}, // Expectation is for this to be called
[](auto&) {});
As per previous discussion, it was decided that the Stream classes
should be constructed on the heap.
While I don't personally agree with this change, it does have the
benefit of avoiding Function object reconstructions due to the lambda
passed to Notifier pointing to a stale object reference. This also has
the benefit of not having to "box" objects for virtual usage, as the
objects come pre-boxed.
However, it means that we now hit the heap everytime we construct a
TCPSocket for instance, which might not be desirable.
Except for tangential accessors such as data(), there is no more feature
of FixedArray that is untested after this large expansion of its test
cases. These tests, with the help of the new NoAllocationGuard, also
test the allocation contract that was fixated in the last commit.
Hopefully this builds confidence in future Kernel uses of FixedArray
as well as its establishment in the real-time parts of the audio
subsystem. I'm excited :^)
FixedArray always *almost* had the following allocation guarantees:
There is (possibly) one allocation in the constructor and one (or more)
deallocation(s) in the destructor. No other operation allocates or
deallocates. With this removal of the public clear() method, which
nobody except the test used anyways, those guarantees are now completely
true and furthermore fixated with an explanatory comment.
Our generator is currently preferring the DST variant of the time zone
display names over the non-DST variant. LibTimeZone currently does not
have DST support, and operates in a mode that basically assumes DST does
not exist. Swap the display names for now just to be consistent until we
have DST support.
Note we will need to generate both of these variants and select the
appropriate one at runtime once we have DST support.
The following table in TR-35 includes a web of fall back rules when the
requested time zone style is unavailable:
https://unicode.org/reports/tr35/tr35-dates.html#dfst-zone
Conveniently, the subset of styles supported by ECMA-402 (and therefore
LibUnicode) all either fall back to GMT offset or to a style that is
unsupported but itself falls back to GMT offset.
This adds an API to use LibTimeZone to convert a time zone such as
"America/New_York" to a GMT offset string like "GMT-5" (short form) or
"GMT-05:00" (long form).
Instead of only having dummy functions that don't work with any input,
let's at least support one time zone: 'UTC'. This matches the basic
Temporal implementation for engines without ECMA-262, for example.
This mechanism was unsafe to use in any multithreaded context, since
the hook function was invoked on a raw pointer *after* decrementing
the local ref count.
Since we don't use it for anything anymore, let's just get rid of it.
This is a rather naive implementation, but serves as a first pass at
determining the GMT offset for a time zone at a particular point in
time. This implementation ignores DST (because we are not parsing any
RULE entries yet), and ignores any offset patterns of the form "Mon>4"
or "lastSun".
Currently, we define a CaseInsensitiveStringTraits structure for String.
Using this structure for StringView involves allocating a String from
that view, and a second string to convert that intermediate string to
lowercase.
This defines CaseInsensitiveStringViewTraits (and the underlying helper
case_insensitive_string_hash) to avoid allocations.
FixedArray now doesn't expose any infallible constructors anymore.
Rather, it exposes fallible methods. Therefore, it can be used for
OOM-safe code.
This commit also converts the rest of the system to use the new API.
However, as an example, VMObject can't take advantage of this yet,
as we would have to endow VMObject with a fallible static
construction method, which would require a very fundamental change
to VMObject's whole inheritance hierarchy.
Add a unit test for each sample pdf file that currently exists in the
anon user's `~/Document/pdf` directory.
- linear.pdf
- non-linearized.pdf
- complex.pdf
Each test ensures that the pdf document is parsed and that the page
count is the expected one.
So far we only had mmap(2) functionality on the /dev/mem device, but now
we can also do read(2) on it.
The test unit was updated to check we are doing it safely.
The previous implementation had some pretty short cycles and two fixed
points (1711463637 and 2389024350). If two keys hashed to one of these
values insertions and lookups would loop forever.
This version is based on a standard xorshift PRNG with period 2**32-1.
The all-zero state is usually forbidden, so we insert it into the cycle
at an arbitrary location.
The evaluation order of method parameters is unspecified in C++, and
so we couldn't rely on parse_statement() being called before
parse_escape() when building a MatchExpression.
With this patch, we explicitly parse what we need in the right order,
before building the MatchExpression object.
The generator parses metaZones.json to form a mapping of meta zones to
time zones (AKA "golden zone" in TR-35). This parser errantly assumed
this was a 1-to-1 mapping.
As it was, negative predicate test for remove_all_matching was
run on empty hash map, and could not remove anything, so test always
returned true. By duplicating it in state where hash maps contains
elements, we make sure that negative predicate has something to
do nothing on.
These were missed in 565a880ce5.
This wasn't an issue because these tests don't pledge/unveil anything,
so they could happily dlopen() the library at runtime. But this is now
needed in order to migrate LibUnicode towards weak symbols instead.
This was currently crashing Half-Life because it was a considered an
"Unknown" specifier. We can use the same case statement as the regular
hex format conversion (lower case 'x'), as the backend
to convert the number already supports upper/lower case input, hence
we get it for free :^)
This exposed a missing exception check in parseWebAssemblyModule(),
which could throw but still return a normal completion (which currently
works as we check VM::exception() at the right point, but breaks when
moving everything to exceptions).
The spec has a note stating that resolve binding will always return a
reference whose [[ReferencedName]] field is name. However this is not
correct as the underlying method GetIdentifierReference may throw on
env.HasBinding(name) thus it can throw. However, there are some
scenarios where it cannot throw because the reference is known to exist
in that case we use MUST with a comment.
Previously we might swallow invalid unicode point which would skip valid
ascii characters. This could be dangerous as we might skip a '"' thus
not closing a string where we should.
This might have been exploitable as it would not have been clear what
code gets executed when looking at a script.
Another approach to this would be simply replacing all invalid
characters with the replacement character (this is what v8 does). But
our lexer and parser are currently not set up for such a change.
It was possible for the "local_socket_read" and "local_socket_write"
tests to fail because we had exited the EventLoop before
BackgroundAction got around to invoking the completion callback.
The crash happened when trying to deferred_invoke() on the background
thread, calling Core::EventLoop::current() after said EventLoop had
returned from exec().
Fix this by not passing a completion callback, since we didn't need
one in the first place.
This is a raffinement of 49cbd4dcca.
Previously, the container was scanned to compute the size in the unhappy
path. Now, using `all_of` happy and unhappy path should be fast.
ISO C requires in section 7.2:
The assert macro is redefined according to the current state of NDEBUG
each time that <assert.h> is included.
Also add tests for `assert` multiple inclusion accordingly.
For setreuid and setresuid syscalls, -1 means to set the current
uid/euid/gid/egid value, to be more convenient for programming.
However, for other syscalls where we pass only one argument, there's no
justification to specify -1.
This behavior is identical to how Linux handles the value -1, and is
influenced by the fact that the manual pages for the group of one
argument syscalls that handle ID operations is ambiguous about this
topic.
The goal of this file is to enable C++ overloaded functions for
standard builtin functions that we use. It contains fallback
implementations for systems that do not have the builtins available.
This unbreaks the /var/run/utmp system which starts out as an empty
string, and is then turned into an object by the first update.
This isn't necessarily the best way for this to work, but it's how
it used to work, so this just fixes the regression for now.
