While adding spec comments to PerformEval, I noticed we were missing
multiple steps.
Namely, these were:
- Checking if the host will allow us to compile the string
(allowing LibWeb to perform CSP for eval)
- The parser's initial state depending on the environment around us
on direct eval:
- Allowing new.target via eval in functions
- Allowing super calls and super properties via eval in classes
- Disallowing the use of the arguments object in class field
initializers at eval's parse time
- Setting ScriptOrModule of eval's execution context
The spec allows us to apply the additional parsing steps in any order.
The method I have gone with is passing in a struct to the parser's
constructor, which overrides the parser's initial state to (dis)allow
the things stated above from the get-go.
It makes no sense to require passing a global object and doing a stack
space check in some cases where running out of stack is highly unlikely,
we can't recover from errors, and currently ignore the result anyway.
This is most commonly in constructors and when setting things up, rather
than regular function calls.
This allows the host of LibJS (notably LibWeb in this case) to override
certain functions such as HostEnqueuePromiseJob, so it can do it's own
thing in certain situations. Notably, LibWeb will override
HostEnqueuePromiseJob to put promise jobs on the microtask queue.
This also makes promise jobs use AK::Function instead of
JS::NativeFunction. This removes the need to go through a JavaScript
function and it more closely matches the spec's idea of "abstract
closures"
Since VM::exception() no longer exists this is now useless. All of these
calls to clear_exception were just to clear the VM state after some
(potentially) failed evaluation and did not use the exception itself.
This commit removes all exception related code:
Remove VM::exception(), VM::throw_exception() etc. Any leftover
throw_exception calls are moved to throw_completion.
The one method left is clear_exception() which is now a no-op. Most of
these calls are just to clear whatever exception might have been thrown
when handling a Completion. So to have a cleaner commit this will be
removed in a next commit.
It also removes the actual Exception and TemporaryClearException classes
since these are no longer used.
In any spot where the exception was actually used an attempt was made to
preserve that behavior. However since it is no longer tracked by the VM
we cannot access exceptions which were thrown in previous calls.
There are two such cases which might have different behavior:
- In Web::DOM::Document::interpreter() the on_call_stack_emptied hook
used to print any uncaught exception but this is now no longer
possible as the VM does not store uncaught exceptions.
- In js the code used to be interruptable by throwing an exception on
the VM. This is no longer possible but was already somewhat fragile
before as you could happen to throw an exception just before a VERIFY.
The hard part of parsing them in import statements and calls was already
done so this is just removing some check which threw before on
assertions. And filtering the assertions based on the result of a new
host hook.
This allows us to load modules from scripts.
This can be dangerous as it can load arbitrary files. Because of that it
fails and throws by default. Currently, only js and JavaScriptTestRunner
enable the default hook.
This also adds tests to test-js which test module code. Because we
form a spec perspective can't "enter" a module this is the easiest way
to run tests without having to modify test-js to have special cases for
modules.
To specify modules in test-js we use the extension '.mjs' this is to
ensure the files are not executed. We do still want to lint these files
so the prettier scripts have changed to look for '.mjs' files as well.
This loads modules with relative paths from the referencing module.
In this commit the only way to run a module is via the interpreter
which can link and evaluate a module (and all its dependencies).
test-js crashes with a segmentation fault when running on macOS on Arm.
Increasing the margin in the test in did_reach_stack_space_limit() to
32 * KiB makes the tests pass. To simplify the code, this is applied
independently of platform, and the previous test for use of an address
sanitizer is removed.
Instead of using plain objects as Iterator records, causes confusion
about the object itself actually being its [[Iterator]] slot, and
requires non-standard type conversion shenanigans fpr the [[NextValue]]
and [[Done]] internal slots, implement a proper Iterator record struct
and use it throughout.
Also annotate the remaining Iterator AOs with spec comments while we're
here.
The spec has a note stating that resolve binding will always return a
reference whose [[ReferencedName]] field is name. However this is not
correct as the underlying method GetIdentifierReference may throw on
env.HasBinding(name) thus it can throw. However, there are some
scenarios where it cannot throw because the reference is known to exist
in that case we use MUST with a comment.
Now that it only needs to deal with ECMAScriptFunctionObject via
internal_call() / internal_construct(), we can:
- Remove the generic FunctionObject parameter
- Move it from the VM to ECMAScriptFunctionObject
- Make it private
Now that it only needs to deal with ECMAScriptFunctionObject via
internal_call() / internal_construct(), we can:
- Remove the generic FunctionObject parameter
- Move it from the VM to ECMAScriptFunctionObject
- Make it private
This patch implements:
- Spec compliant [[Call]] and [[Construct]] internal slots, as virtual
FunctionObject::internal_{call,construct}(). These effectively replace
the old virtual FunctionObject::{call,construct}(), but with several
advantages:
- Clear and consistent naming, following the object internal methods
- Use of completions
- internal_construct() returns an Object, and not Value! This has been
a source of confusion for a long time, since in the spec there's
always an Object returned but the Value return type in LibJS meant
that this could not be fully trusted and something could screw you
over.
- Arguments are passed explicitly in form of a MarkedValueList,
allowing manipulation (BoundFunction). We still put them on the
execution context as a lot of code depends on it (VM::arguments()),
but not from the Call() / Construct() AOs anymore, which now allows
for bypassing them and invoking [[Call]] / [[Construct]] directly.
Nothing but Call() / Construct() themselves do that at the moment,
but future additions to ECMA262 or already existing web specs might.
- Spec compliant, standalone Call() and Construct() AOs: currently the
closest we have is VM::{call,construct}(), but those try to cater to
all the different function object subclasses at once, resulting in a
horrible mess and calling AOs with functions they should never be
called with; most prominently PrepareForOrdinaryCall and
OrdinaryCallBindThis, which are only for ECMAScriptFunctionObject.
As a result this also contains an implicit optimization: we no longer
need to create a new function environment for NativeFunctions - which,
worth mentioning, is what started this whole crusade in the first place
:^)
This patch introduces the "environment coordinate" concept, which
encodes the distance from a variable access to the binding it ends up
resolving to.
EnvironmentCoordinate has two fields:
- hops: The number of hops up the lexical environment chain we have
to make before getting to the resolved binding.
- index: The index of the resolved binding within its declarative
environment record.
Whenever a variable lookup resolves somewhere inside a declarative
environment, we now cache the coordinates and reuse them in subsequent
lookups. This is achieved via a coordinate cache in JS::Identifier.
Note that non-strict direct eval() breaks this optimization and so it
will not be performed if the resolved environment has been permanently
screwed by eval().
This makes variable access *significantly* faster. :^)
This will be used by LibWeb to squirrel away the stack while performing
a microtask checkpoint in some cases. VM will simply consider saved
execution context stacks as GC roots as well.
VM now has a string cache which tracks all live PrimitiveStrings and
reuses an existing one if possible. This drastically reduces the number
of GC-allocated strings in many real-word situations.
Before this we used an ad-hoc combination of references and 'variables'
stored in a hashmap. This worked in most cases but is not spec like.
Additionally hoisting, dynamically naming functions and scope analysis
was not done properly.
This patch fixes all of that by:
- Implement BindingInitialization for destructuring assignment.
- Implementing a new ScopePusher which tracks the lexical and var
scoped declarations. This hoists functions to the top level if no
lexical declaration name overlaps. Furthermore we do checking of
redeclarations in the ScopePusher now requiring less checks all over
the place.
- Add methods for parsing the directives and statement lists instead
of having that code duplicated in multiple places. This allows
declarations to pushed to the appropriate scope more easily.
- Remove the non spec way of storing 'variables' in
DeclarativeEnvironment and make Reference follow the spec instead of
checking both the bindings and 'variables'.
- Remove all scoping related things from the Interpreter. And instead
use environments as specified by the spec. This also includes fixing
that NativeFunctions did not produce a valid FunctionEnvironment
which could cause issues with callbacks and eval. All
FunctionObjects now have a valid NewFunctionEnvironment
implementation.
- Remove execute_statements from Interpreter and instead use
ASTNode::execute everywhere this simplifies AST.cpp as you no longer
need to worry about which method to call.
- Make ScopeNodes setup their own environment. This uses four
different methods specified by the spec
{Block, Function, Eval, Global}DeclarationInstantiation with the
annexB extensions.
- Implement and use NamedEvaluation where specified.
Additionally there are fixes to things exposed by these changes to eval,
{for, for-in, for-of} loops and assignment.
Finally it also fixes some tests in test-js which where passing before
but not now that we have correct behavior :^).
Since there are only a number of statements where labels can actually be
used we now also only store labels when necessary.
Also now tracks the first continue usage of a label since this might not
be valid but that can only be determined after we have parsed the
statement.
Also ensures the correct error does not get wiped by load_state.
We decided that we want to move away from throwing exceptions in AOs
and regular functions implicitly and then returning some
default-constructed value (usually an empty JS::Value) - this requires
remembering to check for an exception at the call site, which is
error-prone. It's also awkward for return values that cannot be
default-constructed, e.g. MarkedValueList.
Instead, the thrown value should be part of the function return value.
The solution to this is moving closer to the spec and using something
they call "completion records":
https://tc39.es/ecma262/#sec-completion-record-specification-type
This has various advantages:
- It becomes crystal clear whether some AO can fail or not, and errors
need to be handled and values unwrapped explicitly (for that reason
compatibility with the TRY() macro is already built-in, and a similar
TRY_OR_DISCARD() macro has been added specifically for use in LibJS,
while the majority of functions doesn't return ThrowCompletionOr yet)
- We no longer need to mix "valid" and "invalid" values of various types
for the success and exception outcomes (e.g. null/non-null AK::String,
empty/non-empty JS::Value)
- Subsequently it's no longer possible to accidentally use an exception
outcome return value as a success outcome return value (e.g. any AO
that returns a numeric type would return 0 even after throwing an
exception, at least before we started making use of Optional for that)
- Eventually the VM will no longer need to store an exception, and
temporarily clearing an exception (e.g. to call a function) becomes
obsolete - instead, completions will simply propagate up to the caller
outside of LibJS, which then can deal with it in any way
- Similar to throw we'll be able to implement the functionality of
break, continue, and return using completions, which will lead to
easier to understand code and fewer workarounds - the current
unwinding mechanism is not even remotely similar to the spec's
approach
The spec's NormalCompletion and ThrowCompletion AOs have been
implemented as simple wrappers around the JS::Completion constructor.
UpdateEmpty has been implemented as a JS::Completion method.
There's also a new VM::throw_completion<T>() helper, which basically
works like VM::throw_exception<T>() - it creates a T object (usually a
JS::Error), and returns it wrapped in a JS::Completion of Type::Throw.
Two temporary usage patterns have emerged:
1. Callee already returns ThrowCompletionOr, but caller doesn't:
auto foo = TRY_OR_DISCARD(bar());
2. Caller already returns ThrowCompletionOr, but callee doesn't:
auto foo = bar();
if (auto* exception = vm.exception())
return throw_completion(exception->value());
Eventually all error handling and unwrapping can be done with just TRY()
or possibly even operator? in the future :^)
Co-authored-by: Andreas Kling <kling@serenityos.org>
This got changed in the spec at some point, replacing the assertion in
step 1 with "... and newTarget (an Object or undefined)" in the
parameter description.
Subsequently, there's now one step less, so the numbers all change.
