beenull/ladybird
1
0
Fork 0
mirror of https://github.com/LadybirdBrowser/ladybird.git synced 2024-11-22 07:30:19 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 6
ladybird/Userland/Libraries/LibJS/Parser.cpp

4057 lines
175 KiB
C++
Raw Normal View History

LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
/*
Everywhere: Use s.unverwerth@serenityos.org :^)
2021-05-29 10:38:28 +00:00
* Copyright (c) 2020, Stephan Unverwerth <s.unverwerth@serenityos.org>
LibJS: Rename RegExpLiteral m_content to m_pattern This is what we call it elsewhere, let's be consistent.
2021-05-10 10:56:08 +00:00
* Copyright (c) 2020-2021, Linus Groh <linusg@serenityos.org>
Everywhere: Use my awesome new serenityos email :^)
2021-10-03 12:16:10 +00:00
* Copyright (c) 2021, David Tuin <davidot@serenityos.org>
LibJS: Make accessing the current function's arguments cheaper Instead of going through an environment record, make arguments of the currently executing function generate references via the argument index, which can later be resolved directly through the ExecutionContext.
2021-10-07 16:43:22 +00:00
* Copyright (c) 2021, Ali Mohammad Pur <mpfard@serenityos.org>
LibJS: Add support for await expressions
2021-11-09 20:52:21 +00:00
* Copyright (c) 2021, Idan Horowitz <idan.horowitz@serenityos.org>
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
*
Everything: Move to SPDX license identifiers in all files. SPDX License Identifiers are a more compact / standardized way of representing file license information. See: https://spdx.dev/resources/use/#identifiers This was done with the `ambr` search and replace tool. ambr --no-parent-ignore --key-from-file --rep-from-file key.txt rep.txt *
2021-04-22 08:24:48 +00:00
* SPDX-License-Identifier: BSD-2-Clause
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
*/
#include "Parser.h"
LibJS: Throw a syntax error when an identifier is a reserved word From the specification: It is a Syntax Error if StringValue of IdentifierName is the same String value as the StringValue of any ReservedWord except for yield or await. It is a Syntax Error if this phrase is contained in strict mode code and the StringValue of IdentifierName is: "implements", "interface", "let", "package", "private", "protected", "public", "static", or "yield".
2021-06-16 22:57:01 +00:00
#include <AK/Array.h>
LibJS: Use the new is_ascii_foo() helpers from AK These constexpr helpers generate nicer code than the LibC ctype.h variants, so let's make use of them. :^)
2021-06-13 08:47:09 +00:00
#include <AK/CharacterTypes.h>
LibJS: Make invalid RegExp flags a SyntaxError at parse time This patch changes the validation of RegExp flags (checking for invalid and duplicate values) from a SyntaxError at runtime to a SyntaxError at parse time - it's not something that's supposed to be catchable. As a nice side effect, this simplifies the RegExpObject constructor a bit, as it can no longer throw an exception and doesn't have to validate the flags itself.
2021-05-10 11:01:38 +00:00
#include <AK/HashTable.h>
LibJS: Hoist variable declarations to the nearest relevant scope "var" declarations are hoisted to the nearest function scope, while "let" and "const" are hoisted to the nearest block scope. This is done by the parser, which keeps two scope stacks, one stack for the current var scope and one for the current let/const scope. When the interpreter enters a scope, we walk all of the declarations and insert them into the variable environment. We don't support the temporal dead zone for let/const yet.
2020-04-13 14:42:54 +00:00
#include <AK/ScopeGuard.h>
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
#include <AK/StdLibExtras.h>
Everywhere: Add missing <AK/TemporaryChange.h> includes Don't rely on HashTable.h pulling this in.
2020-10-15 18:46:52 +00:00
#include <AK/TemporaryChange.h>
LibJS: Parse RegExp literals at AST creation time, not execution time The spec requires that invalid RegExp literals must cause a Syntax Error before the JavaScript is executed. See: https://tc39.es/ecma262/#sec-patterns-static-semantics-early-errors This is explicitly tested in the RegExp/property-escapes test262 tests. For example, see unsupported-property-Line_Break.js: $DONOTEVALUATE(); /\p{Line_Break}/u; That RegExp literal is invalid because Line_Break is not a supported Unicode property. $DONOTEVALUATE() just throws an exception when it is executed. The test expects that this file will fail to be parsed. Note that RegExp patterns can still be parsed at execution time by way of "new RegExp(...)".
2021-07-29 14:34:37 +00:00
#include <LibJS/Runtime/RegExpObject.h>
#include <LibRegex/Regex.h>
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
namespace JS {
LibJS: Add operator precedence parsing Obey precedence and associativity rules when parsing expressions with chained operators.
2020-03-12 22:02:41 +00:00
LibJS: Hoist variable declarations to the nearest relevant scope "var" declarations are hoisted to the nearest function scope, while "let" and "const" are hoisted to the nearest block scope. This is done by the parser, which keeps two scope stacks, one stack for the current var scope and one for the current let/const scope. When the interpreter enters a scope, we walk all of the declarations and insert them into the variable environment. We don't support the temporal dead zone for let/const yet.
2020-04-13 14:42:54 +00:00
class ScopePusher {
LibJS: Make scoping follow the spec Before this we used an ad-hoc combination of references and 'variables' stored in a hashmap. This worked in most cases but is not spec like. Additionally hoisting, dynamically naming functions and scope analysis was not done properly. This patch fixes all of that by: - Implement BindingInitialization for destructuring assignment. - Implementing a new ScopePusher which tracks the lexical and var scoped declarations. This hoists functions to the top level if no lexical declaration name overlaps. Furthermore we do checking of redeclarations in the ScopePusher now requiring less checks all over the place. - Add methods for parsing the directives and statement lists instead of having that code duplicated in multiple places. This allows declarations to pushed to the appropriate scope more easily. - Remove the non spec way of storing 'variables' in DeclarativeEnvironment and make Reference follow the spec instead of checking both the bindings and 'variables'. - Remove all scoping related things from the Interpreter. And instead use environments as specified by the spec. This also includes fixing that NativeFunctions did not produce a valid FunctionEnvironment which could cause issues with callbacks and eval. All FunctionObjects now have a valid NewFunctionEnvironment implementation. - Remove execute_statements from Interpreter and instead use ASTNode::execute everywhere this simplifies AST.cpp as you no longer need to worry about which method to call. - Make ScopeNodes setup their own environment. This uses four different methods specified by the spec {Block, Function, Eval, Global}DeclarationInstantiation with the annexB extensions. - Implement and use NamedEvaluation where specified. Additionally there are fixes to things exposed by these changes to eval, {for, for-in, for-of} loops and assignment. Finally it also fixes some tests in test-js which where passing before but not now that we have correct behavior :^).
2021-09-22 10:44:56 +00:00
private:
ScopePusher(Parser& parser, ScopeNode* node, bool is_top_level)
LibJS: Hoist variable declarations to the nearest relevant scope "var" declarations are hoisted to the nearest function scope, while "let" and "const" are hoisted to the nearest block scope. This is done by the parser, which keeps two scope stacks, one stack for the current var scope and one for the current let/const scope. When the interpreter enters a scope, we walk all of the declarations and insert them into the variable environment. We don't support the temporal dead zone for let/const yet.
2020-04-13 14:42:54 +00:00
: m_parser(parser)
LibJS: Make scoping follow the spec Before this we used an ad-hoc combination of references and 'variables' stored in a hashmap. This worked in most cases but is not spec like. Additionally hoisting, dynamically naming functions and scope analysis was not done properly. This patch fixes all of that by: - Implement BindingInitialization for destructuring assignment. - Implementing a new ScopePusher which tracks the lexical and var scoped declarations. This hoists functions to the top level if no lexical declaration name overlaps. Furthermore we do checking of redeclarations in the ScopePusher now requiring less checks all over the place. - Add methods for parsing the directives and statement lists instead of having that code duplicated in multiple places. This allows declarations to pushed to the appropriate scope more easily. - Remove the non spec way of storing 'variables' in DeclarativeEnvironment and make Reference follow the spec instead of checking both the bindings and 'variables'. - Remove all scoping related things from the Interpreter. And instead use environments as specified by the spec. This also includes fixing that NativeFunctions did not produce a valid FunctionEnvironment which could cause issues with callbacks and eval. All FunctionObjects now have a valid NewFunctionEnvironment implementation. - Remove execute_statements from Interpreter and instead use ASTNode::execute everywhere this simplifies AST.cpp as you no longer need to worry about which method to call. - Make ScopeNodes setup their own environment. This uses four different methods specified by the spec {Block, Function, Eval, Global}DeclarationInstantiation with the annexB extensions. - Implement and use NamedEvaluation where specified. Additionally there are fixes to things exposed by these changes to eval, {for, for-in, for-of} loops and assignment. Finally it also fixes some tests in test-js which where passing before but not now that we have correct behavior :^).
2021-09-22 10:44:56 +00:00
, m_is_top_level(is_top_level)
LibJS: Hoist variable declarations to the nearest relevant scope "var" declarations are hoisted to the nearest function scope, while "let" and "const" are hoisted to the nearest block scope. This is done by the parser, which keeps two scope stacks, one stack for the current var scope and one for the current let/const scope. When the interpreter enters a scope, we walk all of the declarations and insert them into the variable environment. We don't support the temporal dead zone for let/const yet.
2020-04-13 14:42:54 +00:00
{
LibJS: Make scoping follow the spec Before this we used an ad-hoc combination of references and 'variables' stored in a hashmap. This worked in most cases but is not spec like. Additionally hoisting, dynamically naming functions and scope analysis was not done properly. This patch fixes all of that by: - Implement BindingInitialization for destructuring assignment. - Implementing a new ScopePusher which tracks the lexical and var scoped declarations. This hoists functions to the top level if no lexical declaration name overlaps. Furthermore we do checking of redeclarations in the ScopePusher now requiring less checks all over the place. - Add methods for parsing the directives and statement lists instead of having that code duplicated in multiple places. This allows declarations to pushed to the appropriate scope more easily. - Remove the non spec way of storing 'variables' in DeclarativeEnvironment and make Reference follow the spec instead of checking both the bindings and 'variables'. - Remove all scoping related things from the Interpreter. And instead use environments as specified by the spec. This also includes fixing that NativeFunctions did not produce a valid FunctionEnvironment which could cause issues with callbacks and eval. All FunctionObjects now have a valid NewFunctionEnvironment implementation. - Remove execute_statements from Interpreter and instead use ASTNode::execute everywhere this simplifies AST.cpp as you no longer need to worry about which method to call. - Make ScopeNodes setup their own environment. This uses four different methods specified by the spec {Block, Function, Eval, Global}DeclarationInstantiation with the annexB extensions. - Implement and use NamedEvaluation where specified. Additionally there are fixes to things exposed by these changes to eval, {for, for-in, for-of} loops and assignment. Finally it also fixes some tests in test-js which where passing before but not now that we have correct behavior :^).
2021-09-22 10:44:56 +00:00
m_parent_scope = exchange(m_parser.m_state.current_scope_pusher, this);
VERIFY(node || (m_parent_scope && !is_top_level));
if (!node)
m_node = m_parent_scope->m_node;
else
m_node = node;
VERIFY(m_node);
LibJS: Improve function hoisting across blocks The parser now keeps track of a scope chain so that it can hoist function declarations to the closest function scope.
2021-07-04 01:15:52 +00:00
LibJS: Make scoping follow the spec Before this we used an ad-hoc combination of references and 'variables' stored in a hashmap. This worked in most cases but is not spec like. Additionally hoisting, dynamically naming functions and scope analysis was not done properly. This patch fixes all of that by: - Implement BindingInitialization for destructuring assignment. - Implementing a new ScopePusher which tracks the lexical and var scoped declarations. This hoists functions to the top level if no lexical declaration name overlaps. Furthermore we do checking of redeclarations in the ScopePusher now requiring less checks all over the place. - Add methods for parsing the directives and statement lists instead of having that code duplicated in multiple places. This allows declarations to pushed to the appropriate scope more easily. - Remove the non spec way of storing 'variables' in DeclarativeEnvironment and make Reference follow the spec instead of checking both the bindings and 'variables'. - Remove all scoping related things from the Interpreter. And instead use environments as specified by the spec. This also includes fixing that NativeFunctions did not produce a valid FunctionEnvironment which could cause issues with callbacks and eval. All FunctionObjects now have a valid NewFunctionEnvironment implementation. - Remove execute_statements from Interpreter and instead use ASTNode::execute everywhere this simplifies AST.cpp as you no longer need to worry about which method to call. - Make ScopeNodes setup their own environment. This uses four different methods specified by the spec {Block, Function, Eval, Global}DeclarationInstantiation with the annexB extensions. - Implement and use NamedEvaluation where specified. Additionally there are fixes to things exposed by these changes to eval, {for, for-in, for-of} loops and assignment. Finally it also fixes some tests in test-js which where passing before but not now that we have correct behavior :^).
2021-09-22 10:44:56 +00:00
if (!is_top_level)
m_top_level_scope = m_parent_scope->m_top_level_scope;
else
m_top_level_scope = this;
LibJS: Hoist variable declarations to the nearest relevant scope "var" declarations are hoisted to the nearest function scope, while "let" and "const" are hoisted to the nearest block scope. This is done by the parser, which keeps two scope stacks, one stack for the current var scope and one for the current let/const scope. When the interpreter enters a scope, we walk all of the declarations and insert them into the variable environment. We don't support the temporal dead zone for let/const yet.
2020-04-13 14:42:54 +00:00
}
LibJS: Make scoping follow the spec Before this we used an ad-hoc combination of references and 'variables' stored in a hashmap. This worked in most cases but is not spec like. Additionally hoisting, dynamically naming functions and scope analysis was not done properly. This patch fixes all of that by: - Implement BindingInitialization for destructuring assignment. - Implementing a new ScopePusher which tracks the lexical and var scoped declarations. This hoists functions to the top level if no lexical declaration name overlaps. Furthermore we do checking of redeclarations in the ScopePusher now requiring less checks all over the place. - Add methods for parsing the directives and statement lists instead of having that code duplicated in multiple places. This allows declarations to pushed to the appropriate scope more easily. - Remove the non spec way of storing 'variables' in DeclarativeEnvironment and make Reference follow the spec instead of checking both the bindings and 'variables'. - Remove all scoping related things from the Interpreter. And instead use environments as specified by the spec. This also includes fixing that NativeFunctions did not produce a valid FunctionEnvironment which could cause issues with callbacks and eval. All FunctionObjects now have a valid NewFunctionEnvironment implementation. - Remove execute_statements from Interpreter and instead use ASTNode::execute everywhere this simplifies AST.cpp as you no longer need to worry about which method to call. - Make ScopeNodes setup their own environment. This uses four different methods specified by the spec {Block, Function, Eval, Global}DeclarationInstantiation with the annexB extensions. - Implement and use NamedEvaluation where specified. Additionally there are fixes to things exposed by these changes to eval, {for, for-in, for-of} loops and assignment. Finally it also fixes some tests in test-js which where passing before but not now that we have correct behavior :^).
2021-09-22 10:44:56 +00:00
public:
static ScopePusher function_scope(Parser& parser, FunctionBody& function_body, Vector<FunctionDeclaration::Parameter> const& parameters)
LibJS: Hoist variable declarations to the nearest relevant scope "var" declarations are hoisted to the nearest function scope, while "let" and "const" are hoisted to the nearest block scope. This is done by the parser, which keeps two scope stacks, one stack for the current var scope and one for the current let/const scope. When the interpreter enters a scope, we walk all of the declarations and insert them into the variable environment. We don't support the temporal dead zone for let/const yet.
2020-04-13 14:42:54 +00:00
{
LibJS: Make scoping follow the spec Before this we used an ad-hoc combination of references and 'variables' stored in a hashmap. This worked in most cases but is not spec like. Additionally hoisting, dynamically naming functions and scope analysis was not done properly. This patch fixes all of that by: - Implement BindingInitialization for destructuring assignment. - Implementing a new ScopePusher which tracks the lexical and var scoped declarations. This hoists functions to the top level if no lexical declaration name overlaps. Furthermore we do checking of redeclarations in the ScopePusher now requiring less checks all over the place. - Add methods for parsing the directives and statement lists instead of having that code duplicated in multiple places. This allows declarations to pushed to the appropriate scope more easily. - Remove the non spec way of storing 'variables' in DeclarativeEnvironment and make Reference follow the spec instead of checking both the bindings and 'variables'. - Remove all scoping related things from the Interpreter. And instead use environments as specified by the spec. This also includes fixing that NativeFunctions did not produce a valid FunctionEnvironment which could cause issues with callbacks and eval. All FunctionObjects now have a valid NewFunctionEnvironment implementation. - Remove execute_statements from Interpreter and instead use ASTNode::execute everywhere this simplifies AST.cpp as you no longer need to worry about which method to call. - Make ScopeNodes setup their own environment. This uses four different methods specified by the spec {Block, Function, Eval, Global}DeclarationInstantiation with the annexB extensions. - Implement and use NamedEvaluation where specified. Additionally there are fixes to things exposed by these changes to eval, {for, for-in, for-of} loops and assignment. Finally it also fixes some tests in test-js which where passing before but not now that we have correct behavior :^).
2021-09-22 10:44:56 +00:00
ScopePusher scope_pusher(parser, &function_body, true);
LibJS: Make accessing the current function's arguments cheaper Instead of going through an environment record, make arguments of the currently executing function generate references via the argument index, which can later be resolved directly through the ExecutionContext.
2021-10-07 16:43:22 +00:00
scope_pusher.m_function_parameters = parameters;
LibJS: Make scoping follow the spec Before this we used an ad-hoc combination of references and 'variables' stored in a hashmap. This worked in most cases but is not spec like. Additionally hoisting, dynamically naming functions and scope analysis was not done properly. This patch fixes all of that by: - Implement BindingInitialization for destructuring assignment. - Implementing a new ScopePusher which tracks the lexical and var scoped declarations. This hoists functions to the top level if no lexical declaration name overlaps. Furthermore we do checking of redeclarations in the ScopePusher now requiring less checks all over the place. - Add methods for parsing the directives and statement lists instead of having that code duplicated in multiple places. This allows declarations to pushed to the appropriate scope more easily. - Remove the non spec way of storing 'variables' in DeclarativeEnvironment and make Reference follow the spec instead of checking both the bindings and 'variables'. - Remove all scoping related things from the Interpreter. And instead use environments as specified by the spec. This also includes fixing that NativeFunctions did not produce a valid FunctionEnvironment which could cause issues with callbacks and eval. All FunctionObjects now have a valid NewFunctionEnvironment implementation. - Remove execute_statements from Interpreter and instead use ASTNode::execute everywhere this simplifies AST.cpp as you no longer need to worry about which method to call. - Make ScopeNodes setup their own environment. This uses four different methods specified by the spec {Block, Function, Eval, Global}DeclarationInstantiation with the annexB extensions. - Implement and use NamedEvaluation where specified. Additionally there are fixes to things exposed by these changes to eval, {for, for-in, for-of} loops and assignment. Finally it also fixes some tests in test-js which where passing before but not now that we have correct behavior :^).
2021-09-22 10:44:56 +00:00
for (auto& parameter : parameters) {
parameter.binding.visit(
[&](FlyString const& name) {
scope_pusher.m_forbidden_lexical_names.set(name);
},
[&](NonnullRefPtr<BindingPattern> const& binding_pattern) {
binding_pattern->for_each_bound_name([&](auto const& name) {
scope_pusher.m_forbidden_lexical_names.set(name);
});
});
}
return scope_pusher;
}
LibJS: Improve function hoisting across blocks The parser now keeps track of a scope chain so that it can hoist function declarations to the closest function scope.
2021-07-04 01:15:52 +00:00
LibJS: Make scoping follow the spec Before this we used an ad-hoc combination of references and 'variables' stored in a hashmap. This worked in most cases but is not spec like. Additionally hoisting, dynamically naming functions and scope analysis was not done properly. This patch fixes all of that by: - Implement BindingInitialization for destructuring assignment. - Implementing a new ScopePusher which tracks the lexical and var scoped declarations. This hoists functions to the top level if no lexical declaration name overlaps. Furthermore we do checking of redeclarations in the ScopePusher now requiring less checks all over the place. - Add methods for parsing the directives and statement lists instead of having that code duplicated in multiple places. This allows declarations to pushed to the appropriate scope more easily. - Remove the non spec way of storing 'variables' in DeclarativeEnvironment and make Reference follow the spec instead of checking both the bindings and 'variables'. - Remove all scoping related things from the Interpreter. And instead use environments as specified by the spec. This also includes fixing that NativeFunctions did not produce a valid FunctionEnvironment which could cause issues with callbacks and eval. All FunctionObjects now have a valid NewFunctionEnvironment implementation. - Remove execute_statements from Interpreter and instead use ASTNode::execute everywhere this simplifies AST.cpp as you no longer need to worry about which method to call. - Make ScopeNodes setup their own environment. This uses four different methods specified by the spec {Block, Function, Eval, Global}DeclarationInstantiation with the annexB extensions. - Implement and use NamedEvaluation where specified. Additionally there are fixes to things exposed by these changes to eval, {for, for-in, for-of} loops and assignment. Finally it also fixes some tests in test-js which where passing before but not now that we have correct behavior :^).
2021-09-22 10:44:56 +00:00
static ScopePusher program_scope(Parser& parser, Program& program)
{
return ScopePusher(parser, &program, true);
}
LibJS: Don't hoist functions under certain circumstances When a lexical declaration with the same name as a function exists, the function is not hoisted (annex B).
2021-07-05 19:45:34 +00:00
LibJS: Make scoping follow the spec Before this we used an ad-hoc combination of references and 'variables' stored in a hashmap. This worked in most cases but is not spec like. Additionally hoisting, dynamically naming functions and scope analysis was not done properly. This patch fixes all of that by: - Implement BindingInitialization for destructuring assignment. - Implementing a new ScopePusher which tracks the lexical and var scoped declarations. This hoists functions to the top level if no lexical declaration name overlaps. Furthermore we do checking of redeclarations in the ScopePusher now requiring less checks all over the place. - Add methods for parsing the directives and statement lists instead of having that code duplicated in multiple places. This allows declarations to pushed to the appropriate scope more easily. - Remove the non spec way of storing 'variables' in DeclarativeEnvironment and make Reference follow the spec instead of checking both the bindings and 'variables'. - Remove all scoping related things from the Interpreter. And instead use environments as specified by the spec. This also includes fixing that NativeFunctions did not produce a valid FunctionEnvironment which could cause issues with callbacks and eval. All FunctionObjects now have a valid NewFunctionEnvironment implementation. - Remove execute_statements from Interpreter and instead use ASTNode::execute everywhere this simplifies AST.cpp as you no longer need to worry about which method to call. - Make ScopeNodes setup their own environment. This uses four different methods specified by the spec {Block, Function, Eval, Global}DeclarationInstantiation with the annexB extensions. - Implement and use NamedEvaluation where specified. Additionally there are fixes to things exposed by these changes to eval, {for, for-in, for-of} loops and assignment. Finally it also fixes some tests in test-js which where passing before but not now that we have correct behavior :^).
2021-09-22 10:44:56 +00:00
static ScopePusher block_scope(Parser& parser, ScopeNode& node)
{
return ScopePusher(parser, &node, false);
LibJS: Improve function hoisting across blocks The parser now keeps track of a scope chain so that it can hoist function declarations to the closest function scope.
2021-07-04 01:15:52 +00:00
}
LibJS: Make scoping follow the spec Before this we used an ad-hoc combination of references and 'variables' stored in a hashmap. This worked in most cases but is not spec like. Additionally hoisting, dynamically naming functions and scope analysis was not done properly. This patch fixes all of that by: - Implement BindingInitialization for destructuring assignment. - Implementing a new ScopePusher which tracks the lexical and var scoped declarations. This hoists functions to the top level if no lexical declaration name overlaps. Furthermore we do checking of redeclarations in the ScopePusher now requiring less checks all over the place. - Add methods for parsing the directives and statement lists instead of having that code duplicated in multiple places. This allows declarations to pushed to the appropriate scope more easily. - Remove the non spec way of storing 'variables' in DeclarativeEnvironment and make Reference follow the spec instead of checking both the bindings and 'variables'. - Remove all scoping related things from the Interpreter. And instead use environments as specified by the spec. This also includes fixing that NativeFunctions did not produce a valid FunctionEnvironment which could cause issues with callbacks and eval. All FunctionObjects now have a valid NewFunctionEnvironment implementation. - Remove execute_statements from Interpreter and instead use ASTNode::execute everywhere this simplifies AST.cpp as you no longer need to worry about which method to call. - Make ScopeNodes setup their own environment. This uses four different methods specified by the spec {Block, Function, Eval, Global}DeclarationInstantiation with the annexB extensions. - Implement and use NamedEvaluation where specified. Additionally there are fixes to things exposed by these changes to eval, {for, for-in, for-of} loops and assignment. Finally it also fixes some tests in test-js which where passing before but not now that we have correct behavior :^).
2021-09-22 10:44:56 +00:00
static ScopePusher for_loop_scope(Parser& parser, RefPtr<ASTNode> const& init)
LibJS: Improve function hoisting across blocks The parser now keeps track of a scope chain so that it can hoist function declarations to the closest function scope.
2021-07-04 01:15:52 +00:00
{
LibJS: Make scoping follow the spec Before this we used an ad-hoc combination of references and 'variables' stored in a hashmap. This worked in most cases but is not spec like. Additionally hoisting, dynamically naming functions and scope analysis was not done properly. This patch fixes all of that by: - Implement BindingInitialization for destructuring assignment. - Implementing a new ScopePusher which tracks the lexical and var scoped declarations. This hoists functions to the top level if no lexical declaration name overlaps. Furthermore we do checking of redeclarations in the ScopePusher now requiring less checks all over the place. - Add methods for parsing the directives and statement lists instead of having that code duplicated in multiple places. This allows declarations to pushed to the appropriate scope more easily. - Remove the non spec way of storing 'variables' in DeclarativeEnvironment and make Reference follow the spec instead of checking both the bindings and 'variables'. - Remove all scoping related things from the Interpreter. And instead use environments as specified by the spec. This also includes fixing that NativeFunctions did not produce a valid FunctionEnvironment which could cause issues with callbacks and eval. All FunctionObjects now have a valid NewFunctionEnvironment implementation. - Remove execute_statements from Interpreter and instead use ASTNode::execute everywhere this simplifies AST.cpp as you no longer need to worry about which method to call. - Make ScopeNodes setup their own environment. This uses four different methods specified by the spec {Block, Function, Eval, Global}DeclarationInstantiation with the annexB extensions. - Implement and use NamedEvaluation where specified. Additionally there are fixes to things exposed by these changes to eval, {for, for-in, for-of} loops and assignment. Finally it also fixes some tests in test-js which where passing before but not now that we have correct behavior :^).
2021-09-22 10:44:56 +00:00
ScopePusher scope_pusher(parser, nullptr, false);
if (init && is<VariableDeclaration>(*init)) {
auto& variable_declaration = static_cast<VariableDeclaration const&>(*init);
if (variable_declaration.declaration_kind() != DeclarationKind::Var) {
variable_declaration.for_each_bound_name([&](auto const& name) {
scope_pusher.m_forbidden_var_names.set(name);
});
}
}
LibJS: Improve function hoisting across blocks The parser now keeps track of a scope chain so that it can hoist function declarations to the closest function scope.
2021-07-04 01:15:52 +00:00
LibJS: Make scoping follow the spec Before this we used an ad-hoc combination of references and 'variables' stored in a hashmap. This worked in most cases but is not spec like. Additionally hoisting, dynamically naming functions and scope analysis was not done properly. This patch fixes all of that by: - Implement BindingInitialization for destructuring assignment. - Implementing a new ScopePusher which tracks the lexical and var scoped declarations. This hoists functions to the top level if no lexical declaration name overlaps. Furthermore we do checking of redeclarations in the ScopePusher now requiring less checks all over the place. - Add methods for parsing the directives and statement lists instead of having that code duplicated in multiple places. This allows declarations to pushed to the appropriate scope more easily. - Remove the non spec way of storing 'variables' in DeclarativeEnvironment and make Reference follow the spec instead of checking both the bindings and 'variables'. - Remove all scoping related things from the Interpreter. And instead use environments as specified by the spec. This also includes fixing that NativeFunctions did not produce a valid FunctionEnvironment which could cause issues with callbacks and eval. All FunctionObjects now have a valid NewFunctionEnvironment implementation. - Remove execute_statements from Interpreter and instead use ASTNode::execute everywhere this simplifies AST.cpp as you no longer need to worry about which method to call. - Make ScopeNodes setup their own environment. This uses four different methods specified by the spec {Block, Function, Eval, Global}DeclarationInstantiation with the annexB extensions. - Implement and use NamedEvaluation where specified. Additionally there are fixes to things exposed by these changes to eval, {for, for-in, for-of} loops and assignment. Finally it also fixes some tests in test-js which where passing before but not now that we have correct behavior :^).
2021-09-22 10:44:56 +00:00
return scope_pusher;
}
LibJS: Don't hoist functions under certain circumstances When a lexical declaration with the same name as a function exists, the function is not hoisted (annex B).
2021-07-05 19:45:34 +00:00
LibJS: Treat the Catch binding identifier as a var binding
2021-10-08 10:04:13 +00:00
static ScopePusher catch_scope(Parser& parser, RefPtr<BindingPattern> const& pattern, FlyString const& parameter)
LibJS: Make scoping follow the spec Before this we used an ad-hoc combination of references and 'variables' stored in a hashmap. This worked in most cases but is not spec like. Additionally hoisting, dynamically naming functions and scope analysis was not done properly. This patch fixes all of that by: - Implement BindingInitialization for destructuring assignment. - Implementing a new ScopePusher which tracks the lexical and var scoped declarations. This hoists functions to the top level if no lexical declaration name overlaps. Furthermore we do checking of redeclarations in the ScopePusher now requiring less checks all over the place. - Add methods for parsing the directives and statement lists instead of having that code duplicated in multiple places. This allows declarations to pushed to the appropriate scope more easily. - Remove the non spec way of storing 'variables' in DeclarativeEnvironment and make Reference follow the spec instead of checking both the bindings and 'variables'. - Remove all scoping related things from the Interpreter. And instead use environments as specified by the spec. This also includes fixing that NativeFunctions did not produce a valid FunctionEnvironment which could cause issues with callbacks and eval. All FunctionObjects now have a valid NewFunctionEnvironment implementation. - Remove execute_statements from Interpreter and instead use ASTNode::execute everywhere this simplifies AST.cpp as you no longer need to worry about which method to call. - Make ScopeNodes setup their own environment. This uses four different methods specified by the spec {Block, Function, Eval, Global}DeclarationInstantiation with the annexB extensions. - Implement and use NamedEvaluation where specified. Additionally there are fixes to things exposed by these changes to eval, {for, for-in, for-of} loops and assignment. Finally it also fixes some tests in test-js which where passing before but not now that we have correct behavior :^).
2021-09-22 10:44:56 +00:00
{
ScopePusher scope_pusher(parser, nullptr, false);
if (pattern) {
pattern->for_each_bound_name([&](auto const& name) {
scope_pusher.m_forbidden_var_names.set(name);
});
LibJS: Treat the Catch binding identifier as a var binding
2021-10-08 10:04:13 +00:00
} else if (!parameter.is_empty()) {
scope_pusher.m_var_names.set(parameter);
LibJS: Don't hoist functions under certain circumstances When a lexical declaration with the same name as a function exists, the function is not hoisted (annex B).
2021-07-05 19:45:34 +00:00
}
LibJS: Make scoping follow the spec Before this we used an ad-hoc combination of references and 'variables' stored in a hashmap. This worked in most cases but is not spec like. Additionally hoisting, dynamically naming functions and scope analysis was not done properly. This patch fixes all of that by: - Implement BindingInitialization for destructuring assignment. - Implementing a new ScopePusher which tracks the lexical and var scoped declarations. This hoists functions to the top level if no lexical declaration name overlaps. Furthermore we do checking of redeclarations in the ScopePusher now requiring less checks all over the place. - Add methods for parsing the directives and statement lists instead of having that code duplicated in multiple places. This allows declarations to pushed to the appropriate scope more easily. - Remove the non spec way of storing 'variables' in DeclarativeEnvironment and make Reference follow the spec instead of checking both the bindings and 'variables'. - Remove all scoping related things from the Interpreter. And instead use environments as specified by the spec. This also includes fixing that NativeFunctions did not produce a valid FunctionEnvironment which could cause issues with callbacks and eval. All FunctionObjects now have a valid NewFunctionEnvironment implementation. - Remove execute_statements from Interpreter and instead use ASTNode::execute everywhere this simplifies AST.cpp as you no longer need to worry about which method to call. - Make ScopeNodes setup their own environment. This uses four different methods specified by the spec {Block, Function, Eval, Global}DeclarationInstantiation with the annexB extensions. - Implement and use NamedEvaluation where specified. Additionally there are fixes to things exposed by these changes to eval, {for, for-in, for-of} loops and assignment. Finally it also fixes some tests in test-js which where passing before but not now that we have correct behavior :^).
2021-09-22 10:44:56 +00:00
return scope_pusher;
LibJS: Don't hoist functions under certain circumstances When a lexical declaration with the same name as a function exists, the function is not hoisted (annex B).
2021-07-05 19:45:34 +00:00
}
LibJS: Add static initializers to classes
2021-10-20 19:29:47 +00:00
static ScopePusher static_init_block_scope(Parser& parser, ScopeNode& node)
{
return ScopePusher(parser, &node, true);
}
LibJS: Use ClassFieldInitializerStatement for class fields This is necessary as we might have to perform named evaluation with the field name. Ideally we would also skip some setup parts of the function like function_declaration_instantiation however this would require bigger changes to ECMAScriptFunctionObject.
2021-10-13 17:59:38 +00:00
static ScopePusher class_field_scope(Parser& parser)
{
return ScopePusher(parser, nullptr, false);
}
LibJS: Make scoping follow the spec Before this we used an ad-hoc combination of references and 'variables' stored in a hashmap. This worked in most cases but is not spec like. Additionally hoisting, dynamically naming functions and scope analysis was not done properly. This patch fixes all of that by: - Implement BindingInitialization for destructuring assignment. - Implementing a new ScopePusher which tracks the lexical and var scoped declarations. This hoists functions to the top level if no lexical declaration name overlaps. Furthermore we do checking of redeclarations in the ScopePusher now requiring less checks all over the place. - Add methods for parsing the directives and statement lists instead of having that code duplicated in multiple places. This allows declarations to pushed to the appropriate scope more easily. - Remove the non spec way of storing 'variables' in DeclarativeEnvironment and make Reference follow the spec instead of checking both the bindings and 'variables'. - Remove all scoping related things from the Interpreter. And instead use environments as specified by the spec. This also includes fixing that NativeFunctions did not produce a valid FunctionEnvironment which could cause issues with callbacks and eval. All FunctionObjects now have a valid NewFunctionEnvironment implementation. - Remove execute_statements from Interpreter and instead use ASTNode::execute everywhere this simplifies AST.cpp as you no longer need to worry about which method to call. - Make ScopeNodes setup their own environment. This uses four different methods specified by the spec {Block, Function, Eval, Global}DeclarationInstantiation with the annexB extensions. - Implement and use NamedEvaluation where specified. Additionally there are fixes to things exposed by these changes to eval, {for, for-in, for-of} loops and assignment. Finally it also fixes some tests in test-js which where passing before but not now that we have correct behavior :^).
2021-09-22 10:44:56 +00:00
void add_declaration(NonnullRefPtr<Declaration> declaration)
LibJS: Don't hoist functions under certain circumstances When a lexical declaration with the same name as a function exists, the function is not hoisted (annex B).
2021-07-05 19:45:34 +00:00
{
LibJS: Make scoping follow the spec Before this we used an ad-hoc combination of references and 'variables' stored in a hashmap. This worked in most cases but is not spec like. Additionally hoisting, dynamically naming functions and scope analysis was not done properly. This patch fixes all of that by: - Implement BindingInitialization for destructuring assignment. - Implementing a new ScopePusher which tracks the lexical and var scoped declarations. This hoists functions to the top level if no lexical declaration name overlaps. Furthermore we do checking of redeclarations in the ScopePusher now requiring less checks all over the place. - Add methods for parsing the directives and statement lists instead of having that code duplicated in multiple places. This allows declarations to pushed to the appropriate scope more easily. - Remove the non spec way of storing 'variables' in DeclarativeEnvironment and make Reference follow the spec instead of checking both the bindings and 'variables'. - Remove all scoping related things from the Interpreter. And instead use environments as specified by the spec. This also includes fixing that NativeFunctions did not produce a valid FunctionEnvironment which could cause issues with callbacks and eval. All FunctionObjects now have a valid NewFunctionEnvironment implementation. - Remove execute_statements from Interpreter and instead use ASTNode::execute everywhere this simplifies AST.cpp as you no longer need to worry about which method to call. - Make ScopeNodes setup their own environment. This uses four different methods specified by the spec {Block, Function, Eval, Global}DeclarationInstantiation with the annexB extensions. - Implement and use NamedEvaluation where specified. Additionally there are fixes to things exposed by these changes to eval, {for, for-in, for-of} loops and assignment. Finally it also fixes some tests in test-js which where passing before but not now that we have correct behavior :^).
2021-09-22 10:44:56 +00:00
if (declaration->is_lexical_declaration()) {
declaration->for_each_bound_name([&](auto const& name) {
if (m_var_names.contains(name) || m_forbidden_lexical_names.contains(name) || m_function_names.contains(name))
throw_identifier_declared(name, declaration);
if (m_lexical_names.set(name) != AK::HashSetResult::InsertedNewEntry)
throw_identifier_declared(name, declaration);
});
m_node->add_lexical_declaration(move(declaration));
} else if (!declaration->is_function_declaration()) {
declaration->for_each_bound_name([&](auto const& name) {
ScopePusher* pusher = this;
while (true) {
if (pusher->m_lexical_names.contains(name)
|| pusher->m_function_names.contains(name)
|| pusher->m_forbidden_var_names.contains(name))
throw_identifier_declared(name, declaration);
pusher->m_var_names.set(name);
if (pusher->m_is_top_level)
break;
VERIFY(pusher->m_parent_scope != nullptr);
pusher = pusher->m_parent_scope;
}
VERIFY(pusher->m_is_top_level && pusher->m_node);
pusher->m_node->add_var_scoped_declaration(declaration);
});
VERIFY(m_top_level_scope);
m_top_level_scope->m_node->add_var_scoped_declaration(move(declaration));
} else {
if (m_is_top_level) {
declaration->for_each_bound_name([&](auto const& name) {
m_var_names.set(name);
});
m_node->add_var_scoped_declaration(move(declaration));
} else {
VERIFY(is<FunctionDeclaration>(*declaration));
auto& function_declaration = static_cast<FunctionDeclaration const&>(*declaration);
auto& function_name = function_declaration.name();
if (m_var_names.contains(function_name) || m_lexical_names.contains(function_name))
throw_identifier_declared(function_name, declaration);
if (function_declaration.kind() != FunctionKind::Regular || m_parser.m_state.strict_mode) {
if (m_function_names.contains(function_name))
throw_identifier_declared(function_name, declaration);
m_lexical_names.set(function_name);
m_node->add_lexical_declaration(move(declaration));
return;
}
m_function_names.set(function_name);
if (!m_lexical_names.contains(function_name))
m_functions_to_hoist.append(static_ptr_cast<FunctionDeclaration>(declaration));
m_node->add_lexical_declaration(move(declaration));
LibJS: Don't hoist functions under certain circumstances When a lexical declaration with the same name as a function exists, the function is not hoisted (annex B).
2021-07-05 19:45:34 +00:00
}
}
LibJS: Make scoping follow the spec Before this we used an ad-hoc combination of references and 'variables' stored in a hashmap. This worked in most cases but is not spec like. Additionally hoisting, dynamically naming functions and scope analysis was not done properly. This patch fixes all of that by: - Implement BindingInitialization for destructuring assignment. - Implementing a new ScopePusher which tracks the lexical and var scoped declarations. This hoists functions to the top level if no lexical declaration name overlaps. Furthermore we do checking of redeclarations in the ScopePusher now requiring less checks all over the place. - Add methods for parsing the directives and statement lists instead of having that code duplicated in multiple places. This allows declarations to pushed to the appropriate scope more easily. - Remove the non spec way of storing 'variables' in DeclarativeEnvironment and make Reference follow the spec instead of checking both the bindings and 'variables'. - Remove all scoping related things from the Interpreter. And instead use environments as specified by the spec. This also includes fixing that NativeFunctions did not produce a valid FunctionEnvironment which could cause issues with callbacks and eval. All FunctionObjects now have a valid NewFunctionEnvironment implementation. - Remove execute_statements from Interpreter and instead use ASTNode::execute everywhere this simplifies AST.cpp as you no longer need to worry about which method to call. - Make ScopeNodes setup their own environment. This uses four different methods specified by the spec {Block, Function, Eval, Global}DeclarationInstantiation with the annexB extensions. - Implement and use NamedEvaluation where specified. Additionally there are fixes to things exposed by these changes to eval, {for, for-in, for-of} loops and assignment. Finally it also fixes some tests in test-js which where passing before but not now that we have correct behavior :^).
2021-09-22 10:44:56 +00:00
}
LibJS: Make accessing the current function's arguments cheaper Instead of going through an environment record, make arguments of the currently executing function generate references via the argument index, which can later be resolved directly through the ExecutionContext.
2021-10-07 16:43:22 +00:00
ScopePusher const* last_function_scope() const
{
for (auto scope_ptr = this; scope_ptr; scope_ptr = scope_ptr->m_parent_scope) {
if (scope_ptr->m_function_parameters.has_value())
return scope_ptr;
}
return nullptr;
}
Vector<FunctionDeclaration::Parameter> const& function_parameters() const
{
return *m_function_parameters;
}
ScopePusher* parent_scope() { return m_parent_scope; }
ScopePusher const* parent_scope() const { return m_parent_scope; }
[[nodiscard]] bool has_declaration(StringView name) const
{
return m_lexical_names.contains(name) || m_var_names.contains(name) || !m_functions_to_hoist.find_if([&name](auto& function) { return function->name() == name; }).is_end();
}
bool contains_direct_call_to_eval() const { return m_contains_direct_call_to_eval; }
bool contains_access_to_arguments_object() const { return m_contains_access_to_arguments_object; }
void set_contains_direct_call_to_eval() { m_contains_direct_call_to_eval = true; }
void set_contains_access_to_arguments_object() { m_contains_access_to_arguments_object = true; }
LibJS: Make scoping follow the spec Before this we used an ad-hoc combination of references and 'variables' stored in a hashmap. This worked in most cases but is not spec like. Additionally hoisting, dynamically naming functions and scope analysis was not done properly. This patch fixes all of that by: - Implement BindingInitialization for destructuring assignment. - Implementing a new ScopePusher which tracks the lexical and var scoped declarations. This hoists functions to the top level if no lexical declaration name overlaps. Furthermore we do checking of redeclarations in the ScopePusher now requiring less checks all over the place. - Add methods for parsing the directives and statement lists instead of having that code duplicated in multiple places. This allows declarations to pushed to the appropriate scope more easily. - Remove the non spec way of storing 'variables' in DeclarativeEnvironment and make Reference follow the spec instead of checking both the bindings and 'variables'. - Remove all scoping related things from the Interpreter. And instead use environments as specified by the spec. This also includes fixing that NativeFunctions did not produce a valid FunctionEnvironment which could cause issues with callbacks and eval. All FunctionObjects now have a valid NewFunctionEnvironment implementation. - Remove execute_statements from Interpreter and instead use ASTNode::execute everywhere this simplifies AST.cpp as you no longer need to worry about which method to call. - Make ScopeNodes setup their own environment. This uses four different methods specified by the spec {Block, Function, Eval, Global}DeclarationInstantiation with the annexB extensions. - Implement and use NamedEvaluation where specified. Additionally there are fixes to things exposed by these changes to eval, {for, for-in, for-of} loops and assignment. Finally it also fixes some tests in test-js which where passing before but not now that we have correct behavior :^).
2021-09-22 10:44:56 +00:00
~ScopePusher()
{
VERIFY(m_is_top_level || m_parent_scope);
LibJS: Make accessing the current function's arguments cheaper Instead of going through an environment record, make arguments of the currently executing function generate references via the argument index, which can later be resolved directly through the ExecutionContext.
2021-10-07 16:43:22 +00:00
if (!m_contains_access_to_arguments_object) {
for (auto& it : m_identifier_and_argument_index_associations) {
for (auto& identifier : it.value) {
if (!has_declaration(identifier.string()))
identifier.set_lexically_bound_function_argument_index(it.key);
}
}
}
LibJS: Make scoping follow the spec Before this we used an ad-hoc combination of references and 'variables' stored in a hashmap. This worked in most cases but is not spec like. Additionally hoisting, dynamically naming functions and scope analysis was not done properly. This patch fixes all of that by: - Implement BindingInitialization for destructuring assignment. - Implementing a new ScopePusher which tracks the lexical and var scoped declarations. This hoists functions to the top level if no lexical declaration name overlaps. Furthermore we do checking of redeclarations in the ScopePusher now requiring less checks all over the place. - Add methods for parsing the directives and statement lists instead of having that code duplicated in multiple places. This allows declarations to pushed to the appropriate scope more easily. - Remove the non spec way of storing 'variables' in DeclarativeEnvironment and make Reference follow the spec instead of checking both the bindings and 'variables'. - Remove all scoping related things from the Interpreter. And instead use environments as specified by the spec. This also includes fixing that NativeFunctions did not produce a valid FunctionEnvironment which could cause issues with callbacks and eval. All FunctionObjects now have a valid NewFunctionEnvironment implementation. - Remove execute_statements from Interpreter and instead use ASTNode::execute everywhere this simplifies AST.cpp as you no longer need to worry about which method to call. - Make ScopeNodes setup their own environment. This uses four different methods specified by the spec {Block, Function, Eval, Global}DeclarationInstantiation with the annexB extensions. - Implement and use NamedEvaluation where specified. Additionally there are fixes to things exposed by these changes to eval, {for, for-in, for-of} loops and assignment. Finally it also fixes some tests in test-js which where passing before but not now that we have correct behavior :^).
2021-09-22 10:44:56 +00:00
for (size_t i = 0; i < m_functions_to_hoist.size(); i++) {
auto const& function_declaration = m_functions_to_hoist[i];
if (m_lexical_names.contains(function_declaration.name()) || m_forbidden_var_names.contains(function_declaration.name()))
continue;
if (m_is_top_level)
m_node->add_hoisted_function(move(m_functions_to_hoist[i]));
else
m_parent_scope->m_functions_to_hoist.append(move(m_functions_to_hoist[i]));
}
LibJS: Make accessing the current function's arguments cheaper Instead of going through an environment record, make arguments of the currently executing function generate references via the argument index, which can later be resolved directly through the ExecutionContext.
2021-10-07 16:43:22 +00:00
if (m_parent_scope && !m_function_parameters.has_value()) {
m_parent_scope->m_contains_access_to_arguments_object |= m_contains_access_to_arguments_object;
m_parent_scope->m_contains_direct_call_to_eval |= m_contains_direct_call_to_eval;
}
LibJS: Make scoping follow the spec Before this we used an ad-hoc combination of references and 'variables' stored in a hashmap. This worked in most cases but is not spec like. Additionally hoisting, dynamically naming functions and scope analysis was not done properly. This patch fixes all of that by: - Implement BindingInitialization for destructuring assignment. - Implementing a new ScopePusher which tracks the lexical and var scoped declarations. This hoists functions to the top level if no lexical declaration name overlaps. Furthermore we do checking of redeclarations in the ScopePusher now requiring less checks all over the place. - Add methods for parsing the directives and statement lists instead of having that code duplicated in multiple places. This allows declarations to pushed to the appropriate scope more easily. - Remove the non spec way of storing 'variables' in DeclarativeEnvironment and make Reference follow the spec instead of checking both the bindings and 'variables'. - Remove all scoping related things from the Interpreter. And instead use environments as specified by the spec. This also includes fixing that NativeFunctions did not produce a valid FunctionEnvironment which could cause issues with callbacks and eval. All FunctionObjects now have a valid NewFunctionEnvironment implementation. - Remove execute_statements from Interpreter and instead use ASTNode::execute everywhere this simplifies AST.cpp as you no longer need to worry about which method to call. - Make ScopeNodes setup their own environment. This uses four different methods specified by the spec {Block, Function, Eval, Global}DeclarationInstantiation with the annexB extensions. - Implement and use NamedEvaluation where specified. Additionally there are fixes to things exposed by these changes to eval, {for, for-in, for-of} loops and assignment. Finally it also fixes some tests in test-js which where passing before but not now that we have correct behavior :^).
2021-09-22 10:44:56 +00:00
VERIFY(m_parser.m_state.current_scope_pusher == this);
m_parser.m_state.current_scope_pusher = m_parent_scope;
}
LibJS: Make accessing the current function's arguments cheaper Instead of going through an environment record, make arguments of the currently executing function generate references via the argument index, which can later be resolved directly through the ExecutionContext.
2021-10-07 16:43:22 +00:00
void associate_identifier_with_argument_index(NonnullRefPtr<Identifier> identifier, size_t index)
{
m_identifier_and_argument_index_associations.ensure(index).append(move(identifier));
}
LibJS: Make scoping follow the spec Before this we used an ad-hoc combination of references and 'variables' stored in a hashmap. This worked in most cases but is not spec like. Additionally hoisting, dynamically naming functions and scope analysis was not done properly. This patch fixes all of that by: - Implement BindingInitialization for destructuring assignment. - Implementing a new ScopePusher which tracks the lexical and var scoped declarations. This hoists functions to the top level if no lexical declaration name overlaps. Furthermore we do checking of redeclarations in the ScopePusher now requiring less checks all over the place. - Add methods for parsing the directives and statement lists instead of having that code duplicated in multiple places. This allows declarations to pushed to the appropriate scope more easily. - Remove the non spec way of storing 'variables' in DeclarativeEnvironment and make Reference follow the spec instead of checking both the bindings and 'variables'. - Remove all scoping related things from the Interpreter. And instead use environments as specified by the spec. This also includes fixing that NativeFunctions did not produce a valid FunctionEnvironment which could cause issues with callbacks and eval. All FunctionObjects now have a valid NewFunctionEnvironment implementation. - Remove execute_statements from Interpreter and instead use ASTNode::execute everywhere this simplifies AST.cpp as you no longer need to worry about which method to call. - Make ScopeNodes setup their own environment. This uses four different methods specified by the spec {Block, Function, Eval, Global}DeclarationInstantiation with the annexB extensions. - Implement and use NamedEvaluation where specified. Additionally there are fixes to things exposed by these changes to eval, {for, for-in, for-of} loops and assignment. Finally it also fixes some tests in test-js which where passing before but not now that we have correct behavior :^).
2021-09-22 10:44:56 +00:00
private:
void throw_identifier_declared(FlyString const& name, NonnullRefPtr<Declaration> const& declaration)
{
m_parser.syntax_error(String::formatted("Identifier '{}' already declared", name), declaration->source_range().start);
LibJS: Hoist variable declarations to the nearest relevant scope "var" declarations are hoisted to the nearest function scope, while "let" and "const" are hoisted to the nearest block scope. This is done by the parser, which keeps two scope stacks, one stack for the current var scope and one for the current let/const scope. When the interpreter enters a scope, we walk all of the declarations and insert them into the variable environment. We don't support the temporal dead zone for let/const yet.
2020-04-13 14:42:54 +00:00
}
Parser& m_parser;
LibJS: Make scoping follow the spec Before this we used an ad-hoc combination of references and 'variables' stored in a hashmap. This worked in most cases but is not spec like. Additionally hoisting, dynamically naming functions and scope analysis was not done properly. This patch fixes all of that by: - Implement BindingInitialization for destructuring assignment. - Implementing a new ScopePusher which tracks the lexical and var scoped declarations. This hoists functions to the top level if no lexical declaration name overlaps. Furthermore we do checking of redeclarations in the ScopePusher now requiring less checks all over the place. - Add methods for parsing the directives and statement lists instead of having that code duplicated in multiple places. This allows declarations to pushed to the appropriate scope more easily. - Remove the non spec way of storing 'variables' in DeclarativeEnvironment and make Reference follow the spec instead of checking both the bindings and 'variables'. - Remove all scoping related things from the Interpreter. And instead use environments as specified by the spec. This also includes fixing that NativeFunctions did not produce a valid FunctionEnvironment which could cause issues with callbacks and eval. All FunctionObjects now have a valid NewFunctionEnvironment implementation. - Remove execute_statements from Interpreter and instead use ASTNode::execute everywhere this simplifies AST.cpp as you no longer need to worry about which method to call. - Make ScopeNodes setup their own environment. This uses four different methods specified by the spec {Block, Function, Eval, Global}DeclarationInstantiation with the annexB extensions. - Implement and use NamedEvaluation where specified. Additionally there are fixes to things exposed by these changes to eval, {for, for-in, for-of} loops and assignment. Finally it also fixes some tests in test-js which where passing before but not now that we have correct behavior :^).
2021-09-22 10:44:56 +00:00
ScopeNode* m_node { nullptr };
bool m_is_top_level { false };
ScopePusher* m_parent_scope { nullptr };
ScopePusher* m_top_level_scope { nullptr };
HashTable<FlyString> m_lexical_names;
HashTable<FlyString> m_var_names;
HashTable<FlyString> m_function_names;
HashTable<FlyString> m_forbidden_lexical_names;
HashTable<FlyString> m_forbidden_var_names;
NonnullRefPtrVector<FunctionDeclaration> m_functions_to_hoist;
LibJS: Make accessing the current function's arguments cheaper Instead of going through an environment record, make arguments of the currently executing function generate references via the argument index, which can later be resolved directly through the ExecutionContext.
2021-10-07 16:43:22 +00:00
Optional<Vector<FunctionDeclaration::Parameter>> m_function_parameters;
HashMap<size_t, NonnullRefPtrVector<Identifier>> m_identifier_and_argument_index_associations;
bool m_contains_access_to_arguments_object { false };
bool m_contains_direct_call_to_eval { false };
LibJS: Hoist variable declarations to the nearest relevant scope "var" declarations are hoisted to the nearest function scope, while "let" and "const" are hoisted to the nearest block scope. This is done by the parser, which keeps two scope stacks, one stack for the current var scope and one for the current let/const scope. When the interpreter enters a scope, we walk all of the declarations and insert them into the variable environment. We don't support the temporal dead zone for let/const yet.
2020-04-13 14:42:54 +00:00
};
LibJS: Parser refactored to use constexpr precedence table Replaced implementation dependent on HashMap with a constexpr PrecedenceTable based on array lookup.
2020-08-18 16:46:36 +00:00
class OperatorPrecedenceTable {
public:
constexpr OperatorPrecedenceTable()
: m_token_precedence()
{
for (size_t i = 0; i < array_size(m_operator_precedence); ++i) {
auto& op = m_operator_precedence[i];
m_token_precedence[static_cast<size_t>(op.token)] = op.precedence;
}
}
LibJS: Add Parser save_state() and load_state() functions These functions allow us to try to parse ambiguous expressions (such as arrow function arguments in parentheses), and rewind the state of the Parser if an expression candidate failed to parse.
2020-03-30 13:24:43 +00:00
LibJS: Parser refactored to use constexpr precedence table Replaced implementation dependent on HashMap with a constexpr PrecedenceTable based on array lookup.
2020-08-18 16:46:36 +00:00
constexpr int get(TokenType token) const
{
int p = m_token_precedence[static_cast<size_t>(token)];
if (p == 0) {
LibJS: Use new format functions everywhere This changes the remaining uses of the following functions across LibJS: - String::format() => String::formatted() - dbg() => dbgln() - printf() => out(), outln() - fprintf() => warnln() I also removed the relevant 'LogStream& operator<<' overloads as they're not needed anymore.
2020-12-06 16:55:19 +00:00
warnln("Internal Error: No precedence for operator {}", Token::name(token));
Everywhere: Rename ASSERT => VERIFY (...and ASSERT_NOT_REACHED => VERIFY_NOT_REACHED) Since all of these checks are done in release builds as well, let's rename them to VERIFY to prevent confusion, as everyone is used to assertions being compiled out in release. We can introduce a new ASSERT macro that is specifically for debug checks, but I'm doing this wholesale conversion first since we've accumulated thousands of these already, and it's not immediately obvious which ones are suitable for ASSERT.
2021-02-23 19:42:32 +00:00
VERIFY_NOT_REACHED();
LibJS: Parser refactored to use constexpr precedence table Replaced implementation dependent on HashMap with a constexpr PrecedenceTable based on array lookup.
2020-08-18 16:46:36 +00:00
return -1;
}
LibJS: Add operator precedence parsing Obey precedence and associativity rules when parsing expressions with chained operators.
2020-03-12 22:02:41 +00:00
LibJS: Parser refactored to use constexpr precedence table Replaced implementation dependent on HashMap with a constexpr PrecedenceTable based on array lookup.
2020-08-18 16:46:36 +00:00
return p;
}
LibJS: Add operator precedence parsing Obey precedence and associativity rules when parsing expressions with chained operators.
2020-03-12 22:02:41 +00:00
LibJS: Parser refactored to use constexpr precedence table Replaced implementation dependent on HashMap with a constexpr PrecedenceTable based on array lookup.
2020-08-18 16:46:36 +00:00
private:
int m_token_precedence[cs_num_of_js_tokens];
LibJS: Add operator precedence parsing Obey precedence and associativity rules when parsing expressions with chained operators.
2020-03-12 22:02:41 +00:00
LibJS: Parser refactored to use constexpr precedence table Replaced implementation dependent on HashMap with a constexpr PrecedenceTable based on array lookup.
2020-08-18 16:46:36 +00:00
struct OperatorPrecedence {
TokenType token;
int precedence;
};
LibJS: Add operator precedence parsing Obey precedence and associativity rules when parsing expressions with chained operators.
2020-03-12 22:02:41 +00:00
LibJS: Parser refactored to use constexpr precedence table Replaced implementation dependent on HashMap with a constexpr PrecedenceTable based on array lookup.
2020-08-18 16:46:36 +00:00
// https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Operator_Precedence
static constexpr const OperatorPrecedence m_operator_precedence[] = {
{ TokenType::Period, 20 },
{ TokenType::BracketOpen, 20 },
{ TokenType::ParenOpen, 20 },
{ TokenType::QuestionMarkPeriod, 20 },
LibJS: Add operator precedence parsing Obey precedence and associativity rules when parsing expressions with chained operators.
2020-03-12 22:02:41 +00:00
LibJS: Parser refactored to use constexpr precedence table Replaced implementation dependent on HashMap with a constexpr PrecedenceTable based on array lookup.
2020-08-18 16:46:36 +00:00
{ TokenType::New, 19 },
LibJS: Add operator precedence parsing Obey precedence and associativity rules when parsing expressions with chained operators.
2020-03-12 22:02:41 +00:00
LibJS: Parser refactored to use constexpr precedence table Replaced implementation dependent on HashMap with a constexpr PrecedenceTable based on array lookup.
2020-08-18 16:46:36 +00:00
{ TokenType::PlusPlus, 18 },
{ TokenType::MinusMinus, 18 },
LibJS: Add operator precedence parsing Obey precedence and associativity rules when parsing expressions with chained operators.
2020-03-12 22:02:41 +00:00
LibJS: Parser refactored to use constexpr precedence table Replaced implementation dependent on HashMap with a constexpr PrecedenceTable based on array lookup.
2020-08-18 16:46:36 +00:00
{ TokenType::ExclamationMark, 17 },
{ TokenType::Tilde, 17 },
{ TokenType::Typeof, 17 },
{ TokenType::Void, 17 },
{ TokenType::Delete, 17 },
{ TokenType::Await, 17 },
LibJS: Add operator precedence parsing Obey precedence and associativity rules when parsing expressions with chained operators.
2020-03-12 22:02:41 +00:00
LibJS: Parser refactored to use constexpr precedence table Replaced implementation dependent on HashMap with a constexpr PrecedenceTable based on array lookup.
2020-08-18 16:46:36 +00:00
{ TokenType::DoubleAsterisk, 16 },
LibJS: Add operator precedence parsing Obey precedence and associativity rules when parsing expressions with chained operators.
2020-03-12 22:02:41 +00:00
LibJS: Parser refactored to use constexpr precedence table Replaced implementation dependent on HashMap with a constexpr PrecedenceTable based on array lookup.
2020-08-18 16:46:36 +00:00
{ TokenType::Asterisk, 15 },
{ TokenType::Slash, 15 },
{ TokenType::Percent, 15 },
LibJS: Add operator precedence parsing Obey precedence and associativity rules when parsing expressions with chained operators.
2020-03-12 22:02:41 +00:00
LibJS: Parser refactored to use constexpr precedence table Replaced implementation dependent on HashMap with a constexpr PrecedenceTable based on array lookup.
2020-08-18 16:46:36 +00:00
{ TokenType::Plus, 14 },
{ TokenType::Minus, 14 },
LibJS: Add operator precedence parsing Obey precedence and associativity rules when parsing expressions with chained operators.
2020-03-12 22:02:41 +00:00
LibJS: Parser refactored to use constexpr precedence table Replaced implementation dependent on HashMap with a constexpr PrecedenceTable based on array lookup.
2020-08-18 16:46:36 +00:00
{ TokenType::ShiftLeft, 13 },
{ TokenType::ShiftRight, 13 },
{ TokenType::UnsignedShiftRight, 13 },
LibJS: Add operator precedence parsing Obey precedence and associativity rules when parsing expressions with chained operators.
2020-03-12 22:02:41 +00:00
LibJS: Parser refactored to use constexpr precedence table Replaced implementation dependent on HashMap with a constexpr PrecedenceTable based on array lookup.
2020-08-18 16:46:36 +00:00
{ TokenType::LessThan, 12 },
{ TokenType::LessThanEquals, 12 },
{ TokenType::GreaterThan, 12 },
{ TokenType::GreaterThanEquals, 12 },
{ TokenType::In, 12 },
{ TokenType::Instanceof, 12 },
LibJS: Add operator precedence parsing Obey precedence and associativity rules when parsing expressions with chained operators.
2020-03-12 22:02:41 +00:00
LibJS: Parser refactored to use constexpr precedence table Replaced implementation dependent on HashMap with a constexpr PrecedenceTable based on array lookup.
2020-08-18 16:46:36 +00:00
{ TokenType::EqualsEquals, 11 },
{ TokenType::ExclamationMarkEquals, 11 },
{ TokenType::EqualsEqualsEquals, 11 },
{ TokenType::ExclamationMarkEqualsEquals, 11 },
LibJS: Add operator precedence parsing Obey precedence and associativity rules when parsing expressions with chained operators.
2020-03-12 22:02:41 +00:00
LibJS: Parser refactored to use constexpr precedence table Replaced implementation dependent on HashMap with a constexpr PrecedenceTable based on array lookup.
2020-08-18 16:46:36 +00:00
{ TokenType::Ampersand, 10 },
LibJS: Add operator precedence parsing Obey precedence and associativity rules when parsing expressions with chained operators.
2020-03-12 22:02:41 +00:00
LibJS: Parser refactored to use constexpr precedence table Replaced implementation dependent on HashMap with a constexpr PrecedenceTable based on array lookup.
2020-08-18 16:46:36 +00:00
{ TokenType::Caret, 9 },
LibJS: Add operator precedence parsing Obey precedence and associativity rules when parsing expressions with chained operators.
2020-03-12 22:02:41 +00:00
LibJS: Parser refactored to use constexpr precedence table Replaced implementation dependent on HashMap with a constexpr PrecedenceTable based on array lookup.
2020-08-18 16:46:36 +00:00
{ TokenType::Pipe, 8 },
LibJS: Add operator precedence parsing Obey precedence and associativity rules when parsing expressions with chained operators.
2020-03-12 22:02:41 +00:00
LibJS: Parser refactored to use constexpr precedence table Replaced implementation dependent on HashMap with a constexpr PrecedenceTable based on array lookup.
2020-08-18 16:46:36 +00:00
{ TokenType::DoubleQuestionMark, 7 },
LibJS: Add operator precedence parsing Obey precedence and associativity rules when parsing expressions with chained operators.
2020-03-12 22:02:41 +00:00
LibJS: Parser refactored to use constexpr precedence table Replaced implementation dependent on HashMap with a constexpr PrecedenceTable based on array lookup.
2020-08-18 16:46:36 +00:00
{ TokenType::DoubleAmpersand, 6 },
LibJS: Add operator precedence parsing Obey precedence and associativity rules when parsing expressions with chained operators.
2020-03-12 22:02:41 +00:00
LibJS: Parser refactored to use constexpr precedence table Replaced implementation dependent on HashMap with a constexpr PrecedenceTable based on array lookup.
2020-08-18 16:46:36 +00:00
{ TokenType::DoublePipe, 5 },
{ TokenType::QuestionMark, 4 },
{ TokenType::Equals, 3 },
{ TokenType::PlusEquals, 3 },
{ TokenType::MinusEquals, 3 },
{ TokenType::DoubleAsteriskEquals, 3 },
{ TokenType::AsteriskEquals, 3 },
{ TokenType::SlashEquals, 3 },
{ TokenType::PercentEquals, 3 },
{ TokenType::ShiftLeftEquals, 3 },
{ TokenType::ShiftRightEquals, 3 },
{ TokenType::UnsignedShiftRightEquals, 3 },
{ TokenType::AmpersandEquals, 3 },
{ TokenType::CaretEquals, 3 },
LibJS: Implement logical assignment operators (&&=, ||=, ??=) TC39 proposal, stage 4 as of 2020-07. https://tc39.es/proposal-logical-assignment/
2020-10-05 15:49:43 +00:00
{ TokenType::PipeEquals, 3 },
{ TokenType::DoubleAmpersandEquals, 3 },
{ TokenType::DoublePipeEquals, 3 },
{ TokenType::DoubleQuestionMarkEquals, 3 },
LibJS: Parser refactored to use constexpr precedence table Replaced implementation dependent on HashMap with a constexpr PrecedenceTable based on array lookup.
2020-08-18 16:46:36 +00:00
{ TokenType::Yield, 2 },
{ TokenType::Comma, 1 },
};
};
constexpr OperatorPrecedenceTable g_operator_precedence;
LibJS: Add operator precedence parsing Obey precedence and associativity rules when parsing expressions with chained operators.
2020-03-12 22:02:41 +00:00
LibJS: Add a mode to parse JS as a module In a module strict mode should be enabled at the start of parsing and we allow import and export statements.
2021-08-14 15:30:37 +00:00
Parser::ParserState::ParserState(Lexer l, Program::Type program_type)
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
: lexer(move(l))
LibJS: Add operator precedence parsing Obey precedence and associativity rules when parsing expressions with chained operators.
2020-03-12 22:02:41 +00:00
{
LibJS: Add a mode to parse JS as a module In a module strict mode should be enabled at the start of parsing and we allow import and export statements.
2021-08-14 15:30:37 +00:00
if (program_type == Program::Type::Module)
lexer.disallow_html_comments();
current_token = lexer.next();
LibJS: Parser refactored to use constexpr precedence table Replaced implementation dependent on HashMap with a constexpr PrecedenceTable based on array lookup.
2020-08-18 16:46:36 +00:00
}
LibJS: Add operator precedence parsing Obey precedence and associativity rules when parsing expressions with chained operators.
2020-03-12 22:02:41 +00:00
LibJS: Add a mode to parse JS as a module In a module strict mode should be enabled at the start of parsing and we allow import and export statements.
2021-08-14 15:30:37 +00:00
Parser::Parser(Lexer lexer, Program::Type program_type)
: m_state(move(lexer), program_type)
, m_program_type(program_type)
LibJS: Parser refactored to use constexpr precedence table Replaced implementation dependent on HashMap with a constexpr PrecedenceTable based on array lookup.
2020-08-18 16:46:36 +00:00
{
LibJS: Add operator precedence parsing Obey precedence and associativity rules when parsing expressions with chained operators.
2020-03-12 22:02:41 +00:00
}
Associativity Parser::operator_associativity(TokenType type) const
{
switch (type) {
case TokenType::Period:
case TokenType::BracketOpen:
case TokenType::ParenOpen:
case TokenType::QuestionMarkPeriod:
case TokenType::Asterisk:
case TokenType::Slash:
case TokenType::Percent:
case TokenType::Plus:
case TokenType::Minus:
case TokenType::ShiftLeft:
case TokenType::ShiftRight:
case TokenType::UnsignedShiftRight:
case TokenType::LessThan:
case TokenType::LessThanEquals:
case TokenType::GreaterThan:
case TokenType::GreaterThanEquals:
case TokenType::In:
case TokenType::Instanceof:
case TokenType::EqualsEquals:
case TokenType::ExclamationMarkEquals:
case TokenType::EqualsEqualsEquals:
case TokenType::ExclamationMarkEqualsEquals:
LibJS: Fix broken parsing of `!o.a` Unary expressions parsing now respects precedence and associativity of operators. This patch also makes `typeof` left-associative which was an oversight. Thanks to Conrad for helping me work this out. :^)
2020-03-28 10:48:52 +00:00
case TokenType::Typeof:
LibJS: Implement void operator
2020-04-15 16:55:03 +00:00
case TokenType::Void:
LibJS: Implement basic support for the "delete" operator It turns out "delete" is actually a unary op :) This patch implements deletion of object properties, it doesn't yet work for casually deleting properties from the global object. When deleting a property from an object, we switch that object to having a unique shape, no longer sharing shapes with others. Once an object has a unique shape, it no longer needs to care about shape transitions.
2020-04-26 11:53:40 +00:00
case TokenType::Delete:
LibJS: Add support for await expressions
2021-11-09 20:52:21 +00:00
case TokenType::Await:
LibJS: Add operator precedence parsing Obey precedence and associativity rules when parsing expressions with chained operators.
2020-03-12 22:02:41 +00:00
case TokenType::Ampersand:
case TokenType::Caret:
case TokenType::Pipe:
case TokenType::DoubleQuestionMark:
case TokenType::DoubleAmpersand:
case TokenType::DoublePipe:
case TokenType::Comma:
return Associativity::Left;
default:
return Associativity::Right;
}
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
}
LibJS: Make scoping follow the spec Before this we used an ad-hoc combination of references and 'variables' stored in a hashmap. This worked in most cases but is not spec like. Additionally hoisting, dynamically naming functions and scope analysis was not done properly. This patch fixes all of that by: - Implement BindingInitialization for destructuring assignment. - Implementing a new ScopePusher which tracks the lexical and var scoped declarations. This hoists functions to the top level if no lexical declaration name overlaps. Furthermore we do checking of redeclarations in the ScopePusher now requiring less checks all over the place. - Add methods for parsing the directives and statement lists instead of having that code duplicated in multiple places. This allows declarations to pushed to the appropriate scope more easily. - Remove the non spec way of storing 'variables' in DeclarativeEnvironment and make Reference follow the spec instead of checking both the bindings and 'variables'. - Remove all scoping related things from the Interpreter. And instead use environments as specified by the spec. This also includes fixing that NativeFunctions did not produce a valid FunctionEnvironment which could cause issues with callbacks and eval. All FunctionObjects now have a valid NewFunctionEnvironment implementation. - Remove execute_statements from Interpreter and instead use ASTNode::execute everywhere this simplifies AST.cpp as you no longer need to worry about which method to call. - Make ScopeNodes setup their own environment. This uses four different methods specified by the spec {Block, Function, Eval, Global}DeclarationInstantiation with the annexB extensions. - Implement and use NamedEvaluation where specified. Additionally there are fixes to things exposed by these changes to eval, {for, for-in, for-of} loops and assignment. Finally it also fixes some tests in test-js which where passing before but not now that we have correct behavior :^).
2021-09-22 10:44:56 +00:00
bool Parser::parse_directive(ScopeNode& body)
{
bool found_use_strict = false;
while (!done() && match(TokenType::StringLiteral)) {
// It cannot be a labelled function since we hit a string literal.
auto statement = parse_statement(AllowLabelledFunction::No);
body.append(statement);
VERIFY(is<ExpressionStatement>(*statement));
auto& expression = static_cast<ExpressionStatement const&>(*statement).expression();
if (!is<StringLiteral>(expression))
break;
auto& string_literal = static_cast<StringLiteral const&>(expression);
if (string_literal.is_use_strict_directive()) {
found_use_strict = true;
if (m_state.string_legacy_octal_escape_sequence_in_scope)
syntax_error("Octal escape sequence in string literal not allowed in strict mode");
break;
}
}
m_state.string_legacy_octal_escape_sequence_in_scope = false;
return found_use_strict;
}
LibJS: Correct behaviour of direct vs. indirect eval eval only has direct access to the local scope when accessed through the name eval. This includes locals named eval, because of course it does.
2021-06-20 03:13:53 +00:00
NonnullRefPtr<Program> Parser::parse_program(bool starts_in_strict_mode)
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
{
LibJS: Track source positions all the way down to exceptions This makes exceptions have a trace of source positions too, which could probably be helpful in making fancier error tracebacks.
2020-12-28 17:15:22 +00:00
auto rule_start = push_start();
LibJS: Add a mode to parse JS as a module In a module strict mode should be enabled at the start of parsing and we allow import and export statements.
2021-08-14 15:30:37 +00:00
auto program = adopt_ref(*new Program({ m_filename, rule_start.position(), position() }, m_program_type));
LibJS: Make scoping follow the spec Before this we used an ad-hoc combination of references and 'variables' stored in a hashmap. This worked in most cases but is not spec like. Additionally hoisting, dynamically naming functions and scope analysis was not done properly. This patch fixes all of that by: - Implement BindingInitialization for destructuring assignment. - Implementing a new ScopePusher which tracks the lexical and var scoped declarations. This hoists functions to the top level if no lexical declaration name overlaps. Furthermore we do checking of redeclarations in the ScopePusher now requiring less checks all over the place. - Add methods for parsing the directives and statement lists instead of having that code duplicated in multiple places. This allows declarations to pushed to the appropriate scope more easily. - Remove the non spec way of storing 'variables' in DeclarativeEnvironment and make Reference follow the spec instead of checking both the bindings and 'variables'. - Remove all scoping related things from the Interpreter. And instead use environments as specified by the spec. This also includes fixing that NativeFunctions did not produce a valid FunctionEnvironment which could cause issues with callbacks and eval. All FunctionObjects now have a valid NewFunctionEnvironment implementation. - Remove execute_statements from Interpreter and instead use ASTNode::execute everywhere this simplifies AST.cpp as you no longer need to worry about which method to call. - Make ScopeNodes setup their own environment. This uses four different methods specified by the spec {Block, Function, Eval, Global}DeclarationInstantiation with the annexB extensions. - Implement and use NamedEvaluation where specified. Additionally there are fixes to things exposed by these changes to eval, {for, for-in, for-of} loops and assignment. Finally it also fixes some tests in test-js which where passing before but not now that we have correct behavior :^).
2021-09-22 10:44:56 +00:00
ScopePusher program_scope = ScopePusher::program_scope(*this, *program);
if (starts_in_strict_mode || m_program_type == Program::Type::Module)
m_state.strict_mode = true;
bool has_use_strict = parse_directive(program);
if (m_state.strict_mode || has_use_strict) {
LibJS: Correct behaviour of direct vs. indirect eval eval only has direct access to the local scope when accessed through the name eval. This includes locals named eval, because of course it does.
2021-06-20 03:13:53 +00:00
program->set_strict_mode();
m_state.strict_mode = true;
}
LibJS: Add strict mode Adds the ability for a scope (either a function or the entire program) to be in strict mode. Scopes default to non-strict mode. There are two ways to determine the strict-ness of the JS engine: 1. In the parser, this can be accessed with the parser_state variable m_is_strict_mode boolean. If true, the Parser is currently parsing in strict mode. This is done so that the Parser can generate syntax errors at parse time, which is required in some cases. 2. With Interpreter.is_strict_mode(). This allows strict mode checking at runtime as opposed to compile time. Additionally, in order to test this, a global isStrictMode() function has been added to the JS ReplObject under the test-mode flag.
2020-05-28 05:22:08 +00:00
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
while (!done()) {
LibJS: Make scoping follow the spec Before this we used an ad-hoc combination of references and 'variables' stored in a hashmap. This worked in most cases but is not spec like. Additionally hoisting, dynamically naming functions and scope analysis was not done properly. This patch fixes all of that by: - Implement BindingInitialization for destructuring assignment. - Implementing a new ScopePusher which tracks the lexical and var scoped declarations. This hoists functions to the top level if no lexical declaration name overlaps. Furthermore we do checking of redeclarations in the ScopePusher now requiring less checks all over the place. - Add methods for parsing the directives and statement lists instead of having that code duplicated in multiple places. This allows declarations to pushed to the appropriate scope more easily. - Remove the non spec way of storing 'variables' in DeclarativeEnvironment and make Reference follow the spec instead of checking both the bindings and 'variables'. - Remove all scoping related things from the Interpreter. And instead use environments as specified by the spec. This also includes fixing that NativeFunctions did not produce a valid FunctionEnvironment which could cause issues with callbacks and eval. All FunctionObjects now have a valid NewFunctionEnvironment implementation. - Remove execute_statements from Interpreter and instead use ASTNode::execute everywhere this simplifies AST.cpp as you no longer need to worry about which method to call. - Make ScopeNodes setup their own environment. This uses four different methods specified by the spec {Block, Function, Eval, Global}DeclarationInstantiation with the annexB extensions. - Implement and use NamedEvaluation where specified. Additionally there are fixes to things exposed by these changes to eval, {for, for-in, for-of} loops and assignment. Finally it also fixes some tests in test-js which where passing before but not now that we have correct behavior :^).
2021-09-22 10:44:56 +00:00
parse_statement_list(program, AllowLabelledFunction::Yes);
LibJS: Be less strict about 'use strict' It is allowed to have (bogus) directives e.g. 'does nothing'; 'same'; 'use strict'; Should still trigger strict mode.
2021-07-11 23:25:32 +00:00
LibJS: Make scoping follow the spec Before this we used an ad-hoc combination of references and 'variables' stored in a hashmap. This worked in most cases but is not spec like. Additionally hoisting, dynamically naming functions and scope analysis was not done properly. This patch fixes all of that by: - Implement BindingInitialization for destructuring assignment. - Implementing a new ScopePusher which tracks the lexical and var scoped declarations. This hoists functions to the top level if no lexical declaration name overlaps. Furthermore we do checking of redeclarations in the ScopePusher now requiring less checks all over the place. - Add methods for parsing the directives and statement lists instead of having that code duplicated in multiple places. This allows declarations to pushed to the appropriate scope more easily. - Remove the non spec way of storing 'variables' in DeclarativeEnvironment and make Reference follow the spec instead of checking both the bindings and 'variables'. - Remove all scoping related things from the Interpreter. And instead use environments as specified by the spec. This also includes fixing that NativeFunctions did not produce a valid FunctionEnvironment which could cause issues with callbacks and eval. All FunctionObjects now have a valid NewFunctionEnvironment implementation. - Remove execute_statements from Interpreter and instead use ASTNode::execute everywhere this simplifies AST.cpp as you no longer need to worry about which method to call. - Make ScopeNodes setup their own environment. This uses four different methods specified by the spec {Block, Function, Eval, Global}DeclarationInstantiation with the annexB extensions. - Implement and use NamedEvaluation where specified. Additionally there are fixes to things exposed by these changes to eval, {for, for-in, for-of} loops and assignment. Finally it also fixes some tests in test-js which where passing before but not now that we have correct behavior :^).
2021-09-22 10:44:56 +00:00
if (done())
break;
LibJS: Be less strict about 'use strict' It is allowed to have (bogus) directives e.g. 'does nothing'; 'same'; 'use strict'; Should still trigger strict mode.
2021-07-11 23:25:32 +00:00
LibJS: Make scoping follow the spec Before this we used an ad-hoc combination of references and 'variables' stored in a hashmap. This worked in most cases but is not spec like. Additionally hoisting, dynamically naming functions and scope analysis was not done properly. This patch fixes all of that by: - Implement BindingInitialization for destructuring assignment. - Implementing a new ScopePusher which tracks the lexical and var scoped declarations. This hoists functions to the top level if no lexical declaration name overlaps. Furthermore we do checking of redeclarations in the ScopePusher now requiring less checks all over the place. - Add methods for parsing the directives and statement lists instead of having that code duplicated in multiple places. This allows declarations to pushed to the appropriate scope more easily. - Remove the non spec way of storing 'variables' in DeclarativeEnvironment and make Reference follow the spec instead of checking both the bindings and 'variables'. - Remove all scoping related things from the Interpreter. And instead use environments as specified by the spec. This also includes fixing that NativeFunctions did not produce a valid FunctionEnvironment which could cause issues with callbacks and eval. All FunctionObjects now have a valid NewFunctionEnvironment implementation. - Remove execute_statements from Interpreter and instead use ASTNode::execute everywhere this simplifies AST.cpp as you no longer need to worry about which method to call. - Make ScopeNodes setup their own environment. This uses four different methods specified by the spec {Block, Function, Eval, Global}DeclarationInstantiation with the annexB extensions. - Implement and use NamedEvaluation where specified. Additionally there are fixes to things exposed by these changes to eval, {for, for-in, for-of} loops and assignment. Finally it also fixes some tests in test-js which where passing before but not now that we have correct behavior :^).
2021-09-22 10:44:56 +00:00
if (match_export_or_import()) {
LibJS: Parse and partially execute import and export statements We produce the import and export entries as per the spec. However we do not yet verify that named things that are exported are declared somewhere.
2021-08-14 15:42:30 +00:00
VERIFY(m_state.current_token.type() == TokenType::Export || m_state.current_token.type() == TokenType::Import);
if (m_state.current_token.type() == TokenType::Export)
program->append_export(parse_export_statement(*program));
else
program->append_import(parse_import_statement(*program));
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
} else {
LibJS: Distinguish between statement and declaration This separates matching/parsing of statements and declarations and fixes a few edge cases where the parser would incorrectly accept a declaration where only a statement is allowed - for example: if (foo) const a = 1; for (var bar;;) function b() {} while (baz) class c {}
2020-10-22 22:30:07 +00:00
expected("statement or declaration");
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
consume();
}
}
LibJS: Make scoping follow the spec Before this we used an ad-hoc combination of references and 'variables' stored in a hashmap. This worked in most cases but is not spec like. Additionally hoisting, dynamically naming functions and scope analysis was not done properly. This patch fixes all of that by: - Implement BindingInitialization for destructuring assignment. - Implementing a new ScopePusher which tracks the lexical and var scoped declarations. This hoists functions to the top level if no lexical declaration name overlaps. Furthermore we do checking of redeclarations in the ScopePusher now requiring less checks all over the place. - Add methods for parsing the directives and statement lists instead of having that code duplicated in multiple places. This allows declarations to pushed to the appropriate scope more easily. - Remove the non spec way of storing 'variables' in DeclarativeEnvironment and make Reference follow the spec instead of checking both the bindings and 'variables'. - Remove all scoping related things from the Interpreter. And instead use environments as specified by the spec. This also includes fixing that NativeFunctions did not produce a valid FunctionEnvironment which could cause issues with callbacks and eval. All FunctionObjects now have a valid NewFunctionEnvironment implementation. - Remove execute_statements from Interpreter and instead use ASTNode::execute everywhere this simplifies AST.cpp as you no longer need to worry about which method to call. - Make ScopeNodes setup their own environment. This uses four different methods specified by the spec {Block, Function, Eval, Global}DeclarationInstantiation with the annexB extensions. - Implement and use NamedEvaluation where specified. Additionally there are fixes to things exposed by these changes to eval, {for, for-in, for-of} loops and assignment. Finally it also fixes some tests in test-js which where passing before but not now that we have correct behavior :^).
2021-09-22 10:44:56 +00:00
LibJS: Track source positions all the way down to exceptions This makes exceptions have a trace of source positions too, which could probably be helpful in making fancier error tracebacks.
2020-12-28 17:15:22 +00:00
program->source_range().end = position();
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
return program;
}
LibJS: Distinguish between statement and declaration This separates matching/parsing of statements and declarations and fixes a few edge cases where the parser would incorrectly accept a declaration where only a statement is allowed - for example: if (foo) const a = 1; for (var bar;;) function b() {} while (baz) class c {}
2020-10-22 22:30:07 +00:00
NonnullRefPtr<Declaration> Parser::parse_declaration()
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
{
LibJS: Track source positions all the way down to exceptions This makes exceptions have a trace of source positions too, which could probably be helpful in making fancier error tracebacks.
2020-12-28 17:15:22 +00:00
auto rule_start = push_start();
LibJS: Add support for async functions This commit adds support for the most bare bones version of async functions, support for async generator functions, async arrow functions and await expressions are TODO.
2021-11-09 18:39:22 +00:00
if (m_state.current_token.type() == TokenType::Async && next_token().type() == TokenType::Function)
return parse_function_node<FunctionDeclaration>();
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
switch (m_state.current_token.type()) {
LibJS: Initial class implementation; allow super expressions in object literal methods; add EnvrionmentRecord fields and methods to LexicalEnvironment Adding EnvrionmentRecord's fields and methods lets us throw an exception when |this| is not initialized, which occurs when the super constructor in a derived class has not yet been called, or when |this| has already been initialized (the super constructor was already called).
2020-06-08 18:31:21 +00:00
case TokenType::Class:
LibJS: Avoid unnecessary lambda Especially when it's evaluated immediately and unconditionally.
2020-08-29 11:25:37 +00:00
return parse_class_declaration();
LibJS: Make scoping follow the spec Before this we used an ad-hoc combination of references and 'variables' stored in a hashmap. This worked in most cases but is not spec like. Additionally hoisting, dynamically naming functions and scope analysis was not done properly. This patch fixes all of that by: - Implement BindingInitialization for destructuring assignment. - Implementing a new ScopePusher which tracks the lexical and var scoped declarations. This hoists functions to the top level if no lexical declaration name overlaps. Furthermore we do checking of redeclarations in the ScopePusher now requiring less checks all over the place. - Add methods for parsing the directives and statement lists instead of having that code duplicated in multiple places. This allows declarations to pushed to the appropriate scope more easily. - Remove the non spec way of storing 'variables' in DeclarativeEnvironment and make Reference follow the spec instead of checking both the bindings and 'variables'. - Remove all scoping related things from the Interpreter. And instead use environments as specified by the spec. This also includes fixing that NativeFunctions did not produce a valid FunctionEnvironment which could cause issues with callbacks and eval. All FunctionObjects now have a valid NewFunctionEnvironment implementation. - Remove execute_statements from Interpreter and instead use ASTNode::execute everywhere this simplifies AST.cpp as you no longer need to worry about which method to call. - Make ScopeNodes setup their own environment. This uses four different methods specified by the spec {Block, Function, Eval, Global}DeclarationInstantiation with the annexB extensions. - Implement and use NamedEvaluation where specified. Additionally there are fixes to things exposed by these changes to eval, {for, for-in, for-of} loops and assignment. Finally it also fixes some tests in test-js which where passing before but not now that we have correct behavior :^).
2021-09-22 10:44:56 +00:00
case TokenType::Function:
return parse_function_node<FunctionDeclaration>();
LibJS: Distinguish between statement and declaration This separates matching/parsing of statements and declarations and fixes a few edge cases where the parser would incorrectly accept a declaration where only a statement is allowed - for example: if (foo) const a = 1; for (var bar;;) function b() {} while (baz) class c {}
2020-10-22 22:30:07 +00:00
case TokenType::Let:
case TokenType::Const:
return parse_variable_declaration();
default:
expected("declaration");
consume();
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
return create_ast_node<ErrorDeclaration>({ m_state.current_token.filename(), rule_start.position(), position() });
LibJS: Distinguish between statement and declaration This separates matching/parsing of statements and declarations and fixes a few edge cases where the parser would incorrectly accept a declaration where only a statement is allowed - for example: if (foo) const a = 1; for (var bar;;) function b() {} while (baz) class c {}
2020-10-22 22:30:07 +00:00
}
}
LibJS: Allow labelled functions in certain contexts Also disallow duplicated labels.
2021-07-24 23:01:22 +00:00
NonnullRefPtr<Statement> Parser::parse_statement(AllowLabelledFunction allow_labelled_function)
LibJS: Distinguish between statement and declaration This separates matching/parsing of statements and declarations and fixes a few edge cases where the parser would incorrectly accept a declaration where only a statement is allowed - for example: if (foo) const a = 1; for (var bar;;) function b() {} while (baz) class c {}
2020-10-22 22:30:07 +00:00
{
LibJS: Track source positions all the way down to exceptions This makes exceptions have a trace of source positions too, which could probably be helpful in making fancier error tracebacks.
2020-12-28 17:15:22 +00:00
auto rule_start = push_start();
LibJS: Fix small issues in parser - Fix some places where escaped keywords are (not) allowed. - Be more strict about parameters for functions with 'use strict'. - Fix that expressions statements allowed functions and classes. - Fix that class expressions were not allowed. - Added a new next_token() method for checking the look ahead. - Fix that continue labels could jump to non iterating targets. - Fix that generator functions cannot be declared in if statements.
2021-08-28 15:04:37 +00:00
auto type = m_state.current_token.type();
switch (type) {
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
case TokenType::CurlyOpen:
return parse_block_statement();
case TokenType::Return:
return parse_return_statement();
LibJS: Make scoping follow the spec Before this we used an ad-hoc combination of references and 'variables' stored in a hashmap. This worked in most cases but is not spec like. Additionally hoisting, dynamically naming functions and scope analysis was not done properly. This patch fixes all of that by: - Implement BindingInitialization for destructuring assignment. - Implementing a new ScopePusher which tracks the lexical and var scoped declarations. This hoists functions to the top level if no lexical declaration name overlaps. Furthermore we do checking of redeclarations in the ScopePusher now requiring less checks all over the place. - Add methods for parsing the directives and statement lists instead of having that code duplicated in multiple places. This allows declarations to pushed to the appropriate scope more easily. - Remove the non spec way of storing 'variables' in DeclarativeEnvironment and make Reference follow the spec instead of checking both the bindings and 'variables'. - Remove all scoping related things from the Interpreter. And instead use environments as specified by the spec. This also includes fixing that NativeFunctions did not produce a valid FunctionEnvironment which could cause issues with callbacks and eval. All FunctionObjects now have a valid NewFunctionEnvironment implementation. - Remove execute_statements from Interpreter and instead use ASTNode::execute everywhere this simplifies AST.cpp as you no longer need to worry about which method to call. - Make ScopeNodes setup their own environment. This uses four different methods specified by the spec {Block, Function, Eval, Global}DeclarationInstantiation with the annexB extensions. - Implement and use NamedEvaluation where specified. Additionally there are fixes to things exposed by these changes to eval, {for, for-in, for-of} loops and assignment. Finally it also fixes some tests in test-js which where passing before but not now that we have correct behavior :^).
2021-09-22 10:44:56 +00:00
case TokenType::Var: {
auto declaration = parse_variable_declaration();
m_state.current_scope_pusher->add_declaration(declaration);
return declaration;
}
LibJS: Implement for statement
2020-03-12 12:12:12 +00:00
case TokenType::For:
return parse_for_statement();
LibJS: Parse "if" statements This patch implements basic parsing of "if" statements. We don't yet support parsing "else", so I added a FIXME about that.
2020-03-21 17:40:17 +00:00
case TokenType::If:
return parse_if_statement();
LibJS: Implement "throw" You can now throw an expression to the nearest catcher! :^) To support throwing arbitrary values, I added an Exception class that sits as a wrapper around whatever is thrown. In the future it will be a logical place to store a call stack.
2020-03-24 21:03:50 +00:00
case TokenType::Throw:
return parse_throw_statement();
LibJS: Parse "try", "catch" and "finally" This is the first step towards support exceptions. :^)
2020-03-24 13:03:55 +00:00
case TokenType::Try:
return parse_try_statement();
LibJS: Lexer and parser support for "switch" statements
2020-03-29 11:09:54 +00:00
case TokenType::Break:
return parse_break_statement();
LibJS: Add support for "continue" inside "for" statements :^)
2020-04-04 22:22:42 +00:00
case TokenType::Continue:
LibJS: Avoid unnecessary lambda Especially when it's evaluated immediately and unconditionally.
2020-08-29 11:25:37 +00:00
return parse_continue_statement();
LibJS: Lexer and parser support for "switch" statements
2020-03-29 11:09:54 +00:00
case TokenType::Switch:
return parse_switch_statement();
LibJS: Add support for do..while statements
2020-04-04 19:29:23 +00:00
case TokenType::Do:
return parse_do_while_statement();
LibJS: Parse while statements
2020-04-21 18:27:57 +00:00
case TokenType::While:
return parse_while_statement();
LibJS: Parse "with" statements :^)
2020-11-28 14:05:57 +00:00
case TokenType::With:
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
if (m_state.strict_mode)
LibJS: Disallow 'with' statement in strict mode
2020-11-28 19:17:33 +00:00
syntax_error("'with' statement not allowed in strict mode");
LibJS: Parse "with" statements :^)
2020-11-28 14:05:57 +00:00
return parse_with_statement();
LibJS: Implement (no-op) debugger statement
2020-04-30 16:26:27 +00:00
case TokenType::Debugger:
LibJS: Avoid unnecessary lambda Especially when it's evaluated immediately and unconditionally.
2020-08-29 11:25:37 +00:00
return parse_debugger_statement();
LibJS: Support empty statements We already skipped random semicolons in Parser::parse_program(), but now they are properly matched and parsed as empty statements - and thus recognized as a valid body of an if / else / while / ... statement.
2020-05-03 09:59:00 +00:00
case TokenType::Semicolon:
consume();
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
return create_ast_node<EmptyStatement>({ m_state.current_token.filename(), rule_start.position(), position() });
LibJS: Force the lexer to parse a regex when expecting a statement
2021-07-29 21:28:28 +00:00
case TokenType::Slash:
case TokenType::SlashEquals:
m_state.current_token = m_state.lexer.force_slash_as_regex();
[[fallthrough]];
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
default:
LibJS: Handle escaped keywords in more cases and handle 'await' labels
2021-09-18 21:02:50 +00:00
if (match_invalid_escaped_keyword())
LibJS: Fix some small remaining issues with parsing unicode escapes Added a test to ensure the behavior stays the same. We now throw on a direct usage of an escaped keywords with a specific error to make it more clear to the user.
2021-08-21 09:27:20 +00:00
syntax_error("Keyword must not contain escaped characters");
LibJS: Allow labelled functions in certain contexts Also disallow duplicated labels.
2021-07-24 23:01:22 +00:00
if (match_identifier_name()) {
auto result = try_parse_labelled_statement(allow_labelled_function);
LibJS: Integrate labels into the Interpreter The interpreter now considers a statement or block's label when considering whether or not to break. All statements can be labelled.
2020-05-28 20:36:59 +00:00
if (!result.is_null())
return result.release_nonnull();
}
LibJS: Implement automatic semicolon insertion
2020-04-17 13:05:58 +00:00
if (match_expression()) {
LibJS: Add support for async functions This commit adds support for the most bare bones version of async functions, support for async generator functions, async arrow functions and await expressions are TODO.
2021-11-09 18:39:22 +00:00
if (match(TokenType::Function) || (match(TokenType::Async) && next_token().type() == TokenType::Function) || match(TokenType::Class))
LibJS: Fix small issues in parser - Fix some places where escaped keywords are (not) allowed. - Be more strict about parameters for functions with 'use strict'. - Fix that expressions statements allowed functions and classes. - Fix that class expressions were not allowed. - Added a new next_token() method for checking the look ahead. - Fix that continue labels could jump to non iterating targets. - Fix that generator functions cannot be declared in if statements.
2021-08-28 15:04:37 +00:00
syntax_error(String::formatted("{} declaration not allowed in single-statement context", m_state.current_token.name()));
if (match(TokenType::Let) && next_token().type() == TokenType::BracketOpen)
syntax_error(String::formatted("let followed by [ is not allowed in single-statement context"));
LibJS: Implement automatic semicolon insertion
2020-04-17 13:05:58 +00:00
auto expr = parse_expression(0);
consume_or_insert_semicolon();
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
return create_ast_node<ExpressionStatement>({ m_state.current_token.filename(), rule_start.position(), position() }, move(expr));
LibJS: Implement automatic semicolon insertion
2020-04-17 13:05:58 +00:00
}
LibJS: Distinguish between statement and declaration This separates matching/parsing of statements and declarations and fixes a few edge cases where the parser would incorrectly accept a declaration where only a statement is allowed - for example: if (foo) const a = 1; for (var bar;;) function b() {} while (baz) class c {}
2020-10-22 22:30:07 +00:00
expected("statement");
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
consume();
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
return create_ast_node<ErrorStatement>({ m_state.current_token.filename(), rule_start.position(), position() });
LibJS: Avoid unnecessary lambda Especially when it's evaluated immediately and unconditionally.
2020-08-29 11:25:37 +00:00
}
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
}
LibJS: Handle escaped keywords in more cases and handle 'await' labels
2021-09-18 21:02:50 +00:00
bool Parser::match_invalid_escaped_keyword() const
{
if (m_state.current_token.type() != TokenType::EscapedKeyword)
return false;
auto token_value = m_state.current_token.value();
if (token_value == "await"sv) {
return m_program_type == Program::Type::Module;
}
if (m_state.strict_mode) {
return true;
}
return token_value != "yield"sv && token_value != "let"sv;
}
LibJS: Be more strict about reserved and special identifiers
2021-07-11 23:27:35 +00:00
static constexpr AK::Array<StringView, 9> strict_reserved_words = { "implements", "interface", "let", "package", "private", "protected", "public", "static", "yield" };
static bool is_strict_reserved_word(StringView str)
{
Everywhere: Pass AK::StringView by value
2021-11-10 23:55:02 +00:00
return any_of(strict_reserved_words, [&str](StringView word) {
LibJS: Be more strict about reserved and special identifiers
2021-07-11 23:27:35 +00:00
return word == str;
});
}
LibJS: Fix small issues in parser - Fix some places where escaped keywords are (not) allowed. - Be more strict about parameters for functions with 'use strict'. - Fix that expressions statements allowed functions and classes. - Fix that class expressions were not allowed. - Added a new next_token() method for checking the look ahead. - Fix that continue labels could jump to non iterating targets. - Fix that generator functions cannot be declared in if statements.
2021-08-28 15:04:37 +00:00
static bool is_simple_parameter_list(Vector<FunctionNode::Parameter> const& parameters)
{
return all_of(parameters, [](FunctionNode::Parameter const& parameter) {
return !parameter.is_rest && parameter.default_value.is_null() && parameter.binding.has<FlyString>();
});
}
LibJS: Parse async arrow functions
2021-11-14 23:47:16 +00:00
RefPtr<FunctionExpression> Parser::try_parse_arrow_function_expression(bool expect_parens, bool is_async)
LibJS: Add support for arrow functions
2020-03-30 13:26:09 +00:00
{
LibJS: Parse async arrow functions
2021-11-14 23:47:16 +00:00
if (is_async)
VERIFY(match(TokenType::Async));
if (!expect_parens && !is_async) {
LibJS: Add a fast failure path to try_parse_arrow_function_expression() The save/load of parser state performed by lookahead parsing is quite expensive so let's try to avoid it in the most common case. This is a 15-20% speedup on various chunks of JS I've tested. :^)
2021-09-14 00:51:16 +00:00
// NOTE: This is a fast path where we try to fail early in case this can't possibly
// be a match. The idea is to avoid the expensive parser state save/load mechanism.
// The logic is duplicated below in the "real" !expect_parens branch.
if (!match_identifier() && !match(TokenType::Yield) && !match(TokenType::Await))
return nullptr;
LibJS: Do not save state for peeking at the next token from the lexer This saves having to save and load the parser state. This could give an incorrect token in some cases where the parser communicates to the lexer. However this is not applicable in any of the current usages and this would require one to parse the current token as normal which is exactly what you don't want to do in that scenario.
2021-10-07 22:38:24 +00:00
auto token = next_token();
LibJS: Add a fast failure path to try_parse_arrow_function_expression() The save/load of parser state performed by lookahead parsing is quite expensive so let's try to avoid it in the most common case. This is a 15-20% speedup on various chunks of JS I've tested. :^)
2021-09-14 00:51:16 +00:00
if (token.trivia_contains_line_terminator())
return nullptr;
if (token.type() != TokenType::Arrow)
return nullptr;
}
LibJS: Add support for arrow functions
2020-03-30 13:26:09 +00:00
save_state();
LibJS: `save_state()' before creating a RulePosition Fixes #4617.
2020-12-29 05:12:02 +00:00
auto rule_start = push_start();
LibJS: Hoist variable declarations to the nearest relevant scope "var" declarations are hoisted to the nearest function scope, while "let" and "const" are hoisted to the nearest block scope. This is done by the parser, which keeps two scope stacks, one stack for the current var scope and one for the current let/const scope. When the interpreter enters a scope, we walk all of the declarations and insert them into the variable environment. We don't support the temporal dead zone for let/const yet.
2020-04-13 14:42:54 +00:00
ArmedScopeGuard state_rollback_guard = [&] {
load_state();
};
LibJS: Add support for arrow functions
2020-03-30 13:26:09 +00:00
LibJS: Parse async arrow functions
2021-11-14 23:47:16 +00:00
auto function_kind = FunctionKind::Regular;
if (is_async) {
consume(TokenType::Async);
function_kind = FunctionKind::Async;
if (m_state.current_token.trivia_contains_line_terminator())
return nullptr;
// Since we have async it can be followed by paren open in the expect_parens case
// so we also consume that token.
if (expect_parens) {
VERIFY(match(TokenType::ParenOpen));
consume(TokenType::ParenOpen);
}
}
LibJS: Add function default arguments Adds the ability for function arguments to have default values. This works for standard functions as well as arrow functions. Default values are not printed in a <function>.toString() call, as nodes cannot print their source string representation.
2020-05-02 18:46:39 +00:00
Vector<FunctionNode::Parameter> parameters;
LibJS: Function.length respects default and rest parameters "[Function.length is] the number of formal parameters. This number excludes the rest parameter and only includes parameters before the first one with a default value." - MDN
2020-05-06 03:02:14 +00:00
i32 function_length = -1;
LibJS: Share parameter parsing between regular and arrow functions This simplifies try_parse_arrow_function_expression() and fixes a few cases that should not produce an arrow function AST but did: (a,,) => {} (a b) => {} (a ...b) => {} (...b a) => {} The new parsing logic checks whether parens are expected and uses parse_function_parameters() if so, rolling back if a new syntax error occurs during that. Otherwise it's just an identifier in which case we parse the single parameter ourselves.
2020-10-18 23:26:41 +00:00
if (expect_parens) {
// We have parens around the function parameters and can re-use the same parsing
// logic used for regular functions: multiple parameters, default values, rest
// parameter, maybe a trailing comma. If we have a new syntax error afterwards we
LibJS: Implement rules for duplicate function parameters - A regular function can have duplicate parameters except in strict mode or if its parameter list is not "simple" (has a default or rest parameter) - An arrow function can never have duplicate parameters Compared to other engines I opted for more useful syntax error messages than a generic "duplicate parameter name not allowed in this context": "use strict"; function test(foo, foo) {} ^ Uncaught exception: [SyntaxError]: Duplicate parameter 'foo' not allowed in strict mode (line: 1, column: 34) function test(foo, foo = 1) {} ^ Uncaught exception: [SyntaxError]: Duplicate parameter 'foo' not allowed in function with default parameter (line: 1, column: 20) function test(foo, ...foo) {} ^ Uncaught exception: [SyntaxError]: Duplicate parameter 'foo' not allowed in function with rest parameter (line: 1, column: 23) (foo, foo) => {} ^ Uncaught exception: [SyntaxError]: Duplicate parameter 'foo' not allowed in arrow function (line: 1, column: 7)
2020-10-25 11:14:04 +00:00
// check if it's about a wrong token (something like duplicate parameter name must
// not abort), know parsing failed and rollback the parser state.
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
auto previous_syntax_errors = m_state.errors.size();
LibJS: Parse async arrow functions
2021-11-14 23:47:16 +00:00
TemporaryChange in_async_context(m_state.in_async_function_context, is_async || m_state.in_async_function_context);
parameters = parse_formal_parameters(function_length, FunctionNodeParseOptions::IsArrowFunction | (is_async ? FunctionNodeParseOptions::IsAsyncFunction : 0));
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
if (m_state.errors.size() > previous_syntax_errors && m_state.errors[previous_syntax_errors].message.starts_with("Unexpected token"))
LibJS: Share parameter parsing between regular and arrow functions This simplifies try_parse_arrow_function_expression() and fixes a few cases that should not produce an arrow function AST but did: (a,,) => {} (a b) => {} (a ...b) => {} (...b a) => {} The new parsing logic checks whether parens are expected and uses parse_function_parameters() if so, rolling back if a new syntax error occurs during that. Otherwise it's just an identifier in which case we parse the single parameter ourselves.
2020-10-18 23:26:41 +00:00
return nullptr;
if (!match(TokenType::ParenClose))
return nullptr;
consume();
} else {
// No parens - this must be an identifier followed by arrow. That's it.
LibJS: Rework Identifier parsing to match the spec more closely
2021-07-11 11:04:55 +00:00
if (!match_identifier() && !match(TokenType::Yield) && !match(TokenType::Await))
LibJS: Share parameter parsing between regular and arrow functions This simplifies try_parse_arrow_function_expression() and fixes a few cases that should not produce an arrow function AST but did: (a,,) => {} (a b) => {} (a ...b) => {} (...b a) => {} The new parsing logic checks whether parens are expected and uses parse_function_parameters() if so, rolling back if a new syntax error occurs during that. Otherwise it's just an identifier in which case we parse the single parameter ourselves.
2020-10-18 23:26:41 +00:00
return nullptr;
LibJS: Rework Identifier parsing to match the spec more closely
2021-07-11 11:04:55 +00:00
auto token = consume_identifier_reference();
if (m_state.strict_mode && token.value().is_one_of("arguments"sv, "eval"sv))
syntax_error("BindingIdentifier may not be 'arguments' or 'eval' in strict mode");
LibJS: Parse async arrow functions
2021-11-14 23:47:16 +00:00
if (is_async && token.value() == "await"sv)
syntax_error("'await' is a reserved identifier in async functions");
LibJS: Rework Identifier parsing to match the spec more closely
2021-07-11 11:04:55 +00:00
parameters.append({ FlyString { token.value() }, {} });
LibJS: Add support for arrow functions
2020-03-30 13:26:09 +00:00
}
LibJS: Don't parse arrow function with newline between ) and => If there's a newline between the closing paren and arrow it's not a valid arrow function, ASI should kick in instead (it'll then fail with "Unexpected token Arrow")
2020-10-18 23:29:17 +00:00
// If there's a newline between the closing paren and arrow it's not a valid arrow function,
// ASI should kick in instead (it'll then fail with "Unexpected token Arrow")
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
if (m_state.current_token.trivia_contains_line_terminator())
LibJS: Don't parse arrow function with newline between ) and => If there's a newline between the closing paren and arrow it's not a valid arrow function, ASI should kick in instead (it'll then fail with "Unexpected token Arrow")
2020-10-18 23:29:17 +00:00
return nullptr;
LibJS: Share parameter parsing between regular and arrow functions This simplifies try_parse_arrow_function_expression() and fixes a few cases that should not produce an arrow function AST but did: (a,,) => {} (a b) => {} (a ...b) => {} (...b a) => {} The new parsing logic checks whether parens are expected and uses parse_function_parameters() if so, rolling back if a new syntax error occurs during that. Otherwise it's just an identifier in which case we parse the single parameter ourselves.
2020-10-18 23:26:41 +00:00
if (!match(TokenType::Arrow))
LibJS: Add support for arrow functions
2020-03-30 13:26:09 +00:00
return nullptr;
LibJS: Share parameter parsing between regular and arrow functions This simplifies try_parse_arrow_function_expression() and fixes a few cases that should not produce an arrow function AST but did: (a,,) => {} (a b) => {} (a ...b) => {} (...b a) => {} The new parsing logic checks whether parens are expected and uses parse_function_parameters() if so, rolling back if a new syntax error occurs during that. Otherwise it's just an identifier in which case we parse the single parameter ourselves.
2020-10-18 23:26:41 +00:00
consume();
LibJS: Add support for arrow functions
2020-03-30 13:26:09 +00:00
LibJS: Parse comma operator into SequenceExpression
2020-05-11 16:27:31 +00:00
if (function_length == -1)
LibJS: Function.length respects default and rest parameters "[Function.length is] the number of formal parameters. This number excludes the rest parameter and only includes parameters before the first one with a default value." - MDN
2020-05-06 03:02:14 +00:00
function_length = parameters.size();
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
auto old_labels_in_scope = move(m_state.labels_in_scope);
LibJS: break or continue with nonexistent label is a syntax error
2020-10-08 17:49:08 +00:00
ScopeGuard guard([&]() {
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
m_state.labels_in_scope = move(old_labels_in_scope);
LibJS: break or continue with nonexistent label is a syntax error
2020-10-08 17:49:08 +00:00
});
LibJS: Propagate "contains direct call to eval()" flag from parser We now propagate this flag to FunctionDeclaration, and then also into ECMAScriptFunctionObject. This will be used to disable optimizations that aren't safe in the presence of direct eval().
2021-10-08 10:43:38 +00:00
bool contains_direct_call_to_eval = false;
LibJS: Make scoping follow the spec Before this we used an ad-hoc combination of references and 'variables' stored in a hashmap. This worked in most cases but is not spec like. Additionally hoisting, dynamically naming functions and scope analysis was not done properly. This patch fixes all of that by: - Implement BindingInitialization for destructuring assignment. - Implementing a new ScopePusher which tracks the lexical and var scoped declarations. This hoists functions to the top level if no lexical declaration name overlaps. Furthermore we do checking of redeclarations in the ScopePusher now requiring less checks all over the place. - Add methods for parsing the directives and statement lists instead of having that code duplicated in multiple places. This allows declarations to pushed to the appropriate scope more easily. - Remove the non spec way of storing 'variables' in DeclarativeEnvironment and make Reference follow the spec instead of checking both the bindings and 'variables'. - Remove all scoping related things from the Interpreter. And instead use environments as specified by the spec. This also includes fixing that NativeFunctions did not produce a valid FunctionEnvironment which could cause issues with callbacks and eval. All FunctionObjects now have a valid NewFunctionEnvironment implementation. - Remove execute_statements from Interpreter and instead use ASTNode::execute everywhere this simplifies AST.cpp as you no longer need to worry about which method to call. - Make ScopeNodes setup their own environment. This uses four different methods specified by the spec {Block, Function, Eval, Global}DeclarationInstantiation with the annexB extensions. - Implement and use NamedEvaluation where specified. Additionally there are fixes to things exposed by these changes to eval, {for, for-in, for-of} loops and assignment. Finally it also fixes some tests in test-js which where passing before but not now that we have correct behavior :^).
2021-09-22 10:44:56 +00:00
auto function_body_result = [&]() -> RefPtr<FunctionBody> {
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
TemporaryChange change(m_state.in_arrow_function_context, true);
LibJS: Parse async arrow functions
2021-11-14 23:47:16 +00:00
TemporaryChange async_context_change(m_state.in_async_function_context, is_async);
LibJS: Add support for arrow functions
2020-03-30 13:26:09 +00:00
if (match(TokenType::CurlyOpen)) {
// Parse a function body with statements
LibJS: Parse async arrow functions
2021-11-14 23:47:16 +00:00
return parse_function_body(parameters, function_kind, contains_direct_call_to_eval);
LibJS: Add support for arrow functions
2020-03-30 13:26:09 +00:00
}
if (match_expression()) {
// Parse a function body which returns a single expression
// FIXME: We synthesize a block with a return statement
// for arrow function bodies which are a single expression.
// Esprima generates a single "ArrowFunctionExpression"
// with a "body" property.
LibJS: Make scoping follow the spec Before this we used an ad-hoc combination of references and 'variables' stored in a hashmap. This worked in most cases but is not spec like. Additionally hoisting, dynamically naming functions and scope analysis was not done properly. This patch fixes all of that by: - Implement BindingInitialization for destructuring assignment. - Implementing a new ScopePusher which tracks the lexical and var scoped declarations. This hoists functions to the top level if no lexical declaration name overlaps. Furthermore we do checking of redeclarations in the ScopePusher now requiring less checks all over the place. - Add methods for parsing the directives and statement lists instead of having that code duplicated in multiple places. This allows declarations to pushed to the appropriate scope more easily. - Remove the non spec way of storing 'variables' in DeclarativeEnvironment and make Reference follow the spec instead of checking both the bindings and 'variables'. - Remove all scoping related things from the Interpreter. And instead use environments as specified by the spec. This also includes fixing that NativeFunctions did not produce a valid FunctionEnvironment which could cause issues with callbacks and eval. All FunctionObjects now have a valid NewFunctionEnvironment implementation. - Remove execute_statements from Interpreter and instead use ASTNode::execute everywhere this simplifies AST.cpp as you no longer need to worry about which method to call. - Make ScopeNodes setup their own environment. This uses four different methods specified by the spec {Block, Function, Eval, Global}DeclarationInstantiation with the annexB extensions. - Implement and use NamedEvaluation where specified. Additionally there are fixes to things exposed by these changes to eval, {for, for-in, for-of} loops and assignment. Finally it also fixes some tests in test-js which where passing before but not now that we have correct behavior :^).
2021-09-22 10:44:56 +00:00
auto return_block = create_ast_node<FunctionBody>({ m_state.current_token.filename(), rule_start.position(), position() });
LibJS: Make accessing the current function's arguments cheaper Instead of going through an environment record, make arguments of the currently executing function generate references via the argument index, which can later be resolved directly through the ExecutionContext.
2021-10-07 16:43:22 +00:00
ScopePusher function_scope = ScopePusher::function_scope(*this, return_block, parameters);
auto return_expression = parse_expression(2);
LibJS: Keep track of file names, lines and columns inside the AST
2021-02-28 09:42:34 +00:00
return_block->append<ReturnStatement>({ m_filename, rule_start.position(), position() }, move(return_expression));
LibJS: Make scoping follow the spec Before this we used an ad-hoc combination of references and 'variables' stored in a hashmap. This worked in most cases but is not spec like. Additionally hoisting, dynamically naming functions and scope analysis was not done properly. This patch fixes all of that by: - Implement BindingInitialization for destructuring assignment. - Implementing a new ScopePusher which tracks the lexical and var scoped declarations. This hoists functions to the top level if no lexical declaration name overlaps. Furthermore we do checking of redeclarations in the ScopePusher now requiring less checks all over the place. - Add methods for parsing the directives and statement lists instead of having that code duplicated in multiple places. This allows declarations to pushed to the appropriate scope more easily. - Remove the non spec way of storing 'variables' in DeclarativeEnvironment and make Reference follow the spec instead of checking both the bindings and 'variables'. - Remove all scoping related things from the Interpreter. And instead use environments as specified by the spec. This also includes fixing that NativeFunctions did not produce a valid FunctionEnvironment which could cause issues with callbacks and eval. All FunctionObjects now have a valid NewFunctionEnvironment implementation. - Remove execute_statements from Interpreter and instead use ASTNode::execute everywhere this simplifies AST.cpp as you no longer need to worry about which method to call. - Make ScopeNodes setup their own environment. This uses four different methods specified by the spec {Block, Function, Eval, Global}DeclarationInstantiation with the annexB extensions. - Implement and use NamedEvaluation where specified. Additionally there are fixes to things exposed by these changes to eval, {for, for-in, for-of} loops and assignment. Finally it also fixes some tests in test-js which where passing before but not now that we have correct behavior :^).
2021-09-22 10:44:56 +00:00
if (m_state.strict_mode)
return_block->set_strict_mode();
LibJS: Propagate "contains direct call to eval()" flag from parser We now propagate this flag to FunctionDeclaration, and then also into ECMAScriptFunctionObject. This will be used to disable optimizations that aren't safe in the presence of direct eval().
2021-10-08 10:43:38 +00:00
contains_direct_call_to_eval = function_scope.contains_direct_call_to_eval();
LibJS: Add support for arrow functions
2020-03-30 13:26:09 +00:00
return return_block;
}
// Invalid arrow function body
return nullptr;
}();
LibJS: Handle strict mode for functions more correctly If a function is strict (has 'use strict' directive) it cannot have bindings, cannot have duplicated parameter names and cannot have some reserved keywords and identifiers as parameter names. The logic partly applies depending on whether we are already in strict mode or the function contains 'use strict';
2021-07-11 23:29:07 +00:00
if (function_body_result.is_null())
return nullptr;
state_rollback_guard.disarm();
discard_saved_state();
auto body = function_body_result.release_nonnull();
LibJS: Make scoping follow the spec Before this we used an ad-hoc combination of references and 'variables' stored in a hashmap. This worked in most cases but is not spec like. Additionally hoisting, dynamically naming functions and scope analysis was not done properly. This patch fixes all of that by: - Implement BindingInitialization for destructuring assignment. - Implementing a new ScopePusher which tracks the lexical and var scoped declarations. This hoists functions to the top level if no lexical declaration name overlaps. Furthermore we do checking of redeclarations in the ScopePusher now requiring less checks all over the place. - Add methods for parsing the directives and statement lists instead of having that code duplicated in multiple places. This allows declarations to pushed to the appropriate scope more easily. - Remove the non spec way of storing 'variables' in DeclarativeEnvironment and make Reference follow the spec instead of checking both the bindings and 'variables'. - Remove all scoping related things from the Interpreter. And instead use environments as specified by the spec. This also includes fixing that NativeFunctions did not produce a valid FunctionEnvironment which could cause issues with callbacks and eval. All FunctionObjects now have a valid NewFunctionEnvironment implementation. - Remove execute_statements from Interpreter and instead use ASTNode::execute everywhere this simplifies AST.cpp as you no longer need to worry about which method to call. - Make ScopeNodes setup their own environment. This uses four different methods specified by the spec {Block, Function, Eval, Global}DeclarationInstantiation with the annexB extensions. - Implement and use NamedEvaluation where specified. Additionally there are fixes to things exposed by these changes to eval, {for, for-in, for-of} loops and assignment. Finally it also fixes some tests in test-js which where passing before but not now that we have correct behavior :^).
2021-09-22 10:44:56 +00:00
if (body->in_strict_mode()) {
LibJS: Handle strict mode for functions more correctly If a function is strict (has 'use strict' directive) it cannot have bindings, cannot have duplicated parameter names and cannot have some reserved keywords and identifiers as parameter names. The logic partly applies depending on whether we are already in strict mode or the function contains 'use strict';
2021-07-11 23:29:07 +00:00
for (auto& parameter : parameters) {
parameter.binding.visit(
[&](FlyString const& name) {
check_identifier_name_for_assignment_validity(name, true);
},
[&](auto const&) {});
}
LibJS: Add support for arrow functions
2020-03-30 13:26:09 +00:00
}
LibJS: Handle strict mode for functions more correctly If a function is strict (has 'use strict' directive) it cannot have bindings, cannot have duplicated parameter names and cannot have some reserved keywords and identifiers as parameter names. The logic partly applies depending on whether we are already in strict mode or the function contains 'use strict';
2021-07-11 23:29:07 +00:00
return create_ast_node<FunctionExpression>(
{ m_state.current_token.filename(), rule_start.position(), position() }, "", move(body),
LibJS: Parse async arrow functions
2021-11-14 23:47:16 +00:00
move(parameters), function_length, function_kind, body->in_strict_mode(),
LibJS: Propagate "contains direct call to eval()" flag from parser We now propagate this flag to FunctionDeclaration, and then also into ECMAScriptFunctionObject. This will be used to disable optimizations that aren't safe in the presence of direct eval().
2021-10-08 10:43:38 +00:00
/* might_need_arguments_object */ false, contains_direct_call_to_eval, /* is_arrow_function */ true);
LibJS: Add support for arrow functions
2020-03-30 13:26:09 +00:00
}
LibJS: Allow labelled functions in certain contexts Also disallow duplicated labels.
2021-07-24 23:01:22 +00:00
RefPtr<Statement> Parser::try_parse_labelled_statement(AllowLabelledFunction allow_function)
LibJS: Parse labelled statements All statements now have an optional label string that can be null.
2020-05-28 18:09:19 +00:00
{
LibJS: Add fast failure path to try_parse_labelled_statement() If the next token isn't a TokenType::Colon (:), this can't possibly be a labelled statement, so we can fail before having to save_state(). This improves parsing time on a large chunk of JS by ~12.5%.
2021-09-18 17:39:20 +00:00
{
// NOTE: This is a fast path where we try to fail early to avoid the expensive save_state+load_state.
LibJS: Do not save state for peeking at the next token from the lexer This saves having to save and load the parser state. This could give an incorrect token in some cases where the parser communicates to the lexer. However this is not applicable in any of the current usages and this would require one to parse the current token as normal which is exactly what you don't want to do in that scenario.
2021-10-07 22:38:24 +00:00
if (next_token().type() != TokenType::Colon)
LibJS: Add fast failure path to try_parse_labelled_statement() If the next token isn't a TokenType::Colon (:), this can't possibly be a labelled statement, so we can fail before having to save_state(). This improves parsing time on a large chunk of JS by ~12.5%.
2021-09-18 17:39:20 +00:00
return {};
}
LibJS: Parse labelled statements All statements now have an optional label string that can be null.
2020-05-28 18:09:19 +00:00
save_state();
LibJS: `save_state()' before creating a RulePosition Fixes #4617.
2020-12-29 05:12:02 +00:00
auto rule_start = push_start();
LibJS: Parse labelled statements All statements now have an optional label string that can be null.
2020-05-28 18:09:19 +00:00
ArmedScopeGuard state_rollback_guard = [&] {
load_state();
};
LibJS: Fix some small remaining issues with parsing unicode escapes Added a test to ensure the behavior stays the same. We now throw on a direct usage of an escaped keywords with a specific error to make it more clear to the user.
2021-08-21 09:27:20 +00:00
if (m_state.current_token.value() == "yield"sv && (m_state.strict_mode || m_state.in_generator_function_context)) {
LibJS: Handle escaped keywords in more cases and handle 'await' labels
2021-09-18 21:02:50 +00:00
return {};
}
LibJS: Add support for async functions This commit adds support for the most bare bones version of async functions, support for async generator functions, async arrow functions and await expressions are TODO.
2021-11-09 18:39:22 +00:00
if (m_state.current_token.value() == "await"sv && (m_program_type == Program::Type::Module || m_state.in_async_function_context)) {
LibJS: Allow labelled functions in certain contexts Also disallow duplicated labels.
2021-07-24 23:01:22 +00:00
return {};
}
LibJS: Allow multiple labels on the same statement Since there are only a number of statements where labels can actually be used we now also only store labels when necessary. Also now tracks the first continue usage of a label since this might not be valid but that can only be determined after we have parsed the statement. Also ensures the correct error does not get wiped by load_state.
2021-09-18 21:01:54 +00:00
auto identifier = [&] {
if (m_state.current_token.value() == "await"sv) {
return consume().value();
}
return consume_identifier_reference().value();
}();
LibJS: Parse labelled statements All statements now have an optional label string that can be null.
2020-05-28 18:09:19 +00:00
if (!match(TokenType::Colon))
return {};
consume(TokenType::Colon);
if (!match_statement())
return {};
LibJS: Allow labelled functions in certain contexts Also disallow duplicated labels.
2021-07-24 23:01:22 +00:00
LibJS: Allow multiple labels on the same statement Since there are only a number of statements where labels can actually be used we now also only store labels when necessary. Also now tracks the first continue usage of a label since this might not be valid but that can only be determined after we have parsed the statement. Also ensures the correct error does not get wiped by load_state.
2021-09-18 21:01:54 +00:00
state_rollback_guard.disarm();
discard_saved_state();
if (m_state.strict_mode && identifier == "let"sv) {
syntax_error("Strict mode reserved word 'let' is not allowed in label", rule_start.position());
return {};
}
LibJS: Allow labelled functions in certain contexts Also disallow duplicated labels.
2021-07-24 23:01:22 +00:00
if (match(TokenType::Function) && (allow_function == AllowLabelledFunction::No || m_state.strict_mode)) {
syntax_error("Not allowed to declare a function here");
return {};
}
if (m_state.labels_in_scope.contains(identifier))
syntax_error(String::formatted("Label '{}' has already been declared", identifier));
RefPtr<Statement> labelled_statement;
LibJS: Allow multiple labels on the same statement Since there are only a number of statements where labels can actually be used we now also only store labels when necessary. Also now tracks the first continue usage of a label since this might not be valid but that can only be determined after we have parsed the statement. Also ensures the correct error does not get wiped by load_state.
2021-09-18 21:01:54 +00:00
auto is_iteration_statement = false;
LibJS: Allow labelled functions in certain contexts Also disallow duplicated labels.
2021-07-24 23:01:22 +00:00
if (match(TokenType::Function)) {
LibJS: Allow multiple labels on the same statement Since there are only a number of statements where labels can actually be used we now also only store labels when necessary. Also now tracks the first continue usage of a label since this might not be valid but that can only be determined after we have parsed the statement. Also ensures the correct error does not get wiped by load_state.
2021-09-18 21:01:54 +00:00
m_state.labels_in_scope.set(identifier, {});
LibJS: Allow labelled functions in certain contexts Also disallow duplicated labels.
2021-07-24 23:01:22 +00:00
auto function_declaration = parse_function_node<FunctionDeclaration>();
LibJS: Make scoping follow the spec Before this we used an ad-hoc combination of references and 'variables' stored in a hashmap. This worked in most cases but is not spec like. Additionally hoisting, dynamically naming functions and scope analysis was not done properly. This patch fixes all of that by: - Implement BindingInitialization for destructuring assignment. - Implementing a new ScopePusher which tracks the lexical and var scoped declarations. This hoists functions to the top level if no lexical declaration name overlaps. Furthermore we do checking of redeclarations in the ScopePusher now requiring less checks all over the place. - Add methods for parsing the directives and statement lists instead of having that code duplicated in multiple places. This allows declarations to pushed to the appropriate scope more easily. - Remove the non spec way of storing 'variables' in DeclarativeEnvironment and make Reference follow the spec instead of checking both the bindings and 'variables'. - Remove all scoping related things from the Interpreter. And instead use environments as specified by the spec. This also includes fixing that NativeFunctions did not produce a valid FunctionEnvironment which could cause issues with callbacks and eval. All FunctionObjects now have a valid NewFunctionEnvironment implementation. - Remove execute_statements from Interpreter and instead use ASTNode::execute everywhere this simplifies AST.cpp as you no longer need to worry about which method to call. - Make ScopeNodes setup their own environment. This uses four different methods specified by the spec {Block, Function, Eval, Global}DeclarationInstantiation with the annexB extensions. - Implement and use NamedEvaluation where specified. Additionally there are fixes to things exposed by these changes to eval, {for, for-in, for-of} loops and assignment. Finally it also fixes some tests in test-js which where passing before but not now that we have correct behavior :^).
2021-09-22 10:44:56 +00:00
VERIFY(m_state.current_scope_pusher);
m_state.current_scope_pusher->add_declaration(function_declaration);
LibJS: Allow labelled functions in certain contexts Also disallow duplicated labels.
2021-07-24 23:01:22 +00:00
if (function_declaration->kind() == FunctionKind::Generator)
syntax_error("Generator functions cannot be defined in labelled statements");
LibJS: Add support for async functions This commit adds support for the most bare bones version of async functions, support for async generator functions, async arrow functions and await expressions are TODO.
2021-11-09 18:39:22 +00:00
if (function_declaration->kind() == FunctionKind::Async)
syntax_error("Async functions cannot be defined in labelled statements");
LibJS: Allow labelled functions in certain contexts Also disallow duplicated labels.
2021-07-24 23:01:22 +00:00
labelled_statement = move(function_declaration);
} else {
LibJS: Allow multiple labels on the same statement Since there are only a number of statements where labels can actually be used we now also only store labels when necessary. Also now tracks the first continue usage of a label since this might not be valid but that can only be determined after we have parsed the statement. Also ensures the correct error does not get wiped by load_state.
2021-09-18 21:01:54 +00:00
m_state.labels_in_scope.set(identifier, {});
labelled_statement = parse_statement(allow_function);
if (is<IterationStatement>(*labelled_statement)) {
is_iteration_statement = true;
static_cast<IterationStatement&>(*labelled_statement).add_label(identifier);
} else if (is<LabelableStatement>(*labelled_statement)) {
static_cast<LabelableStatement&>(*labelled_statement).add_label(identifier);
}
}
if (!is_iteration_statement) {
if (auto entry = m_state.labels_in_scope.find(identifier); entry != m_state.labels_in_scope.end() && entry->value.has_value())
syntax_error("labelled continue statement cannot use non iterating statement", m_state.labels_in_scope.get(identifier).value());
LibJS: Allow labelled functions in certain contexts Also disallow duplicated labels.
2021-07-24 23:01:22 +00:00
}
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
m_state.labels_in_scope.remove(identifier);
LibJS: Parse labelled statements All statements now have an optional label string that can be null.
2020-05-28 18:09:19 +00:00
LibJS: Allow labelled functions in certain contexts Also disallow duplicated labels.
2021-07-24 23:01:22 +00:00
return labelled_statement.release_nonnull();
LibJS: Parse labelled statements All statements now have an optional label string that can be null.
2020-05-28 18:09:19 +00:00
}
LibJS: Implement 'new.target' This adds a new MetaProperty AST node which will be used for 'new.target' and 'import.meta' meta properties. The parser now distinguishes between "in function context" and "in arrow function context" (which is required for this). When encountering TokenType::New we will attempt to parse it as meta property and resort to regular new expression parsing if that fails, much like the parsing of labelled statements.
2020-11-02 21:27:42 +00:00
RefPtr<MetaProperty> Parser::try_parse_new_target_expression()
{
save_state();
LibJS: `save_state()' before creating a RulePosition Fixes #4617.
2020-12-29 05:12:02 +00:00
auto rule_start = push_start();
LibJS: Implement 'new.target' This adds a new MetaProperty AST node which will be used for 'new.target' and 'import.meta' meta properties. The parser now distinguishes between "in function context" and "in arrow function context" (which is required for this). When encountering TokenType::New we will attempt to parse it as meta property and resort to regular new expression parsing if that fails, much like the parsing of labelled statements.
2020-11-02 21:27:42 +00:00
ArmedScopeGuard state_rollback_guard = [&] {
load_state();
};
consume(TokenType::New);
if (!match(TokenType::Period))
return {};
consume();
if (!match(TokenType::Identifier))
return {};
LibJS: Fix some small remaining issues with parsing unicode escapes Added a test to ensure the behavior stays the same. We now throw on a direct usage of an escaped keywords with a specific error to make it more clear to the user.
2021-08-21 09:27:20 +00:00
// The string 'target' cannot have escapes so we check original value.
if (consume().original_value() != "target"sv)
LibJS: Implement 'new.target' This adds a new MetaProperty AST node which will be used for 'new.target' and 'import.meta' meta properties. The parser now distinguishes between "in function context" and "in arrow function context" (which is required for this). When encountering TokenType::New we will attempt to parse it as meta property and resort to regular new expression parsing if that fails, much like the parsing of labelled statements.
2020-11-02 21:27:42 +00:00
return {};
state_rollback_guard.disarm();
LibJS: Don't save rule start positions along with the parser state This fixes #4617. Also fixes the small problem where some save states would be leaked.
2020-12-29 13:17:39 +00:00
discard_saved_state();
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
return create_ast_node<MetaProperty>({ m_state.current_token.filename(), rule_start.position(), position() }, MetaProperty::Type::NewTarget);
LibJS: Implement 'new.target' This adds a new MetaProperty AST node which will be used for 'new.target' and 'import.meta' meta properties. The parser now distinguishes between "in function context" and "in arrow function context" (which is required for this). When encountering TokenType::New we will attempt to parse it as meta property and resort to regular new expression parsing if that fails, much like the parsing of labelled statements.
2020-11-02 21:27:42 +00:00
}
LibJS: Initial class implementation; allow super expressions in object literal methods; add EnvrionmentRecord fields and methods to LexicalEnvironment Adding EnvrionmentRecord's fields and methods lets us throw an exception when |this| is not initialized, which occurs when the super constructor in a derived class has not yet been called, or when |this| has already been initialized (the super constructor was already called).
2020-06-08 18:31:21 +00:00
NonnullRefPtr<ClassDeclaration> Parser::parse_class_declaration()
{
LibJS: Track source positions all the way down to exceptions This makes exceptions have a trace of source positions too, which could probably be helpful in making fancier error tracebacks.
2020-12-28 17:15:22 +00:00
auto rule_start = push_start();
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
return create_ast_node<ClassDeclaration>({ m_state.current_token.filename(), rule_start.position(), position() }, parse_class_expression(true));
LibJS: Initial class implementation; allow super expressions in object literal methods; add EnvrionmentRecord fields and methods to LexicalEnvironment Adding EnvrionmentRecord's fields and methods lets us throw an exception when |this| is not initialized, which occurs when the super constructor in a derived class has not yet been called, or when |this| has already been initialized (the super constructor was already called).
2020-06-08 18:31:21 +00:00
}
NonnullRefPtr<ClassExpression> Parser::parse_class_expression(bool expect_class_name)
{
LibJS: Track source positions all the way down to exceptions This makes exceptions have a trace of source positions too, which could probably be helpful in making fancier error tracebacks.
2020-12-28 17:15:22 +00:00
auto rule_start = push_start();
LibJS: Initial class implementation; allow super expressions in object literal methods; add EnvrionmentRecord fields and methods to LexicalEnvironment Adding EnvrionmentRecord's fields and methods lets us throw an exception when |this| is not initialized, which occurs when the super constructor in a derived class has not yet been called, or when |this| has already been initialized (the super constructor was already called).
2020-06-08 18:31:21 +00:00
// Classes are always in strict mode.
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
TemporaryChange strict_mode_rollback(m_state.strict_mode, true);
LibJS: Initial class implementation; allow super expressions in object literal methods; add EnvrionmentRecord fields and methods to LexicalEnvironment Adding EnvrionmentRecord's fields and methods lets us throw an exception when |this| is not initialized, which occurs when the super constructor in a derived class has not yet been called, or when |this| has already been initialized (the super constructor was already called).
2020-06-08 18:31:21 +00:00
consume(TokenType::Class);
LibJS: Make class definition evaluation more spec like in ordering
2021-10-06 23:09:04 +00:00
NonnullRefPtrVector<ClassElement> elements;
LibJS: Initial class implementation; allow super expressions in object literal methods; add EnvrionmentRecord fields and methods to LexicalEnvironment Adding EnvrionmentRecord's fields and methods lets us throw an exception when |this| is not initialized, which occurs when the super constructor in a derived class has not yet been called, or when |this| has already been initialized (the super constructor was already called).
2020-06-08 18:31:21 +00:00
RefPtr<Expression> super_class;
RefPtr<FunctionExpression> constructor;
LibJS: Add parsing and evaluation of private fields and methods
2021-10-12 20:45:52 +00:00
HashTable<FlyString> found_private_names;
LibJS: Initial class implementation; allow super expressions in object literal methods; add EnvrionmentRecord fields and methods to LexicalEnvironment Adding EnvrionmentRecord's fields and methods lets us throw an exception when |this| is not initialized, which occurs when the super constructor in a derived class has not yet been called, or when |this| has already been initialized (the super constructor was already called).
2020-06-08 18:31:21 +00:00
LibJS: Rework Identifier parsing to match the spec more closely
2021-07-11 11:04:55 +00:00
String class_name = expect_class_name || match_identifier() || match(TokenType::Yield) || match(TokenType::Await)
? consume_identifier_reference().value().to_string()
: "";
LibJS: Initial class implementation; allow super expressions in object literal methods; add EnvrionmentRecord fields and methods to LexicalEnvironment Adding EnvrionmentRecord's fields and methods lets us throw an exception when |this| is not initialized, which occurs when the super constructor in a derived class has not yet been called, or when |this| has already been initialized (the super constructor was already called).
2020-06-08 18:31:21 +00:00
LibJS: Be more strict about reserved and special identifiers
2021-07-11 23:27:35 +00:00
check_identifier_name_for_assignment_validity(class_name, true);
LibJS: Initial class implementation; allow super expressions in object literal methods; add EnvrionmentRecord fields and methods to LexicalEnvironment Adding EnvrionmentRecord's fields and methods lets us throw an exception when |this| is not initialized, which occurs when the super constructor in a derived class has not yet been called, or when |this| has already been initialized (the super constructor was already called).
2020-06-08 18:31:21 +00:00
if (match(TokenType::Extends)) {
consume();
LibJS: Correctly parse yield-from expressions This commit implements parsing for `yield *expr`, and the multiple ways something can or can't be parsed like that. Also makes yield-from a TODO in the bytecode generator. Behold, the glory of javascript syntax: ```js // 'yield' = expression in generators. function* foo() { yield *bar; // <- Syntax error here, expression can't start with * } // 'yield' = identifier anywhere else. function foo() { yield *bar; // Perfectly fine, this is just `yield * bar` } ```
2021-06-14 11:16:41 +00:00
auto [expression, should_continue_parsing] = parse_primary_expression();
LibJS: Extend class 'extends' RHS expression parsing Instead of only parsing a primary expression, we should also allow member expressions, call expressions, and tagged template literals (and optional chains, which we don't have yet). In the spec, all of this is covered by `LeftHandSideExpression` (https://tc39.es/ecma262/#prod-LeftHandSideExpression).
2021-07-17 22:19:03 +00:00
// Basically a (much) simplified parse_secondary_expression().
for (;;) {
if (match(TokenType::TemplateLiteralStart)) {
auto template_literal = parse_template_literal(true);
expression = create_ast_node<TaggedTemplateLiteral>({ m_state.current_token.filename(), rule_start.position(), position() }, move(expression), move(template_literal));
continue;
}
if (match(TokenType::BracketOpen) || match(TokenType::Period) || match(TokenType::ParenOpen)) {
auto precedence = g_operator_precedence.get(m_state.current_token.type());
expression = parse_secondary_expression(move(expression), precedence);
continue;
}
break;
}
LibJS: Correctly parse yield-from expressions This commit implements parsing for `yield *expr`, and the multiple ways something can or can't be parsed like that. Also makes yield-from a TODO in the bytecode generator. Behold, the glory of javascript syntax: ```js // 'yield' = expression in generators. function* foo() { yield *bar; // <- Syntax error here, expression can't start with * } // 'yield' = identifier anywhere else. function foo() { yield *bar; // Perfectly fine, this is just `yield * bar` } ```
2021-06-14 11:16:41 +00:00
super_class = move(expression);
(void)should_continue_parsing;
LibJS: Initial class implementation; allow super expressions in object literal methods; add EnvrionmentRecord fields and methods to LexicalEnvironment Adding EnvrionmentRecord's fields and methods lets us throw an exception when |this| is not initialized, which occurs when the super constructor in a derived class has not yet been called, or when |this| has already been initialized (the super constructor was already called).
2020-06-08 18:31:21 +00:00
}
consume(TokenType::CurlyOpen);
LibJS: Add parsing and evaluation of private fields and methods
2021-10-12 20:45:52 +00:00
HashTable<StringView> referenced_private_names;
HashTable<StringView>* outer_referenced_private_names = m_state.referenced_private_names;
m_state.referenced_private_names = &referenced_private_names;
ScopeGuard restore_private_name_table = [&] {
m_state.referenced_private_names = outer_referenced_private_names;
};
LibJS: Initial class implementation; allow super expressions in object literal methods; add EnvrionmentRecord fields and methods to LexicalEnvironment Adding EnvrionmentRecord's fields and methods lets us throw an exception when |this| is not initialized, which occurs when the super constructor in a derived class has not yet been called, or when |this| has already been initialized (the super constructor was already called).
2020-06-08 18:31:21 +00:00
while (!done() && !match(TokenType::CurlyClose)) {
RefPtr<Expression> property_key;
bool is_static = false;
bool is_constructor = false;
LibJS: Parse generator functions in class expressions too
2021-07-02 10:07:00 +00:00
bool is_generator = false;
LibJS: Add support for async functions This commit adds support for the most bare bones version of async functions, support for async generator functions, async arrow functions and await expressions are TODO.
2021-11-09 18:39:22 +00:00
bool is_async = false;
LibJS: Initial class implementation; allow super expressions in object literal methods; add EnvrionmentRecord fields and methods to LexicalEnvironment Adding EnvrionmentRecord's fields and methods lets us throw an exception when |this| is not initialized, which occurs when the super constructor in a derived class has not yet been called, or when |this| has already been initialized (the super constructor was already called).
2020-06-08 18:31:21 +00:00
auto method_kind = ClassMethod::Kind::Method;
if (match(TokenType::Semicolon)) {
consume();
continue;
}
LibJS: Add support for async functions This commit adds support for the most bare bones version of async functions, support for async generator functions, async arrow functions and await expressions are TODO.
2021-11-09 18:39:22 +00:00
if (match(TokenType::Async)) {
consume();
is_async = true;
}
LibJS: Parse generator functions in class expressions too
2021-07-02 10:07:00 +00:00
if (match(TokenType::Asterisk)) {
consume();
is_generator = true;
}
LibJS: Add support for public fields in classes
2021-08-28 15:11:05 +00:00
StringView name;
LibJS: Add parsing and evaluation of private fields and methods
2021-10-12 20:45:52 +00:00
if (match_property_key() || match(TokenType::PrivateIdentifier)) {
LibJS: Add support for async functions This commit adds support for the most bare bones version of async functions, support for async generator functions, async arrow functions and await expressions are TODO.
2021-11-09 18:39:22 +00:00
if (!is_generator && !is_async && m_state.current_token.original_value() == "static"sv) {
LibJS: Parse generator functions in class expressions too
2021-07-02 10:07:00 +00:00
if (match(TokenType::Identifier)) {
consume();
is_static = true;
LibJS: Add support for async functions This commit adds support for the most bare bones version of async functions, support for async generator functions, async arrow functions and await expressions are TODO.
2021-11-09 18:39:22 +00:00
if (match(TokenType::Async)) {
consume();
is_async = true;
}
LibJS: Parse generator functions in class expressions too
2021-07-02 10:07:00 +00:00
if (match(TokenType::Asterisk)) {
consume();
is_generator = true;
}
}
LibJS: Initial class implementation; allow super expressions in object literal methods; add EnvrionmentRecord fields and methods to LexicalEnvironment Adding EnvrionmentRecord's fields and methods lets us throw an exception when |this| is not initialized, which occurs when the super constructor in a derived class has not yet been called, or when |this| has already been initialized (the super constructor was already called).
2020-06-08 18:31:21 +00:00
}
if (match(TokenType::Identifier)) {
LibJS: Add support for public fields in classes
2021-08-28 15:11:05 +00:00
auto identifier_name = m_state.current_token.original_value();
LibJS: Initial class implementation; allow super expressions in object literal methods; add EnvrionmentRecord fields and methods to LexicalEnvironment Adding EnvrionmentRecord's fields and methods lets us throw an exception when |this| is not initialized, which occurs when the super constructor in a derived class has not yet been called, or when |this| has already been initialized (the super constructor was already called).
2020-06-08 18:31:21 +00:00
LibJS: Add support for public fields in classes
2021-08-28 15:11:05 +00:00
if (identifier_name == "get"sv) {
LibJS: Initial class implementation; allow super expressions in object literal methods; add EnvrionmentRecord fields and methods to LexicalEnvironment Adding EnvrionmentRecord's fields and methods lets us throw an exception when |this| is not initialized, which occurs when the super constructor in a derived class has not yet been called, or when |this| has already been initialized (the super constructor was already called).
2020-06-08 18:31:21 +00:00
method_kind = ClassMethod::Kind::Getter;
consume();
LibJS: Add support for public fields in classes
2021-08-28 15:11:05 +00:00
} else if (identifier_name == "set"sv) {
LibJS: Initial class implementation; allow super expressions in object literal methods; add EnvrionmentRecord fields and methods to LexicalEnvironment Adding EnvrionmentRecord's fields and methods lets us throw an exception when |this| is not initialized, which occurs when the super constructor in a derived class has not yet been called, or when |this| has already been initialized (the super constructor was already called).
2020-06-08 18:31:21 +00:00
method_kind = ClassMethod::Kind::Setter;
consume();
}
}
LibJS: Add parsing and evaluation of private fields and methods
2021-10-12 20:45:52 +00:00
if (match_property_key() || match(TokenType::PrivateIdentifier)) {
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
switch (m_state.current_token.type()) {
LibJS: Initial class implementation; allow super expressions in object literal methods; add EnvrionmentRecord fields and methods to LexicalEnvironment Adding EnvrionmentRecord's fields and methods lets us throw an exception when |this| is not initialized, which occurs when the super constructor in a derived class has not yet been called, or when |this| has already been initialized (the super constructor was already called).
2020-06-08 18:31:21 +00:00
case TokenType::Identifier:
name = consume().value();
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
property_key = create_ast_node<StringLiteral>({ m_state.current_token.filename(), rule_start.position(), position() }, name);
LibJS: Initial class implementation; allow super expressions in object literal methods; add EnvrionmentRecord fields and methods to LexicalEnvironment Adding EnvrionmentRecord's fields and methods lets us throw an exception when |this| is not initialized, which occurs when the super constructor in a derived class has not yet been called, or when |this| has already been initialized (the super constructor was already called).
2020-06-08 18:31:21 +00:00
break;
LibJS: Add parsing and evaluation of private fields and methods
2021-10-12 20:45:52 +00:00
case TokenType::PrivateIdentifier:
name = consume().value();
if (name == "#constructor")
syntax_error("Private property with name '#constructor' is not allowed");
if (method_kind != ClassMethod::Kind::Method) {
// It is a Syntax Error if PrivateBoundIdentifiers of ClassElementList contains any duplicate entries,
// unless the name is used once for a getter and once for a setter and in no other entries,
// and the getter and setter are either both static or both non-static.
for (auto& element : elements) {
auto private_name = element.private_bound_identifier();
if (!private_name.has_value() || private_name.value() != name)
continue;
if (element.class_element_kind() != ClassElement::ElementKind::Method
|| element.is_static() != is_static) {
syntax_error(String::formatted("Duplicate private field or method named '{}'", name));
break;
}
VERIFY(is<ClassMethod>(element));
auto& class_method_element = static_cast<ClassMethod const&>(element);
if (class_method_element.kind() == ClassMethod::Kind::Method || class_method_element.kind() == method_kind) {
syntax_error(String::formatted("Duplicate private field or method named '{}'", name));
break;
}
}
found_private_names.set(name);
} else if (found_private_names.set(name) != AK::HashSetResult::InsertedNewEntry) {
syntax_error(String::formatted("Duplicate private field or method named '{}'", name));
}
property_key = create_ast_node<PrivateIdentifier>({ m_state.current_token.filename(), rule_start.position(), position() }, name);
break;
LibJS: Initial class implementation; allow super expressions in object literal methods; add EnvrionmentRecord fields and methods to LexicalEnvironment Adding EnvrionmentRecord's fields and methods lets us throw an exception when |this| is not initialized, which occurs when the super constructor in a derived class has not yet been called, or when |this| has already been initialized (the super constructor was already called).
2020-06-08 18:31:21 +00:00
case TokenType::StringLiteral: {
auto string_literal = parse_string_literal(consume());
name = string_literal->value();
property_key = move(string_literal);
break;
}
default:
property_key = parse_property_key();
break;
}
LibJS: Add an optimization to avoid needless arguments object creation This gives FunctionNode a "might need arguments object" boolean flag and sets it based on the simplest possible heuristic for this: if we encounter an identifier called "arguments" or "eval" up to the next (nested) function declaration or expression, we won't need an arguments object. Otherwise, we *might* need one - the final decision is made in the FunctionDeclarationInstantiation AO. Now, this is obviously not perfect. Even if you avoid eval, something like `foo.arguments` will still trigger a false positive - but it's a start and already massively cuts down on needlessly allocated objects, especially in real-world code that is often minified, and so a full "arguments" identifier will be an actual arguments object more often than not. To illustrate the actual impact of this change, here's the number of allocated arguments objects during a full test-js run: Before: - Unmapped arguments objects: 78765 - Mapped arguments objects: 2455 After: - Unmapped arguments objects: 18 - Mapped arguments objects: 37 This results in a ~5% speedup of test-js on my Linux host machine, and about 3.5% on i686 Serenity in QEMU (warm runs, average of 5). The following microbenchmark (calling an empty function 1M times) runs 25% faster on Linux and 45% on Serenity: function foo() {} for (var i = 0; i < 1_000_000; ++i) foo(); test262 reports no changes in either direction, apart from a speedup :^)
2021-10-05 07:44:58 +00:00
// https://tc39.es/ecma262/#sec-class-definitions-static-semantics-early-errors
LibJS: Disallow static methods named prototype in classes
2021-07-11 23:30:04 +00:00
// ClassElement : static MethodDefinition
// It is a Syntax Error if PropName of MethodDefinition is "prototype".
if (is_static && name == "prototype"sv)
syntax_error("Classes may not have a static property named 'prototype'");
LibJS: Add support for async functions This commit adds support for the most bare bones version of async functions, support for async generator functions, async arrow functions and await expressions are TODO.
2021-11-09 18:39:22 +00:00
} else if ((match(TokenType::ParenOpen) || match(TokenType::Equals)) && (is_static || is_async || method_kind != ClassMethod::Kind::Method)) {
LibJS: Allow class methods named "get", "set" or "static"
2021-07-25 09:50:12 +00:00
switch (method_kind) {
case ClassMethod::Kind::Method:
LibJS: Add support for async functions This commit adds support for the most bare bones version of async functions, support for async generator functions, async arrow functions and await expressions are TODO.
2021-11-09 18:39:22 +00:00
if (is_async) {
name = "async";
is_async = false;
} else {
VERIFY(is_static);
name = "static";
is_static = false;
}
LibJS: Allow class methods named "get", "set" or "static"
2021-07-25 09:50:12 +00:00
break;
case ClassMethod::Kind::Getter:
name = "get";
method_kind = ClassMethod::Kind::Method;
break;
case ClassMethod::Kind::Setter:
name = "set";
method_kind = ClassMethod::Kind::Method;
break;
}
property_key = create_ast_node<StringLiteral>({ m_state.current_token.filename(), rule_start.position(), position() }, name);
LibJS: Add static initializers to classes
2021-10-20 19:29:47 +00:00
} else if (match(TokenType::CurlyOpen) && is_static) {
auto static_start = push_start();
consume(TokenType::CurlyOpen);
auto static_init_block = create_ast_node<FunctionBody>({ m_state.current_token.filename(), rule_start.position(), position() });
TemporaryChange break_context_rollback(m_state.in_break_context, false);
TemporaryChange continue_context_rollback(m_state.in_continue_context, false);
TemporaryChange function_context_rollback(m_state.in_function_context, false);
TemporaryChange generator_function_context_rollback(m_state.in_generator_function_context, false);
LibJS: Add support for async functions This commit adds support for the most bare bones version of async functions, support for async generator functions, async arrow functions and await expressions are TODO.
2021-11-09 18:39:22 +00:00
TemporaryChange async_function_context_rollback(m_state.in_async_function_context, false);
LibJS: Add support for await expressions
2021-11-09 20:52:21 +00:00
TemporaryChange class_field_initializer_rollback(m_state.in_class_field_initializer, true);
TemporaryChange class_static_init_block_rollback(m_state.in_class_static_init_block, true);
LibJS: Add static initializers to classes
2021-10-20 19:29:47 +00:00
ScopePusher static_init_scope = ScopePusher::static_init_block_scope(*this, *static_init_block);
parse_statement_list(static_init_block);
consume(TokenType::CurlyClose);
elements.append(create_ast_node<StaticInitializer>({ m_state.current_token.filename(), static_start.position(), position() }, move(static_init_block), static_init_scope.contains_direct_call_to_eval()));
continue;
LibJS: Initial class implementation; allow super expressions in object literal methods; add EnvrionmentRecord fields and methods to LexicalEnvironment Adding EnvrionmentRecord's fields and methods lets us throw an exception when |this| is not initialized, which occurs when the super constructor in a derived class has not yet been called, or when |this| has already been initialized (the super constructor was already called).
2020-06-08 18:31:21 +00:00
} else {
expected("property key");
}
// Constructor may be a StringLiteral or an Identifier.
LibJS: Add support for public fields in classes
2021-08-28 15:11:05 +00:00
if (!is_static && name == "constructor"sv) {
LibJS: Initial class implementation; allow super expressions in object literal methods; add EnvrionmentRecord fields and methods to LexicalEnvironment Adding EnvrionmentRecord's fields and methods lets us throw an exception when |this| is not initialized, which occurs when the super constructor in a derived class has not yet been called, or when |this| has already been initialized (the super constructor was already called).
2020-06-08 18:31:21 +00:00
if (method_kind != ClassMethod::Kind::Method)
syntax_error("Class constructor may not be an accessor");
if (!constructor.is_null())
syntax_error("Classes may not have more than one constructor");
LibJS: Parse generator functions in class expressions too
2021-07-02 10:07:00 +00:00
if (is_generator)
syntax_error("Class constructor may not be a generator");
LibJS: Add support for async functions This commit adds support for the most bare bones version of async functions, support for async generator functions, async arrow functions and await expressions are TODO.
2021-11-09 18:39:22 +00:00
if (is_async)
syntax_error("Class constructor may not be async");
LibJS: Initial class implementation; allow super expressions in object literal methods; add EnvrionmentRecord fields and methods to LexicalEnvironment Adding EnvrionmentRecord's fields and methods lets us throw an exception when |this| is not initialized, which occurs when the super constructor in a derived class has not yet been called, or when |this| has already been initialized (the super constructor was already called).
2020-06-08 18:31:21 +00:00
is_constructor = true;
}
}
if (match(TokenType::ParenOpen)) {
LibJS: Refactor parse_function_node() bool parameters into bit flags I'm about to add even more options and a bunch of unnamed true/false arguments is really not helpful. Let's make this a single parse options parameter using bit flags.
2020-10-20 16:56:49 +00:00
u8 parse_options = FunctionNodeParseOptions::AllowSuperPropertyLookup;
LibJS: Allow class methods named "get", "set" or "static"
2021-07-25 09:50:12 +00:00
if (!super_class.is_null() && !is_static && is_constructor)
LibJS: Refactor parse_function_node() bool parameters into bit flags I'm about to add even more options and a bunch of unnamed true/false arguments is really not helpful. Let's make this a single parse options parameter using bit flags.
2020-10-20 16:56:49 +00:00
parse_options |= FunctionNodeParseOptions::AllowSuperConstructorCall;
LibJS: Move checks for invalid getter/setter params to parse_function_node This allows us to provide better error messages as we can point the syntax error location to the exact first invalid parameter instead of always the end of the function within a object literal or class definition. Before this change: const Foo = { set bar() {} } ^ Uncaught exception: [SyntaxError]: Object setter property must have one argument (line: 1, column: 28) class Foo { set bar() {} } ^ Uncaught exception: [SyntaxError]: Class setter method must have one argument (line: 1, column: 26) After this change: const Foo = { set bar() {} } ^ Uncaught exception: [SyntaxError]: Setter function must have one argument (line: 1, column: 23) class Foo { set bar() {} } ^ Uncaught exception: [SyntaxError]: Setter function must have one argument (line: 1, column: 21) The only possible downside of this change is that class getters/setters and functions in objects are not distinguished in the message anymore - I don't think that's important though, and classes are (mostly) just syntactic sugar anyway.
2020-10-20 17:32:51 +00:00
if (method_kind == ClassMethod::Kind::Getter)
parse_options |= FunctionNodeParseOptions::IsGetterFunction;
if (method_kind == ClassMethod::Kind::Setter)
parse_options |= FunctionNodeParseOptions::IsSetterFunction;
LibJS: Parse generator functions in class expressions too
2021-07-02 10:07:00 +00:00
if (is_generator)
parse_options |= FunctionNodeParseOptions::IsGeneratorFunction;
LibJS: Add support for async functions This commit adds support for the most bare bones version of async functions, support for async generator functions, async arrow functions and await expressions are TODO.
2021-11-09 18:39:22 +00:00
if (is_async)
parse_options |= FunctionNodeParseOptions::IsAsyncFunction;
LibJS: Refactor parse_function_node() bool parameters into bit flags I'm about to add even more options and a bunch of unnamed true/false arguments is really not helpful. Let's make this a single parse options parameter using bit flags.
2020-10-20 16:56:49 +00:00
auto function = parse_function_node<FunctionExpression>(parse_options);
LibJS: Initial class implementation; allow super expressions in object literal methods; add EnvrionmentRecord fields and methods to LexicalEnvironment Adding EnvrionmentRecord's fields and methods lets us throw an exception when |this| is not initialized, which occurs when the super constructor in a derived class has not yet been called, or when |this| has already been initialized (the super constructor was already called).
2020-06-08 18:31:21 +00:00
if (is_constructor) {
constructor = move(function);
} else if (!property_key.is_null()) {
LibJS: Make class definition evaluation more spec like in ordering
2021-10-06 23:09:04 +00:00
elements.append(create_ast_node<ClassMethod>({ m_state.current_token.filename(), rule_start.position(), position() }, property_key.release_nonnull(), move(function), method_kind, is_static));
LibJS: Initial class implementation; allow super expressions in object literal methods; add EnvrionmentRecord fields and methods to LexicalEnvironment Adding EnvrionmentRecord's fields and methods lets us throw an exception when |this| is not initialized, which occurs when the super constructor in a derived class has not yet been called, or when |this| has already been initialized (the super constructor was already called).
2020-06-08 18:31:21 +00:00
} else {
syntax_error("No key for class method");
}
LibJS: Add support for async functions This commit adds support for the most bare bones version of async functions, support for async generator functions, async arrow functions and await expressions are TODO.
2021-11-09 18:39:22 +00:00
} else if (is_generator || is_async) {
LibJS: Initial class implementation; allow super expressions in object literal methods; add EnvrionmentRecord fields and methods to LexicalEnvironment Adding EnvrionmentRecord's fields and methods lets us throw an exception when |this| is not initialized, which occurs when the super constructor in a derived class has not yet been called, or when |this| has already been initialized (the super constructor was already called).
2020-06-08 18:31:21 +00:00
expected("ParenOpen");
consume();
LibJS: Add support for public fields in classes
2021-08-28 15:11:05 +00:00
} else if (property_key.is_null()) {
expected("property key");
consume();
} else {
if (name == "constructor"sv)
syntax_error("Class cannot have field named 'constructor'");
RefPtr<Expression> initializer;
LibJS: Use ClassFieldInitializerStatement for class fields This is necessary as we might have to perform named evaluation with the field name. Ideally we would also skip some setup parts of the function like function_declaration_instantiation however this would require bigger changes to ECMAScriptFunctionObject.
2021-10-13 17:59:38 +00:00
bool contains_direct_call_to_eval = false;
LibJS: Add support for public fields in classes
2021-08-28 15:11:05 +00:00
if (match(TokenType::Equals)) {
consume();
TemporaryChange super_property_access_rollback(m_state.allow_super_property_lookup, true);
TemporaryChange field_initializer_rollback(m_state.in_class_field_initializer, true);
LibJS: Use ClassFieldInitializerStatement for class fields This is necessary as we might have to perform named evaluation with the field name. Ideally we would also skip some setup parts of the function like function_declaration_instantiation however this would require bigger changes to ECMAScriptFunctionObject.
2021-10-13 17:59:38 +00:00
auto class_field_scope = ScopePusher::class_field_scope(*this);
LibJS: Add support for public fields in classes
2021-08-28 15:11:05 +00:00
initializer = parse_expression(2);
LibJS: Use ClassFieldInitializerStatement for class fields This is necessary as we might have to perform named evaluation with the field name. Ideally we would also skip some setup parts of the function like function_declaration_instantiation however this would require bigger changes to ECMAScriptFunctionObject.
2021-10-13 17:59:38 +00:00
contains_direct_call_to_eval = class_field_scope.contains_direct_call_to_eval();
LibJS: Add support for public fields in classes
2021-08-28 15:11:05 +00:00
}
LibJS: Use ClassFieldInitializerStatement for class fields This is necessary as we might have to perform named evaluation with the field name. Ideally we would also skip some setup parts of the function like function_declaration_instantiation however this would require bigger changes to ECMAScriptFunctionObject.
2021-10-13 17:59:38 +00:00
elements.append(create_ast_node<ClassField>({ m_state.current_token.filename(), rule_start.position(), position() }, property_key.release_nonnull(), move(initializer), contains_direct_call_to_eval, is_static));
LibJS: Add support for public fields in classes
2021-08-28 15:11:05 +00:00
consume_or_insert_semicolon();
LibJS: Initial class implementation; allow super expressions in object literal methods; add EnvrionmentRecord fields and methods to LexicalEnvironment Adding EnvrionmentRecord's fields and methods lets us throw an exception when |this| is not initialized, which occurs when the super constructor in a derived class has not yet been called, or when |this| has already been initialized (the super constructor was already called).
2020-06-08 18:31:21 +00:00
}
}
consume(TokenType::CurlyClose);
if (constructor.is_null()) {
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
auto constructor_body = create_ast_node<BlockStatement>({ m_state.current_token.filename(), rule_start.position(), position() });
LibJS: Initial class implementation; allow super expressions in object literal methods; add EnvrionmentRecord fields and methods to LexicalEnvironment Adding EnvrionmentRecord's fields and methods lets us throw an exception when |this| is not initialized, which occurs when the super constructor in a derived class has not yet been called, or when |this| has already been initialized (the super constructor was already called).
2020-06-08 18:31:21 +00:00
if (!super_class.is_null()) {
// Set constructor to the result of parsing the source text
// constructor(... args){ super (...args);}
LibJS: Make SuperCall a proper AST node and clean up evaluation
2021-07-02 17:30:38 +00:00
auto super_call = create_ast_node<SuperCall>(
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
{ m_state.current_token.filename(), rule_start.position(), position() },
Vector { CallExpression::Argument { create_ast_node<Identifier>({ m_state.current_token.filename(), rule_start.position(), position() }, "args"), true } });
constructor_body->append(create_ast_node<ExpressionStatement>({ m_state.current_token.filename(), rule_start.position(), position() }, move(super_call)));
LibJS: Initial class implementation; allow super expressions in object literal methods; add EnvrionmentRecord fields and methods to LexicalEnvironment Adding EnvrionmentRecord's fields and methods lets us throw an exception when |this| is not initialized, which occurs when the super constructor in a derived class has not yet been called, or when |this| has already been initialized (the super constructor was already called).
2020-06-08 18:31:21 +00:00
LibJS: Use an enum class instead of 'bool is_generator' This avoid confusion in the order of the multiple boolean parameters that exist.
2021-06-10 23:08:05 +00:00
constructor = create_ast_node<FunctionExpression>(
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
{ m_state.current_token.filename(), rule_start.position(), position() }, class_name, move(constructor_body),
LibJS: Add an optimization to avoid needless arguments object creation This gives FunctionNode a "might need arguments object" boolean flag and sets it based on the simplest possible heuristic for this: if we encounter an identifier called "arguments" or "eval" up to the next (nested) function declaration or expression, we won't need an arguments object. Otherwise, we *might* need one - the final decision is made in the FunctionDeclarationInstantiation AO. Now, this is obviously not perfect. Even if you avoid eval, something like `foo.arguments` will still trigger a false positive - but it's a start and already massively cuts down on needlessly allocated objects, especially in real-world code that is often minified, and so a full "arguments" identifier will be an actual arguments object more often than not. To illustrate the actual impact of this change, here's the number of allocated arguments objects during a full test-js run: Before: - Unmapped arguments objects: 78765 - Mapped arguments objects: 2455 After: - Unmapped arguments objects: 18 - Mapped arguments objects: 37 This results in a ~5% speedup of test-js on my Linux host machine, and about 3.5% on i686 Serenity in QEMU (warm runs, average of 5). The following microbenchmark (calling an empty function 1M times) runs 25% faster on Linux and 45% on Serenity: function foo() {} for (var i = 0; i < 1_000_000; ++i) foo(); test262 reports no changes in either direction, apart from a speedup :^)
2021-10-05 07:44:58 +00:00
Vector { FunctionNode::Parameter { FlyString { "args" }, nullptr, true } }, 0, FunctionKind::Regular,
LibJS: Propagate "contains direct call to eval()" flag from parser We now propagate this flag to FunctionDeclaration, and then also into ECMAScriptFunctionObject. This will be used to disable optimizations that aren't safe in the presence of direct eval().
2021-10-08 10:43:38 +00:00
/* is_strict_mode */ true, /* might_need_arguments_object */ false, /* contains_direct_call_to_eval */ false);
LibJS: Initial class implementation; allow super expressions in object literal methods; add EnvrionmentRecord fields and methods to LexicalEnvironment Adding EnvrionmentRecord's fields and methods lets us throw an exception when |this| is not initialized, which occurs when the super constructor in a derived class has not yet been called, or when |this| has already been initialized (the super constructor was already called).
2020-06-08 18:31:21 +00:00
} else {
LibJS: Use an enum class instead of 'bool is_generator' This avoid confusion in the order of the multiple boolean parameters that exist.
2021-06-10 23:08:05 +00:00
constructor = create_ast_node<FunctionExpression>(
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
{ m_state.current_token.filename(), rule_start.position(), position() }, class_name, move(constructor_body),
LibJS: Add an optimization to avoid needless arguments object creation This gives FunctionNode a "might need arguments object" boolean flag and sets it based on the simplest possible heuristic for this: if we encounter an identifier called "arguments" or "eval" up to the next (nested) function declaration or expression, we won't need an arguments object. Otherwise, we *might* need one - the final decision is made in the FunctionDeclarationInstantiation AO. Now, this is obviously not perfect. Even if you avoid eval, something like `foo.arguments` will still trigger a false positive - but it's a start and already massively cuts down on needlessly allocated objects, especially in real-world code that is often minified, and so a full "arguments" identifier will be an actual arguments object more often than not. To illustrate the actual impact of this change, here's the number of allocated arguments objects during a full test-js run: Before: - Unmapped arguments objects: 78765 - Mapped arguments objects: 2455 After: - Unmapped arguments objects: 18 - Mapped arguments objects: 37 This results in a ~5% speedup of test-js on my Linux host machine, and about 3.5% on i686 Serenity in QEMU (warm runs, average of 5). The following microbenchmark (calling an empty function 1M times) runs 25% faster on Linux and 45% on Serenity: function foo() {} for (var i = 0; i < 1_000_000; ++i) foo(); test262 reports no changes in either direction, apart from a speedup :^)
2021-10-05 07:44:58 +00:00
Vector<FunctionNode::Parameter> {}, 0, FunctionKind::Regular,
LibJS: Propagate "contains direct call to eval()" flag from parser We now propagate this flag to FunctionDeclaration, and then also into ECMAScriptFunctionObject. This will be used to disable optimizations that aren't safe in the presence of direct eval().
2021-10-08 10:43:38 +00:00
/* is_strict_mode */ true, /* might_need_arguments_object */ false, /* contains_direct_call_to_eval */ false);
LibJS: Initial class implementation; allow super expressions in object literal methods; add EnvrionmentRecord fields and methods to LexicalEnvironment Adding EnvrionmentRecord's fields and methods lets us throw an exception when |this| is not initialized, which occurs when the super constructor in a derived class has not yet been called, or when |this| has already been initialized (the super constructor was already called).
2020-06-08 18:31:21 +00:00
}
}
LibJS: Add parsing and evaluation of private fields and methods
2021-10-12 20:45:52 +00:00
// We could be in a subclass defined within the main class so must move all non declared private names to outer.
for (auto& private_name : referenced_private_names) {
if (found_private_names.contains(private_name))
continue;
if (outer_referenced_private_names)
outer_referenced_private_names->set(private_name);
else // FIXME: Make these error appear in the appropriate places.
syntax_error(String::formatted("Reference to undeclared private field or method '{}'", private_name));
}
LibJS: Make class definition evaluation more spec like in ordering
2021-10-06 23:09:04 +00:00
return create_ast_node<ClassExpression>({ m_state.current_token.filename(), rule_start.position(), position() }, move(class_name), move(constructor), move(super_class), move(elements));
LibJS: Initial class implementation; allow super expressions in object literal methods; add EnvrionmentRecord fields and methods to LexicalEnvironment Adding EnvrionmentRecord's fields and methods lets us throw an exception when |this| is not initialized, which occurs when the super constructor in a derived class has not yet been called, or when |this| has already been initialized (the super constructor was already called).
2020-06-08 18:31:21 +00:00
}
LibJS: Correctly parse yield-from expressions This commit implements parsing for `yield *expr`, and the multiple ways something can or can't be parsed like that. Also makes yield-from a TODO in the bytecode generator. Behold, the glory of javascript syntax: ```js // 'yield' = expression in generators. function* foo() { yield *bar; // <- Syntax error here, expression can't start with * } // 'yield' = identifier anywhere else. function foo() { yield *bar; // Perfectly fine, this is just `yield * bar` } ```
2021-06-14 11:16:41 +00:00
Parser::PrimaryExpressionParseResult Parser::parse_primary_expression()
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
{
LibJS: Track source positions all the way down to exceptions This makes exceptions have a trace of source positions too, which could probably be helpful in making fancier error tracebacks.
2020-12-28 17:15:22 +00:00
auto rule_start = push_start();
LibJS: Fix parsing of `if (typeof "foo" === "string")` Move parsing of unary expressions into the primary expression parsing step so that `typeof` takes precedence over `===` in the above example.
2020-03-25 08:51:54 +00:00
if (match_unary_prefixed_expression())
LibJS: Correctly parse yield-from expressions This commit implements parsing for `yield *expr`, and the multiple ways something can or can't be parsed like that. Also makes yield-from a TODO in the bytecode generator. Behold, the glory of javascript syntax: ```js // 'yield' = expression in generators. function* foo() { yield *bar; // <- Syntax error here, expression can't start with * } // 'yield' = identifier anywhere else. function foo() { yield *bar; // Perfectly fine, this is just `yield * bar` } ```
2021-06-14 11:16:41 +00:00
return { parse_unary_prefixed_expression() };
LibJS: Fix parsing of `if (typeof "foo" === "string")` Move parsing of unary expressions into the primary expression parsing step so that `typeof` takes precedence over `===` in the above example.
2020-03-25 08:51:54 +00:00
LibJS: Parse async arrow functions
2021-11-14 23:47:16 +00:00
auto try_arrow_function_parse_or_fail = [this](Position const& position, bool expect_paren, bool is_async = false) -> RefPtr<FunctionExpression> {
if (try_parse_arrow_function_expression_failed_at_position(position))
return nullptr;
auto arrow_function = try_parse_arrow_function_expression(expect_paren, is_async);
if (arrow_function)
return arrow_function;
set_try_parse_arrow_function_expression_failed_at_position(position, true);
return nullptr;
};
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
switch (m_state.current_token.type()) {
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
case TokenType::ParenOpen: {
LibJS: Memoize failed calls of try_parse_arrow_function_expression()
2021-04-11 20:41:51 +00:00
auto paren_position = position();
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
consume(TokenType::ParenOpen);
LibJS: Parse async arrow functions
2021-11-14 23:47:16 +00:00
if ((match(TokenType::ParenClose) || match_identifier() || match(TokenType::TripleDot) || match(TokenType::CurlyOpen) || match(TokenType::BracketOpen))) {
if (auto arrow_function_result = try_arrow_function_parse_or_fail(paren_position, true))
LibJS: Treat arrow expression as function and stop parsing after
2021-07-28 15:11:33 +00:00
return { arrow_function_result.release_nonnull(), false };
LibJS: Add support for arrow functions
2020-03-30 13:26:09 +00:00
}
LibJS: Add operator precedence parsing Obey precedence and associativity rules when parsing expressions with chained operators.
2020-03-12 22:02:41 +00:00
auto expression = parse_expression(0);
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
consume(TokenType::ParenClose);
Only apply auto-naming of function expressions based on syntax The auto naming of function expressions is a purely syntactic decision, so shouldn't be decided based on the dynamic type of an assignment. This moves the decision making into the parser. One icky hack is that we add a field to FunctionExpression to indicate whether we can autoname. The real solution is to actually generate a CompoundExpression node so that the parser can make the correct decision, however this would have a potentially significant run time cost. This does not correct the behaviour for class expressions. Patch from Anonymous.
2021-03-22 11:44:07 +00:00
if (is<FunctionExpression>(*expression)) {
LibJS: Treat yield as an identifier in more non-generator contexts And disallow some cases where we are in a generator context.
2021-07-28 23:45:55 +00:00
auto& function = static_cast<FunctionExpression&>(*expression);
if (function.kind() == FunctionKind::Generator && function.name() == "yield"sv)
syntax_error("function is not allowed to be called 'yield' in this context", function.source_range().start);
LibJS: Add support for async functions This commit adds support for the most bare bones version of async functions, support for async generator functions, async arrow functions and await expressions are TODO.
2021-11-09 18:39:22 +00:00
if (function.kind() == FunctionKind::Async && function.name() == "await"sv)
syntax_error("function is not allowed to be called 'await' in this context", function.source_range().start);
Only apply auto-naming of function expressions based on syntax The auto naming of function expressions is a purely syntactic decision, so shouldn't be decided based on the dynamic type of an assignment. This moves the decision making into the parser. One icky hack is that we add a field to FunctionExpression to indicate whether we can autoname. The real solution is to actually generate a CompoundExpression node so that the parser can make the correct decision, however this would have a potentially significant run time cost. This does not correct the behaviour for class expressions. Patch from Anonymous.
2021-03-22 11:44:07 +00:00
}
LibJS: Correctly parse yield-from expressions This commit implements parsing for `yield *expr`, and the multiple ways something can or can't be parsed like that. Also makes yield-from a TODO in the bytecode generator. Behold, the glory of javascript syntax: ```js // 'yield' = expression in generators. function* foo() { yield *bar; // <- Syntax error here, expression can't start with * } // 'yield' = identifier anywhere else. function foo() { yield *bar; // Perfectly fine, this is just `yield * bar` } ```
2021-06-14 11:16:41 +00:00
return { move(expression) };
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
}
LibJS: Parse "this" as ThisExpression
2020-04-12 22:42:14 +00:00
case TokenType::This:
consume();
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
return { create_ast_node<ThisExpression>({ m_state.current_token.filename(), rule_start.position(), position() }) };
LibJS: Initial class implementation; allow super expressions in object literal methods; add EnvrionmentRecord fields and methods to LexicalEnvironment Adding EnvrionmentRecord's fields and methods lets us throw an exception when |this| is not initialized, which occurs when the super constructor in a derived class has not yet been called, or when |this| has already been initialized (the super constructor was already called).
2020-06-08 18:31:21 +00:00
case TokenType::Class:
LibJS: Correctly parse yield-from expressions This commit implements parsing for `yield *expr`, and the multiple ways something can or can't be parsed like that. Also makes yield-from a TODO in the bytecode generator. Behold, the glory of javascript syntax: ```js // 'yield' = expression in generators. function* foo() { yield *bar; // <- Syntax error here, expression can't start with * } // 'yield' = identifier anywhere else. function foo() { yield *bar; // Perfectly fine, this is just `yield * bar` } ```
2021-06-14 11:16:41 +00:00
return { parse_class_expression(false) };
LibJS: Initial class implementation; allow super expressions in object literal methods; add EnvrionmentRecord fields and methods to LexicalEnvironment Adding EnvrionmentRecord's fields and methods lets us throw an exception when |this| is not initialized, which occurs when the super constructor in a derived class has not yet been called, or when |this| has already been initialized (the super constructor was already called).
2020-06-08 18:31:21 +00:00
case TokenType::Super:
consume();
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
if (!m_state.allow_super_property_lookup)
LibJS: Initial class implementation; allow super expressions in object literal methods; add EnvrionmentRecord fields and methods to LexicalEnvironment Adding EnvrionmentRecord's fields and methods lets us throw an exception when |this| is not initialized, which occurs when the super constructor in a derived class has not yet been called, or when |this| has already been initialized (the super constructor was already called).
2020-06-08 18:31:21 +00:00
syntax_error("'super' keyword unexpected here");
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
return { create_ast_node<SuperExpression>({ m_state.current_token.filename(), rule_start.position(), position() }) };
LibJS: Fix some small remaining issues with parsing unicode escapes Added a test to ensure the behavior stays the same. We now throw on a direct usage of an escaped keywords with a specific error to make it more clear to the user.
2021-08-21 09:27:20 +00:00
case TokenType::EscapedKeyword:
LibJS: Add support for async functions This commit adds support for the most bare bones version of async functions, support for async generator functions, async arrow functions and await expressions are TODO.
2021-11-09 18:39:22 +00:00
if (m_state.strict_mode || (m_state.current_token.value() != "let"sv && (m_state.in_generator_function_context || m_state.current_token.value() != "yield"sv) && (m_state.in_async_function_context || m_state.current_token.value() != "await"sv)))
LibJS: Fix small issues in parser - Fix some places where escaped keywords are (not) allowed. - Be more strict about parameters for functions with 'use strict'. - Fix that expressions statements allowed functions and classes. - Fix that class expressions were not allowed. - Added a new next_token() method for checking the look ahead. - Fix that continue labels could jump to non iterating targets. - Fix that generator functions cannot be declared in if statements.
2021-08-28 15:04:37 +00:00
syntax_error("Keyword must not contain escaped characters");
LibJS: Fix some small remaining issues with parsing unicode escapes Added a test to ensure the behavior stays the same. We now throw on a direct usage of an escaped keywords with a specific error to make it more clear to the user.
2021-08-21 09:27:20 +00:00
[[fallthrough]];
LibJS: Add support for arrow functions
2020-03-30 13:26:09 +00:00
case TokenType::Identifier: {
LibJS: Implement generator functions (only in bytecode mode)
2021-06-10 21:08:30 +00:00
read_as_identifier:;
LibJS: Parse async arrow functions
2021-11-14 23:47:16 +00:00
if (auto arrow_function_result = try_arrow_function_parse_or_fail(position(), false))
return { arrow_function_result.release_nonnull(), false };
LibJS: Memoize failed calls of try_parse_arrow_function_expression()
2021-04-11 20:41:51 +00:00
LibJS: Fix small issues in parser - Fix some places where escaped keywords are (not) allowed. - Be more strict about parameters for functions with 'use strict'. - Fix that expressions statements allowed functions and classes. - Fix that class expressions were not allowed. - Added a new next_token() method for checking the look ahead. - Fix that continue labels could jump to non iterating targets. - Fix that generator functions cannot be declared in if statements.
2021-08-28 15:04:37 +00:00
auto string = m_state.current_token.value();
LibJS: Be more strict about reserved and special identifiers
2021-07-11 23:27:35 +00:00
// This could be 'eval' or 'arguments' and thus needs a custom check (`eval[1] = true`)
if (m_state.strict_mode && (string == "let" || is_strict_reserved_word(string)))
syntax_error(String::formatted("Identifier must not be a reserved word in strict mode ('{}')", string));
LibJS: Fix small issues in parser - Fix some places where escaped keywords are (not) allowed. - Be more strict about parameters for functions with 'use strict'. - Fix that expressions statements allowed functions and classes. - Fix that class expressions were not allowed. - Added a new next_token() method for checking the look ahead. - Fix that continue labels could jump to non iterating targets. - Fix that generator functions cannot be declared in if statements.
2021-08-28 15:04:37 +00:00
return { parse_identifier() };
LibJS: Add support for arrow functions
2020-03-30 13:26:09 +00:00
}
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
case TokenType::NumericLiteral:
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
return { create_ast_node<NumericLiteral>({ m_state.current_token.filename(), rule_start.position(), position() }, consume_and_validate_numeric_literal().double_value()) };
LibJS: Add BigInt
2020-06-06 00:14:10 +00:00
case TokenType::BigIntLiteral:
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
return { create_ast_node<BigIntLiteral>({ m_state.current_token.filename(), rule_start.position(), position() }, consume().value()) };
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
case TokenType::BoolLiteral:
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
return { create_ast_node<BooleanLiteral>({ m_state.current_token.filename(), rule_start.position(), position() }, consume().bool_value()) };
LibJS: Implement basic lexing + parsing of StringLiteral This still includes the double-quote characters (") but at least the AST comes out right.
2020-03-12 12:05:06 +00:00
case TokenType::StringLiteral:
LibJS: Correctly parse yield-from expressions This commit implements parsing for `yield *expr`, and the multiple ways something can or can't be parsed like that. Also makes yield-from a TODO in the bytecode generator. Behold, the glory of javascript syntax: ```js // 'yield' = expression in generators. function* foo() { yield *bar; // <- Syntax error here, expression can't start with * } // 'yield' = identifier anywhere else. function foo() { yield *bar; // Perfectly fine, this is just `yield * bar` } ```
2021-06-14 11:16:41 +00:00
return { parse_string_literal(consume()) };
LibJS: Implement null and undefined literals
2020-03-15 21:32:34 +00:00
case TokenType::NullLiteral:
consume();
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
return { create_ast_node<NullLiteral>({ m_state.current_token.filename(), rule_start.position(), position() }) };
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
case TokenType::CurlyOpen:
LibJS: Correctly parse yield-from expressions This commit implements parsing for `yield *expr`, and the multiple ways something can or can't be parsed like that. Also makes yield-from a TODO in the bytecode generator. Behold, the glory of javascript syntax: ```js // 'yield' = expression in generators. function* foo() { yield *bar; // <- Syntax error here, expression can't start with * } // 'yield' = identifier anywhere else. function foo() { yield *bar; // Perfectly fine, this is just `yield * bar` } ```
2021-06-14 11:16:41 +00:00
return { parse_object_expression() };
LibJS: Parse async arrow functions
2021-11-14 23:47:16 +00:00
case TokenType::Async: {
auto lookahead_token = next_token();
// No valid async function (arrow or not) can have a line terminator after the async since asi would kick in.
if (lookahead_token.trivia_contains_line_terminator())
LibJS: Add support for async functions This commit adds support for the most bare bones version of async functions, support for async generator functions, async arrow functions and await expressions are TODO.
2021-11-09 18:39:22 +00:00
goto read_as_identifier;
LibJS: Parse async arrow functions
2021-11-14 23:47:16 +00:00
if (lookahead_token.type() == TokenType::Function)
return { parse_function_node<FunctionExpression>() };
if (lookahead_token.type() == TokenType::ParenOpen) {
if (auto arrow_function_result = try_arrow_function_parse_or_fail(position(), true, true))
return { arrow_function_result.release_nonnull(), false };
} else if (lookahead_token.is_identifier_name()) {
if (auto arrow_function_result = try_arrow_function_parse_or_fail(position(), false, true))
return { arrow_function_result.release_nonnull(), false };
}
goto read_as_identifier;
}
LibJS: Parse FunctionExpressions FunctionExpression is mostly like FunctionDeclaration, except the name is optional. Share the parsing logic in parse_function_node<NodeType>. This allows us to do nice things like: document.addEventListener("DOMContentLoaded", function() { alert("Hello friends!"); });
2020-03-19 10:52:56 +00:00
case TokenType::Function:
LibJS: Correctly parse yield-from expressions This commit implements parsing for `yield *expr`, and the multiple ways something can or can't be parsed like that. Also makes yield-from a TODO in the bytecode generator. Behold, the glory of javascript syntax: ```js // 'yield' = expression in generators. function* foo() { yield *bar; // <- Syntax error here, expression can't start with * } // 'yield' = identifier anywhere else. function foo() { yield *bar; // Perfectly fine, this is just `yield * bar` } ```
2021-06-14 11:16:41 +00:00
return { parse_function_node<FunctionExpression>() };
LibJS: Parse ArrayExpression and start implementing Array objects Note that property lookup is not functional yet.
2020-03-20 19:29:57 +00:00
case TokenType::BracketOpen:
LibJS: Correctly parse yield-from expressions This commit implements parsing for `yield *expr`, and the multiple ways something can or can't be parsed like that. Also makes yield-from a TODO in the bytecode generator. Behold, the glory of javascript syntax: ```js // 'yield' = expression in generators. function* foo() { yield *bar; // <- Syntax error here, expression can't start with * } // 'yield' = identifier anywhere else. function foo() { yield *bar; // Perfectly fine, this is just `yield * bar` } ```
2021-06-14 11:16:41 +00:00
return { parse_array_expression() };
LibJS: Lex and parse regex literals, add RegExp objects This adds regex parsing/lexing, as well as a relatively empty RegExpObject. The purpose of this patch is to allow the engine to not get hung up on parsing regexes. This will aid in finding new syntax errors (say, from google or twitter) without having to replace all of their regexes first!
2020-06-03 23:05:49 +00:00
case TokenType::RegexLiteral:
LibJS: Correctly parse yield-from expressions This commit implements parsing for `yield *expr`, and the multiple ways something can or can't be parsed like that. Also makes yield-from a TODO in the bytecode generator. Behold, the glory of javascript syntax: ```js // 'yield' = expression in generators. function* foo() { yield *bar; // <- Syntax error here, expression can't start with * } // 'yield' = identifier anywhere else. function foo() { yield *bar; // Perfectly fine, this is just `yield * bar` } ```
2021-06-14 11:16:41 +00:00
return { parse_regexp_literal() };
LibJS: Add template literals Adds fully functioning template literals. Because template literals contain expressions, most of the work has to be done in the Lexer rather than the Parser. And because of the complexity of template literals (expressions, nesting, escapes, etc), the Lexer needs to have some template-related state. When entering a new template literal, a TemplateLiteralStart token is emitted. When inside a literal, all text will be parsed up until a '${' or '`' (or EOF, but that's a syntax error) is seen, and then a TemplateLiteralExprStart token is emitted. At this point, the Lexer proceeds as normal, however it keeps track of the number of opening and closing curly braces it has seen in order to determine the close of the expression. Once it finds a matching curly brace for the '${', a TemplateLiteralExprEnd token is emitted and the state is updated accordingly. When the Lexer is inside of a template literal, but not an expression, and sees a '`', this must be the closing grave: a TemplateLiteralEnd token is emitted. The state required to correctly parse template strings consists of a vector (for nesting) of two pieces of information: whether or not we are in a template expression (as opposed to a template string); and the count of the number of unmatched open curly braces we have seen (only applicable if the Lexer is currently in a template expression). TODO: Add support for template literal newlines in the JS REPL (this will cause a syntax error currently): > `foo > bar` 'foo bar'
2020-05-03 22:41:14 +00:00
case TokenType::TemplateLiteralStart:
LibJS: Correctly parse yield-from expressions This commit implements parsing for `yield *expr`, and the multiple ways something can or can't be parsed like that. Also makes yield-from a TODO in the bytecode generator. Behold, the glory of javascript syntax: ```js // 'yield' = expression in generators. function* foo() { yield *bar; // <- Syntax error here, expression can't start with * } // 'yield' = identifier anywhere else. function foo() { yield *bar; // Perfectly fine, this is just `yield * bar` } ```
2021-06-14 11:16:41 +00:00
return { parse_template_literal(false) };
LibJS: Implement 'new.target' This adds a new MetaProperty AST node which will be used for 'new.target' and 'import.meta' meta properties. The parser now distinguishes between "in function context" and "in arrow function context" (which is required for this). When encountering TokenType::New we will attempt to parse it as meta property and resort to regular new expression parsing if that fails, much like the parsing of labelled statements.
2020-11-02 21:27:42 +00:00
case TokenType::New: {
auto new_start = position();
auto new_target_result = try_parse_new_target_expression();
if (!new_target_result.is_null()) {
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
if (!m_state.in_function_context)
LibJS: Implement 'new.target' This adds a new MetaProperty AST node which will be used for 'new.target' and 'import.meta' meta properties. The parser now distinguishes between "in function context" and "in arrow function context" (which is required for this). When encountering TokenType::New we will attempt to parse it as meta property and resort to regular new expression parsing if that fails, much like the parsing of labelled statements.
2020-11-02 21:27:42 +00:00
syntax_error("'new.target' not allowed outside of a function", new_start);
LibJS: Correctly parse yield-from expressions This commit implements parsing for `yield *expr`, and the multiple ways something can or can't be parsed like that. Also makes yield-from a TODO in the bytecode generator. Behold, the glory of javascript syntax: ```js // 'yield' = expression in generators. function* foo() { yield *bar; // <- Syntax error here, expression can't start with * } // 'yield' = identifier anywhere else. function foo() { yield *bar; // Perfectly fine, this is just `yield * bar` } ```
2021-06-14 11:16:41 +00:00
return { new_target_result.release_nonnull() };
LibJS: Implement 'new.target' This adds a new MetaProperty AST node which will be used for 'new.target' and 'import.meta' meta properties. The parser now distinguishes between "in function context" and "in arrow function context" (which is required for this). When encountering TokenType::New we will attempt to parse it as meta property and resort to regular new expression parsing if that fails, much like the parsing of labelled statements.
2020-11-02 21:27:42 +00:00
}
LibJS: Correctly parse yield-from expressions This commit implements parsing for `yield *expr`, and the multiple ways something can or can't be parsed like that. Also makes yield-from a TODO in the bytecode generator. Behold, the glory of javascript syntax: ```js // 'yield' = expression in generators. function* foo() { yield *bar; // <- Syntax error here, expression can't start with * } // 'yield' = identifier anywhere else. function foo() { yield *bar; // Perfectly fine, this is just `yield * bar` } ```
2021-06-14 11:16:41 +00:00
return { parse_new_expression() };
LibJS: Implement 'new.target' This adds a new MetaProperty AST node which will be used for 'new.target' and 'import.meta' meta properties. The parser now distinguishes between "in function context" and "in arrow function context" (which is required for this). When encountering TokenType::New we will attempt to parse it as meta property and resort to regular new expression parsing if that fails, much like the parsing of labelled statements.
2020-11-02 21:27:42 +00:00
}
LibJS: Implement generator functions (only in bytecode mode)
2021-06-10 21:08:30 +00:00
case TokenType::Yield:
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
if (!m_state.in_generator_function_context)
LibJS: Implement generator functions (only in bytecode mode)
2021-06-10 21:08:30 +00:00
goto read_as_identifier;
LibJS: Correctly parse yield-from expressions This commit implements parsing for `yield *expr`, and the multiple ways something can or can't be parsed like that. Also makes yield-from a TODO in the bytecode generator. Behold, the glory of javascript syntax: ```js // 'yield' = expression in generators. function* foo() { yield *bar; // <- Syntax error here, expression can't start with * } // 'yield' = identifier anywhere else. function foo() { yield *bar; // Perfectly fine, this is just `yield * bar` } ```
2021-06-14 11:16:41 +00:00
return { parse_yield_expression(), false };
LibJS: Add support for await expressions
2021-11-09 20:52:21 +00:00
case TokenType::Await:
if (!m_state.in_async_function_context)
goto read_as_identifier;
return { parse_await_expression() };
LibJS: Add support for the '#privateName in obj' expression
2021-10-14 00:05:24 +00:00
case TokenType::PrivateIdentifier:
VERIFY(next_token().type() == TokenType::In);
if (!is_private_identifier_valid())
syntax_error(String::formatted("Reference to undeclared private field or method '{}'", m_state.current_token.value()));
return { create_ast_node<PrivateIdentifier>({ m_state.current_token.filename(), rule_start.position(), position() }, consume().value()) };
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
default:
LibJS: Rework Identifier parsing to match the spec more closely
2021-07-11 11:04:55 +00:00
if (match_identifier_name())
goto read_as_identifier;
LibJS: Distinguish between statement and declaration This separates matching/parsing of statements and declarations and fixes a few edge cases where the parser would incorrectly accept a declaration where only a statement is allowed - for example: if (foo) const a = 1; for (var bar;;) function b() {} while (baz) class c {}
2020-10-22 22:30:07 +00:00
expected("primary expression");
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
consume();
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
return { create_ast_node<ErrorExpression>({ m_state.current_token.filename(), rule_start.position(), position() }) };
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
}
}
LibJS: Lex and parse regex literals, add RegExp objects This adds regex parsing/lexing, as well as a relatively empty RegExpObject. The purpose of this patch is to allow the engine to not get hung up on parsing regexes. This will aid in finding new syntax errors (say, from google or twitter) without having to replace all of their regexes first!
2020-06-03 23:05:49 +00:00
NonnullRefPtr<RegExpLiteral> Parser::parse_regexp_literal()
{
LibJS: Track source positions all the way down to exceptions This makes exceptions have a trace of source positions too, which could probably be helpful in making fancier error tracebacks.
2020-12-28 17:15:22 +00:00
auto rule_start = push_start();
LibJS: Rename RegExpLiteral m_content to m_pattern This is what we call it elsewhere, let's be consistent.
2021-05-10 10:56:08 +00:00
auto pattern = consume().value();
// Remove leading and trailing slash.
pattern = pattern.substring_view(1, pattern.length() - 2);
LibJS: Parse RegExp literals at AST creation time, not execution time The spec requires that invalid RegExp literals must cause a Syntax Error before the JavaScript is executed. See: https://tc39.es/ecma262/#sec-patterns-static-semantics-early-errors This is explicitly tested in the RegExp/property-escapes test262 tests. For example, see unsupported-property-Line_Break.js: $DONOTEVALUATE(); /\p{Line_Break}/u; That RegExp literal is invalid because Line_Break is not a supported Unicode property. $DONOTEVALUATE() just throws an exception when it is executed. The test expects that this file will fail to be parsed. Note that RegExp patterns can still be parsed at execution time by way of "new RegExp(...)".
2021-07-29 14:34:37 +00:00
LibJS: Make invalid RegExp flags a SyntaxError at parse time This patch changes the validation of RegExp flags (checking for invalid and duplicate values) from a SyntaxError at runtime to a SyntaxError at parse time - it's not something that's supposed to be catchable. As a nice side effect, this simplifies the RegExpObject constructor a bit, as it can no longer throw an exception and doesn't have to validate the flags itself.
2021-05-10 11:01:38 +00:00
auto flags = String::empty();
LibJS: Parse RegExp literals at AST creation time, not execution time The spec requires that invalid RegExp literals must cause a Syntax Error before the JavaScript is executed. See: https://tc39.es/ecma262/#sec-patterns-static-semantics-early-errors This is explicitly tested in the RegExp/property-escapes test262 tests. For example, see unsupported-property-Line_Break.js: $DONOTEVALUATE(); /\p{Line_Break}/u; That RegExp literal is invalid because Line_Break is not a supported Unicode property. $DONOTEVALUATE() just throws an exception when it is executed. The test expects that this file will fail to be parsed. Note that RegExp patterns can still be parsed at execution time by way of "new RegExp(...)".
2021-07-29 14:34:37 +00:00
auto parsed_flags = RegExpObject::default_flags;
LibJS: Make invalid RegExp flags a SyntaxError at parse time This patch changes the validation of RegExp flags (checking for invalid and duplicate values) from a SyntaxError at runtime to a SyntaxError at parse time - it's not something that's supposed to be catchable. As a nice side effect, this simplifies the RegExpObject constructor a bit, as it can no longer throw an exception and doesn't have to validate the flags itself.
2021-05-10 11:01:38 +00:00
if (match(TokenType::RegexFlags)) {
auto flags_start = position();
flags = consume().value();
LibJS: Parse RegExp literals at AST creation time, not execution time The spec requires that invalid RegExp literals must cause a Syntax Error before the JavaScript is executed. See: https://tc39.es/ecma262/#sec-patterns-static-semantics-early-errors This is explicitly tested in the RegExp/property-escapes test262 tests. For example, see unsupported-property-Line_Break.js: $DONOTEVALUATE(); /\p{Line_Break}/u; That RegExp literal is invalid because Line_Break is not a supported Unicode property. $DONOTEVALUATE() just throws an exception when it is executed. The test expects that this file will fail to be parsed. Note that RegExp patterns can still be parsed at execution time by way of "new RegExp(...)".
2021-07-29 14:34:37 +00:00
auto parsed_flags_or_error = regex_flags_from_string(flags);
if (parsed_flags_or_error.is_error())
syntax_error(parsed_flags_or_error.release_error(), flags_start);
else
parsed_flags = parsed_flags_or_error.release_value();
LibJS: Make invalid RegExp flags a SyntaxError at parse time This patch changes the validation of RegExp flags (checking for invalid and duplicate values) from a SyntaxError at runtime to a SyntaxError at parse time - it's not something that's supposed to be catchable. As a nice side effect, this simplifies the RegExpObject constructor a bit, as it can no longer throw an exception and doesn't have to validate the flags itself.
2021-05-10 11:01:38 +00:00
}
LibJS: Parse RegExp literals at AST creation time, not execution time The spec requires that invalid RegExp literals must cause a Syntax Error before the JavaScript is executed. See: https://tc39.es/ecma262/#sec-patterns-static-semantics-early-errors This is explicitly tested in the RegExp/property-escapes test262 tests. For example, see unsupported-property-Line_Break.js: $DONOTEVALUATE(); /\p{Line_Break}/u; That RegExp literal is invalid because Line_Break is not a supported Unicode property. $DONOTEVALUATE() just throws an exception when it is executed. The test expects that this file will fail to be parsed. Note that RegExp patterns can still be parsed at execution time by way of "new RegExp(...)".
2021-07-29 14:34:37 +00:00
auto parsed_pattern = parse_regex_pattern(pattern, parsed_flags.has_flag_set(ECMAScriptFlags::Unicode));
auto parsed_regex = Regex<ECMA262>::parse_pattern(parsed_pattern, parsed_flags);
if (parsed_regex.error != regex::Error::NoError)
syntax_error(String::formatted("RegExp compile error: {}", Regex<ECMA262>(parsed_regex, parsed_pattern, parsed_flags).error_string()), rule_start.position());
SourceRange range { m_state.current_token.filename(), rule_start.position(), position() };
return create_ast_node<RegExpLiteral>(move(range), move(parsed_regex), move(parsed_pattern), move(parsed_flags), pattern.to_string(), move(flags));
LibJS: Lex and parse regex literals, add RegExp objects This adds regex parsing/lexing, as well as a relatively empty RegExpObject. The purpose of this patch is to allow the engine to not get hung up on parsing regexes. This will aid in finding new syntax errors (say, from google or twitter) without having to replace all of their regexes first!
2020-06-03 23:05:49 +00:00
}
LibJS: Make the AST reference-counted This allows function objects to outlive the original parsed program without their ScopeNode disappearing.
2020-03-18 10:23:53 +00:00
NonnullRefPtr<Expression> Parser::parse_unary_prefixed_expression()
LibJS/Parser: Implement the parsing of unary/prefixed update expressions
2020-03-14 18:45:51 +00:00
{
LibJS: Track source positions all the way down to exceptions This makes exceptions have a trace of source positions too, which could probably be helpful in making fancier error tracebacks.
2020-12-28 17:15:22 +00:00
auto rule_start = push_start();
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
auto precedence = g_operator_precedence.get(m_state.current_token.type());
auto associativity = operator_associativity(m_state.current_token.type());
switch (m_state.current_token.type()) {
LibJS: Enforce that ++/-- operand is an identifier or member expression
2020-04-29 20:39:00 +00:00
case TokenType::PlusPlus: {
consume();
LibJS: Replace 'size_t line, size_t column' with 'Optional<Position>' This is a bit nicer for two reasons: - The absence of line number/column information isn't based on 'values are zero' anymore but on Optional's value - When reporting syntax errors with position information other than the current token's position we had to store line and column ourselves, like this: auto foo_start_line = m_parser_state.m_current_token.line_number(); auto foo_start_column = m_parser_state.m_current_token.line_column(); ... syntax_error("...", foo_start_line, foo_start_column); Which now becomes: auto foo_start= position(); ... syntax_error("...", foo_start); This makes it easier to report correct positions for syntax errors that only emerge a few tokens later :^)
2020-11-02 21:03:19 +00:00
auto rhs_start = position();
LibJS: Enforce that ++/-- operand is an identifier or member expression
2020-04-29 20:39:00 +00:00
auto rhs = parse_expression(precedence, associativity);
LibJS: Add FIXMEs for parsing increment operators with function LHS/RHS The parser considers it a syntax error at the moment, other engines throw a ReferenceError during runtime for ++foo(), --foo(), foo()++ and foo()--, so I assume the spec defines this.
2020-09-16 19:03:06 +00:00
// FIXME: Apparently for functions this should also not be enforced on a parser level,
// other engines throw ReferenceError for ++foo()
LibJS: Remove hand-rolled type information in JS AST in favor of RTTI
2021-01-01 18:34:07 +00:00
if (!is<Identifier>(*rhs) && !is<MemberExpression>(*rhs))
LibJS: Replace 'size_t line, size_t column' with 'Optional<Position>' This is a bit nicer for two reasons: - The absence of line number/column information isn't based on 'values are zero' anymore but on Optional's value - When reporting syntax errors with position information other than the current token's position we had to store line and column ourselves, like this: auto foo_start_line = m_parser_state.m_current_token.line_number(); auto foo_start_column = m_parser_state.m_current_token.line_column(); ... syntax_error("...", foo_start_line, foo_start_column); Which now becomes: auto foo_start= position(); ... syntax_error("...", foo_start); This makes it easier to report correct positions for syntax errors that only emerge a few tokens later :^)
2020-11-02 21:03:19 +00:00
syntax_error(String::formatted("Right-hand side of prefix increment operator must be identifier or member expression, got {}", rhs->class_name()), rhs_start);
LibJS: Be more strict about reserved and special identifiers
2021-07-11 23:27:35 +00:00
if (m_state.strict_mode && is<Identifier>(*rhs)) {
auto& identifier = static_cast<Identifier&>(*rhs);
auto& name = identifier.string();
check_identifier_name_for_assignment_validity(name);
}
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
return create_ast_node<UpdateExpression>({ m_state.current_token.filename(), rule_start.position(), position() }, UpdateOp::Increment, move(rhs), true);
LibJS: Enforce that ++/-- operand is an identifier or member expression
2020-04-29 20:39:00 +00:00
}
case TokenType::MinusMinus: {
consume();
LibJS: Replace 'size_t line, size_t column' with 'Optional<Position>' This is a bit nicer for two reasons: - The absence of line number/column information isn't based on 'values are zero' anymore but on Optional's value - When reporting syntax errors with position information other than the current token's position we had to store line and column ourselves, like this: auto foo_start_line = m_parser_state.m_current_token.line_number(); auto foo_start_column = m_parser_state.m_current_token.line_column(); ... syntax_error("...", foo_start_line, foo_start_column); Which now becomes: auto foo_start= position(); ... syntax_error("...", foo_start); This makes it easier to report correct positions for syntax errors that only emerge a few tokens later :^)
2020-11-02 21:03:19 +00:00
auto rhs_start = position();
LibJS: Enforce that ++/-- operand is an identifier or member expression
2020-04-29 20:39:00 +00:00
auto rhs = parse_expression(precedence, associativity);
LibJS: Add FIXMEs for parsing increment operators with function LHS/RHS The parser considers it a syntax error at the moment, other engines throw a ReferenceError during runtime for ++foo(), --foo(), foo()++ and foo()--, so I assume the spec defines this.
2020-09-16 19:03:06 +00:00
// FIXME: Apparently for functions this should also not be enforced on a parser level,
// other engines throw ReferenceError for --foo()
LibJS: Remove hand-rolled type information in JS AST in favor of RTTI
2021-01-01 18:34:07 +00:00
if (!is<Identifier>(*rhs) && !is<MemberExpression>(*rhs))
LibJS: Replace 'size_t line, size_t column' with 'Optional<Position>' This is a bit nicer for two reasons: - The absence of line number/column information isn't based on 'values are zero' anymore but on Optional's value - When reporting syntax errors with position information other than the current token's position we had to store line and column ourselves, like this: auto foo_start_line = m_parser_state.m_current_token.line_number(); auto foo_start_column = m_parser_state.m_current_token.line_column(); ... syntax_error("...", foo_start_line, foo_start_column); Which now becomes: auto foo_start= position(); ... syntax_error("...", foo_start); This makes it easier to report correct positions for syntax errors that only emerge a few tokens later :^)
2020-11-02 21:03:19 +00:00
syntax_error(String::formatted("Right-hand side of prefix decrement operator must be identifier or member expression, got {}", rhs->class_name()), rhs_start);
LibJS: Be more strict about reserved and special identifiers
2021-07-11 23:27:35 +00:00
if (m_state.strict_mode && is<Identifier>(*rhs)) {
auto& identifier = static_cast<Identifier&>(*rhs);
auto& name = identifier.string();
check_identifier_name_for_assignment_validity(name);
}
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
return create_ast_node<UpdateExpression>({ m_state.current_token.filename(), rule_start.position(), position() }, UpdateOp::Decrement, move(rhs), true);
LibJS: Enforce that ++/-- operand is an identifier or member expression
2020-04-29 20:39:00 +00:00
}
LibJS/Parser: Implement the parsing of unary/prefixed update expressions
2020-03-14 18:45:51 +00:00
case TokenType::ExclamationMark:
consume();
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
return create_ast_node<UnaryExpression>({ m_state.current_token.filename(), rule_start.position(), position() }, UnaryOp::Not, parse_expression(precedence, associativity));
LibJS/Parser: Implement the parsing of unary/prefixed update expressions
2020-03-14 18:45:51 +00:00
case TokenType::Tilde:
consume();
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
return create_ast_node<UnaryExpression>({ m_state.current_token.filename(), rule_start.position(), position() }, UnaryOp::BitwiseNot, parse_expression(precedence, associativity));
LibJS: Implement unary plus / minus
2020-04-02 16:58:39 +00:00
case TokenType::Plus:
consume();
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
return create_ast_node<UnaryExpression>({ m_state.current_token.filename(), rule_start.position(), position() }, UnaryOp::Plus, parse_expression(precedence, associativity));
LibJS: Implement unary plus / minus
2020-04-02 16:58:39 +00:00
case TokenType::Minus:
consume();
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
return create_ast_node<UnaryExpression>({ m_state.current_token.filename(), rule_start.position(), position() }, UnaryOp::Minus, parse_expression(precedence, associativity));
LibJS: Implement typeof operator
2020-03-17 19:33:32 +00:00
case TokenType::Typeof:
consume();
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
return create_ast_node<UnaryExpression>({ m_state.current_token.filename(), rule_start.position(), position() }, UnaryOp::Typeof, parse_expression(precedence, associativity));
LibJS: Implement void operator
2020-04-15 16:55:03 +00:00
case TokenType::Void:
consume();
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
return create_ast_node<UnaryExpression>({ m_state.current_token.filename(), rule_start.position(), position() }, UnaryOp::Void, parse_expression(precedence, associativity));
LibJS: Disallow unqualified deletes in strict mode
2021-07-11 23:32:24 +00:00
case TokenType::Delete: {
LibJS: Implement basic support for the "delete" operator It turns out "delete" is actually a unary op :) This patch implements deletion of object properties, it doesn't yet work for casually deleting properties from the global object. When deleting a property from an object, we switch that object to having a unique shape, no longer sharing shapes with others. Once an object has a unique shape, it no longer needs to care about shape transitions.
2020-04-26 11:53:40 +00:00
consume();
LibJS: Disallow unqualified deletes in strict mode
2021-07-11 23:32:24 +00:00
auto rhs_start = position();
auto rhs = parse_expression(precedence, associativity);
if (is<Identifier>(*rhs) && m_state.strict_mode) {
syntax_error("Delete of an unqualified identifier in strict mode.", rhs_start);
}
LibJS: Add parsing and evaluation of private fields and methods
2021-10-12 20:45:52 +00:00
if (is<MemberExpression>(*rhs)) {
auto& member_expression = static_cast<MemberExpression const&>(*rhs);
if (member_expression.ends_in_private_name())
syntax_error("Private fields cannot be deleted");
}
LibJS: Disallow unqualified deletes in strict mode
2021-07-11 23:32:24 +00:00
return create_ast_node<UnaryExpression>({ m_state.current_token.filename(), rule_start.position(), position() }, UnaryOp::Delete, move(rhs));
}
LibJS/Parser: Implement the parsing of unary/prefixed update expressions
2020-03-14 18:45:51 +00:00
default:
LibJS: Distinguish between statement and declaration This separates matching/parsing of statements and declarations and fixes a few edge cases where the parser would incorrectly accept a declaration where only a statement is allowed - for example: if (foo) const a = 1; for (var bar;;) function b() {} while (baz) class c {}
2020-10-22 22:30:07 +00:00
expected("primary expression");
LibJS/Parser: Implement the parsing of unary/prefixed update expressions
2020-03-14 18:45:51 +00:00
consume();
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
return create_ast_node<ErrorExpression>({ m_state.current_token.filename(), rule_start.position(), position() });
LibJS/Parser: Implement the parsing of unary/prefixed update expressions
2020-03-14 18:45:51 +00:00
}
}
LibJS: Initial class implementation; allow super expressions in object literal methods; add EnvrionmentRecord fields and methods to LexicalEnvironment Adding EnvrionmentRecord's fields and methods lets us throw an exception when |this| is not initialized, which occurs when the super constructor in a derived class has not yet been called, or when |this| has already been initialized (the super constructor was already called).
2020-06-08 18:31:21 +00:00
NonnullRefPtr<Expression> Parser::parse_property_key()
{
LibJS: Track source positions all the way down to exceptions This makes exceptions have a trace of source positions too, which could probably be helpful in making fancier error tracebacks.
2020-12-28 17:15:22 +00:00
auto rule_start = push_start();
LibJS: Initial class implementation; allow super expressions in object literal methods; add EnvrionmentRecord fields and methods to LexicalEnvironment Adding EnvrionmentRecord's fields and methods lets us throw an exception when |this| is not initialized, which occurs when the super constructor in a derived class has not yet been called, or when |this| has already been initialized (the super constructor was already called).
2020-06-08 18:31:21 +00:00
if (match(TokenType::StringLiteral)) {
return parse_string_literal(consume());
} else if (match(TokenType::NumericLiteral)) {
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
return create_ast_node<NumericLiteral>({ m_state.current_token.filename(), rule_start.position(), position() }, consume().double_value());
LibJS: Initial class implementation; allow super expressions in object literal methods; add EnvrionmentRecord fields and methods to LexicalEnvironment Adding EnvrionmentRecord's fields and methods lets us throw an exception when |this| is not initialized, which occurs when the super constructor in a derived class has not yet been called, or when |this| has already been initialized (the super constructor was already called).
2020-06-08 18:31:21 +00:00
} else if (match(TokenType::BigIntLiteral)) {
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
return create_ast_node<BigIntLiteral>({ m_state.current_token.filename(), rule_start.position(), position() }, consume().value());
LibJS: Initial class implementation; allow super expressions in object literal methods; add EnvrionmentRecord fields and methods to LexicalEnvironment Adding EnvrionmentRecord's fields and methods lets us throw an exception when |this| is not initialized, which occurs when the super constructor in a derived class has not yet been called, or when |this| has already been initialized (the super constructor was already called).
2020-06-08 18:31:21 +00:00
} else if (match(TokenType::BracketOpen)) {
consume(TokenType::BracketOpen);
LibJS: Parse only AssignmentExpressions in ComputedPropertyNames The property name in an object literal can either be a literal or a computed name, in which case any AssignmentExpression can be used, we now only parse AssignmentExpression instead of the previous incorrect behaviour which allowed any Expression (Specifically, comma expressions).
2021-06-11 15:43:28 +00:00
auto result = parse_expression(2);
LibJS: Initial class implementation; allow super expressions in object literal methods; add EnvrionmentRecord fields and methods to LexicalEnvironment Adding EnvrionmentRecord's fields and methods lets us throw an exception when |this| is not initialized, which occurs when the super constructor in a derived class has not yet been called, or when |this| has already been initialized (the super constructor was already called).
2020-06-08 18:31:21 +00:00
consume(TokenType::BracketClose);
return result;
} else {
if (!match_identifier_name())
expected("IdentifierName");
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
return create_ast_node<StringLiteral>({ m_state.current_token.filename(), rule_start.position(), position() }, consume().value());
LibJS: Initial class implementation; allow super expressions in object literal methods; add EnvrionmentRecord fields and methods to LexicalEnvironment Adding EnvrionmentRecord's fields and methods lets us throw an exception when |this| is not initialized, which occurs when the super constructor in a derived class has not yet been called, or when |this| has already been initialized (the super constructor was already called).
2020-06-08 18:31:21 +00:00
}
}
LibJS: Make the AST reference-counted This allows function objects to outlive the original parsed program without their ScopeNode disappearing.
2020-03-18 10:23:53 +00:00
NonnullRefPtr<ObjectExpression> Parser::parse_object_expression()
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
{
LibJS: Track source positions all the way down to exceptions This makes exceptions have a trace of source positions too, which could probably be helpful in making fancier error tracebacks.
2020-12-28 17:15:22 +00:00
auto rule_start = push_start();
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
consume(TokenType::CurlyOpen);
LibJS: Parse object expressions
2020-03-21 00:29:00 +00:00
LibJS: Rewrite Parser.parse_object_expression() This rewrite drastically increases the accuracy of object literals. Additionally, an "assertIsSyntaxError" function has been added to test-common.js to assist in testing syntax errors.
2020-05-29 05:50:06 +00:00
NonnullRefPtrVector<ObjectProperty> properties;
ObjectProperty::Type property_type;
LibJS: Allow 'name = value' in object literals as the spec does Currently, these are _always_ a syntax error, future commits will make it valid in certain contexts.
2021-07-11 10:18:30 +00:00
Optional<SourceRange> invalid_object_literal_property_range;
LibJS: Rewrite Parser.parse_object_expression() This rewrite drastically increases the accuracy of object literals. Additionally, an "assertIsSyntaxError" function has been added to test-common.js to assist in testing syntax errors.
2020-05-29 05:50:06 +00:00
auto skip_to_next_property = [&] {
while (!done() && !match(TokenType::Comma) && !match(TokenType::CurlyOpen))
consume();
};
LibJS: Add object literal getter/setter shorthand Adds support for the following syntax: let foo = { get x() { // ... }, set x(value) { // ... } }
2020-05-22 00:28:28 +00:00
LibJS: Disallow multiple __proto__ keys in object expression
2021-07-11 23:32:01 +00:00
// It is a Syntax Error if PropertyNameList of PropertyDefinitionList contains any duplicate
// entries for "__proto__" and at least two of those entries were obtained from productions of
LibJS: Rename PropertyName to PropertyKey Let's use the same name as the spec. :^)
2021-10-24 14:01:24 +00:00
// the form PropertyDefinition : PropertyKey : AssignmentExpression .
LibJS: Disallow multiple __proto__ keys in object expression
2021-07-11 23:32:01 +00:00
bool has_direct_proto_property = false;
LibJS: Break loop on EOF when parsing object expression
2020-04-07 17:29:37 +00:00
while (!done() && !match(TokenType::CurlyClose)) {
LibJS: Rewrite Parser.parse_object_expression() This rewrite drastically increases the accuracy of object literals. Additionally, an "assertIsSyntaxError" function has been added to test-common.js to assist in testing syntax errors.
2020-05-29 05:50:06 +00:00
property_type = ObjectProperty::Type::KeyValue;
RefPtr<Expression> property_name;
LibJS: Implement computed properties in object expressions
2020-04-23 18:37:53 +00:00
RefPtr<Expression> property_value;
LibJS: Parse generator functions in object literals Also add some parser tests
2021-06-14 10:22:59 +00:00
FunctionKind function_kind { FunctionKind::Regular };
LibJS: Add spreading in object literals Supports spreading strings, arrays, and other objects within object literals.
2020-04-28 04:52:47 +00:00
LibJS: Rewrite Parser.parse_object_expression() This rewrite drastically increases the accuracy of object literals. Additionally, an "assertIsSyntaxError" function has been added to test-common.js to assist in testing syntax errors.
2020-05-29 05:50:06 +00:00
if (match(TokenType::TripleDot)) {
consume();
property_name = parse_expression(4);
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
properties.append(create_ast_node<ObjectProperty>({ m_state.current_token.filename(), rule_start.position(), position() }, *property_name, nullptr, ObjectProperty::Type::Spread, false));
LibJS: Rewrite Parser.parse_object_expression() This rewrite drastically increases the accuracy of object literals. Additionally, an "assertIsSyntaxError" function has been added to test-common.js to assist in testing syntax errors.
2020-05-29 05:50:06 +00:00
if (!match(TokenType::Comma))
break;
consume(TokenType::Comma);
continue;
}
LibJS: Disallow multiple __proto__ keys in object expression
2021-07-11 23:32:01 +00:00
auto type = m_state.current_token.type();
LibJS: Parse async generator functions
2021-11-15 00:53:24 +00:00
if (match(TokenType::Async)) {
auto lookahead_token = next_token();
LibJS: Allow object properties called 'async'
2021-11-26 15:40:27 +00:00
if (lookahead_token.type() != TokenType::ParenOpen && lookahead_token.type() != TokenType::Colon
&& !lookahead_token.trivia_contains_line_terminator()) {
LibJS: Parse async generator functions
2021-11-15 00:53:24 +00:00
consume(TokenType::Async);
function_kind = FunctionKind::Async;
}
}
LibJS: Parse generator functions in object literals Also add some parser tests
2021-06-14 10:22:59 +00:00
if (match(TokenType::Asterisk)) {
consume();
property_type = ObjectProperty::Type::KeyValue;
property_name = parse_property_key();
LibJS: Parse async generator functions
2021-11-15 00:53:24 +00:00
VERIFY(function_kind == FunctionKind::Regular || function_kind == FunctionKind::Async);
function_kind = function_kind == FunctionKind::Regular ? FunctionKind::Generator : FunctionKind::AsyncGenerator;
LibJS: Treat yield as an identifier in more non-generator contexts And disallow some cases where we are in a generator context.
2021-07-28 23:45:55 +00:00
} else if (match_identifier()) {
LibJS: Fix small issues in parser - Fix some places where escaped keywords are (not) allowed. - Be more strict about parameters for functions with 'use strict'. - Fix that expressions statements allowed functions and classes. - Fix that class expressions were not allowed. - Added a new next_token() method for checking the look ahead. - Fix that continue labels could jump to non iterating targets. - Fix that generator functions cannot be declared in if statements.
2021-08-28 15:04:37 +00:00
auto identifier = consume();
LibJS: Parse async generator functions
2021-11-15 00:53:24 +00:00
if (identifier.original_value() == "get"sv && match_property_key()) {
LibJS: Rewrite Parser.parse_object_expression() This rewrite drastically increases the accuracy of object literals. Additionally, an "assertIsSyntaxError" function has been added to test-common.js to assist in testing syntax errors.
2020-05-29 05:50:06 +00:00
property_type = ObjectProperty::Type::Getter;
property_name = parse_property_key();
LibJS: Fix small issues in parser - Fix some places where escaped keywords are (not) allowed. - Be more strict about parameters for functions with 'use strict'. - Fix that expressions statements allowed functions and classes. - Fix that class expressions were not allowed. - Added a new next_token() method for checking the look ahead. - Fix that continue labels could jump to non iterating targets. - Fix that generator functions cannot be declared in if statements.
2021-08-28 15:04:37 +00:00
} else if (identifier.original_value() == "set"sv && match_property_key()) {
LibJS: Rewrite Parser.parse_object_expression() This rewrite drastically increases the accuracy of object literals. Additionally, an "assertIsSyntaxError" function has been added to test-common.js to assist in testing syntax errors.
2020-05-29 05:50:06 +00:00
property_type = ObjectProperty::Type::Setter;
property_name = parse_property_key();
} else {
LibJS: Fix small issues in parser - Fix some places where escaped keywords are (not) allowed. - Be more strict about parameters for functions with 'use strict'. - Fix that expressions statements allowed functions and classes. - Fix that class expressions were not allowed. - Added a new next_token() method for checking the look ahead. - Fix that continue labels could jump to non iterating targets. - Fix that generator functions cannot be declared in if statements.
2021-08-28 15:04:37 +00:00
property_name = create_ast_node<StringLiteral>({ m_state.current_token.filename(), rule_start.position(), position() }, identifier.value());
property_value = create_ast_node<Identifier>({ m_state.current_token.filename(), rule_start.position(), position() }, identifier.value());
LibJS: Add object literal getter/setter shorthand Adds support for the following syntax: let foo = { get x() { // ... }, set x(value) { // ... } }
2020-05-22 00:28:28 +00:00
}
LibJS: Allow parsing numeric and string literals in object expressions Also updated the object-basic.js test to include this change
2020-04-06 20:17:05 +00:00
} else {
LibJS: Rewrite Parser.parse_object_expression() This rewrite drastically increases the accuracy of object literals. Additionally, an "assertIsSyntaxError" function has been added to test-common.js to assist in testing syntax errors.
2020-05-29 05:50:06 +00:00
property_name = parse_property_key();
}
LibJS: Disallow multiple __proto__ keys in object expression
2021-07-11 23:32:01 +00:00
bool is_proto = (type == TokenType::StringLiteral || type == TokenType::Identifier) && is<StringLiteral>(*property_name) && static_cast<StringLiteral const&>(*property_name).value() == "__proto__";
LibJS: Rewrite Parser.parse_object_expression() This rewrite drastically increases the accuracy of object literals. Additionally, an "assertIsSyntaxError" function has been added to test-common.js to assist in testing syntax errors.
2020-05-29 05:50:06 +00:00
if (property_type == ObjectProperty::Type::Getter || property_type == ObjectProperty::Type::Setter) {
if (!match(TokenType::ParenOpen)) {
LibJS: Use expected() instead of syntax_error("Expected ...")
2021-07-11 19:00:55 +00:00
expected("'(' for object getter or setter property");
LibJS: Rewrite Parser.parse_object_expression() This rewrite drastically increases the accuracy of object literals. Additionally, an "assertIsSyntaxError" function has been added to test-common.js to assist in testing syntax errors.
2020-05-29 05:50:06 +00:00
skip_to_next_property();
LibJS: Add object literal getter/setter shorthand Adds support for the following syntax: let foo = { get x() { // ... }, set x(value) { // ... } }
2020-05-22 00:28:28 +00:00
continue;
}
LibJS: Allow parsing numeric and string literals in object expressions Also updated the object-basic.js test to include this change
2020-04-06 20:17:05 +00:00
}
LibJS: Allow 'name = value' in object literals as the spec does Currently, these are _always_ a syntax error, future commits will make it valid in certain contexts.
2021-07-11 10:18:30 +00:00
if (match(TokenType::Equals)) {
// Not a valid object literal, but a valid assignment target
consume();
// Parse the expression and throw it away
auto expression = parse_expression(2);
if (!invalid_object_literal_property_range.has_value())
invalid_object_literal_property_range = expression->source_range();
} else if (match(TokenType::ParenOpen)) {
Everywhere: Rename ASSERT => VERIFY (...and ASSERT_NOT_REACHED => VERIFY_NOT_REACHED) Since all of these checks are done in release builds as well, let's rename them to VERIFY to prevent confusion, as everyone is used to assertions being compiled out in release. We can introduce a new ASSERT macro that is specifically for debug checks, but I'm doing this wholesale conversion first since we've accumulated thousands of these already, and it's not immediately obvious which ones are suitable for ASSERT.
2021-02-23 19:42:32 +00:00
VERIFY(property_name);
LibJS: Move checks for invalid getter/setter params to parse_function_node This allows us to provide better error messages as we can point the syntax error location to the exact first invalid parameter instead of always the end of the function within a object literal or class definition. Before this change: const Foo = { set bar() {} } ^ Uncaught exception: [SyntaxError]: Object setter property must have one argument (line: 1, column: 28) class Foo { set bar() {} } ^ Uncaught exception: [SyntaxError]: Class setter method must have one argument (line: 1, column: 26) After this change: const Foo = { set bar() {} } ^ Uncaught exception: [SyntaxError]: Setter function must have one argument (line: 1, column: 23) class Foo { set bar() {} } ^ Uncaught exception: [SyntaxError]: Setter function must have one argument (line: 1, column: 21) The only possible downside of this change is that class getters/setters and functions in objects are not distinguished in the message anymore - I don't think that's important though, and classes are (mostly) just syntactic sugar anyway.
2020-10-20 17:32:51 +00:00
u8 parse_options = FunctionNodeParseOptions::AllowSuperPropertyLookup;
if (property_type == ObjectProperty::Type::Getter)
parse_options |= FunctionNodeParseOptions::IsGetterFunction;
if (property_type == ObjectProperty::Type::Setter)
parse_options |= FunctionNodeParseOptions::IsSetterFunction;
LibJS: Parse async generator functions
2021-11-15 00:53:24 +00:00
if (function_kind == FunctionKind::Generator || function_kind == FunctionKind::AsyncGenerator)
LibJS: Parse generator functions in object literals Also add some parser tests
2021-06-14 10:22:59 +00:00
parse_options |= FunctionNodeParseOptions::IsGeneratorFunction;
LibJS: Parse async generator functions
2021-11-15 00:53:24 +00:00
if (function_kind == FunctionKind::Async || function_kind == FunctionKind::AsyncGenerator)
LibJS: Add support for async functions This commit adds support for the most bare bones version of async functions, support for async generator functions, async arrow functions and await expressions are TODO.
2021-11-09 18:39:22 +00:00
parse_options |= FunctionNodeParseOptions::IsAsyncFunction;
LibJS: Move checks for invalid getter/setter params to parse_function_node This allows us to provide better error messages as we can point the syntax error location to the exact first invalid parameter instead of always the end of the function within a object literal or class definition. Before this change: const Foo = { set bar() {} } ^ Uncaught exception: [SyntaxError]: Object setter property must have one argument (line: 1, column: 28) class Foo { set bar() {} } ^ Uncaught exception: [SyntaxError]: Class setter method must have one argument (line: 1, column: 26) After this change: const Foo = { set bar() {} } ^ Uncaught exception: [SyntaxError]: Setter function must have one argument (line: 1, column: 23) class Foo { set bar() {} } ^ Uncaught exception: [SyntaxError]: Setter function must have one argument (line: 1, column: 21) The only possible downside of this change is that class getters/setters and functions in objects are not distinguished in the message anymore - I don't think that's important though, and classes are (mostly) just syntactic sugar anyway.
2020-10-20 17:32:51 +00:00
auto function = parse_function_node<FunctionExpression>(parse_options);
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
properties.append(create_ast_node<ObjectProperty>({ m_state.current_token.filename(), rule_start.position(), position() }, *property_name, function, property_type, true));
LibJS: Rewrite Parser.parse_object_expression() This rewrite drastically increases the accuracy of object literals. Additionally, an "assertIsSyntaxError" function has been added to test-common.js to assist in testing syntax errors.
2020-05-29 05:50:06 +00:00
} else if (match(TokenType::Colon)) {
LibJS: Replace some parser assertions by syntax errors When parsing JavaScript, we can get pretty much any sequnce of tokens, and we shouldn't crash if it's not something that we normally expect. Instead, emit syntax errors.
2020-06-01 14:08:34 +00:00
if (!property_name) {
LibJS: Use expected() instead of syntax_error("Expected ...")
2021-07-11 19:00:55 +00:00
expected("a property name");
LibJS: Replace some parser assertions by syntax errors When parsing JavaScript, we can get pretty much any sequnce of tokens, and we shouldn't crash if it's not something that we normally expect. Instead, emit syntax errors.
2020-06-01 14:08:34 +00:00
skip_to_next_property();
continue;
}
LibJS: Rewrite Parser.parse_object_expression() This rewrite drastically increases the accuracy of object literals. Additionally, an "assertIsSyntaxError" function has been added to test-common.js to assist in testing syntax errors.
2020-05-29 05:50:06 +00:00
consume();
LibJS: Disallow multiple __proto__ keys in object expression
2021-07-11 23:32:01 +00:00
if (is_proto) {
if (has_direct_proto_property)
syntax_error("Property name '__proto__' must not appear more than once in object literal");
has_direct_proto_property = true;
}
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
properties.append(create_ast_node<ObjectProperty>({ m_state.current_token.filename(), rule_start.position(), position() }, *property_name, parse_expression(2), property_type, false));
LibJS: Replace some parser assertions by syntax errors When parsing JavaScript, we can get pretty much any sequnce of tokens, and we shouldn't crash if it's not something that we normally expect. Instead, emit syntax errors.
2020-06-01 14:08:34 +00:00
} else if (property_name && property_value) {
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
properties.append(create_ast_node<ObjectProperty>({ m_state.current_token.filename(), rule_start.position(), position() }, *property_name, *property_value, property_type, false));
LibJS: Replace some parser assertions by syntax errors When parsing JavaScript, we can get pretty much any sequnce of tokens, and we shouldn't crash if it's not something that we normally expect. Instead, emit syntax errors.
2020-06-01 14:08:34 +00:00
} else {
LibJS: Use expected() instead of syntax_error("Expected ...")
2021-07-11 19:00:55 +00:00
expected("a property");
LibJS: Replace some parser assertions by syntax errors When parsing JavaScript, we can get pretty much any sequnce of tokens, and we shouldn't crash if it's not something that we normally expect. Instead, emit syntax errors.
2020-06-01 14:08:34 +00:00
skip_to_next_property();
continue;
LibJS: Parse object expressions
2020-03-21 00:29:00 +00:00
}
if (!match(TokenType::Comma))
break;
consume(TokenType::Comma);
}
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
consume(TokenType::CurlyClose);
LibJS: Allow 'name = value' in object literals as the spec does Currently, these are _always_ a syntax error, future commits will make it valid in certain contexts.
2021-07-11 10:18:30 +00:00
return create_ast_node<ObjectExpression>(
{ m_state.current_token.filename(), rule_start.position(), position() },
move(properties),
move(invalid_object_literal_property_range));
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
}
LibJS: Parse ArrayExpression and start implementing Array objects Note that property lookup is not functional yet.
2020-03-20 19:29:57 +00:00
NonnullRefPtr<ArrayExpression> Parser::parse_array_expression()
{
LibJS: Track source positions all the way down to exceptions This makes exceptions have a trace of source positions too, which could probably be helpful in making fancier error tracebacks.
2020-12-28 17:15:22 +00:00
auto rule_start = push_start();
LibJS: Parse ArrayExpression and start implementing Array objects Note that property lookup is not functional yet.
2020-03-20 19:29:57 +00:00
consume(TokenType::BracketOpen);
LibJS: Support empty values in array expression
2020-04-15 19:09:06 +00:00
Vector<RefPtr<Expression>> elements;
LibJS: Add spreading in array literals Implement the syntax and behavor necessary to support array literals such as [...[1, 2, 3]]. A type error is thrown if the target of the spread operator does not evaluate to an array (though it should eventually just check for an iterable). Note that the spread token's name is TripleDot, since the '...' token is used for two features: spread and rest. Calling it anything involving 'spread' or 'rest' would be a bit confusing.
2020-04-27 06:05:37 +00:00
while (match_expression() || match(TokenType::TripleDot) || match(TokenType::Comma)) {
LibJS: Support empty values in array expression
2020-04-15 19:09:06 +00:00
RefPtr<Expression> expression;
LibJS: Add spreading in array literals Implement the syntax and behavor necessary to support array literals such as [...[1, 2, 3]]. A type error is thrown if the target of the spread operator does not evaluate to an array (though it should eventually just check for an iterable). Note that the spread token's name is TripleDot, since the '...' token is used for two features: spread and rest. Calling it anything involving 'spread' or 'rest' would be a bit confusing.
2020-04-27 06:05:37 +00:00
if (match(TokenType::TripleDot)) {
consume(TokenType::TripleDot);
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
expression = create_ast_node<SpreadExpression>({ m_state.current_token.filename(), rule_start.position(), position() }, parse_expression(2));
LibJS: Add spreading in array literals Implement the syntax and behavor necessary to support array literals such as [...[1, 2, 3]]. A type error is thrown if the target of the spread operator does not evaluate to an array (though it should eventually just check for an iterable). Note that the spread token's name is TripleDot, since the '...' token is used for two features: spread and rest. Calling it anything involving 'spread' or 'rest' would be a bit confusing.
2020-04-27 06:05:37 +00:00
} else if (match_expression()) {
LibJS: Parse comma operator into SequenceExpression
2020-05-11 16:27:31 +00:00
expression = parse_expression(2);
LibJS: Add spreading in array literals Implement the syntax and behavor necessary to support array literals such as [...[1, 2, 3]]. A type error is thrown if the target of the spread operator does not evaluate to an array (though it should eventually just check for an iterable). Note that the spread token's name is TripleDot, since the '...' token is used for two features: spread and rest. Calling it anything involving 'spread' or 'rest' would be a bit confusing.
2020-04-27 06:05:37 +00:00
}
LibJS: Support empty values in array expression
2020-04-15 19:09:06 +00:00
elements.append(expression);
LibJS: Parse ArrayExpression and start implementing Array objects Note that property lookup is not functional yet.
2020-03-20 19:29:57 +00:00
if (!match(TokenType::Comma))
break;
consume(TokenType::Comma);
}
consume(TokenType::BracketClose);
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
return create_ast_node<ArrayExpression>({ m_state.current_token.filename(), rule_start.position(), position() }, move(elements));
LibJS: Parse ArrayExpression and start implementing Array objects Note that property lookup is not functional yet.
2020-03-20 19:29:57 +00:00
}
LibJS: Use const references to avoid some copies in the parser
2021-02-24 08:58:24 +00:00
NonnullRefPtr<StringLiteral> Parser::parse_string_literal(const Token& token, bool in_template_literal)
LibJS: Handle hex and unicode escape sequences in string literals Introduces the following syntax: '\x55' '\u26a0' '\u{1f41e}'
2020-05-17 06:27:25 +00:00
{
LibJS: Track source positions all the way down to exceptions This makes exceptions have a trace of source positions too, which could probably be helpful in making fancier error tracebacks.
2020-12-28 17:15:22 +00:00
auto rule_start = push_start();
LibJS: Handle hex and unicode escape sequences in string literals Introduces the following syntax: '\x55' '\u26a0' '\u{1f41e}'
2020-05-17 06:27:25 +00:00
auto status = Token::StringValueStatus::Ok;
auto string = token.string_value(status);
if (status != Token::StringValueStatus::Ok) {
String message;
LibJS: Support LegacyOctalEscapeSequence in string literals https://tc39.es/ecma262/#sec-additional-syntax-string-literals The syntax and semantics of 11.8.4 is extended as follows except that this extension is not allowed for strict mode code: Syntax EscapeSequence:: CharacterEscapeSequence LegacyOctalEscapeSequence NonOctalDecimalEscapeSequence HexEscapeSequence UnicodeEscapeSequence LegacyOctalEscapeSequence:: OctalDigit [lookahead ∉ OctalDigit] ZeroToThree OctalDigit [lookahead ∉ OctalDigit] FourToSeven OctalDigit ZeroToThree OctalDigit OctalDigit ZeroToThree :: one of 0 1 2 3 FourToSeven :: one of 4 5 6 7 NonOctalDecimalEscapeSequence :: one of 8 9 This definition of EscapeSequence is not used in strict mode or when parsing TemplateCharacter. Note It is possible for string literals to precede a Use Strict Directive that places the enclosing code in strict mode, and implementations must take care to not use this extended definition of EscapeSequence with such literals. For example, attempting to parse the following source text must fail: function invalid() { "\7"; "use strict"; }
2020-10-24 12:30:57 +00:00
if (status == Token::StringValueStatus::LegacyOctalEscapeSequence) {
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
m_state.string_legacy_octal_escape_sequence_in_scope = true;
LibJS: Support LegacyOctalEscapeSequence in string literals https://tc39.es/ecma262/#sec-additional-syntax-string-literals The syntax and semantics of 11.8.4 is extended as follows except that this extension is not allowed for strict mode code: Syntax EscapeSequence:: CharacterEscapeSequence LegacyOctalEscapeSequence NonOctalDecimalEscapeSequence HexEscapeSequence UnicodeEscapeSequence LegacyOctalEscapeSequence:: OctalDigit [lookahead ∉ OctalDigit] ZeroToThree OctalDigit [lookahead ∉ OctalDigit] FourToSeven OctalDigit ZeroToThree OctalDigit OctalDigit ZeroToThree :: one of 0 1 2 3 FourToSeven :: one of 4 5 6 7 NonOctalDecimalEscapeSequence :: one of 8 9 This definition of EscapeSequence is not used in strict mode or when parsing TemplateCharacter. Note It is possible for string literals to precede a Use Strict Directive that places the enclosing code in strict mode, and implementations must take care to not use this extended definition of EscapeSequence with such literals. For example, attempting to parse the following source text must fail: function invalid() { "\7"; "use strict"; }
2020-10-24 12:30:57 +00:00
if (in_template_literal)
message = "Octal escape sequence not allowed in template literal";
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
else if (m_state.strict_mode)
LibJS: Support LegacyOctalEscapeSequence in string literals https://tc39.es/ecma262/#sec-additional-syntax-string-literals The syntax and semantics of 11.8.4 is extended as follows except that this extension is not allowed for strict mode code: Syntax EscapeSequence:: CharacterEscapeSequence LegacyOctalEscapeSequence NonOctalDecimalEscapeSequence HexEscapeSequence UnicodeEscapeSequence LegacyOctalEscapeSequence:: OctalDigit [lookahead ∉ OctalDigit] ZeroToThree OctalDigit [lookahead ∉ OctalDigit] FourToSeven OctalDigit ZeroToThree OctalDigit OctalDigit ZeroToThree :: one of 0 1 2 3 FourToSeven :: one of 4 5 6 7 NonOctalDecimalEscapeSequence :: one of 8 9 This definition of EscapeSequence is not used in strict mode or when parsing TemplateCharacter. Note It is possible for string literals to precede a Use Strict Directive that places the enclosing code in strict mode, and implementations must take care to not use this extended definition of EscapeSequence with such literals. For example, attempting to parse the following source text must fail: function invalid() { "\7"; "use strict"; }
2020-10-24 12:30:57 +00:00
message = "Octal escape sequence in string literal not allowed in strict mode";
} else if (status == Token::StringValueStatus::MalformedHexEscape || status == Token::StringValueStatus::MalformedUnicodeEscape) {
LibJS: Handle hex and unicode escape sequences in string literals Introduces the following syntax: '\x55' '\u26a0' '\u{1f41e}'
2020-05-17 06:27:25 +00:00
auto type = status == Token::StringValueStatus::MalformedUnicodeEscape ? "unicode" : "hexadecimal";
LibJS: Use String::formatted() for parser error messages
2020-10-04 13:53:15 +00:00
message = String::formatted("Malformed {} escape sequence", type);
LibJS: Handle hex and unicode escape sequences in string literals Introduces the following syntax: '\x55' '\u26a0' '\u{1f41e}'
2020-05-17 06:27:25 +00:00
} else if (status == Token::StringValueStatus::UnicodeEscapeOverflow) {
Unicode: Try s/codepoint/code_point/g again This time, without trailing 's'. Ran: git grep -l 'codepoint' | xargs sed -ie 's/codepoint/code_point/g
2020-08-05 20:31:20 +00:00
message = "Unicode code_point must not be greater than 0x10ffff in escape sequence";
LibJS: Support LegacyOctalEscapeSequence in string literals https://tc39.es/ecma262/#sec-additional-syntax-string-literals The syntax and semantics of 11.8.4 is extended as follows except that this extension is not allowed for strict mode code: Syntax EscapeSequence:: CharacterEscapeSequence LegacyOctalEscapeSequence NonOctalDecimalEscapeSequence HexEscapeSequence UnicodeEscapeSequence LegacyOctalEscapeSequence:: OctalDigit [lookahead ∉ OctalDigit] ZeroToThree OctalDigit [lookahead ∉ OctalDigit] FourToSeven OctalDigit ZeroToThree OctalDigit OctalDigit ZeroToThree :: one of 0 1 2 3 FourToSeven :: one of 4 5 6 7 NonOctalDecimalEscapeSequence :: one of 8 9 This definition of EscapeSequence is not used in strict mode or when parsing TemplateCharacter. Note It is possible for string literals to precede a Use Strict Directive that places the enclosing code in strict mode, and implementations must take care to not use this extended definition of EscapeSequence with such literals. For example, attempting to parse the following source text must fail: function invalid() { "\7"; "use strict"; }
2020-10-24 12:30:57 +00:00
} else {
Everywhere: Rename ASSERT => VERIFY (...and ASSERT_NOT_REACHED => VERIFY_NOT_REACHED) Since all of these checks are done in release builds as well, let's rename them to VERIFY to prevent confusion, as everyone is used to assertions being compiled out in release. We can introduce a new ASSERT macro that is specifically for debug checks, but I'm doing this wholesale conversion first since we've accumulated thousands of these already, and it's not immediately obvious which ones are suitable for ASSERT.
2021-02-23 19:42:32 +00:00
VERIFY_NOT_REACHED();
LibJS: Handle hex and unicode escape sequences in string literals Introduces the following syntax: '\x55' '\u26a0' '\u{1f41e}'
2020-05-17 06:27:25 +00:00
}
LibJS: Report correct line/column for string literal syntax errors We're passing a token to this function, so m_current_token is actually the next token - which leads to incorrect line/column numbers for string literal syntax errors: "\u" ^ Uncaught exception: [SyntaxError]: Malformed unicode escape sequence (line: 1, column: 5) Rather than: "\u" ^ Uncaught exception: [SyntaxError]: Malformed unicode escape sequence (line: 1, column: 1)
2020-10-24 11:48:25 +00:00
if (!message.is_empty())
LibJS: Replace 'size_t line, size_t column' with 'Optional<Position>' This is a bit nicer for two reasons: - The absence of line number/column information isn't based on 'values are zero' anymore but on Optional's value - When reporting syntax errors with position information other than the current token's position we had to store line and column ourselves, like this: auto foo_start_line = m_parser_state.m_current_token.line_number(); auto foo_start_column = m_parser_state.m_current_token.line_column(); ... syntax_error("...", foo_start_line, foo_start_column); Which now becomes: auto foo_start= position(); ... syntax_error("...", foo_start); This makes it easier to report correct positions for syntax errors that only emerge a few tokens later :^)
2020-11-02 21:03:19 +00:00
syntax_error(message, Position { token.line_number(), token.line_column() });
LibJS: Handle hex and unicode escape sequences in string literals Introduces the following syntax: '\x55' '\u26a0' '\u{1f41e}'
2020-05-17 06:27:25 +00:00
}
LibJS: Add strict mode Adds the ability for a scope (either a function or the entire program) to be in strict mode. Scopes default to non-strict mode. There are two ways to determine the strict-ness of the JS engine: 1. In the parser, this can be accessed with the parser_state variable m_is_strict_mode boolean. If true, the Parser is currently parsing in strict mode. This is done so that the Parser can generate syntax errors at parse time, which is required in some cases. 2. With Interpreter.is_strict_mode(). This allows strict mode checking at runtime as opposed to compile time. Additionally, in order to test this, a global isStrictMode() function has been added to the JS ReplObject under the test-mode flag.
2020-05-28 05:22:08 +00:00
LibJS: Fix "use strict" directive false positives By having the "is this a use strict directive?" logic in parse_string_literal() we would apply it to *any* string literal, which is incorrect and would lead to false positives - e.g.: "use strict" + 1 `"use strict"` "\123"; ({"use strict": ...}) Relevant part from the spec which is now implemented properly: [...] and where each ExpressionStatement in the sequence consists entirely of a StringLiteral token [...] I also got rid of UseStrictDirectiveState which is not needed anymore. Fixes #3903.
2020-11-01 21:49:25 +00:00
auto is_use_strict_directive = !in_template_literal && (token.value() == "'use strict'" || token.value() == "\"use strict\"");
LibJS: Support LegacyOctalEscapeSequence in string literals https://tc39.es/ecma262/#sec-additional-syntax-string-literals The syntax and semantics of 11.8.4 is extended as follows except that this extension is not allowed for strict mode code: Syntax EscapeSequence:: CharacterEscapeSequence LegacyOctalEscapeSequence NonOctalDecimalEscapeSequence HexEscapeSequence UnicodeEscapeSequence LegacyOctalEscapeSequence:: OctalDigit [lookahead ∉ OctalDigit] ZeroToThree OctalDigit [lookahead ∉ OctalDigit] FourToSeven OctalDigit ZeroToThree OctalDigit OctalDigit ZeroToThree :: one of 0 1 2 3 FourToSeven :: one of 4 5 6 7 NonOctalDecimalEscapeSequence :: one of 8 9 This definition of EscapeSequence is not used in strict mode or when parsing TemplateCharacter. Note It is possible for string literals to precede a Use Strict Directive that places the enclosing code in strict mode, and implementations must take care to not use this extended definition of EscapeSequence with such literals. For example, attempting to parse the following source text must fail: function invalid() { "\7"; "use strict"; }
2020-10-24 12:30:57 +00:00
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
return create_ast_node<StringLiteral>({ m_state.current_token.filename(), rule_start.position(), position() }, string, is_use_strict_directive);
LibJS: Handle hex and unicode escape sequences in string literals Introduces the following syntax: '\x55' '\u26a0' '\u{1f41e}'
2020-05-17 06:27:25 +00:00
}
LibJS: Add raw strings to tagged template literals When calling a function with a tagged template, the first array that is passed in now contains a "raw" property with the raw, escaped strings.
2020-05-06 23:34:14 +00:00
NonnullRefPtr<TemplateLiteral> Parser::parse_template_literal(bool is_tagged)
LibJS: Add template literals Adds fully functioning template literals. Because template literals contain expressions, most of the work has to be done in the Lexer rather than the Parser. And because of the complexity of template literals (expressions, nesting, escapes, etc), the Lexer needs to have some template-related state. When entering a new template literal, a TemplateLiteralStart token is emitted. When inside a literal, all text will be parsed up until a '${' or '`' (or EOF, but that's a syntax error) is seen, and then a TemplateLiteralExprStart token is emitted. At this point, the Lexer proceeds as normal, however it keeps track of the number of opening and closing curly braces it has seen in order to determine the close of the expression. Once it finds a matching curly brace for the '${', a TemplateLiteralExprEnd token is emitted and the state is updated accordingly. When the Lexer is inside of a template literal, but not an expression, and sees a '`', this must be the closing grave: a TemplateLiteralEnd token is emitted. The state required to correctly parse template strings consists of a vector (for nesting) of two pieces of information: whether or not we are in a template expression (as opposed to a template string); and the count of the number of unmatched open curly braces we have seen (only applicable if the Lexer is currently in a template expression). TODO: Add support for template literal newlines in the JS REPL (this will cause a syntax error currently): > `foo > bar` 'foo bar'
2020-05-03 22:41:14 +00:00
{
LibJS: Track source positions all the way down to exceptions This makes exceptions have a trace of source positions too, which could probably be helpful in making fancier error tracebacks.
2020-12-28 17:15:22 +00:00
auto rule_start = push_start();
LibJS: Add template literals Adds fully functioning template literals. Because template literals contain expressions, most of the work has to be done in the Lexer rather than the Parser. And because of the complexity of template literals (expressions, nesting, escapes, etc), the Lexer needs to have some template-related state. When entering a new template literal, a TemplateLiteralStart token is emitted. When inside a literal, all text will be parsed up until a '${' or '`' (or EOF, but that's a syntax error) is seen, and then a TemplateLiteralExprStart token is emitted. At this point, the Lexer proceeds as normal, however it keeps track of the number of opening and closing curly braces it has seen in order to determine the close of the expression. Once it finds a matching curly brace for the '${', a TemplateLiteralExprEnd token is emitted and the state is updated accordingly. When the Lexer is inside of a template literal, but not an expression, and sees a '`', this must be the closing grave: a TemplateLiteralEnd token is emitted. The state required to correctly parse template strings consists of a vector (for nesting) of two pieces of information: whether or not we are in a template expression (as opposed to a template string); and the count of the number of unmatched open curly braces we have seen (only applicable if the Lexer is currently in a template expression). TODO: Add support for template literal newlines in the JS REPL (this will cause a syntax error currently): > `foo > bar` 'foo bar'
2020-05-03 22:41:14 +00:00
consume(TokenType::TemplateLiteralStart);
NonnullRefPtrVector<Expression> expressions;
LibJS: Add raw strings to tagged template literals When calling a function with a tagged template, the first array that is passed in now contains a "raw" property with the raw, escaped strings.
2020-05-06 23:34:14 +00:00
NonnullRefPtrVector<Expression> raw_strings;
LibJS: Track source positions all the way down to exceptions This makes exceptions have a trace of source positions too, which could probably be helpful in making fancier error tracebacks.
2020-12-28 17:15:22 +00:00
auto append_empty_string = [this, &rule_start, &expressions, &raw_strings, is_tagged]() {
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
auto string_literal = create_ast_node<StringLiteral>({ m_state.current_token.filename(), rule_start.position(), position() }, "");
LibJS: Add raw strings to tagged template literals When calling a function with a tagged template, the first array that is passed in now contains a "raw" property with the raw, escaped strings.
2020-05-06 23:34:14 +00:00
expressions.append(string_literal);
if (is_tagged)
raw_strings.append(string_literal);
};
LibJS: Add template literals Adds fully functioning template literals. Because template literals contain expressions, most of the work has to be done in the Lexer rather than the Parser. And because of the complexity of template literals (expressions, nesting, escapes, etc), the Lexer needs to have some template-related state. When entering a new template literal, a TemplateLiteralStart token is emitted. When inside a literal, all text will be parsed up until a '${' or '`' (or EOF, but that's a syntax error) is seen, and then a TemplateLiteralExprStart token is emitted. At this point, the Lexer proceeds as normal, however it keeps track of the number of opening and closing curly braces it has seen in order to determine the close of the expression. Once it finds a matching curly brace for the '${', a TemplateLiteralExprEnd token is emitted and the state is updated accordingly. When the Lexer is inside of a template literal, but not an expression, and sees a '`', this must be the closing grave: a TemplateLiteralEnd token is emitted. The state required to correctly parse template strings consists of a vector (for nesting) of two pieces of information: whether or not we are in a template expression (as opposed to a template string); and the count of the number of unmatched open curly braces we have seen (only applicable if the Lexer is currently in a template expression). TODO: Add support for template literal newlines in the JS REPL (this will cause a syntax error currently): > `foo > bar` 'foo bar'
2020-05-03 22:41:14 +00:00
LibJS: Implement tagged template literals (foo`bar`) To make processing tagged template literals easier, template literals will now add one empty StringLiteral before and after each template expression *if* there's no other string - e.g.: `${foo}` -> "", foo, "" `test${foo}${bar}test` -> "test", foo, "", bar, "test" This also matches the behaviour of many other parsers.
2020-05-06 09:17:35 +00:00
if (!match(TokenType::TemplateLiteralString))
LibJS: Add raw strings to tagged template literals When calling a function with a tagged template, the first array that is passed in now contains a "raw" property with the raw, escaped strings.
2020-05-06 23:34:14 +00:00
append_empty_string();
LibJS: Implement tagged template literals (foo`bar`) To make processing tagged template literals easier, template literals will now add one empty StringLiteral before and after each template expression *if* there's no other string - e.g.: `${foo}` -> "", foo, "" `test${foo}${bar}test` -> "test", foo, "", bar, "test" This also matches the behaviour of many other parsers.
2020-05-06 09:17:35 +00:00
LibJS: Fix Parser.parse_template_literal looping forever parse_template_literal now breaks out of the loop if it sees something it doesn't expect. Additionally, it now checks for EOFs.
2020-06-03 22:36:25 +00:00
while (!done() && !match(TokenType::TemplateLiteralEnd) && !match(TokenType::UnterminatedTemplateLiteral)) {
LibJS: Add template literals Adds fully functioning template literals. Because template literals contain expressions, most of the work has to be done in the Lexer rather than the Parser. And because of the complexity of template literals (expressions, nesting, escapes, etc), the Lexer needs to have some template-related state. When entering a new template literal, a TemplateLiteralStart token is emitted. When inside a literal, all text will be parsed up until a '${' or '`' (or EOF, but that's a syntax error) is seen, and then a TemplateLiteralExprStart token is emitted. At this point, the Lexer proceeds as normal, however it keeps track of the number of opening and closing curly braces it has seen in order to determine the close of the expression. Once it finds a matching curly brace for the '${', a TemplateLiteralExprEnd token is emitted and the state is updated accordingly. When the Lexer is inside of a template literal, but not an expression, and sees a '`', this must be the closing grave: a TemplateLiteralEnd token is emitted. The state required to correctly parse template strings consists of a vector (for nesting) of two pieces of information: whether or not we are in a template expression (as opposed to a template string); and the count of the number of unmatched open curly braces we have seen (only applicable if the Lexer is currently in a template expression). TODO: Add support for template literal newlines in the JS REPL (this will cause a syntax error currently): > `foo > bar` 'foo bar'
2020-05-03 22:41:14 +00:00
if (match(TokenType::TemplateLiteralString)) {
LibJS: Add raw strings to tagged template literals When calling a function with a tagged template, the first array that is passed in now contains a "raw" property with the raw, escaped strings.
2020-05-06 23:34:14 +00:00
auto token = consume();
LibJS: Support LegacyOctalEscapeSequence in string literals https://tc39.es/ecma262/#sec-additional-syntax-string-literals The syntax and semantics of 11.8.4 is extended as follows except that this extension is not allowed for strict mode code: Syntax EscapeSequence:: CharacterEscapeSequence LegacyOctalEscapeSequence NonOctalDecimalEscapeSequence HexEscapeSequence UnicodeEscapeSequence LegacyOctalEscapeSequence:: OctalDigit [lookahead ∉ OctalDigit] ZeroToThree OctalDigit [lookahead ∉ OctalDigit] FourToSeven OctalDigit ZeroToThree OctalDigit OctalDigit ZeroToThree :: one of 0 1 2 3 FourToSeven :: one of 4 5 6 7 NonOctalDecimalEscapeSequence :: one of 8 9 This definition of EscapeSequence is not used in strict mode or when parsing TemplateCharacter. Note It is possible for string literals to precede a Use Strict Directive that places the enclosing code in strict mode, and implementations must take care to not use this extended definition of EscapeSequence with such literals. For example, attempting to parse the following source text must fail: function invalid() { "\7"; "use strict"; }
2020-10-24 12:30:57 +00:00
expressions.append(parse_string_literal(token, true));
LibJS: Add raw strings to tagged template literals When calling a function with a tagged template, the first array that is passed in now contains a "raw" property with the raw, escaped strings.
2020-05-06 23:34:14 +00:00
if (is_tagged)
LibJS: Fix that windows style line endings were not ignored or converted These are tested by test262 but the current test262-runner reads the files in python which automatically converts \r\n to \n. This meant that we passed the tests while we should not have.
2021-09-01 16:34:19 +00:00
raw_strings.append(create_ast_node<StringLiteral>({ m_state.current_token.filename(), rule_start.position(), position() }, token.raw_template_value()));
LibJS: Add template literals Adds fully functioning template literals. Because template literals contain expressions, most of the work has to be done in the Lexer rather than the Parser. And because of the complexity of template literals (expressions, nesting, escapes, etc), the Lexer needs to have some template-related state. When entering a new template literal, a TemplateLiteralStart token is emitted. When inside a literal, all text will be parsed up until a '${' or '`' (or EOF, but that's a syntax error) is seen, and then a TemplateLiteralExprStart token is emitted. At this point, the Lexer proceeds as normal, however it keeps track of the number of opening and closing curly braces it has seen in order to determine the close of the expression. Once it finds a matching curly brace for the '${', a TemplateLiteralExprEnd token is emitted and the state is updated accordingly. When the Lexer is inside of a template literal, but not an expression, and sees a '`', this must be the closing grave: a TemplateLiteralEnd token is emitted. The state required to correctly parse template strings consists of a vector (for nesting) of two pieces of information: whether or not we are in a template expression (as opposed to a template string); and the count of the number of unmatched open curly braces we have seen (only applicable if the Lexer is currently in a template expression). TODO: Add support for template literal newlines in the JS REPL (this will cause a syntax error currently): > `foo > bar` 'foo bar'
2020-05-03 22:41:14 +00:00
} else if (match(TokenType::TemplateLiteralExprStart)) {
consume(TokenType::TemplateLiteralExprStart);
if (match(TokenType::TemplateLiteralExprEnd)) {
syntax_error("Empty template literal expression block");
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
return create_ast_node<TemplateLiteral>({ m_state.current_token.filename(), rule_start.position(), position() }, expressions);
LibJS: Add template literals Adds fully functioning template literals. Because template literals contain expressions, most of the work has to be done in the Lexer rather than the Parser. And because of the complexity of template literals (expressions, nesting, escapes, etc), the Lexer needs to have some template-related state. When entering a new template literal, a TemplateLiteralStart token is emitted. When inside a literal, all text will be parsed up until a '${' or '`' (or EOF, but that's a syntax error) is seen, and then a TemplateLiteralExprStart token is emitted. At this point, the Lexer proceeds as normal, however it keeps track of the number of opening and closing curly braces it has seen in order to determine the close of the expression. Once it finds a matching curly brace for the '${', a TemplateLiteralExprEnd token is emitted and the state is updated accordingly. When the Lexer is inside of a template literal, but not an expression, and sees a '`', this must be the closing grave: a TemplateLiteralEnd token is emitted. The state required to correctly parse template strings consists of a vector (for nesting) of two pieces of information: whether or not we are in a template expression (as opposed to a template string); and the count of the number of unmatched open curly braces we have seen (only applicable if the Lexer is currently in a template expression). TODO: Add support for template literal newlines in the JS REPL (this will cause a syntax error currently): > `foo > bar` 'foo bar'
2020-05-03 22:41:14 +00:00
}
expressions.append(parse_expression(0));
if (match(TokenType::UnterminatedTemplateLiteral)) {
syntax_error("Unterminated template literal");
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
return create_ast_node<TemplateLiteral>({ m_state.current_token.filename(), rule_start.position(), position() }, expressions);
LibJS: Add template literals Adds fully functioning template literals. Because template literals contain expressions, most of the work has to be done in the Lexer rather than the Parser. And because of the complexity of template literals (expressions, nesting, escapes, etc), the Lexer needs to have some template-related state. When entering a new template literal, a TemplateLiteralStart token is emitted. When inside a literal, all text will be parsed up until a '${' or '`' (or EOF, but that's a syntax error) is seen, and then a TemplateLiteralExprStart token is emitted. At this point, the Lexer proceeds as normal, however it keeps track of the number of opening and closing curly braces it has seen in order to determine the close of the expression. Once it finds a matching curly brace for the '${', a TemplateLiteralExprEnd token is emitted and the state is updated accordingly. When the Lexer is inside of a template literal, but not an expression, and sees a '`', this must be the closing grave: a TemplateLiteralEnd token is emitted. The state required to correctly parse template strings consists of a vector (for nesting) of two pieces of information: whether or not we are in a template expression (as opposed to a template string); and the count of the number of unmatched open curly braces we have seen (only applicable if the Lexer is currently in a template expression). TODO: Add support for template literal newlines in the JS REPL (this will cause a syntax error currently): > `foo > bar` 'foo bar'
2020-05-03 22:41:14 +00:00
}
consume(TokenType::TemplateLiteralExprEnd);
LibJS: Implement tagged template literals (foo`bar`) To make processing tagged template literals easier, template literals will now add one empty StringLiteral before and after each template expression *if* there's no other string - e.g.: `${foo}` -> "", foo, "" `test${foo}${bar}test` -> "test", foo, "", bar, "test" This also matches the behaviour of many other parsers.
2020-05-06 09:17:35 +00:00
if (!match(TokenType::TemplateLiteralString))
LibJS: Add raw strings to tagged template literals When calling a function with a tagged template, the first array that is passed in now contains a "raw" property with the raw, escaped strings.
2020-05-06 23:34:14 +00:00
append_empty_string();
LibJS: Fix Parser.parse_template_literal looping forever parse_template_literal now breaks out of the loop if it sees something it doesn't expect. Additionally, it now checks for EOFs.
2020-06-03 22:36:25 +00:00
} else {
expected("Template literal string or expression");
break;
LibJS: Add template literals Adds fully functioning template literals. Because template literals contain expressions, most of the work has to be done in the Lexer rather than the Parser. And because of the complexity of template literals (expressions, nesting, escapes, etc), the Lexer needs to have some template-related state. When entering a new template literal, a TemplateLiteralStart token is emitted. When inside a literal, all text will be parsed up until a '${' or '`' (or EOF, but that's a syntax error) is seen, and then a TemplateLiteralExprStart token is emitted. At this point, the Lexer proceeds as normal, however it keeps track of the number of opening and closing curly braces it has seen in order to determine the close of the expression. Once it finds a matching curly brace for the '${', a TemplateLiteralExprEnd token is emitted and the state is updated accordingly. When the Lexer is inside of a template literal, but not an expression, and sees a '`', this must be the closing grave: a TemplateLiteralEnd token is emitted. The state required to correctly parse template strings consists of a vector (for nesting) of two pieces of information: whether or not we are in a template expression (as opposed to a template string); and the count of the number of unmatched open curly braces we have seen (only applicable if the Lexer is currently in a template expression). TODO: Add support for template literal newlines in the JS REPL (this will cause a syntax error currently): > `foo > bar` 'foo bar'
2020-05-03 22:41:14 +00:00
}
}
if (match(TokenType::UnterminatedTemplateLiteral)) {
syntax_error("Unterminated template literal");
} else {
consume(TokenType::TemplateLiteralEnd);
}
LibJS: Add raw strings to tagged template literals When calling a function with a tagged template, the first array that is passed in now contains a "raw" property with the raw, escaped strings.
2020-05-06 23:34:14 +00:00
if (is_tagged)
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
return create_ast_node<TemplateLiteral>({ m_state.current_token.filename(), rule_start.position(), position() }, expressions, raw_strings);
return create_ast_node<TemplateLiteral>({ m_state.current_token.filename(), rule_start.position(), position() }, expressions);
LibJS: Add template literals Adds fully functioning template literals. Because template literals contain expressions, most of the work has to be done in the Lexer rather than the Parser. And because of the complexity of template literals (expressions, nesting, escapes, etc), the Lexer needs to have some template-related state. When entering a new template literal, a TemplateLiteralStart token is emitted. When inside a literal, all text will be parsed up until a '${' or '`' (or EOF, but that's a syntax error) is seen, and then a TemplateLiteralExprStart token is emitted. At this point, the Lexer proceeds as normal, however it keeps track of the number of opening and closing curly braces it has seen in order to determine the close of the expression. Once it finds a matching curly brace for the '${', a TemplateLiteralExprEnd token is emitted and the state is updated accordingly. When the Lexer is inside of a template literal, but not an expression, and sees a '`', this must be the closing grave: a TemplateLiteralEnd token is emitted. The state required to correctly parse template strings consists of a vector (for nesting) of two pieces of information: whether or not we are in a template expression (as opposed to a template string); and the count of the number of unmatched open curly braces we have seen (only applicable if the Lexer is currently in a template expression). TODO: Add support for template literal newlines in the JS REPL (this will cause a syntax error currently): > `foo > bar` 'foo bar'
2020-05-03 22:41:14 +00:00
}
LibJS: Use const references to avoid some copies in the parser
2021-02-24 08:58:24 +00:00
NonnullRefPtr<Expression> Parser::parse_expression(int min_precedence, Associativity associativity, const Vector<TokenType>& forbidden)
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
{
LibJS: Track source positions all the way down to exceptions This makes exceptions have a trace of source positions too, which could probably be helpful in making fancier error tracebacks.
2020-12-28 17:15:22 +00:00
auto rule_start = push_start();
LibJS: Correctly parse yield-from expressions This commit implements parsing for `yield *expr`, and the multiple ways something can or can't be parsed like that. Also makes yield-from a TODO in the bytecode generator. Behold, the glory of javascript syntax: ```js // 'yield' = expression in generators. function* foo() { yield *bar; // <- Syntax error here, expression can't start with * } // 'yield' = identifier anywhere else. function foo() { yield *bar; // Perfectly fine, this is just `yield * bar` } ```
2021-06-14 11:16:41 +00:00
auto [expression, should_continue_parsing] = parse_primary_expression();
LibJS: Allow 'name = value' in object literals as the spec does Currently, these are _always_ a syntax error, future commits will make it valid in certain contexts.
2021-07-11 10:18:30 +00:00
auto check_for_invalid_object_property = [&](auto& expression) {
if (is<ObjectExpression>(*expression)) {
if (auto range = static_cast<ObjectExpression&>(*expression).invalid_property_range(); range.has_value())
syntax_error("Invalid property in object literal", range->start);
}
};
LibJS: Make accessing the current function's arguments cheaper Instead of going through an environment record, make arguments of the currently executing function generate references via the argument index, which can later be resolved directly through the ExecutionContext.
2021-10-07 16:43:22 +00:00
if (is<Identifier>(*expression) && m_state.current_scope_pusher) {
auto identifier_instance = static_ptr_cast<Identifier>(expression);
auto function_scope = m_state.current_scope_pusher->last_function_scope();
auto function_parent_scope = function_scope ? function_scope->parent_scope() : nullptr;
bool has_not_been_declared_as_variable = true;
for (auto scope = m_state.current_scope_pusher; scope != function_parent_scope; scope = scope->parent_scope()) {
if (scope->has_declaration(identifier_instance->string())) {
has_not_been_declared_as_variable = false;
break;
}
}
if (has_not_been_declared_as_variable) {
if (identifier_instance->string() == "arguments"sv)
m_state.current_scope_pusher->set_contains_access_to_arguments_object();
}
}
LibJS: Implement tagged template literals (foo`bar`) To make processing tagged template literals easier, template literals will now add one empty StringLiteral before and after each template expression *if* there's no other string - e.g.: `${foo}` -> "", foo, "" `test${foo}${bar}test` -> "test", foo, "", bar, "test" This also matches the behaviour of many other parsers.
2020-05-06 09:17:35 +00:00
while (match(TokenType::TemplateLiteralStart)) {
LibJS: Add raw strings to tagged template literals When calling a function with a tagged template, the first array that is passed in now contains a "raw" property with the raw, escaped strings.
2020-05-06 23:34:14 +00:00
auto template_literal = parse_template_literal(true);
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
expression = create_ast_node<TaggedTemplateLiteral>({ m_state.current_token.filename(), rule_start.position(), position() }, move(expression), move(template_literal));
LibJS: Implement tagged template literals (foo`bar`) To make processing tagged template literals easier, template literals will now add one empty StringLiteral before and after each template expression *if* there's no other string - e.g.: `${foo}` -> "", foo, "" `test${foo}${bar}test` -> "test", foo, "", bar, "test" This also matches the behaviour of many other parsers.
2020-05-06 09:17:35 +00:00
}
LibJS: Correctly parse yield-from expressions This commit implements parsing for `yield *expr`, and the multiple ways something can or can't be parsed like that. Also makes yield-from a TODO in the bytecode generator. Behold, the glory of javascript syntax: ```js // 'yield' = expression in generators. function* foo() { yield *bar; // <- Syntax error here, expression can't start with * } // 'yield' = identifier anywhere else. function foo() { yield *bar; // Perfectly fine, this is just `yield * bar` } ```
2021-06-14 11:16:41 +00:00
if (should_continue_parsing) {
while (match_secondary_expression(forbidden)) {
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
int new_precedence = g_operator_precedence.get(m_state.current_token.type());
LibJS: Correctly parse yield-from expressions This commit implements parsing for `yield *expr`, and the multiple ways something can or can't be parsed like that. Also makes yield-from a TODO in the bytecode generator. Behold, the glory of javascript syntax: ```js // 'yield' = expression in generators. function* foo() { yield *bar; // <- Syntax error here, expression can't start with * } // 'yield' = identifier anywhere else. function foo() { yield *bar; // Perfectly fine, this is just `yield * bar` } ```
2021-06-14 11:16:41 +00:00
if (new_precedence < min_precedence)
break;
if (new_precedence == min_precedence && associativity == Associativity::Left)
break;
LibJS: Allow 'name = value' in object literals as the spec does Currently, these are _always_ a syntax error, future commits will make it valid in certain contexts.
2021-07-11 10:18:30 +00:00
check_for_invalid_object_property(expression);
LibJS: Add operator precedence parsing Obey precedence and associativity rules when parsing expressions with chained operators.
2020-03-12 22:02:41 +00:00
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
Associativity new_associativity = operator_associativity(m_state.current_token.type());
LibJS: Correctly parse yield-from expressions This commit implements parsing for `yield *expr`, and the multiple ways something can or can't be parsed like that. Also makes yield-from a TODO in the bytecode generator. Behold, the glory of javascript syntax: ```js // 'yield' = expression in generators. function* foo() { yield *bar; // <- Syntax error here, expression can't start with * } // 'yield' = identifier anywhere else. function foo() { yield *bar; // Perfectly fine, this is just `yield * bar` } ```
2021-06-14 11:16:41 +00:00
expression = parse_secondary_expression(move(expression), new_precedence, new_associativity);
LibJS: Handle '++' and '--' more correctly within expression
2021-08-14 15:02:15 +00:00
while (match(TokenType::TemplateLiteralStart) && !is<UpdateExpression>(*expression)) {
LibJS: Correctly parse yield-from expressions This commit implements parsing for `yield *expr`, and the multiple ways something can or can't be parsed like that. Also makes yield-from a TODO in the bytecode generator. Behold, the glory of javascript syntax: ```js // 'yield' = expression in generators. function* foo() { yield *bar; // <- Syntax error here, expression can't start with * } // 'yield' = identifier anywhere else. function foo() { yield *bar; // Perfectly fine, this is just `yield * bar` } ```
2021-06-14 11:16:41 +00:00
auto template_literal = parse_template_literal(true);
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
expression = create_ast_node<TaggedTemplateLiteral>({ m_state.current_token.filename(), rule_start.position(), position() }, move(expression), move(template_literal));
LibJS: Correctly parse yield-from expressions This commit implements parsing for `yield *expr`, and the multiple ways something can or can't be parsed like that. Also makes yield-from a TODO in the bytecode generator. Behold, the glory of javascript syntax: ```js // 'yield' = expression in generators. function* foo() { yield *bar; // <- Syntax error here, expression can't start with * } // 'yield' = identifier anywhere else. function foo() { yield *bar; // Perfectly fine, this is just `yield * bar` } ```
2021-06-14 11:16:41 +00:00
}
LibJS: Implement tagged template literals (foo`bar`) To make processing tagged template literals easier, template literals will now add one empty StringLiteral before and after each template expression *if* there's no other string - e.g.: `${foo}` -> "", foo, "" `test${foo}${bar}test` -> "test", foo, "", bar, "test" This also matches the behaviour of many other parsers.
2020-05-06 09:17:35 +00:00
}
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
}
LibJS: Allow 'name = value' in object literals as the spec does Currently, these are _always_ a syntax error, future commits will make it valid in certain contexts.
2021-07-11 10:18:30 +00:00
LibJS: Disallow standalone super expression
2021-08-14 15:08:09 +00:00
if (is<SuperExpression>(*expression))
syntax_error("'super' keyword unexpected here");
LibJS: Allow 'name = value' in object literals as the spec does Currently, these are _always_ a syntax error, future commits will make it valid in certain contexts.
2021-07-11 10:18:30 +00:00
check_for_invalid_object_property(expression);
LibJS: Make accessing the current function's arguments cheaper Instead of going through an environment record, make arguments of the currently executing function generate references via the argument index, which can later be resolved directly through the ExecutionContext.
2021-10-07 16:43:22 +00:00
if (is<CallExpression>(*expression) && m_state.current_scope_pusher) {
auto& callee = static_ptr_cast<CallExpression>(expression)->callee();
if (is<Identifier>(callee)) {
auto& identifier_instance = static_cast<Identifier const&>(callee);
if (identifier_instance.string() == "eval"sv) {
bool has_not_been_declared_as_variable = true;
for (auto scope = m_state.current_scope_pusher; scope; scope = scope->parent_scope()) {
if (scope->has_declaration(identifier_instance.string())) {
has_not_been_declared_as_variable = false;
break;
}
}
if (has_not_been_declared_as_variable)
m_state.current_scope_pusher->set_contains_direct_call_to_eval();
}
}
}
LibJS: Parse comma operator into SequenceExpression
2020-05-11 16:27:31 +00:00
if (match(TokenType::Comma) && min_precedence <= 1) {
NonnullRefPtrVector<Expression> expressions;
expressions.append(expression);
while (match(TokenType::Comma)) {
consume();
expressions.append(parse_expression(2));
}
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
expression = create_ast_node<SequenceExpression>({ m_state.current_token.filename(), rule_start.position(), position() }, move(expressions));
LibJS: Parse comma operator into SequenceExpression
2020-05-11 16:27:31 +00:00
}
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
return expression;
}
LibJS: Make the AST reference-counted This allows function objects to outlive the original parsed program without their ScopeNode disappearing.
2020-03-18 10:23:53 +00:00
NonnullRefPtr<Expression> Parser::parse_secondary_expression(NonnullRefPtr<Expression> lhs, int min_precedence, Associativity associativity)
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
{
LibJS: Track source positions all the way down to exceptions This makes exceptions have a trace of source positions too, which could probably be helpful in making fancier error tracebacks.
2020-12-28 17:15:22 +00:00
auto rule_start = push_start();
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
switch (m_state.current_token.type()) {
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
case TokenType::Plus:
consume();
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
return create_ast_node<BinaryExpression>({ m_state.current_token.filename(), rule_start.position(), position() }, BinaryOp::Addition, move(lhs), parse_expression(min_precedence, associativity));
LibJS: Implement +=, -=, *=, and /= assignment operators
2020-03-12 12:09:15 +00:00
case TokenType::PlusEquals:
LibJS: Validate all assignment expressions, not just "=" The check for invalid lhs and assignment to eval/arguments in strict mode should happen for all kinds of assignment expressions, not just AssignmentOp::Assignment.
2020-10-04 22:58:57 +00:00
return parse_assignment_expression(AssignmentOp::AdditionAssignment, move(lhs), min_precedence, associativity);
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
case TokenType::Minus:
consume();
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
return create_ast_node<BinaryExpression>({ m_state.current_token.filename(), rule_start.position(), position() }, BinaryOp::Subtraction, move(lhs), parse_expression(min_precedence, associativity));
LibJS: Implement +=, -=, *=, and /= assignment operators
2020-03-12 12:09:15 +00:00
case TokenType::MinusEquals:
LibJS: Validate all assignment expressions, not just "=" The check for invalid lhs and assignment to eval/arguments in strict mode should happen for all kinds of assignment expressions, not just AssignmentOp::Assignment.
2020-10-04 22:58:57 +00:00
return parse_assignment_expression(AssignmentOp::SubtractionAssignment, move(lhs), min_precedence, associativity);
LibJS: Implement multiplication and division operators
2020-03-12 12:04:52 +00:00
case TokenType::Asterisk:
consume();
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
return create_ast_node<BinaryExpression>({ m_state.current_token.filename(), rule_start.position(), position() }, BinaryOp::Multiplication, move(lhs), parse_expression(min_precedence, associativity));
LibJS: Implement +=, -=, *=, and /= assignment operators
2020-03-12 12:09:15 +00:00
case TokenType::AsteriskEquals:
LibJS: Validate all assignment expressions, not just "=" The check for invalid lhs and assignment to eval/arguments in strict mode should happen for all kinds of assignment expressions, not just AssignmentOp::Assignment.
2020-10-04 22:58:57 +00:00
return parse_assignment_expression(AssignmentOp::MultiplicationAssignment, move(lhs), min_precedence, associativity);
LibJS: Implement multiplication and division operators
2020-03-12 12:04:52 +00:00
case TokenType::Slash:
consume();
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
return create_ast_node<BinaryExpression>({ m_state.current_token.filename(), rule_start.position(), position() }, BinaryOp::Division, move(lhs), parse_expression(min_precedence, associativity));
LibJS: Implement +=, -=, *=, and /= assignment operators
2020-03-12 12:09:15 +00:00
case TokenType::SlashEquals:
LibJS: Validate all assignment expressions, not just "=" The check for invalid lhs and assignment to eval/arguments in strict mode should happen for all kinds of assignment expressions, not just AssignmentOp::Assignment.
2020-10-04 22:58:57 +00:00
return parse_assignment_expression(AssignmentOp::DivisionAssignment, move(lhs), min_precedence, associativity);
LibJS: Add basic support for modulo (%) in binary expressions
2020-04-04 19:17:34 +00:00
case TokenType::Percent:
consume();
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
return create_ast_node<BinaryExpression>({ m_state.current_token.filename(), rule_start.position(), position() }, BinaryOp::Modulo, move(lhs), parse_expression(min_precedence, associativity));
LibJS: Implement modulo assignment operator (%=)
2020-05-04 22:07:05 +00:00
case TokenType::PercentEquals:
LibJS: Validate all assignment expressions, not just "=" The check for invalid lhs and assignment to eval/arguments in strict mode should happen for all kinds of assignment expressions, not just AssignmentOp::Assignment.
2020-10-04 22:58:57 +00:00
return parse_assignment_expression(AssignmentOp::ModuloAssignment, move(lhs), min_precedence, associativity);
LibJS: Parse "this" as ThisExpression
2020-04-12 22:42:14 +00:00
case TokenType::DoubleAsterisk:
LibJS: Implement exponentiation (** operator)
2020-04-05 12:40:00 +00:00
consume();
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
return create_ast_node<BinaryExpression>({ m_state.current_token.filename(), rule_start.position(), position() }, BinaryOp::Exponentiation, move(lhs), parse_expression(min_precedence, associativity));
LibJS: Implement exponentiation assignment operator (**=)
2020-05-04 22:03:35 +00:00
case TokenType::DoubleAsteriskEquals:
LibJS: Validate all assignment expressions, not just "=" The check for invalid lhs and assignment to eval/arguments in strict mode should happen for all kinds of assignment expressions, not just AssignmentOp::Assignment.
2020-10-04 22:58:57 +00:00
return parse_assignment_expression(AssignmentOp::ExponentiationAssignment, move(lhs), min_precedence, associativity);
LibJS: Parse > and < binary operators
2020-03-12 12:10:27 +00:00
case TokenType::GreaterThan:
consume();
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
return create_ast_node<BinaryExpression>({ m_state.current_token.filename(), rule_start.position(), position() }, BinaryOp::GreaterThan, move(lhs), parse_expression(min_precedence, associativity));
LibJS: Implement <= and >= binary operators
2020-03-12 12:07:08 +00:00
case TokenType::GreaterThanEquals:
consume();
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
return create_ast_node<BinaryExpression>({ m_state.current_token.filename(), rule_start.position(), position() }, BinaryOp::GreaterThanEquals, move(lhs), parse_expression(min_precedence, associativity));
LibJS: Parse > and < binary operators
2020-03-12 12:10:27 +00:00
case TokenType::LessThan:
consume();
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
return create_ast_node<BinaryExpression>({ m_state.current_token.filename(), rule_start.position(), position() }, BinaryOp::LessThan, move(lhs), parse_expression(min_precedence, associativity));
LibJS: Implement <= and >= binary operators
2020-03-12 12:07:08 +00:00
case TokenType::LessThanEquals:
consume();
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
return create_ast_node<BinaryExpression>({ m_state.current_token.filename(), rule_start.position(), position() }, BinaryOp::LessThanEquals, move(lhs), parse_expression(min_precedence, associativity));
LibJS: Parse === and !== binary operators
2020-03-12 12:11:33 +00:00
case TokenType::EqualsEqualsEquals:
consume();
LibJS: Rename {Abstract,Typed => Loosely,Strictly}{Equals,Inequals} This affects the AST's BinaryOp enum as well as the Bytecode's ENUMERATE_BYTECODE_OPS and JS_ENUMERATE_COMMON_BINARY_OPS macros.
2021-09-23 22:06:10 +00:00
return create_ast_node<BinaryExpression>({ m_state.current_token.filename(), rule_start.position(), position() }, BinaryOp::StrictlyEquals, move(lhs), parse_expression(min_precedence, associativity));
LibJS: Parse === and !== binary operators
2020-03-12 12:11:33 +00:00
case TokenType::ExclamationMarkEqualsEquals:
consume();
LibJS: Rename {Abstract,Typed => Loosely,Strictly}{Equals,Inequals} This affects the AST's BinaryOp enum as well as the Bytecode's ENUMERATE_BYTECODE_OPS and JS_ENUMERATE_COMMON_BINARY_OPS macros.
2021-09-23 22:06:10 +00:00
return create_ast_node<BinaryExpression>({ m_state.current_token.filename(), rule_start.position(), position() }, BinaryOp::StrictlyInequals, move(lhs), parse_expression(min_precedence, associativity));
LibJS: Implement abstract equality and inequality
2020-03-15 22:23:38 +00:00
case TokenType::EqualsEquals:
consume();
LibJS: Rename {Abstract,Typed => Loosely,Strictly}{Equals,Inequals} This affects the AST's BinaryOp enum as well as the Bytecode's ENUMERATE_BYTECODE_OPS and JS_ENUMERATE_COMMON_BINARY_OPS macros.
2021-09-23 22:06:10 +00:00
return create_ast_node<BinaryExpression>({ m_state.current_token.filename(), rule_start.position(), position() }, BinaryOp::LooselyEquals, move(lhs), parse_expression(min_precedence, associativity));
LibJS: Implement abstract equality and inequality
2020-03-15 22:23:38 +00:00
case TokenType::ExclamationMarkEquals:
consume();
LibJS: Rename {Abstract,Typed => Loosely,Strictly}{Equals,Inequals} This affects the AST's BinaryOp enum as well as the Bytecode's ENUMERATE_BYTECODE_OPS and JS_ENUMERATE_COMMON_BINARY_OPS macros.
2021-09-23 22:06:10 +00:00
return create_ast_node<BinaryExpression>({ m_state.current_token.filename(), rule_start.position(), position() }, BinaryOp::LooselyInequals, move(lhs), parse_expression(min_precedence, associativity));
LibJS: Implement 'in' operator
2020-04-23 15:06:01 +00:00
case TokenType::In:
consume();
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
return create_ast_node<BinaryExpression>({ m_state.current_token.filename(), rule_start.position(), position() }, BinaryOp::In, move(lhs), parse_expression(min_precedence, associativity));
LibJS: Implement the "instanceof" operator This operator walks the prototype chain of the RHS value and looks for a "prototype" property with the same value as the prototype of the LHS. This is pretty cool. :^)
2020-03-28 15:56:54 +00:00
case TokenType::Instanceof:
consume();
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
return create_ast_node<BinaryExpression>({ m_state.current_token.filename(), rule_start.position(), position() }, BinaryOp::InstanceOf, move(lhs), parse_expression(min_precedence, associativity));
LibJS: Parse binary bitwise operators The Lexer and AST already have all the functionality required in place, so this is just updating Parser::match_secondary_expression() and Parser::parse_expression() to handle TokenType::{Ampersand,Pipe,Caret}, as well as adding some tests.
2020-04-03 12:02:31 +00:00
case TokenType::Ampersand:
consume();
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
return create_ast_node<BinaryExpression>({ m_state.current_token.filename(), rule_start.position(), position() }, BinaryOp::BitwiseAnd, move(lhs), parse_expression(min_precedence, associativity));
LibJS: Implement bitwise assignment operators (&=, |=, ^=)
2020-05-04 21:34:45 +00:00
case TokenType::AmpersandEquals:
LibJS: Validate all assignment expressions, not just "=" The check for invalid lhs and assignment to eval/arguments in strict mode should happen for all kinds of assignment expressions, not just AssignmentOp::Assignment.
2020-10-04 22:58:57 +00:00
return parse_assignment_expression(AssignmentOp::BitwiseAndAssignment, move(lhs), min_precedence, associativity);
LibJS: Parse binary bitwise operators The Lexer and AST already have all the functionality required in place, so this is just updating Parser::match_secondary_expression() and Parser::parse_expression() to handle TokenType::{Ampersand,Pipe,Caret}, as well as adding some tests.
2020-04-03 12:02:31 +00:00
case TokenType::Pipe:
consume();
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
return create_ast_node<BinaryExpression>({ m_state.current_token.filename(), rule_start.position(), position() }, BinaryOp::BitwiseOr, move(lhs), parse_expression(min_precedence, associativity));
LibJS: Implement bitwise assignment operators (&=, |=, ^=)
2020-05-04 21:34:45 +00:00
case TokenType::PipeEquals:
LibJS: Validate all assignment expressions, not just "=" The check for invalid lhs and assignment to eval/arguments in strict mode should happen for all kinds of assignment expressions, not just AssignmentOp::Assignment.
2020-10-04 22:58:57 +00:00
return parse_assignment_expression(AssignmentOp::BitwiseOrAssignment, move(lhs), min_precedence, associativity);
LibJS: Parse binary bitwise operators The Lexer and AST already have all the functionality required in place, so this is just updating Parser::match_secondary_expression() and Parser::parse_expression() to handle TokenType::{Ampersand,Pipe,Caret}, as well as adding some tests.
2020-04-03 12:02:31 +00:00
case TokenType::Caret:
consume();
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
return create_ast_node<BinaryExpression>({ m_state.current_token.filename(), rule_start.position(), position() }, BinaryOp::BitwiseXor, move(lhs), parse_expression(min_precedence, associativity));
LibJS: Implement bitwise assignment operators (&=, |=, ^=)
2020-05-04 21:34:45 +00:00
case TokenType::CaretEquals:
LibJS: Validate all assignment expressions, not just "=" The check for invalid lhs and assignment to eval/arguments in strict mode should happen for all kinds of assignment expressions, not just AssignmentOp::Assignment.
2020-10-04 22:58:57 +00:00
return parse_assignment_expression(AssignmentOp::BitwiseXorAssignment, move(lhs), min_precedence, associativity);
LibJS: Implement bitwise left shift operator (<<)
2020-04-23 12:36:14 +00:00
case TokenType::ShiftLeft:
consume();
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
return create_ast_node<BinaryExpression>({ m_state.current_token.filename(), rule_start.position(), position() }, BinaryOp::LeftShift, move(lhs), parse_expression(min_precedence, associativity));
LibJS: Implement bitwise left shift operator (<<)
2020-04-23 12:36:14 +00:00
case TokenType::ShiftLeftEquals:
LibJS: Validate all assignment expressions, not just "=" The check for invalid lhs and assignment to eval/arguments in strict mode should happen for all kinds of assignment expressions, not just AssignmentOp::Assignment.
2020-10-04 22:58:57 +00:00
return parse_assignment_expression(AssignmentOp::LeftShiftAssignment, move(lhs), min_precedence, associativity);
LibJS: Implement bitwise right shift operator (>>)
2020-04-23 12:45:19 +00:00
case TokenType::ShiftRight:
consume();
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
return create_ast_node<BinaryExpression>({ m_state.current_token.filename(), rule_start.position(), position() }, BinaryOp::RightShift, move(lhs), parse_expression(min_precedence, associativity));
LibJS: Implement bitwise right shift operator (>>)
2020-04-23 12:45:19 +00:00
case TokenType::ShiftRightEquals:
LibJS: Validate all assignment expressions, not just "=" The check for invalid lhs and assignment to eval/arguments in strict mode should happen for all kinds of assignment expressions, not just AssignmentOp::Assignment.
2020-10-04 22:58:57 +00:00
return parse_assignment_expression(AssignmentOp::RightShiftAssignment, move(lhs), min_precedence, associativity);
LibJS: Implement bitwise unsigned right shift operator (>>>)
2020-04-23 14:43:10 +00:00
case TokenType::UnsignedShiftRight:
consume();
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
return create_ast_node<BinaryExpression>({ m_state.current_token.filename(), rule_start.position(), position() }, BinaryOp::UnsignedRightShift, move(lhs), parse_expression(min_precedence, associativity));
LibJS: Implement bitwise unsigned right shift operator (>>>)
2020-04-23 14:43:10 +00:00
case TokenType::UnsignedShiftRightEquals:
LibJS: Validate all assignment expressions, not just "=" The check for invalid lhs and assignment to eval/arguments in strict mode should happen for all kinds of assignment expressions, not just AssignmentOp::Assignment.
2020-10-04 22:58:57 +00:00
return parse_assignment_expression(AssignmentOp::UnsignedRightShiftAssignment, move(lhs), min_precedence, associativity);
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
case TokenType::ParenOpen:
return parse_call_expression(move(lhs));
case TokenType::Equals:
LibJS: Validate all assignment expressions, not just "=" The check for invalid lhs and assignment to eval/arguments in strict mode should happen for all kinds of assignment expressions, not just AssignmentOp::Assignment.
2020-10-04 22:58:57 +00:00
return parse_assignment_expression(AssignmentOp::Assignment, move(lhs), min_precedence, associativity);
LibJS: Implement basic MemberExpression parsing At last we can parse "hello friends".length :^)
2020-03-12 12:05:57 +00:00
case TokenType::Period:
consume();
LibJS: Add parsing and evaluation of private fields and methods
2021-10-12 20:45:52 +00:00
if (match(TokenType::PrivateIdentifier)) {
if (!is_private_identifier_valid())
syntax_error(String::formatted("Reference to undeclared private field or method '{}'", m_state.current_token.value()));
else if (is<SuperExpression>(*lhs))
syntax_error(String::formatted("Cannot access private field or method '{}' on super", m_state.current_token.value()));
return create_ast_node<MemberExpression>({ m_state.current_token.filename(), rule_start.position(), position() }, move(lhs), create_ast_node<PrivateIdentifier>({ m_state.current_token.filename(), rule_start.position(), position() }, consume().value()));
} else if (!match_identifier_name()) {
LibJS: Allow reserved words as keys in object expressions.
2020-04-18 18:31:27 +00:00
expected("IdentifierName");
LibJS: Add parsing and evaluation of private fields and methods
2021-10-12 20:45:52 +00:00
}
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
return create_ast_node<MemberExpression>({ m_state.current_token.filename(), rule_start.position(), position() }, move(lhs), create_ast_node<Identifier>({ m_state.current_token.filename(), rule_start.position(), position() }, consume().value()));
LibJS: Parse computed MemberExpressions MemberExpression comes in two flavors: computed: a[b] non-computed: a.b We can now parse both of the types. :^)
2020-03-20 19:51:03 +00:00
case TokenType::BracketOpen: {
consume(TokenType::BracketOpen);
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
auto expression = create_ast_node<MemberExpression>({ m_state.current_token.filename(), rule_start.position(), position() }, move(lhs), parse_expression(0), true);
LibJS: Parse computed MemberExpressions MemberExpression comes in two flavors: computed: a[b] non-computed: a.b We can now parse both of the types. :^)
2020-03-20 19:51:03 +00:00
consume(TokenType::BracketClose);
return expression;
}
LibJS: Implement update expressions Note that currently only the non-prefixed variant is supported (i.e i++ not ++i), this variant returns the value of the argument before the update.
2020-03-12 11:45:45 +00:00
case TokenType::PlusPlus:
LibJS: Add FIXMEs for parsing increment operators with function LHS/RHS The parser considers it a syntax error at the moment, other engines throw a ReferenceError during runtime for ++foo(), --foo(), foo()++ and foo()--, so I assume the spec defines this.
2020-09-16 19:03:06 +00:00
// FIXME: Apparently for functions this should also not be enforced on a parser level,
// other engines throw ReferenceError for foo()++
LibJS: Remove hand-rolled type information in JS AST in favor of RTTI
2021-01-01 18:34:07 +00:00
if (!is<Identifier>(*lhs) && !is<MemberExpression>(*lhs))
LibJS: Use String::formatted() for parser error messages
2020-10-04 13:53:15 +00:00
syntax_error(String::formatted("Left-hand side of postfix increment operator must be identifier or member expression, got {}", lhs->class_name()));
LibJS: Be more strict about reserved and special identifiers
2021-07-11 23:27:35 +00:00
if (m_state.strict_mode && is<Identifier>(*lhs)) {
auto& identifier = static_cast<Identifier&>(*lhs);
auto& name = identifier.string();
check_identifier_name_for_assignment_validity(name);
}
LibJS: Implement update expressions Note that currently only the non-prefixed variant is supported (i.e i++ not ++i), this variant returns the value of the argument before the update.
2020-03-12 11:45:45 +00:00
consume();
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
return create_ast_node<UpdateExpression>({ m_state.current_token.filename(), rule_start.position(), position() }, UpdateOp::Increment, move(lhs));
LibJS: Implement update expressions Note that currently only the non-prefixed variant is supported (i.e i++ not ++i), this variant returns the value of the argument before the update.
2020-03-12 11:45:45 +00:00
case TokenType::MinusMinus:
LibJS: Add FIXMEs for parsing increment operators with function LHS/RHS The parser considers it a syntax error at the moment, other engines throw a ReferenceError during runtime for ++foo(), --foo(), foo()++ and foo()--, so I assume the spec defines this.
2020-09-16 19:03:06 +00:00
// FIXME: Apparently for functions this should also not be enforced on a parser level,
// other engines throw ReferenceError for foo()--
LibJS: Remove hand-rolled type information in JS AST in favor of RTTI
2021-01-01 18:34:07 +00:00
if (!is<Identifier>(*lhs) && !is<MemberExpression>(*lhs))
LibJS: Use String::formatted() for parser error messages
2020-10-04 13:53:15 +00:00
syntax_error(String::formatted("Left-hand side of postfix increment operator must be identifier or member expression, got {}", lhs->class_name()));
LibJS: Be more strict about reserved and special identifiers
2021-07-11 23:27:35 +00:00
if (m_state.strict_mode && is<Identifier>(*lhs)) {
auto& identifier = static_cast<Identifier&>(*lhs);
auto& name = identifier.string();
check_identifier_name_for_assignment_validity(name);
}
LibJS: Implement update expressions Note that currently only the non-prefixed variant is supported (i.e i++ not ++i), this variant returns the value of the argument before the update.
2020-03-12 11:45:45 +00:00
consume();
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
return create_ast_node<UpdateExpression>({ m_state.current_token.filename(), rule_start.position(), position() }, UpdateOp::Decrement, move(lhs));
LibJS/Parser: Parse logical expressions
2020-03-15 21:35:22 +00:00
case TokenType::DoubleAmpersand:
consume();
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
return create_ast_node<LogicalExpression>({ m_state.current_token.filename(), rule_start.position(), position() }, LogicalOp::And, move(lhs), parse_expression(min_precedence, associativity));
LibJS: Implement logical assignment operators (&&=, ||=, ??=) TC39 proposal, stage 4 as of 2020-07. https://tc39.es/proposal-logical-assignment/
2020-10-05 15:49:43 +00:00
case TokenType::DoubleAmpersandEquals:
return parse_assignment_expression(AssignmentOp::AndAssignment, move(lhs), min_precedence, associativity);
LibJS/Parser: Parse logical expressions
2020-03-15 21:35:22 +00:00
case TokenType::DoublePipe:
consume();
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
return create_ast_node<LogicalExpression>({ m_state.current_token.filename(), rule_start.position(), position() }, LogicalOp::Or, move(lhs), parse_expression(min_precedence, associativity));
LibJS: Implement logical assignment operators (&&=, ||=, ??=) TC39 proposal, stage 4 as of 2020-07. https://tc39.es/proposal-logical-assignment/
2020-10-05 15:49:43 +00:00
case TokenType::DoublePipeEquals:
return parse_assignment_expression(AssignmentOp::OrAssignment, move(lhs), min_precedence, associativity);
LibJS: Implement nullish coalescing operator (??)
2020-04-17 23:49:11 +00:00
case TokenType::DoubleQuestionMark:
consume();
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
return create_ast_node<LogicalExpression>({ m_state.current_token.filename(), rule_start.position(), position() }, LogicalOp::NullishCoalescing, move(lhs), parse_expression(min_precedence, associativity));
LibJS: Implement logical assignment operators (&&=, ||=, ??=) TC39 proposal, stage 4 as of 2020-07. https://tc39.es/proposal-logical-assignment/
2020-10-05 15:49:43 +00:00
case TokenType::DoubleQuestionMarkEquals:
return parse_assignment_expression(AssignmentOp::NullishAssignment, move(lhs), min_precedence, associativity);
LibJS: Implement ConditionalExpression (ternary "?:" operator)
2020-04-03 10:14:28 +00:00
case TokenType::QuestionMark:
return parse_conditional_expression(move(lhs));
LibJS: Implement parsing and execution of optional chains
2021-09-14 02:26:31 +00:00
case TokenType::QuestionMarkPeriod:
// FIXME: This should allow `(new Foo)?.bar', but as our parser strips parenthesis,
// we can't really tell if `lhs' was parenthesized at this point.
if (is<NewExpression>(lhs.ptr())) {
syntax_error("'new' cannot be used with optional chaining", position());
consume();
return lhs;
}
return parse_optional_chain(move(lhs));
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
default:
LibJS: Distinguish between statement and declaration This separates matching/parsing of statements and declarations and fixes a few edge cases where the parser would incorrectly accept a declaration where only a statement is allowed - for example: if (foo) const a = 1; for (var bar;;) function b() {} while (baz) class c {}
2020-10-22 22:30:07 +00:00
expected("secondary expression");
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
consume();
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
return create_ast_node<ErrorExpression>({ m_state.current_token.filename(), rule_start.position(), position() });
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
}
}
LibJS: Add parsing and evaluation of private fields and methods
2021-10-12 20:45:52 +00:00
bool Parser::is_private_identifier_valid() const
{
VERIFY(match(TokenType::PrivateIdentifier));
if (!m_state.referenced_private_names)
return false;
// We might not have hit the declaration yet so class will check this in the end
m_state.referenced_private_names->set(m_state.current_token.value());
return true;
}
LibJS: Allow member expressions in binding patterns Also allows literal string and numbers as property names in object binding patterns.
2021-09-17 23:11:32 +00:00
RefPtr<BindingPattern> Parser::synthesize_binding_pattern(Expression const& expression)
{
VERIFY(is<ArrayExpression>(expression) || is<ObjectExpression>(expression));
// Clear any syntax error that has occurred in the range that 'expression' spans.
m_state.errors.remove_all_matching([range = expression.source_range()](auto const& error) {
return error.position.has_value() && range.contains(*error.position);
});
// Make a parser and parse the source for this expression as a binding pattern.
auto source = m_state.lexer.source().substring_view(expression.source_range().start.offset - 2, expression.source_range().end.offset - expression.source_range().start.offset);
Lexer lexer { source, m_state.lexer.filename(), expression.source_range().start.line, expression.source_range().start.column };
Parser parser { lexer };
parser.m_state.strict_mode = m_state.strict_mode;
parser.m_state.allow_super_property_lookup = m_state.allow_super_property_lookup;
parser.m_state.allow_super_constructor_call = m_state.allow_super_constructor_call;
parser.m_state.in_function_context = m_state.in_function_context;
parser.m_state.in_formal_parameter_context = m_state.in_formal_parameter_context;
parser.m_state.in_generator_function_context = m_state.in_generator_function_context;
LibJS: Add support for async functions This commit adds support for the most bare bones version of async functions, support for async generator functions, async arrow functions and await expressions are TODO.
2021-11-09 18:39:22 +00:00
parser.m_state.in_async_function_context = m_state.in_async_function_context;
LibJS: Allow member expressions in binding patterns Also allows literal string and numbers as property names in object binding patterns.
2021-09-17 23:11:32 +00:00
parser.m_state.in_arrow_function_context = m_state.in_arrow_function_context;
parser.m_state.in_break_context = m_state.in_break_context;
parser.m_state.in_continue_context = m_state.in_continue_context;
parser.m_state.string_legacy_octal_escape_sequence_in_scope = m_state.string_legacy_octal_escape_sequence_in_scope;
parser.m_state.in_class_field_initializer = m_state.in_class_field_initializer;
LibJS: Add support for await expressions
2021-11-09 20:52:21 +00:00
parser.m_state.in_class_static_init_block = m_state.in_class_static_init_block;
LibJS: Allow member expressions in binding patterns Also allows literal string and numbers as property names in object binding patterns.
2021-09-17 23:11:32 +00:00
auto result = parser.parse_binding_pattern(AllowDuplicates::Yes, AllowMemberExpressions::Yes);
if (parser.has_errors())
m_state.errors.extend(parser.errors());
return result;
}
LibJS: Validate all assignment expressions, not just "=" The check for invalid lhs and assignment to eval/arguments in strict mode should happen for all kinds of assignment expressions, not just AssignmentOp::Assignment.
2020-10-04 22:58:57 +00:00
NonnullRefPtr<AssignmentExpression> Parser::parse_assignment_expression(AssignmentOp assignment_op, NonnullRefPtr<Expression> lhs, int min_precedence, Associativity associativity)
{
LibJS: Track source positions all the way down to exceptions This makes exceptions have a trace of source positions too, which could probably be helpful in making fancier error tracebacks.
2020-12-28 17:15:22 +00:00
auto rule_start = push_start();
Everywhere: Rename ASSERT => VERIFY (...and ASSERT_NOT_REACHED => VERIFY_NOT_REACHED) Since all of these checks are done in release builds as well, let's rename them to VERIFY to prevent confusion, as everyone is used to assertions being compiled out in release. We can introduce a new ASSERT macro that is specifically for debug checks, but I'm doing this wholesale conversion first since we've accumulated thousands of these already, and it's not immediately obvious which ones are suitable for ASSERT.
2021-02-23 19:42:32 +00:00
VERIFY(match(TokenType::Equals)
LibJS: Validate all assignment expressions, not just "=" The check for invalid lhs and assignment to eval/arguments in strict mode should happen for all kinds of assignment expressions, not just AssignmentOp::Assignment.
2020-10-04 22:58:57 +00:00
|| match(TokenType::PlusEquals)
|| match(TokenType::MinusEquals)
|| match(TokenType::AsteriskEquals)
|| match(TokenType::SlashEquals)
|| match(TokenType::PercentEquals)
|| match(TokenType::DoubleAsteriskEquals)
|| match(TokenType::AmpersandEquals)
|| match(TokenType::PipeEquals)
|| match(TokenType::CaretEquals)
|| match(TokenType::ShiftLeftEquals)
|| match(TokenType::ShiftRightEquals)
LibJS: Implement logical assignment operators (&&=, ||=, ??=) TC39 proposal, stage 4 as of 2020-07. https://tc39.es/proposal-logical-assignment/
2020-10-05 15:49:43 +00:00
|| match(TokenType::UnsignedShiftRightEquals)
|| match(TokenType::DoubleAmpersandEquals)
|| match(TokenType::DoublePipeEquals)
|| match(TokenType::DoubleQuestionMarkEquals));
LibJS: Validate all assignment expressions, not just "=" The check for invalid lhs and assignment to eval/arguments in strict mode should happen for all kinds of assignment expressions, not just AssignmentOp::Assignment.
2020-10-04 22:58:57 +00:00
consume();
LibJS: Implement parsing and evaluation for AssignmentPatterns e.g. `[...foo] = bar` can now be evaluated :^)
2021-07-10 20:46:17 +00:00
if (assignment_op == AssignmentOp::Assignment) {
if (is<ArrayExpression>(*lhs) || is<ObjectExpression>(*lhs)) {
auto binding_pattern = synthesize_binding_pattern(*lhs);
if (binding_pattern) {
auto rhs = parse_expression(min_precedence, associativity);
return create_ast_node<AssignmentExpression>(
{ m_state.current_token.filename(), rule_start.position(), position() },
assignment_op,
binding_pattern.release_nonnull(),
move(rhs));
}
}
}
LibJS: Remove hand-rolled type information in JS AST in favor of RTTI
2021-01-01 18:34:07 +00:00
if (!is<Identifier>(*lhs) && !is<MemberExpression>(*lhs) && !is<CallExpression>(*lhs)) {
LibJS: Validate all assignment expressions, not just "=" The check for invalid lhs and assignment to eval/arguments in strict mode should happen for all kinds of assignment expressions, not just AssignmentOp::Assignment.
2020-10-04 22:58:57 +00:00
syntax_error("Invalid left-hand side in assignment");
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
} else if (m_state.strict_mode && is<Identifier>(*lhs)) {
LibJS: Validate all assignment expressions, not just "=" The check for invalid lhs and assignment to eval/arguments in strict mode should happen for all kinds of assignment expressions, not just AssignmentOp::Assignment.
2020-10-04 22:58:57 +00:00
auto name = static_cast<const Identifier&>(*lhs).string();
LibJS: Be more strict about reserved and special identifiers
2021-07-11 23:27:35 +00:00
check_identifier_name_for_assignment_validity(name);
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
} else if (m_state.strict_mode && is<CallExpression>(*lhs)) {
LibJS: Make assignment to CallExpression a syntax error in strict mode
2020-10-04 23:00:10 +00:00
syntax_error("Cannot assign to function call");
LibJS: Validate all assignment expressions, not just "=" The check for invalid lhs and assignment to eval/arguments in strict mode should happen for all kinds of assignment expressions, not just AssignmentOp::Assignment.
2020-10-04 22:58:57 +00:00
}
Only apply auto-naming of function expressions based on syntax The auto naming of function expressions is a purely syntactic decision, so shouldn't be decided based on the dynamic type of an assignment. This moves the decision making into the parser. One icky hack is that we add a field to FunctionExpression to indicate whether we can autoname. The real solution is to actually generate a CompoundExpression node so that the parser can make the correct decision, however this would have a potentially significant run time cost. This does not correct the behaviour for class expressions. Patch from Anonymous.
2021-03-22 11:44:07 +00:00
auto rhs = parse_expression(min_precedence, associativity);
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
return create_ast_node<AssignmentExpression>({ m_state.current_token.filename(), rule_start.position(), position() }, assignment_op, move(lhs), move(rhs));
LibJS: Validate all assignment expressions, not just "=" The check for invalid lhs and assignment to eval/arguments in strict mode should happen for all kinds of assignment expressions, not just AssignmentOp::Assignment.
2020-10-04 22:58:57 +00:00
}
LibJS: Restructure and fully implement BindingPatterns
2021-06-13 01:04:28 +00:00
NonnullRefPtr<Identifier> Parser::parse_identifier()
{
auto identifier_start = position();
LibJS: Rework Identifier parsing to match the spec more closely
2021-07-11 11:04:55 +00:00
auto token = consume_identifier();
LibJS: Add support for public fields in classes
2021-08-28 15:11:05 +00:00
if (m_state.in_class_field_initializer && token.value() == "arguments"sv)
syntax_error("'arguments' is not allowed in class field initializer");
LibJS: Restructure and fully implement BindingPatterns
2021-06-13 01:04:28 +00:00
return create_ast_node<Identifier>(
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
{ m_state.current_token.filename(), identifier_start, position() },
LibJS: Restructure and fully implement BindingPatterns
2021-06-13 01:04:28 +00:00
token.value());
}
LibJS: Implement parsing and execution of optional chains
2021-09-14 02:26:31 +00:00
Vector<CallExpression::Argument> Parser::parse_arguments()
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
{
LibJS: Add function call spreading Adds support for the following syntax: myFunction(...x, ...[1, 2, 3], ...o.foo, ...'abcd')
2020-05-06 05:36:24 +00:00
Vector<CallExpression::Argument> arguments;
LibJS: Parse CallExpression arguments
2020-03-12 18:35:23 +00:00
LibJS: Implement parsing and execution of optional chains
2021-09-14 02:26:31 +00:00
consume(TokenType::ParenOpen);
LibJS: Add function call spreading Adds support for the following syntax: myFunction(...x, ...[1, 2, 3], ...o.foo, ...'abcd')
2020-05-06 05:36:24 +00:00
while (match_expression() || match(TokenType::TripleDot)) {
if (match(TokenType::TripleDot)) {
consume();
LibJS: Parse comma operator into SequenceExpression
2020-05-11 16:27:31 +00:00
arguments.append({ parse_expression(2), true });
LibJS: Add function call spreading Adds support for the following syntax: myFunction(...x, ...[1, 2, 3], ...o.foo, ...'abcd')
2020-05-06 05:36:24 +00:00
} else {
LibJS: Parse comma operator into SequenceExpression
2020-05-11 16:27:31 +00:00
arguments.append({ parse_expression(2), false });
LibJS: Add function call spreading Adds support for the following syntax: myFunction(...x, ...[1, 2, 3], ...o.foo, ...'abcd')
2020-05-06 05:36:24 +00:00
}
LibJS: Fix broken parsing of 0-argument CallExpression
2020-03-12 19:03:12 +00:00
if (!match(TokenType::Comma))
break;
consume();
LibJS: Parse CallExpression arguments
2020-03-12 18:35:23 +00:00
}
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
consume(TokenType::ParenClose);
LibJS: Implement parsing and execution of optional chains
2021-09-14 02:26:31 +00:00
return arguments;
}
NonnullRefPtr<Expression> Parser::parse_call_expression(NonnullRefPtr<Expression> lhs)
{
auto rule_start = push_start();
if (!m_state.allow_super_constructor_call && is<SuperExpression>(*lhs))
syntax_error("'super' keyword unexpected here");
auto arguments = parse_arguments();
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
LibJS: Make SuperCall a proper AST node and clean up evaluation
2021-07-02 17:30:38 +00:00
if (is<SuperExpression>(*lhs))
return create_ast_node<SuperCall>({ m_state.current_token.filename(), rule_start.position(), position() }, move(arguments));
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
return create_ast_node<CallExpression>({ m_state.current_token.filename(), rule_start.position(), position() }, move(lhs), move(arguments));
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
}
LibJS: Implement basic support for the "new" keyword NewExpression mostly piggybacks on the existing CallExpression. The big difference is that "new" creates a new Object and passes it as |this| to the callee.
2020-03-28 15:33:52 +00:00
NonnullRefPtr<NewExpression> Parser::parse_new_expression()
{
LibJS: Track source positions all the way down to exceptions This makes exceptions have a trace of source positions too, which could probably be helpful in making fancier error tracebacks.
2020-12-28 17:15:22 +00:00
auto rule_start = push_start();
LibJS: Implement basic support for the "new" keyword NewExpression mostly piggybacks on the existing CallExpression. The big difference is that "new" creates a new Object and passes it as |this| to the callee.
2020-03-28 15:33:52 +00:00
consume(TokenType::New);
LibJS: Implement parsing and execution of optional chains
2021-09-14 02:26:31 +00:00
auto callee = parse_expression(g_operator_precedence.get(TokenType::New), Associativity::Right, { TokenType::ParenOpen, TokenType::QuestionMarkPeriod });
LibJS: Implement basic support for the "new" keyword NewExpression mostly piggybacks on the existing CallExpression. The big difference is that "new" creates a new Object and passes it as |this| to the callee.
2020-03-28 15:33:52 +00:00
LibJS: Add function call spreading Adds support for the following syntax: myFunction(...x, ...[1, 2, 3], ...o.foo, ...'abcd')
2020-05-06 05:36:24 +00:00
Vector<CallExpression::Argument> arguments;
LibJS: Implement basic support for the "new" keyword NewExpression mostly piggybacks on the existing CallExpression. The big difference is that "new" creates a new Object and passes it as |this| to the callee.
2020-03-28 15:33:52 +00:00
if (match(TokenType::ParenOpen)) {
consume(TokenType::ParenOpen);
LibJS: Add function call spreading Adds support for the following syntax: myFunction(...x, ...[1, 2, 3], ...o.foo, ...'abcd')
2020-05-06 05:36:24 +00:00
while (match_expression() || match(TokenType::TripleDot)) {
if (match(TokenType::TripleDot)) {
consume();
LibJS: Parse comma operator into SequenceExpression
2020-05-11 16:27:31 +00:00
arguments.append({ parse_expression(2), true });
LibJS: Add function call spreading Adds support for the following syntax: myFunction(...x, ...[1, 2, 3], ...o.foo, ...'abcd')
2020-05-06 05:36:24 +00:00
} else {
LibJS: Parse comma operator into SequenceExpression
2020-05-11 16:27:31 +00:00
arguments.append({ parse_expression(2), false });
LibJS: Add function call spreading Adds support for the following syntax: myFunction(...x, ...[1, 2, 3], ...o.foo, ...'abcd')
2020-05-06 05:36:24 +00:00
}
LibJS: Implement basic support for the "new" keyword NewExpression mostly piggybacks on the existing CallExpression. The big difference is that "new" creates a new Object and passes it as |this| to the callee.
2020-03-28 15:33:52 +00:00
if (!match(TokenType::Comma))
break;
consume();
}
consume(TokenType::ParenClose);
}
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
return create_ast_node<NewExpression>({ m_state.current_token.filename(), rule_start.position(), position() }, move(callee), move(arguments));
LibJS: Implement basic support for the "new" keyword NewExpression mostly piggybacks on the existing CallExpression. The big difference is that "new" creates a new Object and passes it as |this| to the callee.
2020-03-28 15:33:52 +00:00
}
LibJS: Implement generator functions (only in bytecode mode)
2021-06-10 21:08:30 +00:00
NonnullRefPtr<YieldExpression> Parser::parse_yield_expression()
{
auto rule_start = push_start();
LibJS: Disallow yield expression correctly in formal parameters And add ZERO WIDTH NO BREAK SPACE to valid whitespace.
2021-08-21 09:31:36 +00:00
if (m_state.in_formal_parameter_context)
syntax_error("'Yield' expression is not allowed in formal parameters of generator function");
LibJS: Implement generator functions (only in bytecode mode)
2021-06-10 21:08:30 +00:00
consume(TokenType::Yield);
RefPtr<Expression> argument;
LibJS: Correctly parse yield-from expressions This commit implements parsing for `yield *expr`, and the multiple ways something can or can't be parsed like that. Also makes yield-from a TODO in the bytecode generator. Behold, the glory of javascript syntax: ```js // 'yield' = expression in generators. function* foo() { yield *bar; // <- Syntax error here, expression can't start with * } // 'yield' = identifier anywhere else. function foo() { yield *bar; // Perfectly fine, this is just `yield * bar` } ```
2021-06-14 11:16:41 +00:00
bool yield_from = false;
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
if (!m_state.current_token.trivia_contains_line_terminator()) {
LibJS: Correctly parse yield-from expressions This commit implements parsing for `yield *expr`, and the multiple ways something can or can't be parsed like that. Also makes yield-from a TODO in the bytecode generator. Behold, the glory of javascript syntax: ```js // 'yield' = expression in generators. function* foo() { yield *bar; // <- Syntax error here, expression can't start with * } // 'yield' = identifier anywhere else. function foo() { yield *bar; // Perfectly fine, this is just `yield * bar` } ```
2021-06-14 11:16:41 +00:00
if (match(TokenType::Asterisk)) {
consume();
yield_from = true;
}
LibJS: Allow yielding a class
2021-07-29 00:03:38 +00:00
if (yield_from || match_expression() || match(TokenType::Class))
LibJS: Correctly parse yield-from expressions This commit implements parsing for `yield *expr`, and the multiple ways something can or can't be parsed like that. Also makes yield-from a TODO in the bytecode generator. Behold, the glory of javascript syntax: ```js // 'yield' = expression in generators. function* foo() { yield *bar; // <- Syntax error here, expression can't start with * } // 'yield' = identifier anywhere else. function foo() { yield *bar; // Perfectly fine, this is just `yield * bar` } ```
2021-06-14 11:16:41 +00:00
argument = parse_expression(0);
}
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
return create_ast_node<YieldExpression>({ m_state.current_token.filename(), rule_start.position(), position() }, move(argument), yield_from);
LibJS: Implement generator functions (only in bytecode mode)
2021-06-10 21:08:30 +00:00
}
LibJS: Add support for await expressions
2021-11-09 20:52:21 +00:00
NonnullRefPtr<AwaitExpression> Parser::parse_await_expression()
{
auto rule_start = push_start();
if (m_state.in_formal_parameter_context)
syntax_error("'Await' expression is not allowed in formal parameters of an async function");
consume(TokenType::Await);
auto precedence = g_operator_precedence.get(TokenType::Await);
auto associativity = operator_associativity(TokenType::Await);
auto argument = parse_expression(precedence, associativity);
return create_ast_node<AwaitExpression>({ m_state.current_token.filename(), rule_start.position(), position() }, move(argument));
}
LibJS: Make the AST reference-counted This allows function objects to outlive the original parsed program without their ScopeNode disappearing.
2020-03-18 10:23:53 +00:00
NonnullRefPtr<ReturnStatement> Parser::parse_return_statement()
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
{
LibJS: Track source positions all the way down to exceptions This makes exceptions have a trace of source positions too, which could probably be helpful in making fancier error tracebacks.
2020-12-28 17:15:22 +00:00
auto rule_start = push_start();
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
if (!m_state.in_function_context && !m_state.in_arrow_function_context)
LibJS: Disallow 'return' outside of a function
2020-10-07 18:33:48 +00:00
syntax_error("'return' not allowed outside of a function");
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
consume(TokenType::Return);
LibJS: Implement automatic semicolon insertion
2020-04-17 13:05:58 +00:00
// Automatic semicolon insertion: terminate statement when return is followed by newline
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
if (m_state.current_token.trivia_contains_line_terminator())
return create_ast_node<ReturnStatement>({ m_state.current_token.filename(), rule_start.position(), position() }, nullptr);
LibJS: Implement automatic semicolon insertion
2020-04-17 13:05:58 +00:00
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
if (match_expression()) {
LibJS: Fix parsing of IfStatement, fixes #1829
2020-04-18 12:00:43 +00:00
auto expression = parse_expression(0);
consume_or_insert_semicolon();
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
return create_ast_node<ReturnStatement>({ m_state.current_token.filename(), rule_start.position(), position() }, move(expression));
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
}
LibJS: Fix parsing of IfStatement, fixes #1829
2020-04-18 12:00:43 +00:00
LibJS: Implement automatic semicolon insertion
2020-04-17 13:05:58 +00:00
consume_or_insert_semicolon();
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
return create_ast_node<ReturnStatement>({ m_state.current_token.filename(), rule_start.position(), position() }, nullptr);
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
}
LibJS: Make scoping follow the spec Before this we used an ad-hoc combination of references and 'variables' stored in a hashmap. This worked in most cases but is not spec like. Additionally hoisting, dynamically naming functions and scope analysis was not done properly. This patch fixes all of that by: - Implement BindingInitialization for destructuring assignment. - Implementing a new ScopePusher which tracks the lexical and var scoped declarations. This hoists functions to the top level if no lexical declaration name overlaps. Furthermore we do checking of redeclarations in the ScopePusher now requiring less checks all over the place. - Add methods for parsing the directives and statement lists instead of having that code duplicated in multiple places. This allows declarations to pushed to the appropriate scope more easily. - Remove the non spec way of storing 'variables' in DeclarativeEnvironment and make Reference follow the spec instead of checking both the bindings and 'variables'. - Remove all scoping related things from the Interpreter. And instead use environments as specified by the spec. This also includes fixing that NativeFunctions did not produce a valid FunctionEnvironment which could cause issues with callbacks and eval. All FunctionObjects now have a valid NewFunctionEnvironment implementation. - Remove execute_statements from Interpreter and instead use ASTNode::execute everywhere this simplifies AST.cpp as you no longer need to worry about which method to call. - Make ScopeNodes setup their own environment. This uses four different methods specified by the spec {Block, Function, Eval, Global}DeclarationInstantiation with the annexB extensions. - Implement and use NamedEvaluation where specified. Additionally there are fixes to things exposed by these changes to eval, {for, for-in, for-of} loops and assignment. Finally it also fixes some tests in test-js which where passing before but not now that we have correct behavior :^).
2021-09-22 10:44:56 +00:00
void Parser::parse_statement_list(ScopeNode& output_node, AllowLabelledFunction allow_labelled_functions)
LibJS: Strict mode is now handled by Functions and Programs, not Blocks Since blocks can't be strict by themselves, it makes no sense for them to store whether or not they are strict. Strict-ness is now stored in the Program and FunctionNode ASTNodes. Fixes issue #3641
2020-10-04 00:02:43 +00:00
{
LibJS: Make scoping follow the spec Before this we used an ad-hoc combination of references and 'variables' stored in a hashmap. This worked in most cases but is not spec like. Additionally hoisting, dynamically naming functions and scope analysis was not done properly. This patch fixes all of that by: - Implement BindingInitialization for destructuring assignment. - Implementing a new ScopePusher which tracks the lexical and var scoped declarations. This hoists functions to the top level if no lexical declaration name overlaps. Furthermore we do checking of redeclarations in the ScopePusher now requiring less checks all over the place. - Add methods for parsing the directives and statement lists instead of having that code duplicated in multiple places. This allows declarations to pushed to the appropriate scope more easily. - Remove the non spec way of storing 'variables' in DeclarativeEnvironment and make Reference follow the spec instead of checking both the bindings and 'variables'. - Remove all scoping related things from the Interpreter. And instead use environments as specified by the spec. This also includes fixing that NativeFunctions did not produce a valid FunctionEnvironment which could cause issues with callbacks and eval. All FunctionObjects now have a valid NewFunctionEnvironment implementation. - Remove execute_statements from Interpreter and instead use ASTNode::execute everywhere this simplifies AST.cpp as you no longer need to worry about which method to call. - Make ScopeNodes setup their own environment. This uses four different methods specified by the spec {Block, Function, Eval, Global}DeclarationInstantiation with the annexB extensions. - Implement and use NamedEvaluation where specified. Additionally there are fixes to things exposed by these changes to eval, {for, for-in, for-of} loops and assignment. Finally it also fixes some tests in test-js which where passing before but not now that we have correct behavior :^).
2021-09-22 10:44:56 +00:00
while (!done()) {
if (match_declaration()) {
auto declaration = parse_declaration();
VERIFY(m_state.current_scope_pusher);
m_state.current_scope_pusher->add_declaration(declaration);
output_node.append(move(declaration));
} else if (match_statement()) {
output_node.append(parse_statement(allow_labelled_functions));
} else {
break;
}
}
LibJS: Strict mode is now handled by Functions and Programs, not Blocks Since blocks can't be strict by themselves, it makes no sense for them to store whether or not they are strict. Strict-ness is now stored in the Program and FunctionNode ASTNodes. Fixes issue #3641
2020-10-04 00:02:43 +00:00
}
LibJS: Make scoping follow the spec Before this we used an ad-hoc combination of references and 'variables' stored in a hashmap. This worked in most cases but is not spec like. Additionally hoisting, dynamically naming functions and scope analysis was not done properly. This patch fixes all of that by: - Implement BindingInitialization for destructuring assignment. - Implementing a new ScopePusher which tracks the lexical and var scoped declarations. This hoists functions to the top level if no lexical declaration name overlaps. Furthermore we do checking of redeclarations in the ScopePusher now requiring less checks all over the place. - Add methods for parsing the directives and statement lists instead of having that code duplicated in multiple places. This allows declarations to pushed to the appropriate scope more easily. - Remove the non spec way of storing 'variables' in DeclarativeEnvironment and make Reference follow the spec instead of checking both the bindings and 'variables'. - Remove all scoping related things from the Interpreter. And instead use environments as specified by the spec. This also includes fixing that NativeFunctions did not produce a valid FunctionEnvironment which could cause issues with callbacks and eval. All FunctionObjects now have a valid NewFunctionEnvironment implementation. - Remove execute_statements from Interpreter and instead use ASTNode::execute everywhere this simplifies AST.cpp as you no longer need to worry about which method to call. - Make ScopeNodes setup their own environment. This uses four different methods specified by the spec {Block, Function, Eval, Global}DeclarationInstantiation with the annexB extensions. - Implement and use NamedEvaluation where specified. Additionally there are fixes to things exposed by these changes to eval, {for, for-in, for-of} loops and assignment. Finally it also fixes some tests in test-js which where passing before but not now that we have correct behavior :^).
2021-09-22 10:44:56 +00:00
// FunctionBody, https://tc39.es/ecma262/#prod-FunctionBody
LibJS: Propagate "contains direct call to eval()" flag from parser We now propagate this flag to FunctionDeclaration, and then also into ECMAScriptFunctionObject. This will be used to disable optimizations that aren't safe in the presence of direct eval().
2021-10-08 10:43:38 +00:00
NonnullRefPtr<FunctionBody> Parser::parse_function_body(Vector<FunctionDeclaration::Parameter> const& parameters, FunctionKind function_kind, bool& contains_direct_call_to_eval)
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
{
LibJS: Track source positions all the way down to exceptions This makes exceptions have a trace of source positions too, which could probably be helpful in making fancier error tracebacks.
2020-12-28 17:15:22 +00:00
auto rule_start = push_start();
LibJS: Make scoping follow the spec Before this we used an ad-hoc combination of references and 'variables' stored in a hashmap. This worked in most cases but is not spec like. Additionally hoisting, dynamically naming functions and scope analysis was not done properly. This patch fixes all of that by: - Implement BindingInitialization for destructuring assignment. - Implementing a new ScopePusher which tracks the lexical and var scoped declarations. This hoists functions to the top level if no lexical declaration name overlaps. Furthermore we do checking of redeclarations in the ScopePusher now requiring less checks all over the place. - Add methods for parsing the directives and statement lists instead of having that code duplicated in multiple places. This allows declarations to pushed to the appropriate scope more easily. - Remove the non spec way of storing 'variables' in DeclarativeEnvironment and make Reference follow the spec instead of checking both the bindings and 'variables'. - Remove all scoping related things from the Interpreter. And instead use environments as specified by the spec. This also includes fixing that NativeFunctions did not produce a valid FunctionEnvironment which could cause issues with callbacks and eval. All FunctionObjects now have a valid NewFunctionEnvironment implementation. - Remove execute_statements from Interpreter and instead use ASTNode::execute everywhere this simplifies AST.cpp as you no longer need to worry about which method to call. - Make ScopeNodes setup their own environment. This uses four different methods specified by the spec {Block, Function, Eval, Global}DeclarationInstantiation with the annexB extensions. - Implement and use NamedEvaluation where specified. Additionally there are fixes to things exposed by these changes to eval, {for, for-in, for-of} loops and assignment. Finally it also fixes some tests in test-js which where passing before but not now that we have correct behavior :^).
2021-09-22 10:44:56 +00:00
auto function_body = create_ast_node<FunctionBody>({ m_state.current_token.filename(), rule_start.position(), position() });
ScopePusher function_scope = ScopePusher::function_scope(*this, function_body, parameters); // FIXME <-
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
consume(TokenType::CurlyOpen);
LibJS: Make scoping follow the spec Before this we used an ad-hoc combination of references and 'variables' stored in a hashmap. This worked in most cases but is not spec like. Additionally hoisting, dynamically naming functions and scope analysis was not done properly. This patch fixes all of that by: - Implement BindingInitialization for destructuring assignment. - Implementing a new ScopePusher which tracks the lexical and var scoped declarations. This hoists functions to the top level if no lexical declaration name overlaps. Furthermore we do checking of redeclarations in the ScopePusher now requiring less checks all over the place. - Add methods for parsing the directives and statement lists instead of having that code duplicated in multiple places. This allows declarations to pushed to the appropriate scope more easily. - Remove the non spec way of storing 'variables' in DeclarativeEnvironment and make Reference follow the spec instead of checking both the bindings and 'variables'. - Remove all scoping related things from the Interpreter. And instead use environments as specified by the spec. This also includes fixing that NativeFunctions did not produce a valid FunctionEnvironment which could cause issues with callbacks and eval. All FunctionObjects now have a valid NewFunctionEnvironment implementation. - Remove execute_statements from Interpreter and instead use ASTNode::execute everywhere this simplifies AST.cpp as you no longer need to worry about which method to call. - Make ScopeNodes setup their own environment. This uses four different methods specified by the spec {Block, Function, Eval, Global}DeclarationInstantiation with the annexB extensions. - Implement and use NamedEvaluation where specified. Additionally there are fixes to things exposed by these changes to eval, {for, for-in, for-of} loops and assignment. Finally it also fixes some tests in test-js which where passing before but not now that we have correct behavior :^).
2021-09-22 10:44:56 +00:00
auto has_use_strict = parse_directive(function_body);
bool previous_strict_mode = m_state.strict_mode;
if (has_use_strict) {
m_state.strict_mode = true;
function_body->set_strict_mode();
if (!is_simple_parameter_list(parameters))
syntax_error("Illegal 'use strict' directive in function with non-simple parameter list");
} else if (previous_strict_mode) {
function_body->set_strict_mode();
}
LibJS: Add strict mode Adds the ability for a scope (either a function or the entire program) to be in strict mode. Scopes default to non-strict mode. There are two ways to determine the strict-ness of the JS engine: 1. In the parser, this can be accessed with the parser_state variable m_is_strict_mode boolean. If true, the Parser is currently parsing in strict mode. This is done so that the Parser can generate syntax errors at parse time, which is required in some cases. 2. With Interpreter.is_strict_mode(). This allows strict mode checking at runtime as opposed to compile time. Additionally, in order to test this, a global isStrictMode() function has been added to the JS ReplObject under the test-mode flag.
2020-05-28 05:22:08 +00:00
LibJS: Make scoping follow the spec Before this we used an ad-hoc combination of references and 'variables' stored in a hashmap. This worked in most cases but is not spec like. Additionally hoisting, dynamically naming functions and scope analysis was not done properly. This patch fixes all of that by: - Implement BindingInitialization for destructuring assignment. - Implementing a new ScopePusher which tracks the lexical and var scoped declarations. This hoists functions to the top level if no lexical declaration name overlaps. Furthermore we do checking of redeclarations in the ScopePusher now requiring less checks all over the place. - Add methods for parsing the directives and statement lists instead of having that code duplicated in multiple places. This allows declarations to pushed to the appropriate scope more easily. - Remove the non spec way of storing 'variables' in DeclarativeEnvironment and make Reference follow the spec instead of checking both the bindings and 'variables'. - Remove all scoping related things from the Interpreter. And instead use environments as specified by the spec. This also includes fixing that NativeFunctions did not produce a valid FunctionEnvironment which could cause issues with callbacks and eval. All FunctionObjects now have a valid NewFunctionEnvironment implementation. - Remove execute_statements from Interpreter and instead use ASTNode::execute everywhere this simplifies AST.cpp as you no longer need to worry about which method to call. - Make ScopeNodes setup their own environment. This uses four different methods specified by the spec {Block, Function, Eval, Global}DeclarationInstantiation with the annexB extensions. - Implement and use NamedEvaluation where specified. Additionally there are fixes to things exposed by these changes to eval, {for, for-in, for-of} loops and assignment. Finally it also fixes some tests in test-js which where passing before but not now that we have correct behavior :^).
2021-09-22 10:44:56 +00:00
parse_statement_list(function_body);
LibJS: Add strict mode Adds the ability for a scope (either a function or the entire program) to be in strict mode. Scopes default to non-strict mode. There are two ways to determine the strict-ness of the JS engine: 1. In the parser, this can be accessed with the parser_state variable m_is_strict_mode boolean. If true, the Parser is currently parsing in strict mode. This is done so that the Parser can generate syntax errors at parse time, which is required in some cases. 2. With Interpreter.is_strict_mode(). This allows strict mode checking at runtime as opposed to compile time. Additionally, in order to test this, a global isStrictMode() function has been added to the JS ReplObject under the test-mode flag.
2020-05-28 05:22:08 +00:00
LibJS: Make scoping follow the spec Before this we used an ad-hoc combination of references and 'variables' stored in a hashmap. This worked in most cases but is not spec like. Additionally hoisting, dynamically naming functions and scope analysis was not done properly. This patch fixes all of that by: - Implement BindingInitialization for destructuring assignment. - Implementing a new ScopePusher which tracks the lexical and var scoped declarations. This hoists functions to the top level if no lexical declaration name overlaps. Furthermore we do checking of redeclarations in the ScopePusher now requiring less checks all over the place. - Add methods for parsing the directives and statement lists instead of having that code duplicated in multiple places. This allows declarations to pushed to the appropriate scope more easily. - Remove the non spec way of storing 'variables' in DeclarativeEnvironment and make Reference follow the spec instead of checking both the bindings and 'variables'. - Remove all scoping related things from the Interpreter. And instead use environments as specified by the spec. This also includes fixing that NativeFunctions did not produce a valid FunctionEnvironment which could cause issues with callbacks and eval. All FunctionObjects now have a valid NewFunctionEnvironment implementation. - Remove execute_statements from Interpreter and instead use ASTNode::execute everywhere this simplifies AST.cpp as you no longer need to worry about which method to call. - Make ScopeNodes setup their own environment. This uses four different methods specified by the spec {Block, Function, Eval, Global}DeclarationInstantiation with the annexB extensions. - Implement and use NamedEvaluation where specified. Additionally there are fixes to things exposed by these changes to eval, {for, for-in, for-of} loops and assignment. Finally it also fixes some tests in test-js which where passing before but not now that we have correct behavior :^).
2021-09-22 10:44:56 +00:00
// If the function contains 'use strict' we need to check the parameters (again).
if (function_body->in_strict_mode() || function_kind != FunctionKind::Regular) {
Vector<StringView> parameter_names;
for (auto& parameter : parameters) {
parameter.binding.visit(
[&](FlyString const& parameter_name) {
check_identifier_name_for_assignment_validity(parameter_name, function_body->in_strict_mode());
if (function_kind == FunctionKind::Generator && parameter_name == "yield"sv)
syntax_error("Parameter name 'yield' not allowed in this context");
LibJS: Add support for async functions This commit adds support for the most bare bones version of async functions, support for async generator functions, async arrow functions and await expressions are TODO.
2021-11-09 18:39:22 +00:00
if (function_kind == FunctionKind::Async && parameter_name == "await"sv)
syntax_error("Parameter name 'await' not allowed in this context");
LibJS: Make scoping follow the spec Before this we used an ad-hoc combination of references and 'variables' stored in a hashmap. This worked in most cases but is not spec like. Additionally hoisting, dynamically naming functions and scope analysis was not done properly. This patch fixes all of that by: - Implement BindingInitialization for destructuring assignment. - Implementing a new ScopePusher which tracks the lexical and var scoped declarations. This hoists functions to the top level if no lexical declaration name overlaps. Furthermore we do checking of redeclarations in the ScopePusher now requiring less checks all over the place. - Add methods for parsing the directives and statement lists instead of having that code duplicated in multiple places. This allows declarations to pushed to the appropriate scope more easily. - Remove the non spec way of storing 'variables' in DeclarativeEnvironment and make Reference follow the spec instead of checking both the bindings and 'variables'. - Remove all scoping related things from the Interpreter. And instead use environments as specified by the spec. This also includes fixing that NativeFunctions did not produce a valid FunctionEnvironment which could cause issues with callbacks and eval. All FunctionObjects now have a valid NewFunctionEnvironment implementation. - Remove execute_statements from Interpreter and instead use ASTNode::execute everywhere this simplifies AST.cpp as you no longer need to worry about which method to call. - Make ScopeNodes setup their own environment. This uses four different methods specified by the spec {Block, Function, Eval, Global}DeclarationInstantiation with the annexB extensions. - Implement and use NamedEvaluation where specified. Additionally there are fixes to things exposed by these changes to eval, {for, for-in, for-of} loops and assignment. Finally it also fixes some tests in test-js which where passing before but not now that we have correct behavior :^).
2021-09-22 10:44:56 +00:00
for (auto& previous_name : parameter_names) {
if (previous_name == parameter_name) {
syntax_error(String::formatted("Duplicate parameter '{}' not allowed in strict mode", parameter_name));
}
LibJS: Be less strict about 'use strict' It is allowed to have (bogus) directives e.g. 'does nothing'; 'same'; 'use strict'; Should still trigger strict mode.
2021-07-11 23:25:32 +00:00
}
LibJS: Make scoping follow the spec Before this we used an ad-hoc combination of references and 'variables' stored in a hashmap. This worked in most cases but is not spec like. Additionally hoisting, dynamically naming functions and scope analysis was not done properly. This patch fixes all of that by: - Implement BindingInitialization for destructuring assignment. - Implementing a new ScopePusher which tracks the lexical and var scoped declarations. This hoists functions to the top level if no lexical declaration name overlaps. Furthermore we do checking of redeclarations in the ScopePusher now requiring less checks all over the place. - Add methods for parsing the directives and statement lists instead of having that code duplicated in multiple places. This allows declarations to pushed to the appropriate scope more easily. - Remove the non spec way of storing 'variables' in DeclarativeEnvironment and make Reference follow the spec instead of checking both the bindings and 'variables'. - Remove all scoping related things from the Interpreter. And instead use environments as specified by the spec. This also includes fixing that NativeFunctions did not produce a valid FunctionEnvironment which could cause issues with callbacks and eval. All FunctionObjects now have a valid NewFunctionEnvironment implementation. - Remove execute_statements from Interpreter and instead use ASTNode::execute everywhere this simplifies AST.cpp as you no longer need to worry about which method to call. - Make ScopeNodes setup their own environment. This uses four different methods specified by the spec {Block, Function, Eval, Global}DeclarationInstantiation with the annexB extensions. - Implement and use NamedEvaluation where specified. Additionally there are fixes to things exposed by these changes to eval, {for, for-in, for-of} loops and assignment. Finally it also fixes some tests in test-js which where passing before but not now that we have correct behavior :^).
2021-09-22 10:44:56 +00:00
parameter_names.append(parameter_name);
},
[&](NonnullRefPtr<BindingPattern> const& binding) {
binding->for_each_bound_name([&](auto& bound_name) {
if (function_kind == FunctionKind::Generator && bound_name == "yield"sv)
syntax_error("Parameter name 'yield' not allowed in this context");
LibJS: Be less strict about 'use strict' It is allowed to have (bogus) directives e.g. 'does nothing'; 'same'; 'use strict'; Should still trigger strict mode.
2021-07-11 23:25:32 +00:00
LibJS: Add support for async functions This commit adds support for the most bare bones version of async functions, support for async generator functions, async arrow functions and await expressions are TODO.
2021-11-09 18:39:22 +00:00
if (function_kind == FunctionKind::Async && bound_name == "await"sv)
syntax_error("Parameter name 'await' not allowed in this context");
LibJS: Make scoping follow the spec Before this we used an ad-hoc combination of references and 'variables' stored in a hashmap. This worked in most cases but is not spec like. Additionally hoisting, dynamically naming functions and scope analysis was not done properly. This patch fixes all of that by: - Implement BindingInitialization for destructuring assignment. - Implementing a new ScopePusher which tracks the lexical and var scoped declarations. This hoists functions to the top level if no lexical declaration name overlaps. Furthermore we do checking of redeclarations in the ScopePusher now requiring less checks all over the place. - Add methods for parsing the directives and statement lists instead of having that code duplicated in multiple places. This allows declarations to pushed to the appropriate scope more easily. - Remove the non spec way of storing 'variables' in DeclarativeEnvironment and make Reference follow the spec instead of checking both the bindings and 'variables'. - Remove all scoping related things from the Interpreter. And instead use environments as specified by the spec. This also includes fixing that NativeFunctions did not produce a valid FunctionEnvironment which could cause issues with callbacks and eval. All FunctionObjects now have a valid NewFunctionEnvironment implementation. - Remove execute_statements from Interpreter and instead use ASTNode::execute everywhere this simplifies AST.cpp as you no longer need to worry about which method to call. - Make ScopeNodes setup their own environment. This uses four different methods specified by the spec {Block, Function, Eval, Global}DeclarationInstantiation with the annexB extensions. - Implement and use NamedEvaluation where specified. Additionally there are fixes to things exposed by these changes to eval, {for, for-in, for-of} loops and assignment. Finally it also fixes some tests in test-js which where passing before but not now that we have correct behavior :^).
2021-09-22 10:44:56 +00:00
for (auto& previous_name : parameter_names) {
if (previous_name == bound_name) {
syntax_error(String::formatted("Duplicate parameter '{}' not allowed in strict mode", bound_name));
break;
}
}
parameter_names.append(bound_name);
});
});
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
}
}
LibJS: Make scoping follow the spec Before this we used an ad-hoc combination of references and 'variables' stored in a hashmap. This worked in most cases but is not spec like. Additionally hoisting, dynamically naming functions and scope analysis was not done properly. This patch fixes all of that by: - Implement BindingInitialization for destructuring assignment. - Implementing a new ScopePusher which tracks the lexical and var scoped declarations. This hoists functions to the top level if no lexical declaration name overlaps. Furthermore we do checking of redeclarations in the ScopePusher now requiring less checks all over the place. - Add methods for parsing the directives and statement lists instead of having that code duplicated in multiple places. This allows declarations to pushed to the appropriate scope more easily. - Remove the non spec way of storing 'variables' in DeclarativeEnvironment and make Reference follow the spec instead of checking both the bindings and 'variables'. - Remove all scoping related things from the Interpreter. And instead use environments as specified by the spec. This also includes fixing that NativeFunctions did not produce a valid FunctionEnvironment which could cause issues with callbacks and eval. All FunctionObjects now have a valid NewFunctionEnvironment implementation. - Remove execute_statements from Interpreter and instead use ASTNode::execute everywhere this simplifies AST.cpp as you no longer need to worry about which method to call. - Make ScopeNodes setup their own environment. This uses four different methods specified by the spec {Block, Function, Eval, Global}DeclarationInstantiation with the annexB extensions. - Implement and use NamedEvaluation where specified. Additionally there are fixes to things exposed by these changes to eval, {for, for-in, for-of} loops and assignment. Finally it also fixes some tests in test-js which where passing before but not now that we have correct behavior :^).
2021-09-22 10:44:56 +00:00
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
consume(TokenType::CurlyClose);
LibJS: Make scoping follow the spec Before this we used an ad-hoc combination of references and 'variables' stored in a hashmap. This worked in most cases but is not spec like. Additionally hoisting, dynamically naming functions and scope analysis was not done properly. This patch fixes all of that by: - Implement BindingInitialization for destructuring assignment. - Implementing a new ScopePusher which tracks the lexical and var scoped declarations. This hoists functions to the top level if no lexical declaration name overlaps. Furthermore we do checking of redeclarations in the ScopePusher now requiring less checks all over the place. - Add methods for parsing the directives and statement lists instead of having that code duplicated in multiple places. This allows declarations to pushed to the appropriate scope more easily. - Remove the non spec way of storing 'variables' in DeclarativeEnvironment and make Reference follow the spec instead of checking both the bindings and 'variables'. - Remove all scoping related things from the Interpreter. And instead use environments as specified by the spec. This also includes fixing that NativeFunctions did not produce a valid FunctionEnvironment which could cause issues with callbacks and eval. All FunctionObjects now have a valid NewFunctionEnvironment implementation. - Remove execute_statements from Interpreter and instead use ASTNode::execute everywhere this simplifies AST.cpp as you no longer need to worry about which method to call. - Make ScopeNodes setup their own environment. This uses four different methods specified by the spec {Block, Function, Eval, Global}DeclarationInstantiation with the annexB extensions. - Implement and use NamedEvaluation where specified. Additionally there are fixes to things exposed by these changes to eval, {for, for-in, for-of} loops and assignment. Finally it also fixes some tests in test-js which where passing before but not now that we have correct behavior :^).
2021-09-22 10:44:56 +00:00
m_state.strict_mode = previous_strict_mode;
LibJS: Propagate "contains direct call to eval()" flag from parser We now propagate this flag to FunctionDeclaration, and then also into ECMAScriptFunctionObject. This will be used to disable optimizations that aren't safe in the presence of direct eval().
2021-10-08 10:43:38 +00:00
contains_direct_call_to_eval = function_scope.contains_direct_call_to_eval();
LibJS: Make scoping follow the spec Before this we used an ad-hoc combination of references and 'variables' stored in a hashmap. This worked in most cases but is not spec like. Additionally hoisting, dynamically naming functions and scope analysis was not done properly. This patch fixes all of that by: - Implement BindingInitialization for destructuring assignment. - Implementing a new ScopePusher which tracks the lexical and var scoped declarations. This hoists functions to the top level if no lexical declaration name overlaps. Furthermore we do checking of redeclarations in the ScopePusher now requiring less checks all over the place. - Add methods for parsing the directives and statement lists instead of having that code duplicated in multiple places. This allows declarations to pushed to the appropriate scope more easily. - Remove the non spec way of storing 'variables' in DeclarativeEnvironment and make Reference follow the spec instead of checking both the bindings and 'variables'. - Remove all scoping related things from the Interpreter. And instead use environments as specified by the spec. This also includes fixing that NativeFunctions did not produce a valid FunctionEnvironment which could cause issues with callbacks and eval. All FunctionObjects now have a valid NewFunctionEnvironment implementation. - Remove execute_statements from Interpreter and instead use ASTNode::execute everywhere this simplifies AST.cpp as you no longer need to worry about which method to call. - Make ScopeNodes setup their own environment. This uses four different methods specified by the spec {Block, Function, Eval, Global}DeclarationInstantiation with the annexB extensions. - Implement and use NamedEvaluation where specified. Additionally there are fixes to things exposed by these changes to eval, {for, for-in, for-of} loops and assignment. Finally it also fixes some tests in test-js which where passing before but not now that we have correct behavior :^).
2021-09-22 10:44:56 +00:00
return function_body;
}
NonnullRefPtr<BlockStatement> Parser::parse_block_statement()
{
auto rule_start = push_start();
auto block = create_ast_node<BlockStatement>({ m_state.current_token.filename(), rule_start.position(), position() });
ScopePusher block_scope = ScopePusher::block_scope(*this, block);
consume(TokenType::CurlyOpen);
parse_statement_list(block);
consume(TokenType::CurlyClose);
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
return block;
}
LibJS: Parse FunctionExpressions FunctionExpression is mostly like FunctionDeclaration, except the name is optional. Share the parsing logic in parse_function_node<NodeType>. This allows us to do nice things like: document.addEventListener("DOMContentLoaded", function() { alert("Hello friends!"); });
2020-03-19 10:52:56 +00:00
template<typename FunctionNodeType>
LibJS: Refactor parse_function_node() bool parameters into bit flags I'm about to add even more options and a bunch of unnamed true/false arguments is really not helpful. Let's make this a single parse options parameter using bit flags.
2020-10-20 16:56:49 +00:00
NonnullRefPtr<FunctionNodeType> Parser::parse_function_node(u8 parse_options)
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
{
LibJS: Track source positions all the way down to exceptions This makes exceptions have a trace of source positions too, which could probably be helpful in making fancier error tracebacks.
2020-12-28 17:15:22 +00:00
auto rule_start = push_start();
Everywhere: Rename ASSERT => VERIFY (...and ASSERT_NOT_REACHED => VERIFY_NOT_REACHED) Since all of these checks are done in release builds as well, let's rename them to VERIFY to prevent confusion, as everyone is used to assertions being compiled out in release. We can introduce a new ASSERT macro that is specifically for debug checks, but I'm doing this wholesale conversion first since we've accumulated thousands of these already, and it's not immediately obvious which ones are suitable for ASSERT.
2021-02-23 19:42:32 +00:00
VERIFY(!(parse_options & FunctionNodeParseOptions::IsGetterFunction && parse_options & FunctionNodeParseOptions::IsSetterFunction));
LibJS: Move checks for invalid getter/setter params to parse_function_node This allows us to provide better error messages as we can point the syntax error location to the exact first invalid parameter instead of always the end of the function within a object literal or class definition. Before this change: const Foo = { set bar() {} } ^ Uncaught exception: [SyntaxError]: Object setter property must have one argument (line: 1, column: 28) class Foo { set bar() {} } ^ Uncaught exception: [SyntaxError]: Class setter method must have one argument (line: 1, column: 26) After this change: const Foo = { set bar() {} } ^ Uncaught exception: [SyntaxError]: Setter function must have one argument (line: 1, column: 23) class Foo { set bar() {} } ^ Uncaught exception: [SyntaxError]: Setter function must have one argument (line: 1, column: 21) The only possible downside of this change is that class getters/setters and functions in objects are not distinguished in the message anymore - I don't think that's important though, and classes are (mostly) just syntactic sugar anyway.
2020-10-20 17:32:51 +00:00
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
TemporaryChange super_property_access_rollback(m_state.allow_super_property_lookup, !!(parse_options & FunctionNodeParseOptions::AllowSuperPropertyLookup));
TemporaryChange super_constructor_call_rollback(m_state.allow_super_constructor_call, !!(parse_options & FunctionNodeParseOptions::AllowSuperConstructorCall));
LibJS: Make functions reset break and continue context
2021-07-25 09:51:33 +00:00
TemporaryChange break_context_rollback(m_state.in_break_context, false);
TemporaryChange continue_context_rollback(m_state.in_continue_context, false);
LibJS: Add support for public fields in classes
2021-08-28 15:11:05 +00:00
TemporaryChange class_field_initializer_rollback(m_state.in_class_field_initializer, false);
LibJS: Parse async generator functions
2021-11-15 00:53:24 +00:00
TemporaryChange class_static_initializer_rollback(m_state.in_class_static_init_block, false);
LibJS: Add an optimization to avoid needless arguments object creation This gives FunctionNode a "might need arguments object" boolean flag and sets it based on the simplest possible heuristic for this: if we encounter an identifier called "arguments" or "eval" up to the next (nested) function declaration or expression, we won't need an arguments object. Otherwise, we *might* need one - the final decision is made in the FunctionDeclarationInstantiation AO. Now, this is obviously not perfect. Even if you avoid eval, something like `foo.arguments` will still trigger a false positive - but it's a start and already massively cuts down on needlessly allocated objects, especially in real-world code that is often minified, and so a full "arguments" identifier will be an actual arguments object more often than not. To illustrate the actual impact of this change, here's the number of allocated arguments objects during a full test-js run: Before: - Unmapped arguments objects: 78765 - Mapped arguments objects: 2455 After: - Unmapped arguments objects: 18 - Mapped arguments objects: 37 This results in a ~5% speedup of test-js on my Linux host machine, and about 3.5% on i686 Serenity in QEMU (warm runs, average of 5). The following microbenchmark (calling an empty function 1M times) runs 25% faster on Linux and 45% on Serenity: function foo() {} for (var i = 0; i < 1_000_000; ++i) foo(); test262 reports no changes in either direction, apart from a speedup :^)
2021-10-05 07:44:58 +00:00
TemporaryChange might_need_arguments_object_rollback(m_state.function_might_need_arguments_object, false);
LibJS: Initial class implementation; allow super expressions in object literal methods; add EnvrionmentRecord fields and methods to LexicalEnvironment Adding EnvrionmentRecord's fields and methods lets us throw an exception when |this| is not initialized, which occurs when the super constructor in a derived class has not yet been called, or when |this| has already been initialized (the super constructor was already called).
2020-06-08 18:31:21 +00:00
LibJS: Allow 'yield' and 'await' as function expression names The spec says so, and test262 checks for this too.
2021-07-02 10:41:11 +00:00
constexpr auto is_function_expression = IsSame<FunctionNodeType, FunctionExpression>;
LibJS: Add support for async functions This commit adds support for the most bare bones version of async functions, support for async generator functions, async arrow functions and await expressions are TODO.
2021-11-09 18:39:22 +00:00
FunctionKind function_kind;
if ((parse_options & FunctionNodeParseOptions::IsGeneratorFunction) != 0 && (parse_options & FunctionNodeParseOptions::IsAsyncFunction) != 0)
LibJS: Parse async generator functions
2021-11-15 00:53:24 +00:00
function_kind = FunctionKind::AsyncGenerator;
LibJS: Add support for async functions This commit adds support for the most bare bones version of async functions, support for async generator functions, async arrow functions and await expressions are TODO.
2021-11-09 18:39:22 +00:00
else if ((parse_options & FunctionNodeParseOptions::IsGeneratorFunction) != 0)
function_kind = FunctionKind::Generator;
else if ((parse_options & FunctionNodeParseOptions::IsAsyncFunction) != 0)
function_kind = FunctionKind::Async;
else
function_kind = FunctionKind::Regular;
LibJS: Parse FunctionExpressions FunctionExpression is mostly like FunctionDeclaration, except the name is optional. Share the parsing logic in parse_function_node<NodeType>. This allows us to do nice things like: document.addEventListener("DOMContentLoaded", function() { alert("Hello friends!"); });
2020-03-19 10:52:56 +00:00
String name;
LibJS: Refactor parse_function_node() bool parameters into bit flags I'm about to add even more options and a bunch of unnamed true/false arguments is really not helpful. Let's make this a single parse options parameter using bit flags.
2020-10-20 16:56:49 +00:00
if (parse_options & FunctionNodeParseOptions::CheckForFunctionAndName) {
LibJS: Add support for async functions This commit adds support for the most bare bones version of async functions, support for async generator functions, async arrow functions and await expressions are TODO.
2021-11-09 18:39:22 +00:00
if (function_kind == FunctionKind::Regular && match(TokenType::Async) && !next_token().trivia_contains_line_terminator()) {
function_kind = FunctionKind::Async;
consume(TokenType::Async);
parse_options = parse_options | FunctionNodeParseOptions::IsAsyncFunction;
}
LibJS: Refactor parse_function_node() bool parameters into bit flags I'm about to add even more options and a bunch of unnamed true/false arguments is really not helpful. Let's make this a single parse options parameter using bit flags.
2020-10-20 16:56:49 +00:00
consume(TokenType::Function);
LibJS: Parse async generator functions
2021-11-15 00:53:24 +00:00
if (match(TokenType::Asterisk)) {
function_kind = function_kind == FunctionKind::Regular ? FunctionKind::Generator : FunctionKind::AsyncGenerator;
LibJS: Add support for async functions This commit adds support for the most bare bones version of async functions, support for async generator functions, async arrow functions and await expressions are TODO.
2021-11-09 18:39:22 +00:00
consume(TokenType::Asterisk);
parse_options = parse_options | FunctionNodeParseOptions::IsGeneratorFunction;
LibJS: Parse generator functions in object literals Also add some parser tests
2021-06-14 10:22:59 +00:00
}
LibJS: Implement generator functions (only in bytecode mode)
2021-06-10 21:08:30 +00:00
LibJS: Rework Identifier parsing to match the spec more closely
2021-07-11 11:04:55 +00:00
if (FunctionNodeType::must_have_name() || match_identifier())
name = consume_identifier().value();
LibJS: Allow 'yield' and 'await' as function expression names The spec says so, and test262 checks for this too.
2021-07-02 10:41:11 +00:00
else if (is_function_expression && (match(TokenType::Yield) || match(TokenType::Await)))
name = consume().value();
LibJS: Be more strict about reserved and special identifiers
2021-07-11 23:27:35 +00:00
check_identifier_name_for_assignment_validity(name);
LibJS: Parse FunctionExpressions FunctionExpression is mostly like FunctionDeclaration, except the name is optional. Share the parsing logic in parse_function_node<NodeType>. This allows us to do nice things like: document.addEventListener("DOMContentLoaded", function() { alert("Hello friends!"); });
2020-03-19 10:52:56 +00:00
}
LibJS: Parse async generator functions
2021-11-15 00:53:24 +00:00
TemporaryChange generator_change(m_state.in_generator_function_context, function_kind == FunctionKind::Generator || function_kind == FunctionKind::AsyncGenerator);
TemporaryChange async_change(m_state.in_async_function_context, function_kind == FunctionKind::Async || function_kind == FunctionKind::AsyncGenerator);
LibJS: Treat yield as an identifier in more non-generator contexts And disallow some cases where we are in a generator context.
2021-07-28 23:45:55 +00:00
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
consume(TokenType::ParenOpen);
LibJS: Function.length respects default and rest parameters "[Function.length is] the number of formal parameters. This number excludes the rest parameter and only includes parameters before the first one with a default value." - MDN
2020-05-06 03:02:14 +00:00
i32 function_length = -1;
LibJS: Implement destructuring assignments and function parameters
2021-05-29 11:33:19 +00:00
auto parameters = parse_formal_parameters(function_length, parse_options);
LibJS: Share parameter parsing between regular and arrow functions This simplifies try_parse_arrow_function_expression() and fixes a few cases that should not produce an arrow function AST but did: (a,,) => {} (a b) => {} (a ...b) => {} (...b a) => {} The new parsing logic checks whether parens are expected and uses parse_function_parameters() if so, rolling back if a new syntax error occurs during that. Otherwise it's just an identifier in which case we parse the single parameter ourselves.
2020-10-18 23:26:41 +00:00
consume(TokenType::ParenClose);
if (function_length == -1)
function_length = parameters.size();
LibJS: Add an optimization to avoid needless arguments object creation This gives FunctionNode a "might need arguments object" boolean flag and sets it based on the simplest possible heuristic for this: if we encounter an identifier called "arguments" or "eval" up to the next (nested) function declaration or expression, we won't need an arguments object. Otherwise, we *might* need one - the final decision is made in the FunctionDeclarationInstantiation AO. Now, this is obviously not perfect. Even if you avoid eval, something like `foo.arguments` will still trigger a false positive - but it's a start and already massively cuts down on needlessly allocated objects, especially in real-world code that is often minified, and so a full "arguments" identifier will be an actual arguments object more often than not. To illustrate the actual impact of this change, here's the number of allocated arguments objects during a full test-js run: Before: - Unmapped arguments objects: 78765 - Mapped arguments objects: 2455 After: - Unmapped arguments objects: 18 - Mapped arguments objects: 37 This results in a ~5% speedup of test-js on my Linux host machine, and about 3.5% on i686 Serenity in QEMU (warm runs, average of 5). The following microbenchmark (calling an empty function 1M times) runs 25% faster on Linux and 45% on Serenity: function foo() {} for (var i = 0; i < 1_000_000; ++i) foo(); test262 reports no changes in either direction, apart from a speedup :^)
2021-10-05 07:44:58 +00:00
TemporaryChange function_context_rollback(m_state.in_function_context, true);
LibJS: Treat yield as an identifier in more non-generator contexts And disallow some cases where we are in a generator context.
2021-07-28 23:45:55 +00:00
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
auto old_labels_in_scope = move(m_state.labels_in_scope);
LibJS: Share parameter parsing between regular and arrow functions This simplifies try_parse_arrow_function_expression() and fixes a few cases that should not produce an arrow function AST but did: (a,,) => {} (a b) => {} (a ...b) => {} (...b a) => {} The new parsing logic checks whether parens are expected and uses parse_function_parameters() if so, rolling back if a new syntax error occurs during that. Otherwise it's just an identifier in which case we parse the single parameter ourselves.
2020-10-18 23:26:41 +00:00
ScopeGuard guard([&]() {
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
m_state.labels_in_scope = move(old_labels_in_scope);
LibJS: Share parameter parsing between regular and arrow functions This simplifies try_parse_arrow_function_expression() and fixes a few cases that should not produce an arrow function AST but did: (a,,) => {} (a b) => {} (a ...b) => {} (...b a) => {} The new parsing logic checks whether parens are expected and uses parse_function_parameters() if so, rolling back if a new syntax error occurs during that. Otherwise it's just an identifier in which case we parse the single parameter ourselves.
2020-10-18 23:26:41 +00:00
});
LibJS: Propagate "contains direct call to eval()" flag from parser We now propagate this flag to FunctionDeclaration, and then also into ECMAScriptFunctionObject. This will be used to disable optimizations that aren't safe in the presence of direct eval().
2021-10-08 10:43:38 +00:00
bool contains_direct_call_to_eval = false;
auto body = parse_function_body(parameters, function_kind, contains_direct_call_to_eval);
LibJS: Treat yield as an identifier in more non-generator contexts And disallow some cases where we are in a generator context.
2021-07-28 23:45:55 +00:00
LibJS: Make scoping follow the spec Before this we used an ad-hoc combination of references and 'variables' stored in a hashmap. This worked in most cases but is not spec like. Additionally hoisting, dynamically naming functions and scope analysis was not done properly. This patch fixes all of that by: - Implement BindingInitialization for destructuring assignment. - Implementing a new ScopePusher which tracks the lexical and var scoped declarations. This hoists functions to the top level if no lexical declaration name overlaps. Furthermore we do checking of redeclarations in the ScopePusher now requiring less checks all over the place. - Add methods for parsing the directives and statement lists instead of having that code duplicated in multiple places. This allows declarations to pushed to the appropriate scope more easily. - Remove the non spec way of storing 'variables' in DeclarativeEnvironment and make Reference follow the spec instead of checking both the bindings and 'variables'. - Remove all scoping related things from the Interpreter. And instead use environments as specified by the spec. This also includes fixing that NativeFunctions did not produce a valid FunctionEnvironment which could cause issues with callbacks and eval. All FunctionObjects now have a valid NewFunctionEnvironment implementation. - Remove execute_statements from Interpreter and instead use ASTNode::execute everywhere this simplifies AST.cpp as you no longer need to worry about which method to call. - Make ScopeNodes setup their own environment. This uses four different methods specified by the spec {Block, Function, Eval, Global}DeclarationInstantiation with the annexB extensions. - Implement and use NamedEvaluation where specified. Additionally there are fixes to things exposed by these changes to eval, {for, for-in, for-of} loops and assignment. Finally it also fixes some tests in test-js which where passing before but not now that we have correct behavior :^).
2021-09-22 10:44:56 +00:00
auto has_strict_directive = body->in_strict_mode();
LibJS: Handle strict mode for functions more correctly If a function is strict (has 'use strict' directive) it cannot have bindings, cannot have duplicated parameter names and cannot have some reserved keywords and identifiers as parameter names. The logic partly applies depending on whether we are already in strict mode or the function contains 'use strict';
2021-07-11 23:29:07 +00:00
LibJS: Make scoping follow the spec Before this we used an ad-hoc combination of references and 'variables' stored in a hashmap. This worked in most cases but is not spec like. Additionally hoisting, dynamically naming functions and scope analysis was not done properly. This patch fixes all of that by: - Implement BindingInitialization for destructuring assignment. - Implementing a new ScopePusher which tracks the lexical and var scoped declarations. This hoists functions to the top level if no lexical declaration name overlaps. Furthermore we do checking of redeclarations in the ScopePusher now requiring less checks all over the place. - Add methods for parsing the directives and statement lists instead of having that code duplicated in multiple places. This allows declarations to pushed to the appropriate scope more easily. - Remove the non spec way of storing 'variables' in DeclarativeEnvironment and make Reference follow the spec instead of checking both the bindings and 'variables'. - Remove all scoping related things from the Interpreter. And instead use environments as specified by the spec. This also includes fixing that NativeFunctions did not produce a valid FunctionEnvironment which could cause issues with callbacks and eval. All FunctionObjects now have a valid NewFunctionEnvironment implementation. - Remove execute_statements from Interpreter and instead use ASTNode::execute everywhere this simplifies AST.cpp as you no longer need to worry about which method to call. - Make ScopeNodes setup their own environment. This uses four different methods specified by the spec {Block, Function, Eval, Global}DeclarationInstantiation with the annexB extensions. - Implement and use NamedEvaluation where specified. Additionally there are fixes to things exposed by these changes to eval, {for, for-in, for-of} loops and assignment. Finally it also fixes some tests in test-js which where passing before but not now that we have correct behavior :^).
2021-09-22 10:44:56 +00:00
if (has_strict_directive)
check_identifier_name_for_assignment_validity(name, true);
LibJS: Improve function hoisting across blocks The parser now keeps track of a scope chain so that it can hoist function declarations to the closest function scope.
2021-07-04 01:15:52 +00:00
LibJS: Use an enum class instead of 'bool is_generator' This avoid confusion in the order of the multiple boolean parameters that exist.
2021-06-10 23:08:05 +00:00
return create_ast_node<FunctionNodeType>(
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
{ m_state.current_token.filename(), rule_start.position(), position() },
LibJS: Remove variables from FunctionNode They weren't consumed anywhere outside the AST and went against the usual concept of having declaration in ScopeNode.
2021-07-04 01:12:27 +00:00
name, move(body), move(parameters), function_length,
LibJS: Propagate "contains direct call to eval()" flag from parser We now propagate this flag to FunctionDeclaration, and then also into ECMAScriptFunctionObject. This will be used to disable optimizations that aren't safe in the presence of direct eval().
2021-10-08 10:43:38 +00:00
function_kind, has_strict_directive, m_state.function_might_need_arguments_object,
contains_direct_call_to_eval);
LibJS: Share parameter parsing between regular and arrow functions This simplifies try_parse_arrow_function_expression() and fixes a few cases that should not produce an arrow function AST but did: (a,,) => {} (a b) => {} (a ...b) => {} (...b a) => {} The new parsing logic checks whether parens are expected and uses parse_function_parameters() if so, rolling back if a new syntax error occurs during that. Otherwise it's just an identifier in which case we parse the single parameter ourselves.
2020-10-18 23:26:41 +00:00
}
LibJS: Implement destructuring assignments and function parameters
2021-05-29 11:33:19 +00:00
Vector<FunctionNode::Parameter> Parser::parse_formal_parameters(int& function_length, u8 parse_options)
LibJS: Share parameter parsing between regular and arrow functions This simplifies try_parse_arrow_function_expression() and fixes a few cases that should not produce an arrow function AST but did: (a,,) => {} (a b) => {} (a ...b) => {} (...b a) => {} The new parsing logic checks whether parens are expected and uses parse_function_parameters() if so, rolling back if a new syntax error occurs during that. Otherwise it's just an identifier in which case we parse the single parameter ourselves.
2020-10-18 23:26:41 +00:00
{
LibJS: Track source positions all the way down to exceptions This makes exceptions have a trace of source positions too, which could probably be helpful in making fancier error tracebacks.
2020-12-28 17:15:22 +00:00
auto rule_start = push_start();
LibJS: Implement rules for duplicate function parameters - A regular function can have duplicate parameters except in strict mode or if its parameter list is not "simple" (has a default or rest parameter) - An arrow function can never have duplicate parameters Compared to other engines I opted for more useful syntax error messages than a generic "duplicate parameter name not allowed in this context": "use strict"; function test(foo, foo) {} ^ Uncaught exception: [SyntaxError]: Duplicate parameter 'foo' not allowed in strict mode (line: 1, column: 34) function test(foo, foo = 1) {} ^ Uncaught exception: [SyntaxError]: Duplicate parameter 'foo' not allowed in function with default parameter (line: 1, column: 20) function test(foo, ...foo) {} ^ Uncaught exception: [SyntaxError]: Duplicate parameter 'foo' not allowed in function with rest parameter (line: 1, column: 23) (foo, foo) => {} ^ Uncaught exception: [SyntaxError]: Duplicate parameter 'foo' not allowed in arrow function (line: 1, column: 7)
2020-10-25 11:14:04 +00:00
bool has_default_parameter = false;
bool has_rest_parameter = false;
LibJS: Disallow yield expression correctly in formal parameters And add ZERO WIDTH NO BREAK SPACE to valid whitespace.
2021-08-21 09:31:36 +00:00
TemporaryChange formal_parameter_context_change { m_state.in_formal_parameter_context, true };
LibJS: Implement rules for duplicate function parameters - A regular function can have duplicate parameters except in strict mode or if its parameter list is not "simple" (has a default or rest parameter) - An arrow function can never have duplicate parameters Compared to other engines I opted for more useful syntax error messages than a generic "duplicate parameter name not allowed in this context": "use strict"; function test(foo, foo) {} ^ Uncaught exception: [SyntaxError]: Duplicate parameter 'foo' not allowed in strict mode (line: 1, column: 34) function test(foo, foo = 1) {} ^ Uncaught exception: [SyntaxError]: Duplicate parameter 'foo' not allowed in function with default parameter (line: 1, column: 20) function test(foo, ...foo) {} ^ Uncaught exception: [SyntaxError]: Duplicate parameter 'foo' not allowed in function with rest parameter (line: 1, column: 23) (foo, foo) => {} ^ Uncaught exception: [SyntaxError]: Duplicate parameter 'foo' not allowed in arrow function (line: 1, column: 7)
2020-10-25 11:14:04 +00:00
LibJS: Share parameter parsing between regular and arrow functions This simplifies try_parse_arrow_function_expression() and fixes a few cases that should not produce an arrow function AST but did: (a,,) => {} (a b) => {} (a ...b) => {} (...b a) => {} The new parsing logic checks whether parens are expected and uses parse_function_parameters() if so, rolling back if a new syntax error occurs during that. Otherwise it's just an identifier in which case we parse the single parameter ourselves.
2020-10-18 23:26:41 +00:00
Vector<FunctionNode::Parameter> parameters;
LibJS: Implement rules for duplicate function parameters - A regular function can have duplicate parameters except in strict mode or if its parameter list is not "simple" (has a default or rest parameter) - An arrow function can never have duplicate parameters Compared to other engines I opted for more useful syntax error messages than a generic "duplicate parameter name not allowed in this context": "use strict"; function test(foo, foo) {} ^ Uncaught exception: [SyntaxError]: Duplicate parameter 'foo' not allowed in strict mode (line: 1, column: 34) function test(foo, foo = 1) {} ^ Uncaught exception: [SyntaxError]: Duplicate parameter 'foo' not allowed in function with default parameter (line: 1, column: 20) function test(foo, ...foo) {} ^ Uncaught exception: [SyntaxError]: Duplicate parameter 'foo' not allowed in function with rest parameter (line: 1, column: 23) (foo, foo) => {} ^ Uncaught exception: [SyntaxError]: Duplicate parameter 'foo' not allowed in arrow function (line: 1, column: 7)
2020-10-25 11:14:04 +00:00
LibJS: Implement destructuring assignments and function parameters
2021-05-29 11:33:19 +00:00
auto consume_identifier_or_binding_pattern = [&]() -> Variant<FlyString, NonnullRefPtr<BindingPattern>> {
LibJS: Allow member expressions in binding patterns Also allows literal string and numbers as property names in object binding patterns.
2021-09-17 23:11:32 +00:00
if (auto pattern = parse_binding_pattern(AllowDuplicates::No, AllowMemberExpressions::Yes))
LibJS: Implement destructuring assignments and function parameters
2021-05-29 11:33:19 +00:00
return pattern.release_nonnull();
LibJS: Rework Identifier parsing to match the spec more closely
2021-07-11 11:04:55 +00:00
auto token = consume_identifier();
LibJS: Implement rules for duplicate function parameters - A regular function can have duplicate parameters except in strict mode or if its parameter list is not "simple" (has a default or rest parameter) - An arrow function can never have duplicate parameters Compared to other engines I opted for more useful syntax error messages than a generic "duplicate parameter name not allowed in this context": "use strict"; function test(foo, foo) {} ^ Uncaught exception: [SyntaxError]: Duplicate parameter 'foo' not allowed in strict mode (line: 1, column: 34) function test(foo, foo = 1) {} ^ Uncaught exception: [SyntaxError]: Duplicate parameter 'foo' not allowed in function with default parameter (line: 1, column: 20) function test(foo, ...foo) {} ^ Uncaught exception: [SyntaxError]: Duplicate parameter 'foo' not allowed in function with rest parameter (line: 1, column: 23) (foo, foo) => {} ^ Uncaught exception: [SyntaxError]: Duplicate parameter 'foo' not allowed in arrow function (line: 1, column: 7)
2020-10-25 11:14:04 +00:00
auto parameter_name = token.value();
LibJS: Be more strict about reserved and special identifiers
2021-07-11 23:27:35 +00:00
check_identifier_name_for_assignment_validity(parameter_name);
LibJS: Implement rules for duplicate function parameters - A regular function can have duplicate parameters except in strict mode or if its parameter list is not "simple" (has a default or rest parameter) - An arrow function can never have duplicate parameters Compared to other engines I opted for more useful syntax error messages than a generic "duplicate parameter name not allowed in this context": "use strict"; function test(foo, foo) {} ^ Uncaught exception: [SyntaxError]: Duplicate parameter 'foo' not allowed in strict mode (line: 1, column: 34) function test(foo, foo = 1) {} ^ Uncaught exception: [SyntaxError]: Duplicate parameter 'foo' not allowed in function with default parameter (line: 1, column: 20) function test(foo, ...foo) {} ^ Uncaught exception: [SyntaxError]: Duplicate parameter 'foo' not allowed in function with rest parameter (line: 1, column: 23) (foo, foo) => {} ^ Uncaught exception: [SyntaxError]: Duplicate parameter 'foo' not allowed in arrow function (line: 1, column: 7)
2020-10-25 11:14:04 +00:00
for (auto& parameter : parameters) {
LibJS: Handle strict mode for functions more correctly If a function is strict (has 'use strict' directive) it cannot have bindings, cannot have duplicated parameter names and cannot have some reserved keywords and identifiers as parameter names. The logic partly applies depending on whether we are already in strict mode or the function contains 'use strict';
2021-07-11 23:29:07 +00:00
bool has_same_name = parameter.binding.visit(
[&](FlyString const& name) {
return name == parameter_name;
},
[&](NonnullRefPtr<BindingPattern> const& bindings) {
bool found_duplicate = false;
bindings->for_each_bound_name([&](auto& bound_name) {
if (bound_name == parameter_name)
found_duplicate = true;
});
return found_duplicate;
});
if (!has_same_name)
LibJS: Implement rules for duplicate function parameters - A regular function can have duplicate parameters except in strict mode or if its parameter list is not "simple" (has a default or rest parameter) - An arrow function can never have duplicate parameters Compared to other engines I opted for more useful syntax error messages than a generic "duplicate parameter name not allowed in this context": "use strict"; function test(foo, foo) {} ^ Uncaught exception: [SyntaxError]: Duplicate parameter 'foo' not allowed in strict mode (line: 1, column: 34) function test(foo, foo = 1) {} ^ Uncaught exception: [SyntaxError]: Duplicate parameter 'foo' not allowed in function with default parameter (line: 1, column: 20) function test(foo, ...foo) {} ^ Uncaught exception: [SyntaxError]: Duplicate parameter 'foo' not allowed in function with rest parameter (line: 1, column: 23) (foo, foo) => {} ^ Uncaught exception: [SyntaxError]: Duplicate parameter 'foo' not allowed in arrow function (line: 1, column: 7)
2020-10-25 11:14:04 +00:00
continue;
LibJS: Handle strict mode for functions more correctly If a function is strict (has 'use strict' directive) it cannot have bindings, cannot have duplicated parameter names and cannot have some reserved keywords and identifiers as parameter names. The logic partly applies depending on whether we are already in strict mode or the function contains 'use strict';
2021-07-11 23:29:07 +00:00
LibJS: Implement rules for duplicate function parameters - A regular function can have duplicate parameters except in strict mode or if its parameter list is not "simple" (has a default or rest parameter) - An arrow function can never have duplicate parameters Compared to other engines I opted for more useful syntax error messages than a generic "duplicate parameter name not allowed in this context": "use strict"; function test(foo, foo) {} ^ Uncaught exception: [SyntaxError]: Duplicate parameter 'foo' not allowed in strict mode (line: 1, column: 34) function test(foo, foo = 1) {} ^ Uncaught exception: [SyntaxError]: Duplicate parameter 'foo' not allowed in function with default parameter (line: 1, column: 20) function test(foo, ...foo) {} ^ Uncaught exception: [SyntaxError]: Duplicate parameter 'foo' not allowed in function with rest parameter (line: 1, column: 23) (foo, foo) => {} ^ Uncaught exception: [SyntaxError]: Duplicate parameter 'foo' not allowed in arrow function (line: 1, column: 7)
2020-10-25 11:14:04 +00:00
String message;
if (parse_options & FunctionNodeParseOptions::IsArrowFunction)
message = String::formatted("Duplicate parameter '{}' not allowed in arrow function", parameter_name);
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
else if (m_state.strict_mode)
LibJS: Implement rules for duplicate function parameters - A regular function can have duplicate parameters except in strict mode or if its parameter list is not "simple" (has a default or rest parameter) - An arrow function can never have duplicate parameters Compared to other engines I opted for more useful syntax error messages than a generic "duplicate parameter name not allowed in this context": "use strict"; function test(foo, foo) {} ^ Uncaught exception: [SyntaxError]: Duplicate parameter 'foo' not allowed in strict mode (line: 1, column: 34) function test(foo, foo = 1) {} ^ Uncaught exception: [SyntaxError]: Duplicate parameter 'foo' not allowed in function with default parameter (line: 1, column: 20) function test(foo, ...foo) {} ^ Uncaught exception: [SyntaxError]: Duplicate parameter 'foo' not allowed in function with rest parameter (line: 1, column: 23) (foo, foo) => {} ^ Uncaught exception: [SyntaxError]: Duplicate parameter 'foo' not allowed in arrow function (line: 1, column: 7)
2020-10-25 11:14:04 +00:00
message = String::formatted("Duplicate parameter '{}' not allowed in strict mode", parameter_name);
else if (has_default_parameter || match(TokenType::Equals))
message = String::formatted("Duplicate parameter '{}' not allowed in function with default parameter", parameter_name);
else if (has_rest_parameter)
message = String::formatted("Duplicate parameter '{}' not allowed in function with rest parameter", parameter_name);
if (!message.is_empty())
LibJS: Replace 'size_t line, size_t column' with 'Optional<Position>' This is a bit nicer for two reasons: - The absence of line number/column information isn't based on 'values are zero' anymore but on Optional's value - When reporting syntax errors with position information other than the current token's position we had to store line and column ourselves, like this: auto foo_start_line = m_parser_state.m_current_token.line_number(); auto foo_start_column = m_parser_state.m_current_token.line_column(); ... syntax_error("...", foo_start_line, foo_start_column); Which now becomes: auto foo_start= position(); ... syntax_error("...", foo_start); This makes it easier to report correct positions for syntax errors that only emerge a few tokens later :^)
2020-11-02 21:03:19 +00:00
syntax_error(message, Position { token.line_number(), token.line_column() });
LibJS: Implement rules for duplicate function parameters - A regular function can have duplicate parameters except in strict mode or if its parameter list is not "simple" (has a default or rest parameter) - An arrow function can never have duplicate parameters Compared to other engines I opted for more useful syntax error messages than a generic "duplicate parameter name not allowed in this context": "use strict"; function test(foo, foo) {} ^ Uncaught exception: [SyntaxError]: Duplicate parameter 'foo' not allowed in strict mode (line: 1, column: 34) function test(foo, foo = 1) {} ^ Uncaught exception: [SyntaxError]: Duplicate parameter 'foo' not allowed in function with default parameter (line: 1, column: 20) function test(foo, ...foo) {} ^ Uncaught exception: [SyntaxError]: Duplicate parameter 'foo' not allowed in function with rest parameter (line: 1, column: 23) (foo, foo) => {} ^ Uncaught exception: [SyntaxError]: Duplicate parameter 'foo' not allowed in arrow function (line: 1, column: 7)
2020-10-25 11:14:04 +00:00
break;
}
LibJS: Implement destructuring assignments and function parameters
2021-05-29 11:33:19 +00:00
return FlyString { token.value() };
LibJS: Implement rules for duplicate function parameters - A regular function can have duplicate parameters except in strict mode or if its parameter list is not "simple" (has a default or rest parameter) - An arrow function can never have duplicate parameters Compared to other engines I opted for more useful syntax error messages than a generic "duplicate parameter name not allowed in this context": "use strict"; function test(foo, foo) {} ^ Uncaught exception: [SyntaxError]: Duplicate parameter 'foo' not allowed in strict mode (line: 1, column: 34) function test(foo, foo = 1) {} ^ Uncaught exception: [SyntaxError]: Duplicate parameter 'foo' not allowed in function with default parameter (line: 1, column: 20) function test(foo, ...foo) {} ^ Uncaught exception: [SyntaxError]: Duplicate parameter 'foo' not allowed in function with rest parameter (line: 1, column: 23) (foo, foo) => {} ^ Uncaught exception: [SyntaxError]: Duplicate parameter 'foo' not allowed in arrow function (line: 1, column: 7)
2020-10-25 11:14:04 +00:00
};
LibJS: Rework Identifier parsing to match the spec more closely
2021-07-11 11:04:55 +00:00
while (match(TokenType::CurlyOpen) || match(TokenType::BracketOpen) || match_identifier() || match(TokenType::TripleDot)) {
LibJS: Move checks for invalid getter/setter params to parse_function_node This allows us to provide better error messages as we can point the syntax error location to the exact first invalid parameter instead of always the end of the function within a object literal or class definition. Before this change: const Foo = { set bar() {} } ^ Uncaught exception: [SyntaxError]: Object setter property must have one argument (line: 1, column: 28) class Foo { set bar() {} } ^ Uncaught exception: [SyntaxError]: Class setter method must have one argument (line: 1, column: 26) After this change: const Foo = { set bar() {} } ^ Uncaught exception: [SyntaxError]: Setter function must have one argument (line: 1, column: 23) class Foo { set bar() {} } ^ Uncaught exception: [SyntaxError]: Setter function must have one argument (line: 1, column: 21) The only possible downside of this change is that class getters/setters and functions in objects are not distinguished in the message anymore - I don't think that's important though, and classes are (mostly) just syntactic sugar anyway.
2020-10-20 17:32:51 +00:00
if (parse_options & FunctionNodeParseOptions::IsGetterFunction)
syntax_error("Getter function must have no arguments");
LibJS: Rest parameter in setter functions is a syntax error
2020-10-20 17:43:58 +00:00
if (parse_options & FunctionNodeParseOptions::IsSetterFunction && (parameters.size() >= 1 || match(TokenType::TripleDot)))
LibJS: Move checks for invalid getter/setter params to parse_function_node This allows us to provide better error messages as we can point the syntax error location to the exact first invalid parameter instead of always the end of the function within a object literal or class definition. Before this change: const Foo = { set bar() {} } ^ Uncaught exception: [SyntaxError]: Object setter property must have one argument (line: 1, column: 28) class Foo { set bar() {} } ^ Uncaught exception: [SyntaxError]: Class setter method must have one argument (line: 1, column: 26) After this change: const Foo = { set bar() {} } ^ Uncaught exception: [SyntaxError]: Setter function must have one argument (line: 1, column: 23) class Foo { set bar() {} } ^ Uncaught exception: [SyntaxError]: Setter function must have one argument (line: 1, column: 21) The only possible downside of this change is that class getters/setters and functions in objects are not distinguished in the message anymore - I don't think that's important though, and classes are (mostly) just syntactic sugar anyway.
2020-10-20 17:32:51 +00:00
syntax_error("Setter function must have one argument");
LibJS: Implement destructuring assignments and function parameters
2021-05-29 11:33:19 +00:00
auto is_rest = false;
LibJS: Implement rest parameters
2020-05-04 15:05:13 +00:00
if (match(TokenType::TripleDot)) {
consume();
LibJS: Implement rules for duplicate function parameters - A regular function can have duplicate parameters except in strict mode or if its parameter list is not "simple" (has a default or rest parameter) - An arrow function can never have duplicate parameters Compared to other engines I opted for more useful syntax error messages than a generic "duplicate parameter name not allowed in this context": "use strict"; function test(foo, foo) {} ^ Uncaught exception: [SyntaxError]: Duplicate parameter 'foo' not allowed in strict mode (line: 1, column: 34) function test(foo, foo = 1) {} ^ Uncaught exception: [SyntaxError]: Duplicate parameter 'foo' not allowed in function with default parameter (line: 1, column: 20) function test(foo, ...foo) {} ^ Uncaught exception: [SyntaxError]: Duplicate parameter 'foo' not allowed in function with rest parameter (line: 1, column: 23) (foo, foo) => {} ^ Uncaught exception: [SyntaxError]: Duplicate parameter 'foo' not allowed in arrow function (line: 1, column: 7)
2020-10-25 11:14:04 +00:00
has_rest_parameter = true;
LibJS: Function.length respects default and rest parameters "[Function.length is] the number of formal parameters. This number excludes the rest parameter and only includes parameters before the first one with a default value." - MDN
2020-05-06 03:02:14 +00:00
function_length = parameters.size();
LibJS: Implement destructuring assignments and function parameters
2021-05-29 11:33:19 +00:00
is_rest = true;
LibJS: Implement rest parameters
2020-05-04 15:05:13 +00:00
}
LibJS: Implement destructuring assignments and function parameters
2021-05-29 11:33:19 +00:00
auto parameter = consume_identifier_or_binding_pattern();
LibJS: Add function default arguments Adds the ability for function arguments to have default values. This works for standard functions as well as arrow functions. Default values are not printed in a <function>.toString() call, as nodes cannot print their source string representation.
2020-05-02 18:46:39 +00:00
RefPtr<Expression> default_value;
if (match(TokenType::Equals)) {
LibJS: Implement rules for duplicate function parameters - A regular function can have duplicate parameters except in strict mode or if its parameter list is not "simple" (has a default or rest parameter) - An arrow function can never have duplicate parameters Compared to other engines I opted for more useful syntax error messages than a generic "duplicate parameter name not allowed in this context": "use strict"; function test(foo, foo) {} ^ Uncaught exception: [SyntaxError]: Duplicate parameter 'foo' not allowed in strict mode (line: 1, column: 34) function test(foo, foo = 1) {} ^ Uncaught exception: [SyntaxError]: Duplicate parameter 'foo' not allowed in function with default parameter (line: 1, column: 20) function test(foo, ...foo) {} ^ Uncaught exception: [SyntaxError]: Duplicate parameter 'foo' not allowed in function with rest parameter (line: 1, column: 23) (foo, foo) => {} ^ Uncaught exception: [SyntaxError]: Duplicate parameter 'foo' not allowed in arrow function (line: 1, column: 7)
2020-10-25 11:14:04 +00:00
consume();
LibJS: Tighten default values in formal parameter parsing Disallow default parameter for rest parameters. Disallow yield expressions as default values.
2021-07-28 23:47:36 +00:00
if (is_rest)
syntax_error("Rest parameter may not have a default initializer");
LibJS: Treat default parameter values as being in function context e.g. `new.target` should be permitted in `function foo(x = new.target) {}`.
2021-07-10 18:18:25 +00:00
TemporaryChange change(m_state.in_function_context, true);
LibJS: Implement rules for duplicate function parameters - A regular function can have duplicate parameters except in strict mode or if its parameter list is not "simple" (has a default or rest parameter) - An arrow function can never have duplicate parameters Compared to other engines I opted for more useful syntax error messages than a generic "duplicate parameter name not allowed in this context": "use strict"; function test(foo, foo) {} ^ Uncaught exception: [SyntaxError]: Duplicate parameter 'foo' not allowed in strict mode (line: 1, column: 34) function test(foo, foo = 1) {} ^ Uncaught exception: [SyntaxError]: Duplicate parameter 'foo' not allowed in function with default parameter (line: 1, column: 20) function test(foo, ...foo) {} ^ Uncaught exception: [SyntaxError]: Duplicate parameter 'foo' not allowed in function with rest parameter (line: 1, column: 23) (foo, foo) => {} ^ Uncaught exception: [SyntaxError]: Duplicate parameter 'foo' not allowed in arrow function (line: 1, column: 7)
2020-10-25 11:14:04 +00:00
has_default_parameter = true;
LibJS: Function.length respects default and rest parameters "[Function.length is] the number of formal parameters. This number excludes the rest parameter and only includes parameters before the first one with a default value." - MDN
2020-05-06 03:02:14 +00:00
function_length = parameters.size();
LibJS: Parse comma operator into SequenceExpression
2020-05-11 16:27:31 +00:00
default_value = parse_expression(2);
LibJS: Disallow 'yield' identifier initializer in GeneratorFunctions
2021-06-17 17:14:45 +00:00
bool is_generator = parse_options & FunctionNodeParseOptions::IsGeneratorFunction;
LibJS: Be more strict about reserved and special identifiers
2021-07-11 23:27:35 +00:00
if ((is_generator || m_state.strict_mode) && default_value && default_value->fast_is<Identifier>() && static_cast<Identifier&>(*default_value).string() == "yield"sv)
LibJS: Disallow 'yield' identifier initializer in GeneratorFunctions
2021-06-17 17:14:45 +00:00
syntax_error("Generator function parameter initializer cannot contain a reference to an identifier named \"yield\"");
LibJS: Add function default arguments Adds the ability for function arguments to have default values. This works for standard functions as well as arrow functions. Default values are not printed in a <function>.toString() call, as nodes cannot print their source string representation.
2020-05-02 18:46:39 +00:00
}
LibJS: Implement destructuring assignments and function parameters
2021-05-29 11:33:19 +00:00
parameters.append({ move(parameter), default_value, is_rest });
LibJS: Disallow comma after rest parameter in formal parameters
2021-09-27 20:29:49 +00:00
if (match(TokenType::ParenClose) || is_rest)
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
break;
consume(TokenType::Comma);
}
LibJS: Move checks for invalid getter/setter params to parse_function_node This allows us to provide better error messages as we can point the syntax error location to the exact first invalid parameter instead of always the end of the function within a object literal or class definition. Before this change: const Foo = { set bar() {} } ^ Uncaught exception: [SyntaxError]: Object setter property must have one argument (line: 1, column: 28) class Foo { set bar() {} } ^ Uncaught exception: [SyntaxError]: Class setter method must have one argument (line: 1, column: 26) After this change: const Foo = { set bar() {} } ^ Uncaught exception: [SyntaxError]: Setter function must have one argument (line: 1, column: 23) class Foo { set bar() {} } ^ Uncaught exception: [SyntaxError]: Setter function must have one argument (line: 1, column: 21) The only possible downside of this change is that class getters/setters and functions in objects are not distinguished in the message anymore - I don't think that's important though, and classes are (mostly) just syntactic sugar anyway.
2020-10-20 17:32:51 +00:00
if (parse_options & FunctionNodeParseOptions::IsSetterFunction && parameters.is_empty())
syntax_error("Setter function must have one argument");
LibJS: Share parameter parsing between regular and arrow functions This simplifies try_parse_arrow_function_expression() and fixes a few cases that should not produce an arrow function AST but did: (a,,) => {} (a b) => {} (a ...b) => {} (...b a) => {} The new parsing logic checks whether parens are expected and uses parse_function_parameters() if so, rolling back if a new syntax error occurs during that. Otherwise it's just an identifier in which case we parse the single parameter ourselves.
2020-10-18 23:26:41 +00:00
return parameters;
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
}
LibJS: Rework Identifier parsing to match the spec more closely
2021-07-11 11:04:55 +00:00
static constexpr AK::Array<StringView, 36> s_reserved_words = { "break", "case", "catch", "class", "const", "continue", "debugger", "default", "delete", "do", "else", "enum", "export", "extends", "false", "finally", "for", "function", "if", "import", "in", "instanceof", "new", "null", "return", "super", "switch", "this", "throw", "true", "try", "typeof", "var", "void", "while", "with" };
LibJS: Add optional extra strict checks in parse_binding_pattern
2021-07-28 23:49:25 +00:00
LibJS: Allow member expressions in binding patterns Also allows literal string and numbers as property names in object binding patterns.
2021-09-17 23:11:32 +00:00
RefPtr<BindingPattern> Parser::parse_binding_pattern(Parser::AllowDuplicates allow_duplicates, Parser::AllowMemberExpressions allow_member_expressions)
LibJS: Implement destructuring assignments and function parameters
2021-05-29 11:33:19 +00:00
{
auto rule_start = push_start();
TokenType closing_token;
LibJS: Restructure and fully implement BindingPatterns
2021-06-13 01:04:28 +00:00
bool is_object = true;
LibJS: Implement destructuring assignments and function parameters
2021-05-29 11:33:19 +00:00
if (match(TokenType::BracketOpen)) {
consume();
closing_token = TokenType::BracketClose;
LibJS: Restructure and fully implement BindingPatterns
2021-06-13 01:04:28 +00:00
is_object = false;
LibJS: Implement destructuring assignments and function parameters
2021-05-29 11:33:19 +00:00
} else if (match(TokenType::CurlyOpen)) {
consume();
closing_token = TokenType::CurlyClose;
} else {
return {};
}
LibJS: Restructure and fully implement BindingPatterns
2021-06-13 01:04:28 +00:00
Vector<BindingPattern::BindingEntry> entries;
LibJS: Implement destructuring assignments and function parameters
2021-05-29 11:33:19 +00:00
while (!match(closing_token)) {
LibJS: Restructure and fully implement BindingPatterns
2021-06-13 01:04:28 +00:00
if (!is_object && match(TokenType::Comma)) {
LibJS: Implement destructuring assignments and function parameters
2021-05-29 11:33:19 +00:00
consume();
LibJS: Restructure and fully implement BindingPatterns
2021-06-13 01:04:28 +00:00
entries.append(BindingPattern::BindingEntry {});
continue;
}
LibJS: Implement destructuring assignments and function parameters
2021-05-29 11:33:19 +00:00
auto is_rest = false;
if (match(TokenType::TripleDot)) {
consume();
is_rest = true;
}
LibJS: Restructure and fully implement BindingPatterns
2021-06-13 01:04:28 +00:00
decltype(BindingPattern::BindingEntry::name) name = Empty {};
decltype(BindingPattern::BindingEntry::alias) alias = Empty {};
RefPtr<Expression> initializer = {};
LibJS: Implement destructuring assignments and function parameters
2021-05-29 11:33:19 +00:00
LibJS: Restructure and fully implement BindingPatterns
2021-06-13 01:04:28 +00:00
if (is_object) {
LibJS: Allow member expressions in binding patterns Also allows literal string and numbers as property names in object binding patterns.
2021-09-17 23:11:32 +00:00
bool needs_alias = false;
if (allow_member_expressions == AllowMemberExpressions::Yes && is_rest) {
auto expression_position = position();
auto expression = parse_expression(2, Associativity::Right, { TokenType::Equals });
if (is<MemberExpression>(*expression))
alias = static_ptr_cast<MemberExpression>(expression);
else if (is<Identifier>(*expression))
name = static_ptr_cast<Identifier>(expression);
else
syntax_error("Invalid destructuring assignment target", expression_position);
} else if (match_identifier_name() || match(TokenType::StringLiteral) || match(TokenType::NumericLiteral)) {
if (match(TokenType::StringLiteral) || match(TokenType::NumericLiteral))
needs_alias = true;
if (match(TokenType::StringLiteral)) {
auto token = consume(TokenType::StringLiteral);
auto string_literal = parse_string_literal(token);
name = create_ast_node<Identifier>(
{ m_state.current_token.filename(), rule_start.position(), position() },
string_literal->value());
} else {
name = create_ast_node<Identifier>(
{ m_state.current_token.filename(), rule_start.position(), position() },
consume().value());
}
LibJS: Restructure and fully implement BindingPatterns
2021-06-13 01:04:28 +00:00
} else if (match(TokenType::BracketOpen)) {
LibJS: Implement destructuring assignments and function parameters
2021-05-29 11:33:19 +00:00
consume();
LibJS: Add optional extra strict checks in parse_binding_pattern
2021-07-28 23:49:25 +00:00
auto expression = parse_expression(0);
name = move(expression);
LibJS: Fix computed property ending token in binding pattern parsing The syntax is supposed to be '[expression]', not '[expression['.
2021-07-11 10:19:42 +00:00
consume(TokenType::BracketClose);
LibJS: Restructure and fully implement BindingPatterns
2021-06-13 01:04:28 +00:00
} else {
LibJS: Use expected() instead of syntax_error("Expected ...")
2021-07-11 19:00:55 +00:00
expected("identifier or computed property name");
LibJS: Restructure and fully implement BindingPatterns
2021-06-13 01:04:28 +00:00
return {};
LibJS: Implement destructuring assignments and function parameters
2021-05-29 11:33:19 +00:00
}
LibJS: Restructure and fully implement BindingPatterns
2021-06-13 01:04:28 +00:00
if (!is_rest && match(TokenType::Colon)) {
LibJS: Implement destructuring assignments and function parameters
2021-05-29 11:33:19 +00:00
consume();
LibJS: Allow member expressions in binding patterns Also allows literal string and numbers as property names in object binding patterns.
2021-09-17 23:11:32 +00:00
if (allow_member_expressions == AllowMemberExpressions::Yes) {
auto expression_position = position();
auto expression = parse_expression(2, Associativity::Right, { TokenType::Equals });
if (is<ArrayExpression>(*expression) || is<ObjectExpression>(*expression)) {
if (auto synthesized_binding_pattern = synthesize_binding_pattern(*expression))
alias = synthesized_binding_pattern.release_nonnull();
else
syntax_error("Invalid destructuring assignment target", expression_position);
} else if (is<MemberExpression>(*expression)) {
alias = static_ptr_cast<MemberExpression>(expression);
} else if (is<Identifier>(*expression)) {
alias = static_ptr_cast<Identifier>(expression);
} else {
syntax_error("Invalid destructuring assignment target", expression_position);
}
} else if (match(TokenType::CurlyOpen) || match(TokenType::BracketOpen)) {
auto binding_pattern = parse_binding_pattern(allow_duplicates, allow_member_expressions);
LibJS: Restructure and fully implement BindingPatterns
2021-06-13 01:04:28 +00:00
if (!binding_pattern)
return {};
alias = binding_pattern.release_nonnull();
} else if (match_identifier_name()) {
LibJS: Rework Identifier parsing to match the spec more closely
2021-07-11 11:04:55 +00:00
alias = create_ast_node<Identifier>(
{ m_state.current_token.filename(), rule_start.position(), position() },
LibJS: Allow member expressions in binding patterns Also allows literal string and numbers as property names in object binding patterns.
2021-09-17 23:11:32 +00:00
consume().value());
LibJS: Restructure and fully implement BindingPatterns
2021-06-13 01:04:28 +00:00
} else {
LibJS: Use expected() instead of syntax_error("Expected ...")
2021-07-11 19:00:55 +00:00
expected("identifier or binding pattern");
LibJS: Restructure and fully implement BindingPatterns
2021-06-13 01:04:28 +00:00
return {};
}
LibJS: Allow member expressions in binding patterns Also allows literal string and numbers as property names in object binding patterns.
2021-09-17 23:11:32 +00:00
} else if (needs_alias) {
expected("alias for string or numeric literal name");
return {};
LibJS: Restructure and fully implement BindingPatterns
2021-06-13 01:04:28 +00:00
}
} else {
LibJS: Allow member expressions in binding patterns Also allows literal string and numbers as property names in object binding patterns.
2021-09-17 23:11:32 +00:00
if (allow_member_expressions == AllowMemberExpressions::Yes) {
auto expression_position = position();
auto expression = parse_expression(2, Associativity::Right, { TokenType::Equals });
if (is<ArrayExpression>(*expression) || is<ObjectExpression>(*expression)) {
if (auto synthesized_binding_pattern = synthesize_binding_pattern(*expression))
alias = synthesized_binding_pattern.release_nonnull();
else
syntax_error("Invalid destructuring assignment target", expression_position);
} else if (is<MemberExpression>(*expression)) {
alias = static_ptr_cast<MemberExpression>(expression);
} else if (is<Identifier>(*expression)) {
alias = static_ptr_cast<Identifier>(expression);
} else {
syntax_error("Invalid destructuring assignment target", expression_position);
LibJS: Add optional extra strict checks in parse_binding_pattern
2021-07-28 23:49:25 +00:00
}
LibJS: Restructure and fully implement BindingPatterns
2021-06-13 01:04:28 +00:00
} else if (match(TokenType::BracketOpen) || match(TokenType::CurlyOpen)) {
LibJS: Allow member expressions in binding patterns Also allows literal string and numbers as property names in object binding patterns.
2021-09-17 23:11:32 +00:00
auto pattern = parse_binding_pattern(allow_duplicates, allow_member_expressions);
LibJS: Restructure and fully implement BindingPatterns
2021-06-13 01:04:28 +00:00
if (!pattern) {
LibJS: Use expected() instead of syntax_error("Expected ...")
2021-07-11 19:00:55 +00:00
expected("binding pattern");
LibJS: Restructure and fully implement BindingPatterns
2021-06-13 01:04:28 +00:00
return {};
}
alias = pattern.release_nonnull();
LibJS: Allow member expressions in binding patterns Also allows literal string and numbers as property names in object binding patterns.
2021-09-17 23:11:32 +00:00
} else if (match_identifier_name()) {
// BindingElement must always have an Empty name field
auto identifier_name = consume_identifier().value();
alias = create_ast_node<Identifier>(
{ m_state.current_token.filename(), rule_start.position(), position() },
identifier_name);
LibJS: Restructure and fully implement BindingPatterns
2021-06-13 01:04:28 +00:00
} else {
LibJS: Use expected() instead of syntax_error("Expected ...")
2021-07-11 19:00:55 +00:00
expected("identifier or binding pattern");
LibJS: Restructure and fully implement BindingPatterns
2021-06-13 01:04:28 +00:00
return {};
LibJS: Implement destructuring assignments and function parameters
2021-05-29 11:33:19 +00:00
}
}
LibJS: Restructure and fully implement BindingPatterns
2021-06-13 01:04:28 +00:00
if (match(TokenType::Equals)) {
if (is_rest) {
syntax_error("Unexpected initializer after rest element");
return {};
LibJS: Implement destructuring assignments and function parameters
2021-05-29 11:33:19 +00:00
}
LibJS: Restructure and fully implement BindingPatterns
2021-06-13 01:04:28 +00:00
consume();
initializer = parse_expression(2);
if (!initializer) {
LibJS: Use expected() instead of syntax_error("Expected ...")
2021-07-11 19:00:55 +00:00
expected("initialization expression");
LibJS: Restructure and fully implement BindingPatterns
2021-06-13 01:04:28 +00:00
return {};
}
LibJS: Implement destructuring assignments and function parameters
2021-05-29 11:33:19 +00:00
}
LibJS: Restructure and fully implement BindingPatterns
2021-06-13 01:04:28 +00:00
entries.append(BindingPattern::BindingEntry { move(name), move(alias), move(initializer), is_rest });
if (match(TokenType::Comma)) {
if (is_rest) {
syntax_error("Rest element may not be followed by a comma");
return {};
}
consume();
}
LibJS: Implement destructuring assignments and function parameters
2021-05-29 11:33:19 +00:00
}
LibJS: Restructure and fully implement BindingPatterns
2021-06-13 01:04:28 +00:00
while (!is_object && match(TokenType::Comma))
LibJS: Implement destructuring assignments and function parameters
2021-05-29 11:33:19 +00:00
consume();
consume(closing_token);
LibJS: Restructure and fully implement BindingPatterns
2021-06-13 01:04:28 +00:00
auto kind = is_object ? BindingPattern::Kind::Object : BindingPattern::Kind::Array;
auto pattern = adopt_ref(*new BindingPattern);
pattern->entries = move(entries);
pattern->kind = kind;
LibJS: Add optional extra strict checks in parse_binding_pattern
2021-07-28 23:49:25 +00:00
Vector<StringView> bound_names;
pattern->for_each_bound_name([&](auto& name) {
LibJS: Allow member expressions in binding patterns Also allows literal string and numbers as property names in object binding patterns.
2021-09-17 23:11:32 +00:00
if (allow_duplicates == AllowDuplicates::No) {
LibJS: Add optional extra strict checks in parse_binding_pattern
2021-07-28 23:49:25 +00:00
if (bound_names.contains_slow(name))
syntax_error("Duplicate parameter names in bindings");
bound_names.append(name);
}
check_identifier_name_for_assignment_validity(name);
});
LibJS: Rework Identifier parsing to match the spec more closely
2021-07-11 11:04:55 +00:00
LibJS: Implement destructuring assignments and function parameters
2021-05-29 11:33:19 +00:00
return pattern;
}
LibJS: Require initializer for 'const' variable declaration
2020-10-30 19:08:45 +00:00
NonnullRefPtr<VariableDeclaration> Parser::parse_variable_declaration(bool for_loop_variable_declaration)
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
{
LibJS: Track source positions all the way down to exceptions This makes exceptions have a trace of source positions too, which could probably be helpful in making fancier error tracebacks.
2020-12-28 17:15:22 +00:00
auto rule_start = push_start();
LibJS: rename JS::DeclarationType => JS::DeclarationKind Many other parsers call it with this name. Also Type can be confusing in this context since the DeclarationType is not the type (number, string, etc.) of the variables that are being declared by the VariableDeclaration.
2020-04-08 09:59:18 +00:00
DeclarationKind declaration_kind;
LibJS/Parser: Parse let declarations
2020-03-12 11:16:48 +00:00
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
switch (m_state.current_token.type()) {
LibJS/Parser: Parse let declarations
2020-03-12 11:16:48 +00:00
case TokenType::Var:
LibJS: rename JS::DeclarationType => JS::DeclarationKind Many other parsers call it with this name. Also Type can be confusing in this context since the DeclarationType is not the type (number, string, etc.) of the variables that are being declared by the VariableDeclaration.
2020-04-08 09:59:18 +00:00
declaration_kind = DeclarationKind::Var;
LibJS/Parser: Parse let declarations
2020-03-12 11:16:48 +00:00
break;
case TokenType::Let:
LibJS: rename JS::DeclarationType => JS::DeclarationKind Many other parsers call it with this name. Also Type can be confusing in this context since the DeclarationType is not the type (number, string, etc.) of the variables that are being declared by the VariableDeclaration.
2020-04-08 09:59:18 +00:00
declaration_kind = DeclarationKind::Let;
LibJS/Parser: Parse let declarations
2020-03-12 11:16:48 +00:00
break;
LibJS: Implement const variable declarations This also tightens the means of redeclaration of a variable by proxy, since we now have a way of knowing how a variable was initially declared, we can check if it was declared using `let` or `const` and not tolerate redeclaration like we did previously.
2020-03-12 12:24:34 +00:00
case TokenType::Const:
LibJS: rename JS::DeclarationType => JS::DeclarationKind Many other parsers call it with this name. Also Type can be confusing in this context since the DeclarationType is not the type (number, string, etc.) of the variables that are being declared by the VariableDeclaration.
2020-04-08 09:59:18 +00:00
declaration_kind = DeclarationKind::Const;
LibJS: Implement const variable declarations This also tightens the means of redeclaration of a variable by proxy, since we now have a way of knowing how a variable was initially declared, we can check if it was declared using `let` or `const` and not tolerate redeclaration like we did previously.
2020-03-12 12:24:34 +00:00
break;
LibJS/Parser: Parse let declarations
2020-03-12 11:16:48 +00:00
default:
Everywhere: Rename ASSERT => VERIFY (...and ASSERT_NOT_REACHED => VERIFY_NOT_REACHED) Since all of these checks are done in release builds as well, let's rename them to VERIFY to prevent confusion, as everyone is used to assertions being compiled out in release. We can introduce a new ASSERT macro that is specifically for debug checks, but I'm doing this wholesale conversion first since we've accumulated thousands of these already, and it's not immediately obvious which ones are suitable for ASSERT.
2021-02-23 19:42:32 +00:00
VERIFY_NOT_REACHED();
LibJS/Parser: Parse let declarations
2020-03-12 11:16:48 +00:00
}
LibJS: Require initializer for 'const' variable declaration
2020-10-30 19:08:45 +00:00
consume();
LibJS: Support VariableDeclaration with multiple declarators This patch adds support in the parser and interpreter for this: var a = 1, b = 2, c = a + b; VariableDeclaration is now a sequence of VariableDeclarators. :^)
2020-04-04 19:46:25 +00:00
NonnullRefPtrVector<VariableDeclarator> declarations;
for (;;) {
Libraries: Use AK::Variant default initialization where appropriate
2021-09-19 21:00:45 +00:00
Variant<NonnullRefPtr<Identifier>, NonnullRefPtr<BindingPattern>, Empty> target {};
LibJS: Rework Identifier parsing to match the spec more closely
2021-07-11 11:04:55 +00:00
if (match_identifier()) {
LibJS: Disallow duplicated variable declarations
2021-07-11 23:29:47 +00:00
auto identifier_start = push_start();
LibJS: Rework Identifier parsing to match the spec more closely
2021-07-11 11:04:55 +00:00
auto name = consume_identifier().value();
LibJS: Implement destructuring assignments and function parameters
2021-05-29 11:33:19 +00:00
target = create_ast_node<Identifier>(
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
{ m_state.current_token.filename(), rule_start.position(), position() },
LibJS: Rework Identifier parsing to match the spec more closely
2021-07-11 11:04:55 +00:00
name);
LibJS: Be more strict about reserved and special identifiers
2021-07-11 23:27:35 +00:00
check_identifier_name_for_assignment_validity(name);
LibJS: Rework Identifier parsing to match the spec more closely
2021-07-11 11:04:55 +00:00
if ((declaration_kind == DeclarationKind::Let || declaration_kind == DeclarationKind::Const) && name == "let"sv)
syntax_error("Lexical binding may not be called 'let'");
LibJS: Allow member expressions in binding patterns Also allows literal string and numbers as property names in object binding patterns.
2021-09-17 23:11:32 +00:00
} else if (auto pattern = parse_binding_pattern(declaration_kind != DeclarationKind::Var ? AllowDuplicates::No : AllowDuplicates::Yes, AllowMemberExpressions::No)) {
LibJS: Implement destructuring assignments and function parameters
2021-05-29 11:33:19 +00:00
target = pattern.release_nonnull();
LibJS: Rework Identifier parsing to match the spec more closely
2021-07-11 11:04:55 +00:00
if ((declaration_kind == DeclarationKind::Let || declaration_kind == DeclarationKind::Const)) {
LibJS: Allow member expressions in binding patterns Also allows literal string and numbers as property names in object binding patterns.
2021-09-17 23:11:32 +00:00
target.get<NonnullRefPtr<BindingPattern>>()->for_each_bound_name([this](auto& name) {
LibJS: Rework Identifier parsing to match the spec more closely
2021-07-11 11:04:55 +00:00
if (name == "let"sv)
syntax_error("Lexical binding may not be called 'let'");
});
}
LibJS: Allow 'yield' as a variable name outside of generator functions
2021-07-02 10:25:34 +00:00
} else if (!m_state.in_generator_function_context && match(TokenType::Yield)) {
LibJS: Be more strict about reserved and special identifiers
2021-07-11 23:27:35 +00:00
if (m_state.strict_mode)
syntax_error("Identifier must not be a reserved word in strict mode ('yield')");
LibJS: Add support for async functions This commit adds support for the most bare bones version of async functions, support for async generator functions, async arrow functions and await expressions are TODO.
2021-11-09 18:39:22 +00:00
target = create_ast_node<Identifier>(
{ m_state.current_token.filename(), rule_start.position(), position() },
consume().value());
} else if (!m_state.in_async_function_context && match(TokenType::Async)) {
if (m_program_type == Program::Type::Module)
syntax_error("Identifier must not be a reserved word in modules ('async')");
LibJS: Allow 'yield' as a variable name outside of generator functions
2021-07-02 10:25:34 +00:00
target = create_ast_node<Identifier>(
{ m_state.current_token.filename(), rule_start.position(), position() },
consume().value());
LibJS: Implement destructuring assignments and function parameters
2021-05-29 11:33:19 +00:00
}
if (target.has<Empty>()) {
LibJS: Use expected() instead of syntax_error("Expected ...")
2021-07-11 19:00:55 +00:00
expected("identifier or a binding pattern");
LibJS: Implement destructuring assignments and function parameters
2021-05-29 11:33:19 +00:00
if (match(TokenType::Comma)) {
consume();
continue;
}
break;
}
LibJS: Support VariableDeclaration with multiple declarators This patch adds support in the parser and interpreter for this: var a = 1, b = 2, c = a + b; VariableDeclaration is now a sequence of VariableDeclarators. :^)
2020-04-04 19:46:25 +00:00
RefPtr<Expression> init;
if (match(TokenType::Equals)) {
consume();
LibJS: Make scoping follow the spec Before this we used an ad-hoc combination of references and 'variables' stored in a hashmap. This worked in most cases but is not spec like. Additionally hoisting, dynamically naming functions and scope analysis was not done properly. This patch fixes all of that by: - Implement BindingInitialization for destructuring assignment. - Implementing a new ScopePusher which tracks the lexical and var scoped declarations. This hoists functions to the top level if no lexical declaration name overlaps. Furthermore we do checking of redeclarations in the ScopePusher now requiring less checks all over the place. - Add methods for parsing the directives and statement lists instead of having that code duplicated in multiple places. This allows declarations to pushed to the appropriate scope more easily. - Remove the non spec way of storing 'variables' in DeclarativeEnvironment and make Reference follow the spec instead of checking both the bindings and 'variables'. - Remove all scoping related things from the Interpreter. And instead use environments as specified by the spec. This also includes fixing that NativeFunctions did not produce a valid FunctionEnvironment which could cause issues with callbacks and eval. All FunctionObjects now have a valid NewFunctionEnvironment implementation. - Remove execute_statements from Interpreter and instead use ASTNode::execute everywhere this simplifies AST.cpp as you no longer need to worry about which method to call. - Make ScopeNodes setup their own environment. This uses four different methods specified by the spec {Block, Function, Eval, Global}DeclarationInstantiation with the annexB extensions. - Implement and use NamedEvaluation where specified. Additionally there are fixes to things exposed by these changes to eval, {for, for-in, for-of} loops and assignment. Finally it also fixes some tests in test-js which where passing before but not now that we have correct behavior :^).
2021-09-22 10:44:56 +00:00
// In a for loop 'in' can be ambiguous so we do not allow it
// 14.7.4 The for Statement, https://tc39.es/ecma262/#prod-ForStatement and 14.7.5 The for-in, for-of, and for-await-of Statements, https://tc39.es/ecma262/#prod-ForInOfStatement
if (for_loop_variable_declaration)
init = parse_expression(2, Associativity::Right, { TokenType::In });
else
init = parse_expression(2);
LibJS: Require initializer for 'const' variable declaration
2020-10-30 19:08:45 +00:00
} else if (!for_loop_variable_declaration && declaration_kind == DeclarationKind::Const) {
syntax_error("Missing initializer in 'const' variable declaration");
LibJS: Allow binding patterns as for in/of targets
2021-07-02 10:25:53 +00:00
} else if (!for_loop_variable_declaration && target.has<NonnullRefPtr<BindingPattern>>()) {
LibJS: Implement destructuring assignments and function parameters
2021-05-29 11:33:19 +00:00
syntax_error("Missing initializer in destructuring assignment");
LibJS: Support VariableDeclaration with multiple declarators This patch adds support in the parser and interpreter for this: var a = 1, b = 2, c = a + b; VariableDeclaration is now a sequence of VariableDeclarators. :^)
2020-04-04 19:46:25 +00:00
}
LibJS: Implement destructuring assignments and function parameters
2021-05-29 11:33:19 +00:00
declarations.append(create_ast_node<VariableDeclarator>(
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
{ m_state.current_token.filename(), rule_start.position(), position() },
AK: Undo bogus Variant::downcast() rename I accidentally renamed these to verify_cast() when doing the global AK::downcast() rename.
2021-06-26 19:22:49 +00:00
move(target).downcast<NonnullRefPtr<Identifier>, NonnullRefPtr<BindingPattern>>(),
LibJS: Implement destructuring assignments and function parameters
2021-05-29 11:33:19 +00:00
move(init)));
LibJS: Support VariableDeclaration with multiple declarators This patch adds support in the parser and interpreter for this: var a = 1, b = 2, c = a + b; VariableDeclaration is now a sequence of VariableDeclarators. :^)
2020-04-04 19:46:25 +00:00
if (match(TokenType::Comma)) {
consume();
continue;
}
break;
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
}
LibJS: Require initializer for 'const' variable declaration
2020-10-30 19:08:45 +00:00
if (!for_loop_variable_declaration)
LibJS: Implement basic for..in and for..of loops
2020-04-21 18:21:26 +00:00
consume_or_insert_semicolon();
LibJS: Implement automatic semicolon insertion
2020-04-17 13:05:58 +00:00
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
auto declaration = create_ast_node<VariableDeclaration>({ m_state.current_token.filename(), rule_start.position(), position() }, declaration_kind, move(declarations));
LibJS: Hoist variable declarations to the nearest relevant scope "var" declarations are hoisted to the nearest function scope, while "let" and "const" are hoisted to the nearest block scope. This is done by the parser, which keeps two scope stacks, one stack for the current var scope and one for the current let/const scope. When the interpreter enters a scope, we walk all of the declarations and insert them into the variable environment. We don't support the temporal dead zone for let/const yet.
2020-04-13 14:42:54 +00:00
return declaration;
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
}
LibJS: Implement "throw" You can now throw an expression to the nearest catcher! :^) To support throwing arbitrary values, I added an Exception class that sits as a wrapper around whatever is thrown. In the future it will be a logical place to store a call stack.
2020-03-24 21:03:50 +00:00
NonnullRefPtr<ThrowStatement> Parser::parse_throw_statement()
{
LibJS: Track source positions all the way down to exceptions This makes exceptions have a trace of source positions too, which could probably be helpful in making fancier error tracebacks.
2020-12-28 17:15:22 +00:00
auto rule_start = push_start();
LibJS: Implement "throw" You can now throw an expression to the nearest catcher! :^) To support throwing arbitrary values, I added an Exception class that sits as a wrapper around whatever is thrown. In the future it will be a logical place to store a call stack.
2020-03-24 21:03:50 +00:00
consume(TokenType::Throw);
LibJS: Implement automatic semicolon insertion
2020-04-17 13:05:58 +00:00
// Automatic semicolon insertion: terminate statement when throw is followed by newline
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
if (m_state.current_token.trivia_contains_line_terminator()) {
LibJS: Add Parser::syntax_error() helper Instead of having fprintf()s all over the place we can now use syntax_error("message") or syntax_error("message", line, column). This takes care of a consistent format, appending a newline and getting the line number and column of the current token if the last two params are omitted.
2020-04-29 20:37:51 +00:00
syntax_error("No line break is allowed between 'throw' and its expression");
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
return create_ast_node<ThrowStatement>({ m_state.current_token.filename(), rule_start.position(), position() }, create_ast_node<ErrorExpression>({ m_state.current_token.filename(), rule_start.position(), position() }));
LibJS: Implement automatic semicolon insertion
2020-04-17 13:05:58 +00:00
}
auto expression = parse_expression(0);
consume_or_insert_semicolon();
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
return create_ast_node<ThrowStatement>({ m_state.current_token.filename(), rule_start.position(), position() }, move(expression));
LibJS: Implement "throw" You can now throw an expression to the nearest catcher! :^) To support throwing arbitrary values, I added an Exception class that sits as a wrapper around whatever is thrown. In the future it will be a logical place to store a call stack.
2020-03-24 21:03:50 +00:00
}
LibJS: Lexer and parser support for "switch" statements
2020-03-29 11:09:54 +00:00
NonnullRefPtr<BreakStatement> Parser::parse_break_statement()
{
LibJS: Track source positions all the way down to exceptions This makes exceptions have a trace of source positions too, which could probably be helpful in making fancier error tracebacks.
2020-12-28 17:15:22 +00:00
auto rule_start = push_start();
LibJS: Lexer and parser support for "switch" statements
2020-03-29 11:09:54 +00:00
consume(TokenType::Break);
LibJS: Parse labels in continue and break statements
2020-05-28 19:02:32 +00:00
FlyString target_label;
if (match(TokenType::Semicolon)) {
consume();
LibJS: Disallow 'continue' & 'break' outside of their respective scopes 'continue' is no longer allowed outside of a loop, and an unlabeled 'break' is not longer allowed outside of a loop or switch statement. Labeled 'break' statements are still allowed everywhere, even if the label does not exist.
2020-10-07 19:21:15 +00:00
} else {
LibJS: Allow multiple labels on the same statement Since there are only a number of statements where labels can actually be used we now also only store labels when necessary. Also now tracks the first continue usage of a label since this might not be valid but that can only be determined after we have parsed the statement. Also ensures the correct error does not get wiped by load_state.
2021-09-18 21:01:54 +00:00
if (!m_state.current_token.trivia_contains_line_terminator() && match_identifier()) {
LibJS: Disallow 'continue' & 'break' outside of their respective scopes 'continue' is no longer allowed outside of a loop, and an unlabeled 'break' is not longer allowed outside of a loop or switch statement. Labeled 'break' statements are still allowed everywhere, even if the label does not exist.
2020-10-07 19:21:15 +00:00
target_label = consume().value();
LibJS: Fix small issues in parser - Fix some places where escaped keywords are (not) allowed. - Be more strict about parameters for functions with 'use strict'. - Fix that expressions statements allowed functions and classes. - Fix that class expressions were not allowed. - Added a new next_token() method for checking the look ahead. - Fix that continue labels could jump to non iterating targets. - Fix that generator functions cannot be declared in if statements.
2021-08-28 15:04:37 +00:00
auto label = m_state.labels_in_scope.find(target_label);
if (label == m_state.labels_in_scope.end())
LibJS: break or continue with nonexistent label is a syntax error
2020-10-08 17:49:08 +00:00
syntax_error(String::formatted("Label '{}' not found", target_label));
}
LibJS: Disallow 'continue' & 'break' outside of their respective scopes 'continue' is no longer allowed outside of a loop, and an unlabeled 'break' is not longer allowed outside of a loop or switch statement. Labeled 'break' statements are still allowed everywhere, even if the label does not exist.
2020-10-07 19:21:15 +00:00
consume_or_insert_semicolon();
LibJS: Parse labels in continue and break statements
2020-05-28 19:02:32 +00:00
}
LibJS: Disallow 'continue' & 'break' outside of their respective scopes 'continue' is no longer allowed outside of a loop, and an unlabeled 'break' is not longer allowed outside of a loop or switch statement. Labeled 'break' statements are still allowed everywhere, even if the label does not exist.
2020-10-07 19:21:15 +00:00
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
if (target_label.is_null() && !m_state.in_break_context)
LibJS: Disallow 'continue' & 'break' outside of their respective scopes 'continue' is no longer allowed outside of a loop, and an unlabeled 'break' is not longer allowed outside of a loop or switch statement. Labeled 'break' statements are still allowed everywhere, even if the label does not exist.
2020-10-07 19:21:15 +00:00
syntax_error("Unlabeled 'break' not allowed outside of a loop or switch statement");
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
return create_ast_node<BreakStatement>({ m_state.current_token.filename(), rule_start.position(), position() }, target_label);
LibJS: Lexer and parser support for "switch" statements
2020-03-29 11:09:54 +00:00
}
LibJS: Add support for "continue" inside "for" statements :^)
2020-04-04 22:22:42 +00:00
NonnullRefPtr<ContinueStatement> Parser::parse_continue_statement()
{
LibJS: Track source positions all the way down to exceptions This makes exceptions have a trace of source positions too, which could probably be helpful in making fancier error tracebacks.
2020-12-28 17:15:22 +00:00
auto rule_start = push_start();
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
if (!m_state.in_continue_context)
LibJS: Disallow 'continue' & 'break' outside of their respective scopes 'continue' is no longer allowed outside of a loop, and an unlabeled 'break' is not longer allowed outside of a loop or switch statement. Labeled 'break' statements are still allowed everywhere, even if the label does not exist.
2020-10-07 19:21:15 +00:00
syntax_error("'continue' not allow outside of a loop");
LibJS: Add support for "continue" inside "for" statements :^)
2020-04-04 22:22:42 +00:00
consume(TokenType::Continue);
LibJS: Parse labels in continue and break statements
2020-05-28 19:02:32 +00:00
FlyString target_label;
if (match(TokenType::Semicolon)) {
consume();
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
return create_ast_node<ContinueStatement>({ m_state.current_token.filename(), rule_start.position(), position() }, target_label);
LibJS: Parse labels in continue and break statements
2020-05-28 19:02:32 +00:00
}
LibJS: Allow multiple labels on the same statement Since there are only a number of statements where labels can actually be used we now also only store labels when necessary. Also now tracks the first continue usage of a label since this might not be valid but that can only be determined after we have parsed the statement. Also ensures the correct error does not get wiped by load_state.
2021-09-18 21:01:54 +00:00
if (!m_state.current_token.trivia_contains_line_terminator() && match_identifier()) {
auto label_position = position();
LibJS: Parse labels in continue and break statements
2020-05-28 19:02:32 +00:00
target_label = consume().value();
LibJS: Fix small issues in parser - Fix some places where escaped keywords are (not) allowed. - Be more strict about parameters for functions with 'use strict'. - Fix that expressions statements allowed functions and classes. - Fix that class expressions were not allowed. - Added a new next_token() method for checking the look ahead. - Fix that continue labels could jump to non iterating targets. - Fix that generator functions cannot be declared in if statements.
2021-08-28 15:04:37 +00:00
auto label = m_state.labels_in_scope.find(target_label);
LibJS: Allow multiple labels on the same statement Since there are only a number of statements where labels can actually be used we now also only store labels when necessary. Also now tracks the first continue usage of a label since this might not be valid but that can only be determined after we have parsed the statement. Also ensures the correct error does not get wiped by load_state.
2021-09-18 21:01:54 +00:00
if (label == m_state.labels_in_scope.end())
LibJS: Fix small issues in parser - Fix some places where escaped keywords are (not) allowed. - Be more strict about parameters for functions with 'use strict'. - Fix that expressions statements allowed functions and classes. - Fix that class expressions were not allowed. - Added a new next_token() method for checking the look ahead. - Fix that continue labels could jump to non iterating targets. - Fix that generator functions cannot be declared in if statements.
2021-08-28 15:04:37 +00:00
syntax_error(String::formatted("Label '{}' not found or invalid", target_label));
LibJS: Allow multiple labels on the same statement Since there are only a number of statements where labels can actually be used we now also only store labels when necessary. Also now tracks the first continue usage of a label since this might not be valid but that can only be determined after we have parsed the statement. Also ensures the correct error does not get wiped by load_state.
2021-09-18 21:01:54 +00:00
else
label->value = label_position;
LibJS: break or continue with nonexistent label is a syntax error
2020-10-08 17:49:08 +00:00
}
LibJS: Implement automatic semicolon insertion
2020-04-17 13:05:58 +00:00
consume_or_insert_semicolon();
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
return create_ast_node<ContinueStatement>({ m_state.current_token.filename(), rule_start.position(), position() }, target_label);
LibJS: Add support for "continue" inside "for" statements :^)
2020-04-04 22:22:42 +00:00
}
LibJS: Implement ConditionalExpression (ternary "?:" operator)
2020-04-03 10:14:28 +00:00
NonnullRefPtr<ConditionalExpression> Parser::parse_conditional_expression(NonnullRefPtr<Expression> test)
{
LibJS: Track source positions all the way down to exceptions This makes exceptions have a trace of source positions too, which could probably be helpful in making fancier error tracebacks.
2020-12-28 17:15:22 +00:00
auto rule_start = push_start();
LibJS: Implement ConditionalExpression (ternary "?:" operator)
2020-04-03 10:14:28 +00:00
consume(TokenType::QuestionMark);
LibJS: Fix conditional expression precedence This fixes the following from parsing incorrectly due to the comma that occurs after the conditional: let o = { foo: true ? 1 : 2, bar: 'baz', };
2020-05-28 21:42:20 +00:00
auto consequent = parse_expression(2);
LibJS: Implement ConditionalExpression (ternary "?:" operator)
2020-04-03 10:14:28 +00:00
consume(TokenType::Colon);
LibJS: Fix conditional expression precedence This fixes the following from parsing incorrectly due to the comma that occurs after the conditional: let o = { foo: true ? 1 : 2, bar: 'baz', };
2020-05-28 21:42:20 +00:00
auto alternate = parse_expression(2);
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
return create_ast_node<ConditionalExpression>({ m_state.current_token.filename(), rule_start.position(), position() }, move(test), move(consequent), move(alternate));
LibJS: Implement ConditionalExpression (ternary "?:" operator)
2020-04-03 10:14:28 +00:00
}
LibJS: Implement parsing and execution of optional chains
2021-09-14 02:26:31 +00:00
NonnullRefPtr<OptionalChain> Parser::parse_optional_chain(NonnullRefPtr<Expression> base)
{
auto rule_start = push_start();
Vector<OptionalChain::Reference> chain;
do {
if (match(TokenType::QuestionMarkPeriod)) {
consume(TokenType::QuestionMarkPeriod);
switch (m_state.current_token.type()) {
case TokenType::ParenOpen:
chain.append(OptionalChain::Call { parse_arguments(), OptionalChain::Mode::Optional });
break;
case TokenType::BracketOpen:
consume();
chain.append(OptionalChain::ComputedReference { parse_expression(0), OptionalChain::Mode::Optional });
consume(TokenType::BracketClose);
break;
LibJS: Implement private identifiers in optional chains
2021-10-18 21:32:47 +00:00
case TokenType::PrivateIdentifier: {
if (!is_private_identifier_valid())
syntax_error(String::formatted("Reference to undeclared private field or method '{}'", m_state.current_token.value()));
auto start = position();
auto private_identifier = consume();
chain.append(OptionalChain::PrivateMemberReference {
create_ast_node<PrivateIdentifier>({ m_state.current_token.filename(), start, position() }, private_identifier.value()),
OptionalChain::Mode::Optional });
break;
}
LibJS: Implement parsing and execution of optional chains
2021-09-14 02:26:31 +00:00
case TokenType::TemplateLiteralStart:
// 13.3.1.1 - Static Semantics: Early Errors
// OptionalChain :
// ?. TemplateLiteral
// OptionalChain TemplateLiteral
// This is a hard error.
syntax_error("Invalid tagged template literal after ?.", position());
break;
default:
if (match_identifier_name()) {
auto start = position();
auto identifier = consume();
chain.append(OptionalChain::MemberReference {
create_ast_node<Identifier>({ m_state.current_token.filename(), start, position() }, identifier.value()),
OptionalChain::Mode::Optional,
});
} else {
syntax_error("Invalid optional chain reference after ?.", position());
}
break;
}
} else if (match(TokenType::ParenOpen)) {
chain.append(OptionalChain::Call { parse_arguments(), OptionalChain::Mode::NotOptional });
} else if (match(TokenType::Period)) {
consume();
LibJS: Implement private identifiers in optional chains
2021-10-18 21:32:47 +00:00
if (match(TokenType::PrivateIdentifier)) {
auto start = position();
auto private_identifier = consume();
chain.append(OptionalChain::PrivateMemberReference {
create_ast_node<PrivateIdentifier>({ m_state.current_token.filename(), start, position() }, private_identifier.value()),
OptionalChain::Mode::NotOptional,
});
} else if (match_identifier_name()) {
LibJS: Implement parsing and execution of optional chains
2021-09-14 02:26:31 +00:00
auto start = position();
auto identifier = consume();
chain.append(OptionalChain::MemberReference {
create_ast_node<Identifier>({ m_state.current_token.filename(), start, position() }, identifier.value()),
OptionalChain::Mode::NotOptional,
});
} else {
expected("an identifier");
break;
}
} else if (match(TokenType::TemplateLiteralStart)) {
// 13.3.1.1 - Static Semantics: Early Errors
// OptionalChain :
// ?. TemplateLiteral
// OptionalChain TemplateLiteral
syntax_error("Invalid tagged template literal after optional chain", position());
break;
} else if (match(TokenType::BracketOpen)) {
consume();
chain.append(OptionalChain::ComputedReference { parse_expression(2), OptionalChain::Mode::NotOptional });
consume(TokenType::BracketClose);
} else {
break;
}
} while (!done());
return create_ast_node<OptionalChain>(
{ m_state.current_token.filename(), rule_start.position(), position() },
move(base),
move(chain));
}
LibJS: Parse "try", "catch" and "finally" This is the first step towards support exceptions. :^)
2020-03-24 13:03:55 +00:00
NonnullRefPtr<TryStatement> Parser::parse_try_statement()
{
LibJS: Track source positions all the way down to exceptions This makes exceptions have a trace of source positions too, which could probably be helpful in making fancier error tracebacks.
2020-12-28 17:15:22 +00:00
auto rule_start = push_start();
LibJS: Parse "try", "catch" and "finally" This is the first step towards support exceptions. :^)
2020-03-24 13:03:55 +00:00
consume(TokenType::Try);
auto block = parse_block_statement();
LibJS: Allow try statement with only finally clause This was a regression introduced by 9ffe45b - a TryStatement without 'catch' clause *is* allowed, if it has a 'finally' clause. It is now checked properly that at least one of both is present.
2020-10-23 19:54:00 +00:00
RefPtr<CatchClause> handler;
if (match(TokenType::Catch))
handler = parse_catch_clause();
LibJS: Parse "try", "catch" and "finally" This is the first step towards support exceptions. :^)
2020-03-24 13:03:55 +00:00
RefPtr<BlockStatement> finalizer;
if (match(TokenType::Finally)) {
consume();
finalizer = parse_block_statement();
}
LibJS: Allow try statement with only finally clause This was a regression introduced by 9ffe45b - a TryStatement without 'catch' clause *is* allowed, if it has a 'finally' clause. It is now checked properly that at least one of both is present.
2020-10-23 19:54:00 +00:00
if (!handler && !finalizer)
syntax_error("try statement must have a 'catch' or 'finally' clause");
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
return create_ast_node<TryStatement>({ m_state.current_token.filename(), rule_start.position(), position() }, move(block), move(handler), move(finalizer));
LibJS: Parse "try", "catch" and "finally" This is the first step towards support exceptions. :^)
2020-03-24 13:03:55 +00:00
}
LibJS: Add support for do..while statements
2020-04-04 19:29:23 +00:00
NonnullRefPtr<DoWhileStatement> Parser::parse_do_while_statement()
{
LibJS: Track source positions all the way down to exceptions This makes exceptions have a trace of source positions too, which could probably be helpful in making fancier error tracebacks.
2020-12-28 17:15:22 +00:00
auto rule_start = push_start();
LibJS: Add support for do..while statements
2020-04-04 19:29:23 +00:00
consume(TokenType::Do);
LibJS: Fix do..while parsing by consuming parentheses explicitly (#1652) Before this patch the parser accepted conditions without enclosing parentheses (like: .."while number < 9").
2020-04-05 14:57:01 +00:00
LibJS: Disallow 'continue' & 'break' outside of their respective scopes 'continue' is no longer allowed outside of a loop, and an unlabeled 'break' is not longer allowed outside of a loop or switch statement. Labeled 'break' statements are still allowed everywhere, even if the label does not exist.
2020-10-07 19:21:15 +00:00
auto body = [&]() -> NonnullRefPtr<Statement> {
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
TemporaryChange break_change(m_state.in_break_context, true);
TemporaryChange continue_change(m_state.in_continue_context, true);
LibJS: Disallow 'continue' & 'break' outside of their respective scopes 'continue' is no longer allowed outside of a loop, and an unlabeled 'break' is not longer allowed outside of a loop or switch statement. Labeled 'break' statements are still allowed everywhere, even if the label does not exist.
2020-10-07 19:21:15 +00:00
return parse_statement();
}();
LibJS: Fix do..while parsing by consuming parentheses explicitly (#1652) Before this patch the parser accepted conditions without enclosing parentheses (like: .."while number < 9").
2020-04-05 14:57:01 +00:00
LibJS: Add support for do..while statements
2020-04-04 19:29:23 +00:00
consume(TokenType::While);
LibJS: Fix do..while parsing by consuming parentheses explicitly (#1652) Before this patch the parser accepted conditions without enclosing parentheses (like: .."while number < 9").
2020-04-05 14:57:01 +00:00
consume(TokenType::ParenOpen);
LibJS: Add support for do..while statements
2020-04-04 19:29:23 +00:00
auto test = parse_expression(0);
LibJS: Fix do..while parsing by consuming parentheses explicitly (#1652) Before this patch the parser accepted conditions without enclosing parentheses (like: .."while number < 9").
2020-04-05 14:57:01 +00:00
consume(TokenType::ParenClose);
LibJS: Always insert semicolon after do-while statement if missing https://tc39.es/ecma262/#sec-additions-and-changes-that-introduce-incompatibilities-with-prior-editions 11.9.1: In ECMAScript 2015, Automatic Semicolon Insertion adds a semicolon at the end of a do-while statement if the semicolon is missing. This change aligns the specification with the actual behaviour of most existing implementations.
2020-10-27 19:16:23 +00:00
// Since ES 2015 a missing semicolon is inserted here, despite the regular ASI rules not applying
if (match(TokenType::Semicolon))
consume();
LibJS: Fix do..while parsing by consuming parentheses explicitly (#1652) Before this patch the parser accepted conditions without enclosing parentheses (like: .."while number < 9").
2020-04-05 14:57:01 +00:00
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
return create_ast_node<DoWhileStatement>({ m_state.current_token.filename(), rule_start.position(), position() }, move(test), move(body));
LibJS: Add support for do..while statements
2020-04-04 19:29:23 +00:00
}
LibJS: Parse while statements
2020-04-21 18:27:57 +00:00
NonnullRefPtr<WhileStatement> Parser::parse_while_statement()
{
LibJS: Track source positions all the way down to exceptions This makes exceptions have a trace of source positions too, which could probably be helpful in making fancier error tracebacks.
2020-12-28 17:15:22 +00:00
auto rule_start = push_start();
LibJS: Parse while statements
2020-04-21 18:27:57 +00:00
consume(TokenType::While);
consume(TokenType::ParenOpen);
auto test = parse_expression(0);
consume(TokenType::ParenClose);
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
TemporaryChange break_change(m_state.in_break_context, true);
TemporaryChange continue_change(m_state.in_continue_context, true);
LibJS: Parse while statements
2020-04-21 18:27:57 +00:00
auto body = parse_statement();
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
return create_ast_node<WhileStatement>({ m_state.current_token.filename(), rule_start.position(), position() }, move(test), move(body));
LibJS: Parse while statements
2020-04-21 18:27:57 +00:00
}
LibJS: Lexer and parser support for "switch" statements
2020-03-29 11:09:54 +00:00
NonnullRefPtr<SwitchStatement> Parser::parse_switch_statement()
{
LibJS: Track source positions all the way down to exceptions This makes exceptions have a trace of source positions too, which could probably be helpful in making fancier error tracebacks.
2020-12-28 17:15:22 +00:00
auto rule_start = push_start();
LibJS: Lexer and parser support for "switch" statements
2020-03-29 11:09:54 +00:00
consume(TokenType::Switch);
consume(TokenType::ParenOpen);
auto determinant = parse_expression(0);
consume(TokenType::ParenClose);
consume(TokenType::CurlyOpen);
NonnullRefPtrVector<SwitchCase> cases;
LibJS: Make scoping follow the spec Before this we used an ad-hoc combination of references and 'variables' stored in a hashmap. This worked in most cases but is not spec like. Additionally hoisting, dynamically naming functions and scope analysis was not done properly. This patch fixes all of that by: - Implement BindingInitialization for destructuring assignment. - Implementing a new ScopePusher which tracks the lexical and var scoped declarations. This hoists functions to the top level if no lexical declaration name overlaps. Furthermore we do checking of redeclarations in the ScopePusher now requiring less checks all over the place. - Add methods for parsing the directives and statement lists instead of having that code duplicated in multiple places. This allows declarations to pushed to the appropriate scope more easily. - Remove the non spec way of storing 'variables' in DeclarativeEnvironment and make Reference follow the spec instead of checking both the bindings and 'variables'. - Remove all scoping related things from the Interpreter. And instead use environments as specified by the spec. This also includes fixing that NativeFunctions did not produce a valid FunctionEnvironment which could cause issues with callbacks and eval. All FunctionObjects now have a valid NewFunctionEnvironment implementation. - Remove execute_statements from Interpreter and instead use ASTNode::execute everywhere this simplifies AST.cpp as you no longer need to worry about which method to call. - Make ScopeNodes setup their own environment. This uses four different methods specified by the spec {Block, Function, Eval, Global}DeclarationInstantiation with the annexB extensions. - Implement and use NamedEvaluation where specified. Additionally there are fixes to things exposed by these changes to eval, {for, for-in, for-of} loops and assignment. Finally it also fixes some tests in test-js which where passing before but not now that we have correct behavior :^).
2021-09-22 10:44:56 +00:00
auto switch_statement = create_ast_node<SwitchStatement>({ m_state.current_token.filename(), rule_start.position(), position() }, move(determinant));
ScopePusher switch_scope = ScopePusher::block_scope(*this, switch_statement);
LibJS: Multiple 'default' clauses in switch statement are a syntax error
2020-10-18 22:12:27 +00:00
auto has_default = false;
while (match(TokenType::Case) || match(TokenType::Default)) {
if (match(TokenType::Default)) {
if (has_default)
syntax_error("Multiple 'default' clauses in switch statement");
has_default = true;
}
LibJS: Make scoping follow the spec Before this we used an ad-hoc combination of references and 'variables' stored in a hashmap. This worked in most cases but is not spec like. Additionally hoisting, dynamically naming functions and scope analysis was not done properly. This patch fixes all of that by: - Implement BindingInitialization for destructuring assignment. - Implementing a new ScopePusher which tracks the lexical and var scoped declarations. This hoists functions to the top level if no lexical declaration name overlaps. Furthermore we do checking of redeclarations in the ScopePusher now requiring less checks all over the place. - Add methods for parsing the directives and statement lists instead of having that code duplicated in multiple places. This allows declarations to pushed to the appropriate scope more easily. - Remove the non spec way of storing 'variables' in DeclarativeEnvironment and make Reference follow the spec instead of checking both the bindings and 'variables'. - Remove all scoping related things from the Interpreter. And instead use environments as specified by the spec. This also includes fixing that NativeFunctions did not produce a valid FunctionEnvironment which could cause issues with callbacks and eval. All FunctionObjects now have a valid NewFunctionEnvironment implementation. - Remove execute_statements from Interpreter and instead use ASTNode::execute everywhere this simplifies AST.cpp as you no longer need to worry about which method to call. - Make ScopeNodes setup their own environment. This uses four different methods specified by the spec {Block, Function, Eval, Global}DeclarationInstantiation with the annexB extensions. - Implement and use NamedEvaluation where specified. Additionally there are fixes to things exposed by these changes to eval, {for, for-in, for-of} loops and assignment. Finally it also fixes some tests in test-js which where passing before but not now that we have correct behavior :^).
2021-09-22 10:44:56 +00:00
switch_statement->add_case(parse_switch_case());
LibJS: Multiple 'default' clauses in switch statement are a syntax error
2020-10-18 22:12:27 +00:00
}
LibJS: Lexer and parser support for "switch" statements
2020-03-29 11:09:54 +00:00
consume(TokenType::CurlyClose);
LibJS: Make scoping follow the spec Before this we used an ad-hoc combination of references and 'variables' stored in a hashmap. This worked in most cases but is not spec like. Additionally hoisting, dynamically naming functions and scope analysis was not done properly. This patch fixes all of that by: - Implement BindingInitialization for destructuring assignment. - Implementing a new ScopePusher which tracks the lexical and var scoped declarations. This hoists functions to the top level if no lexical declaration name overlaps. Furthermore we do checking of redeclarations in the ScopePusher now requiring less checks all over the place. - Add methods for parsing the directives and statement lists instead of having that code duplicated in multiple places. This allows declarations to pushed to the appropriate scope more easily. - Remove the non spec way of storing 'variables' in DeclarativeEnvironment and make Reference follow the spec instead of checking both the bindings and 'variables'. - Remove all scoping related things from the Interpreter. And instead use environments as specified by the spec. This also includes fixing that NativeFunctions did not produce a valid FunctionEnvironment which could cause issues with callbacks and eval. All FunctionObjects now have a valid NewFunctionEnvironment implementation. - Remove execute_statements from Interpreter and instead use ASTNode::execute everywhere this simplifies AST.cpp as you no longer need to worry about which method to call. - Make ScopeNodes setup their own environment. This uses four different methods specified by the spec {Block, Function, Eval, Global}DeclarationInstantiation with the annexB extensions. - Implement and use NamedEvaluation where specified. Additionally there are fixes to things exposed by these changes to eval, {for, for-in, for-of} loops and assignment. Finally it also fixes some tests in test-js which where passing before but not now that we have correct behavior :^).
2021-09-22 10:44:56 +00:00
return switch_statement;
LibJS: Lexer and parser support for "switch" statements
2020-03-29 11:09:54 +00:00
}
LibJS: Parse "with" statements :^)
2020-11-28 14:05:57 +00:00
NonnullRefPtr<WithStatement> Parser::parse_with_statement()
{
LibJS: Track source positions all the way down to exceptions This makes exceptions have a trace of source positions too, which could probably be helpful in making fancier error tracebacks.
2020-12-28 17:15:22 +00:00
auto rule_start = push_start();
LibJS: Parse "with" statements :^)
2020-11-28 14:05:57 +00:00
consume(TokenType::With);
consume(TokenType::ParenOpen);
auto object = parse_expression(0);
consume(TokenType::ParenClose);
auto body = parse_statement();
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
return create_ast_node<WithStatement>({ m_state.current_token.filename(), rule_start.position(), position() }, move(object), move(body));
LibJS: Parse "with" statements :^)
2020-11-28 14:05:57 +00:00
}
LibJS: Lexer and parser support for "switch" statements
2020-03-29 11:09:54 +00:00
NonnullRefPtr<SwitchCase> Parser::parse_switch_case()
{
LibJS: Track source positions all the way down to exceptions This makes exceptions have a trace of source positions too, which could probably be helpful in making fancier error tracebacks.
2020-12-28 17:15:22 +00:00
auto rule_start = push_start();
LibJS: Lexer and parser support for "switch" statements
2020-03-29 11:09:54 +00:00
RefPtr<Expression> test;
if (consume().type() == TokenType::Case) {
test = parse_expression(0);
}
consume(TokenType::Colon);
NonnullRefPtrVector<Statement> consequent;
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
TemporaryChange break_change(m_state.in_break_context, true);
LibJS: Make scoping follow the spec Before this we used an ad-hoc combination of references and 'variables' stored in a hashmap. This worked in most cases but is not spec like. Additionally hoisting, dynamically naming functions and scope analysis was not done properly. This patch fixes all of that by: - Implement BindingInitialization for destructuring assignment. - Implementing a new ScopePusher which tracks the lexical and var scoped declarations. This hoists functions to the top level if no lexical declaration name overlaps. Furthermore we do checking of redeclarations in the ScopePusher now requiring less checks all over the place. - Add methods for parsing the directives and statement lists instead of having that code duplicated in multiple places. This allows declarations to pushed to the appropriate scope more easily. - Remove the non spec way of storing 'variables' in DeclarativeEnvironment and make Reference follow the spec instead of checking both the bindings and 'variables'. - Remove all scoping related things from the Interpreter. And instead use environments as specified by the spec. This also includes fixing that NativeFunctions did not produce a valid FunctionEnvironment which could cause issues with callbacks and eval. All FunctionObjects now have a valid NewFunctionEnvironment implementation. - Remove execute_statements from Interpreter and instead use ASTNode::execute everywhere this simplifies AST.cpp as you no longer need to worry about which method to call. - Make ScopeNodes setup their own environment. This uses four different methods specified by the spec {Block, Function, Eval, Global}DeclarationInstantiation with the annexB extensions. - Implement and use NamedEvaluation where specified. Additionally there are fixes to things exposed by these changes to eval, {for, for-in, for-of} loops and assignment. Finally it also fixes some tests in test-js which where passing before but not now that we have correct behavior :^).
2021-09-22 10:44:56 +00:00
auto switch_case = create_ast_node<SwitchCase>({ m_state.current_token.filename(), rule_start.position(), position() }, move(test));
parse_statement_list(switch_case);
LibJS: Lexer and parser support for "switch" statements
2020-03-29 11:09:54 +00:00
LibJS: Make scoping follow the spec Before this we used an ad-hoc combination of references and 'variables' stored in a hashmap. This worked in most cases but is not spec like. Additionally hoisting, dynamically naming functions and scope analysis was not done properly. This patch fixes all of that by: - Implement BindingInitialization for destructuring assignment. - Implementing a new ScopePusher which tracks the lexical and var scoped declarations. This hoists functions to the top level if no lexical declaration name overlaps. Furthermore we do checking of redeclarations in the ScopePusher now requiring less checks all over the place. - Add methods for parsing the directives and statement lists instead of having that code duplicated in multiple places. This allows declarations to pushed to the appropriate scope more easily. - Remove the non spec way of storing 'variables' in DeclarativeEnvironment and make Reference follow the spec instead of checking both the bindings and 'variables'. - Remove all scoping related things from the Interpreter. And instead use environments as specified by the spec. This also includes fixing that NativeFunctions did not produce a valid FunctionEnvironment which could cause issues with callbacks and eval. All FunctionObjects now have a valid NewFunctionEnvironment implementation. - Remove execute_statements from Interpreter and instead use ASTNode::execute everywhere this simplifies AST.cpp as you no longer need to worry about which method to call. - Make ScopeNodes setup their own environment. This uses four different methods specified by the spec {Block, Function, Eval, Global}DeclarationInstantiation with the annexB extensions. - Implement and use NamedEvaluation where specified. Additionally there are fixes to things exposed by these changes to eval, {for, for-in, for-of} loops and assignment. Finally it also fixes some tests in test-js which where passing before but not now that we have correct behavior :^).
2021-09-22 10:44:56 +00:00
return switch_case;
LibJS: Lexer and parser support for "switch" statements
2020-03-29 11:09:54 +00:00
}
LibJS: Parse "try", "catch" and "finally" This is the first step towards support exceptions. :^)
2020-03-24 13:03:55 +00:00
NonnullRefPtr<CatchClause> Parser::parse_catch_clause()
{
LibJS: Track source positions all the way down to exceptions This makes exceptions have a trace of source positions too, which could probably be helpful in making fancier error tracebacks.
2020-12-28 17:15:22 +00:00
auto rule_start = push_start();
LibJS: Parse "try", "catch" and "finally" This is the first step towards support exceptions. :^)
2020-03-24 13:03:55 +00:00
consume(TokenType::Catch);
LibJS: Add support for binding patterns in catch clauses `try { ... } catch({a=foo}) {}` is valid, and now we parse and evaluate it correctly :^)
2021-07-11 10:45:38 +00:00
FlyString parameter;
RefPtr<BindingPattern> pattern_parameter;
auto should_expect_parameter = false;
LibJS: Parse "try", "catch" and "finally" This is the first step towards support exceptions. :^)
2020-03-24 13:03:55 +00:00
if (match(TokenType::ParenOpen)) {
LibJS: Add support for binding patterns in catch clauses `try { ... } catch({a=foo}) {}` is valid, and now we parse and evaluate it correctly :^)
2021-07-11 10:45:38 +00:00
should_expect_parameter = true;
LibJS: Parse "try", "catch" and "finally" This is the first step towards support exceptions. :^)
2020-03-24 13:03:55 +00:00
consume();
LibJS: Add support for async functions This commit adds support for the most bare bones version of async functions, support for async generator functions, async arrow functions and await expressions are TODO.
2021-11-09 18:39:22 +00:00
if (match_identifier_name() && (!match(TokenType::Yield) || !m_state.in_generator_function_context) && (!match(TokenType::Async) || !m_state.in_async_function_context))
LibJS: Rework Identifier parsing to match the spec more closely
2021-07-11 11:04:55 +00:00
parameter = consume().value();
LibJS: Add support for binding patterns in catch clauses `try { ... } catch({a=foo}) {}` is valid, and now we parse and evaluate it correctly :^)
2021-07-11 10:45:38 +00:00
else
LibJS: Allow member expressions in binding patterns Also allows literal string and numbers as property names in object binding patterns.
2021-09-17 23:11:32 +00:00
pattern_parameter = parse_binding_pattern(AllowDuplicates::No, AllowMemberExpressions::No);
LibJS: Parse "try", "catch" and "finally" This is the first step towards support exceptions. :^)
2020-03-24 13:03:55 +00:00
consume(TokenType::ParenClose);
}
LibJS: Add support for binding patterns in catch clauses `try { ... } catch({a=foo}) {}` is valid, and now we parse and evaluate it correctly :^)
2021-07-11 10:45:38 +00:00
if (should_expect_parameter && parameter.is_empty() && !pattern_parameter)
LibJS: Use expected() instead of syntax_error("Expected ...")
2021-07-11 19:00:55 +00:00
expected("an identifier or a binding pattern");
LibJS: Add support for binding patterns in catch clauses `try { ... } catch({a=foo}) {}` is valid, and now we parse and evaluate it correctly :^)
2021-07-11 10:45:38 +00:00
LibJS: Make scoping follow the spec Before this we used an ad-hoc combination of references and 'variables' stored in a hashmap. This worked in most cases but is not spec like. Additionally hoisting, dynamically naming functions and scope analysis was not done properly. This patch fixes all of that by: - Implement BindingInitialization for destructuring assignment. - Implementing a new ScopePusher which tracks the lexical and var scoped declarations. This hoists functions to the top level if no lexical declaration name overlaps. Furthermore we do checking of redeclarations in the ScopePusher now requiring less checks all over the place. - Add methods for parsing the directives and statement lists instead of having that code duplicated in multiple places. This allows declarations to pushed to the appropriate scope more easily. - Remove the non spec way of storing 'variables' in DeclarativeEnvironment and make Reference follow the spec instead of checking both the bindings and 'variables'. - Remove all scoping related things from the Interpreter. And instead use environments as specified by the spec. This also includes fixing that NativeFunctions did not produce a valid FunctionEnvironment which could cause issues with callbacks and eval. All FunctionObjects now have a valid NewFunctionEnvironment implementation. - Remove execute_statements from Interpreter and instead use ASTNode::execute everywhere this simplifies AST.cpp as you no longer need to worry about which method to call. - Make ScopeNodes setup their own environment. This uses four different methods specified by the spec {Block, Function, Eval, Global}DeclarationInstantiation with the annexB extensions. - Implement and use NamedEvaluation where specified. Additionally there are fixes to things exposed by these changes to eval, {for, for-in, for-of} loops and assignment. Finally it also fixes some tests in test-js which where passing before but not now that we have correct behavior :^).
2021-09-22 10:44:56 +00:00
HashTable<FlyString> bound_names;
LibJS: Rework Identifier parsing to match the spec more closely
2021-07-11 11:04:55 +00:00
LibJS: Make scoping follow the spec Before this we used an ad-hoc combination of references and 'variables' stored in a hashmap. This worked in most cases but is not spec like. Additionally hoisting, dynamically naming functions and scope analysis was not done properly. This patch fixes all of that by: - Implement BindingInitialization for destructuring assignment. - Implementing a new ScopePusher which tracks the lexical and var scoped declarations. This hoists functions to the top level if no lexical declaration name overlaps. Furthermore we do checking of redeclarations in the ScopePusher now requiring less checks all over the place. - Add methods for parsing the directives and statement lists instead of having that code duplicated in multiple places. This allows declarations to pushed to the appropriate scope more easily. - Remove the non spec way of storing 'variables' in DeclarativeEnvironment and make Reference follow the spec instead of checking both the bindings and 'variables'. - Remove all scoping related things from the Interpreter. And instead use environments as specified by the spec. This also includes fixing that NativeFunctions did not produce a valid FunctionEnvironment which could cause issues with callbacks and eval. All FunctionObjects now have a valid NewFunctionEnvironment implementation. - Remove execute_statements from Interpreter and instead use ASTNode::execute everywhere this simplifies AST.cpp as you no longer need to worry about which method to call. - Make ScopeNodes setup their own environment. This uses four different methods specified by the spec {Block, Function, Eval, Global}DeclarationInstantiation with the annexB extensions. - Implement and use NamedEvaluation where specified. Additionally there are fixes to things exposed by these changes to eval, {for, for-in, for-of} loops and assignment. Finally it also fixes some tests in test-js which where passing before but not now that we have correct behavior :^).
2021-09-22 10:44:56 +00:00
if (pattern_parameter) {
pattern_parameter->for_each_bound_name(
[&](auto& name) {
check_identifier_name_for_assignment_validity(name);
bound_names.set(name);
});
}
if (!parameter.is_empty()) {
LibJS: Rework Identifier parsing to match the spec more closely
2021-07-11 11:04:55 +00:00
check_identifier_name_for_assignment_validity(parameter);
LibJS: Make scoping follow the spec Before this we used an ad-hoc combination of references and 'variables' stored in a hashmap. This worked in most cases but is not spec like. Additionally hoisting, dynamically naming functions and scope analysis was not done properly. This patch fixes all of that by: - Implement BindingInitialization for destructuring assignment. - Implementing a new ScopePusher which tracks the lexical and var scoped declarations. This hoists functions to the top level if no lexical declaration name overlaps. Furthermore we do checking of redeclarations in the ScopePusher now requiring less checks all over the place. - Add methods for parsing the directives and statement lists instead of having that code duplicated in multiple places. This allows declarations to pushed to the appropriate scope more easily. - Remove the non spec way of storing 'variables' in DeclarativeEnvironment and make Reference follow the spec instead of checking both the bindings and 'variables'. - Remove all scoping related things from the Interpreter. And instead use environments as specified by the spec. This also includes fixing that NativeFunctions did not produce a valid FunctionEnvironment which could cause issues with callbacks and eval. All FunctionObjects now have a valid NewFunctionEnvironment implementation. - Remove execute_statements from Interpreter and instead use ASTNode::execute everywhere this simplifies AST.cpp as you no longer need to worry about which method to call. - Make ScopeNodes setup their own environment. This uses four different methods specified by the spec {Block, Function, Eval, Global}DeclarationInstantiation with the annexB extensions. - Implement and use NamedEvaluation where specified. Additionally there are fixes to things exposed by these changes to eval, {for, for-in, for-of} loops and assignment. Finally it also fixes some tests in test-js which where passing before but not now that we have correct behavior :^).
2021-09-22 10:44:56 +00:00
bound_names.set(parameter);
}
LibJS: Rework Identifier parsing to match the spec more closely
2021-07-11 11:04:55 +00:00
LibJS: Treat the Catch binding identifier as a var binding
2021-10-08 10:04:13 +00:00
ScopePusher catch_scope = ScopePusher::catch_scope(*this, pattern_parameter, parameter);
LibJS: Parse "try", "catch" and "finally" This is the first step towards support exceptions. :^)
2020-03-24 13:03:55 +00:00
auto body = parse_block_statement();
LibJS: Make scoping follow the spec Before this we used an ad-hoc combination of references and 'variables' stored in a hashmap. This worked in most cases but is not spec like. Additionally hoisting, dynamically naming functions and scope analysis was not done properly. This patch fixes all of that by: - Implement BindingInitialization for destructuring assignment. - Implementing a new ScopePusher which tracks the lexical and var scoped declarations. This hoists functions to the top level if no lexical declaration name overlaps. Furthermore we do checking of redeclarations in the ScopePusher now requiring less checks all over the place. - Add methods for parsing the directives and statement lists instead of having that code duplicated in multiple places. This allows declarations to pushed to the appropriate scope more easily. - Remove the non spec way of storing 'variables' in DeclarativeEnvironment and make Reference follow the spec instead of checking both the bindings and 'variables'. - Remove all scoping related things from the Interpreter. And instead use environments as specified by the spec. This also includes fixing that NativeFunctions did not produce a valid FunctionEnvironment which could cause issues with callbacks and eval. All FunctionObjects now have a valid NewFunctionEnvironment implementation. - Remove execute_statements from Interpreter and instead use ASTNode::execute everywhere this simplifies AST.cpp as you no longer need to worry about which method to call. - Make ScopeNodes setup their own environment. This uses four different methods specified by the spec {Block, Function, Eval, Global}DeclarationInstantiation with the annexB extensions. - Implement and use NamedEvaluation where specified. Additionally there are fixes to things exposed by these changes to eval, {for, for-in, for-of} loops and assignment. Finally it also fixes some tests in test-js which where passing before but not now that we have correct behavior :^).
2021-09-22 10:44:56 +00:00
body->for_each_lexically_declared_name([&](auto const& name) {
if (bound_names.contains(name))
syntax_error(String::formatted("Identifier '{}' already declared as catch parameter", name));
});
LibJS: Add support for binding patterns in catch clauses `try { ... } catch({a=foo}) {}` is valid, and now we parse and evaluate it correctly :^)
2021-07-11 10:45:38 +00:00
if (pattern_parameter) {
return create_ast_node<CatchClause>(
{ m_state.current_token.filename(), rule_start.position(), position() },
pattern_parameter.release_nonnull(),
move(body));
}
return create_ast_node<CatchClause>(
{ m_state.current_token.filename(), rule_start.position(), position() },
move(parameter),
move(body));
LibJS: Parse "try", "catch" and "finally" This is the first step towards support exceptions. :^)
2020-03-24 13:03:55 +00:00
}
LibJS: Parse "if" statements This patch implements basic parsing of "if" statements. We don't yet support parsing "else", so I added a FIXME about that.
2020-03-21 17:40:17 +00:00
NonnullRefPtr<IfStatement> Parser::parse_if_statement()
{
LibJS: Track source positions all the way down to exceptions This makes exceptions have a trace of source positions too, which could probably be helpful in making fancier error tracebacks.
2020-12-28 17:15:22 +00:00
auto rule_start = push_start();
LibJS: Function declarations in if statement clauses https://tc39.es/ecma262/#sec-functiondeclarations-in-ifstatement-statement-clauses B.3.4 FunctionDeclarations in IfStatement Statement Clauses The following augments the IfStatement production in 13.6: IfStatement[Yield, Await, Return] : if ( Expression[+In, ?Yield, ?Await] ) FunctionDeclaration[?Yield, ?Await, ~Default] else Statement[?Yield, ?Await, ?Return] if ( Expression[+In, ?Yield, ?Await] ) Statement[?Yield, ?Await, ?Return] else FunctionDeclaration[?Yield, ?Await, ~Default] if ( Expression[+In, ?Yield, ?Await] ) FunctionDeclaration[?Yield, ?Await, ~Default] else FunctionDeclaration[?Yield, ?Await, ~Default] if ( Expression[+In, ?Yield, ?Await] ) FunctionDeclaration[?Yield, ?Await, ~Default] This production only applies when parsing non-strict code. Code matching this production is processed as if each matching occurrence of FunctionDeclaration[?Yield, ?Await, ~Default] was the sole StatementListItem of a BlockStatement occupying that position in the source code. The semantics of such a synthetic BlockStatement includes the web legacy compatibility semantics specified in B.3.3.
2020-10-31 13:37:09 +00:00
auto parse_function_declaration_as_block_statement = [&] {
// https://tc39.es/ecma262/#sec-functiondeclarations-in-ifstatement-statement-clauses
// Code matching this production is processed as if each matching occurrence of
// FunctionDeclaration[?Yield, ?Await, ~Default] was the sole StatementListItem
// of a BlockStatement occupying that position in the source code.
LibJS: Fix small issues in parser - Fix some places where escaped keywords are (not) allowed. - Be more strict about parameters for functions with 'use strict'. - Fix that expressions statements allowed functions and classes. - Fix that class expressions were not allowed. - Added a new next_token() method for checking the look ahead. - Fix that continue labels could jump to non iterating targets. - Fix that generator functions cannot be declared in if statements.
2021-08-28 15:04:37 +00:00
VERIFY(match(TokenType::Function));
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
auto block = create_ast_node<BlockStatement>({ m_state.current_token.filename(), rule_start.position(), position() });
LibJS: Make scoping follow the spec Before this we used an ad-hoc combination of references and 'variables' stored in a hashmap. This worked in most cases but is not spec like. Additionally hoisting, dynamically naming functions and scope analysis was not done properly. This patch fixes all of that by: - Implement BindingInitialization for destructuring assignment. - Implementing a new ScopePusher which tracks the lexical and var scoped declarations. This hoists functions to the top level if no lexical declaration name overlaps. Furthermore we do checking of redeclarations in the ScopePusher now requiring less checks all over the place. - Add methods for parsing the directives and statement lists instead of having that code duplicated in multiple places. This allows declarations to pushed to the appropriate scope more easily. - Remove the non spec way of storing 'variables' in DeclarativeEnvironment and make Reference follow the spec instead of checking both the bindings and 'variables'. - Remove all scoping related things from the Interpreter. And instead use environments as specified by the spec. This also includes fixing that NativeFunctions did not produce a valid FunctionEnvironment which could cause issues with callbacks and eval. All FunctionObjects now have a valid NewFunctionEnvironment implementation. - Remove execute_statements from Interpreter and instead use ASTNode::execute everywhere this simplifies AST.cpp as you no longer need to worry about which method to call. - Make ScopeNodes setup their own environment. This uses four different methods specified by the spec {Block, Function, Eval, Global}DeclarationInstantiation with the annexB extensions. - Implement and use NamedEvaluation where specified. Additionally there are fixes to things exposed by these changes to eval, {for, for-in, for-of} loops and assignment. Finally it also fixes some tests in test-js which where passing before but not now that we have correct behavior :^).
2021-09-22 10:44:56 +00:00
ScopePusher block_scope = ScopePusher::block_scope(*this, *block);
LibJS: Fix small issues in parser - Fix some places where escaped keywords are (not) allowed. - Be more strict about parameters for functions with 'use strict'. - Fix that expressions statements allowed functions and classes. - Fix that class expressions were not allowed. - Added a new next_token() method for checking the look ahead. - Fix that continue labels could jump to non iterating targets. - Fix that generator functions cannot be declared in if statements.
2021-08-28 15:04:37 +00:00
auto declaration = parse_declaration();
LibJS: Make scoping follow the spec Before this we used an ad-hoc combination of references and 'variables' stored in a hashmap. This worked in most cases but is not spec like. Additionally hoisting, dynamically naming functions and scope analysis was not done properly. This patch fixes all of that by: - Implement BindingInitialization for destructuring assignment. - Implementing a new ScopePusher which tracks the lexical and var scoped declarations. This hoists functions to the top level if no lexical declaration name overlaps. Furthermore we do checking of redeclarations in the ScopePusher now requiring less checks all over the place. - Add methods for parsing the directives and statement lists instead of having that code duplicated in multiple places. This allows declarations to pushed to the appropriate scope more easily. - Remove the non spec way of storing 'variables' in DeclarativeEnvironment and make Reference follow the spec instead of checking both the bindings and 'variables'. - Remove all scoping related things from the Interpreter. And instead use environments as specified by the spec. This also includes fixing that NativeFunctions did not produce a valid FunctionEnvironment which could cause issues with callbacks and eval. All FunctionObjects now have a valid NewFunctionEnvironment implementation. - Remove execute_statements from Interpreter and instead use ASTNode::execute everywhere this simplifies AST.cpp as you no longer need to worry about which method to call. - Make ScopeNodes setup their own environment. This uses four different methods specified by the spec {Block, Function, Eval, Global}DeclarationInstantiation with the annexB extensions. - Implement and use NamedEvaluation where specified. Additionally there are fixes to things exposed by these changes to eval, {for, for-in, for-of} loops and assignment. Finally it also fixes some tests in test-js which where passing before but not now that we have correct behavior :^).
2021-09-22 10:44:56 +00:00
VERIFY(m_state.current_scope_pusher);
block_scope.add_declaration(declaration);
LibJS: Fix small issues in parser - Fix some places where escaped keywords are (not) allowed. - Be more strict about parameters for functions with 'use strict'. - Fix that expressions statements allowed functions and classes. - Fix that class expressions were not allowed. - Added a new next_token() method for checking the look ahead. - Fix that continue labels could jump to non iterating targets. - Fix that generator functions cannot be declared in if statements.
2021-08-28 15:04:37 +00:00
VERIFY(is<FunctionDeclaration>(*declaration));
auto& function_declaration = static_cast<FunctionDeclaration const&>(*declaration);
if (function_declaration.kind() == FunctionKind::Generator)
syntax_error("Generator functions can only be declared in top-level or within a block");
LibJS: Add support for async functions This commit adds support for the most bare bones version of async functions, support for async generator functions, async arrow functions and await expressions are TODO.
2021-11-09 18:39:22 +00:00
if (function_declaration.kind() == FunctionKind::Async)
syntax_error("Async functions can only be declared in top-level or within a block");
LibJS: Fix small issues in parser - Fix some places where escaped keywords are (not) allowed. - Be more strict about parameters for functions with 'use strict'. - Fix that expressions statements allowed functions and classes. - Fix that class expressions were not allowed. - Added a new next_token() method for checking the look ahead. - Fix that continue labels could jump to non iterating targets. - Fix that generator functions cannot be declared in if statements.
2021-08-28 15:04:37 +00:00
block->append(move(declaration));
LibJS: Function declarations in if statement clauses https://tc39.es/ecma262/#sec-functiondeclarations-in-ifstatement-statement-clauses B.3.4 FunctionDeclarations in IfStatement Statement Clauses The following augments the IfStatement production in 13.6: IfStatement[Yield, Await, Return] : if ( Expression[+In, ?Yield, ?Await] ) FunctionDeclaration[?Yield, ?Await, ~Default] else Statement[?Yield, ?Await, ?Return] if ( Expression[+In, ?Yield, ?Await] ) Statement[?Yield, ?Await, ?Return] else FunctionDeclaration[?Yield, ?Await, ~Default] if ( Expression[+In, ?Yield, ?Await] ) FunctionDeclaration[?Yield, ?Await, ~Default] else FunctionDeclaration[?Yield, ?Await, ~Default] if ( Expression[+In, ?Yield, ?Await] ) FunctionDeclaration[?Yield, ?Await, ~Default] This production only applies when parsing non-strict code. Code matching this production is processed as if each matching occurrence of FunctionDeclaration[?Yield, ?Await, ~Default] was the sole StatementListItem of a BlockStatement occupying that position in the source code. The semantics of such a synthetic BlockStatement includes the web legacy compatibility semantics specified in B.3.3.
2020-10-31 13:37:09 +00:00
return block;
};
LibJS: Parse "if" statements This patch implements basic parsing of "if" statements. We don't yet support parsing "else", so I added a FIXME about that.
2020-03-21 17:40:17 +00:00
consume(TokenType::If);
consume(TokenType::ParenOpen);
auto predicate = parse_expression(0);
consume(TokenType::ParenClose);
LibJS: Function declarations in if statement clauses https://tc39.es/ecma262/#sec-functiondeclarations-in-ifstatement-statement-clauses B.3.4 FunctionDeclarations in IfStatement Statement Clauses The following augments the IfStatement production in 13.6: IfStatement[Yield, Await, Return] : if ( Expression[+In, ?Yield, ?Await] ) FunctionDeclaration[?Yield, ?Await, ~Default] else Statement[?Yield, ?Await, ?Return] if ( Expression[+In, ?Yield, ?Await] ) Statement[?Yield, ?Await, ?Return] else FunctionDeclaration[?Yield, ?Await, ~Default] if ( Expression[+In, ?Yield, ?Await] ) FunctionDeclaration[?Yield, ?Await, ~Default] else FunctionDeclaration[?Yield, ?Await, ~Default] if ( Expression[+In, ?Yield, ?Await] ) FunctionDeclaration[?Yield, ?Await, ~Default] This production only applies when parsing non-strict code. Code matching this production is processed as if each matching occurrence of FunctionDeclaration[?Yield, ?Await, ~Default] was the sole StatementListItem of a BlockStatement occupying that position in the source code. The semantics of such a synthetic BlockStatement includes the web legacy compatibility semantics specified in B.3.3.
2020-10-31 13:37:09 +00:00
RefPtr<Statement> consequent;
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
if (!m_state.strict_mode && match(TokenType::Function))
LibJS: Function declarations in if statement clauses https://tc39.es/ecma262/#sec-functiondeclarations-in-ifstatement-statement-clauses B.3.4 FunctionDeclarations in IfStatement Statement Clauses The following augments the IfStatement production in 13.6: IfStatement[Yield, Await, Return] : if ( Expression[+In, ?Yield, ?Await] ) FunctionDeclaration[?Yield, ?Await, ~Default] else Statement[?Yield, ?Await, ?Return] if ( Expression[+In, ?Yield, ?Await] ) Statement[?Yield, ?Await, ?Return] else FunctionDeclaration[?Yield, ?Await, ~Default] if ( Expression[+In, ?Yield, ?Await] ) FunctionDeclaration[?Yield, ?Await, ~Default] else FunctionDeclaration[?Yield, ?Await, ~Default] if ( Expression[+In, ?Yield, ?Await] ) FunctionDeclaration[?Yield, ?Await, ~Default] This production only applies when parsing non-strict code. Code matching this production is processed as if each matching occurrence of FunctionDeclaration[?Yield, ?Await, ~Default] was the sole StatementListItem of a BlockStatement occupying that position in the source code. The semantics of such a synthetic BlockStatement includes the web legacy compatibility semantics specified in B.3.3.
2020-10-31 13:37:09 +00:00
consequent = parse_function_declaration_as_block_statement();
else
consequent = parse_statement();
LibJS: Implement "else" parsing We can now handle scripts with if/else in LibJS. Most of the changes are about fixing IfStatement to store the consequent and alternate node as Statements. Interpreter now also runs Statements, rather than running ScopeNodes.
2020-03-23 15:46:41 +00:00
RefPtr<Statement> alternate;
if (match(TokenType::Else)) {
LibJS: Function declarations in if statement clauses https://tc39.es/ecma262/#sec-functiondeclarations-in-ifstatement-statement-clauses B.3.4 FunctionDeclarations in IfStatement Statement Clauses The following augments the IfStatement production in 13.6: IfStatement[Yield, Await, Return] : if ( Expression[+In, ?Yield, ?Await] ) FunctionDeclaration[?Yield, ?Await, ~Default] else Statement[?Yield, ?Await, ?Return] if ( Expression[+In, ?Yield, ?Await] ) Statement[?Yield, ?Await, ?Return] else FunctionDeclaration[?Yield, ?Await, ~Default] if ( Expression[+In, ?Yield, ?Await] ) FunctionDeclaration[?Yield, ?Await, ~Default] else FunctionDeclaration[?Yield, ?Await, ~Default] if ( Expression[+In, ?Yield, ?Await] ) FunctionDeclaration[?Yield, ?Await, ~Default] This production only applies when parsing non-strict code. Code matching this production is processed as if each matching occurrence of FunctionDeclaration[?Yield, ?Await, ~Default] was the sole StatementListItem of a BlockStatement occupying that position in the source code. The semantics of such a synthetic BlockStatement includes the web legacy compatibility semantics specified in B.3.3.
2020-10-31 13:37:09 +00:00
consume();
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
if (!m_state.strict_mode && match(TokenType::Function))
LibJS: Function declarations in if statement clauses https://tc39.es/ecma262/#sec-functiondeclarations-in-ifstatement-statement-clauses B.3.4 FunctionDeclarations in IfStatement Statement Clauses The following augments the IfStatement production in 13.6: IfStatement[Yield, Await, Return] : if ( Expression[+In, ?Yield, ?Await] ) FunctionDeclaration[?Yield, ?Await, ~Default] else Statement[?Yield, ?Await, ?Return] if ( Expression[+In, ?Yield, ?Await] ) Statement[?Yield, ?Await, ?Return] else FunctionDeclaration[?Yield, ?Await, ~Default] if ( Expression[+In, ?Yield, ?Await] ) FunctionDeclaration[?Yield, ?Await, ~Default] else FunctionDeclaration[?Yield, ?Await, ~Default] if ( Expression[+In, ?Yield, ?Await] ) FunctionDeclaration[?Yield, ?Await, ~Default] This production only applies when parsing non-strict code. Code matching this production is processed as if each matching occurrence of FunctionDeclaration[?Yield, ?Await, ~Default] was the sole StatementListItem of a BlockStatement occupying that position in the source code. The semantics of such a synthetic BlockStatement includes the web legacy compatibility semantics specified in B.3.3.
2020-10-31 13:37:09 +00:00
alternate = parse_function_declaration_as_block_statement();
else
alternate = parse_statement();
LibJS: Implement "else" parsing We can now handle scripts with if/else in LibJS. Most of the changes are about fixing IfStatement to store the consequent and alternate node as Statements. Interpreter now also runs Statements, rather than running ScopeNodes.
2020-03-23 15:46:41 +00:00
}
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
return create_ast_node<IfStatement>({ m_state.current_token.filename(), rule_start.position(), position() }, move(predicate), move(*consequent), move(alternate));
LibJS: Parse "if" statements This patch implements basic parsing of "if" statements. We don't yet support parsing "else", so I added a FIXME about that.
2020-03-21 17:40:17 +00:00
}
LibJS: Implement basic for..in and for..of loops
2020-04-21 18:21:26 +00:00
NonnullRefPtr<Statement> Parser::parse_for_statement()
LibJS: Implement for statement
2020-03-12 12:12:12 +00:00
{
LibJS: Track source positions all the way down to exceptions This makes exceptions have a trace of source positions too, which could probably be helpful in making fancier error tracebacks.
2020-12-28 17:15:22 +00:00
auto rule_start = push_start();
LibJS: Implement parsing and executing for-await-of loops
2021-11-23 15:09:28 +00:00
auto is_await_loop = IsForAwaitLoop::No;
LibJS: Parse async arrow functions
2021-11-14 23:47:16 +00:00
auto match_of = [&](Token const& token) {
return token.type() == TokenType::Identifier && token.original_value() == "of"sv;
};
LibJS: Implement parsing and executing for-await-of loops
2021-11-23 15:09:28 +00:00
auto match_for_in_of = [&]() {
bool is_of = match_of(m_state.current_token);
if (is_await_loop == IsForAwaitLoop::Yes) {
if (!is_of)
syntax_error("for await loop is only valid with 'of'");
else if (!m_state.in_async_function_context)
syntax_error("for await loop is only valid in async function or generator");
return true;
}
return match(TokenType::In) || is_of;
LibJS: Implement basic for..in and for..of loops
2020-04-21 18:21:26 +00:00
};
LibJS: Implement for statement
2020-03-12 12:12:12 +00:00
consume(TokenType::For);
LibJS: Implement parsing and executing for-await-of loops
2021-11-23 15:09:28 +00:00
if (match(TokenType::Await)) {
consume();
if (!m_state.in_async_function_context)
syntax_error("for-await-of is only allowed in async function context");
is_await_loop = IsForAwaitLoop::Yes;
}
LibJS: Implement for statement
2020-03-12 12:12:12 +00:00
consume(TokenType::ParenOpen);
LibJS: Use ScopePusher to correctly push the scope in for statements We were previously pushing a scope but forgetting to actually set it as the current scope.
2021-09-16 17:39:28 +00:00
Optional<ScopePusher> scope_pusher;
LibJS: Fix runaway let scope when parsing for-in/of statements This requires variables that should be exported to the script host or to other scripts to be declared as var (such as in test-common.js), since lexically-scoped variables won't be visible.
2021-07-05 19:31:51 +00:00
LibJS: Consume semicolon at the end of a statement A bunch of code was relying on this not happenind, in particular the parsing of "for" statements. Reorganized things so they work again.
2020-03-23 18:08:32 +00:00
RefPtr<ASTNode> init;
LibJS: Implement basic for..in and for..of loops
2020-04-21 18:21:26 +00:00
if (!match(TokenType::Semicolon)) {
LibJS: Rework Identifier parsing to match the spec more closely
2021-07-11 11:04:55 +00:00
if (match_variable_declaration()) {
LibJS: Make scoping follow the spec Before this we used an ad-hoc combination of references and 'variables' stored in a hashmap. This worked in most cases but is not spec like. Additionally hoisting, dynamically naming functions and scope analysis was not done properly. This patch fixes all of that by: - Implement BindingInitialization for destructuring assignment. - Implementing a new ScopePusher which tracks the lexical and var scoped declarations. This hoists functions to the top level if no lexical declaration name overlaps. Furthermore we do checking of redeclarations in the ScopePusher now requiring less checks all over the place. - Add methods for parsing the directives and statement lists instead of having that code duplicated in multiple places. This allows declarations to pushed to the appropriate scope more easily. - Remove the non spec way of storing 'variables' in DeclarativeEnvironment and make Reference follow the spec instead of checking both the bindings and 'variables'. - Remove all scoping related things from the Interpreter. And instead use environments as specified by the spec. This also includes fixing that NativeFunctions did not produce a valid FunctionEnvironment which could cause issues with callbacks and eval. All FunctionObjects now have a valid NewFunctionEnvironment implementation. - Remove execute_statements from Interpreter and instead use ASTNode::execute everywhere this simplifies AST.cpp as you no longer need to worry about which method to call. - Make ScopeNodes setup their own environment. This uses four different methods specified by the spec {Block, Function, Eval, Global}DeclarationInstantiation with the annexB extensions. - Implement and use NamedEvaluation where specified. Additionally there are fixes to things exposed by these changes to eval, {for, for-in, for-of} loops and assignment. Finally it also fixes some tests in test-js which where passing before but not now that we have correct behavior :^).
2021-09-22 10:44:56 +00:00
auto declaration = parse_variable_declaration(true);
if (declaration->declaration_kind() == DeclarationKind::Var) {
m_state.current_scope_pusher->add_declaration(declaration);
} else {
// This does not follow the normal declaration structure so we need additional checks.
HashTable<FlyString> bound_names;
declaration->for_each_bound_name([&](auto const& name) {
if (bound_names.set(name) != AK::HashSetResult::InsertedNewEntry)
syntax_error(String::formatted("Identifier '{}' already declared in for loop initializer", name), declaration->source_range().start);
});
}
init = move(declaration);
LibJS: Implement basic for..in and for..of loops
2020-04-21 18:21:26 +00:00
if (match_for_in_of())
LibJS: Implement parsing and executing for-await-of loops
2021-11-23 15:09:28 +00:00
return parse_for_in_of_statement(*init, is_await_loop);
LibJS: Require initializer for 'const' variable declaration
2020-10-30 19:08:45 +00:00
if (static_cast<VariableDeclaration&>(*init).declaration_kind() == DeclarationKind::Const) {
LibJS: Make scoping follow the spec Before this we used an ad-hoc combination of references and 'variables' stored in a hashmap. This worked in most cases but is not spec like. Additionally hoisting, dynamically naming functions and scope analysis was not done properly. This patch fixes all of that by: - Implement BindingInitialization for destructuring assignment. - Implementing a new ScopePusher which tracks the lexical and var scoped declarations. This hoists functions to the top level if no lexical declaration name overlaps. Furthermore we do checking of redeclarations in the ScopePusher now requiring less checks all over the place. - Add methods for parsing the directives and statement lists instead of having that code duplicated in multiple places. This allows declarations to pushed to the appropriate scope more easily. - Remove the non spec way of storing 'variables' in DeclarativeEnvironment and make Reference follow the spec instead of checking both the bindings and 'variables'. - Remove all scoping related things from the Interpreter. And instead use environments as specified by the spec. This also includes fixing that NativeFunctions did not produce a valid FunctionEnvironment which could cause issues with callbacks and eval. All FunctionObjects now have a valid NewFunctionEnvironment implementation. - Remove execute_statements from Interpreter and instead use ASTNode::execute everywhere this simplifies AST.cpp as you no longer need to worry about which method to call. - Make ScopeNodes setup their own environment. This uses four different methods specified by the spec {Block, Function, Eval, Global}DeclarationInstantiation with the annexB extensions. - Implement and use NamedEvaluation where specified. Additionally there are fixes to things exposed by these changes to eval, {for, for-in, for-of} loops and assignment. Finally it also fixes some tests in test-js which where passing before but not now that we have correct behavior :^).
2021-09-22 10:44:56 +00:00
for (auto& variable : static_cast<VariableDeclaration&>(*init).declarations()) {
if (!variable.init())
LibJS: Require initializer for 'const' variable declaration
2020-10-30 19:08:45 +00:00
syntax_error("Missing initializer in 'const' variable declaration");
}
}
LibJS: Rework Identifier parsing to match the spec more closely
2021-07-11 11:04:55 +00:00
} else if (match_expression()) {
LibJS: Parse async arrow functions
2021-11-14 23:47:16 +00:00
auto lookahead_token = next_token();
bool starts_with_async_of = match(TokenType::Async) && match_of(lookahead_token);
LibJS: Rework Identifier parsing to match the spec more closely
2021-07-11 11:04:55 +00:00
init = parse_expression(0, Associativity::Right, { TokenType::In });
LibJS: Parse async arrow functions
2021-11-14 23:47:16 +00:00
if (match_for_in_of()) {
LibJS: Implement parsing and executing for-await-of loops
2021-11-23 15:09:28 +00:00
if (is_await_loop != IsForAwaitLoop::Yes
&& starts_with_async_of && match_of(m_state.current_token))
LibJS: Parse async arrow functions
2021-11-14 23:47:16 +00:00
syntax_error("for-of loop may not start with async of");
LibJS: Implement parsing and executing for-await-of loops
2021-11-23 15:09:28 +00:00
return parse_for_in_of_statement(*init, is_await_loop);
LibJS: Parse async arrow functions
2021-11-14 23:47:16 +00:00
}
LibJS: Implement automatic semicolon insertion
2020-04-17 13:05:58 +00:00
} else {
LibJS: Replace some parser assertions by syntax errors When parsing JavaScript, we can get pretty much any sequnce of tokens, and we shouldn't crash if it's not something that we normally expect. Instead, emit syntax errors.
2020-06-01 14:08:34 +00:00
syntax_error("Unexpected token in for loop");
LibJS: Implement automatic semicolon insertion
2020-04-17 13:05:58 +00:00
}
LibJS: Implement for statement
2020-03-12 12:12:12 +00:00
}
LibJS: Implement basic for..in and for..of loops
2020-04-21 18:21:26 +00:00
consume(TokenType::Semicolon);
LibJS: Implement for statement
2020-03-12 12:12:12 +00:00
LibJS: Make the AST reference-counted This allows function objects to outlive the original parsed program without their ScopeNode disappearing.
2020-03-18 10:23:53 +00:00
RefPtr<Expression> test;
LibJS: Implement basic for..in and for..of loops
2020-04-21 18:21:26 +00:00
if (!match(TokenType::Semicolon))
LibJS: Add operator precedence parsing Obey precedence and associativity rules when parsing expressions with chained operators.
2020-03-12 22:02:41 +00:00
test = parse_expression(0);
LibJS: Implement for statement
2020-03-12 12:12:12 +00:00
consume(TokenType::Semicolon);
LibJS: Make the AST reference-counted This allows function objects to outlive the original parsed program without their ScopeNode disappearing.
2020-03-18 10:23:53 +00:00
RefPtr<Expression> update;
LibJS: Implement basic for..in and for..of loops
2020-04-21 18:21:26 +00:00
if (!match(TokenType::ParenClose))
LibJS: Add operator precedence parsing Obey precedence and associativity rules when parsing expressions with chained operators.
2020-03-12 22:02:41 +00:00
update = parse_expression(0);
LibJS: Implement for statement
2020-03-12 12:12:12 +00:00
consume(TokenType::ParenClose);
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
TemporaryChange break_change(m_state.in_break_context, true);
TemporaryChange continue_change(m_state.in_continue_context, true);
LibJS: Make scoping follow the spec Before this we used an ad-hoc combination of references and 'variables' stored in a hashmap. This worked in most cases but is not spec like. Additionally hoisting, dynamically naming functions and scope analysis was not done properly. This patch fixes all of that by: - Implement BindingInitialization for destructuring assignment. - Implementing a new ScopePusher which tracks the lexical and var scoped declarations. This hoists functions to the top level if no lexical declaration name overlaps. Furthermore we do checking of redeclarations in the ScopePusher now requiring less checks all over the place. - Add methods for parsing the directives and statement lists instead of having that code duplicated in multiple places. This allows declarations to pushed to the appropriate scope more easily. - Remove the non spec way of storing 'variables' in DeclarativeEnvironment and make Reference follow the spec instead of checking both the bindings and 'variables'. - Remove all scoping related things from the Interpreter. And instead use environments as specified by the spec. This also includes fixing that NativeFunctions did not produce a valid FunctionEnvironment which could cause issues with callbacks and eval. All FunctionObjects now have a valid NewFunctionEnvironment implementation. - Remove execute_statements from Interpreter and instead use ASTNode::execute everywhere this simplifies AST.cpp as you no longer need to worry about which method to call. - Make ScopeNodes setup their own environment. This uses four different methods specified by the spec {Block, Function, Eval, Global}DeclarationInstantiation with the annexB extensions. - Implement and use NamedEvaluation where specified. Additionally there are fixes to things exposed by these changes to eval, {for, for-in, for-of} loops and assignment. Finally it also fixes some tests in test-js which where passing before but not now that we have correct behavior :^).
2021-09-22 10:44:56 +00:00
ScopePusher for_loop_scope = ScopePusher::for_loop_scope(*this, init);
LibJS: Allow "for" statement without curly braces around body
2020-04-04 19:09:06 +00:00
auto body = parse_statement();
LibJS: Implement for statement
2020-03-12 12:12:12 +00:00
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
return create_ast_node<ForStatement>({ m_state.current_token.filename(), rule_start.position(), position() }, move(init), move(test), move(update), move(body));
LibJS: Implement for statement
2020-03-12 12:12:12 +00:00
}
LibJS: Implement parsing and executing for-await-of loops
2021-11-23 15:09:28 +00:00
NonnullRefPtr<Statement> Parser::parse_for_in_of_statement(NonnullRefPtr<ASTNode> lhs, IsForAwaitLoop is_for_await_loop)
LibJS: Implement basic for..in and for..of loops
2020-04-21 18:21:26 +00:00
{
LibJS: Make scoping follow the spec Before this we used an ad-hoc combination of references and 'variables' stored in a hashmap. This worked in most cases but is not spec like. Additionally hoisting, dynamically naming functions and scope analysis was not done properly. This patch fixes all of that by: - Implement BindingInitialization for destructuring assignment. - Implementing a new ScopePusher which tracks the lexical and var scoped declarations. This hoists functions to the top level if no lexical declaration name overlaps. Furthermore we do checking of redeclarations in the ScopePusher now requiring less checks all over the place. - Add methods for parsing the directives and statement lists instead of having that code duplicated in multiple places. This allows declarations to pushed to the appropriate scope more easily. - Remove the non spec way of storing 'variables' in DeclarativeEnvironment and make Reference follow the spec instead of checking both the bindings and 'variables'. - Remove all scoping related things from the Interpreter. And instead use environments as specified by the spec. This also includes fixing that NativeFunctions did not produce a valid FunctionEnvironment which could cause issues with callbacks and eval. All FunctionObjects now have a valid NewFunctionEnvironment implementation. - Remove execute_statements from Interpreter and instead use ASTNode::execute everywhere this simplifies AST.cpp as you no longer need to worry about which method to call. - Make ScopeNodes setup their own environment. This uses four different methods specified by the spec {Block, Function, Eval, Global}DeclarationInstantiation with the annexB extensions. - Implement and use NamedEvaluation where specified. Additionally there are fixes to things exposed by these changes to eval, {for, for-in, for-of} loops and assignment. Finally it also fixes some tests in test-js which where passing before but not now that we have correct behavior :^).
2021-09-22 10:44:56 +00:00
Variant<NonnullRefPtr<ASTNode>, NonnullRefPtr<BindingPattern>> for_declaration = lhs;
LibJS: Track source positions all the way down to exceptions This makes exceptions have a trace of source positions too, which could probably be helpful in making fancier error tracebacks.
2020-12-28 17:15:22 +00:00
auto rule_start = push_start();
LibJS: Make scoping follow the spec Before this we used an ad-hoc combination of references and 'variables' stored in a hashmap. This worked in most cases but is not spec like. Additionally hoisting, dynamically naming functions and scope analysis was not done properly. This patch fixes all of that by: - Implement BindingInitialization for destructuring assignment. - Implementing a new ScopePusher which tracks the lexical and var scoped declarations. This hoists functions to the top level if no lexical declaration name overlaps. Furthermore we do checking of redeclarations in the ScopePusher now requiring less checks all over the place. - Add methods for parsing the directives and statement lists instead of having that code duplicated in multiple places. This allows declarations to pushed to the appropriate scope more easily. - Remove the non spec way of storing 'variables' in DeclarativeEnvironment and make Reference follow the spec instead of checking both the bindings and 'variables'. - Remove all scoping related things from the Interpreter. And instead use environments as specified by the spec. This also includes fixing that NativeFunctions did not produce a valid FunctionEnvironment which could cause issues with callbacks and eval. All FunctionObjects now have a valid NewFunctionEnvironment implementation. - Remove execute_statements from Interpreter and instead use ASTNode::execute everywhere this simplifies AST.cpp as you no longer need to worry about which method to call. - Make ScopeNodes setup their own environment. This uses four different methods specified by the spec {Block, Function, Eval, Global}DeclarationInstantiation with the annexB extensions. - Implement and use NamedEvaluation where specified. Additionally there are fixes to things exposed by these changes to eval, {for, for-in, for-of} loops and assignment. Finally it also fixes some tests in test-js which where passing before but not now that we have correct behavior :^).
2021-09-22 10:44:56 +00:00
auto has_annexB_for_in_init_extension = false;
LibJS: Remove hand-rolled type information in JS AST in favor of RTTI
2021-01-01 18:34:07 +00:00
if (is<VariableDeclaration>(*lhs)) {
LibJS: Make scoping follow the spec Before this we used an ad-hoc combination of references and 'variables' stored in a hashmap. This worked in most cases but is not spec like. Additionally hoisting, dynamically naming functions and scope analysis was not done properly. This patch fixes all of that by: - Implement BindingInitialization for destructuring assignment. - Implementing a new ScopePusher which tracks the lexical and var scoped declarations. This hoists functions to the top level if no lexical declaration name overlaps. Furthermore we do checking of redeclarations in the ScopePusher now requiring less checks all over the place. - Add methods for parsing the directives and statement lists instead of having that code duplicated in multiple places. This allows declarations to pushed to the appropriate scope more easily. - Remove the non spec way of storing 'variables' in DeclarativeEnvironment and make Reference follow the spec instead of checking both the bindings and 'variables'. - Remove all scoping related things from the Interpreter. And instead use environments as specified by the spec. This also includes fixing that NativeFunctions did not produce a valid FunctionEnvironment which could cause issues with callbacks and eval. All FunctionObjects now have a valid NewFunctionEnvironment implementation. - Remove execute_statements from Interpreter and instead use ASTNode::execute everywhere this simplifies AST.cpp as you no longer need to worry about which method to call. - Make ScopeNodes setup their own environment. This uses four different methods specified by the spec {Block, Function, Eval, Global}DeclarationInstantiation with the annexB extensions. - Implement and use NamedEvaluation where specified. Additionally there are fixes to things exposed by these changes to eval, {for, for-in, for-of} loops and assignment. Finally it also fixes some tests in test-js which where passing before but not now that we have correct behavior :^).
2021-09-22 10:44:56 +00:00
auto& declaration = static_cast<VariableDeclaration&>(*lhs);
if (declaration.declarations().size() > 1) {
syntax_error("Multiple declarations not allowed in for..in/of");
} else if (declaration.declarations().size() < 1) {
syntax_error("Need exactly one variable declaration in for..in/of");
} else {
// AnnexB extension B.3.5 Initializers in ForIn Statement Heads, https://tc39.es/ecma262/#sec-initializers-in-forin-statement-heads
auto& variable = declaration.declarations().first();
if (variable.init()) {
if (m_state.strict_mode || declaration.declaration_kind() != DeclarationKind::Var || !variable.target().has<NonnullRefPtr<Identifier>>())
syntax_error("Variable initializer not allowed in for..in/of");
else
has_annexB_for_in_init_extension = true;
}
}
} else if (!lhs->is_identifier() && !is<MemberExpression>(*lhs)) {
bool valid = false;
if (is<ObjectExpression>(*lhs) || is<ArrayExpression>(*lhs)) {
auto synthesized_binding_pattern = synthesize_binding_pattern(static_cast<Expression const&>(*lhs));
if (synthesized_binding_pattern) {
for_declaration = synthesized_binding_pattern.release_nonnull();
valid = true;
}
}
if (!valid)
syntax_error(String::formatted("Invalid left-hand side in for-loop ('{}')", lhs->class_name()));
LibJS: Implement basic for..in and for..of loops
2020-04-21 18:21:26 +00:00
}
auto in_or_of = consume();
LibJS: Make scoping follow the spec Before this we used an ad-hoc combination of references and 'variables' stored in a hashmap. This worked in most cases but is not spec like. Additionally hoisting, dynamically naming functions and scope analysis was not done properly. This patch fixes all of that by: - Implement BindingInitialization for destructuring assignment. - Implementing a new ScopePusher which tracks the lexical and var scoped declarations. This hoists functions to the top level if no lexical declaration name overlaps. Furthermore we do checking of redeclarations in the ScopePusher now requiring less checks all over the place. - Add methods for parsing the directives and statement lists instead of having that code duplicated in multiple places. This allows declarations to pushed to the appropriate scope more easily. - Remove the non spec way of storing 'variables' in DeclarativeEnvironment and make Reference follow the spec instead of checking both the bindings and 'variables'. - Remove all scoping related things from the Interpreter. And instead use environments as specified by the spec. This also includes fixing that NativeFunctions did not produce a valid FunctionEnvironment which could cause issues with callbacks and eval. All FunctionObjects now have a valid NewFunctionEnvironment implementation. - Remove execute_statements from Interpreter and instead use ASTNode::execute everywhere this simplifies AST.cpp as you no longer need to worry about which method to call. - Make ScopeNodes setup their own environment. This uses four different methods specified by the spec {Block, Function, Eval, Global}DeclarationInstantiation with the annexB extensions. - Implement and use NamedEvaluation where specified. Additionally there are fixes to things exposed by these changes to eval, {for, for-in, for-of} loops and assignment. Finally it also fixes some tests in test-js which where passing before but not now that we have correct behavior :^).
2021-09-22 10:44:56 +00:00
auto is_in = in_or_of.type() == TokenType::In;
LibJS: Be more strict about the lhs of a for in/of loop This is not entirely correct as really Object- and ArrayExpressions are not allowed but that requires a bigger refactoring of for statement parsing.
2021-07-28 21:09:57 +00:00
LibJS: Make scoping follow the spec Before this we used an ad-hoc combination of references and 'variables' stored in a hashmap. This worked in most cases but is not spec like. Additionally hoisting, dynamically naming functions and scope analysis was not done properly. This patch fixes all of that by: - Implement BindingInitialization for destructuring assignment. - Implementing a new ScopePusher which tracks the lexical and var scoped declarations. This hoists functions to the top level if no lexical declaration name overlaps. Furthermore we do checking of redeclarations in the ScopePusher now requiring less checks all over the place. - Add methods for parsing the directives and statement lists instead of having that code duplicated in multiple places. This allows declarations to pushed to the appropriate scope more easily. - Remove the non spec way of storing 'variables' in DeclarativeEnvironment and make Reference follow the spec instead of checking both the bindings and 'variables'. - Remove all scoping related things from the Interpreter. And instead use environments as specified by the spec. This also includes fixing that NativeFunctions did not produce a valid FunctionEnvironment which could cause issues with callbacks and eval. All FunctionObjects now have a valid NewFunctionEnvironment implementation. - Remove execute_statements from Interpreter and instead use ASTNode::execute everywhere this simplifies AST.cpp as you no longer need to worry about which method to call. - Make ScopeNodes setup their own environment. This uses four different methods specified by the spec {Block, Function, Eval, Global}DeclarationInstantiation with the annexB extensions. - Implement and use NamedEvaluation where specified. Additionally there are fixes to things exposed by these changes to eval, {for, for-in, for-of} loops and assignment. Finally it also fixes some tests in test-js which where passing before but not now that we have correct behavior :^).
2021-09-22 10:44:56 +00:00
if (!is_in) {
LibJS: Be more strict about the lhs of a for in/of loop This is not entirely correct as really Object- and ArrayExpressions are not allowed but that requires a bigger refactoring of for statement parsing.
2021-07-28 21:09:57 +00:00
if (is<MemberExpression>(*lhs)) {
auto& member = static_cast<MemberExpression const&>(*lhs);
if (member.object().is_identifier() && static_cast<Identifier const&>(member.object()).string() == "let"sv)
syntax_error("For of statement may not start with let.");
}
LibJS: Make scoping follow the spec Before this we used an ad-hoc combination of references and 'variables' stored in a hashmap. This worked in most cases but is not spec like. Additionally hoisting, dynamically naming functions and scope analysis was not done properly. This patch fixes all of that by: - Implement BindingInitialization for destructuring assignment. - Implementing a new ScopePusher which tracks the lexical and var scoped declarations. This hoists functions to the top level if no lexical declaration name overlaps. Furthermore we do checking of redeclarations in the ScopePusher now requiring less checks all over the place. - Add methods for parsing the directives and statement lists instead of having that code duplicated in multiple places. This allows declarations to pushed to the appropriate scope more easily. - Remove the non spec way of storing 'variables' in DeclarativeEnvironment and make Reference follow the spec instead of checking both the bindings and 'variables'. - Remove all scoping related things from the Interpreter. And instead use environments as specified by the spec. This also includes fixing that NativeFunctions did not produce a valid FunctionEnvironment which could cause issues with callbacks and eval. All FunctionObjects now have a valid NewFunctionEnvironment implementation. - Remove execute_statements from Interpreter and instead use ASTNode::execute everywhere this simplifies AST.cpp as you no longer need to worry about which method to call. - Make ScopeNodes setup their own environment. This uses four different methods specified by the spec {Block, Function, Eval, Global}DeclarationInstantiation with the annexB extensions. - Implement and use NamedEvaluation where specified. Additionally there are fixes to things exposed by these changes to eval, {for, for-in, for-of} loops and assignment. Finally it also fixes some tests in test-js which where passing before but not now that we have correct behavior :^).
2021-09-22 10:44:56 +00:00
if (has_annexB_for_in_init_extension)
syntax_error("Variable initializer not allowed in for..of", rule_start.position());
LibJS: Be more strict about the lhs of a for in/of loop This is not entirely correct as really Object- and ArrayExpressions are not allowed but that requires a bigger refactoring of for statement parsing.
2021-07-28 21:09:57 +00:00
}
LibJS: Make scoping follow the spec Before this we used an ad-hoc combination of references and 'variables' stored in a hashmap. This worked in most cases but is not spec like. Additionally hoisting, dynamically naming functions and scope analysis was not done properly. This patch fixes all of that by: - Implement BindingInitialization for destructuring assignment. - Implementing a new ScopePusher which tracks the lexical and var scoped declarations. This hoists functions to the top level if no lexical declaration name overlaps. Furthermore we do checking of redeclarations in the ScopePusher now requiring less checks all over the place. - Add methods for parsing the directives and statement lists instead of having that code duplicated in multiple places. This allows declarations to pushed to the appropriate scope more easily. - Remove the non spec way of storing 'variables' in DeclarativeEnvironment and make Reference follow the spec instead of checking both the bindings and 'variables'. - Remove all scoping related things from the Interpreter. And instead use environments as specified by the spec. This also includes fixing that NativeFunctions did not produce a valid FunctionEnvironment which could cause issues with callbacks and eval. All FunctionObjects now have a valid NewFunctionEnvironment implementation. - Remove execute_statements from Interpreter and instead use ASTNode::execute everywhere this simplifies AST.cpp as you no longer need to worry about which method to call. - Make ScopeNodes setup their own environment. This uses four different methods specified by the spec {Block, Function, Eval, Global}DeclarationInstantiation with the annexB extensions. - Implement and use NamedEvaluation where specified. Additionally there are fixes to things exposed by these changes to eval, {for, for-in, for-of} loops and assignment. Finally it also fixes some tests in test-js which where passing before but not now that we have correct behavior :^).
2021-09-22 10:44:56 +00:00
auto rhs = parse_expression(is_in ? 0 : 2);
LibJS: Implement basic for..in and for..of loops
2020-04-21 18:21:26 +00:00
consume(TokenType::ParenClose);
LibJS: Disallow 'continue' & 'break' outside of their respective scopes 'continue' is no longer allowed outside of a loop, and an unlabeled 'break' is not longer allowed outside of a loop or switch statement. Labeled 'break' statements are still allowed everywhere, even if the label does not exist.
2020-10-07 19:21:15 +00:00
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
TemporaryChange break_change(m_state.in_break_context, true);
TemporaryChange continue_change(m_state.in_continue_context, true);
LibJS: Make scoping follow the spec Before this we used an ad-hoc combination of references and 'variables' stored in a hashmap. This worked in most cases but is not spec like. Additionally hoisting, dynamically naming functions and scope analysis was not done properly. This patch fixes all of that by: - Implement BindingInitialization for destructuring assignment. - Implementing a new ScopePusher which tracks the lexical and var scoped declarations. This hoists functions to the top level if no lexical declaration name overlaps. Furthermore we do checking of redeclarations in the ScopePusher now requiring less checks all over the place. - Add methods for parsing the directives and statement lists instead of having that code duplicated in multiple places. This allows declarations to pushed to the appropriate scope more easily. - Remove the non spec way of storing 'variables' in DeclarativeEnvironment and make Reference follow the spec instead of checking both the bindings and 'variables'. - Remove all scoping related things from the Interpreter. And instead use environments as specified by the spec. This also includes fixing that NativeFunctions did not produce a valid FunctionEnvironment which could cause issues with callbacks and eval. All FunctionObjects now have a valid NewFunctionEnvironment implementation. - Remove execute_statements from Interpreter and instead use ASTNode::execute everywhere this simplifies AST.cpp as you no longer need to worry about which method to call. - Make ScopeNodes setup their own environment. This uses four different methods specified by the spec {Block, Function, Eval, Global}DeclarationInstantiation with the annexB extensions. - Implement and use NamedEvaluation where specified. Additionally there are fixes to things exposed by these changes to eval, {for, for-in, for-of} loops and assignment. Finally it also fixes some tests in test-js which where passing before but not now that we have correct behavior :^).
2021-09-22 10:44:56 +00:00
ScopePusher for_loop_scope = ScopePusher::for_loop_scope(*this, lhs);
LibJS: Implement basic for..in and for..of loops
2020-04-21 18:21:26 +00:00
auto body = parse_statement();
LibJS: Make scoping follow the spec Before this we used an ad-hoc combination of references and 'variables' stored in a hashmap. This worked in most cases but is not spec like. Additionally hoisting, dynamically naming functions and scope analysis was not done properly. This patch fixes all of that by: - Implement BindingInitialization for destructuring assignment. - Implementing a new ScopePusher which tracks the lexical and var scoped declarations. This hoists functions to the top level if no lexical declaration name overlaps. Furthermore we do checking of redeclarations in the ScopePusher now requiring less checks all over the place. - Add methods for parsing the directives and statement lists instead of having that code duplicated in multiple places. This allows declarations to pushed to the appropriate scope more easily. - Remove the non spec way of storing 'variables' in DeclarativeEnvironment and make Reference follow the spec instead of checking both the bindings and 'variables'. - Remove all scoping related things from the Interpreter. And instead use environments as specified by the spec. This also includes fixing that NativeFunctions did not produce a valid FunctionEnvironment which could cause issues with callbacks and eval. All FunctionObjects now have a valid NewFunctionEnvironment implementation. - Remove execute_statements from Interpreter and instead use ASTNode::execute everywhere this simplifies AST.cpp as you no longer need to worry about which method to call. - Make ScopeNodes setup their own environment. This uses four different methods specified by the spec {Block, Function, Eval, Global}DeclarationInstantiation with the annexB extensions. - Implement and use NamedEvaluation where specified. Additionally there are fixes to things exposed by these changes to eval, {for, for-in, for-of} loops and assignment. Finally it also fixes some tests in test-js which where passing before but not now that we have correct behavior :^).
2021-09-22 10:44:56 +00:00
if (is_in)
return create_ast_node<ForInStatement>({ m_state.current_token.filename(), rule_start.position(), position() }, move(for_declaration), move(rhs), move(body));
LibJS: Implement parsing and executing for-await-of loops
2021-11-23 15:09:28 +00:00
if (is_for_await_loop == IsForAwaitLoop::Yes)
return create_ast_node<ForAwaitOfStatement>({ m_state.current_token.filename(), rule_start.position(), position() }, move(for_declaration), move(rhs), move(body));
LibJS: Make scoping follow the spec Before this we used an ad-hoc combination of references and 'variables' stored in a hashmap. This worked in most cases but is not spec like. Additionally hoisting, dynamically naming functions and scope analysis was not done properly. This patch fixes all of that by: - Implement BindingInitialization for destructuring assignment. - Implementing a new ScopePusher which tracks the lexical and var scoped declarations. This hoists functions to the top level if no lexical declaration name overlaps. Furthermore we do checking of redeclarations in the ScopePusher now requiring less checks all over the place. - Add methods for parsing the directives and statement lists instead of having that code duplicated in multiple places. This allows declarations to pushed to the appropriate scope more easily. - Remove the non spec way of storing 'variables' in DeclarativeEnvironment and make Reference follow the spec instead of checking both the bindings and 'variables'. - Remove all scoping related things from the Interpreter. And instead use environments as specified by the spec. This also includes fixing that NativeFunctions did not produce a valid FunctionEnvironment which could cause issues with callbacks and eval. All FunctionObjects now have a valid NewFunctionEnvironment implementation. - Remove execute_statements from Interpreter and instead use ASTNode::execute everywhere this simplifies AST.cpp as you no longer need to worry about which method to call. - Make ScopeNodes setup their own environment. This uses four different methods specified by the spec {Block, Function, Eval, Global}DeclarationInstantiation with the annexB extensions. - Implement and use NamedEvaluation where specified. Additionally there are fixes to things exposed by these changes to eval, {for, for-in, for-of} loops and assignment. Finally it also fixes some tests in test-js which where passing before but not now that we have correct behavior :^).
2021-09-22 10:44:56 +00:00
return create_ast_node<ForOfStatement>({ m_state.current_token.filename(), rule_start.position(), position() }, move(for_declaration), move(rhs), move(body));
LibJS: Implement basic for..in and for..of loops
2020-04-21 18:21:26 +00:00
}
LibJS: Implement (no-op) debugger statement
2020-04-30 16:26:27 +00:00
NonnullRefPtr<DebuggerStatement> Parser::parse_debugger_statement()
{
LibJS: Track source positions all the way down to exceptions This makes exceptions have a trace of source positions too, which could probably be helpful in making fancier error tracebacks.
2020-12-28 17:15:22 +00:00
auto rule_start = push_start();
LibJS: Implement (no-op) debugger statement
2020-04-30 16:26:27 +00:00
consume(TokenType::Debugger);
consume_or_insert_semicolon();
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
return create_ast_node<DebuggerStatement>({ m_state.current_token.filename(), rule_start.position(), position() });
LibJS: Implement (no-op) debugger statement
2020-04-30 16:26:27 +00:00
}
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
bool Parser::match(TokenType type) const
{
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
return m_state.current_token.type() == type;
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
}
bool Parser::match_expression() const
{
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
auto type = m_state.current_token.type();
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
return type == TokenType::BoolLiteral
|| type == TokenType::NumericLiteral
LibJS: Add BigInt
2020-06-06 00:14:10 +00:00
|| type == TokenType::BigIntLiteral
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
|| type == TokenType::StringLiteral
LibJS: Add template literals Adds fully functioning template literals. Because template literals contain expressions, most of the work has to be done in the Lexer rather than the Parser. And because of the complexity of template literals (expressions, nesting, escapes, etc), the Lexer needs to have some template-related state. When entering a new template literal, a TemplateLiteralStart token is emitted. When inside a literal, all text will be parsed up until a '${' or '`' (or EOF, but that's a syntax error) is seen, and then a TemplateLiteralExprStart token is emitted. At this point, the Lexer proceeds as normal, however it keeps track of the number of opening and closing curly braces it has seen in order to determine the close of the expression. Once it finds a matching curly brace for the '${', a TemplateLiteralExprEnd token is emitted and the state is updated accordingly. When the Lexer is inside of a template literal, but not an expression, and sees a '`', this must be the closing grave: a TemplateLiteralEnd token is emitted. The state required to correctly parse template strings consists of a vector (for nesting) of two pieces of information: whether or not we are in a template expression (as opposed to a template string); and the count of the number of unmatched open curly braces we have seen (only applicable if the Lexer is currently in a template expression). TODO: Add support for template literal newlines in the JS REPL (this will cause a syntax error currently): > `foo > bar` 'foo bar'
2020-05-03 22:41:14 +00:00
|| type == TokenType::TemplateLiteralStart
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
|| type == TokenType::NullLiteral
LibJS: Rework Identifier parsing to match the spec more closely
2021-07-11 11:04:55 +00:00
|| match_identifier()
LibJS: Add parsing and evaluation of private fields and methods
2021-10-12 20:45:52 +00:00
|| (type == TokenType::PrivateIdentifier && next_token().type() == TokenType::In)
LibJS: Handle escaped keywords in more cases and handle 'await' labels
2021-09-18 21:02:50 +00:00
|| type == TokenType::Await
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
|| type == TokenType::New
LibJS: Fix small issues in parser - Fix some places where escaped keywords are (not) allowed. - Be more strict about parameters for functions with 'use strict'. - Fix that expressions statements allowed functions and classes. - Fix that class expressions were not allowed. - Added a new next_token() method for checking the look ahead. - Fix that continue labels could jump to non iterating targets. - Fix that generator functions cannot be declared in if statements.
2021-08-28 15:04:37 +00:00
|| type == TokenType::Class
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
|| type == TokenType::CurlyOpen
|| type == TokenType::BracketOpen
LibJS/Parser: Implement the parsing of unary/prefixed update expressions
2020-03-14 18:45:51 +00:00
|| type == TokenType::ParenOpen
LibJS: Parse FunctionExpressions FunctionExpression is mostly like FunctionDeclaration, except the name is optional. Share the parsing logic in parse_function_node<NodeType>. This allows us to do nice things like: document.addEventListener("DOMContentLoaded", function() { alert("Hello friends!"); });
2020-03-19 10:52:56 +00:00
|| type == TokenType::Function
LibJS: Add support for async functions This commit adds support for the most bare bones version of async functions, support for async generator functions, async arrow functions and await expressions are TODO.
2021-11-09 18:39:22 +00:00
|| type == TokenType::Async
LibJS: Parse "this" as ThisExpression
2020-04-12 22:42:14 +00:00
|| type == TokenType::This
LibJS: Initial class implementation; allow super expressions in object literal methods; add EnvrionmentRecord fields and methods to LexicalEnvironment Adding EnvrionmentRecord's fields and methods lets us throw an exception when |this| is not initialized, which occurs when the super constructor in a derived class has not yet been called, or when |this| has already been initialized (the super constructor was already called).
2020-06-08 18:31:21 +00:00
|| type == TokenType::Super
LibJS: Lex and parse regex literals, add RegExp objects This adds regex parsing/lexing, as well as a relatively empty RegExpObject. The purpose of this patch is to allow the engine to not get hung up on parsing regexes. This will aid in finding new syntax errors (say, from google or twitter) without having to replace all of their regexes first!
2020-06-03 23:05:49 +00:00
|| type == TokenType::RegexLiteral
LibJS: Force the lexer to parse a regex when expecting a statement
2021-07-29 21:28:28 +00:00
|| type == TokenType::Slash // Wrongly recognized regex by lexer
|| type == TokenType::SlashEquals // Wrongly recognized regex by lexer (/=a/ is a valid regex)
LibJS: Implement generator functions (only in bytecode mode)
2021-06-10 21:08:30 +00:00
|| type == TokenType::Yield
LibJS/Parser: Implement the parsing of unary/prefixed update expressions
2020-03-14 18:45:51 +00:00
|| match_unary_prefixed_expression();
}
bool Parser::match_unary_prefixed_expression() const
{
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
auto type = m_state.current_token.type();
LibJS/Parser: Implement the parsing of unary/prefixed update expressions
2020-03-14 18:45:51 +00:00
return type == TokenType::PlusPlus
|| type == TokenType::MinusMinus
|| type == TokenType::ExclamationMark
LibJS: Implement typeof operator
2020-03-17 19:33:32 +00:00
|| type == TokenType::Tilde
LibJS: Implement unary plus / minus
2020-04-02 16:58:39 +00:00
|| type == TokenType::Plus
|| type == TokenType::Minus
LibJS: Implement void operator
2020-04-15 16:55:03 +00:00
|| type == TokenType::Typeof
LibJS: Implement basic support for the "delete" operator It turns out "delete" is actually a unary op :) This patch implements deletion of object properties, it doesn't yet work for casually deleting properties from the global object. When deleting a property from an object, we switch that object to having a unique shape, no longer sharing shapes with others. Once an object has a unique shape, it no longer needs to care about shape transitions.
2020-04-26 11:53:40 +00:00
|| type == TokenType::Void
|| type == TokenType::Delete;
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
}
LibJS: Use const references to avoid some copies in the parser
2021-02-24 08:58:24 +00:00
bool Parser::match_secondary_expression(const Vector<TokenType>& forbidden) const
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
{
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
auto type = m_state.current_token.type();
LibJS: Implement basic for..in and for..of loops
2020-04-21 18:21:26 +00:00
if (forbidden.contains_slow(type))
return false;
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
return type == TokenType::Plus
LibJS: Implement +=, -=, *=, and /= assignment operators
2020-03-12 12:09:15 +00:00
|| type == TokenType::PlusEquals
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
|| type == TokenType::Minus
LibJS: Implement +=, -=, *=, and /= assignment operators
2020-03-12 12:09:15 +00:00
|| type == TokenType::MinusEquals
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
|| type == TokenType::Asterisk
LibJS: Implement +=, -=, *=, and /= assignment operators
2020-03-12 12:09:15 +00:00
|| type == TokenType::AsteriskEquals
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
|| type == TokenType::Slash
LibJS: Implement +=, -=, *=, and /= assignment operators
2020-03-12 12:09:15 +00:00
|| type == TokenType::SlashEquals
LibJS: Add basic support for modulo (%) in binary expressions
2020-04-04 19:17:34 +00:00
|| type == TokenType::Percent
LibJS: Implement modulo assignment operator (%=)
2020-05-04 22:07:05 +00:00
|| type == TokenType::PercentEquals
LibJS: Implement exponentiation (** operator)
2020-04-05 12:40:00 +00:00
|| type == TokenType::DoubleAsterisk
LibJS: Implement exponentiation assignment operator (**=)
2020-05-04 22:03:35 +00:00
|| type == TokenType::DoubleAsteriskEquals
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
|| type == TokenType::Equals
LibJS: Parse === and !== binary operators
2020-03-12 12:11:33 +00:00
|| type == TokenType::EqualsEqualsEquals
|| type == TokenType::ExclamationMarkEqualsEquals
LibJS: Implement abstract equality and inequality
2020-03-15 22:23:38 +00:00
|| type == TokenType::EqualsEquals
|| type == TokenType::ExclamationMarkEquals
LibJS: Parse > and < binary operators
2020-03-12 12:10:27 +00:00
|| type == TokenType::GreaterThan
LibJS: Implement <= and >= binary operators
2020-03-12 12:07:08 +00:00
|| type == TokenType::GreaterThanEquals
LibJS: Parse > and < binary operators
2020-03-12 12:10:27 +00:00
|| type == TokenType::LessThan
LibJS: Implement <= and >= binary operators
2020-03-12 12:07:08 +00:00
|| type == TokenType::LessThanEquals
LibJS: Implement basic MemberExpression parsing At last we can parse "hello friends".length :^)
2020-03-12 12:05:57 +00:00
|| type == TokenType::ParenOpen
LibJS: Implement update expressions Note that currently only the non-prefixed variant is supported (i.e i++ not ++i), this variant returns the value of the argument before the update.
2020-03-12 11:45:45 +00:00
|| type == TokenType::Period
LibJS: Parse computed MemberExpressions MemberExpression comes in two flavors: computed: a[b] non-computed: a.b We can now parse both of the types. :^)
2020-03-20 19:51:03 +00:00
|| type == TokenType::BracketOpen
LibJS: Handle '++' and '--' more correctly within expression
2021-08-14 15:02:15 +00:00
|| (type == TokenType::PlusPlus && !m_state.current_token.trivia_contains_line_terminator())
|| (type == TokenType::MinusMinus && !m_state.current_token.trivia_contains_line_terminator())
LibJS: Implement 'in' operator
2020-04-23 15:06:01 +00:00
|| type == TokenType::In
LibJS: Implement ConditionalExpression (ternary "?:" operator)
2020-04-03 10:14:28 +00:00
|| type == TokenType::Instanceof
LibJS: Parse binary bitwise operators The Lexer and AST already have all the functionality required in place, so this is just updating Parser::match_secondary_expression() and Parser::parse_expression() to handle TokenType::{Ampersand,Pipe,Caret}, as well as adding some tests.
2020-04-03 12:02:31 +00:00
|| type == TokenType::QuestionMark
|| type == TokenType::Ampersand
LibJS: Implement bitwise assignment operators (&=, |=, ^=)
2020-05-04 21:34:45 +00:00
|| type == TokenType::AmpersandEquals
LibJS: Parse binary bitwise operators The Lexer and AST already have all the functionality required in place, so this is just updating Parser::match_secondary_expression() and Parser::parse_expression() to handle TokenType::{Ampersand,Pipe,Caret}, as well as adding some tests.
2020-04-03 12:02:31 +00:00
|| type == TokenType::Pipe
LibJS: Implement bitwise assignment operators (&=, |=, ^=)
2020-05-04 21:34:45 +00:00
|| type == TokenType::PipeEquals
LibJS: Fix logical expressions The current implementation of logical AND (&&) and logical OR (||) always returns a boolean - this is not correct.
2020-04-03 13:33:28 +00:00
|| type == TokenType::Caret
LibJS: Implement bitwise assignment operators (&=, |=, ^=)
2020-05-04 21:34:45 +00:00
|| type == TokenType::CaretEquals
LibJS: Implement bitwise left shift operator (<<)
2020-04-23 12:36:14 +00:00
|| type == TokenType::ShiftLeft
|| type == TokenType::ShiftLeftEquals
LibJS: Implement bitwise right shift operator (>>)
2020-04-23 12:45:19 +00:00
|| type == TokenType::ShiftRight
|| type == TokenType::ShiftRightEquals
LibJS: Implement bitwise unsigned right shift operator (>>>)
2020-04-23 14:43:10 +00:00
|| type == TokenType::UnsignedShiftRight
|| type == TokenType::UnsignedShiftRightEquals
LibJS: Fix logical expressions The current implementation of logical AND (&&) and logical OR (||) always returns a boolean - this is not correct.
2020-04-03 13:33:28 +00:00
|| type == TokenType::DoubleAmpersand
LibJS: Implement logical assignment operators (&&=, ||=, ??=) TC39 proposal, stage 4 as of 2020-07. https://tc39.es/proposal-logical-assignment/
2020-10-05 15:49:43 +00:00
|| type == TokenType::DoubleAmpersandEquals
LibJS: Implement nullish coalescing operator (??)
2020-04-17 23:49:11 +00:00
|| type == TokenType::DoublePipe
LibJS: Implement logical assignment operators (&&=, ||=, ??=) TC39 proposal, stage 4 as of 2020-07. https://tc39.es/proposal-logical-assignment/
2020-10-05 15:49:43 +00:00
|| type == TokenType::DoublePipeEquals
|| type == TokenType::DoubleQuestionMark
LibJS: Implement parsing and execution of optional chains
2021-09-14 02:26:31 +00:00
|| type == TokenType::DoubleQuestionMarkEquals
|| type == TokenType::QuestionMarkPeriod;
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
}
bool Parser::match_statement() const
{
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
auto type = m_state.current_token.type();
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
return match_expression()
|| type == TokenType::Return
LibJS: Implement generator functions (only in bytecode mode)
2021-06-10 21:08:30 +00:00
|| type == TokenType::Yield
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
|| type == TokenType::Do
|| type == TokenType::If
LibJS: Implement "throw" You can now throw an expression to the nearest catcher! :^) To support throwing arbitrary values, I added an Exception class that sits as a wrapper around whatever is thrown. In the future it will be a logical place to store a call stack.
2020-03-24 21:03:50 +00:00
|| type == TokenType::Throw
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
|| type == TokenType::Try
|| type == TokenType::While
LibJS: Parse "with" statements :^)
2020-11-28 14:05:57 +00:00
|| type == TokenType::With
LibJS: Implement for statement
2020-03-12 12:12:12 +00:00
|| type == TokenType::For
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
|| type == TokenType::CurlyOpen
LibJS: Lexer and parser support for "switch" statements
2020-03-29 11:09:54 +00:00
|| type == TokenType::Switch
|| type == TokenType::Break
LibJS: Add support for "continue" inside "for" statements :^)
2020-04-04 22:22:42 +00:00
|| type == TokenType::Continue
LibJS: Implement (no-op) debugger statement
2020-04-30 16:26:27 +00:00
|| type == TokenType::Var
LibJS: Support empty statements We already skipped random semicolons in Parser::parse_program(), but now they are properly matched and parsed as empty statements - and thus recognized as a valid body of an if / else / while / ... statement.
2020-05-03 09:59:00 +00:00
|| type == TokenType::Debugger
|| type == TokenType::Semicolon;
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
}
LibJS: Parse and partially execute import and export statements We produce the import and export entries as per the spec. However we do not yet verify that named things that are exported are declared somewhere.
2021-08-14 15:42:30 +00:00
bool Parser::match_export_or_import() const
{
auto type = m_state.current_token.type();
return type == TokenType::Export
|| type == TokenType::Import;
}
LibJS: Do not save state for peeking at the next token from the lexer This saves having to save and load the parser state. This could give an incorrect token in some cases where the parser communicates to the lexer. However this is not applicable in any of the current usages and this would require one to parse the current token as normal which is exactly what you don't want to do in that scenario.
2021-10-07 22:38:24 +00:00
bool Parser::match_declaration() const
LibJS: Distinguish between statement and declaration This separates matching/parsing of statements and declarations and fixes a few edge cases where the parser would incorrectly accept a declaration where only a statement is allowed - for example: if (foo) const a = 1; for (var bar;;) function b() {} while (baz) class c {}
2020-10-22 22:30:07 +00:00
{
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
auto type = m_state.current_token.type();
LibJS: Check that 'let' is followed by declaration before matching it Since 'let' is a valid variable name (in non-strict mode) let may not be the start of a declaration but just an identifier.
2021-07-29 00:03:16 +00:00
if (type == TokenType::Let && !m_state.strict_mode) {
return try_match_let_declaration();
}
LibJS: Distinguish between statement and declaration This separates matching/parsing of statements and declarations and fixes a few edge cases where the parser would incorrectly accept a declaration where only a statement is allowed - for example: if (foo) const a = 1; for (var bar;;) function b() {} while (baz) class c {}
2020-10-22 22:30:07 +00:00
return type == TokenType::Function
|| type == TokenType::Class
|| type == TokenType::Const
LibJS: Add support for async functions This commit adds support for the most bare bones version of async functions, support for async generator functions, async arrow functions and await expressions are TODO.
2021-11-09 18:39:22 +00:00
|| type == TokenType::Let
|| (type == TokenType::Async && next_token().type() == TokenType::Function);
LibJS: Distinguish between statement and declaration This separates matching/parsing of statements and declarations and fixes a few edge cases where the parser would incorrectly accept a declaration where only a statement is allowed - for example: if (foo) const a = 1; for (var bar;;) function b() {} while (baz) class c {}
2020-10-22 22:30:07 +00:00
}
LibJS: Do not save state for peeking at the next token from the lexer This saves having to save and load the parser state. This could give an incorrect token in some cases where the parser communicates to the lexer. However this is not applicable in any of the current usages and this would require one to parse the current token as normal which is exactly what you don't want to do in that scenario.
2021-10-07 22:38:24 +00:00
Token Parser::next_token() const
LibJS: Check that 'let' is followed by declaration before matching it Since 'let' is a valid variable name (in non-strict mode) let may not be the start of a declaration but just an identifier.
2021-07-29 00:03:16 +00:00
{
LibJS: Do not save state for peeking at the next token from the lexer This saves having to save and load the parser state. This could give an incorrect token in some cases where the parser communicates to the lexer. However this is not applicable in any of the current usages and this would require one to parse the current token as normal which is exactly what you don't want to do in that scenario.
2021-10-07 22:38:24 +00:00
Lexer lookahead_lexer = m_state.lexer;
LibJS: Check that 'let' is followed by declaration before matching it Since 'let' is a valid variable name (in non-strict mode) let may not be the start of a declaration but just an identifier.
2021-07-29 00:03:16 +00:00
LibJS: Do not save state for peeking at the next token from the lexer This saves having to save and load the parser state. This could give an incorrect token in some cases where the parser communicates to the lexer. However this is not applicable in any of the current usages and this would require one to parse the current token as normal which is exactly what you don't want to do in that scenario.
2021-10-07 22:38:24 +00:00
auto token_after = lookahead_lexer.next();
LibJS: Fix small issues in parser - Fix some places where escaped keywords are (not) allowed. - Be more strict about parameters for functions with 'use strict'. - Fix that expressions statements allowed functions and classes. - Fix that class expressions were not allowed. - Added a new next_token() method for checking the look ahead. - Fix that continue labels could jump to non iterating targets. - Fix that generator functions cannot be declared in if statements.
2021-08-28 15:04:37 +00:00
return token_after;
}
LibJS: Check that 'let' is followed by declaration before matching it Since 'let' is a valid variable name (in non-strict mode) let may not be the start of a declaration but just an identifier.
2021-07-29 00:03:16 +00:00
LibJS: Do not save state for peeking at the next token from the lexer This saves having to save and load the parser state. This could give an incorrect token in some cases where the parser communicates to the lexer. However this is not applicable in any of the current usages and this would require one to parse the current token as normal which is exactly what you don't want to do in that scenario.
2021-10-07 22:38:24 +00:00
bool Parser::try_match_let_declaration() const
LibJS: Fix small issues in parser - Fix some places where escaped keywords are (not) allowed. - Be more strict about parameters for functions with 'use strict'. - Fix that expressions statements allowed functions and classes. - Fix that class expressions were not allowed. - Added a new next_token() method for checking the look ahead. - Fix that continue labels could jump to non iterating targets. - Fix that generator functions cannot be declared in if statements.
2021-08-28 15:04:37 +00:00
{
VERIFY(m_state.current_token.type() == TokenType::Let);
auto token_after = next_token();
LibJS: Check that 'let' is followed by declaration before matching it Since 'let' is a valid variable name (in non-strict mode) let may not be the start of a declaration but just an identifier.
2021-07-29 00:03:16 +00:00
LibJS: Fix small issues in parser - Fix some places where escaped keywords are (not) allowed. - Be more strict about parameters for functions with 'use strict'. - Fix that expressions statements allowed functions and classes. - Fix that class expressions were not allowed. - Added a new next_token() method for checking the look ahead. - Fix that continue labels could jump to non iterating targets. - Fix that generator functions cannot be declared in if statements.
2021-08-28 15:04:37 +00:00
if (token_after.is_identifier_name() && token_after.value() != "in"sv)
LibJS: Check that 'let' is followed by declaration before matching it Since 'let' is a valid variable name (in non-strict mode) let may not be the start of a declaration but just an identifier.
2021-07-29 00:03:16 +00:00
return true;
LibJS: Fix small issues in parser - Fix some places where escaped keywords are (not) allowed. - Be more strict about parameters for functions with 'use strict'. - Fix that expressions statements allowed functions and classes. - Fix that class expressions were not allowed. - Added a new next_token() method for checking the look ahead. - Fix that continue labels could jump to non iterating targets. - Fix that generator functions cannot be declared in if statements.
2021-08-28 15:04:37 +00:00
if (token_after.type() == TokenType::CurlyOpen || token_after.type() == TokenType::BracketOpen)
LibJS: Check that 'let' is followed by declaration before matching it Since 'let' is a valid variable name (in non-strict mode) let may not be the start of a declaration but just an identifier.
2021-07-29 00:03:16 +00:00
return true;
return false;
}
LibJS: Do not save state for peeking at the next token from the lexer This saves having to save and load the parser state. This could give an incorrect token in some cases where the parser communicates to the lexer. However this is not applicable in any of the current usages and this would require one to parse the current token as normal which is exactly what you don't want to do in that scenario.
2021-10-07 22:38:24 +00:00
bool Parser::match_variable_declaration() const
LibJS: Distinguish between statement and declaration This separates matching/parsing of statements and declarations and fixes a few edge cases where the parser would incorrectly accept a declaration where only a statement is allowed - for example: if (foo) const a = 1; for (var bar;;) function b() {} while (baz) class c {}
2020-10-22 22:30:07 +00:00
{
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
auto type = m_state.current_token.type();
LibJS: Check that 'let' is followed by declaration before matching it Since 'let' is a valid variable name (in non-strict mode) let may not be the start of a declaration but just an identifier.
2021-07-29 00:03:16 +00:00
if (type == TokenType::Let && !m_state.strict_mode) {
return try_match_let_declaration();
}
LibJS: Distinguish between statement and declaration This separates matching/parsing of statements and declarations and fixes a few edge cases where the parser would incorrectly accept a declaration where only a statement is allowed - for example: if (foo) const a = 1; for (var bar;;) function b() {} while (baz) class c {}
2020-10-22 22:30:07 +00:00
return type == TokenType::Var
|| type == TokenType::Let
|| type == TokenType::Const;
}
LibJS: Rework Identifier parsing to match the spec more closely
2021-07-11 11:04:55 +00:00
bool Parser::match_identifier() const
{
LibJS: Fix some small remaining issues with parsing unicode escapes Added a test to ensure the behavior stays the same. We now throw on a direct usage of an escaped keywords with a specific error to make it more clear to the user.
2021-08-21 09:27:20 +00:00
if (m_state.current_token.type() == TokenType::EscapedKeyword) {
if (m_state.current_token.value() == "let"sv)
return !m_state.strict_mode;
if (m_state.current_token.value() == "yield"sv)
return !m_state.strict_mode && !m_state.in_generator_function_context;
LibJS: Handle escaped keywords in more cases and handle 'await' labels
2021-09-18 21:02:50 +00:00
if (m_state.current_token.value() == "await"sv)
LibJS: Add support for await expressions
2021-11-09 20:52:21 +00:00
return m_program_type != Program::Type::Module && !m_state.in_async_function_context && !m_state.in_class_static_init_block;
LibJS: Fix some small remaining issues with parsing unicode escapes Added a test to ensure the behavior stays the same. We now throw on a direct usage of an escaped keywords with a specific error to make it more clear to the user.
2021-08-21 09:27:20 +00:00
return true;
}
LibJS: Rework Identifier parsing to match the spec more closely
2021-07-11 11:04:55 +00:00
return m_state.current_token.type() == TokenType::Identifier
LibJS: Add support for async functions This commit adds support for the most bare bones version of async functions, support for async generator functions, async arrow functions and await expressions are TODO.
2021-11-09 18:39:22 +00:00
|| m_state.current_token.type() == TokenType::Async
LibJS: Treat yield as an identifier in more non-generator contexts And disallow some cases where we are in a generator context.
2021-07-28 23:45:55 +00:00
|| (m_state.current_token.type() == TokenType::Let && !m_state.strict_mode)
LibJS: Add support for await expressions
2021-11-09 20:52:21 +00:00
|| (m_state.current_token.type() == TokenType::Await && m_program_type != Program::Type::Module && !m_state.in_async_function_context && !m_state.in_class_static_init_block)
LibJS: Treat yield as an identifier in more non-generator contexts And disallow some cases where we are in a generator context.
2021-07-28 23:45:55 +00:00
|| (m_state.current_token.type() == TokenType::Yield && !m_state.in_generator_function_context && !m_state.strict_mode); // See note in Parser::parse_identifier().
LibJS: Rework Identifier parsing to match the spec more closely
2021-07-11 11:04:55 +00:00
}
LibJS: Allow reserved words as keys in object expressions.
2020-04-18 18:31:27 +00:00
bool Parser::match_identifier_name() const
{
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
return m_state.current_token.is_identifier_name();
LibJS: Allow reserved words as keys in object expressions.
2020-04-18 18:31:27 +00:00
}
LibJS: Initial class implementation; allow super expressions in object literal methods; add EnvrionmentRecord fields and methods to LexicalEnvironment Adding EnvrionmentRecord's fields and methods lets us throw an exception when |this| is not initialized, which occurs when the super constructor in a derived class has not yet been called, or when |this| has already been initialized (the super constructor was already called).
2020-06-08 18:31:21 +00:00
bool Parser::match_property_key() const
{
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
auto type = m_state.current_token.type();
LibJS: Initial class implementation; allow super expressions in object literal methods; add EnvrionmentRecord fields and methods to LexicalEnvironment Adding EnvrionmentRecord's fields and methods lets us throw an exception when |this| is not initialized, which occurs when the super constructor in a derived class has not yet been called, or when |this| has already been initialized (the super constructor was already called).
2020-06-08 18:31:21 +00:00
return match_identifier_name()
|| type == TokenType::BracketOpen
|| type == TokenType::StringLiteral
|| type == TokenType::NumericLiteral
|| type == TokenType::BigIntLiteral;
}
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
bool Parser::done() const
{
return match(TokenType::Eof);
}
Token Parser::consume()
{
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
auto old_token = m_state.current_token;
m_state.current_token = m_state.lexer.next();
LibJS: Add an optimization to avoid needless arguments object creation This gives FunctionNode a "might need arguments object" boolean flag and sets it based on the simplest possible heuristic for this: if we encounter an identifier called "arguments" or "eval" up to the next (nested) function declaration or expression, we won't need an arguments object. Otherwise, we *might* need one - the final decision is made in the FunctionDeclarationInstantiation AO. Now, this is obviously not perfect. Even if you avoid eval, something like `foo.arguments` will still trigger a false positive - but it's a start and already massively cuts down on needlessly allocated objects, especially in real-world code that is often minified, and so a full "arguments" identifier will be an actual arguments object more often than not. To illustrate the actual impact of this change, here's the number of allocated arguments objects during a full test-js run: Before: - Unmapped arguments objects: 78765 - Mapped arguments objects: 2455 After: - Unmapped arguments objects: 18 - Mapped arguments objects: 37 This results in a ~5% speedup of test-js on my Linux host machine, and about 3.5% on i686 Serenity in QEMU (warm runs, average of 5). The following microbenchmark (calling an empty function 1M times) runs 25% faster on Linux and 45% on Serenity: function foo() {} for (var i = 0; i < 1_000_000; ++i) foo(); test262 reports no changes in either direction, apart from a speedup :^)
2021-10-05 07:44:58 +00:00
// NOTE: This is the bare minimum needed to decide whether we might need an arguments object
// in a function expression or declaration. ("might" because the AST implements some further
// conditions from the spec that rule out the need for allocating one)
if (old_token.type() == TokenType::Identifier && old_token.value().is_one_of("arguments"sv, "eval"sv))
m_state.function_might_need_arguments_object = true;
LibJS: Add operator precedence parsing Obey precedence and associativity rules when parsing expressions with chained operators.
2020-03-12 22:02:41 +00:00
return old_token;
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
}
LibJS: Implement automatic semicolon insertion
2020-04-17 13:05:58 +00:00
void Parser::consume_or_insert_semicolon()
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
{
LibJS: Implement automatic semicolon insertion
2020-04-17 13:05:58 +00:00
// Semicolon was found and will be consumed
if (match(TokenType::Semicolon)) {
consume();
return;
}
// Insert semicolon if...
Everywhere: Fix typos Mostly in comments, but sprintf() now prints "August" instead of "Auguest" so that's something.
2020-10-02 13:59:28 +00:00
// ...token is preceded by one or more newlines
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
if (m_state.current_token.trivia_contains_line_terminator())
LibJS: Implement automatic semicolon insertion
2020-04-17 13:05:58 +00:00
return;
LibJS: Fix semicolon insertion
2020-04-17 13:27:51 +00:00
// ...token is a closing curly brace
if (match(TokenType::CurlyClose))
LibJS: Implement automatic semicolon insertion
2020-04-17 13:05:58 +00:00
return;
// ...token is eof
if (match(TokenType::Eof))
return;
// No rule for semicolon insertion applies -> syntax error
expected("Semicolon");
}
LibJS: Rework Identifier parsing to match the spec more closely
2021-07-11 11:04:55 +00:00
Token Parser::consume_identifier()
{
if (match(TokenType::Identifier))
return consume(TokenType::Identifier);
LibJS: Fix some small remaining issues with parsing unicode escapes Added a test to ensure the behavior stays the same. We now throw on a direct usage of an escaped keywords with a specific error to make it more clear to the user.
2021-08-21 09:27:20 +00:00
if (match(TokenType::EscapedKeyword))
return consume(TokenType::EscapedKeyword);
LibJS: Rework Identifier parsing to match the spec more closely
2021-07-11 11:04:55 +00:00
// Note that 'let' is not a reserved keyword, but our lexer considers it such
// As it's pretty nice to have that (for syntax highlighting and such), we'll
// special-case it here instead.
if (match(TokenType::Let)) {
if (m_state.strict_mode)
syntax_error("'let' is not allowed as an identifier in strict mode");
return consume();
}
LibJS: Treat yield as an identifier in more non-generator contexts And disallow some cases where we are in a generator context.
2021-07-28 23:45:55 +00:00
if (match(TokenType::Yield)) {
if (m_state.strict_mode || m_state.in_generator_function_context)
syntax_error("Identifier must not be a reserved word in strict mode ('yield')");
return consume();
}
LibJS: Add support for async functions This commit adds support for the most bare bones version of async functions, support for async generator functions, async arrow functions and await expressions are TODO.
2021-11-09 18:39:22 +00:00
if (match(TokenType::Await)) {
LibJS: Add support for await expressions
2021-11-09 20:52:21 +00:00
if (m_program_type == Program::Type::Module || m_state.in_async_function_context || m_state.in_class_static_init_block)
LibJS: Add support for async functions This commit adds support for the most bare bones version of async functions, support for async generator functions, async arrow functions and await expressions are TODO.
2021-11-09 18:39:22 +00:00
syntax_error("Identifier must not be a reserved word in modules ('await')");
return consume();
}
if (match(TokenType::Async))
return consume();
LibJS: Rework Identifier parsing to match the spec more closely
2021-07-11 11:04:55 +00:00
expected("Identifier");
return consume();
}
// https://tc39.es/ecma262/#prod-IdentifierReference
Token Parser::consume_identifier_reference()
{
if (match(TokenType::Identifier))
return consume(TokenType::Identifier);
LibJS: Fix some small remaining issues with parsing unicode escapes Added a test to ensure the behavior stays the same. We now throw on a direct usage of an escaped keywords with a specific error to make it more clear to the user.
2021-08-21 09:27:20 +00:00
if (match(TokenType::EscapedKeyword)) {
auto name = m_state.current_token.value();
LibJS: Handle escaped keywords in more cases and handle 'await' labels
2021-09-18 21:02:50 +00:00
if (m_state.strict_mode && (name == "let"sv || name == "yield"sv))
LibJS: Fix some small remaining issues with parsing unicode escapes Added a test to ensure the behavior stays the same. We now throw on a direct usage of an escaped keywords with a specific error to make it more clear to the user.
2021-08-21 09:27:20 +00:00
syntax_error(String::formatted("'{}' is not allowed as an identifier in strict mode", name));
LibJS: Handle escaped keywords in more cases and handle 'await' labels
2021-09-18 21:02:50 +00:00
if (m_program_type == Program::Type::Module && name == "await"sv)
syntax_error("'await' is not allowed as an identifier in module");
LibJS: Fix some small remaining issues with parsing unicode escapes Added a test to ensure the behavior stays the same. We now throw on a direct usage of an escaped keywords with a specific error to make it more clear to the user.
2021-08-21 09:27:20 +00:00
return consume();
}
LibJS: Rework Identifier parsing to match the spec more closely
2021-07-11 11:04:55 +00:00
// See note in Parser::parse_identifier().
if (match(TokenType::Let)) {
if (m_state.strict_mode)
syntax_error("'let' is not allowed as an identifier in strict mode");
return consume();
}
if (match(TokenType::Yield)) {
if (m_state.strict_mode)
syntax_error("Identifier reference may not be 'yield' in strict mode");
return consume();
}
if (match(TokenType::Await)) {
LibJS: Handle escaped keywords in more cases and handle 'await' labels
2021-09-18 21:02:50 +00:00
if (m_program_type == Program::Type::Module)
syntax_error("'await' is not allowed as an identifier in module");
LibJS: Rework Identifier parsing to match the spec more closely
2021-07-11 11:04:55 +00:00
return consume();
}
LibJS: Add support for async functions This commit adds support for the most bare bones version of async functions, support for async generator functions, async arrow functions and await expressions are TODO.
2021-11-09 18:39:22 +00:00
if (match(TokenType::Async))
return consume();
LibJS: Rework Identifier parsing to match the spec more closely
2021-07-11 11:04:55 +00:00
expected(Token::name(TokenType::Identifier));
return consume();
}
LibJS: Implement automatic semicolon insertion
2020-04-17 13:05:58 +00:00
Token Parser::consume(TokenType expected_type)
{
LibJS: Implement basic for..in and for..of loops
2020-04-21 18:21:26 +00:00
if (!match(expected_type)) {
LibJS: Implement automatic semicolon insertion
2020-04-17 13:05:58 +00:00
expected(Token::name(expected_type));
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
}
LibJS: Throw a syntax error when an identifier is a reserved word From the specification: It is a Syntax Error if StringValue of IdentifierName is the same String value as the StringValue of any ReservedWord except for yield or await. It is a Syntax Error if this phrase is contained in strict mode code and the StringValue of IdentifierName is: "implements", "interface", "let", "package", "private", "protected", "public", "static", or "yield".
2021-06-16 22:57:01 +00:00
auto token = consume();
if (expected_type == TokenType::Identifier) {
LibJS: Be more strict about reserved and special identifiers
2021-07-11 23:27:35 +00:00
if (m_state.strict_mode && is_strict_reserved_word(token.value()))
syntax_error(String::formatted("Identifier must not be a reserved word in strict mode ('{}')", token.value()));
LibJS: Throw a syntax error when an identifier is a reserved word From the specification: It is a Syntax Error if StringValue of IdentifierName is the same String value as the StringValue of any ReservedWord except for yield or await. It is a Syntax Error if this phrase is contained in strict mode code and the StringValue of IdentifierName is: "implements", "interface", "let", "package", "private", "protected", "public", "static", or "yield".
2021-06-16 22:57:01 +00:00
}
return token;
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
}
LibJS: Unprefixed octal numbers are a syntax error in strict mode
2020-10-19 17:01:28 +00:00
Token Parser::consume_and_validate_numeric_literal()
{
Everywhere: Pass AK::StringView by value
2021-11-10 23:55:02 +00:00
auto is_unprefixed_octal_number = [](StringView value) {
LibJS: Use the new is_ascii_foo() helpers from AK These constexpr helpers generate nicer code than the LibC ctype.h variants, so let's make use of them. :^)
2021-06-13 08:47:09 +00:00
return value.length() > 1 && value[0] == '0' && is_ascii_digit(value[1]);
LibJS: Unprefixed octal numbers are a syntax error in strict mode
2020-10-19 17:01:28 +00:00
};
LibJS: Replace 'size_t line, size_t column' with 'Optional<Position>' This is a bit nicer for two reasons: - The absence of line number/column information isn't based on 'values are zero' anymore but on Optional's value - When reporting syntax errors with position information other than the current token's position we had to store line and column ourselves, like this: auto foo_start_line = m_parser_state.m_current_token.line_number(); auto foo_start_column = m_parser_state.m_current_token.line_column(); ... syntax_error("...", foo_start_line, foo_start_column); Which now becomes: auto foo_start= position(); ... syntax_error("...", foo_start); This makes it easier to report correct positions for syntax errors that only emerge a few tokens later :^)
2020-11-02 21:03:19 +00:00
auto literal_start = position();
LibJS: Unprefixed octal numbers are a syntax error in strict mode
2020-10-19 17:01:28 +00:00
auto token = consume(TokenType::NumericLiteral);
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
if (m_state.strict_mode && is_unprefixed_octal_number(token.value()))
LibJS: Replace 'size_t line, size_t column' with 'Optional<Position>' This is a bit nicer for two reasons: - The absence of line number/column information isn't based on 'values are zero' anymore but on Optional's value - When reporting syntax errors with position information other than the current token's position we had to store line and column ourselves, like this: auto foo_start_line = m_parser_state.m_current_token.line_number(); auto foo_start_column = m_parser_state.m_current_token.line_column(); ... syntax_error("...", foo_start_line, foo_start_column); Which now becomes: auto foo_start= position(); ... syntax_error("...", foo_start); This makes it easier to report correct positions for syntax errors that only emerge a few tokens later :^)
2020-11-02 21:03:19 +00:00
syntax_error("Unprefixed octal number not allowed in strict mode", literal_start);
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
if (match_identifier_name() && m_state.current_token.trivia().is_empty())
LibJS: Disallow NumericLiteral immediately followed by Identifier From the spec: https://tc39.es/ecma262/#sec-literals-numeric-literals The SourceCharacter immediately following a NumericLiteral must not be an IdentifierStart or DecimalDigit. For example: 3in is an error and not the two input elements 3 and in.
2020-10-22 21:18:31 +00:00
syntax_error("Numeric literal must not be immediately followed by identifier");
LibJS: Unprefixed octal numbers are a syntax error in strict mode
2020-10-19 17:01:28 +00:00
return token;
}
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
void Parser::expected(const char* what)
LibJS: Add Parser::syntax_error() helper Instead of having fprintf()s all over the place we can now use syntax_error("message") or syntax_error("message", line, column). This takes care of a consistent format, appending a newline and getting the line number and column of the current token if the last two params are omitted.
2020-04-29 20:37:51 +00:00
{
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
auto message = m_state.current_token.message();
LibJS: Use message from invalid token in syntax error
2020-10-26 20:19:36 +00:00
if (message.is_empty())
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
message = String::formatted("Unexpected token {}. Expected {}", m_state.current_token.name(), what);
LibJS: Use message from invalid token in syntax error
2020-10-26 20:19:36 +00:00
syntax_error(message);
LibJS: Add Parser::syntax_error() helper Instead of having fprintf()s all over the place we can now use syntax_error("message") or syntax_error("message", line, column). This takes care of a consistent format, appending a newline and getting the line number and column of the current token if the last two params are omitted.
2020-04-29 20:37:51 +00:00
}
LibJS: Track source positions all the way down to exceptions This makes exceptions have a trace of source positions too, which could probably be helpful in making fancier error tracebacks.
2020-12-28 17:15:22 +00:00
Position Parser::position() const
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
{
LibJS: Replace 'size_t line, size_t column' with 'Optional<Position>' This is a bit nicer for two reasons: - The absence of line number/column information isn't based on 'values are zero' anymore but on Optional's value - When reporting syntax errors with position information other than the current token's position we had to store line and column ourselves, like this: auto foo_start_line = m_parser_state.m_current_token.line_number(); auto foo_start_column = m_parser_state.m_current_token.line_column(); ... syntax_error("...", foo_start_line, foo_start_column); Which now becomes: auto foo_start= position(); ... syntax_error("...", foo_start); This makes it easier to report correct positions for syntax errors that only emerge a few tokens later :^)
2020-11-02 21:03:19 +00:00
return {
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
m_state.current_token.line_number(),
LibJS: Implement parsing and evaluation for AssignmentPatterns e.g. `[...foo] = bar` can now be evaluated :^)
2021-07-10 20:46:17 +00:00
m_state.current_token.line_column(),
m_state.current_token.offset(),
LibJS: Replace 'size_t line, size_t column' with 'Optional<Position>' This is a bit nicer for two reasons: - The absence of line number/column information isn't based on 'values are zero' anymore but on Optional's value - When reporting syntax errors with position information other than the current token's position we had to store line and column ourselves, like this: auto foo_start_line = m_parser_state.m_current_token.line_number(); auto foo_start_column = m_parser_state.m_current_token.line_column(); ... syntax_error("...", foo_start_line, foo_start_column); Which now becomes: auto foo_start= position(); ... syntax_error("...", foo_start); This makes it easier to report correct positions for syntax errors that only emerge a few tokens later :^)
2020-11-02 21:03:19 +00:00
};
}
LibJS: Memoize failed calls of try_parse_arrow_function_expression()
2021-04-11 20:41:51 +00:00
bool Parser::try_parse_arrow_function_expression_failed_at_position(const Position& position) const
{
auto it = m_token_memoizations.find(position);
if (it == m_token_memoizations.end())
return false;
return (*it).value.try_parse_arrow_function_expression_failed;
}
void Parser::set_try_parse_arrow_function_expression_failed_at_position(const Position& position, bool failed)
{
m_token_memoizations.set(position, { failed });
}
LibJS: Replace 'size_t line, size_t column' with 'Optional<Position>' This is a bit nicer for two reasons: - The absence of line number/column information isn't based on 'values are zero' anymore but on Optional's value - When reporting syntax errors with position information other than the current token's position we had to store line and column ourselves, like this: auto foo_start_line = m_parser_state.m_current_token.line_number(); auto foo_start_column = m_parser_state.m_current_token.line_column(); ... syntax_error("...", foo_start_line, foo_start_column); Which now becomes: auto foo_start= position(); ... syntax_error("...", foo_start); This makes it easier to report correct positions for syntax errors that only emerge a few tokens later :^)
2020-11-02 21:03:19 +00:00
void Parser::syntax_error(const String& message, Optional<Position> position)
{
if (!position.has_value())
position = this->position();
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
m_state.errors.append({ message, position });
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
}
LibJS: Add Parser save_state() and load_state() functions These functions allow us to try to parse ambiguous expressions (such as arrow function arguments in parentheses), and rewind the state of the Parser if an expression candidate failed to parse.
2020-03-30 13:24:43 +00:00
void Parser::save_state()
{
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
m_saved_state.append(m_state);
LibJS: Add Parser save_state() and load_state() functions These functions allow us to try to parse ambiguous expressions (such as arrow function arguments in parentheses), and rewind the state of the Parser if an expression candidate failed to parse.
2020-03-30 13:24:43 +00:00
}
void Parser::load_state()
{
Everywhere: Rename ASSERT => VERIFY (...and ASSERT_NOT_REACHED => VERIFY_NOT_REACHED) Since all of these checks are done in release builds as well, let's rename them to VERIFY to prevent confusion, as everyone is used to assertions being compiled out in release. We can introduce a new ASSERT macro that is specifically for debug checks, but I'm doing this wholesale conversion first since we've accumulated thousands of these already, and it's not immediately obvious which ones are suitable for ASSERT.
2021-02-23 19:42:32 +00:00
VERIFY(!m_saved_state.is_empty());
LibJS: Rename Parser::m_parser_state => m_state Also remove the "m_" prefix from all the public data members.
2021-06-19 12:43:09 +00:00
m_state = m_saved_state.take_last();
LibJS: Add Parser save_state() and load_state() functions These functions allow us to try to parse ambiguous expressions (such as arrow function arguments in parentheses), and rewind the state of the Parser if an expression candidate failed to parse.
2020-03-30 13:24:43 +00:00
}
LibJS: Add Parser::syntax_error() helper Instead of having fprintf()s all over the place we can now use syntax_error("message") or syntax_error("message", line, column). This takes care of a consistent format, appending a newline and getting the line number and column of the current token if the last two params are omitted.
2020-04-29 20:37:51 +00:00
LibJS: Don't save rule start positions along with the parser state This fixes #4617. Also fixes the small problem where some save states would be leaked.
2020-12-29 13:17:39 +00:00
void Parser::discard_saved_state()
{
m_saved_state.take_last();
}
LibJS: Be more strict about reserved and special identifiers
2021-07-11 23:27:35 +00:00
void Parser::check_identifier_name_for_assignment_validity(StringView name, bool force_strict)
LibJS: Rework Identifier parsing to match the spec more closely
2021-07-11 11:04:55 +00:00
{
LibJS: Be more strict about reserved and special identifiers
2021-07-11 23:27:35 +00:00
// FIXME: this is now called from multiple places maybe the error message should be dynamic?
Userland: Make use of container version of any_of Problem: - New `any_of` implementation takes the entire container so the user does not need to pass explicit begin/end iterators. This is unused except is in tests. Solution: - Make use of the new and more user-friendly version where possible.
2021-07-25 20:21:27 +00:00
if (any_of(s_reserved_words, [&](auto& value) { return name == value; })) {
LibJS: Rework Identifier parsing to match the spec more closely
2021-07-11 11:04:55 +00:00
syntax_error("Binding pattern target may not be a reserved word");
LibJS: Be more strict about reserved and special identifiers
2021-07-11 23:27:35 +00:00
} else if (m_state.strict_mode || force_strict) {
LibJS: Rework Identifier parsing to match the spec more closely
2021-07-11 11:04:55 +00:00
if (name.is_one_of("arguments"sv, "eval"sv))
syntax_error("Binding pattern target may not be called 'arguments' or 'eval' in strict mode");
LibJS: Be more strict about reserved and special identifiers
2021-07-11 23:27:35 +00:00
else if (is_strict_reserved_word(name))
LibJS: Treat yield as an identifier in more non-generator contexts And disallow some cases where we are in a generator context.
2021-07-28 23:45:55 +00:00
syntax_error(String::formatted("Binding pattern target may not be called '{}' in strict mode", name));
LibJS: Rework Identifier parsing to match the spec more closely
2021-07-11 11:04:55 +00:00
}
}
LibJS: Parse and partially execute import and export statements We produce the import and export entries as per the spec. However we do not yet verify that named things that are exported are declared somewhere.
2021-08-14 15:42:30 +00:00
NonnullRefPtr<ImportStatement> Parser::parse_import_statement(Program& program)
{
auto rule_start = push_start();
if (program.type() != Program::Type::Module)
syntax_error("Cannot use import statement outside a module");
consume(TokenType::Import);
if (match(TokenType::StringLiteral)) {
auto module_name = consume(TokenType::StringLiteral).value();
return create_ast_node<ImportStatement>({ m_state.current_token.filename(), rule_start.position(), position() }, module_name);
}
auto match_imported_binding = [&] {
return match_identifier() || match(TokenType::Yield) || match(TokenType::Await);
};
auto match_as = [&] {
LibJS: Allow Unicode escape sequences in identifiers For example, "property.br\u{64}wn" should resolve to "property.brown". To support this behavior, this commit changes the Token class to hold both the evaluated identifier name and a view into the original source for the unevaluated name. There are some contexts in which identifiers are not allowed to contain Unicode escape sequences; for example, export statements of the form "export {} from foo.js" forbid escapes in the identifier "from". The test file is added to .prettierignore because prettier will replace all escaped Unicode sequences with their unescaped value.
2021-08-18 20:34:25 +00:00
return match(TokenType::Identifier) && m_state.current_token.original_value() == "as"sv;
LibJS: Parse and partially execute import and export statements We produce the import and export entries as per the spec. However we do not yet verify that named things that are exported are declared somewhere.
2021-08-14 15:42:30 +00:00
};
bool continue_parsing = true;
struct ImportWithLocation {
ImportStatement::ImportEntry entry;
Position position;
};
Vector<ImportWithLocation> entries_with_location;
if (match_imported_binding()) {
auto id_position = position();
auto bound_name = consume().value();
entries_with_location.append({ { "default", bound_name }, id_position });
if (match(TokenType::Comma)) {
consume(TokenType::Comma);
} else {
continue_parsing = false;
}
}
if (!continue_parsing) {
// skip the rest
} else if (match(TokenType::Asterisk)) {
consume(TokenType::Asterisk);
if (!match_as())
syntax_error(String::formatted("Unexpected token: {}", m_state.current_token.name()));
consume(TokenType::Identifier);
if (match_imported_binding()) {
auto namespace_position = position();
auto namespace_name = consume().value();
entries_with_location.append({ { "*", namespace_name }, namespace_position });
} else {
syntax_error(String::formatted("Unexpected token: {}", m_state.current_token.name()));
}
} else if (match(TokenType::CurlyOpen)) {
consume(TokenType::CurlyOpen);
while (!done() && !match(TokenType::CurlyClose)) {
if (match_identifier_name()) {
auto require_as = !match_imported_binding();
auto name_position = position();
auto name = consume().value();
if (match_as()) {
consume(TokenType::Identifier);
auto alias_position = position();
auto alias = consume_identifier().value();
check_identifier_name_for_assignment_validity(alias);
entries_with_location.append({ { name, alias }, alias_position });
} else if (require_as) {
syntax_error(String::formatted("Unexpected reserved word '{}'", name));
} else {
check_identifier_name_for_assignment_validity(name);
entries_with_location.append({ { name, name }, name_position });
}
} else {
expected("identifier");
break;
}
if (!match(TokenType::Comma))
break;
consume(TokenType::Comma);
}
consume(TokenType::CurlyClose);
} else {
expected("import clauses");
}
LibJS: Handle escaped keywords in more cases and handle 'await' labels
2021-09-18 21:02:50 +00:00
auto from_statement = consume(TokenType::Identifier).original_value();
LibJS: Parse and partially execute import and export statements We produce the import and export entries as per the spec. However we do not yet verify that named things that are exported are declared somewhere.
2021-08-14 15:42:30 +00:00
if (from_statement != "from"sv)
syntax_error(String::formatted("Expected 'from' got {}", from_statement));
auto module_name = consume(TokenType::StringLiteral).value();
Vector<ImportStatement::ImportEntry> entries;
entries.ensure_capacity(entries_with_location.size());
for (auto& entry : entries_with_location) {
for (auto& import_statement : program.imports()) {
if (import_statement.has_bound_name(entry.entry.local_name))
syntax_error(String::formatted("Identifier '{}' already declared", entry.entry.local_name), entry.position);
}
for (auto& new_entry : entries) {
if (new_entry.local_name == entry.entry.local_name)
syntax_error(String::formatted("Identifier '{}' already declared", entry.entry.local_name), entry.position);
}
entries.append(move(entry.entry));
}
return create_ast_node<ImportStatement>({ m_state.current_token.filename(), rule_start.position(), position() }, module_name, move(entries));
}
NonnullRefPtr<ExportStatement> Parser::parse_export_statement(Program& program)
{
auto rule_start = push_start();
if (program.type() != Program::Type::Module)
syntax_error("Cannot use export statement outside a module");
auto match_as = [&] {
LibJS: Allow Unicode escape sequences in identifiers For example, "property.br\u{64}wn" should resolve to "property.brown". To support this behavior, this commit changes the Token class to hold both the evaluated identifier name and a view into the original source for the unevaluated name. There are some contexts in which identifiers are not allowed to contain Unicode escape sequences; for example, export statements of the form "export {} from foo.js" forbid escapes in the identifier "from". The test file is added to .prettierignore because prettier will replace all escaped Unicode sequences with their unescaped value.
2021-08-18 20:34:25 +00:00
return match(TokenType::Identifier) && m_state.current_token.original_value() == "as"sv;
LibJS: Parse and partially execute import and export statements We produce the import and export entries as per the spec. However we do not yet verify that named things that are exported are declared somewhere.
2021-08-14 15:42:30 +00:00
};
auto match_from = [&] {
LibJS: Allow Unicode escape sequences in identifiers For example, "property.br\u{64}wn" should resolve to "property.brown". To support this behavior, this commit changes the Token class to hold both the evaluated identifier name and a view into the original source for the unevaluated name. There are some contexts in which identifiers are not allowed to contain Unicode escape sequences; for example, export statements of the form "export {} from foo.js" forbid escapes in the identifier "from". The test file is added to .prettierignore because prettier will replace all escaped Unicode sequences with their unescaped value.
2021-08-18 20:34:25 +00:00
return match(TokenType::Identifier) && m_state.current_token.original_value() == "from"sv;
};
auto match_default = [&] {
return match(TokenType::Default) && m_state.current_token.original_value() == "default"sv;
LibJS: Parse and partially execute import and export statements We produce the import and export entries as per the spec. However we do not yet verify that named things that are exported are declared somewhere.
2021-08-14 15:42:30 +00:00
};
consume(TokenType::Export);
struct EntryAndLocation {
ExportStatement::ExportEntry entry;
Position position;
void to_module_request(String from_module)
{
entry.kind = ExportStatement::ExportEntry::Kind::ModuleRequest;
entry.module_request = from_module;
}
};
Vector<EntryAndLocation> entries_with_location;
RefPtr<ASTNode> expression = {};
LibJS: Allow Unicode escape sequences in identifiers For example, "property.br\u{64}wn" should resolve to "property.brown". To support this behavior, this commit changes the Token class to hold both the evaluated identifier name and a view into the original source for the unevaluated name. There are some contexts in which identifiers are not allowed to contain Unicode escape sequences; for example, export statements of the form "export {} from foo.js" forbid escapes in the identifier "from". The test file is added to .prettierignore because prettier will replace all escaped Unicode sequences with their unescaped value.
2021-08-18 20:34:25 +00:00
if (match_default()) {
LibJS: Parse and partially execute import and export statements We produce the import and export entries as per the spec. However we do not yet verify that named things that are exported are declared somewhere.
2021-08-14 15:42:30 +00:00
auto default_position = position();
consume(TokenType::Default);
String local_name;
if (match(TokenType::Class)) {
auto class_expression = parse_class_expression(false);
local_name = class_expression->name();
expression = move(class_expression);
LibJS: Add support for async functions This commit adds support for the most bare bones version of async functions, support for async generator functions, async arrow functions and await expressions are TODO.
2021-11-09 18:39:22 +00:00
} else if (match(TokenType::Function) || (match(TokenType::Async) && next_token().type() == TokenType::Function)) {
LibJS: Parse and partially execute import and export statements We produce the import and export entries as per the spec. However we do not yet verify that named things that are exported are declared somewhere.
2021-08-14 15:42:30 +00:00
auto func_expr = parse_function_node<FunctionExpression>();
local_name = func_expr->name();
expression = move(func_expr);
} else if (match_expression()) {
expression = parse_expression(2);
consume_or_insert_semicolon();
local_name = "*default*";
} else {
expected("Declaration or assignment expression");
}
entries_with_location.append({ { "default", local_name }, default_position });
} else {
enum FromSpecifier {
NotAllowed,
Optional,
Required
} check_for_from { NotAllowed };
if (match(TokenType::Asterisk)) {
auto asterisk_position = position();
consume(TokenType::Asterisk);
if (match_as()) {
consume(TokenType::Identifier);
if (match_identifier_name()) {
auto namespace_position = position();
auto exported_name = consume().value();
entries_with_location.append({ { exported_name, "*" }, namespace_position });
} else {
expected("identifier");
}
} else {
entries_with_location.append({ { {}, "*" }, asterisk_position });
}
check_for_from = Required;
} else if (match_declaration()) {
auto decl_position = position();
auto declaration = parse_declaration();
if (is<FunctionDeclaration>(*declaration)) {
auto& func = static_cast<FunctionDeclaration&>(*declaration);
entries_with_location.append({ { func.name(), func.name() }, func.source_range().start });
} else if (is<ClassDeclaration>(*declaration)) {
auto& class_declaration = static_cast<ClassDeclaration&>(*declaration);
entries_with_location.append({ { class_declaration.class_name(), class_declaration.class_name() }, class_declaration.source_range().start });
} else {
VERIFY(is<VariableDeclaration>(*declaration));
auto& variables = static_cast<VariableDeclaration&>(*declaration);
for (auto& decl : variables.declarations()) {
decl.target().visit(
[&](NonnullRefPtr<Identifier> const& identifier) {
entries_with_location.append({ { identifier->string(), identifier->string() }, identifier->source_range().start });
},
[&](NonnullRefPtr<BindingPattern> const& binding) {
binding->for_each_bound_name([&](auto& name) {
entries_with_location.append({ { name, name }, decl_position });
});
});
}
}
expression = declaration;
} else if (match(TokenType::Var)) {
auto variable_position = position();
auto variable_declaration = parse_variable_declaration();
for (auto& decl : variable_declaration->declarations()) {
decl.target().visit(
[&](NonnullRefPtr<Identifier> const& identifier) {
entries_with_location.append({ { identifier->string(), identifier->string() }, identifier->source_range().start });
},
[&](NonnullRefPtr<BindingPattern> const& binding) {
binding->for_each_bound_name([&](auto& name) {
entries_with_location.append({ { name, name }, variable_position });
});
});
}
expression = variable_declaration;
} else if (match(TokenType::CurlyOpen)) {
consume(TokenType::CurlyOpen);
while (!done() && !match(TokenType::CurlyClose)) {
if (match_identifier_name()) {
auto identifier_position = position();
auto identifier = consume().value();
if (match_as()) {
consume(TokenType::Identifier);
if (match_identifier_name()) {
auto export_name = consume().value();
entries_with_location.append({ { export_name, identifier }, identifier_position });
} else {
expected("identifier name");
}
} else {
entries_with_location.append({ { identifier, identifier }, identifier_position });
}
} else {
expected("identifier");
break;
}
if (!match(TokenType::Comma))
break;
consume(TokenType::Comma);
}
consume(TokenType::CurlyClose);
check_for_from = Optional;
} else {
syntax_error("Unexpected token 'export'", rule_start.position());
}
if (check_for_from != NotAllowed && match_from()) {
consume(TokenType::Identifier);
if (match(TokenType::StringLiteral)) {
auto from_specifier = consume().value();
for (auto& entry : entries_with_location)
entry.to_module_request(from_specifier);
} else {
expected("ModuleSpecifier");
}
} else if (check_for_from == Required) {
expected("from");
}
if (check_for_from != NotAllowed)
consume_or_insert_semicolon();
}
Vector<ExportStatement::ExportEntry> entries;
entries.ensure_capacity(entries_with_location.size());
for (auto& entry : entries_with_location) {
for (auto& export_statement : program.exports()) {
if (export_statement.has_export(entry.entry.export_name))
syntax_error(String::formatted("Duplicate export with name: '{}'", entry.entry.export_name), entry.position);
}
for (auto& new_entry : entries) {
if (new_entry.export_name == entry.entry.export_name)
syntax_error(String::formatted("Duplicate export with name: '{}'", entry.entry.export_name), entry.position);
}
entries.append(move(entry.entry));
}
return create_ast_node<ExportStatement>({ m_state.current_token.filename(), rule_start.position(), position() }, move(expression), move(entries));
}
LibJS: Add Javascript lexer and parser This adds a basic Javascript lexer and parser. It can parse the currently existing demo programs. More work needs to be done to turn it into a complete parser than can parse arbitrary JS Code. The lexer outputs tokens with preceeding whitespace and comments in the trivia member. This should allow us to generate the exact source code by concatenating the generated tokens. The parser is written in a way that it always returns a complete syntax tree. Error conditions are represented as nodes in the tree. This simplifies the code and allows it to be used as an early stage parser, e.g for parsing JS documents in an IDE while editing the source code.:
2020-03-11 18:27:43 +00:00
}
Reference in a new issue Copy permalink