beenull/ladybird
1
0
Fork 0
mirror of https://github.com/LadybirdBrowser/ladybird.git synced 2024-11-22 15:40:19 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 6
ladybird/Libraries/LibRegex/RegexOptions.h

149 lines
7.1 KiB
C
Raw Normal View History

LibRegex: Add a regular expression library This commit is a mix of several commits, squashed into one because the commits before 'Move regex to own Library and fix all the broken stuff' were not fixable in any elegant way. The commits are listed below for "historical" purposes: - AK: Add options/flags and Errors for regular expressions Flags can be provided for any possible flavour by adding a new scoped enum. Handling of flags is done by templated Options class and the overloaded '|' and '&' operators. - AK: Add Lexer for regular expressions The lexer parses the input and extracts tokens needed to parse a regular expression. - AK: Add regex Parser and PosixExtendedParser This patchset adds a abstract parser class that can be derived to implement different parsers. A parser produces bytecode to be executed within the regex matcher. - AK: Add regex matcher This patchset adds an regex matcher based on the principles of the T-REX VM. The bytecode pruduced by the respective Parser is put into the matcher and the VM will recursively execute the bytecode according to the available OpCodes. Possible improvement: the recursion could be replaced by multi threading capabilities. To match a Regular expression, e.g. for the Posix standard regular expression matcher use the following API: ``` Pattern<PosixExtendedParser> pattern("^.*$"); auto result = pattern.match("Well, hello friends!\nHello World!"); // Match whole needle EXPECT(result.count == 1); EXPECT(result.matches.at(0).view.starts_with("Well")); EXPECT(result.matches.at(0).view.end() == "!"); result = pattern.match("Well, hello friends!\nHello World!", PosixFlags::Multiline); // Match line by line EXPECT(result.count == 2); EXPECT(result.matches.at(0).view == "Well, hello friends!"); EXPECT(result.matches.at(1).view == "Hello World!"); EXPECT(pattern.has_match("Well,....")); // Just check if match without a result, which saves some resources. ``` - AK: Rework regex to work with opcodes objects This patchsets reworks the matcher to work on a more structured base. For that an abstract OpCode class and derived classes for the specific OpCodes have been added. The respective opcode logic is contained in each respective execute() method. - AK: Add benchmark for regex - AK: Some optimization in regex for runtime and memory - LibRegex: Move regex to own Library and fix all the broken stuff Now regex works again and grep utility is also in place for testing. This commit also fixes the use of regex.h in C by making `regex_t` an opaque (-ish) type, which makes its behaviour consistent between C and C++ compilers. Previously, <regex.h> would've blown C compilers up, and even if it didn't, would've caused a leak in C code, and not in C++ code (due to the existence of `OwnPtr` inside the struct). To make this whole ordeal easier to deal with (for now), this pulls the definitions of `reg*()` into LibRegex. pros: - The circular dependency between LibC and LibRegex is broken - Eaiser to test (without accidentally pulling in the host's libc!) cons: - Using any of the regex.h functions will require the user to link -lregex - The symbols will be missing from libc, which will be a big surprise down the line (especially with shared libs). Co-Authored-By: Ali Mohammad Pur <ali.mpfard@gmail.com>
2020-04-26 12:45:10 +00:00
/*
* Copyright (c) 2020, Emanuel Sprung <emanuel.sprung@gmail.com>
*
Everything: Move to SPDX license identifiers in all files. SPDX License Identifiers are a more compact / standardized way of representing file license information. See: https://spdx.dev/resources/use/#identifiers This was done with the `ambr` search and replace tool. ambr --no-parent-ignore --key-from-file --rep-from-file key.txt rep.txt *
2021-04-22 08:24:48 +00:00
* SPDX-License-Identifier: BSD-2-Clause
LibRegex: Add a regular expression library This commit is a mix of several commits, squashed into one because the commits before 'Move regex to own Library and fix all the broken stuff' were not fixable in any elegant way. The commits are listed below for "historical" purposes: - AK: Add options/flags and Errors for regular expressions Flags can be provided for any possible flavour by adding a new scoped enum. Handling of flags is done by templated Options class and the overloaded '|' and '&' operators. - AK: Add Lexer for regular expressions The lexer parses the input and extracts tokens needed to parse a regular expression. - AK: Add regex Parser and PosixExtendedParser This patchset adds a abstract parser class that can be derived to implement different parsers. A parser produces bytecode to be executed within the regex matcher. - AK: Add regex matcher This patchset adds an regex matcher based on the principles of the T-REX VM. The bytecode pruduced by the respective Parser is put into the matcher and the VM will recursively execute the bytecode according to the available OpCodes. Possible improvement: the recursion could be replaced by multi threading capabilities. To match a Regular expression, e.g. for the Posix standard regular expression matcher use the following API: ``` Pattern<PosixExtendedParser> pattern("^.*$"); auto result = pattern.match("Well, hello friends!\nHello World!"); // Match whole needle EXPECT(result.count == 1); EXPECT(result.matches.at(0).view.starts_with("Well")); EXPECT(result.matches.at(0).view.end() == "!"); result = pattern.match("Well, hello friends!\nHello World!", PosixFlags::Multiline); // Match line by line EXPECT(result.count == 2); EXPECT(result.matches.at(0).view == "Well, hello friends!"); EXPECT(result.matches.at(1).view == "Hello World!"); EXPECT(pattern.has_match("Well,....")); // Just check if match without a result, which saves some resources. ``` - AK: Rework regex to work with opcodes objects This patchsets reworks the matcher to work on a more structured base. For that an abstract OpCode class and derived classes for the specific OpCodes have been added. The respective opcode logic is contained in each respective execute() method. - AK: Add benchmark for regex - AK: Some optimization in regex for runtime and memory - LibRegex: Move regex to own Library and fix all the broken stuff Now regex works again and grep utility is also in place for testing. This commit also fixes the use of regex.h in C by making `regex_t` an opaque (-ish) type, which makes its behaviour consistent between C and C++ compilers. Previously, <regex.h> would've blown C compilers up, and even if it didn't, would've caused a leak in C code, and not in C++ code (due to the existence of `OwnPtr` inside the struct). To make this whole ordeal easier to deal with (for now), this pulls the definitions of `reg*()` into LibRegex. pros: - The circular dependency between LibC and LibRegex is broken - Eaiser to test (without accidentally pulling in the host's libc!) cons: - Using any of the regex.h functions will require the user to link -lregex - The symbols will be missing from libc, which will be a big surprise down the line (especially with shared libs). Co-Authored-By: Ali Mohammad Pur <ali.mpfard@gmail.com>
2020-04-26 12:45:10 +00:00
*/
#pragma once
LibC+LibRegex: Move regex_defs.h from LibC to LibRegex This is needed to avoid including LibC headers in Lagom builds.
2023-04-29 20:50:54 +00:00
#include "RegexDefs.h"
LibRegex: Add a regular expression library This commit is a mix of several commits, squashed into one because the commits before 'Move regex to own Library and fix all the broken stuff' were not fixable in any elegant way. The commits are listed below for "historical" purposes: - AK: Add options/flags and Errors for regular expressions Flags can be provided for any possible flavour by adding a new scoped enum. Handling of flags is done by templated Options class and the overloaded '|' and '&' operators. - AK: Add Lexer for regular expressions The lexer parses the input and extracts tokens needed to parse a regular expression. - AK: Add regex Parser and PosixExtendedParser This patchset adds a abstract parser class that can be derived to implement different parsers. A parser produces bytecode to be executed within the regex matcher. - AK: Add regex matcher This patchset adds an regex matcher based on the principles of the T-REX VM. The bytecode pruduced by the respective Parser is put into the matcher and the VM will recursively execute the bytecode according to the available OpCodes. Possible improvement: the recursion could be replaced by multi threading capabilities. To match a Regular expression, e.g. for the Posix standard regular expression matcher use the following API: ``` Pattern<PosixExtendedParser> pattern("^.*$"); auto result = pattern.match("Well, hello friends!\nHello World!"); // Match whole needle EXPECT(result.count == 1); EXPECT(result.matches.at(0).view.starts_with("Well")); EXPECT(result.matches.at(0).view.end() == "!"); result = pattern.match("Well, hello friends!\nHello World!", PosixFlags::Multiline); // Match line by line EXPECT(result.count == 2); EXPECT(result.matches.at(0).view == "Well, hello friends!"); EXPECT(result.matches.at(1).view == "Hello World!"); EXPECT(pattern.has_match("Well,....")); // Just check if match without a result, which saves some resources. ``` - AK: Rework regex to work with opcodes objects This patchsets reworks the matcher to work on a more structured base. For that an abstract OpCode class and derived classes for the specific OpCodes have been added. The respective opcode logic is contained in each respective execute() method. - AK: Add benchmark for regex - AK: Some optimization in regex for runtime and memory - LibRegex: Move regex to own Library and fix all the broken stuff Now regex works again and grep utility is also in place for testing. This commit also fixes the use of regex.h in C by making `regex_t` an opaque (-ish) type, which makes its behaviour consistent between C and C++ compilers. Previously, <regex.h> would've blown C compilers up, and even if it didn't, would've caused a leak in C code, and not in C++ code (due to the existence of `OwnPtr` inside the struct). To make this whole ordeal easier to deal with (for now), this pulls the definitions of `reg*()` into LibRegex. pros: - The circular dependency between LibC and LibRegex is broken - Eaiser to test (without accidentally pulling in the host's libc!) cons: - Using any of the regex.h functions will require the user to link -lregex - The symbols will be missing from libc, which will be a big surprise down the line (especially with shared libs). Co-Authored-By: Ali Mohammad Pur <ali.mpfard@gmail.com>
2020-04-26 12:45:10 +00:00
#include <AK/Types.h>
#include <stdio.h>
namespace regex {
LibRegex: Implement ECMA262 multiline matching without splitting lines As ECMA262 regex allows `[^]` and literal newlines to match newlines in the input string, we shouldn't split the input string into lines, rather simply make boundaries and catchall patterns capable of checking for these conditions specifically.
2022-01-25 10:00:27 +00:00
using FlagsUnderlyingType = u32;
LibRegex: Add a regular expression library This commit is a mix of several commits, squashed into one because the commits before 'Move regex to own Library and fix all the broken stuff' were not fixable in any elegant way. The commits are listed below for "historical" purposes: - AK: Add options/flags and Errors for regular expressions Flags can be provided for any possible flavour by adding a new scoped enum. Handling of flags is done by templated Options class and the overloaded '|' and '&' operators. - AK: Add Lexer for regular expressions The lexer parses the input and extracts tokens needed to parse a regular expression. - AK: Add regex Parser and PosixExtendedParser This patchset adds a abstract parser class that can be derived to implement different parsers. A parser produces bytecode to be executed within the regex matcher. - AK: Add regex matcher This patchset adds an regex matcher based on the principles of the T-REX VM. The bytecode pruduced by the respective Parser is put into the matcher and the VM will recursively execute the bytecode according to the available OpCodes. Possible improvement: the recursion could be replaced by multi threading capabilities. To match a Regular expression, e.g. for the Posix standard regular expression matcher use the following API: ``` Pattern<PosixExtendedParser> pattern("^.*$"); auto result = pattern.match("Well, hello friends!\nHello World!"); // Match whole needle EXPECT(result.count == 1); EXPECT(result.matches.at(0).view.starts_with("Well")); EXPECT(result.matches.at(0).view.end() == "!"); result = pattern.match("Well, hello friends!\nHello World!", PosixFlags::Multiline); // Match line by line EXPECT(result.count == 2); EXPECT(result.matches.at(0).view == "Well, hello friends!"); EXPECT(result.matches.at(1).view == "Hello World!"); EXPECT(pattern.has_match("Well,....")); // Just check if match without a result, which saves some resources. ``` - AK: Rework regex to work with opcodes objects This patchsets reworks the matcher to work on a more structured base. For that an abstract OpCode class and derived classes for the specific OpCodes have been added. The respective opcode logic is contained in each respective execute() method. - AK: Add benchmark for regex - AK: Some optimization in regex for runtime and memory - LibRegex: Move regex to own Library and fix all the broken stuff Now regex works again and grep utility is also in place for testing. This commit also fixes the use of regex.h in C by making `regex_t` an opaque (-ish) type, which makes its behaviour consistent between C and C++ compilers. Previously, <regex.h> would've blown C compilers up, and even if it didn't, would've caused a leak in C code, and not in C++ code (due to the existence of `OwnPtr` inside the struct). To make this whole ordeal easier to deal with (for now), this pulls the definitions of `reg*()` into LibRegex. pros: - The circular dependency between LibC and LibRegex is broken - Eaiser to test (without accidentally pulling in the host's libc!) cons: - Using any of the regex.h functions will require the user to link -lregex - The symbols will be missing from libc, which will be a big surprise down the line (especially with shared libs). Co-Authored-By: Ali Mohammad Pur <ali.mpfard@gmail.com>
2020-04-26 12:45:10 +00:00
enum class AllFlags {
LibRegex: Make '.' reject matching LF / LS / PS as per the ECMA262 spec Previously we allowed it to match those, but the ECMA262 spec disallows these (except in DotAll).
2023-02-15 06:25:43 +00:00
Default = 0,
Global = __Regex_Global, // All matches (don't return after first match)
Insensitive = __Regex_Insensitive, // Case insensitive match (ignores case of [a-zA-Z])
Ungreedy = __Regex_Ungreedy, // The match becomes lazy by default. Now a ? following a quantifier makes it greedy
Unicode = __Regex_Unicode, // Enable all unicode features and interpret all unicode escape sequences as such
Extended = __Regex_Extended, // Ignore whitespaces. Spaces and text after a # in the pattern are ignored
Extra = __Regex_Extra, // Disallow meaningless escapes. A \ followed by a letter with no special meaning is faulted
MatchNotBeginOfLine = __Regex_MatchNotBeginOfLine, // Pattern is not forced to ^ -> search in whole string!
MatchNotEndOfLine = __Regex_MatchNotEndOfLine, // Don't Force the dollar sign, $, to always match end of the string, instead of end of the line. This option is ignored if the Multiline-flag is set
SkipSubExprResults = __Regex_SkipSubExprResults, // Do not return sub expressions in the result
StringCopyMatches = __Regex_StringCopyMatches, // Do explicitly copy results into new allocated string instead of StringView to original string.
SingleLine = __Regex_SingleLine, // Dot matches newline characters
Sticky = __Regex_Sticky, // Force the pattern to only match consecutive matches from where the previous match ended.
Multiline = __Regex_Multiline, // Handle newline characters. Match each line, one by one.
SkipTrimEmptyMatches = __Regex_SkipTrimEmptyMatches, // Do not remove empty capture group results.
SingleMatch = __Regex_SingleMatch, // Stop after acquiring a single match.
UnicodeSets = __Regex_UnicodeSets, // Only for ECMA262, Allow set operations in character classes.
Internal_Stateful = __Regex_Internal_Stateful, // Make global matches match one result at a time, and further match() calls on the same instance continue where the previous one left off.
Internal_BrowserExtended = __Regex_Internal_BrowserExtended, // Only for ECMA262, Enable the behaviors defined in section B.1.4. of the ECMA262 spec.
Internal_ConsiderNewline = __Regex_Internal_ConsiderNewline, // Only for ECMA262, Allow multiline matches to consider newlines as line boundaries.
Internal_ECMA262DotSemantics = __Regex_Internal_ECMA262DotSemantics, // Use ECMA262 dot semantics: disallow matching CR/LF/LS/PS instead of just CR.
LibRegex: Implement section B.1.4. of the ECMA262 spec This allows the parser to deal with crazy patterns like the one in #5517.
2021-02-26 19:19:34 +00:00
Last = Internal_BrowserExtended,
LibRegex: Add a regular expression library This commit is a mix of several commits, squashed into one because the commits before 'Move regex to own Library and fix all the broken stuff' were not fixable in any elegant way. The commits are listed below for "historical" purposes: - AK: Add options/flags and Errors for regular expressions Flags can be provided for any possible flavour by adding a new scoped enum. Handling of flags is done by templated Options class and the overloaded '|' and '&' operators. - AK: Add Lexer for regular expressions The lexer parses the input and extracts tokens needed to parse a regular expression. - AK: Add regex Parser and PosixExtendedParser This patchset adds a abstract parser class that can be derived to implement different parsers. A parser produces bytecode to be executed within the regex matcher. - AK: Add regex matcher This patchset adds an regex matcher based on the principles of the T-REX VM. The bytecode pruduced by the respective Parser is put into the matcher and the VM will recursively execute the bytecode according to the available OpCodes. Possible improvement: the recursion could be replaced by multi threading capabilities. To match a Regular expression, e.g. for the Posix standard regular expression matcher use the following API: ``` Pattern<PosixExtendedParser> pattern("^.*$"); auto result = pattern.match("Well, hello friends!\nHello World!"); // Match whole needle EXPECT(result.count == 1); EXPECT(result.matches.at(0).view.starts_with("Well")); EXPECT(result.matches.at(0).view.end() == "!"); result = pattern.match("Well, hello friends!\nHello World!", PosixFlags::Multiline); // Match line by line EXPECT(result.count == 2); EXPECT(result.matches.at(0).view == "Well, hello friends!"); EXPECT(result.matches.at(1).view == "Hello World!"); EXPECT(pattern.has_match("Well,....")); // Just check if match without a result, which saves some resources. ``` - AK: Rework regex to work with opcodes objects This patchsets reworks the matcher to work on a more structured base. For that an abstract OpCode class and derived classes for the specific OpCodes have been added. The respective opcode logic is contained in each respective execute() method. - AK: Add benchmark for regex - AK: Some optimization in regex for runtime and memory - LibRegex: Move regex to own Library and fix all the broken stuff Now regex works again and grep utility is also in place for testing. This commit also fixes the use of regex.h in C by making `regex_t` an opaque (-ish) type, which makes its behaviour consistent between C and C++ compilers. Previously, <regex.h> would've blown C compilers up, and even if it didn't, would've caused a leak in C code, and not in C++ code (due to the existence of `OwnPtr` inside the struct). To make this whole ordeal easier to deal with (for now), this pulls the definitions of `reg*()` into LibRegex. pros: - The circular dependency between LibC and LibRegex is broken - Eaiser to test (without accidentally pulling in the host's libc!) cons: - Using any of the regex.h functions will require the user to link -lregex - The symbols will be missing from libc, which will be a big surprise down the line (especially with shared libs). Co-Authored-By: Ali Mohammad Pur <ali.mpfard@gmail.com>
2020-04-26 12:45:10 +00:00
};
enum class PosixFlags : FlagsUnderlyingType {
LibRegex: Make '.' reject matching LF / LS / PS as per the ECMA262 spec Previously we allowed it to match those, but the ECMA262 spec disallows these (except in DotAll).
2023-02-15 06:25:43 +00:00
Default = 0,
LibRegex: Add a regular expression library This commit is a mix of several commits, squashed into one because the commits before 'Move regex to own Library and fix all the broken stuff' were not fixable in any elegant way. The commits are listed below for "historical" purposes: - AK: Add options/flags and Errors for regular expressions Flags can be provided for any possible flavour by adding a new scoped enum. Handling of flags is done by templated Options class and the overloaded '|' and '&' operators. - AK: Add Lexer for regular expressions The lexer parses the input and extracts tokens needed to parse a regular expression. - AK: Add regex Parser and PosixExtendedParser This patchset adds a abstract parser class that can be derived to implement different parsers. A parser produces bytecode to be executed within the regex matcher. - AK: Add regex matcher This patchset adds an regex matcher based on the principles of the T-REX VM. The bytecode pruduced by the respective Parser is put into the matcher and the VM will recursively execute the bytecode according to the available OpCodes. Possible improvement: the recursion could be replaced by multi threading capabilities. To match a Regular expression, e.g. for the Posix standard regular expression matcher use the following API: ``` Pattern<PosixExtendedParser> pattern("^.*$"); auto result = pattern.match("Well, hello friends!\nHello World!"); // Match whole needle EXPECT(result.count == 1); EXPECT(result.matches.at(0).view.starts_with("Well")); EXPECT(result.matches.at(0).view.end() == "!"); result = pattern.match("Well, hello friends!\nHello World!", PosixFlags::Multiline); // Match line by line EXPECT(result.count == 2); EXPECT(result.matches.at(0).view == "Well, hello friends!"); EXPECT(result.matches.at(1).view == "Hello World!"); EXPECT(pattern.has_match("Well,....")); // Just check if match without a result, which saves some resources. ``` - AK: Rework regex to work with opcodes objects This patchsets reworks the matcher to work on a more structured base. For that an abstract OpCode class and derived classes for the specific OpCodes have been added. The respective opcode logic is contained in each respective execute() method. - AK: Add benchmark for regex - AK: Some optimization in regex for runtime and memory - LibRegex: Move regex to own Library and fix all the broken stuff Now regex works again and grep utility is also in place for testing. This commit also fixes the use of regex.h in C by making `regex_t` an opaque (-ish) type, which makes its behaviour consistent between C and C++ compilers. Previously, <regex.h> would've blown C compilers up, and even if it didn't, would've caused a leak in C code, and not in C++ code (due to the existence of `OwnPtr` inside the struct). To make this whole ordeal easier to deal with (for now), this pulls the definitions of `reg*()` into LibRegex. pros: - The circular dependency between LibC and LibRegex is broken - Eaiser to test (without accidentally pulling in the host's libc!) cons: - Using any of the regex.h functions will require the user to link -lregex - The symbols will be missing from libc, which will be a big surprise down the line (especially with shared libs). Co-Authored-By: Ali Mohammad Pur <ali.mpfard@gmail.com>
2020-04-26 12:45:10 +00:00
Global = (FlagsUnderlyingType)AllFlags::Global,
Insensitive = (FlagsUnderlyingType)AllFlags::Insensitive,
Ungreedy = (FlagsUnderlyingType)AllFlags::Ungreedy,
Unicode = (FlagsUnderlyingType)AllFlags::Unicode,
Extended = (FlagsUnderlyingType)AllFlags::Extended,
Extra = (FlagsUnderlyingType)AllFlags::Extra,
MatchNotBeginOfLine = (FlagsUnderlyingType)AllFlags::MatchNotBeginOfLine,
MatchNotEndOfLine = (FlagsUnderlyingType)AllFlags::MatchNotEndOfLine,
SkipSubExprResults = (FlagsUnderlyingType)AllFlags::SkipSubExprResults,
LibRegex: Add RegexStringView wrapper to support utf8 and utf32 views
2020-06-08 22:15:09 +00:00
SkipTrimEmptyMatches = (FlagsUnderlyingType)AllFlags::SkipTrimEmptyMatches,
LibRegex: Add a regular expression library This commit is a mix of several commits, squashed into one because the commits before 'Move regex to own Library and fix all the broken stuff' were not fixable in any elegant way. The commits are listed below for "historical" purposes: - AK: Add options/flags and Errors for regular expressions Flags can be provided for any possible flavour by adding a new scoped enum. Handling of flags is done by templated Options class and the overloaded '|' and '&' operators. - AK: Add Lexer for regular expressions The lexer parses the input and extracts tokens needed to parse a regular expression. - AK: Add regex Parser and PosixExtendedParser This patchset adds a abstract parser class that can be derived to implement different parsers. A parser produces bytecode to be executed within the regex matcher. - AK: Add regex matcher This patchset adds an regex matcher based on the principles of the T-REX VM. The bytecode pruduced by the respective Parser is put into the matcher and the VM will recursively execute the bytecode according to the available OpCodes. Possible improvement: the recursion could be replaced by multi threading capabilities. To match a Regular expression, e.g. for the Posix standard regular expression matcher use the following API: ``` Pattern<PosixExtendedParser> pattern("^.*$"); auto result = pattern.match("Well, hello friends!\nHello World!"); // Match whole needle EXPECT(result.count == 1); EXPECT(result.matches.at(0).view.starts_with("Well")); EXPECT(result.matches.at(0).view.end() == "!"); result = pattern.match("Well, hello friends!\nHello World!", PosixFlags::Multiline); // Match line by line EXPECT(result.count == 2); EXPECT(result.matches.at(0).view == "Well, hello friends!"); EXPECT(result.matches.at(1).view == "Hello World!"); EXPECT(pattern.has_match("Well,....")); // Just check if match without a result, which saves some resources. ``` - AK: Rework regex to work with opcodes objects This patchsets reworks the matcher to work on a more structured base. For that an abstract OpCode class and derived classes for the specific OpCodes have been added. The respective opcode logic is contained in each respective execute() method. - AK: Add benchmark for regex - AK: Some optimization in regex for runtime and memory - LibRegex: Move regex to own Library and fix all the broken stuff Now regex works again and grep utility is also in place for testing. This commit also fixes the use of regex.h in C by making `regex_t` an opaque (-ish) type, which makes its behaviour consistent between C and C++ compilers. Previously, <regex.h> would've blown C compilers up, and even if it didn't, would've caused a leak in C code, and not in C++ code (due to the existence of `OwnPtr` inside the struct). To make this whole ordeal easier to deal with (for now), this pulls the definitions of `reg*()` into LibRegex. pros: - The circular dependency between LibC and LibRegex is broken - Eaiser to test (without accidentally pulling in the host's libc!) cons: - Using any of the regex.h functions will require the user to link -lregex - The symbols will be missing from libc, which will be a big surprise down the line (especially with shared libs). Co-Authored-By: Ali Mohammad Pur <ali.mpfard@gmail.com>
2020-04-26 12:45:10 +00:00
Multiline = (FlagsUnderlyingType)AllFlags::Multiline,
LibRegex: Allow the SingleMatch flag to be used as a PosixFlag
2022-12-17 08:50:04 +00:00
SingleMatch = (FlagsUnderlyingType)AllFlags::SingleMatch,
LibRegex: Add a regular expression library This commit is a mix of several commits, squashed into one because the commits before 'Move regex to own Library and fix all the broken stuff' were not fixable in any elegant way. The commits are listed below for "historical" purposes: - AK: Add options/flags and Errors for regular expressions Flags can be provided for any possible flavour by adding a new scoped enum. Handling of flags is done by templated Options class and the overloaded '|' and '&' operators. - AK: Add Lexer for regular expressions The lexer parses the input and extracts tokens needed to parse a regular expression. - AK: Add regex Parser and PosixExtendedParser This patchset adds a abstract parser class that can be derived to implement different parsers. A parser produces bytecode to be executed within the regex matcher. - AK: Add regex matcher This patchset adds an regex matcher based on the principles of the T-REX VM. The bytecode pruduced by the respective Parser is put into the matcher and the VM will recursively execute the bytecode according to the available OpCodes. Possible improvement: the recursion could be replaced by multi threading capabilities. To match a Regular expression, e.g. for the Posix standard regular expression matcher use the following API: ``` Pattern<PosixExtendedParser> pattern("^.*$"); auto result = pattern.match("Well, hello friends!\nHello World!"); // Match whole needle EXPECT(result.count == 1); EXPECT(result.matches.at(0).view.starts_with("Well")); EXPECT(result.matches.at(0).view.end() == "!"); result = pattern.match("Well, hello friends!\nHello World!", PosixFlags::Multiline); // Match line by line EXPECT(result.count == 2); EXPECT(result.matches.at(0).view == "Well, hello friends!"); EXPECT(result.matches.at(1).view == "Hello World!"); EXPECT(pattern.has_match("Well,....")); // Just check if match without a result, which saves some resources. ``` - AK: Rework regex to work with opcodes objects This patchsets reworks the matcher to work on a more structured base. For that an abstract OpCode class and derived classes for the specific OpCodes have been added. The respective opcode logic is contained in each respective execute() method. - AK: Add benchmark for regex - AK: Some optimization in regex for runtime and memory - LibRegex: Move regex to own Library and fix all the broken stuff Now regex works again and grep utility is also in place for testing. This commit also fixes the use of regex.h in C by making `regex_t` an opaque (-ish) type, which makes its behaviour consistent between C and C++ compilers. Previously, <regex.h> would've blown C compilers up, and even if it didn't, would've caused a leak in C code, and not in C++ code (due to the existence of `OwnPtr` inside the struct). To make this whole ordeal easier to deal with (for now), this pulls the definitions of `reg*()` into LibRegex. pros: - The circular dependency between LibC and LibRegex is broken - Eaiser to test (without accidentally pulling in the host's libc!) cons: - Using any of the regex.h functions will require the user to link -lregex - The symbols will be missing from libc, which will be a big surprise down the line (especially with shared libs). Co-Authored-By: Ali Mohammad Pur <ali.mpfard@gmail.com>
2020-04-26 12:45:10 +00:00
StringCopyMatches = (FlagsUnderlyingType)AllFlags::StringCopyMatches,
};
enum class ECMAScriptFlags : FlagsUnderlyingType {
LibRegex: Make '.' reject matching LF / LS / PS as per the ECMA262 spec Previously we allowed it to match those, but the ECMA262 spec disallows these (except in DotAll).
2023-02-15 06:25:43 +00:00
Default = (FlagsUnderlyingType)AllFlags::Internal_ECMA262DotSemantics,
LibJS: Hook up Regex<ECMA262> to RegExpObject and implement `test()' This makes RegExpObject compile and store a Regex<ECMA262>, adds all flag-related properties, and implements `RegExpPrototype.test()` (complete with 'lastIndex' support) :^) It should be noted that this only implements `test()' using the builtin `exec()'.
2020-11-18 22:20:00 +00:00
Global = (FlagsUnderlyingType)AllFlags::Global | (FlagsUnderlyingType)AllFlags::Internal_Stateful, // Note: ECMAScript "Global" creates a stateful regex.
LibRegex: Add a regular expression library This commit is a mix of several commits, squashed into one because the commits before 'Move regex to own Library and fix all the broken stuff' were not fixable in any elegant way. The commits are listed below for "historical" purposes: - AK: Add options/flags and Errors for regular expressions Flags can be provided for any possible flavour by adding a new scoped enum. Handling of flags is done by templated Options class and the overloaded '|' and '&' operators. - AK: Add Lexer for regular expressions The lexer parses the input and extracts tokens needed to parse a regular expression. - AK: Add regex Parser and PosixExtendedParser This patchset adds a abstract parser class that can be derived to implement different parsers. A parser produces bytecode to be executed within the regex matcher. - AK: Add regex matcher This patchset adds an regex matcher based on the principles of the T-REX VM. The bytecode pruduced by the respective Parser is put into the matcher and the VM will recursively execute the bytecode according to the available OpCodes. Possible improvement: the recursion could be replaced by multi threading capabilities. To match a Regular expression, e.g. for the Posix standard regular expression matcher use the following API: ``` Pattern<PosixExtendedParser> pattern("^.*$"); auto result = pattern.match("Well, hello friends!\nHello World!"); // Match whole needle EXPECT(result.count == 1); EXPECT(result.matches.at(0).view.starts_with("Well")); EXPECT(result.matches.at(0).view.end() == "!"); result = pattern.match("Well, hello friends!\nHello World!", PosixFlags::Multiline); // Match line by line EXPECT(result.count == 2); EXPECT(result.matches.at(0).view == "Well, hello friends!"); EXPECT(result.matches.at(1).view == "Hello World!"); EXPECT(pattern.has_match("Well,....")); // Just check if match without a result, which saves some resources. ``` - AK: Rework regex to work with opcodes objects This patchsets reworks the matcher to work on a more structured base. For that an abstract OpCode class and derived classes for the specific OpCodes have been added. The respective opcode logic is contained in each respective execute() method. - AK: Add benchmark for regex - AK: Some optimization in regex for runtime and memory - LibRegex: Move regex to own Library and fix all the broken stuff Now regex works again and grep utility is also in place for testing. This commit also fixes the use of regex.h in C by making `regex_t` an opaque (-ish) type, which makes its behaviour consistent between C and C++ compilers. Previously, <regex.h> would've blown C compilers up, and even if it didn't, would've caused a leak in C code, and not in C++ code (due to the existence of `OwnPtr` inside the struct). To make this whole ordeal easier to deal with (for now), this pulls the definitions of `reg*()` into LibRegex. pros: - The circular dependency between LibC and LibRegex is broken - Eaiser to test (without accidentally pulling in the host's libc!) cons: - Using any of the regex.h functions will require the user to link -lregex - The symbols will be missing from libc, which will be a big surprise down the line (especially with shared libs). Co-Authored-By: Ali Mohammad Pur <ali.mpfard@gmail.com>
2020-04-26 12:45:10 +00:00
Insensitive = (FlagsUnderlyingType)AllFlags::Insensitive,
Ungreedy = (FlagsUnderlyingType)AllFlags::Ungreedy,
Unicode = (FlagsUnderlyingType)AllFlags::Unicode,
Extended = (FlagsUnderlyingType)AllFlags::Extended,
Extra = (FlagsUnderlyingType)AllFlags::Extra,
SingleLine = (FlagsUnderlyingType)AllFlags::SingleLine,
Sticky = (FlagsUnderlyingType)AllFlags::Sticky,
Multiline = (FlagsUnderlyingType)AllFlags::Multiline,
StringCopyMatches = (FlagsUnderlyingType)AllFlags::StringCopyMatches,
LibRegex: Partially implement the ECMAScript unicodeSets proposal This skips the new string unicode properties additions, along with \q{}.
2022-07-20 18:52:07 +00:00
UnicodeSets = (FlagsUnderlyingType)AllFlags::UnicodeSets,
LibRegex: Implement section B.1.4. of the ECMA262 spec This allows the parser to deal with crazy patterns like the one in #5517.
2021-02-26 19:19:34 +00:00
BrowserExtended = (FlagsUnderlyingType)AllFlags::Internal_BrowserExtended,
LibRegex: Add a regular expression library This commit is a mix of several commits, squashed into one because the commits before 'Move regex to own Library and fix all the broken stuff' were not fixable in any elegant way. The commits are listed below for "historical" purposes: - AK: Add options/flags and Errors for regular expressions Flags can be provided for any possible flavour by adding a new scoped enum. Handling of flags is done by templated Options class and the overloaded '|' and '&' operators. - AK: Add Lexer for regular expressions The lexer parses the input and extracts tokens needed to parse a regular expression. - AK: Add regex Parser and PosixExtendedParser This patchset adds a abstract parser class that can be derived to implement different parsers. A parser produces bytecode to be executed within the regex matcher. - AK: Add regex matcher This patchset adds an regex matcher based on the principles of the T-REX VM. The bytecode pruduced by the respective Parser is put into the matcher and the VM will recursively execute the bytecode according to the available OpCodes. Possible improvement: the recursion could be replaced by multi threading capabilities. To match a Regular expression, e.g. for the Posix standard regular expression matcher use the following API: ``` Pattern<PosixExtendedParser> pattern("^.*$"); auto result = pattern.match("Well, hello friends!\nHello World!"); // Match whole needle EXPECT(result.count == 1); EXPECT(result.matches.at(0).view.starts_with("Well")); EXPECT(result.matches.at(0).view.end() == "!"); result = pattern.match("Well, hello friends!\nHello World!", PosixFlags::Multiline); // Match line by line EXPECT(result.count == 2); EXPECT(result.matches.at(0).view == "Well, hello friends!"); EXPECT(result.matches.at(1).view == "Hello World!"); EXPECT(pattern.has_match("Well,....")); // Just check if match without a result, which saves some resources. ``` - AK: Rework regex to work with opcodes objects This patchsets reworks the matcher to work on a more structured base. For that an abstract OpCode class and derived classes for the specific OpCodes have been added. The respective opcode logic is contained in each respective execute() method. - AK: Add benchmark for regex - AK: Some optimization in regex for runtime and memory - LibRegex: Move regex to own Library and fix all the broken stuff Now regex works again and grep utility is also in place for testing. This commit also fixes the use of regex.h in C by making `regex_t` an opaque (-ish) type, which makes its behaviour consistent between C and C++ compilers. Previously, <regex.h> would've blown C compilers up, and even if it didn't, would've caused a leak in C code, and not in C++ code (due to the existence of `OwnPtr` inside the struct). To make this whole ordeal easier to deal with (for now), this pulls the definitions of `reg*()` into LibRegex. pros: - The circular dependency between LibC and LibRegex is broken - Eaiser to test (without accidentally pulling in the host's libc!) cons: - Using any of the regex.h functions will require the user to link -lregex - The symbols will be missing from libc, which will be a big surprise down the line (especially with shared libs). Co-Authored-By: Ali Mohammad Pur <ali.mpfard@gmail.com>
2020-04-26 12:45:10 +00:00
};
template<class T>
class RegexOptions {
public:
using FlagsType = T;
RegexOptions() = default;
LibRegex: Allow RegexOptions to be declared at compile time
2021-07-29 13:30:58 +00:00
constexpr RegexOptions(T flags)
LibRegex: Make '.' reject matching LF / LS / PS as per the ECMA262 spec Previously we allowed it to match those, but the ECMA262 spec disallows these (except in DotAll).
2023-02-15 06:25:43 +00:00
: m_flags(static_cast<T>(to_underlying(flags) | to_underlying(T::Default)))
LibRegex: Add a regular expression library This commit is a mix of several commits, squashed into one because the commits before 'Move regex to own Library and fix all the broken stuff' were not fixable in any elegant way. The commits are listed below for "historical" purposes: - AK: Add options/flags and Errors for regular expressions Flags can be provided for any possible flavour by adding a new scoped enum. Handling of flags is done by templated Options class and the overloaded '|' and '&' operators. - AK: Add Lexer for regular expressions The lexer parses the input and extracts tokens needed to parse a regular expression. - AK: Add regex Parser and PosixExtendedParser This patchset adds a abstract parser class that can be derived to implement different parsers. A parser produces bytecode to be executed within the regex matcher. - AK: Add regex matcher This patchset adds an regex matcher based on the principles of the T-REX VM. The bytecode pruduced by the respective Parser is put into the matcher and the VM will recursively execute the bytecode according to the available OpCodes. Possible improvement: the recursion could be replaced by multi threading capabilities. To match a Regular expression, e.g. for the Posix standard regular expression matcher use the following API: ``` Pattern<PosixExtendedParser> pattern("^.*$"); auto result = pattern.match("Well, hello friends!\nHello World!"); // Match whole needle EXPECT(result.count == 1); EXPECT(result.matches.at(0).view.starts_with("Well")); EXPECT(result.matches.at(0).view.end() == "!"); result = pattern.match("Well, hello friends!\nHello World!", PosixFlags::Multiline); // Match line by line EXPECT(result.count == 2); EXPECT(result.matches.at(0).view == "Well, hello friends!"); EXPECT(result.matches.at(1).view == "Hello World!"); EXPECT(pattern.has_match("Well,....")); // Just check if match without a result, which saves some resources. ``` - AK: Rework regex to work with opcodes objects This patchsets reworks the matcher to work on a more structured base. For that an abstract OpCode class and derived classes for the specific OpCodes have been added. The respective opcode logic is contained in each respective execute() method. - AK: Add benchmark for regex - AK: Some optimization in regex for runtime and memory - LibRegex: Move regex to own Library and fix all the broken stuff Now regex works again and grep utility is also in place for testing. This commit also fixes the use of regex.h in C by making `regex_t` an opaque (-ish) type, which makes its behaviour consistent between C and C++ compilers. Previously, <regex.h> would've blown C compilers up, and even if it didn't, would've caused a leak in C code, and not in C++ code (due to the existence of `OwnPtr` inside the struct). To make this whole ordeal easier to deal with (for now), this pulls the definitions of `reg*()` into LibRegex. pros: - The circular dependency between LibC and LibRegex is broken - Eaiser to test (without accidentally pulling in the host's libc!) cons: - Using any of the regex.h functions will require the user to link -lregex - The symbols will be missing from libc, which will be a big surprise down the line (especially with shared libs). Co-Authored-By: Ali Mohammad Pur <ali.mpfard@gmail.com>
2020-04-26 12:45:10 +00:00
{
}
template<class U>
LibRegex: Allow RegexOptions to be declared at compile time
2021-07-29 13:30:58 +00:00
constexpr RegexOptions(RegexOptions<U> other)
LibRegex: Make '.' reject matching LF / LS / PS as per the ECMA262 spec Previously we allowed it to match those, but the ECMA262 spec disallows these (except in DotAll).
2023-02-15 06:25:43 +00:00
: RegexOptions(static_cast<T>(to_underlying(other.value())))
LibRegex: Add a regular expression library This commit is a mix of several commits, squashed into one because the commits before 'Move regex to own Library and fix all the broken stuff' were not fixable in any elegant way. The commits are listed below for "historical" purposes: - AK: Add options/flags and Errors for regular expressions Flags can be provided for any possible flavour by adding a new scoped enum. Handling of flags is done by templated Options class and the overloaded '|' and '&' operators. - AK: Add Lexer for regular expressions The lexer parses the input and extracts tokens needed to parse a regular expression. - AK: Add regex Parser and PosixExtendedParser This patchset adds a abstract parser class that can be derived to implement different parsers. A parser produces bytecode to be executed within the regex matcher. - AK: Add regex matcher This patchset adds an regex matcher based on the principles of the T-REX VM. The bytecode pruduced by the respective Parser is put into the matcher and the VM will recursively execute the bytecode according to the available OpCodes. Possible improvement: the recursion could be replaced by multi threading capabilities. To match a Regular expression, e.g. for the Posix standard regular expression matcher use the following API: ``` Pattern<PosixExtendedParser> pattern("^.*$"); auto result = pattern.match("Well, hello friends!\nHello World!"); // Match whole needle EXPECT(result.count == 1); EXPECT(result.matches.at(0).view.starts_with("Well")); EXPECT(result.matches.at(0).view.end() == "!"); result = pattern.match("Well, hello friends!\nHello World!", PosixFlags::Multiline); // Match line by line EXPECT(result.count == 2); EXPECT(result.matches.at(0).view == "Well, hello friends!"); EXPECT(result.matches.at(1).view == "Hello World!"); EXPECT(pattern.has_match("Well,....")); // Just check if match without a result, which saves some resources. ``` - AK: Rework regex to work with opcodes objects This patchsets reworks the matcher to work on a more structured base. For that an abstract OpCode class and derived classes for the specific OpCodes have been added. The respective opcode logic is contained in each respective execute() method. - AK: Add benchmark for regex - AK: Some optimization in regex for runtime and memory - LibRegex: Move regex to own Library and fix all the broken stuff Now regex works again and grep utility is also in place for testing. This commit also fixes the use of regex.h in C by making `regex_t` an opaque (-ish) type, which makes its behaviour consistent between C and C++ compilers. Previously, <regex.h> would've blown C compilers up, and even if it didn't, would've caused a leak in C code, and not in C++ code (due to the existence of `OwnPtr` inside the struct). To make this whole ordeal easier to deal with (for now), this pulls the definitions of `reg*()` into LibRegex. pros: - The circular dependency between LibC and LibRegex is broken - Eaiser to test (without accidentally pulling in the host's libc!) cons: - Using any of the regex.h functions will require the user to link -lregex - The symbols will be missing from libc, which will be a big surprise down the line (especially with shared libs). Co-Authored-By: Ali Mohammad Pur <ali.mpfard@gmail.com>
2020-04-26 12:45:10 +00:00
{
}
operator bool() const { return !!*this; }
bool operator!() const { return (FlagsUnderlyingType)m_flags == 0; }
LibRegex: Allow RegexOptions to be declared at compile time
2021-07-29 13:30:58 +00:00
constexpr RegexOptions<T> operator|(T flag) const { return RegexOptions<T> { (T)((FlagsUnderlyingType)m_flags | (FlagsUnderlyingType)flag) }; }
constexpr RegexOptions<T> operator&(T flag) const { return RegexOptions<T> { (T)((FlagsUnderlyingType)m_flags & (FlagsUnderlyingType)flag) }; }
LibRegex: Add a regular expression library This commit is a mix of several commits, squashed into one because the commits before 'Move regex to own Library and fix all the broken stuff' were not fixable in any elegant way. The commits are listed below for "historical" purposes: - AK: Add options/flags and Errors for regular expressions Flags can be provided for any possible flavour by adding a new scoped enum. Handling of flags is done by templated Options class and the overloaded '|' and '&' operators. - AK: Add Lexer for regular expressions The lexer parses the input and extracts tokens needed to parse a regular expression. - AK: Add regex Parser and PosixExtendedParser This patchset adds a abstract parser class that can be derived to implement different parsers. A parser produces bytecode to be executed within the regex matcher. - AK: Add regex matcher This patchset adds an regex matcher based on the principles of the T-REX VM. The bytecode pruduced by the respective Parser is put into the matcher and the VM will recursively execute the bytecode according to the available OpCodes. Possible improvement: the recursion could be replaced by multi threading capabilities. To match a Regular expression, e.g. for the Posix standard regular expression matcher use the following API: ``` Pattern<PosixExtendedParser> pattern("^.*$"); auto result = pattern.match("Well, hello friends!\nHello World!"); // Match whole needle EXPECT(result.count == 1); EXPECT(result.matches.at(0).view.starts_with("Well")); EXPECT(result.matches.at(0).view.end() == "!"); result = pattern.match("Well, hello friends!\nHello World!", PosixFlags::Multiline); // Match line by line EXPECT(result.count == 2); EXPECT(result.matches.at(0).view == "Well, hello friends!"); EXPECT(result.matches.at(1).view == "Hello World!"); EXPECT(pattern.has_match("Well,....")); // Just check if match without a result, which saves some resources. ``` - AK: Rework regex to work with opcodes objects This patchsets reworks the matcher to work on a more structured base. For that an abstract OpCode class and derived classes for the specific OpCodes have been added. The respective opcode logic is contained in each respective execute() method. - AK: Add benchmark for regex - AK: Some optimization in regex for runtime and memory - LibRegex: Move regex to own Library and fix all the broken stuff Now regex works again and grep utility is also in place for testing. This commit also fixes the use of regex.h in C by making `regex_t` an opaque (-ish) type, which makes its behaviour consistent between C and C++ compilers. Previously, <regex.h> would've blown C compilers up, and even if it didn't, would've caused a leak in C code, and not in C++ code (due to the existence of `OwnPtr` inside the struct). To make this whole ordeal easier to deal with (for now), this pulls the definitions of `reg*()` into LibRegex. pros: - The circular dependency between LibC and LibRegex is broken - Eaiser to test (without accidentally pulling in the host's libc!) cons: - Using any of the regex.h functions will require the user to link -lregex - The symbols will be missing from libc, which will be a big surprise down the line (especially with shared libs). Co-Authored-By: Ali Mohammad Pur <ali.mpfard@gmail.com>
2020-04-26 12:45:10 +00:00
LibRegex: Allow RegexOptions to be declared at compile time
2021-07-29 13:30:58 +00:00
constexpr RegexOptions<T>& operator|=(T flag)
LibRegex: Add a regular expression library This commit is a mix of several commits, squashed into one because the commits before 'Move regex to own Library and fix all the broken stuff' were not fixable in any elegant way. The commits are listed below for "historical" purposes: - AK: Add options/flags and Errors for regular expressions Flags can be provided for any possible flavour by adding a new scoped enum. Handling of flags is done by templated Options class and the overloaded '|' and '&' operators. - AK: Add Lexer for regular expressions The lexer parses the input and extracts tokens needed to parse a regular expression. - AK: Add regex Parser and PosixExtendedParser This patchset adds a abstract parser class that can be derived to implement different parsers. A parser produces bytecode to be executed within the regex matcher. - AK: Add regex matcher This patchset adds an regex matcher based on the principles of the T-REX VM. The bytecode pruduced by the respective Parser is put into the matcher and the VM will recursively execute the bytecode according to the available OpCodes. Possible improvement: the recursion could be replaced by multi threading capabilities. To match a Regular expression, e.g. for the Posix standard regular expression matcher use the following API: ``` Pattern<PosixExtendedParser> pattern("^.*$"); auto result = pattern.match("Well, hello friends!\nHello World!"); // Match whole needle EXPECT(result.count == 1); EXPECT(result.matches.at(0).view.starts_with("Well")); EXPECT(result.matches.at(0).view.end() == "!"); result = pattern.match("Well, hello friends!\nHello World!", PosixFlags::Multiline); // Match line by line EXPECT(result.count == 2); EXPECT(result.matches.at(0).view == "Well, hello friends!"); EXPECT(result.matches.at(1).view == "Hello World!"); EXPECT(pattern.has_match("Well,....")); // Just check if match without a result, which saves some resources. ``` - AK: Rework regex to work with opcodes objects This patchsets reworks the matcher to work on a more structured base. For that an abstract OpCode class and derived classes for the specific OpCodes have been added. The respective opcode logic is contained in each respective execute() method. - AK: Add benchmark for regex - AK: Some optimization in regex for runtime and memory - LibRegex: Move regex to own Library and fix all the broken stuff Now regex works again and grep utility is also in place for testing. This commit also fixes the use of regex.h in C by making `regex_t` an opaque (-ish) type, which makes its behaviour consistent between C and C++ compilers. Previously, <regex.h> would've blown C compilers up, and even if it didn't, would've caused a leak in C code, and not in C++ code (due to the existence of `OwnPtr` inside the struct). To make this whole ordeal easier to deal with (for now), this pulls the definitions of `reg*()` into LibRegex. pros: - The circular dependency between LibC and LibRegex is broken - Eaiser to test (without accidentally pulling in the host's libc!) cons: - Using any of the regex.h functions will require the user to link -lregex - The symbols will be missing from libc, which will be a big surprise down the line (especially with shared libs). Co-Authored-By: Ali Mohammad Pur <ali.mpfard@gmail.com>
2020-04-26 12:45:10 +00:00
{
m_flags = (T)((FlagsUnderlyingType)m_flags | (FlagsUnderlyingType)flag);
return *this;
}
LibRegex: Allow RegexOptions to be declared at compile time
2021-07-29 13:30:58 +00:00
constexpr RegexOptions<T>& operator&=(T flag)
LibRegex: Add a regular expression library This commit is a mix of several commits, squashed into one because the commits before 'Move regex to own Library and fix all the broken stuff' were not fixable in any elegant way. The commits are listed below for "historical" purposes: - AK: Add options/flags and Errors for regular expressions Flags can be provided for any possible flavour by adding a new scoped enum. Handling of flags is done by templated Options class and the overloaded '|' and '&' operators. - AK: Add Lexer for regular expressions The lexer parses the input and extracts tokens needed to parse a regular expression. - AK: Add regex Parser and PosixExtendedParser This patchset adds a abstract parser class that can be derived to implement different parsers. A parser produces bytecode to be executed within the regex matcher. - AK: Add regex matcher This patchset adds an regex matcher based on the principles of the T-REX VM. The bytecode pruduced by the respective Parser is put into the matcher and the VM will recursively execute the bytecode according to the available OpCodes. Possible improvement: the recursion could be replaced by multi threading capabilities. To match a Regular expression, e.g. for the Posix standard regular expression matcher use the following API: ``` Pattern<PosixExtendedParser> pattern("^.*$"); auto result = pattern.match("Well, hello friends!\nHello World!"); // Match whole needle EXPECT(result.count == 1); EXPECT(result.matches.at(0).view.starts_with("Well")); EXPECT(result.matches.at(0).view.end() == "!"); result = pattern.match("Well, hello friends!\nHello World!", PosixFlags::Multiline); // Match line by line EXPECT(result.count == 2); EXPECT(result.matches.at(0).view == "Well, hello friends!"); EXPECT(result.matches.at(1).view == "Hello World!"); EXPECT(pattern.has_match("Well,....")); // Just check if match without a result, which saves some resources. ``` - AK: Rework regex to work with opcodes objects This patchsets reworks the matcher to work on a more structured base. For that an abstract OpCode class and derived classes for the specific OpCodes have been added. The respective opcode logic is contained in each respective execute() method. - AK: Add benchmark for regex - AK: Some optimization in regex for runtime and memory - LibRegex: Move regex to own Library and fix all the broken stuff Now regex works again and grep utility is also in place for testing. This commit also fixes the use of regex.h in C by making `regex_t` an opaque (-ish) type, which makes its behaviour consistent between C and C++ compilers. Previously, <regex.h> would've blown C compilers up, and even if it didn't, would've caused a leak in C code, and not in C++ code (due to the existence of `OwnPtr` inside the struct). To make this whole ordeal easier to deal with (for now), this pulls the definitions of `reg*()` into LibRegex. pros: - The circular dependency between LibC and LibRegex is broken - Eaiser to test (without accidentally pulling in the host's libc!) cons: - Using any of the regex.h functions will require the user to link -lregex - The symbols will be missing from libc, which will be a big surprise down the line (especially with shared libs). Co-Authored-By: Ali Mohammad Pur <ali.mpfard@gmail.com>
2020-04-26 12:45:10 +00:00
{
m_flags = (T)((FlagsUnderlyingType)m_flags & (FlagsUnderlyingType)flag);
return *this;
}
void reset_flags() { m_flags = (T)0; }
void reset_flag(T flag) { m_flags = (T)((FlagsUnderlyingType)m_flags & ~(FlagsUnderlyingType)flag); }
void set_flag(T flag) { *this |= flag; }
LibJS: Hook up Regex<ECMA262> to RegExpObject and implement `test()' This makes RegExpObject compile and store a Regex<ECMA262>, adds all flag-related properties, and implements `RegExpPrototype.test()` (complete with 'lastIndex' support) :^) It should be noted that this only implements `test()' using the builtin `exec()'.
2020-11-18 22:20:00 +00:00
bool has_flag_set(T flag) const { return (FlagsUnderlyingType)flag == ((FlagsUnderlyingType)m_flags & (FlagsUnderlyingType)flag); }
LibRegex: Make ^ and $ accept all `LineTerminator`s instead of just '\n' Also adds a couple tests.
2023-03-25 13:06:05 +00:00
constexpr T value() const { return m_flags; }
LibRegex: Add a regular expression library This commit is a mix of several commits, squashed into one because the commits before 'Move regex to own Library and fix all the broken stuff' were not fixable in any elegant way. The commits are listed below for "historical" purposes: - AK: Add options/flags and Errors for regular expressions Flags can be provided for any possible flavour by adding a new scoped enum. Handling of flags is done by templated Options class and the overloaded '|' and '&' operators. - AK: Add Lexer for regular expressions The lexer parses the input and extracts tokens needed to parse a regular expression. - AK: Add regex Parser and PosixExtendedParser This patchset adds a abstract parser class that can be derived to implement different parsers. A parser produces bytecode to be executed within the regex matcher. - AK: Add regex matcher This patchset adds an regex matcher based on the principles of the T-REX VM. The bytecode pruduced by the respective Parser is put into the matcher and the VM will recursively execute the bytecode according to the available OpCodes. Possible improvement: the recursion could be replaced by multi threading capabilities. To match a Regular expression, e.g. for the Posix standard regular expression matcher use the following API: ``` Pattern<PosixExtendedParser> pattern("^.*$"); auto result = pattern.match("Well, hello friends!\nHello World!"); // Match whole needle EXPECT(result.count == 1); EXPECT(result.matches.at(0).view.starts_with("Well")); EXPECT(result.matches.at(0).view.end() == "!"); result = pattern.match("Well, hello friends!\nHello World!", PosixFlags::Multiline); // Match line by line EXPECT(result.count == 2); EXPECT(result.matches.at(0).view == "Well, hello friends!"); EXPECT(result.matches.at(1).view == "Hello World!"); EXPECT(pattern.has_match("Well,....")); // Just check if match without a result, which saves some resources. ``` - AK: Rework regex to work with opcodes objects This patchsets reworks the matcher to work on a more structured base. For that an abstract OpCode class and derived classes for the specific OpCodes have been added. The respective opcode logic is contained in each respective execute() method. - AK: Add benchmark for regex - AK: Some optimization in regex for runtime and memory - LibRegex: Move regex to own Library and fix all the broken stuff Now regex works again and grep utility is also in place for testing. This commit also fixes the use of regex.h in C by making `regex_t` an opaque (-ish) type, which makes its behaviour consistent between C and C++ compilers. Previously, <regex.h> would've blown C compilers up, and even if it didn't, would've caused a leak in C code, and not in C++ code (due to the existence of `OwnPtr` inside the struct). To make this whole ordeal easier to deal with (for now), this pulls the definitions of `reg*()` into LibRegex. pros: - The circular dependency between LibC and LibRegex is broken - Eaiser to test (without accidentally pulling in the host's libc!) cons: - Using any of the regex.h functions will require the user to link -lregex - The symbols will be missing from libc, which will be a big surprise down the line (especially with shared libs). Co-Authored-By: Ali Mohammad Pur <ali.mpfard@gmail.com>
2020-04-26 12:45:10 +00:00
private:
LibRegex: Make '.' reject matching LF / LS / PS as per the ECMA262 spec Previously we allowed it to match those, but the ECMA262 spec disallows these (except in DotAll).
2023-02-15 06:25:43 +00:00
T m_flags { T::Default };
LibRegex: Add a regular expression library This commit is a mix of several commits, squashed into one because the commits before 'Move regex to own Library and fix all the broken stuff' were not fixable in any elegant way. The commits are listed below for "historical" purposes: - AK: Add options/flags and Errors for regular expressions Flags can be provided for any possible flavour by adding a new scoped enum. Handling of flags is done by templated Options class and the overloaded '|' and '&' operators. - AK: Add Lexer for regular expressions The lexer parses the input and extracts tokens needed to parse a regular expression. - AK: Add regex Parser and PosixExtendedParser This patchset adds a abstract parser class that can be derived to implement different parsers. A parser produces bytecode to be executed within the regex matcher. - AK: Add regex matcher This patchset adds an regex matcher based on the principles of the T-REX VM. The bytecode pruduced by the respective Parser is put into the matcher and the VM will recursively execute the bytecode according to the available OpCodes. Possible improvement: the recursion could be replaced by multi threading capabilities. To match a Regular expression, e.g. for the Posix standard regular expression matcher use the following API: ``` Pattern<PosixExtendedParser> pattern("^.*$"); auto result = pattern.match("Well, hello friends!\nHello World!"); // Match whole needle EXPECT(result.count == 1); EXPECT(result.matches.at(0).view.starts_with("Well")); EXPECT(result.matches.at(0).view.end() == "!"); result = pattern.match("Well, hello friends!\nHello World!", PosixFlags::Multiline); // Match line by line EXPECT(result.count == 2); EXPECT(result.matches.at(0).view == "Well, hello friends!"); EXPECT(result.matches.at(1).view == "Hello World!"); EXPECT(pattern.has_match("Well,....")); // Just check if match without a result, which saves some resources. ``` - AK: Rework regex to work with opcodes objects This patchsets reworks the matcher to work on a more structured base. For that an abstract OpCode class and derived classes for the specific OpCodes have been added. The respective opcode logic is contained in each respective execute() method. - AK: Add benchmark for regex - AK: Some optimization in regex for runtime and memory - LibRegex: Move regex to own Library and fix all the broken stuff Now regex works again and grep utility is also in place for testing. This commit also fixes the use of regex.h in C by making `regex_t` an opaque (-ish) type, which makes its behaviour consistent between C and C++ compilers. Previously, <regex.h> would've blown C compilers up, and even if it didn't, would've caused a leak in C code, and not in C++ code (due to the existence of `OwnPtr` inside the struct). To make this whole ordeal easier to deal with (for now), this pulls the definitions of `reg*()` into LibRegex. pros: - The circular dependency between LibC and LibRegex is broken - Eaiser to test (without accidentally pulling in the host's libc!) cons: - Using any of the regex.h functions will require the user to link -lregex - The symbols will be missing from libc, which will be a big surprise down the line (especially with shared libs). Co-Authored-By: Ali Mohammad Pur <ali.mpfard@gmail.com>
2020-04-26 12:45:10 +00:00
};
template<class T>
LibRegex: Allow RegexOptions to be declared at compile time
2021-07-29 13:30:58 +00:00
constexpr RegexOptions<T> operator|(T lhs, T rhs)
LibRegex: Add a regular expression library This commit is a mix of several commits, squashed into one because the commits before 'Move regex to own Library and fix all the broken stuff' were not fixable in any elegant way. The commits are listed below for "historical" purposes: - AK: Add options/flags and Errors for regular expressions Flags can be provided for any possible flavour by adding a new scoped enum. Handling of flags is done by templated Options class and the overloaded '|' and '&' operators. - AK: Add Lexer for regular expressions The lexer parses the input and extracts tokens needed to parse a regular expression. - AK: Add regex Parser and PosixExtendedParser This patchset adds a abstract parser class that can be derived to implement different parsers. A parser produces bytecode to be executed within the regex matcher. - AK: Add regex matcher This patchset adds an regex matcher based on the principles of the T-REX VM. The bytecode pruduced by the respective Parser is put into the matcher and the VM will recursively execute the bytecode according to the available OpCodes. Possible improvement: the recursion could be replaced by multi threading capabilities. To match a Regular expression, e.g. for the Posix standard regular expression matcher use the following API: ``` Pattern<PosixExtendedParser> pattern("^.*$"); auto result = pattern.match("Well, hello friends!\nHello World!"); // Match whole needle EXPECT(result.count == 1); EXPECT(result.matches.at(0).view.starts_with("Well")); EXPECT(result.matches.at(0).view.end() == "!"); result = pattern.match("Well, hello friends!\nHello World!", PosixFlags::Multiline); // Match line by line EXPECT(result.count == 2); EXPECT(result.matches.at(0).view == "Well, hello friends!"); EXPECT(result.matches.at(1).view == "Hello World!"); EXPECT(pattern.has_match("Well,....")); // Just check if match without a result, which saves some resources. ``` - AK: Rework regex to work with opcodes objects This patchsets reworks the matcher to work on a more structured base. For that an abstract OpCode class and derived classes for the specific OpCodes have been added. The respective opcode logic is contained in each respective execute() method. - AK: Add benchmark for regex - AK: Some optimization in regex for runtime and memory - LibRegex: Move regex to own Library and fix all the broken stuff Now regex works again and grep utility is also in place for testing. This commit also fixes the use of regex.h in C by making `regex_t` an opaque (-ish) type, which makes its behaviour consistent between C and C++ compilers. Previously, <regex.h> would've blown C compilers up, and even if it didn't, would've caused a leak in C code, and not in C++ code (due to the existence of `OwnPtr` inside the struct). To make this whole ordeal easier to deal with (for now), this pulls the definitions of `reg*()` into LibRegex. pros: - The circular dependency between LibC and LibRegex is broken - Eaiser to test (without accidentally pulling in the host's libc!) cons: - Using any of the regex.h functions will require the user to link -lregex - The symbols will be missing from libc, which will be a big surprise down the line (especially with shared libs). Co-Authored-By: Ali Mohammad Pur <ali.mpfard@gmail.com>
2020-04-26 12:45:10 +00:00
{
return RegexOptions<T> { lhs } |= rhs;
}
template<class T>
LibRegex: Allow RegexOptions to be declared at compile time
2021-07-29 13:30:58 +00:00
constexpr RegexOptions<T> operator&(T lhs, T rhs)
LibRegex: Add a regular expression library This commit is a mix of several commits, squashed into one because the commits before 'Move regex to own Library and fix all the broken stuff' were not fixable in any elegant way. The commits are listed below for "historical" purposes: - AK: Add options/flags and Errors for regular expressions Flags can be provided for any possible flavour by adding a new scoped enum. Handling of flags is done by templated Options class and the overloaded '|' and '&' operators. - AK: Add Lexer for regular expressions The lexer parses the input and extracts tokens needed to parse a regular expression. - AK: Add regex Parser and PosixExtendedParser This patchset adds a abstract parser class that can be derived to implement different parsers. A parser produces bytecode to be executed within the regex matcher. - AK: Add regex matcher This patchset adds an regex matcher based on the principles of the T-REX VM. The bytecode pruduced by the respective Parser is put into the matcher and the VM will recursively execute the bytecode according to the available OpCodes. Possible improvement: the recursion could be replaced by multi threading capabilities. To match a Regular expression, e.g. for the Posix standard regular expression matcher use the following API: ``` Pattern<PosixExtendedParser> pattern("^.*$"); auto result = pattern.match("Well, hello friends!\nHello World!"); // Match whole needle EXPECT(result.count == 1); EXPECT(result.matches.at(0).view.starts_with("Well")); EXPECT(result.matches.at(0).view.end() == "!"); result = pattern.match("Well, hello friends!\nHello World!", PosixFlags::Multiline); // Match line by line EXPECT(result.count == 2); EXPECT(result.matches.at(0).view == "Well, hello friends!"); EXPECT(result.matches.at(1).view == "Hello World!"); EXPECT(pattern.has_match("Well,....")); // Just check if match without a result, which saves some resources. ``` - AK: Rework regex to work with opcodes objects This patchsets reworks the matcher to work on a more structured base. For that an abstract OpCode class and derived classes for the specific OpCodes have been added. The respective opcode logic is contained in each respective execute() method. - AK: Add benchmark for regex - AK: Some optimization in regex for runtime and memory - LibRegex: Move regex to own Library and fix all the broken stuff Now regex works again and grep utility is also in place for testing. This commit also fixes the use of regex.h in C by making `regex_t` an opaque (-ish) type, which makes its behaviour consistent between C and C++ compilers. Previously, <regex.h> would've blown C compilers up, and even if it didn't, would've caused a leak in C code, and not in C++ code (due to the existence of `OwnPtr` inside the struct). To make this whole ordeal easier to deal with (for now), this pulls the definitions of `reg*()` into LibRegex. pros: - The circular dependency between LibC and LibRegex is broken - Eaiser to test (without accidentally pulling in the host's libc!) cons: - Using any of the regex.h functions will require the user to link -lregex - The symbols will be missing from libc, which will be a big surprise down the line (especially with shared libs). Co-Authored-By: Ali Mohammad Pur <ali.mpfard@gmail.com>
2020-04-26 12:45:10 +00:00
{
return RegexOptions<T> { lhs } &= rhs;
}
template<class T>
LibRegex: Allow RegexOptions to be declared at compile time
2021-07-29 13:30:58 +00:00
constexpr T operator~(T flag)
LibRegex: Add a regular expression library This commit is a mix of several commits, squashed into one because the commits before 'Move regex to own Library and fix all the broken stuff' were not fixable in any elegant way. The commits are listed below for "historical" purposes: - AK: Add options/flags and Errors for regular expressions Flags can be provided for any possible flavour by adding a new scoped enum. Handling of flags is done by templated Options class and the overloaded '|' and '&' operators. - AK: Add Lexer for regular expressions The lexer parses the input and extracts tokens needed to parse a regular expression. - AK: Add regex Parser and PosixExtendedParser This patchset adds a abstract parser class that can be derived to implement different parsers. A parser produces bytecode to be executed within the regex matcher. - AK: Add regex matcher This patchset adds an regex matcher based on the principles of the T-REX VM. The bytecode pruduced by the respective Parser is put into the matcher and the VM will recursively execute the bytecode according to the available OpCodes. Possible improvement: the recursion could be replaced by multi threading capabilities. To match a Regular expression, e.g. for the Posix standard regular expression matcher use the following API: ``` Pattern<PosixExtendedParser> pattern("^.*$"); auto result = pattern.match("Well, hello friends!\nHello World!"); // Match whole needle EXPECT(result.count == 1); EXPECT(result.matches.at(0).view.starts_with("Well")); EXPECT(result.matches.at(0).view.end() == "!"); result = pattern.match("Well, hello friends!\nHello World!", PosixFlags::Multiline); // Match line by line EXPECT(result.count == 2); EXPECT(result.matches.at(0).view == "Well, hello friends!"); EXPECT(result.matches.at(1).view == "Hello World!"); EXPECT(pattern.has_match("Well,....")); // Just check if match without a result, which saves some resources. ``` - AK: Rework regex to work with opcodes objects This patchsets reworks the matcher to work on a more structured base. For that an abstract OpCode class and derived classes for the specific OpCodes have been added. The respective opcode logic is contained in each respective execute() method. - AK: Add benchmark for regex - AK: Some optimization in regex for runtime and memory - LibRegex: Move regex to own Library and fix all the broken stuff Now regex works again and grep utility is also in place for testing. This commit also fixes the use of regex.h in C by making `regex_t` an opaque (-ish) type, which makes its behaviour consistent between C and C++ compilers. Previously, <regex.h> would've blown C compilers up, and even if it didn't, would've caused a leak in C code, and not in C++ code (due to the existence of `OwnPtr` inside the struct). To make this whole ordeal easier to deal with (for now), this pulls the definitions of `reg*()` into LibRegex. pros: - The circular dependency between LibC and LibRegex is broken - Eaiser to test (without accidentally pulling in the host's libc!) cons: - Using any of the regex.h functions will require the user to link -lregex - The symbols will be missing from libc, which will be a big surprise down the line (especially with shared libs). Co-Authored-By: Ali Mohammad Pur <ali.mpfard@gmail.com>
2020-04-26 12:45:10 +00:00
{
return (T) ~((FlagsUnderlyingType)flag);
}
using AllOptions = RegexOptions<AllFlags>;
using ECMAScriptOptions = RegexOptions<ECMAScriptFlags>;
using PosixOptions = RegexOptions<PosixFlags>;
}
using regex::ECMAScriptFlags;
using regex::ECMAScriptOptions;
using regex::PosixFlags;
using regex::PosixOptions;
Reference in a new issue Copy permalink