Commit graph

39 commits

Author SHA1 Message Date
Pavel Shliak
cdb54fe504 LibRegex: Clean up #include directives
This change aims to improve the speed of incremental builds.
2024-11-21 14:08:33 +01:00
Ali Mohammad Pur
5a4d657a4e LibRegex: Avoid generating ForkJumps when jumping to the next alt block
Fixes #2398.
2024-11-17 20:12:39 +01:00
Ali Mohammad Pur
00bc22c332 LibRegex: Don't immediately ignore TempInverse in optimizer
fe46b2c141 added the reset-temp-inverse flag, but set it up so all
tempinverse ops were negated at the start of the next op; this commit
makes it so these flags actually persist for one op and not zero.

Fixes #2296.
2024-11-17 09:03:29 -05:00
Ali Mohammad Pur
dabd60180f LibRegex: Don't ignore references that weren't bound in checked blocks
Fixes #2281.
2024-11-12 10:37:57 +01:00
Timothy Flynn
93712b24bf Everywhere: Hoist the Libraries folder to the top-level 2024-11-10 12:50:45 +01:00
Andreas Kling
13d7c09125 Libraries: Move to Userland/Libraries/ 2021-01-12 12:17:46 +01:00
Lenny Maiorani
e6f907a155 AK: Simplify constructors and conversions from nullptr_t
Problem:
- Many constructors are defined as `{}` rather than using the ` =
  default` compiler-provided constructor.
- Some types provide an implicit conversion operator from `nullptr_t`
  instead of requiring the caller to default construct. This violates
  the C++ Core Guidelines suggestion to declare single-argument
  constructors explicit
  (https://isocpp.github.io/CppCoreGuidelines/CppCoreGuidelines#c46-by-default-declare-single-argument-constructors-explicit).

Solution:
- Change default constructors to use the compiler-provided default
  constructor.
- Remove implicit conversion operators from `nullptr_t` and change
  usage to enforce type consistency without conversion.
2021-01-12 09:11:45 +01:00
asynts
6fa42af567 Everywhere: Replace a bundle of dbg with dbgln.
These changes are arbitrarily divided into multiple commits to make it
easier to find potentially introduced bugs with git bisect.

The modifications in this commit were automatically made using the
following command:

    find . -name '*.h' -exec sed -i -E 's/dbg\(\) << ("[^"{]*");/dbgln\(\1\);/' {} \;
2021-01-11 21:49:29 +01:00
Sahan Fernando
fe2b8906d4 Everywhere: Fix incorrect uses of String::format and StringBuilder::appendf
These changes are arbitrarily divided into multiple commits to make it
easier to find potentially introduced bugs with git bisect.
2021-01-11 21:06:32 +01:00
asynts
938e5c7719 Everywhere: Replace a bundle of dbg with dbgln.
These changes are arbitrarily divided into multiple commits to make it
easier to find potentially introduced bugs with git bisect.Everything:

The modifications in this commit were automatically made using the
following command:

    find . -name '*.cpp' -exec sed -i -E 's/dbg\(\) << ("[^"{]*");/dbgln\(\1\);/' {} \;
2021-01-09 21:11:09 +01:00
Luke
0f66589007 Everywhere: Fix more typos 2020-12-31 01:47:41 +01:00
AnotherTest
ee1d9217aa LibRegex: Fail early if eof seen after '(?'
Fixes #4583.
2020-12-28 13:09:22 +01:00
Nathan Lanza
d1891f67ac
AK: Use direct-list-initialization for Vector::empend() (#4564)
clang trunk with -std=c++20 doesn't seem to properly look for an
aggregate initializer here when the type being constructed is a simple
aggregate (e.g. `struct Thing { int a; int b; };`). This template fails
to compile in a usage added 12/16/2020 in `AK/Trie.h`.

Both forms of initialization are supposed to call the
aggregate-initializers but direct-list-initialization delegating to
aggregate initializers is a new addition in c++20 that might not be
implemented yet.
2020-12-27 23:06:37 +01:00
AnotherTest
f12c98b29f LibRegex: Add a basic Regex<...>::replace() 2020-12-08 23:34:38 +01:00
AnotherTest
765d2977bc LibRegex: Add basic support for unicode escapes in ECMA262Parser
This parses unicode escapes (and matches them only for utf8 strings).
2020-12-06 15:38:40 +01:00
AnotherTest
86811683b0 LibRegex: Remove Lexer::slice_back() and just use StringViews 2020-12-06 15:38:40 +01:00
AnotherTest
19bf7734a4 LibRegex: Store 'String' matches inside the bytecode
Also removes an unnecessary 'length' argument (StringView has a length!)
2020-12-06 15:38:40 +01:00
AnotherTest
c1c5225b72 LibRegex: Fix assertion when parsing '(?'
Found by oss-fuzz: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=28186&q=label%3AProj-serenity
2020-12-03 10:51:56 +01:00
AnotherTest
6394720c87 LibRegex: Don't try to consume the escaped character if at EOF
Fixes assert on e.g. `new RegExp("\\")`
2020-11-30 17:45:05 +01:00
AnotherTest
8cada744df LibRegex: Give the bytecode a chance to run when there's no input
Fixes #4246
Also adds a test case.
2020-11-30 11:37:30 +01:00
AnotherTest
ab2c646d5d LibRegex: Fix OOB access in Regex debug print 2020-11-30 11:37:30 +01:00
Linus Groh
8284f87867 LibRegex: Add bounds check to Lexer::back()
If the offset is zero and we're already at the end of the lexer's input
an out of bounds read (m_source[m_position]) would occur.
Also check that the offset is not more than m_position (which should
never be the case, and would result in m_position underflowing).

Fixes #4253.
2020-11-30 11:36:24 +01:00
Linus Groh
eea7cabdbc LibRegex: Use match_ordinary_characters() in ECMA262Parser::parse_atom()
Otherwise we would only match TokenType::Char, making all of these invalid:

- /foo,bar/
- /foo\/bar/
- /foo=bar/
- /foo-bar/
- /foo:bar/

Fixes #4243.
2020-11-29 20:35:52 +01:00
AnotherTest
158fe9d9ca LibRegex: Allow syntax characters (except ']') without escapes in classes
e.g. `[:]`
2020-11-29 20:32:10 +01:00
Linus Groh
cbe4595ec2 LibRegex: Fix clang build errors 2020-11-29 09:29:26 +01:00
Linus Groh
7615649fdf LibRegex: Remove trailing newline from error_string() 2020-11-28 16:46:25 +01:00
Linus Groh
ca8e5b0791 LibRegex: Don't print error message in Regex constructor
It should be up to the caller to decide what to do when parsing of the
pattern fails.
2020-11-28 16:46:25 +01:00
AnotherTest
491e4a8a3b LibRegex: Allow '-' as the last element of a charclass
Fixes #4189.
2020-11-28 10:13:33 +01:00
AnotherTest
e2fa1b40c4 LibRegex: Allow unknown escapes in non-unicode mode (for ECMA262)
This makes regexps like `/\x/` to work as normal.
Partially deals with #4189.
2020-11-28 10:13:33 +01:00
AnotherTest
801750b95a LibRegex: Fix parsing identity escape sequences
Also fixes the propagation of default options (the previous
implementation reset them to zero before parsing...).
Partially deals with #4189.
2020-11-28 10:13:33 +01:00
AnotherTest
e83e7a03c2 LibRegex: Stop trying to read a character class when no tokens remain
e.g. in "[".
Fixes #4186.
2020-11-28 10:13:33 +01:00
AnotherTest
8ba273a2f3 LibJS: Hook up Regex<ECMA262> to RegExpObject and implement `test()'
This makes RegExpObject compile and store a Regex<ECMA262>, adds
all flag-related properties, and implements `RegExpPrototype.test()`
(complete with 'lastIndex' support) :^)
It should be noted that this only implements `test()' using the builtin
`exec()'.
2020-11-27 21:32:41 +01:00
AnotherTest
75081b2bdd LibRegex: Fix reported compare-against value in debug 2020-11-27 21:32:41 +01:00
AnotherTest
dbef2b1ee9 LibRegex: Implement an ECMA262-compatible parser
This also adds support for lookarounds and individually-negated
comparisons.
The only unimplemented part of the parser spec is the unicode stuff.
2020-11-27 21:32:41 +01:00
AnotherTest
3db8ced4c7 LibRegex: Change bytecode value type to a 64-bit value
To allow storing unicode ranges compactly; this is not utilised at the
moment, but changing this later would've been significantly more
difficult.
Also fixes a few debug logs.
2020-11-27 21:32:41 +01:00
AnotherTest
92ea9ed4a5 LibRegex: Fix greedy/reluctant modifiers in PosixExtendedParser
Also fixes the issue with assertions causing early termination when
they fail.
2020-11-27 21:32:41 +01:00
Emanuel Sprung
6add8b9c05 LibRegex: Remove backup file, remove BOM in RegexParser.cpp, run clang-format 2020-11-27 21:32:41 +01:00
Emanuel Sprung
4a630d4b63 LibRegex: Add RegexStringView wrapper to support utf8 and utf32 views 2020-11-27 21:32:41 +01:00
Emanuel Sprung
55450055d8 LibRegex: Add a regular expression library
This commit is a mix of several commits, squashed into one because the
commits before 'Move regex to own Library and fix all the broken stuff'
were not fixable in any elegant way.
The commits are listed below for "historical" purposes:

- AK: Add options/flags and Errors for regular expressions

Flags can be provided for any possible flavour by adding a new scoped enum.
Handling of flags is done by templated Options class and the overloaded
'|' and '&' operators.

- AK: Add Lexer for regular expressions

The lexer parses the input and extracts tokens needed to parse a regular
expression.

- AK: Add regex Parser and PosixExtendedParser

This patchset adds a abstract parser class that can be derived to implement
different parsers. A parser produces bytecode to be executed within the
regex matcher.

- AK: Add regex matcher

This patchset adds an regex matcher based on the principles of the T-REX VM.
The bytecode pruduced by the respective Parser is put into the matcher and
the VM will recursively execute the bytecode according to the available OpCodes.
Possible improvement: the recursion could be replaced by multi threading capabilities.

To match a Regular expression, e.g. for the Posix standard regular expression matcher
use the following API:

```
Pattern<PosixExtendedParser> pattern("^.*$");
auto result = pattern.match("Well, hello friends!\nHello World!"); // Match whole needle

EXPECT(result.count == 1);
EXPECT(result.matches.at(0).view.starts_with("Well"));
EXPECT(result.matches.at(0).view.end() == "!");

result = pattern.match("Well, hello friends!\nHello World!", PosixFlags::Multiline); // Match line by line

EXPECT(result.count == 2);
EXPECT(result.matches.at(0).view == "Well, hello friends!");
EXPECT(result.matches.at(1).view == "Hello World!");

EXPECT(pattern.has_match("Well,....")); // Just check if match without a result, which saves some resources.
```

- AK: Rework regex to work with opcodes objects

This patchsets reworks the matcher to work on a more structured base.
For that an abstract OpCode class and derived classes for the specific
OpCodes have been added. The respective opcode logic is contained in
each respective execute() method.

- AK: Add benchmark for regex

- AK: Some optimization in regex for runtime and memory

- LibRegex: Move regex to own Library and fix all the broken stuff

Now regex works again and grep utility is also in place for testing.
This commit also fixes the use of regex.h in C by making `regex_t`
an opaque (-ish) type, which makes its behaviour consistent between
C and C++ compilers.
Previously, <regex.h> would've blown C compilers up, and even if it
didn't, would've caused a leak in C code, and not in C++ code (due to
the existence of `OwnPtr` inside the struct).

To make this whole ordeal easier to deal with (for now), this pulls the
definitions of `reg*()` into LibRegex.

pros:
- The circular dependency between LibC and LibRegex is broken
- Eaiser to test (without accidentally pulling in the host's libc!)

cons:
- Using any of the regex.h functions will require the user to link -lregex
- The symbols will be missing from libc, which will be a big surprise
  down the line (especially with shared libs).

Co-Authored-By: Ali Mohammad Pur <ali.mpfard@gmail.com>
2020-11-27 21:32:41 +01:00