LibRegex: Add a regular expression library
This commit is a mix of several commits, squashed into one because the
commits before 'Move regex to own Library and fix all the broken stuff'
were not fixable in any elegant way.
The commits are listed below for "historical" purposes:
- AK: Add options/flags and Errors for regular expressions
Flags can be provided for any possible flavour by adding a new scoped enum.
Handling of flags is done by templated Options class and the overloaded
'|' and '&' operators.
- AK: Add Lexer for regular expressions
The lexer parses the input and extracts tokens needed to parse a regular
expression.
- AK: Add regex Parser and PosixExtendedParser
This patchset adds a abstract parser class that can be derived to implement
different parsers. A parser produces bytecode to be executed within the
regex matcher.
- AK: Add regex matcher
This patchset adds an regex matcher based on the principles of the T-REX VM.
The bytecode pruduced by the respective Parser is put into the matcher and
the VM will recursively execute the bytecode according to the available OpCodes.
Possible improvement: the recursion could be replaced by multi threading capabilities.
To match a Regular expression, e.g. for the Posix standard regular expression matcher
use the following API:
```
Pattern<PosixExtendedParser> pattern("^.*$");
auto result = pattern.match("Well, hello friends!\nHello World!"); // Match whole needle
EXPECT(result.count == 1);
EXPECT(result.matches.at(0).view.starts_with("Well"));
EXPECT(result.matches.at(0).view.end() == "!");
result = pattern.match("Well, hello friends!\nHello World!", PosixFlags::Multiline); // Match line by line
EXPECT(result.count == 2);
EXPECT(result.matches.at(0).view == "Well, hello friends!");
EXPECT(result.matches.at(1).view == "Hello World!");
EXPECT(pattern.has_match("Well,....")); // Just check if match without a result, which saves some resources.
```
- AK: Rework regex to work with opcodes objects
This patchsets reworks the matcher to work on a more structured base.
For that an abstract OpCode class and derived classes for the specific
OpCodes have been added. The respective opcode logic is contained in
each respective execute() method.
- AK: Add benchmark for regex
- AK: Some optimization in regex for runtime and memory
- LibRegex: Move regex to own Library and fix all the broken stuff
Now regex works again and grep utility is also in place for testing.
This commit also fixes the use of regex.h in C by making `regex_t`
an opaque (-ish) type, which makes its behaviour consistent between
C and C++ compilers.
Previously, <regex.h> would've blown C compilers up, and even if it
didn't, would've caused a leak in C code, and not in C++ code (due to
the existence of `OwnPtr` inside the struct).
To make this whole ordeal easier to deal with (for now), this pulls the
definitions of `reg*()` into LibRegex.
pros:
- The circular dependency between LibC and LibRegex is broken
- Eaiser to test (without accidentally pulling in the host's libc!)
cons:
- Using any of the regex.h functions will require the user to link -lregex
- The symbols will be missing from libc, which will be a big surprise
down the line (especially with shared libs).
Co-Authored-By: Ali Mohammad Pur <ali.mpfard@gmail.com>
2020-04-26 12:45:10 +00:00
|
|
|
/*
|
|
|
|
* Copyright (c) 2020, Emanuel Sprung <emanuel.sprung@gmail.com>
|
|
|
|
*
|
2021-04-22 08:24:48 +00:00
|
|
|
* SPDX-License-Identifier: BSD-2-Clause
|
LibRegex: Add a regular expression library
This commit is a mix of several commits, squashed into one because the
commits before 'Move regex to own Library and fix all the broken stuff'
were not fixable in any elegant way.
The commits are listed below for "historical" purposes:
- AK: Add options/flags and Errors for regular expressions
Flags can be provided for any possible flavour by adding a new scoped enum.
Handling of flags is done by templated Options class and the overloaded
'|' and '&' operators.
- AK: Add Lexer for regular expressions
The lexer parses the input and extracts tokens needed to parse a regular
expression.
- AK: Add regex Parser and PosixExtendedParser
This patchset adds a abstract parser class that can be derived to implement
different parsers. A parser produces bytecode to be executed within the
regex matcher.
- AK: Add regex matcher
This patchset adds an regex matcher based on the principles of the T-REX VM.
The bytecode pruduced by the respective Parser is put into the matcher and
the VM will recursively execute the bytecode according to the available OpCodes.
Possible improvement: the recursion could be replaced by multi threading capabilities.
To match a Regular expression, e.g. for the Posix standard regular expression matcher
use the following API:
```
Pattern<PosixExtendedParser> pattern("^.*$");
auto result = pattern.match("Well, hello friends!\nHello World!"); // Match whole needle
EXPECT(result.count == 1);
EXPECT(result.matches.at(0).view.starts_with("Well"));
EXPECT(result.matches.at(0).view.end() == "!");
result = pattern.match("Well, hello friends!\nHello World!", PosixFlags::Multiline); // Match line by line
EXPECT(result.count == 2);
EXPECT(result.matches.at(0).view == "Well, hello friends!");
EXPECT(result.matches.at(1).view == "Hello World!");
EXPECT(pattern.has_match("Well,....")); // Just check if match without a result, which saves some resources.
```
- AK: Rework regex to work with opcodes objects
This patchsets reworks the matcher to work on a more structured base.
For that an abstract OpCode class and derived classes for the specific
OpCodes have been added. The respective opcode logic is contained in
each respective execute() method.
- AK: Add benchmark for regex
- AK: Some optimization in regex for runtime and memory
- LibRegex: Move regex to own Library and fix all the broken stuff
Now regex works again and grep utility is also in place for testing.
This commit also fixes the use of regex.h in C by making `regex_t`
an opaque (-ish) type, which makes its behaviour consistent between
C and C++ compilers.
Previously, <regex.h> would've blown C compilers up, and even if it
didn't, would've caused a leak in C code, and not in C++ code (due to
the existence of `OwnPtr` inside the struct).
To make this whole ordeal easier to deal with (for now), this pulls the
definitions of `reg*()` into LibRegex.
pros:
- The circular dependency between LibC and LibRegex is broken
- Eaiser to test (without accidentally pulling in the host's libc!)
cons:
- Using any of the regex.h functions will require the user to link -lregex
- The symbols will be missing from libc, which will be a big surprise
down the line (especially with shared libs).
Co-Authored-By: Ali Mohammad Pur <ali.mpfard@gmail.com>
2020-04-26 12:45:10 +00:00
|
|
|
*/
|
|
|
|
|
|
|
|
#pragma once
|
|
|
|
|
|
|
|
#include "RegexOptions.h"
|
|
|
|
|
2021-07-17 19:54:55 +00:00
|
|
|
#include <AK/FlyString.h>
|
|
|
|
#include <AK/HashMap.h>
|
|
|
|
#include <AK/MemMem.h>
|
|
|
|
#include <AK/String.h>
|
|
|
|
#include <AK/StringBuilder.h>
|
|
|
|
#include <AK/StringView.h>
|
|
|
|
#include <AK/Utf32View.h>
|
2021-07-18 00:37:01 +00:00
|
|
|
#include <AK/Utf8View.h>
|
|
|
|
#include <AK/Variant.h>
|
2021-07-17 19:54:55 +00:00
|
|
|
#include <AK/Vector.h>
|
LibRegex: Add a regular expression library
This commit is a mix of several commits, squashed into one because the
commits before 'Move regex to own Library and fix all the broken stuff'
were not fixable in any elegant way.
The commits are listed below for "historical" purposes:
- AK: Add options/flags and Errors for regular expressions
Flags can be provided for any possible flavour by adding a new scoped enum.
Handling of flags is done by templated Options class and the overloaded
'|' and '&' operators.
- AK: Add Lexer for regular expressions
The lexer parses the input and extracts tokens needed to parse a regular
expression.
- AK: Add regex Parser and PosixExtendedParser
This patchset adds a abstract parser class that can be derived to implement
different parsers. A parser produces bytecode to be executed within the
regex matcher.
- AK: Add regex matcher
This patchset adds an regex matcher based on the principles of the T-REX VM.
The bytecode pruduced by the respective Parser is put into the matcher and
the VM will recursively execute the bytecode according to the available OpCodes.
Possible improvement: the recursion could be replaced by multi threading capabilities.
To match a Regular expression, e.g. for the Posix standard regular expression matcher
use the following API:
```
Pattern<PosixExtendedParser> pattern("^.*$");
auto result = pattern.match("Well, hello friends!\nHello World!"); // Match whole needle
EXPECT(result.count == 1);
EXPECT(result.matches.at(0).view.starts_with("Well"));
EXPECT(result.matches.at(0).view.end() == "!");
result = pattern.match("Well, hello friends!\nHello World!", PosixFlags::Multiline); // Match line by line
EXPECT(result.count == 2);
EXPECT(result.matches.at(0).view == "Well, hello friends!");
EXPECT(result.matches.at(1).view == "Hello World!");
EXPECT(pattern.has_match("Well,....")); // Just check if match without a result, which saves some resources.
```
- AK: Rework regex to work with opcodes objects
This patchsets reworks the matcher to work on a more structured base.
For that an abstract OpCode class and derived classes for the specific
OpCodes have been added. The respective opcode logic is contained in
each respective execute() method.
- AK: Add benchmark for regex
- AK: Some optimization in regex for runtime and memory
- LibRegex: Move regex to own Library and fix all the broken stuff
Now regex works again and grep utility is also in place for testing.
This commit also fixes the use of regex.h in C by making `regex_t`
an opaque (-ish) type, which makes its behaviour consistent between
C and C++ compilers.
Previously, <regex.h> would've blown C compilers up, and even if it
didn't, would've caused a leak in C code, and not in C++ code (due to
the existence of `OwnPtr` inside the struct).
To make this whole ordeal easier to deal with (for now), this pulls the
definitions of `reg*()` into LibRegex.
pros:
- The circular dependency between LibC and LibRegex is broken
- Eaiser to test (without accidentally pulling in the host's libc!)
cons:
- Using any of the regex.h functions will require the user to link -lregex
- The symbols will be missing from libc, which will be a big surprise
down the line (especially with shared libs).
Co-Authored-By: Ali Mohammad Pur <ali.mpfard@gmail.com>
2020-04-26 12:45:10 +00:00
|
|
|
|
|
|
|
namespace regex {
|
|
|
|
|
2020-06-08 22:15:09 +00:00
|
|
|
class RegexStringView {
|
|
|
|
public:
|
|
|
|
RegexStringView(const char* chars)
|
2021-07-18 00:37:01 +00:00
|
|
|
: m_view(StringView { chars })
|
2020-06-08 22:15:09 +00:00
|
|
|
{
|
|
|
|
}
|
|
|
|
|
|
|
|
RegexStringView(const String& string)
|
2021-07-18 00:37:01 +00:00
|
|
|
: m_view(string.view())
|
2020-06-08 22:15:09 +00:00
|
|
|
{
|
|
|
|
}
|
|
|
|
|
|
|
|
RegexStringView(const StringView view)
|
2021-07-18 00:37:01 +00:00
|
|
|
: m_view(view)
|
2020-06-08 22:15:09 +00:00
|
|
|
{
|
|
|
|
}
|
2021-07-18 00:37:01 +00:00
|
|
|
|
|
|
|
RegexStringView(Utf32View view)
|
|
|
|
: m_view(view)
|
|
|
|
{
|
|
|
|
}
|
|
|
|
|
|
|
|
RegexStringView(Utf8View view)
|
|
|
|
: m_view(view)
|
2020-06-08 22:15:09 +00:00
|
|
|
{
|
|
|
|
}
|
|
|
|
|
2021-07-18 00:37:01 +00:00
|
|
|
const StringView& string_view() const
|
|
|
|
{
|
|
|
|
return m_view.get<StringView>();
|
|
|
|
}
|
2020-06-08 22:15:09 +00:00
|
|
|
|
2021-07-18 00:37:01 +00:00
|
|
|
const Utf32View& u32_view() const
|
2020-06-08 22:15:09 +00:00
|
|
|
{
|
2021-07-18 00:37:01 +00:00
|
|
|
return m_view.get<Utf32View>();
|
|
|
|
}
|
2020-06-08 22:15:09 +00:00
|
|
|
|
2021-07-18 00:37:01 +00:00
|
|
|
const Utf8View& u8_view() const
|
2020-06-08 22:15:09 +00:00
|
|
|
{
|
2021-07-18 00:37:01 +00:00
|
|
|
return m_view.get<Utf8View>();
|
|
|
|
}
|
2020-06-08 22:15:09 +00:00
|
|
|
|
|
|
|
bool is_empty() const
|
|
|
|
{
|
2021-07-18 00:37:01 +00:00
|
|
|
return m_view.visit([](auto& view) { return view.is_empty(); });
|
2020-06-08 22:15:09 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
bool is_null() const
|
|
|
|
{
|
2021-07-18 00:37:01 +00:00
|
|
|
return m_view.visit([](auto& view) { return view.is_null(); });
|
2020-06-08 22:15:09 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
size_t length() const
|
|
|
|
{
|
2021-07-18 00:37:01 +00:00
|
|
|
return m_view.visit([](auto& view) { return view.length(); });
|
|
|
|
}
|
|
|
|
|
|
|
|
RegexStringView construct_as_same(Span<u32> data, Optional<String>& optional_string_storage) const
|
|
|
|
{
|
|
|
|
return m_view.visit(
|
|
|
|
[&]<typename T>(T const&) {
|
|
|
|
StringBuilder builder;
|
|
|
|
for (auto ch : data)
|
|
|
|
builder.append(ch); // Note: The type conversion is intentional.
|
|
|
|
optional_string_storage = builder.build();
|
|
|
|
return RegexStringView { T { *optional_string_storage } };
|
|
|
|
},
|
|
|
|
[&](Utf32View) {
|
|
|
|
return RegexStringView { Utf32View { data.data(), data.size() } };
|
|
|
|
});
|
2020-06-08 22:15:09 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
Vector<RegexStringView> lines() const
|
|
|
|
{
|
2021-07-18 00:37:01 +00:00
|
|
|
return m_view.visit(
|
|
|
|
[](StringView view) {
|
|
|
|
auto views = view.lines(false);
|
|
|
|
Vector<RegexStringView> new_views;
|
|
|
|
for (auto& view : views)
|
|
|
|
new_views.empend(view);
|
|
|
|
return new_views;
|
|
|
|
},
|
|
|
|
[](Utf32View view) {
|
|
|
|
Vector<RegexStringView> views;
|
|
|
|
u32 newline = '\n';
|
|
|
|
while (!view.is_empty()) {
|
|
|
|
auto position = AK::memmem_optional(view.code_points(), view.length() * sizeof(u32), &newline, sizeof(u32));
|
|
|
|
if (!position.has_value())
|
|
|
|
break;
|
|
|
|
auto offset = position.value() / sizeof(u32);
|
|
|
|
views.empend(view.substring_view(0, offset));
|
|
|
|
view = view.substring_view(offset + 1, view.length() - offset - 1);
|
|
|
|
}
|
|
|
|
if (!view.is_empty())
|
|
|
|
views.empend(view);
|
|
|
|
return views;
|
|
|
|
},
|
|
|
|
[](Utf8View& view) {
|
|
|
|
Vector<RegexStringView> views;
|
|
|
|
auto it = view.begin();
|
|
|
|
auto previous_newline_position_it = it;
|
|
|
|
for (;;) {
|
|
|
|
if (*it == '\n') {
|
|
|
|
auto previous_offset = view.byte_offset_of(previous_newline_position_it);
|
|
|
|
auto new_offset = view.byte_offset_of(it);
|
|
|
|
auto slice = view.substring_view(previous_offset, new_offset - previous_offset);
|
|
|
|
views.empend(slice);
|
|
|
|
++it;
|
|
|
|
previous_newline_position_it = it;
|
|
|
|
}
|
|
|
|
if (it.done())
|
|
|
|
break;
|
|
|
|
++it;
|
|
|
|
}
|
|
|
|
if (it != previous_newline_position_it) {
|
|
|
|
auto previous_offset = view.byte_offset_of(previous_newline_position_it);
|
|
|
|
auto new_offset = view.byte_offset_of(it);
|
|
|
|
auto slice = view.substring_view(previous_offset, new_offset - previous_offset);
|
|
|
|
views.empend(slice);
|
|
|
|
}
|
|
|
|
return views;
|
|
|
|
});
|
2020-06-08 22:15:09 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
RegexStringView substring_view(size_t offset, size_t length) const
|
|
|
|
{
|
2021-07-18 00:37:01 +00:00
|
|
|
return m_view.visit(
|
|
|
|
[&](auto view) { return RegexStringView { view.substring_view(offset, length) }; },
|
|
|
|
[&](Utf8View const& view) { return RegexStringView { view.unicode_substring_view(offset, length) }; });
|
2020-06-08 22:15:09 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
String to_string() const
|
|
|
|
{
|
2021-07-18 00:37:01 +00:00
|
|
|
return m_view.visit(
|
|
|
|
[](StringView view) { return view.to_string(); },
|
|
|
|
[](auto& view) {
|
|
|
|
StringBuilder builder;
|
|
|
|
for (auto it = view.begin(); it != view.end(); ++it)
|
|
|
|
builder.append_code_point(*it);
|
|
|
|
return builder.to_string();
|
|
|
|
});
|
2020-06-08 22:15:09 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
u32 operator[](size_t index) const
|
|
|
|
{
|
2021-07-18 00:37:01 +00:00
|
|
|
return m_view.visit(
|
|
|
|
[&](StringView view) -> u32 {
|
|
|
|
auto ch = view[index];
|
|
|
|
if (ch < 0)
|
|
|
|
return 256u + ch;
|
|
|
|
return ch;
|
|
|
|
},
|
|
|
|
[&](auto view) -> u32 { return view[index]; },
|
|
|
|
[&](Utf8View& view) -> u32 {
|
|
|
|
size_t i = index;
|
|
|
|
for (auto it = view.begin(); it != view.end(); ++it, --i) {
|
|
|
|
if (i == 0)
|
|
|
|
return *it;
|
|
|
|
}
|
|
|
|
VERIFY_NOT_REACHED();
|
|
|
|
});
|
2020-06-08 22:15:09 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
bool operator==(const char* cstring) const
|
|
|
|
{
|
2021-07-18 00:37:01 +00:00
|
|
|
return m_view.visit(
|
|
|
|
[&](Utf32View) { return to_string() == cstring; },
|
|
|
|
[&](Utf8View const& view) { return view.as_string() == cstring; },
|
|
|
|
[&](StringView view) { return view == cstring; });
|
2020-06-08 22:15:09 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
bool operator!=(const char* cstring) const
|
|
|
|
{
|
|
|
|
return !(*this == cstring);
|
|
|
|
}
|
|
|
|
|
|
|
|
bool operator==(const String& string) const
|
|
|
|
{
|
2021-07-18 00:37:01 +00:00
|
|
|
return m_view.visit(
|
|
|
|
[&](Utf32View) { return to_string() == string; },
|
|
|
|
[&](Utf8View const& view) { return view.as_string() == string; },
|
|
|
|
[&](StringView view) { return view == string; });
|
2020-06-08 22:15:09 +00:00
|
|
|
}
|
|
|
|
|
2021-07-18 00:37:01 +00:00
|
|
|
bool operator==(const StringView& string) const
|
2020-06-08 22:15:09 +00:00
|
|
|
{
|
2021-07-18 00:37:01 +00:00
|
|
|
return m_view.visit(
|
|
|
|
[&](Utf32View) { return to_string() == string; },
|
|
|
|
[&](Utf8View const& view) { return view.as_string() == string; },
|
|
|
|
[&](StringView view) { return view == string; });
|
2020-06-08 22:15:09 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
bool operator!=(const StringView& other) const
|
|
|
|
{
|
|
|
|
return !(*this == other);
|
|
|
|
}
|
|
|
|
|
|
|
|
bool operator==(const Utf32View& other) const
|
|
|
|
{
|
2021-07-18 00:37:01 +00:00
|
|
|
return m_view.visit(
|
|
|
|
[&](Utf32View view) {
|
|
|
|
return view.length() == other.length() && __builtin_memcmp(view.code_points(), other.code_points(), view.length() * sizeof(u32)) == 0;
|
|
|
|
},
|
|
|
|
[&](Utf8View const& view) { return view.as_string() == RegexStringView { other }.to_string(); },
|
|
|
|
[&](StringView view) { return view == RegexStringView { other }.to_string(); });
|
2020-06-08 22:15:09 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
bool operator!=(const Utf32View& other) const
|
|
|
|
{
|
|
|
|
return !(*this == other);
|
|
|
|
}
|
|
|
|
|
2021-07-18 00:37:01 +00:00
|
|
|
bool operator==(const Utf8View& other) const
|
|
|
|
{
|
|
|
|
return m_view.visit(
|
|
|
|
[&](Utf32View) {
|
|
|
|
return to_string() == other.as_string();
|
|
|
|
},
|
|
|
|
[&](Utf8View const& view) { return view.as_string() == other.as_string(); },
|
|
|
|
[&](StringView view) { return other.as_string() == view; });
|
|
|
|
}
|
|
|
|
|
|
|
|
bool operator!=(const Utf8View& other) const
|
|
|
|
{
|
|
|
|
return !(*this == other);
|
|
|
|
}
|
|
|
|
|
|
|
|
bool equals(const RegexStringView& other) const
|
2020-06-08 22:15:09 +00:00
|
|
|
{
|
2021-07-18 00:37:01 +00:00
|
|
|
return other.m_view.visit([&](auto const& view) { return operator==(view); });
|
|
|
|
}
|
2020-06-08 22:15:09 +00:00
|
|
|
|
2021-07-18 00:37:01 +00:00
|
|
|
bool equals_ignoring_case(const RegexStringView& other) const
|
|
|
|
{
|
|
|
|
// FIXME: Implement equals_ignoring_case() for unicode.
|
|
|
|
return m_view.visit(
|
|
|
|
[&](StringView view) {
|
|
|
|
return other.m_view.visit(
|
|
|
|
[&](StringView other_view) { return view.equals_ignoring_case(other_view); },
|
|
|
|
[](auto&) -> bool { TODO(); });
|
|
|
|
},
|
|
|
|
[](auto&) -> bool { TODO(); });
|
2020-06-08 22:15:09 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
bool starts_with(const StringView& str) const
|
|
|
|
{
|
2021-07-18 00:37:01 +00:00
|
|
|
return m_view.visit(
|
|
|
|
[&](Utf32View) -> bool {
|
|
|
|
TODO();
|
|
|
|
},
|
|
|
|
[&](Utf8View const& view) { return view.as_string().starts_with(str); },
|
|
|
|
[&](StringView view) { return view.starts_with(str); });
|
2020-06-08 22:15:09 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
bool starts_with(const Utf32View& str) const
|
|
|
|
{
|
2021-07-18 00:37:01 +00:00
|
|
|
return m_view.visit(
|
|
|
|
[&](Utf32View view) -> bool {
|
|
|
|
if (str.length() > view.length())
|
|
|
|
return false;
|
|
|
|
if (str.length() == view.length())
|
|
|
|
return operator==(str);
|
|
|
|
for (size_t i = 0; i < str.length(); ++i) {
|
|
|
|
if (str.at(i) != view.at(i))
|
|
|
|
return false;
|
|
|
|
}
|
|
|
|
return true;
|
|
|
|
},
|
|
|
|
[&](Utf8View const& view) {
|
|
|
|
auto it = view.begin();
|
|
|
|
for (auto code_point : str) {
|
|
|
|
if (it.done())
|
|
|
|
return false;
|
|
|
|
if (code_point != *it)
|
|
|
|
return false;
|
|
|
|
++it;
|
|
|
|
}
|
|
|
|
return true;
|
|
|
|
},
|
|
|
|
[&](StringView) -> bool { TODO(); });
|
2020-06-08 22:15:09 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
private:
|
2021-07-18 00:37:01 +00:00
|
|
|
Variant<StringView, Utf8View, Utf32View> m_view;
|
2020-06-08 22:15:09 +00:00
|
|
|
};
|
|
|
|
|
LibRegex: Add a regular expression library
This commit is a mix of several commits, squashed into one because the
commits before 'Move regex to own Library and fix all the broken stuff'
were not fixable in any elegant way.
The commits are listed below for "historical" purposes:
- AK: Add options/flags and Errors for regular expressions
Flags can be provided for any possible flavour by adding a new scoped enum.
Handling of flags is done by templated Options class and the overloaded
'|' and '&' operators.
- AK: Add Lexer for regular expressions
The lexer parses the input and extracts tokens needed to parse a regular
expression.
- AK: Add regex Parser and PosixExtendedParser
This patchset adds a abstract parser class that can be derived to implement
different parsers. A parser produces bytecode to be executed within the
regex matcher.
- AK: Add regex matcher
This patchset adds an regex matcher based on the principles of the T-REX VM.
The bytecode pruduced by the respective Parser is put into the matcher and
the VM will recursively execute the bytecode according to the available OpCodes.
Possible improvement: the recursion could be replaced by multi threading capabilities.
To match a Regular expression, e.g. for the Posix standard regular expression matcher
use the following API:
```
Pattern<PosixExtendedParser> pattern("^.*$");
auto result = pattern.match("Well, hello friends!\nHello World!"); // Match whole needle
EXPECT(result.count == 1);
EXPECT(result.matches.at(0).view.starts_with("Well"));
EXPECT(result.matches.at(0).view.end() == "!");
result = pattern.match("Well, hello friends!\nHello World!", PosixFlags::Multiline); // Match line by line
EXPECT(result.count == 2);
EXPECT(result.matches.at(0).view == "Well, hello friends!");
EXPECT(result.matches.at(1).view == "Hello World!");
EXPECT(pattern.has_match("Well,....")); // Just check if match without a result, which saves some resources.
```
- AK: Rework regex to work with opcodes objects
This patchsets reworks the matcher to work on a more structured base.
For that an abstract OpCode class and derived classes for the specific
OpCodes have been added. The respective opcode logic is contained in
each respective execute() method.
- AK: Add benchmark for regex
- AK: Some optimization in regex for runtime and memory
- LibRegex: Move regex to own Library and fix all the broken stuff
Now regex works again and grep utility is also in place for testing.
This commit also fixes the use of regex.h in C by making `regex_t`
an opaque (-ish) type, which makes its behaviour consistent between
C and C++ compilers.
Previously, <regex.h> would've blown C compilers up, and even if it
didn't, would've caused a leak in C code, and not in C++ code (due to
the existence of `OwnPtr` inside the struct).
To make this whole ordeal easier to deal with (for now), this pulls the
definitions of `reg*()` into LibRegex.
pros:
- The circular dependency between LibC and LibRegex is broken
- Eaiser to test (without accidentally pulling in the host's libc!)
cons:
- Using any of the regex.h functions will require the user to link -lregex
- The symbols will be missing from libc, which will be a big surprise
down the line (especially with shared libs).
Co-Authored-By: Ali Mohammad Pur <ali.mpfard@gmail.com>
2020-04-26 12:45:10 +00:00
|
|
|
class Match final {
|
|
|
|
private:
|
|
|
|
Optional<FlyString> string;
|
|
|
|
|
|
|
|
public:
|
|
|
|
Match() = default;
|
|
|
|
~Match() = default;
|
|
|
|
|
2020-06-08 22:15:09 +00:00
|
|
|
Match(const RegexStringView view_, const size_t line_, const size_t column_, const size_t global_offset_)
|
LibRegex: Add a regular expression library
This commit is a mix of several commits, squashed into one because the
commits before 'Move regex to own Library and fix all the broken stuff'
were not fixable in any elegant way.
The commits are listed below for "historical" purposes:
- AK: Add options/flags and Errors for regular expressions
Flags can be provided for any possible flavour by adding a new scoped enum.
Handling of flags is done by templated Options class and the overloaded
'|' and '&' operators.
- AK: Add Lexer for regular expressions
The lexer parses the input and extracts tokens needed to parse a regular
expression.
- AK: Add regex Parser and PosixExtendedParser
This patchset adds a abstract parser class that can be derived to implement
different parsers. A parser produces bytecode to be executed within the
regex matcher.
- AK: Add regex matcher
This patchset adds an regex matcher based on the principles of the T-REX VM.
The bytecode pruduced by the respective Parser is put into the matcher and
the VM will recursively execute the bytecode according to the available OpCodes.
Possible improvement: the recursion could be replaced by multi threading capabilities.
To match a Regular expression, e.g. for the Posix standard regular expression matcher
use the following API:
```
Pattern<PosixExtendedParser> pattern("^.*$");
auto result = pattern.match("Well, hello friends!\nHello World!"); // Match whole needle
EXPECT(result.count == 1);
EXPECT(result.matches.at(0).view.starts_with("Well"));
EXPECT(result.matches.at(0).view.end() == "!");
result = pattern.match("Well, hello friends!\nHello World!", PosixFlags::Multiline); // Match line by line
EXPECT(result.count == 2);
EXPECT(result.matches.at(0).view == "Well, hello friends!");
EXPECT(result.matches.at(1).view == "Hello World!");
EXPECT(pattern.has_match("Well,....")); // Just check if match without a result, which saves some resources.
```
- AK: Rework regex to work with opcodes objects
This patchsets reworks the matcher to work on a more structured base.
For that an abstract OpCode class and derived classes for the specific
OpCodes have been added. The respective opcode logic is contained in
each respective execute() method.
- AK: Add benchmark for regex
- AK: Some optimization in regex for runtime and memory
- LibRegex: Move regex to own Library and fix all the broken stuff
Now regex works again and grep utility is also in place for testing.
This commit also fixes the use of regex.h in C by making `regex_t`
an opaque (-ish) type, which makes its behaviour consistent between
C and C++ compilers.
Previously, <regex.h> would've blown C compilers up, and even if it
didn't, would've caused a leak in C code, and not in C++ code (due to
the existence of `OwnPtr` inside the struct).
To make this whole ordeal easier to deal with (for now), this pulls the
definitions of `reg*()` into LibRegex.
pros:
- The circular dependency between LibC and LibRegex is broken
- Eaiser to test (without accidentally pulling in the host's libc!)
cons:
- Using any of the regex.h functions will require the user to link -lregex
- The symbols will be missing from libc, which will be a big surprise
down the line (especially with shared libs).
Co-Authored-By: Ali Mohammad Pur <ali.mpfard@gmail.com>
2020-04-26 12:45:10 +00:00
|
|
|
: view(view_)
|
|
|
|
, line(line_)
|
|
|
|
, column(column_)
|
|
|
|
, global_offset(global_offset_)
|
|
|
|
, left_column(column_)
|
|
|
|
{
|
|
|
|
}
|
|
|
|
|
|
|
|
Match(const String string_, const size_t line_, const size_t column_, const size_t global_offset_)
|
|
|
|
: string(string_)
|
|
|
|
, view(string.value().view())
|
|
|
|
, line(line_)
|
|
|
|
, column(column_)
|
|
|
|
, global_offset(global_offset_)
|
|
|
|
, left_column(column_)
|
|
|
|
{
|
|
|
|
}
|
|
|
|
|
2020-06-08 22:15:09 +00:00
|
|
|
RegexStringView view { nullptr };
|
LibRegex: Add a regular expression library
This commit is a mix of several commits, squashed into one because the
commits before 'Move regex to own Library and fix all the broken stuff'
were not fixable in any elegant way.
The commits are listed below for "historical" purposes:
- AK: Add options/flags and Errors for regular expressions
Flags can be provided for any possible flavour by adding a new scoped enum.
Handling of flags is done by templated Options class and the overloaded
'|' and '&' operators.
- AK: Add Lexer for regular expressions
The lexer parses the input and extracts tokens needed to parse a regular
expression.
- AK: Add regex Parser and PosixExtendedParser
This patchset adds a abstract parser class that can be derived to implement
different parsers. A parser produces bytecode to be executed within the
regex matcher.
- AK: Add regex matcher
This patchset adds an regex matcher based on the principles of the T-REX VM.
The bytecode pruduced by the respective Parser is put into the matcher and
the VM will recursively execute the bytecode according to the available OpCodes.
Possible improvement: the recursion could be replaced by multi threading capabilities.
To match a Regular expression, e.g. for the Posix standard regular expression matcher
use the following API:
```
Pattern<PosixExtendedParser> pattern("^.*$");
auto result = pattern.match("Well, hello friends!\nHello World!"); // Match whole needle
EXPECT(result.count == 1);
EXPECT(result.matches.at(0).view.starts_with("Well"));
EXPECT(result.matches.at(0).view.end() == "!");
result = pattern.match("Well, hello friends!\nHello World!", PosixFlags::Multiline); // Match line by line
EXPECT(result.count == 2);
EXPECT(result.matches.at(0).view == "Well, hello friends!");
EXPECT(result.matches.at(1).view == "Hello World!");
EXPECT(pattern.has_match("Well,....")); // Just check if match without a result, which saves some resources.
```
- AK: Rework regex to work with opcodes objects
This patchsets reworks the matcher to work on a more structured base.
For that an abstract OpCode class and derived classes for the specific
OpCodes have been added. The respective opcode logic is contained in
each respective execute() method.
- AK: Add benchmark for regex
- AK: Some optimization in regex for runtime and memory
- LibRegex: Move regex to own Library and fix all the broken stuff
Now regex works again and grep utility is also in place for testing.
This commit also fixes the use of regex.h in C by making `regex_t`
an opaque (-ish) type, which makes its behaviour consistent between
C and C++ compilers.
Previously, <regex.h> would've blown C compilers up, and even if it
didn't, would've caused a leak in C code, and not in C++ code (due to
the existence of `OwnPtr` inside the struct).
To make this whole ordeal easier to deal with (for now), this pulls the
definitions of `reg*()` into LibRegex.
pros:
- The circular dependency between LibC and LibRegex is broken
- Eaiser to test (without accidentally pulling in the host's libc!)
cons:
- Using any of the regex.h functions will require the user to link -lregex
- The symbols will be missing from libc, which will be a big surprise
down the line (especially with shared libs).
Co-Authored-By: Ali Mohammad Pur <ali.mpfard@gmail.com>
2020-04-26 12:45:10 +00:00
|
|
|
size_t line { 0 };
|
|
|
|
size_t column { 0 };
|
|
|
|
size_t global_offset { 0 };
|
|
|
|
|
|
|
|
// ugly, as not usable by user, but needed to prevent to create extra vectors that are
|
|
|
|
// able to store the column when the left paren has been found
|
|
|
|
size_t left_column { 0 };
|
|
|
|
};
|
|
|
|
|
|
|
|
struct MatchInput {
|
2020-06-08 22:15:09 +00:00
|
|
|
RegexStringView view { nullptr };
|
LibRegex: Add a regular expression library
This commit is a mix of several commits, squashed into one because the
commits before 'Move regex to own Library and fix all the broken stuff'
were not fixable in any elegant way.
The commits are listed below for "historical" purposes:
- AK: Add options/flags and Errors for regular expressions
Flags can be provided for any possible flavour by adding a new scoped enum.
Handling of flags is done by templated Options class and the overloaded
'|' and '&' operators.
- AK: Add Lexer for regular expressions
The lexer parses the input and extracts tokens needed to parse a regular
expression.
- AK: Add regex Parser and PosixExtendedParser
This patchset adds a abstract parser class that can be derived to implement
different parsers. A parser produces bytecode to be executed within the
regex matcher.
- AK: Add regex matcher
This patchset adds an regex matcher based on the principles of the T-REX VM.
The bytecode pruduced by the respective Parser is put into the matcher and
the VM will recursively execute the bytecode according to the available OpCodes.
Possible improvement: the recursion could be replaced by multi threading capabilities.
To match a Regular expression, e.g. for the Posix standard regular expression matcher
use the following API:
```
Pattern<PosixExtendedParser> pattern("^.*$");
auto result = pattern.match("Well, hello friends!\nHello World!"); // Match whole needle
EXPECT(result.count == 1);
EXPECT(result.matches.at(0).view.starts_with("Well"));
EXPECT(result.matches.at(0).view.end() == "!");
result = pattern.match("Well, hello friends!\nHello World!", PosixFlags::Multiline); // Match line by line
EXPECT(result.count == 2);
EXPECT(result.matches.at(0).view == "Well, hello friends!");
EXPECT(result.matches.at(1).view == "Hello World!");
EXPECT(pattern.has_match("Well,....")); // Just check if match without a result, which saves some resources.
```
- AK: Rework regex to work with opcodes objects
This patchsets reworks the matcher to work on a more structured base.
For that an abstract OpCode class and derived classes for the specific
OpCodes have been added. The respective opcode logic is contained in
each respective execute() method.
- AK: Add benchmark for regex
- AK: Some optimization in regex for runtime and memory
- LibRegex: Move regex to own Library and fix all the broken stuff
Now regex works again and grep utility is also in place for testing.
This commit also fixes the use of regex.h in C by making `regex_t`
an opaque (-ish) type, which makes its behaviour consistent between
C and C++ compilers.
Previously, <regex.h> would've blown C compilers up, and even if it
didn't, would've caused a leak in C code, and not in C++ code (due to
the existence of `OwnPtr` inside the struct).
To make this whole ordeal easier to deal with (for now), this pulls the
definitions of `reg*()` into LibRegex.
pros:
- The circular dependency between LibC and LibRegex is broken
- Eaiser to test (without accidentally pulling in the host's libc!)
cons:
- Using any of the regex.h functions will require the user to link -lregex
- The symbols will be missing from libc, which will be a big surprise
down the line (especially with shared libs).
Co-Authored-By: Ali Mohammad Pur <ali.mpfard@gmail.com>
2020-04-26 12:45:10 +00:00
|
|
|
AllOptions regex_options {};
|
2020-11-18 22:20:00 +00:00
|
|
|
size_t start_offset { 0 }; // For Stateful matches, saved and restored from Regex::start_offset.
|
LibRegex: Add a regular expression library
This commit is a mix of several commits, squashed into one because the
commits before 'Move regex to own Library and fix all the broken stuff'
were not fixable in any elegant way.
The commits are listed below for "historical" purposes:
- AK: Add options/flags and Errors for regular expressions
Flags can be provided for any possible flavour by adding a new scoped enum.
Handling of flags is done by templated Options class and the overloaded
'|' and '&' operators.
- AK: Add Lexer for regular expressions
The lexer parses the input and extracts tokens needed to parse a regular
expression.
- AK: Add regex Parser and PosixExtendedParser
This patchset adds a abstract parser class that can be derived to implement
different parsers. A parser produces bytecode to be executed within the
regex matcher.
- AK: Add regex matcher
This patchset adds an regex matcher based on the principles of the T-REX VM.
The bytecode pruduced by the respective Parser is put into the matcher and
the VM will recursively execute the bytecode according to the available OpCodes.
Possible improvement: the recursion could be replaced by multi threading capabilities.
To match a Regular expression, e.g. for the Posix standard regular expression matcher
use the following API:
```
Pattern<PosixExtendedParser> pattern("^.*$");
auto result = pattern.match("Well, hello friends!\nHello World!"); // Match whole needle
EXPECT(result.count == 1);
EXPECT(result.matches.at(0).view.starts_with("Well"));
EXPECT(result.matches.at(0).view.end() == "!");
result = pattern.match("Well, hello friends!\nHello World!", PosixFlags::Multiline); // Match line by line
EXPECT(result.count == 2);
EXPECT(result.matches.at(0).view == "Well, hello friends!");
EXPECT(result.matches.at(1).view == "Hello World!");
EXPECT(pattern.has_match("Well,....")); // Just check if match without a result, which saves some resources.
```
- AK: Rework regex to work with opcodes objects
This patchsets reworks the matcher to work on a more structured base.
For that an abstract OpCode class and derived classes for the specific
OpCodes have been added. The respective opcode logic is contained in
each respective execute() method.
- AK: Add benchmark for regex
- AK: Some optimization in regex for runtime and memory
- LibRegex: Move regex to own Library and fix all the broken stuff
Now regex works again and grep utility is also in place for testing.
This commit also fixes the use of regex.h in C by making `regex_t`
an opaque (-ish) type, which makes its behaviour consistent between
C and C++ compilers.
Previously, <regex.h> would've blown C compilers up, and even if it
didn't, would've caused a leak in C code, and not in C++ code (due to
the existence of `OwnPtr` inside the struct).
To make this whole ordeal easier to deal with (for now), this pulls the
definitions of `reg*()` into LibRegex.
pros:
- The circular dependency between LibC and LibRegex is broken
- Eaiser to test (without accidentally pulling in the host's libc!)
cons:
- Using any of the regex.h functions will require the user to link -lregex
- The symbols will be missing from libc, which will be a big surprise
down the line (especially with shared libs).
Co-Authored-By: Ali Mohammad Pur <ali.mpfard@gmail.com>
2020-04-26 12:45:10 +00:00
|
|
|
|
|
|
|
size_t match_index { 0 };
|
|
|
|
size_t line { 0 };
|
|
|
|
size_t column { 0 };
|
|
|
|
|
2020-06-08 22:15:09 +00:00
|
|
|
size_t global_offset { 0 }; // For multiline matching, knowing the offset from start could be important
|
2020-11-27 16:03:53 +00:00
|
|
|
|
|
|
|
mutable size_t fail_counter { 0 };
|
|
|
|
mutable Vector<size_t> saved_positions;
|
LibRegex: Add a regular expression library
This commit is a mix of several commits, squashed into one because the
commits before 'Move regex to own Library and fix all the broken stuff'
were not fixable in any elegant way.
The commits are listed below for "historical" purposes:
- AK: Add options/flags and Errors for regular expressions
Flags can be provided for any possible flavour by adding a new scoped enum.
Handling of flags is done by templated Options class and the overloaded
'|' and '&' operators.
- AK: Add Lexer for regular expressions
The lexer parses the input and extracts tokens needed to parse a regular
expression.
- AK: Add regex Parser and PosixExtendedParser
This patchset adds a abstract parser class that can be derived to implement
different parsers. A parser produces bytecode to be executed within the
regex matcher.
- AK: Add regex matcher
This patchset adds an regex matcher based on the principles of the T-REX VM.
The bytecode pruduced by the respective Parser is put into the matcher and
the VM will recursively execute the bytecode according to the available OpCodes.
Possible improvement: the recursion could be replaced by multi threading capabilities.
To match a Regular expression, e.g. for the Posix standard regular expression matcher
use the following API:
```
Pattern<PosixExtendedParser> pattern("^.*$");
auto result = pattern.match("Well, hello friends!\nHello World!"); // Match whole needle
EXPECT(result.count == 1);
EXPECT(result.matches.at(0).view.starts_with("Well"));
EXPECT(result.matches.at(0).view.end() == "!");
result = pattern.match("Well, hello friends!\nHello World!", PosixFlags::Multiline); // Match line by line
EXPECT(result.count == 2);
EXPECT(result.matches.at(0).view == "Well, hello friends!");
EXPECT(result.matches.at(1).view == "Hello World!");
EXPECT(pattern.has_match("Well,....")); // Just check if match without a result, which saves some resources.
```
- AK: Rework regex to work with opcodes objects
This patchsets reworks the matcher to work on a more structured base.
For that an abstract OpCode class and derived classes for the specific
OpCodes have been added. The respective opcode logic is contained in
each respective execute() method.
- AK: Add benchmark for regex
- AK: Some optimization in regex for runtime and memory
- LibRegex: Move regex to own Library and fix all the broken stuff
Now regex works again and grep utility is also in place for testing.
This commit also fixes the use of regex.h in C by making `regex_t`
an opaque (-ish) type, which makes its behaviour consistent between
C and C++ compilers.
Previously, <regex.h> would've blown C compilers up, and even if it
didn't, would've caused a leak in C code, and not in C++ code (due to
the existence of `OwnPtr` inside the struct).
To make this whole ordeal easier to deal with (for now), this pulls the
definitions of `reg*()` into LibRegex.
pros:
- The circular dependency between LibC and LibRegex is broken
- Eaiser to test (without accidentally pulling in the host's libc!)
cons:
- Using any of the regex.h functions will require the user to link -lregex
- The symbols will be missing from libc, which will be a big surprise
down the line (especially with shared libs).
Co-Authored-By: Ali Mohammad Pur <ali.mpfard@gmail.com>
2020-04-26 12:45:10 +00:00
|
|
|
};
|
|
|
|
|
|
|
|
struct MatchState {
|
2021-06-16 10:14:12 +00:00
|
|
|
size_t string_position_before_match { 0 };
|
LibRegex: Add a regular expression library
This commit is a mix of several commits, squashed into one because the
commits before 'Move regex to own Library and fix all the broken stuff'
were not fixable in any elegant way.
The commits are listed below for "historical" purposes:
- AK: Add options/flags and Errors for regular expressions
Flags can be provided for any possible flavour by adding a new scoped enum.
Handling of flags is done by templated Options class and the overloaded
'|' and '&' operators.
- AK: Add Lexer for regular expressions
The lexer parses the input and extracts tokens needed to parse a regular
expression.
- AK: Add regex Parser and PosixExtendedParser
This patchset adds a abstract parser class that can be derived to implement
different parsers. A parser produces bytecode to be executed within the
regex matcher.
- AK: Add regex matcher
This patchset adds an regex matcher based on the principles of the T-REX VM.
The bytecode pruduced by the respective Parser is put into the matcher and
the VM will recursively execute the bytecode according to the available OpCodes.
Possible improvement: the recursion could be replaced by multi threading capabilities.
To match a Regular expression, e.g. for the Posix standard regular expression matcher
use the following API:
```
Pattern<PosixExtendedParser> pattern("^.*$");
auto result = pattern.match("Well, hello friends!\nHello World!"); // Match whole needle
EXPECT(result.count == 1);
EXPECT(result.matches.at(0).view.starts_with("Well"));
EXPECT(result.matches.at(0).view.end() == "!");
result = pattern.match("Well, hello friends!\nHello World!", PosixFlags::Multiline); // Match line by line
EXPECT(result.count == 2);
EXPECT(result.matches.at(0).view == "Well, hello friends!");
EXPECT(result.matches.at(1).view == "Hello World!");
EXPECT(pattern.has_match("Well,....")); // Just check if match without a result, which saves some resources.
```
- AK: Rework regex to work with opcodes objects
This patchsets reworks the matcher to work on a more structured base.
For that an abstract OpCode class and derived classes for the specific
OpCodes have been added. The respective opcode logic is contained in
each respective execute() method.
- AK: Add benchmark for regex
- AK: Some optimization in regex for runtime and memory
- LibRegex: Move regex to own Library and fix all the broken stuff
Now regex works again and grep utility is also in place for testing.
This commit also fixes the use of regex.h in C by making `regex_t`
an opaque (-ish) type, which makes its behaviour consistent between
C and C++ compilers.
Previously, <regex.h> would've blown C compilers up, and even if it
didn't, would've caused a leak in C code, and not in C++ code (due to
the existence of `OwnPtr` inside the struct).
To make this whole ordeal easier to deal with (for now), this pulls the
definitions of `reg*()` into LibRegex.
pros:
- The circular dependency between LibC and LibRegex is broken
- Eaiser to test (without accidentally pulling in the host's libc!)
cons:
- Using any of the regex.h functions will require the user to link -lregex
- The symbols will be missing from libc, which will be a big surprise
down the line (especially with shared libs).
Co-Authored-By: Ali Mohammad Pur <ali.mpfard@gmail.com>
2020-04-26 12:45:10 +00:00
|
|
|
size_t string_position { 0 };
|
|
|
|
size_t instruction_position { 0 };
|
|
|
|
size_t fork_at_position { 0 };
|
2021-07-18 00:37:01 +00:00
|
|
|
Vector<Match> matches;
|
|
|
|
Vector<Vector<Match>> capture_group_matches;
|
|
|
|
Vector<HashMap<String, Match>> named_capture_group_matches;
|
LibRegex: Add a regular expression library
This commit is a mix of several commits, squashed into one because the
commits before 'Move regex to own Library and fix all the broken stuff'
were not fixable in any elegant way.
The commits are listed below for "historical" purposes:
- AK: Add options/flags and Errors for regular expressions
Flags can be provided for any possible flavour by adding a new scoped enum.
Handling of flags is done by templated Options class and the overloaded
'|' and '&' operators.
- AK: Add Lexer for regular expressions
The lexer parses the input and extracts tokens needed to parse a regular
expression.
- AK: Add regex Parser and PosixExtendedParser
This patchset adds a abstract parser class that can be derived to implement
different parsers. A parser produces bytecode to be executed within the
regex matcher.
- AK: Add regex matcher
This patchset adds an regex matcher based on the principles of the T-REX VM.
The bytecode pruduced by the respective Parser is put into the matcher and
the VM will recursively execute the bytecode according to the available OpCodes.
Possible improvement: the recursion could be replaced by multi threading capabilities.
To match a Regular expression, e.g. for the Posix standard regular expression matcher
use the following API:
```
Pattern<PosixExtendedParser> pattern("^.*$");
auto result = pattern.match("Well, hello friends!\nHello World!"); // Match whole needle
EXPECT(result.count == 1);
EXPECT(result.matches.at(0).view.starts_with("Well"));
EXPECT(result.matches.at(0).view.end() == "!");
result = pattern.match("Well, hello friends!\nHello World!", PosixFlags::Multiline); // Match line by line
EXPECT(result.count == 2);
EXPECT(result.matches.at(0).view == "Well, hello friends!");
EXPECT(result.matches.at(1).view == "Hello World!");
EXPECT(pattern.has_match("Well,....")); // Just check if match without a result, which saves some resources.
```
- AK: Rework regex to work with opcodes objects
This patchsets reworks the matcher to work on a more structured base.
For that an abstract OpCode class and derived classes for the specific
OpCodes have been added. The respective opcode logic is contained in
each respective execute() method.
- AK: Add benchmark for regex
- AK: Some optimization in regex for runtime and memory
- LibRegex: Move regex to own Library and fix all the broken stuff
Now regex works again and grep utility is also in place for testing.
This commit also fixes the use of regex.h in C by making `regex_t`
an opaque (-ish) type, which makes its behaviour consistent between
C and C++ compilers.
Previously, <regex.h> would've blown C compilers up, and even if it
didn't, would've caused a leak in C code, and not in C++ code (due to
the existence of `OwnPtr` inside the struct).
To make this whole ordeal easier to deal with (for now), this pulls the
definitions of `reg*()` into LibRegex.
pros:
- The circular dependency between LibC and LibRegex is broken
- Eaiser to test (without accidentally pulling in the host's libc!)
cons:
- Using any of the regex.h functions will require the user to link -lregex
- The symbols will be missing from libc, which will be a big surprise
down the line (especially with shared libs).
Co-Authored-By: Ali Mohammad Pur <ali.mpfard@gmail.com>
2020-04-26 12:45:10 +00:00
|
|
|
};
|
|
|
|
|
|
|
|
struct MatchOutput {
|
|
|
|
size_t operations;
|
|
|
|
Vector<Match> matches;
|
|
|
|
Vector<Vector<Match>> capture_group_matches;
|
|
|
|
Vector<HashMap<String, Match>> named_capture_group_matches;
|
|
|
|
};
|
|
|
|
|
|
|
|
}
|
2020-06-08 22:15:09 +00:00
|
|
|
|
|
|
|
using regex::RegexStringView;
|
2021-01-17 15:06:30 +00:00
|
|
|
|
|
|
|
template<>
|
|
|
|
struct AK::Formatter<regex::RegexStringView> : Formatter<StringView> {
|
|
|
|
void format(FormatBuilder& builder, const regex::RegexStringView& value)
|
|
|
|
{
|
2021-07-18 00:37:01 +00:00
|
|
|
auto string = value.to_string();
|
|
|
|
return Formatter<StringView>::format(builder, string);
|
2021-01-17 15:06:30 +00:00
|
|
|
}
|
|
|
|
};
|