MAX_GENERATED_VALUES_PER_TEST is now the --randomized_runs flag:
$ ./Build/lagom/bin/TestGenerator --randomized_runs 1000
It's sometimes useful to try larger numbers for it instead of the
default of 100.
MAX_GEN_ATTEMPTS_PER_VALUE is now a constexpr. It's not usually needed
to tweak this value; we can recompile with a different value on the rare
occasion.
REJECT and ASSUME are useful for filtering out unwanted generated
values. While this is not ideal, it is ocassionally useful and so we
include it for convenience.
The main loop of RANDOMIZED_TEST_CASE runs the test case 100 times, each
time trying to generate a different set of values. Inside that loop, if
it sees a REJECT (ASSUME is implemented in terms of REJECT), it retries
up to 15 times before giving up (perhaps it's impossible or just very
improbable to generate a value that will survive REJECT or ASSUME).
REJECT("Reason for rejecting") will just outright fail, while
ASSUME(bool) is more of an equivalent of a .filter() method from
functional languages.
This will be very useful as we add the randomized test cases and their
two loops ("generate+test many times" and "shrink once failure is
found"), because without this failure reporting we'd get many FAIL error
messages while still searching for the minimal one.
So, inside randomized test cases we want to only turn the error
reporting on for one last time after all the generating and shrinking.
This will be a foundational part of bootstrapping generators: this is
the way they'll get prerecorded values from / record random values into
RandomRuns. (Generators don't get in contact with RandomRuns
themselves, they just interact with the RandomnessSource.)
This will be used in the randomized tests a lot more than it is in the
unit tests / benchmarks; randomized tests will run the test function
multiple times, check the result and optionally start shrinking the
failing input. Generators will also be able to fail, resulting in some
of the new TestResult variants.
This class had slightly confusing semantics and the added weirdness
doesn't seem worth it just so we can say "." instead of "->" when
iterating over a vector of NNRPs.
This patch replaces NonnullRefPtrVector<T> with Vector<NNRP<T>>.
This is not guaranteed to always work correctly as ArgsParser deals in
StringViews and might have a non-properly-null-terminated string as a
value. As a bonus, using StringView (and DeprecatedString where
necessary) leads to nicer looking code too :^)
This commit moves the implementation of getopt into AK, and converts its
API to understand and use StringView instead of char*.
Everything else is caught in the crossfire of making
Option::accept_value() take a StringView instead of a char const*.
With this, we must now pass a Span<StringView> to ArgsParser::parse(),
applications using LibMain are unaffected, but anything not using that
or taking its own argc/argv has to construct a Vector<StringView> for
this method.
We have a new, improved string type coming up in AK (OOM aware, no null
state), and while it's going to use UTF-8, the name UTF8String is a
mouthful - so let's free up the String name by renaming the existing
class.
Making the old one have an annoying name will hopefully also help with
quick adoption :^)
As many macros as possible are moved to Macros.h, while the
macros to create a test case are moved to TestCase.h. TestCase is now
the only user-facing header for creating a test case. TestSuite and its
helpers have moved into a .cpp file. Instead of requiring a TEST_MAIN
macro to be instantiated into the test file, a TestMain.cpp file is
provided instead that will be linked against each test. This has the
side effect that, if we wanted to have test cases split across multiple
files, it's as simple as adding them all to the same executable.
The test main should be portable to kernel mode as well, so if
there's a set of tests that should be run in self-test mode in kernel
space, we can accomodate that.
A new serenity_test CMake function streamlines adding a new test with
arguments for the test source file, subdirectory under /usr/Tests to
install the test application and an optional list of libraries to link
against the test application. To accomodate future test where the
provided TestMain.cpp is not suitable (e.g. test-js), a CUSTOM_MAIN
parameter can be passed to the function to not link against the
boilerplate main function.
