LibTest: Add the REJECT and ASSUME macros

REJECT and ASSUME are useful for filtering out unwanted generated
values. While this is not ideal, it is ocassionally useful and so we
include it for convenience.

The main loop of RANDOMIZED_TEST_CASE runs the test case 100 times, each
time trying to generate a different set of values. Inside that loop, if
it sees a REJECT (ASSUME is implemented in terms of REJECT), it retries
up to 15 times before giving up (perhaps it's impossible or just very
improbable to generate a value that will survive REJECT or ASSUME).

REJECT("Reason for rejecting") will just outright fail, while
ASSUME(bool) is more of an equivalent of a .filter() method from
functional languages.
This commit is contained in:
Martin Janiczek 2023-10-24 01:34:46 +02:00 committed by Andrew Kaster
parent 1bcfead020
commit 00934bc344
Notes: sideshowbarker 2024-07-16 21:51:02 +09:00
3 changed files with 26 additions and 0 deletions

View file

@ -109,6 +109,25 @@ void disable_reporting();
#define EXPECT_APPROXIMATE(a, b) EXPECT_APPROXIMATE_WITH_ERROR(a, b, 0.0000005) #define EXPECT_APPROXIMATE(a, b) EXPECT_APPROXIMATE_WITH_ERROR(a, b, 0.0000005)
#define REJECT(message) \
do { \
if (::Test::is_reporting_enabled()) \
::AK::warnln("\033[31;1mREJECTED\033[0m: {}:{}: {}", \
__FILE__, __LINE__, #message); \
::Test::set_current_test_result(::Test::TestResult::Rejected); \
} while (false)
#define ASSUME(x) \
do { \
if (!(x)) { \
if (::Test::is_reporting_enabled()) \
::AK::warnln("\033[31;1mREJECTED\033[0m: {}:{}: Couldn't generate random value satisfying ASSUME({})", \
__FILE__, __LINE__, #x); \
::Test::set_current_test_result(::Test::TestResult::Rejected); \
return; \
} \
} while (false)
#define FAIL(message) \ #define FAIL(message) \
do { \ do { \
if (::Test::is_reporting_enabled()) \ if (::Test::is_reporting_enabled()) \

View file

@ -18,6 +18,11 @@ enum class TestResult {
// Didn't get through EXPECT(...). // Didn't get through EXPECT(...).
Failed, Failed,
// Didn't get through the ASSUME(...) filter 15 times in a row
// (in a randomized test).
// Alternatively, user used REJECT(...).
Rejected,
// Ran out of RandomRun data (in a randomized test, when shrinking). // Ran out of RandomRun data (in a randomized test, when shrinking).
// This is fine, we'll just try some other shrink. // This is fine, we'll just try some other shrink.
Overrun, Overrun,

View file

@ -103,6 +103,8 @@ static DeprecatedString test_result_to_string(TestResult result)
return "Completed"; return "Completed";
case TestResult::Failed: case TestResult::Failed:
return "Failed"; return "Failed";
case TestResult::Rejected:
return "Rejected";
case TestResult::Overrun: case TestResult::Overrun:
return "Ran out of randomness"; return "Ran out of randomness";
default: default: