beenull/ladybird
1
0
Fork 0
mirror of https://github.com/LadybirdBrowser/ladybird.git synced 2024-11-22 15:40:19 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 6
ladybird/Userland/Libraries/LibTest/TestCase.h

183 lines
8.7 KiB
C
Raw Normal View History

AK+Userland: Move AK/TestSuite.h into LibTest and rework Tests' CMake As many macros as possible are moved to Macros.h, while the macros to create a test case are moved to TestCase.h. TestCase is now the only user-facing header for creating a test case. TestSuite and its helpers have moved into a .cpp file. Instead of requiring a TEST_MAIN macro to be instantiated into the test file, a TestMain.cpp file is provided instead that will be linked against each test. This has the side effect that, if we wanted to have test cases split across multiple files, it's as simple as adding them all to the same executable. The test main should be portable to kernel mode as well, so if there's a set of tests that should be run in self-test mode in kernel space, we can accomodate that. A new serenity_test CMake function streamlines adding a new test with arguments for the test source file, subdirectory under /usr/Tests to install the test application and an optional list of libraries to link against the test application. To accomodate future test where the provided TestMain.cpp is not suitable (e.g. test-js), a CUSTOM_MAIN parameter can be passed to the function to not link against the boilerplate main function.
2021-04-25 05:53:23 +00:00
/*
* Copyright (c) 2018-2020, Andreas Kling <kling@serenityos.org>
* Copyright (c) 2021, Andrew Kaster <akaster@serenityos.org>
*
* SPDX-License-Identifier: BSD-2-Clause
*/
#pragma once
Everywhere: Rename {Deprecated => Byte}String This commit un-deprecates DeprecatedString, and repurposes it as a byte string. As the null state has already been removed, there are no other particularly hairy blockers in repurposing this type as a byte string (what it _really_ is). This commit is auto-generated: $ xs=$(ack -l \bDeprecatedString\b\|deprecated_string AK Userland \ Meta Ports Ladybird Tests Kernel) $ perl -pie 's/\bDeprecatedString\b/ByteString/g; s/deprecated_string/byte_string/g' $xs $ clang-format --style=file -i \ $(git diff --name-only | grep \.cpp\|\.h) $ gn format $(git ls-files '*.gn' '*.gni')
2023-12-16 14:19:34 +00:00
#include <AK/ByteString.h>
AK+Userland: Move AK/TestSuite.h into LibTest and rework Tests' CMake As many macros as possible are moved to Macros.h, while the macros to create a test case are moved to TestCase.h. TestCase is now the only user-facing header for creating a test case. TestSuite and its helpers have moved into a .cpp file. Instead of requiring a TEST_MAIN macro to be instantiated into the test file, a TestMain.cpp file is provided instead that will be linked against each test. This has the side effect that, if we wanted to have test cases split across multiple files, it's as simple as adding them all to the same executable. The test main should be portable to kernel mode as well, so if there's a set of tests that should be run in self-test mode in kernel space, we can accomodate that. A new serenity_test CMake function streamlines adding a new test with arguments for the test source file, subdirectory under /usr/Tests to install the test application and an optional list of libraries to link against the test application. To accomodate future test where the provided TestMain.cpp is not suitable (e.g. test-js), a CUSTOM_MAIN parameter can be passed to the function to not link against the boilerplate main function.
2021-04-25 05:53:23 +00:00
#include <AK/Function.h>
#include <AK/NonnullRefPtr.h>
#include <AK/RefCounted.h>
LibTest: Change #define-d constants into constexpr and a runtime flag MAX_GENERATED_VALUES_PER_TEST is now the --randomized_runs flag: $ ./Build/lagom/bin/TestGenerator --randomized_runs 1000 It's sometimes useful to try larger numbers for it instead of the default of 100. MAX_GEN_ATTEMPTS_PER_VALUE is now a constexpr. It's not usually needed to tweak this value; we can recompile with a different value on the rare occasion.
2023-10-27 09:47:43 +00:00
#include <LibTest/Macros.h>
#include <LibTest/Randomized/RandomnessSource.h>
#include <LibTest/Randomized/Shrink.h>
LibTest: Add the RANDOMIZED_TEST_CASE macro and its main loop Tests defined like RANDOMIZED_TEST_CASE(test_name) { GEN(dice, Gen::unsigned_int(1,6)); EXPECT(dice >= 1 && dice <= 6); } will be run many times (100x by default, can be overriden with MAX_GENERATED_VALUES_PER_TEST), each time generating different random values, and if any of the test runs fails, we'll shrink the generated values and report the final minimal ones to the user.
2023-10-23 23:46:52 +00:00
AK+Userland: Move AK/TestSuite.h into LibTest and rework Tests' CMake As many macros as possible are moved to Macros.h, while the macros to create a test case are moved to TestCase.h. TestCase is now the only user-facing header for creating a test case. TestSuite and its helpers have moved into a .cpp file. Instead of requiring a TEST_MAIN macro to be instantiated into the test file, a TestMain.cpp file is provided instead that will be linked against each test. This has the side effect that, if we wanted to have test cases split across multiple files, it's as simple as adding them all to the same executable. The test main should be portable to kernel mode as well, so if there's a set of tests that should be run in self-test mode in kernel space, we can accomodate that. A new serenity_test CMake function streamlines adding a new test with arguments for the test source file, subdirectory under /usr/Tests to install the test application and an optional list of libraries to link against the test application. To accomodate future test where the provided TestMain.cpp is not suitable (e.g. test-js), a CUSTOM_MAIN parameter can be passed to the function to not link against the boilerplate main function.
2021-04-25 05:53:23 +00:00
namespace Test {
using TestFunction = Function<void()>;
LibTest: Add the RANDOMIZED_TEST_CASE macro and its main loop Tests defined like RANDOMIZED_TEST_CASE(test_name) { GEN(dice, Gen::unsigned_int(1,6)); EXPECT(dice >= 1 && dice <= 6); } will be run many times (100x by default, can be overriden with MAX_GENERATED_VALUES_PER_TEST), each time generating different random values, and if any of the test runs fails, we'll shrink the generated values and report the final minimal ones to the user.
2023-10-23 23:46:52 +00:00
inline void run_with_randomness_source(Randomized::RandomnessSource source, TestFunction const& test_function)
{
set_randomness_source(move(source));
set_current_test_result(TestResult::NotRun);
test_function();
if (current_test_result() == TestResult::NotRun) {
set_current_test_result(TestResult::Passed);
}
}
AK+Userland: Move AK/TestSuite.h into LibTest and rework Tests' CMake As many macros as possible are moved to Macros.h, while the macros to create a test case are moved to TestCase.h. TestCase is now the only user-facing header for creating a test case. TestSuite and its helpers have moved into a .cpp file. Instead of requiring a TEST_MAIN macro to be instantiated into the test file, a TestMain.cpp file is provided instead that will be linked against each test. This has the side effect that, if we wanted to have test cases split across multiple files, it's as simple as adding them all to the same executable. The test main should be portable to kernel mode as well, so if there's a set of tests that should be run in self-test mode in kernel space, we can accomodate that. A new serenity_test CMake function streamlines adding a new test with arguments for the test source file, subdirectory under /usr/Tests to install the test application and an optional list of libraries to link against the test application. To accomodate future test where the provided TestMain.cpp is not suitable (e.g. test-js), a CUSTOM_MAIN parameter can be passed to the function to not link against the boilerplate main function.
2021-04-25 05:53:23 +00:00
class TestCase : public RefCounted<TestCase> {
public:
Everywhere: Rename {Deprecated => Byte}String This commit un-deprecates DeprecatedString, and repurposes it as a byte string. As the null state has already been removed, there are no other particularly hairy blockers in repurposing this type as a byte string (what it _really_ is). This commit is auto-generated: $ xs=$(ack -l \bDeprecatedString\b\|deprecated_string AK Userland \ Meta Ports Ladybird Tests Kernel) $ perl -pie 's/\bDeprecatedString\b/ByteString/g; s/deprecated_string/byte_string/g' $xs $ clang-format --style=file -i \ $(git diff --name-only | grep \.cpp\|\.h) $ gn format $(git ls-files '*.gn' '*.gni')
2023-12-16 14:19:34 +00:00
TestCase(ByteString const& name, TestFunction&& fn, bool is_benchmark)
AK+Userland: Move AK/TestSuite.h into LibTest and rework Tests' CMake As many macros as possible are moved to Macros.h, while the macros to create a test case are moved to TestCase.h. TestCase is now the only user-facing header for creating a test case. TestSuite and its helpers have moved into a .cpp file. Instead of requiring a TEST_MAIN macro to be instantiated into the test file, a TestMain.cpp file is provided instead that will be linked against each test. This has the side effect that, if we wanted to have test cases split across multiple files, it's as simple as adding them all to the same executable. The test main should be portable to kernel mode as well, so if there's a set of tests that should be run in self-test mode in kernel space, we can accomodate that. A new serenity_test CMake function streamlines adding a new test with arguments for the test source file, subdirectory under /usr/Tests to install the test application and an optional list of libraries to link against the test application. To accomodate future test where the provided TestMain.cpp is not suitable (e.g. test-js), a CUSTOM_MAIN parameter can be passed to the function to not link against the boilerplate main function.
2021-04-25 05:53:23 +00:00
: m_name(name)
, m_function(move(fn))
, m_is_benchmark(is_benchmark)
{
}
bool is_benchmark() const { return m_is_benchmark; }
Everywhere: Rename {Deprecated => Byte}String This commit un-deprecates DeprecatedString, and repurposes it as a byte string. As the null state has already been removed, there are no other particularly hairy blockers in repurposing this type as a byte string (what it _really_ is). This commit is auto-generated: $ xs=$(ack -l \bDeprecatedString\b\|deprecated_string AK Userland \ Meta Ports Ladybird Tests Kernel) $ perl -pie 's/\bDeprecatedString\b/ByteString/g; s/deprecated_string/byte_string/g' $xs $ clang-format --style=file -i \ $(git diff --name-only | grep \.cpp\|\.h) $ gn format $(git ls-files '*.gn' '*.gni')
2023-12-16 14:19:34 +00:00
ByteString const& name() const { return m_name; }
Everywhere: Run clang-format
2022-04-01 17:58:27 +00:00
TestFunction const& func() const { return m_function; }
AK+Userland: Move AK/TestSuite.h into LibTest and rework Tests' CMake As many macros as possible are moved to Macros.h, while the macros to create a test case are moved to TestCase.h. TestCase is now the only user-facing header for creating a test case. TestSuite and its helpers have moved into a .cpp file. Instead of requiring a TEST_MAIN macro to be instantiated into the test file, a TestMain.cpp file is provided instead that will be linked against each test. This has the side effect that, if we wanted to have test cases split across multiple files, it's as simple as adding them all to the same executable. The test main should be portable to kernel mode as well, so if there's a set of tests that should be run in self-test mode in kernel space, we can accomodate that. A new serenity_test CMake function streamlines adding a new test with arguments for the test source file, subdirectory under /usr/Tests to install the test application and an optional list of libraries to link against the test application. To accomodate future test where the provided TestMain.cpp is not suitable (e.g. test-js), a CUSTOM_MAIN parameter can be passed to the function to not link against the boilerplate main function.
2021-04-25 05:53:23 +00:00
Everywhere: Rename {Deprecated => Byte}String This commit un-deprecates DeprecatedString, and repurposes it as a byte string. As the null state has already been removed, there are no other particularly hairy blockers in repurposing this type as a byte string (what it _really_ is). This commit is auto-generated: $ xs=$(ack -l \bDeprecatedString\b\|deprecated_string AK Userland \ Meta Ports Ladybird Tests Kernel) $ perl -pie 's/\bDeprecatedString\b/ByteString/g; s/deprecated_string/byte_string/g' $xs $ clang-format --style=file -i \ $(git diff --name-only | grep \.cpp\|\.h) $ gn format $(git ls-files '*.gn' '*.gni')
2023-12-16 14:19:34 +00:00
static NonnullRefPtr<TestCase> randomized(ByteString const& name, TestFunction&& test_function)
LibTest: Add the RANDOMIZED_TEST_CASE macro and its main loop Tests defined like RANDOMIZED_TEST_CASE(test_name) { GEN(dice, Gen::unsigned_int(1,6)); EXPECT(dice >= 1 && dice <= 6); } will be run many times (100x by default, can be overriden with MAX_GENERATED_VALUES_PER_TEST), each time generating different random values, and if any of the test runs fails, we'll shrink the generated values and report the final minimal ones to the user.
2023-10-23 23:46:52 +00:00
{
using namespace Randomized;
LibTest: Change #define-d constants into constexpr and a runtime flag MAX_GENERATED_VALUES_PER_TEST is now the --randomized_runs flag: $ ./Build/lagom/bin/TestGenerator --randomized_runs 1000 It's sometimes useful to try larger numbers for it instead of the default of 100. MAX_GEN_ATTEMPTS_PER_VALUE is now a constexpr. It's not usually needed to tweak this value; we can recompile with a different value on the rare occasion.
2023-10-27 09:47:43 +00:00
LibTest: Bump up MAX_GEN_ATTEMPTS_PER_VALUE to 30 The original value 15 was too little: it made our `weighted_boolean_fair_false` test fail every now and then. This is because a fair coin (P(false) = 0.5) will hit the same value 15 times in a row with a probability (1/2)^15: around once in a 32k tries. With the bumped up value, this is now once in 1 billion tries. Should lower the test flakiness enough (if our random number generator is truly uniform), while 30 tries is still an OK amount of computation for randomized tests to do, compared to 15.
2023-11-06 09:41:25 +00:00
constexpr u8 MAX_GEN_ATTEMPTS_PER_VALUE = 30;
LibTest: Change #define-d constants into constexpr and a runtime flag MAX_GENERATED_VALUES_PER_TEST is now the --randomized_runs flag: $ ./Build/lagom/bin/TestGenerator --randomized_runs 1000 It's sometimes useful to try larger numbers for it instead of the default of 100. MAX_GEN_ATTEMPTS_PER_VALUE is now a constexpr. It's not usually needed to tweak this value; we can recompile with a different value on the rare occasion.
2023-10-27 09:47:43 +00:00
LibTest: Add the RANDOMIZED_TEST_CASE macro and its main loop Tests defined like RANDOMIZED_TEST_CASE(test_name) { GEN(dice, Gen::unsigned_int(1,6)); EXPECT(dice >= 1 && dice <= 6); } will be run many times (100x by default, can be overriden with MAX_GENERATED_VALUES_PER_TEST), each time generating different random values, and if any of the test runs fails, we'll shrink the generated values and report the final minimal ones to the user.
2023-10-23 23:46:52 +00:00
TestFunction test_case_function = [test_function = move(test_function)]() {
LibTest: Change #define-d constants into constexpr and a runtime flag MAX_GENERATED_VALUES_PER_TEST is now the --randomized_runs flag: $ ./Build/lagom/bin/TestGenerator --randomized_runs 1000 It's sometimes useful to try larger numbers for it instead of the default of 100. MAX_GEN_ATTEMPTS_PER_VALUE is now a constexpr. It's not usually needed to tweak this value; we can recompile with a different value on the rare occasion.
2023-10-27 09:47:43 +00:00
u64 max_randomized_runs = randomized_runs();
for (u64 i = 0; i < max_randomized_runs; ++i) {
LibTest: Add the RANDOMIZED_TEST_CASE macro and its main loop Tests defined like RANDOMIZED_TEST_CASE(test_name) { GEN(dice, Gen::unsigned_int(1,6)); EXPECT(dice >= 1 && dice <= 6); } will be run many times (100x by default, can be overriden with MAX_GENERATED_VALUES_PER_TEST), each time generating different random values, and if any of the test runs fails, we'll shrink the generated values and report the final minimal ones to the user.
2023-10-23 23:46:52 +00:00
bool generated_successfully = false;
u8 gen_attempt;
for (gen_attempt = 0; gen_attempt < MAX_GEN_ATTEMPTS_PER_VALUE && !generated_successfully; ++gen_attempt) {
// We're going to run the test function many times, so let's turn off the reporting until we finish.
disable_reporting();
set_current_test_result(TestResult::NotRun);
run_with_randomness_source(RandomnessSource::live(), test_function);
switch (current_test_result()) {
case TestResult::NotRun:
VERIFY_NOT_REACHED();
break;
case TestResult::Passed: {
generated_successfully = true;
break;
}
case TestResult::Failed: {
generated_successfully = true;
RandomRun first_failure = randomness_source().run();
RandomRun best_failure = shrink(first_failure, test_function);
// Run one last time with reporting on, so that the user can see the minimal failure
enable_reporting();
run_with_randomness_source(RandomnessSource::recorded(best_failure), test_function);
return;
}
case TestResult::Rejected:
break;
case TestResult::Overrun:
break;
default:
VERIFY_NOT_REACHED();
break;
}
}
enable_reporting();
if (!generated_successfully) {
// The loop above got to the full MAX_GEN_ATTEMPTS_PER_VALUE and gave up.
// Run one last time with reporting on, so that the user gets the REJECTED message.
RandomRun last_failure = randomness_source().run();
run_with_randomness_source(RandomnessSource::recorded(last_failure), test_function);
return;
}
}
LibTest: Change #define-d constants into constexpr and a runtime flag MAX_GENERATED_VALUES_PER_TEST is now the --randomized_runs flag: $ ./Build/lagom/bin/TestGenerator --randomized_runs 1000 It's sometimes useful to try larger numbers for it instead of the default of 100. MAX_GEN_ATTEMPTS_PER_VALUE is now a constexpr. It's not usually needed to tweak this value; we can recompile with a different value on the rare occasion.
2023-10-27 09:47:43 +00:00
// All randomized_runs() values generated + passed the test.
LibTest: Add the RANDOMIZED_TEST_CASE macro and its main loop Tests defined like RANDOMIZED_TEST_CASE(test_name) { GEN(dice, Gen::unsigned_int(1,6)); EXPECT(dice >= 1 && dice <= 6); } will be run many times (100x by default, can be overriden with MAX_GENERATED_VALUES_PER_TEST), each time generating different random values, and if any of the test runs fails, we'll shrink the generated values and report the final minimal ones to the user.
2023-10-23 23:46:52 +00:00
};
return make_ref_counted<TestCase>(name, move(test_case_function), false);
}
AK+Userland: Move AK/TestSuite.h into LibTest and rework Tests' CMake As many macros as possible are moved to Macros.h, while the macros to create a test case are moved to TestCase.h. TestCase is now the only user-facing header for creating a test case. TestSuite and its helpers have moved into a .cpp file. Instead of requiring a TEST_MAIN macro to be instantiated into the test file, a TestMain.cpp file is provided instead that will be linked against each test. This has the side effect that, if we wanted to have test cases split across multiple files, it's as simple as adding them all to the same executable. The test main should be portable to kernel mode as well, so if there's a set of tests that should be run in self-test mode in kernel space, we can accomodate that. A new serenity_test CMake function streamlines adding a new test with arguments for the test source file, subdirectory under /usr/Tests to install the test application and an optional list of libraries to link against the test application. To accomodate future test where the provided TestMain.cpp is not suitable (e.g. test-js), a CUSTOM_MAIN parameter can be passed to the function to not link against the boilerplate main function.
2021-04-25 05:53:23 +00:00
private:
Everywhere: Rename {Deprecated => Byte}String This commit un-deprecates DeprecatedString, and repurposes it as a byte string. As the null state has already been removed, there are no other particularly hairy blockers in repurposing this type as a byte string (what it _really_ is). This commit is auto-generated: $ xs=$(ack -l \bDeprecatedString\b\|deprecated_string AK Userland \ Meta Ports Ladybird Tests Kernel) $ perl -pie 's/\bDeprecatedString\b/ByteString/g; s/deprecated_string/byte_string/g' $xs $ clang-format --style=file -i \ $(git diff --name-only | grep \.cpp\|\.h) $ gn format $(git ls-files '*.gn' '*.gni')
2023-12-16 14:19:34 +00:00
ByteString m_name;
AK+Userland: Move AK/TestSuite.h into LibTest and rework Tests' CMake As many macros as possible are moved to Macros.h, while the macros to create a test case are moved to TestCase.h. TestCase is now the only user-facing header for creating a test case. TestSuite and its helpers have moved into a .cpp file. Instead of requiring a TEST_MAIN macro to be instantiated into the test file, a TestMain.cpp file is provided instead that will be linked against each test. This has the side effect that, if we wanted to have test cases split across multiple files, it's as simple as adding them all to the same executable. The test main should be portable to kernel mode as well, so if there's a set of tests that should be run in self-test mode in kernel space, we can accomodate that. A new serenity_test CMake function streamlines adding a new test with arguments for the test source file, subdirectory under /usr/Tests to install the test application and an optional list of libraries to link against the test application. To accomodate future test where the provided TestMain.cpp is not suitable (e.g. test-js), a CUSTOM_MAIN parameter can be passed to the function to not link against the boilerplate main function.
2021-04-25 05:53:23 +00:00
TestFunction m_function;
bool m_is_benchmark;
};
// Helper to hide implementation of TestSuite from users
Everywhere: Run clang-format
2022-04-01 17:58:27 +00:00
void add_test_case_to_suite(NonnullRefPtr<TestCase> const& test_case);
LibTest: Add TEST_SETUP macro that runs before all test cases
2021-08-29 18:55:05 +00:00
void set_suite_setup_function(Function<void()> setup);
AK+Userland: Move AK/TestSuite.h into LibTest and rework Tests' CMake As many macros as possible are moved to Macros.h, while the macros to create a test case are moved to TestCase.h. TestCase is now the only user-facing header for creating a test case. TestSuite and its helpers have moved into a .cpp file. Instead of requiring a TEST_MAIN macro to be instantiated into the test file, a TestMain.cpp file is provided instead that will be linked against each test. This has the side effect that, if we wanted to have test cases split across multiple files, it's as simple as adding them all to the same executable. The test main should be portable to kernel mode as well, so if there's a set of tests that should be run in self-test mode in kernel space, we can accomodate that. A new serenity_test CMake function streamlines adding a new test with arguments for the test source file, subdirectory under /usr/Tests to install the test application and an optional list of libraries to link against the test application. To accomodate future test where the provided TestMain.cpp is not suitable (e.g. test-js), a CUSTOM_MAIN parameter can be passed to the function to not link against the boilerplate main function.
2021-04-25 05:53:23 +00:00
}
Everywhere: Run clang-format
2022-10-16 22:06:11 +00:00
#define TEST_SETUP \
static void __setup(); \
struct __setup_type { \
__setup_type() \
{ \
Test::set_suite_setup_function(__setup); \
} \
}; \
static struct __setup_type __setup_type; \
LibTest: Add TEST_SETUP macro that runs before all test cases
2021-08-29 18:55:05 +00:00
static void __setup()
LibTest: Add the RANDOMIZED_TEST_CASE macro and its main loop Tests defined like RANDOMIZED_TEST_CASE(test_name) { GEN(dice, Gen::unsigned_int(1,6)); EXPECT(dice >= 1 && dice <= 6); } will be run many times (100x by default, can be overriden with MAX_GENERATED_VALUES_PER_TEST), each time generating different random values, and if any of the test runs fails, we'll shrink the generated values and report the final minimal ones to the user.
2023-10-23 23:46:52 +00:00
// Unit test
AK+Userland: Move AK/TestSuite.h into LibTest and rework Tests' CMake As many macros as possible are moved to Macros.h, while the macros to create a test case are moved to TestCase.h. TestCase is now the only user-facing header for creating a test case. TestSuite and its helpers have moved into a .cpp file. Instead of requiring a TEST_MAIN macro to be instantiated into the test file, a TestMain.cpp file is provided instead that will be linked against each test. This has the side effect that, if we wanted to have test cases split across multiple files, it's as simple as adding them all to the same executable. The test main should be portable to kernel mode as well, so if there's a set of tests that should be run in self-test mode in kernel space, we can accomodate that. A new serenity_test CMake function streamlines adding a new test with arguments for the test source file, subdirectory under /usr/Tests to install the test application and an optional list of libraries to link against the test application. To accomodate future test where the provided TestMain.cpp is not suitable (e.g. test-js), a CUSTOM_MAIN parameter can be passed to the function to not link against the boilerplate main function.
2021-04-25 05:53:23 +00:00
#define __TESTCASE_FUNC(x) __test_##x
#define __TESTCASE_TYPE(x) __TestCase_##x
Everywhere: Run clang-format
2022-10-16 22:06:11 +00:00
#define TEST_CASE(x) \
static void __TESTCASE_FUNC(x)(); \
struct __TESTCASE_TYPE(x) { \
__TESTCASE_TYPE(x) \
() \
{ \
add_test_case_to_suite(adopt_ref(*new ::Test::TestCase(#x, __TESTCASE_FUNC(x), false))); \
} \
}; \
static struct __TESTCASE_TYPE(x) __TESTCASE_TYPE(x); \
AK+Userland: Move AK/TestSuite.h into LibTest and rework Tests' CMake As many macros as possible are moved to Macros.h, while the macros to create a test case are moved to TestCase.h. TestCase is now the only user-facing header for creating a test case. TestSuite and its helpers have moved into a .cpp file. Instead of requiring a TEST_MAIN macro to be instantiated into the test file, a TestMain.cpp file is provided instead that will be linked against each test. This has the side effect that, if we wanted to have test cases split across multiple files, it's as simple as adding them all to the same executable. The test main should be portable to kernel mode as well, so if there's a set of tests that should be run in self-test mode in kernel space, we can accomodate that. A new serenity_test CMake function streamlines adding a new test with arguments for the test source file, subdirectory under /usr/Tests to install the test application and an optional list of libraries to link against the test application. To accomodate future test where the provided TestMain.cpp is not suitable (e.g. test-js), a CUSTOM_MAIN parameter can be passed to the function to not link against the boilerplate main function.
2021-04-25 05:53:23 +00:00
static void __TESTCASE_FUNC(x)()
LibTest: Add the RANDOMIZED_TEST_CASE macro and its main loop Tests defined like RANDOMIZED_TEST_CASE(test_name) { GEN(dice, Gen::unsigned_int(1,6)); EXPECT(dice >= 1 && dice <= 6); } will be run many times (100x by default, can be overriden with MAX_GENERATED_VALUES_PER_TEST), each time generating different random values, and if any of the test runs fails, we'll shrink the generated values and report the final minimal ones to the user.
2023-10-23 23:46:52 +00:00
// Benchmark
AK+Userland: Move AK/TestSuite.h into LibTest and rework Tests' CMake As many macros as possible are moved to Macros.h, while the macros to create a test case are moved to TestCase.h. TestCase is now the only user-facing header for creating a test case. TestSuite and its helpers have moved into a .cpp file. Instead of requiring a TEST_MAIN macro to be instantiated into the test file, a TestMain.cpp file is provided instead that will be linked against each test. This has the side effect that, if we wanted to have test cases split across multiple files, it's as simple as adding them all to the same executable. The test main should be portable to kernel mode as well, so if there's a set of tests that should be run in self-test mode in kernel space, we can accomodate that. A new serenity_test CMake function streamlines adding a new test with arguments for the test source file, subdirectory under /usr/Tests to install the test application and an optional list of libraries to link against the test application. To accomodate future test where the provided TestMain.cpp is not suitable (e.g. test-js), a CUSTOM_MAIN parameter can be passed to the function to not link against the boilerplate main function.
2021-04-25 05:53:23 +00:00
#define __BENCHMARK_FUNC(x) __benchmark_##x
#define __BENCHMARK_TYPE(x) __BenchmarkCase_##x
Everywhere: Run clang-format
2022-10-16 22:06:11 +00:00
#define BENCHMARK_CASE(x) \
static void __BENCHMARK_FUNC(x)(); \
struct __BENCHMARK_TYPE(x) { \
__BENCHMARK_TYPE(x) \
() \
{ \
add_test_case_to_suite(adopt_ref(*new ::Test::TestCase(#x, __BENCHMARK_FUNC(x), true))); \
} \
}; \
static struct __BENCHMARK_TYPE(x) __BENCHMARK_TYPE(x); \
AK+Userland: Move AK/TestSuite.h into LibTest and rework Tests' CMake As many macros as possible are moved to Macros.h, while the macros to create a test case are moved to TestCase.h. TestCase is now the only user-facing header for creating a test case. TestSuite and its helpers have moved into a .cpp file. Instead of requiring a TEST_MAIN macro to be instantiated into the test file, a TestMain.cpp file is provided instead that will be linked against each test. This has the side effect that, if we wanted to have test cases split across multiple files, it's as simple as adding them all to the same executable. The test main should be portable to kernel mode as well, so if there's a set of tests that should be run in self-test mode in kernel space, we can accomodate that. A new serenity_test CMake function streamlines adding a new test with arguments for the test source file, subdirectory under /usr/Tests to install the test application and an optional list of libraries to link against the test application. To accomodate future test where the provided TestMain.cpp is not suitable (e.g. test-js), a CUSTOM_MAIN parameter can be passed to the function to not link against the boilerplate main function.
2021-04-25 05:53:23 +00:00
static void __BENCHMARK_FUNC(x)()
LibTest: Add the GEN macro Generators are callable as-is: u32 my_int = Gen::unsigned_int(); // -> 1, 5, 8, 3, 2 But there is little visibility in the test fail message into what went wrong. Showing what values were generated helps a lot, and that's what this macro does: GEN(my_int, Gen::unsigned_int()); expands into the above declaration and (crucially) a conditional warnln() call looking like "my_int = {}". It will only run if error reporting is enabled (see Test::can_report()), so it will only give the final shrunk value instead of spamming the output with each generated value.
2023-10-23 23:43:23 +00:00
LibTest: Add the RANDOMIZED_TEST_CASE macro and its main loop Tests defined like RANDOMIZED_TEST_CASE(test_name) { GEN(dice, Gen::unsigned_int(1,6)); EXPECT(dice >= 1 && dice <= 6); } will be run many times (100x by default, can be overriden with MAX_GENERATED_VALUES_PER_TEST), each time generating different random values, and if any of the test runs fails, we'll shrink the generated values and report the final minimal ones to the user.
2023-10-23 23:46:52 +00:00
// Randomized test
#define __RANDOMIZED_TEST_FUNC(x) __randomized_test_##x
#define __RANDOMIZED_TEST_TYPE(x) __RandomizedTestCase_##x
#define RANDOMIZED_TEST_CASE(x) \
static void __RANDOMIZED_TEST_FUNC(x)(); \
struct __RANDOMIZED_TEST_TYPE(x) { \
__RANDOMIZED_TEST_TYPE(x) \
() \
{ \
add_test_case_to_suite(::Test::TestCase::randomized(#x, __RANDOMIZED_TEST_FUNC(x))); \
} \
}; \
static struct __RANDOMIZED_TEST_TYPE(x) __RANDOMIZED_TEST_TYPE(x); \
static void __RANDOMIZED_TEST_FUNC(x)()
LibTest: Add the GEN macro Generators are callable as-is: u32 my_int = Gen::unsigned_int(); // -> 1, 5, 8, 3, 2 But there is little visibility in the test fail message into what went wrong. Showing what values were generated helps a lot, and that's what this macro does: GEN(my_int, Gen::unsigned_int()); expands into the above declaration and (crucially) a conditional warnln() call looking like "my_int = {}". It will only run if error reporting is enabled (see Test::can_report()), so it will only give the final shrunk value instead of spamming the output with each generated value.
2023-10-23 23:43:23 +00:00
// This allows us to print the generated locals in the test after a failure is fully shrunk.
#define GEN(identifier, value) \
auto identifier = (value); \
if (::Test::current_test_result() == ::Test::TestResult::Overrun) \
return; \
if (::Test::is_reporting_enabled()) \
::AK::warnln("{} = {}", #identifier, (identifier))
Reference in a new issue Copy permalink