Explorar el Código

LibTest: Return 0 if all test cases pass and 1 otherwise from TEST_MAIN

Before, TEST_MAIN used to return the return value of TestSuite::main()
function (which returns the number of test cases that did not pass, so
it can be >=256) directly.

The run-tests utility determines the success / failure of a test suite
binary by examining its (or i.e. TEST_MAIN's) exit status.

But as exit status values are supposed to be between 0 and 255, values
>=256 will get wrapped around (modulo 256), converting a return value of
256 to 0.

So, in a rare case where exactly 256 test cases are failing in your test
suite, run-tests utility will display that the test suite passed without
any failures.

Now, TEST_MAIN just returns 0 if all of the test cases pass and returns
1 otherwise.
ronak69 hace 1 año
padre
commit
a42cf020ea
Se han modificado 2 ficheros con 5 adiciones y 2 borrados
  1. 4 1
      Userland/Libraries/LibTest/TestMain.cpp
  2. 1 1
      Userland/Utilities/run-tests.cpp

+ 4 - 1
Userland/Libraries/LibTest/TestMain.cpp

@@ -29,5 +29,8 @@ int TEST_MAIN(int argc, char** argv)
 
 
     int ret = ::Test::TestSuite::the().main(argv[0], arguments);
     int ret = ::Test::TestSuite::the().main(argv[0], arguments);
     ::Test::TestSuite::release();
     ::Test::TestSuite::release();
-    return ret;
+    // As TestSuite::main() returns the number of test cases that did not pass,
+    // ret can be >=256 which cannot be returned as an exit status directly.
+    // Return 0 if all of the test cases pass and return 1 otherwise.
+    return ret != 0;
 }
 }

+ 1 - 1
Userland/Utilities/run-tests.cpp

@@ -280,7 +280,7 @@ FileResult TestRunner::run_test_file(ByteString const& test_path)
             break; // we'll end up with a failure
             break; // we'll end up with a failure
 
 
         if (WIFEXITED(wstatus)) {
         if (WIFEXITED(wstatus)) {
-            if (wstatus == 0) {
+            if (WEXITSTATUS(wstatus) == 0) {
                 test_result = Test::Result::Pass;
                 test_result = Test::Result::Pass;
             }
             }
             break;
             break;