Commit graph

132 commits

Author SHA1 Message Date
Luke
3bc2527ce7 Lagom/Fuzzers: Add SQL parser fuzzer 2021-05-31 17:07:52 +04:30
Gunnar Beutner
5f18cf75c5 AK: Replace ByteBuffer::grow with resize()/ensure_capacity()
Previously ByteBuffer::grow() behaved like Vector<T>::resize().
However the function name was somewhat ambiguous - and so this patch
updates ByteBuffer to behave more like Vector<T> by replacing grow()
with resize() and adding an ensure_capacity() method.

This also lets the user change the buffer's capacity without affecting
the size which was not previously possible.

Additionally this patch makes the capacity() method public (again).
2021-05-31 14:49:00 +04:30
Ben Wiederhake
774107f37c Fuzz+LibGfx: When fuzzing GIFLoader, try to load all frames 2021-05-30 14:42:34 +01:00
Ali Mohammad Pur
0e4431af33 Meta: Run the Wasm spec tests in CI
Since LibWasm is still not capable of passing all of the spec tests,
ignore failing tests, only fail the build if some segfault/abort/etc
occurs.
2021-05-27 17:28:41 +04:30
Ali Mohammad Pur
b3c13c3e8a LibWasm+Meta: Add test-wasm and optionally test the conformance tests
This only tests "can it be parsed", but the goal of this commit is to
provide a test framework that can be built upon :)
The conformance tests are downloaded, compiled* and installed only if
the INCLUDE_WASM_SPEC_TESTS cmake option is enabled.
(*) Since we do not yet have a wast parser, the compilation is delegated
to an external tool from binaryen, `wasm-as`, which is required for the
test suite download/install to succeed.
This *does* run the tests in CI, but it currently does not include the
spec conformance tests.
2021-05-21 00:15:23 +01:00
Ali Mohammad Pur
ba2fce14d3 Meta: Add a Wasm parser fuzzer 2021-05-21 00:15:23 +01:00
Ali Mohammad Pur
f137c1bfaa LibJS+LibTest: Move out the test-js test runner into LibTest 2021-05-18 18:48:15 +01:00
Ali Mohammad Pur
dfb2178519 Meta: Build LibWasm and the wasm utility as part of Lagom 2021-05-17 23:25:30 +02:00
Gunnar Beutner
f0fa51773a AK+Userland: Fix some compiler warnings and make variables const-ref
This fixes a few compiler warnings and makes some variables const-ref
in preparation for the next commit which changes how ByteBuffer works.
2021-05-16 17:49:42 +02:00
Andrew Kaster
d81f52e529 Lagom: Enable sanitizer builds with gcc
Previously the CMake options for -fsanitize=address, thread and
undefined were gated behind clang, which was unecessary. Only
-fsanitize=fuzzer is clang-only.
2021-05-14 08:34:00 +01:00
DexesTTP
5963f6f9ff LibCrypto: Split BigInteger operations into an Algorithms class
Since the operations are already complicated and will become even more
so soon, let's split them into their own files. We can also integrate
the NumberTheory operations that would better fit there into this class
as well.

This commit doesn't change behaviors, but moves the allocation of some
variables into caller classes.
2021-05-13 19:18:07 +01:00
Dylan Katz
984b6e08cf LibSQL: Add a syntax highlighter 2021-05-09 09:39:05 +02:00
Brian Gianforcaro
7e28ecc305 Meta + Lagom: Fix paths after movement of tests to Tests/
Not sure how this didn't cause a build break when I originally
moved them in #6891. Better fix them now.
2021-05-08 00:04:10 +01:00
Brian Gianforcaro
9720ad3901 Tests: Move Userland/Utilities/test-js to Tests/LibJS 2021-05-08 00:04:10 +01:00
Brian Gianforcaro
dd633b9dd1 Meta: Fix Fuzzers CMakeLists.txt and ReadMe.md to use CXX_COMPILER_ID
Previously the directions omitted that you have to specify
`-CMAKE_CXX_COMPILER` when building the Fuzzers. This
would cause all kinds of weird problems at compilation and
link time. You can't specify one or the other, they must
both be pointing at clang in order for things to work as
experted. Fix this by updating the documentation to specify
that the user should specify both the C and CXX compiler explicitly
to be safe, as well as forcing the cmake clang argument handling
to modify the CXX compiler variable instead of the C version.
2021-05-07 15:19:48 +01:00
Idan Horowitz
5c385d06e8 Lagom/Fuzzers: Add fuzzers for Windows-1251 and Windows-1255 decoders 2021-05-01 17:59:08 +02:00
Carlos César Neves Enumo
69e2431c64 Lagom/CMake: Add -fconstexpr-steps for AppleClang 2021-05-01 07:38:43 +02:00
Idan Horowitz
33fdd402b5 Meta: Increase fconstexpr-steps for Clang Lagom builds
Clang's default constexpr-steps limit is 1048576, which is not enough
for LibGfx's generation of the unicode bidirectional class lookup table
while GCC doesn't have any limit at all, so this patch increases the
limit to an arbitrarily larger value.
2021-04-29 10:36:23 +02:00
Andrew Kaster
35c0a6c54d AK+Userland: Move AK/TestSuite.h into LibTest and rework Tests' CMake
As many macros as possible are moved to Macros.h, while the
macros to create a test case are moved to TestCase.h. TestCase is now
the only user-facing header for creating a test case. TestSuite and its
helpers have moved into a .cpp file. Instead of requiring a TEST_MAIN
macro to be instantiated into the test file, a TestMain.cpp file is
provided instead that will be linked against each test. This has the
side effect that, if we wanted to have test cases split across multiple
files, it's as simple as adding them all to the same executable.

The test main should be portable to kernel mode as well, so if
there's a set of tests that should be run in self-test mode in kernel
space, we can accomodate that.

A new serenity_test CMake function streamlines adding a new test with
arguments for the test source file, subdirectory under /usr/Tests to
install the test application and an optional list of libraries to link
against the test application. To accomodate future test where the
provided TestMain.cpp is not suitable (e.g. test-js), a CUSTOM_MAIN
parameter can be passed to the function to not link against the
boilerplate main function.
2021-04-25 09:36:49 +02:00
Brian Gianforcaro
1682f0b760 Everything: Move to SPDX license identifiers in all files.
SPDX License Identifiers are a more compact / standardized
way of representing file license information.

See: https://spdx.dev/resources/use/#identifiers

This was done with the `ambr` search and replace tool.

 ambr --no-parent-ignore --key-from-file --rep-from-file key.txt rep.txt *
2021-04-22 11:22:27 +02:00
Marco Biscaro
8124719c3d Tests: Reorganize LibCompress unit tests
Move LibCompress unit tests to LibCompress/Tests directory and register
them with CMake's add_test. This allows us to run these tests with
ninja test instead of running a separate executable.

Also split the existing tests in 3 test files that better follow the
source code structure (inspired by AK tests).
2021-04-21 08:00:32 +02:00
Timothy Flynn
110cd98c0a Lagom: Add sql utility and LibSQL unit tests 2021-04-20 18:28:34 +02:00
Andreas Kling
251018676f Everywhere: Update references from ReadMe.md => README.md 2021-04-11 10:52:25 +02:00
Luke
1ff6f2b4b2 Lagom/Fuzzers: Add fuzzer for zip file parser 2021-03-27 16:28:34 +01:00
Andreas Kling
d792200a55 LibJS: Rename GlobalObject::initialize() => initialize_global_object()
This function was shadowing Object::initialize() which cannot be called
on global objects and has a different set of parameters.
2021-03-17 16:53:35 +01:00
Luke
c983e42e8c Lagom/Fuzzers: Add fuzzers for the new Gzip and Deflate compressors 2021-03-14 11:28:12 +01:00
Luke
d27e6f0961 Lagom/Fuzzers: Add fuzzers for Latin 1, Latin 2 and UTF16-BE
No fuzzer for UTF-8 as it (currently) just returns the input.
2021-03-14 11:27:52 +01:00
Andreas Kling
9769542bc2 FuzzilliJs: Add missing <errno.h> include 2021-03-12 17:43:48 +01:00
Luke
28a705d3c7 Lagom/Fuzzers: Add fuzzers for LibCompess
Adds fuzzers for Deflate, Gzip and Zlib.
2021-03-04 09:19:29 +01:00
Andrew Kaster
1aa605bc03 Meta: Run test-js and test-compress as CTest executables
Reduces the number of steps in the Actions workflow this way :^)
2021-03-02 09:00:21 +01:00
Andrew Kaster
669b6c43aa AK/Lagom: Modify TestSuite to return how many tests failed from main
This allows us to remove the FAIL_REGEX logic from the CTest invocation
of AK and LibRegex tests, as they will return a non-zero exit code on
failure :^).

Also means that running a failing TestSuite-enabled test with the
run-test-and-shutdown script will actually print that the test failed.
2021-03-01 11:12:36 +01:00
Luke
a66f96ff62 Lagom/Fuzzers: Add WAV fuzzer 2021-03-01 11:09:09 +01:00
Andrew Kaster
e787738c24 Meta: Build AK and LibRegex tests in Lagom and for Serenity
These tests were never built for the serenity target. Move their Lagom
build steps to the Lagom CMakeLists.txt, and add serenity build steps
for them. Also, fix the build errors when building them with the
serenity cross-compiler :^)
2021-02-28 18:19:37 +01:00
Linus Groh
24120d9dca Lagom: Build with -Wno-literal-suffix when using GCC
This is already set in the root CMakeLists.txt as well as here for Clang
(-Wno-user-defined-literals), but was forgotten for GCC which made an
Lagom-only build (cmake ../Meta/Lagom [...]) fail.
2021-02-27 16:38:21 +01:00
Linus Groh
590ae7a36d Lagom: CMakeLists.txt housekeeping
- Remove unused LIBM_SOURCES
- Some cosmetic and whitespace changes
2021-02-27 16:38:21 +01:00
Linus Groh
e265054c12 Everywhere: Remove a bunch of redundant 'AK::' namespace prefixes
This is basically just for consistency, it's quite strange to see
multiple AK container types next to each other, some with and some
without the namespace prefix - we're 'using AK::Foo;' a lot and should
leverage that. :^)
2021-02-26 16:59:56 +01:00
Brian Gianforcaro
31e1b08e15 AK: Add support for AK::StringView literals with operator""sv
A new operator, operator""sv was added as of C++17 to support
string_view literals. This allows string_views to be constructed
from string literals and with no runtime cost to find the string
length.

See: https://en.cppreference.com/w/cpp/string/basic_string_view/operator%22%22sv

This change implements that functionality in AK::StringView.
We do have to suppress some warnings about implementing reserved
operators as we are essentially implementing STL functions in AK
as we have no STL :).
2021-02-24 14:38:31 +01:00
Andreas Kling
5d180d1f99 Everywhere: Rename ASSERT => VERIFY
(...and ASSERT_NOT_REACHED => VERIFY_NOT_REACHED)

Since all of these checks are done in release builds as well,
let's rename them to VERIFY to prevent confusion, as everyone is
used to assertions being compiled out in release.

We can introduce a new ASSERT macro that is specifically for debug
checks, but I'm doing this wholesale conversion first since we've
accumulated thousands of these already, and it's not immediately
obvious which ones are suitable for ASSERT.
2021-02-23 20:56:54 +01:00
Andreas Kling
5625a1d608 Lagom: Build LibTextCodec into Lagom 2021-02-16 19:00:02 +01:00
Luke
5b891b0c36 Lagom/Fuzzers: Add RSA key parser fuzzer
First issue: #5317
2021-02-13 11:03:06 +01:00
Ben Wiederhake
c6a42ab5c3 Everywhere: Remove unnecessary headers 4/4
Arbitrarily split up to make git bisect easier.

These unnecessary #include's were found by combining an automated tool (which
determined likely candidates) and some brain power (which decided whether
the #include is also semantically superfluous).
2021-02-08 18:03:57 +01:00
Ben Wiederhake
6e9fa350bc Fuzz: Remove unused FuzziliJS header 2021-02-08 18:03:57 +01:00
Ben Wiederhake
648f153951 Lagom/Fuzzers: Recommend enabling UBSan, extend instructions 2021-01-30 09:23:18 +01:00
Ben Wiederhake
1cbc01e288 Lagom: Don't include Shell/main.cpp
This caused some confusion: Apparently, clang has no trouble overriding Shell's
main, and this issue only surfaced when I tried to build the fuzzers with
wrong configuration (i.e., without the clang-injected 'main').

The diff is suggested by, and work of, @alimpfard.
2021-01-30 09:23:18 +01:00
Andreas Kling
7ec8f83a7f Lagom+AK: Remove remains of clang -Wconsumed usage
We stopped using that warning ages ago since it confused the compiler.
2021-01-28 09:14:49 +01:00
asynts
7cf0c7cc0d Meta: Split debug defines into multiple headers.
The following script was used to make these changes:

    #!/bin/bash
    set -e

    tmp=$(mktemp -d)

    echo "tmp=$tmp"

    find Kernel \( -name '*.cpp' -o -name '*.h' \) | sort > $tmp/Kernel.files
    find . \( -path ./Toolchain -prune -o -path ./Build -prune -o -path ./Kernel -prune \) -o \( -name '*.cpp' -o -name '*.h' \) -print | sort > $tmp/EverythingExceptKernel.files

    cat $tmp/Kernel.files | xargs grep -Eho '[A-Z0-9_]+_DEBUG' | sort | uniq > $tmp/Kernel.macros
    cat $tmp/EverythingExceptKernel.files | xargs grep -Eho '[A-Z0-9_]+_DEBUG' | sort | uniq > $tmp/EverythingExceptKernel.macros

    comm -23 $tmp/Kernel.macros $tmp/EverythingExceptKernel.macros > $tmp/Kernel.unique
    comm -1 $tmp/Kernel.macros $tmp/EverythingExceptKernel.macros > $tmp/EverythingExceptKernel.unique

    cat $tmp/Kernel.unique | awk '{ print "#cmakedefine01 "$1 }' > $tmp/Kernel.header
    cat $tmp/EverythingExceptKernel.unique | awk '{ print "#cmakedefine01 "$1 }' > $tmp/EverythingExceptKernel.header

    for macro in $(cat $tmp/Kernel.unique)
    do
        cat $tmp/Kernel.files | xargs grep -l $macro >> $tmp/Kernel.new-includes ||:
    done
    cat $tmp/Kernel.new-includes | sort > $tmp/Kernel.new-includes.sorted

    for macro in $(cat $tmp/EverythingExceptKernel.unique)
    do
        cat $tmp/Kernel.files | xargs grep -l $macro >> $tmp/Kernel.old-includes ||:
    done
    cat $tmp/Kernel.old-includes | sort > $tmp/Kernel.old-includes.sorted

    comm -23 $tmp/Kernel.new-includes.sorted $tmp/Kernel.old-includes.sorted > $tmp/Kernel.includes.new
    comm -13 $tmp/Kernel.new-includes.sorted $tmp/Kernel.old-includes.sorted > $tmp/Kernel.includes.old
    comm -12 $tmp/Kernel.new-includes.sorted $tmp/Kernel.old-includes.sorted > $tmp/Kernel.includes.mixed

    for file in $(cat $tmp/Kernel.includes.new)
    do
        sed -i -E 's/#include <AK\/Debug\.h>/#include <Kernel\/Debug\.h>/' $file
    done

    for file in $(cat $tmp/Kernel.includes.mixed)
    do
        echo "mixed include in $file, requires manual editing."
    done
2021-01-26 21:20:00 +01:00
asynts
1a3a0836c0 Everywhere: Use CMake to generate AK/Debug.h.
This was done with the help of several scripts, I dump them here to
easily find them later:

    awk '/#ifdef/ { print "#cmakedefine01 "$2 }' AK/Debug.h.in

    for debug_macro in $(awk '/#ifdef/ { print $2 }' AK/Debug.h.in)
    do
        find . \( -name '*.cpp' -o -name '*.h' -o -name '*.in' \) -not -path './Toolchain/*' -not -path './Build/*' -exec sed -i -E 's/#ifdef '$debug_macro'/#if '$debug_macro'/' {} \;
    done

    # Remember to remove WRAPPER_GERNERATOR_DEBUG from the list.
    awk '/#cmake/ { print "set("$2" ON)" }' AK/Debug.h.in
2021-01-25 09:47:36 +01:00
Luke
1c18d1380f Lagom/Fuzzers: Fix FuzzilliJs build and update patch for new Fuzzilli version
-fsanitize=fuzzer was being added to LINKER_FLAGS from Lagom/CMakeLists,
which we don't want with FuzzilliJs as we want to define the functions
it provides ourselves.
2021-01-24 00:40:49 +01:00
Nico Weber
7e5c49b755 Meta: Add some more documentation on oss-fuzz 2021-01-20 21:00:58 +01:00
AnotherTest
86f50aa74e Shell: Make tests use PASS/FAIL instead of exit codes
There's no guarantee that the last executed command will have a zero
exit code, and so the shell exit code may or may not be zero, even if
all the tests pass.
Also changes the `test || echo fail && exit` to
`if not test { echo fail && exit }`, since that's nicer-looking.
2021-01-19 08:19:43 +01:00