Commit graph

344 commits

Author SHA1 Message Date
Zaggy1024
bbc053fa56 LibVideo: Add LibVideo test to Lagom 2022-10-09 20:32:40 -06:00
Andreas Kling
a4ccdce011 LibWeb: Generate WebContent IPC endpoints for Lagom's LibWebView
This is pretty hackish but should get us back to building.
2022-10-06 16:35:53 +02:00
Timothy Flynn
e8410bc2ee Meta: Hide WebView behind ENABLE_LAGOM_LIBWEB CMake option
If ENABLE_LAGOM_LIBWEB is OFF, this will fail to build.
2022-10-06 14:12:54 +02:00
Andreas Kling
4d762fd3a6 Lagom: Add Build WebContentClient as part of Lagom's LibWebView
This will allow us to use it in Ladybird. :^)
2022-10-06 09:51:04 +02:00
vl-ms
d8b6f38748 Lagom: Change dl to ${CMAKE_DL_LIBS} for portability 2022-10-03 20:36:04 +01:00
Daniel Bertalan
41a8f3f40f Lagom: Link LibMain statically
This matches how LibMain is used within Serenity. This commit makes it
possible to build Lagom with LTO. Previously, `serenity_main` functions
would be dead-stripped away, as the linker could prove that nothing from
the executable ever called them.
2022-10-01 20:39:48 +02:00
Diego Iastrubni
18257604eb Lagom: Win32 support baby steps
This is the initial port of Lagom to win32. This will enable developers
to use Lagom as an alternative to vanilla STL/StandardC++Library - which
gives a much richer environment (think QtCore - but modern).

My main incentive - is to have a native Windows Ladybird working.

I am starting with AK, which does not yet fully compile (on mingw). When
AK is compiling (currently fails building StringBuffer.cpp) - I will
continue to LibCore and then the rest of the user space libraries
(excluding the GUI, which will be another different effort).

Most of the code is happily stollen from Andrew Kaster's fork - he
deserves the credit.

Co-authored-by: Andrew Kaster <akaster@serenityos.org>
2022-09-29 17:01:22 +01:00
Andreas Kling
34c232def7 Lagom: Build DOMTreeModel from LibWebView
This will be used in Ladybird.
2022-09-25 12:10:28 +02:00
Andreas Kling
5778da765d Lagom: Build a tiny slice of LibGUI
We now build Model, ModelIndex and Icon from LibGUI. These will be used
in Ladybird to view the DOM tree model.
2022-09-25 12:09:36 +02:00
Andrew Kaster
d6a741af6c Meta: Don't build Core::LocalServer on Android
We won't be able to use local servers on Android without some serious
Android work to create background tasks, so just disable this for now,
as it currently relies on Core::Account to take over from SystemServer.
2022-09-22 11:07:21 -04:00
Linus Groh
edfef8e2f5 Everywhere: Rename WrapperGenerator to BindingsGenerator
This code generator no longer creates JS wrappers for platform objects
in the old sense, instead they're JS objects internally themselves.
Most of what we generate now are prototypes - which can be seen as
bindings for the internal C++ methods implementing getters, setters, and
methods - as well as object constructors, i.e. bindings for the internal
create_with_global_object() method.

Also tweak the naming of various CMake glue code existing around this.
2022-09-21 23:06:08 +01:00
Sam Atkins
c4668053d1 LibIDL+WrapperGenerator: Move IDL code into a library
IDL function overload resolution requires knowing each IDL function's
parameters and their types at runtime. The simplest way to do that is
just to make the types the generator uses available to the runtime.

Parsing has moved to LibIDL, but code generation has not, since that is
very specific to WrapperGenerator.
2022-09-17 21:27:17 +02:00
Timothy Flynn
6290a25224 Meta: Enable vptr sanitation by default for Lagom UBSAN builds 2022-09-15 09:45:13 +02:00
Timothy Flynn
10727ade4c Meta: Add Lagom option to disable building LibWeb
Compiling LibWeb for Lagom can add quite a bit of compile time. Add a
CMake option to disable it locally.
2022-09-13 17:05:20 +02:00
davidot
7948897688 LibJS: Import the libjs-test262-runner from linusg/libjs-test262
For now this is a lagom only application as it is not compatible with
serenity in its current state.

The only change is that it is released under a different license with
permission from all the authors.
2022-09-11 20:25:51 +01:00
davidot
d4736d17ae LibJS: Allow negative pointers in Value
Also ensure that all a nullptr input gives null object and you don't
accidentally dereference a nullptr.
2022-09-10 00:05:32 +01:00
Andrew Kaster
5ab3fcf710 Meta: Update jakt build support for fully bootstrapped compiler
Remove the Corrosion dependency, and use the now-builtin
add_jakt_executable function from the Jakt install rules to build our
example application.

By using find_package(Jakt), we now have to set ENABLE_JAKT manually on
both serenity and Lagom at the same time, so the preferred method to do
this for now is:

    cmake -B Build/superbuild<arch><toolchain> \
          -S Meta/CMake/Superbuild \
          -DENABLE_JAKT=ON \
          -DJAKT_SOURCE_DIR=/path/to/jakt

Where omitting JAKT_SOURCE_DIR will still pull from the main branch of
SerenityOS/jakt. This can be done after runing Meta/serenity.sh run.
2022-09-09 11:23:42 +02:00
Timothy Flynn
9e860d973e LibLocale: Move locale source files to the LibLocale library
Everything is now setup to create the LibLocale library and link it
where needed.
2022-09-05 14:37:16 -04:00
Timothy Flynn
b2d2bb43ce LibLocale: Move locale test files to the LibLocale folder 2022-09-05 14:37:16 -04:00
Timothy Flynn
43a3471298 LibLocale: Move locale source files to the LibLocale folder
These are still included in LibUnicode, but this updates their location
and the include paths of other files which include them.
2022-09-05 14:37:16 -04:00
Timothy Flynn
89d1813b5d LibUnicode: Move CLDR data generators to a LibLocale subfolder
To prepare for placing all CLDR generated data in a new library,
LibLocale, this moves the code generators for the CLDR data to the
LibLocale subfolder.
2022-09-05 14:37:16 -04:00
kleines Filmröllchen
c91511b883 Meta+Tests: Allow running FLAC spec tests
The FLAC "spec tests", or rather the test suite by xiph that exercises
weird FLAC features and edge cases, can be found at
https://github.com/ietf-wg-cellar/flac-test-files and is a good
challenge for our FLAC decoder to become more spec compliant. Running
these tests is similar to LibWasm spec tests, you need to pass
INCLUDE_FLAC_SPEC_TESTS to CMake.

As of integrating these tests, 23 out of 63 fail. :yakplus:
2022-09-02 23:54:50 +01:00
Andrew Kaster
2aaaee6744 CI+Lagom: Add Lagom Android CI for arm64-v8a on NDK 24 with API level 30
This will let us validate that we aren't breaking any library compile
steps for arm64.
2022-07-21 16:37:15 +02:00
kleines Filmröllchen
3f59356c79 LibAudio: Rename ConnectionFromClient to ConnectionToServer
The automatic nomenclature change for IPC sockets got this one wrong.
2022-07-19 11:17:45 +01:00
Andrew Kaster
9c2211f246 Meta: Add Android cross-compile support to Lagom
This is a start to properly letting us cross-compile Lagom where both
the Tools and the BUILD_LAGOM=ON build are using Lagom CMakeLists.

The initial cut allows an Android build to succeed, more or less.
But there are issues with namespace clashes when using FetchContent with
this approach.
2022-07-19 10:44:02 +01:00
Linus Groh
9244f1697d LibWeb: Add definitions from '2.2.1. Methods' in the Fetch spec 2022-07-14 00:42:26 +01:00
Timothy Flynn
ea78bac36d LibUnicode: Parse and generate per-locale plural rules from the CLDR
Plural rules in the CLDR are of the form:

"cs": {
    "pluralRule-count-one": "i = 1 and v = 0 @integer 1",
    "pluralRule-count-few": "i = 2..4 and v = 0 @integer 2~4",
    "pluralRule-count-many": "v != 0 @decimal 0.0~1.5, 10.0, 100.0 ...",
    "pluralRule-count-other": "@integer 0, 5~19, 100, 1000, 10000 ..."
}

The syntax is described here:
https://unicode.org/reports/tr35/tr35-numbers.html#Plural_rules_syntax

There are up to 2 sets of rules for each locale, a cardinal set and an
ordinal set. The approach here is to generate a C++ function for each
set of rules. Each condition in the rules (e.g. "i = 1 and v = 0") is
transpiled to a C++ if-statement within its function. Then lookup tables
are generated to match locales to their generated functions.

NOTE: -Wno-parentheses-equality is added to the LibUnicodeData compile
flags because the generated plural rules have lots of extra parentheses
(because e.g. we need to selectively negate and combine rules). The code
to generate only exactly the right number of parentheses is quite hairy,
so this just tells the compiler to ignore the extras.
2022-07-08 11:51:54 +02:00
Andrew Kaster
0a62fcfbdf Meta: Build select Services in Lagom
Add overrides for serenity_bin and serenity_lib to allow the actual
CMakeLists.txt from Userland to be used to build as many services as
possible without adding more clutter to Meta/Lagom/CMakeLists.txt
2022-07-06 14:24:23 +02:00
Andrew Kaster
2b29e611fe Meta: Rename Lagom library target names from LagomFoo to LibFoo
This matches the target names for the main serenity build, and will make
simplifying the Lagom build much easier going forward.

The LagomFoo name came from a time when we had both library builds in
the same CMake generated project and needed to deconflict the names.
2022-07-06 14:24:23 +02:00
Luke Wilde
58f882200c LibWeb: Add the ability to retrieve a WebGL context from getContext 2022-06-13 21:45:27 +01:00
DexesTTP
5b56f3ed5b Lagom: Compile headless-browser on Lagom :^) 2022-05-29 23:00:04 +01:00
DexesTTP
63e841c3ad Lagom: Build LibWeb on Lagom 2022-05-29 23:00:04 +01:00
DexesTTP
b388aa7876 Lagom: Disable the unused-private-field warning on Lagom's clang builds
With the compilation of LibWeb, there's now quite a few cases where this
warning gets triggered. Rather than trying to fix them all right away,
we simply disable the warning for now.

This workaround was proposed by Andrew Kaster and BertalanD who promised
to open an issue about it!
2022-05-29 23:00:04 +01:00
DexesTTP
915ac9edd6 Lagom: Compile LibWebSocket on Lagom 2022-05-29 23:00:04 +01:00
DexesTTP
3c8411eee4 Lagom: Compile all sources of LibGfx
The filters were missing from the compiled sources.
2022-05-29 23:00:04 +01:00
Andrew Kaster
ca42da23c2 Meta+Userland: Add jakt as an optional Lagom Tool
We can now use ENABLE_JAKT to pull jakt as a host tool and use it to
pre-process .jakt files into .cpp files for use in serenity applications
2022-05-23 23:05:45 +02:00
Michiel Visser
d6a5b11f04 LibCompress: Implement Brotli decompressor
This implements the BrotliDecompressionStream, which is a Core::Stream
that can decompress another Core::Stream.
2022-05-21 22:41:40 +02:00
Linus Groh
ee721978f6 Meta: Move compiler flags into standalone CMake files
This way we can have all of them in a single place, similar to how we
structure options added via the serenity_option() macro.
2022-05-14 17:53:06 +02:00
Jelle Raaijmakers
8cfabbcd93 Tests: Implement reference image testing for LibGL
Each LibGL test can now be tested against a reference QOI image.
Initially, these images can be generated by setting `SAVE_OUTPUT` to
`true`, which will save a bunch of QOI images to `/home/anon`.
2022-04-17 09:58:29 +04:30
Simon Wanner
bf6d64c199 LibGfx: Add a loader the Web Open Font Format (WOFF)
This format is 'just' a zlib compressed wrapper for TrueType/OpenType
which makes implementing it rather convenient. :^)
2022-04-09 23:48:18 +02:00
Simon Wanner
206d6ece55 LibGfx: Move other font-related files to LibGfx/Font/ 2022-04-09 23:48:18 +02:00
Simon Wanner
6f8fd91f22 LibGfx: Move TTF files from TrueTypeFont/ to Font/TrueType/ 2022-04-09 23:48:18 +02:00
Stephan Unverwerth
5bb76e9b63 LibGL+LibGPU+LibSoftGPU: Load SoftGPU driver dynamically
This loads libsoftgpu.so during GLContext creation and instantiates the
device class which is then passed into the GLContext constructor.
2022-04-06 11:32:24 +02:00
Timothy Flynn
b36c3a68d8 js: Convert non-UTF-8 encoded files to UTF-8 before parsing 2022-04-05 00:14:29 +01:00
Nico Weber
fd82121319 Tests: Add some test coverage for the TTF parser
This is in Tests/LibTTF instead of Tests/LibGfx because Tests/LibGfx
depends on serenity's file system layout and can't run in lagom,
but this new test runs just fine in lagom.
2022-04-03 19:16:03 +02:00
Nico Weber
9b24600779 Meta: Give Lagom build errors colored diagnostics
Non-lagom builds get this from the root CMakeLists.txt file,
but lagom builds didn't before this change.
2022-04-03 19:16:03 +02:00
Tim Schumacher
bf502ae3b0 CMake: Allow building fuzzing targets without libFuzzer or OSS-Fuzz 2022-03-31 22:11:04 -07:00
Tim Schumacher
e3519b8e5c Meta: Rename Fuzzer flags to ENABLE_FUZZERS_{LIBFUZZER,OSSFUZZ} 2022-03-31 22:11:04 -07:00
Matthew Olsson
5f9d35909d LibPDF: Move font files into their own directory 2022-03-31 18:10:45 +02:00
Kenneth Myhra
de7d333d43 markdown-check: Port to LibMain 2022-03-29 21:28:29 -07:00
Karol Kosek
dcb24e943d Tests: Add a basic UTF-8 to UTF-8 LibTextCodec test 2022-03-29 01:01:32 +02:00
Ali Mohammad Pur
67357fe984 LibXML: Add a fairly basic XML parser
Currently this can parse XML and resolve external resources/references,
and read a DTD (but not apply or verify its rules).
That's good enough for _most_ XHTML documents as the HTML 5 spec
enforces its own rules about document well-formedness, and does not make
use of XML DTDs (aside from a list of predefined entities).

An accompanying `xml` utility is provided that can read and dump XML
documents, and can also run the XML conformance test suite.
2022-03-28 23:11:48 +02:00
Nico Weber
20c6dabaff Lagom: Build with -fsigned-char
When building on an arm host system, char defaults to unsigned,
leading to errors such as:

  serenity/AK/StringBuilder.cpp:198:20:
    error: comparison is always true due to limited range of data type
           [-Werror=type-limits]
    198 |             if (ch >= 0 && ch <= 0x1f)
        |

Building with -fsigned-char makes things work like on Intel, and
it's what we already do in Kernel/CMakeLists.txt for the same reasons.
2022-03-20 19:53:47 -07:00
Brian Gianforcaro
66e7ac1954 Meta: Error out on find_program errors with CMake less than 3.18
We have seen some cases where the build fails for folks, and they are
missing unzip/tar/gzip etc. We can catch some of these in CMake itself,
so lets make sure to handle that uniformly across the build system.

The REQUIRED flag to `find_program` was only added on in CMake 3.18 and
above, so we can't rely on that to actually halt the program execution.
2022-03-19 15:01:22 -07:00
Itamar
3a71748e5d Userland: Rename IPC ClientConnection => ConnectionFromClient
This was done with CLion's automatic rename feature and with:
find . -name ClientConnection.h
    | rename 's/ClientConnection\.h/ConnectionFromClient.h/'

find . -name ClientConnection.cpp
    | rename 's/ClientConnection\.cpp/ConnectionFromClient.cpp/'
2022-02-25 22:35:12 +01:00
Ali Mohammad Pur
bed129a69f LibTest+Spreadsheet: Add some basic spreadsheet runtime behaviour tests
As there's a somewhat active development going on, let's keep the
expected behaviour under tests to make sure nothing blows up :^)
2022-02-23 03:17:12 +03:30
Filiph Sandström
b9dbe248aa Lagom: Port LibSyntax
LibSyntax was already building for lagom without any extra changes
so let's just enable it :^)
2022-02-21 16:42:23 +00:00
Andrew Kaster
0c95d9962c Lagom: Add two-stage build for Fuzzers to enable fuzzing generated code
This allows us to fuzz the generated unicode and timezone database
helpers, and to fuzz things like LibJS using Fuzzilli to get proper
coverage of our unicode handling code.

Update the Azure CI to use the new two-stage build as well, and cleanup
some unused CMake options there.
2022-02-20 19:04:59 +00:00
Timothy Flynn
636a6a2fb8 Meta: Add a CPack installation target for js(1)
This adds a CPack configuration to generate a release package for js(1).
Our current CMake requirement is 3.16, which doesn't have a great story
for automatically installing a binary target's library dependencies. If
we eventually require CMake 3.21 or above, we can remove the helper
.cmake file added here in lieu of RUNTIME_DEPENDENCIES.
2022-02-09 12:19:56 +03:30
kleines Filmröllchen
4931c88b13 LibGUI: Remove GML prefix in favor of proper namespace
Prefixes are very much a C thing which we don't need in C++. This commit
moves all GML-related classes in LibGUI into the GUI::GML namespace, a
change somewhat overdue.
2022-02-07 18:39:50 +01:00
Morten Larsen
2c3b297895 Lagom: Exclude libraries with X86 code when building for macOS on Arm
The CMakeLists.txt for Lagom contains a few libraries and executables
with X86-specific code. By excluding those libraries, Lagom builds
for macOS on Arm as well. The places are marked FIXME to be removed
when the libraries will build for Arm.
2022-02-06 03:15:00 +00:00
davidot
b40308d0a4 Tests+LibJS: Add very simple bytecode LibJS tests
These tests are not meant as a replacement to test-js with the -b option
but are meant to test simple cases until that works.
Before this it was very easy to accidentally break bytecode since no
tests were run in bytecode mode. This hopefully makes it easier to spot
such regressions :^).
2022-01-25 23:26:14 +00:00
Nico Weber
3831e5fcff Utilities+Lagom: Remove test-crypto
After 4d5ffd364a, Tests/{LibCrypto,LibTLS} test the same things
and test-crypto has been redundant since then.
2022-01-22 01:44:05 +00:00
mjz19910
2cf02c6f38 adjtime: Port to LibMain 2022-01-21 01:52:22 +01:00
Lucas CHOLLET
9e1e80954b gml-format: Port to LibMain 2022-01-14 19:42:19 +02:00
Lucas CHOLLET
affef5d1a9 disasm: Port to LibMain 2022-01-14 19:42:19 +02:00
Timothy Flynn
b493c2ca90 LibTimeZone: Add a unit test for generated time zone data 2022-01-11 00:36:45 +01:00
Federico Guerinoni
9aa556fdf0 Userland: Port wasm to LibMain 2022-01-09 04:26:16 -08:00
Lucas CHOLLET
ed0f4bdfaf Shell: Port to LibMain 2022-01-09 03:22:10 +03:30
Simon Woertz
47f4e54cdc Lagom: Execute LibPDF unit tests on the host
Add lagom unit tests for `LibPDF`.
2022-01-08 18:57:55 +01:00
Timothy Flynn
498b741434 LibUnicode: Use LibTimeZone's list of time zone names
LibUnicode no longer needs to generate a list of time zone names that it
parsed from metaZones.json. We can defer to the TZDB for a golden list
of time zones.
2022-01-08 12:45:34 +01:00
Timothy Flynn
8669b25cea LibTimeZone+Meta: Add plumbing for an IANA Time Zone Database generator
The IANA Time Zone Database contains data needed, at least, for various
JavaScript objects. This adds plumbing for a parser and code generator
for this data. The generated data will be made available by LibTimeZone,
much like how UCD and CLDR data is available through LibUnicode.
2022-01-08 12:45:34 +01:00
Timothy Flynn
363eafbc44 Meta: Remove -ldl linkage from LagomUnicode
This was needed for dlopen() on some systems, but is no longer needed as
of commit 1116a29c19.
2022-01-08 12:45:34 +01:00
mjz19910
10ec98dd38 Everywhere: Fix spelling mistakes 2022-01-07 15:44:42 +01:00
Andrew Kaster
c62c10caf0 Meta+CI+Documentation: Bump host gcc requirement up to gcc 11
Bump macOS CI version to macOS 11 while we're here.
2022-01-07 11:02:30 +01:00
Guilherme Gonçalves
f91d471843 LibSQL: Implement LIKE SQL expressions 2022-01-07 10:50:39 +03:30
tuftedocelot
5810467c97 Build: Remove gzip -k usage in PCI/USB ID files and crypt for OpenBSD
OpenBSD gzip does not have the -k flag to keep the original after
extraction. Work around this by copying the original gzip to the dest
and then extracting. A bit of a hack, but only needs to be done for the
first-time or rebuilds

OpenBSD provides crypt in libc, not libcrypt. Adjust if/else to check
for either and proceed accordingly

Remove outdated OpenBSD checks when building the toolchain
2021-12-29 03:46:56 -08:00
Stephan Unverwerth
ad3d5d43bd LibGL+LibSoftGPU: Move rendering related code to LibSoftGPU library
This introduces a new library, LibSoftGPU, that incorporates all
rendering related features that formerly resided within LibGL itself.

Going forward we will make both libraries completely independent from
each other allowing LibGL to load different, possibly accelerated,
rendering backends.
2021-12-24 05:10:28 -08:00
Ali Mohammad Pur
f20e0036bd Meta: Add a cmake flag for building for compiler explorer
We don't need to build tests/binaries for CE.
2021-12-22 20:49:28 +03:30
Timothy Flynn
3fd53baa25 LibUnicode: Dynamically load the generated UnicodeData symbols
The generated data for libunicodedata.so is quite large, and loading it
is a price paid by nearly every application by way of depending on
LibRegex. In order to defer this cost until an application actually uses
one of the surrounding APIs, dynamically load the generated symbols.

To be able to load the symbols dynamically, the generated methods must
have demangled names. Typically, this is accomplished with `extern "C"`
blocks. The clang toolchain complains about this here because the types
returned from the generators are strictly C++ types. So to demangle the
names, we use the asm() compiler directive to manually define a symbol
name; the caveat is that we *must* be sure the symbols are unique. As an
extra precaution, we prefix each symbol name with "unicode_". For more
details, see: https://gcc.gnu.org/onlinedocs/gcc/Asm-Labels.html

This symbol loader used in this implementation provides the additional
benefit of removing many [[maybe_unused]] attributes from the LibUnicode
methods. Internally, if ENABLE_UNICODE_DATABASE_DOWNLOAD is OFF, the
loader is able to stub out the function pointers it returns.

Note that as of this commit, LibUnicode is still directly linked against
LibUnicodeData. This commit is just a first step towards removing that.
2021-12-21 13:09:49 -08:00
Simon Woertz
c2df6b597a Lagom: Add argument WORKING_DIRECTORY to lagom_test(...)
So far the working directory was set in some cases using
`set_tests_properties(...)`, but this requires to know which name is
picked by `lagom_test(...)` when calling `add_test(...)`.
In case of adding multiple test cases using a globbing pattern this
would require to duplicate code to construct the test name from the file
name.
2021-12-15 10:33:27 -08:00
Hendiadyoin1
7a27ecc135 Tests: Add a simple LibGL render-test
At the moment we just check if we *can* render a simple triangle, we do
not yet actually test if the image is indeed the triangle we wanted.

This test also outputs the rendered image when GL_DEBUG is enabled to a
file called "picture.bmp" for manual verification.

Co-authored-by: sunverwerth <s.unverwerth@serenityos.org>
2021-11-29 23:17:05 +03:30
Hendiadyoin1
3a4dd5ff87 Lagom: Add LibGL to the libraries 2021-11-29 23:17:05 +03:30
Hendiadyoin1
849089c406 Lagom: Disable implicit-const-int-float-conversion warnings 2021-11-29 23:17:05 +03:30
Timothy Flynn
a2ea704d21 Meta: Define LagomMain outside of the BUILD_LAGOM branch
This allows code generators to use LagomMain. Otherwise, during CI, they
are built during the superbuild without BUILD_LAGOM=ON.
2021-11-23 22:58:05 +01:00
Linus Groh
cfecfbb214 js: Port to LibMain :^)
This wasn't particularly difficult, and there's not much use for the
nicer interface yet either. While unveil() is of limited use in js(1)
as it should be able to open arbitrary files, I feel like we should be
able to add a pledge() call.
2021-11-22 23:07:43 +01:00
Linus Groh
ba0f89a4d1 Lagom: Add LibMain as a lagom_lib() 2021-11-22 23:07:43 +01:00
Brendan Coles
91de60d912 Lagom/Fuzzers: Add fuzzer for PDF document 2021-10-30 10:33:56 -07:00
Nico Weber
4d555e8b95 Lagom: Do not use -fno-sematic-interposition in fuzzer builds
Apparently it breaks the fuzzer build. There's probably a better fix
for this, but for now just unbreak the fuzzer build.
Keep this for non-fuzzer builds though since it's apparently a 17%
speedup for running test262 tests :^)
2021-10-16 14:45:06 +01:00
Nico Weber
ec9488a58c Lagom: Build with -fno-no-semantic-interposition
Lagom: Build with -fno-no-semantic-interposition

We build with this in non-lagom builds, and serenity's gcc even adds it
to its CC1_SPEC. Let's use it for lagom too.

Reduces the number of dynamic relocations in liblagom-js.so.0.0.0 (per
`objdump -R`) from 15133 to 14534, and increases its size back to 91M
(95156800 bytes), probably due to more inlining being possible.
This might help perf of lagom binaries.
2021-10-15 21:59:42 +01:00
Nico Weber
b11d660ff8 Lagom: Build with -fno-exceptions
We build with this in non-lagom builds, so there's no reason not
to use it in lagom builds as well.

Reduces the size of liblagom-js.so.0.0.0 from 94M to 90M
(from 98352784 to 93831056 bytes to be exact).
2021-10-15 21:59:42 +01:00
Ben Wiederhake
3f88d65b78 markdown-checker: New tool that checks document links 2021-10-10 15:18:55 -07:00
Ben Wiederhake
6d99b7b72e Meta: Re-enable warnings for deprecated copies also for Lagom 2021-10-10 21:21:35 +01:00
Idan Horowitz
9958277317 Meta: Disable -Wmaybe-uninitialized for Lagom 2021-10-07 21:56:03 +03:00
davidot
ac2c3a73b1 LibJS: Add a specific test for invalid unicode characters in the lexer
Also fixes that it tried to make substrings past the end of the source
if we overran the source length.
2021-10-03 17:42:05 +02:00
Andrew Kaster
04d91e90c2 Meta: Fix Lagom RPATH for non-Ubuntu Linux and macOS hosts
Multi-lib distros like Gentoo and Fedora install lagom-core.so into
lagom-install/lib64 rather than lib. Set the install RPATH based on
CMAKE_INSTALL_LIBDIR to avoid the wrong path being set in the binaries.

Also apply macOS specific RPATH rules to fix the build on that platform.
2021-09-16 15:47:13 +00:00
Andrew Kaster
368d2ace55 Meta: Add FIXME for not setting BUILD_SHARED_LIBS in Lagom build
This is really the business of the consuming project. We will need to
make changes to libjs-test262 and to oss-fuzz to address this properly.
2021-09-15 19:04:52 +04:30
Andrew Kaster
b5c98ede08 Meta: Switch to a SuperBuild that splits host and target builds
Replace the old logic where we would start with a host build, and swap
all the CMake compiler and target variables underneath it to trick
CMake into building for Serenity after we configured and built the Lagom
code generators.

The SuperBuild creates two ExternalProjects, one for Lagom and one for
Serenity. The Serenity project depends on the install stage for the
Lagom build. The SuperBuild also generates a CMakeToolchain file for the
Serenity build to use that replaces the old toolchain file that was only
used for Ports.

To ensure that code generators are rebuilt when core libraries such as
AK and LibCore are modified, developers will need to direct their manual
`ninja` invocations to the SuperBuild's binary directory instead of the
Serenity binary directory.

This commit includes warning coalescing and option style cleanup for the
affected CMakeLists in the Kernel, top level, and runtime support
libraries. A large part of the cleanup is replacing USE_CLANG_TOOLCHAIN
with the proper CMAKE_CXX_COMPILER_ID variable, which will no longer be
confused by a host clang compiler.
2021-09-15 19:04:52 +04:30
Andrew Kaster
904a268872 Meta: Move all options to targetname_options.cmake files
This common strategy of having a serenity_option() macro defined in
either the Lagom or top level CMakeLists.txt allows us to do two things:

First, we can more clearly see which options are Serenity-specific,
Lagom-specific, or common between the target and host builds.

Second, it enables the upcoming SuperBuild changes to set() the options
in the SuperBuild's CMake cache and forward each target's options to the
corresponding ExternalProject.
2021-09-15 19:04:52 +04:30
Andrew Kaster
6e7cc40b18 Meta: Add Meta/CMake to the CMAKE_MODULE_PATH for Serenity and Lagom
This makes it so we don't need to specify the full path to all the
helper scripts we include() from different places in the codebase and
feels a lot cleaner.
2021-09-15 19:04:52 +04:30
Peter Elliott
8d2c04821f Tests: Test LibMarkdown against commonmark test suite
TestCommonmark runs the CommonMark test suite
(https://spec.commonmark.org/0.30/spec.json) against LibMarkdown.
Currently 44/652 tests pass.
2021-08-31 16:53:51 +02:00
Brian Gianforcaro
619200774b CMake: Add custom target to build only the generated sources
This is needed so all headers and files exist on disk, so that
the sonar cloud analyzer can find them when executing the compilation
commands contained in compile_commands.json, without actually building.

Co-authored-by: Andrew Kaster <akaster@serenityos.org>
2021-08-30 16:44:16 +02:00
Andrew Kaster
63956b36d0 Everywhere: Move all host tools into the Lagom/Tools subdirectory
This allows us to remove all the add_subdirectory calls from the top
level CMakeLists.txt that referred to targets linking LagomCore.

Segregating the host tools and Serenity targets helps us get to a place
where the main Serenity build can simply use a CMake toolchain file
rather than swapping all the compiler/sysroot variables after building
host libraries and tools.
2021-08-28 08:44:17 +01:00
Andrew Kaster
e88761b2b9 Meta+LibUnicode: Move unicode_data helper to Meta/CMake
Moving this helper CMake file to the centralized Meta/CMake folder helps
to get a better grasp on what extra files are required for the build,
and what files are generated.

While we're at it, don't use add_compile_definitions for
ENABLE_UNICODE_DATA, which only needs to be seen by LibUnicode sources.
2021-08-28 08:44:17 +01:00
Timothy Flynn
831f90c9d4 Meta: Semi-automatically detect ccache for Lagom builds
The top-level CMakeLists.txt already automatically detects ccache, but
CI will invoke CMake with Lagom's CMakeLists.txt. Add an option to Lagom
to do the same detection.
2021-08-17 13:31:43 +01:00
Andrew Kaster
40b0767d88 Meta: Add BUILD_SHARED_LIBS option for Lagom builds
This standard CMake option controls whether add_library() calls will
use STATIC or SHARED by default. The flag is set to on by default
since that's what we want for normal CI jobs and local builds and the
test262 runner, but disabled for oss-fuzz builds.

This should finally fix the oss-fuzz build after it was broken in #9017

oss-fuzz un-breakage was verified by running the following commands in
the oss-fuzz repo:

python infra/helper.py build_image serenity
python infra/helper.py build_fuzzers --sanitizer address --engine afl \
    --architecture x86_64 serenity /path/to/local/checkout/Meta/Lagom
python infra/helper.py check_build --sanitizer address --engine afl \
    --architecture x86_64 serenity
2021-08-02 09:05:28 +02:00
Timothy Flynn
d485cf29d7 LibRegex+LibUnicode: Begin implementing Unicode property escapes
This supports some binary property matching. It does not support any
properties not yet parsed by LibUnicode, nor does it support value
matching (such as Script_Extensions=Latin).
2021-07-30 21:26:31 +01:00
Andrew Kaster
32d076ef54 Lagom: Add proper install rules
Create the proper export files to allow Lagom to be a well-behaved
ExternalProject, based on the example project from the cmake-init
project generator here:

https://github.com/friendlyanon/cmake-init-shared-static
2021-07-29 21:46:25 +01:00
Linus Groh
7392a73066 Lagom: Exclude GMLAutocompleteProvider.cpp from GML library sources 2021-07-29 21:46:25 +01:00
Andrew Kaster
b6e5117f65 Lagom: Change to shared library build for all Lagom code
Split the Lagom build into shared libraries to match the Serenity build.

This reduces the cognitive load when trying to edit the Lagom CMakeLists
significantly. It also reduces the amount of source files that must be
compiled to run each test or host program significantly.

Also re-organize all the build rules into sections. And reorganize the
CMakeLists file in general.
2021-07-29 21:46:25 +01:00
Andrew Kaster
af9be6e093 LibTTF/LibGfx: Remove circular dependency by merging LibTTF into LibGfx
LibTTF has a concrete dependency on LibGfx for things like Gfx::Bitmap,
and LibGfx has a concrete dependency in the TTF::Font class in
Gfx::FontDatabase. This circular dependency works fine for Serenity and
Lagom Linux builds of the two libraries. It also works fine for static
library builds on Lagom macOS builds.

However, future changes will make Lagom use shared libraries, and
circular library dependencies are not tolerated in macOS.
2021-07-29 21:46:25 +01:00
Timothy Flynn
98d8274040 Meta: Add LibUnicode (and its tests) to Lagom
This is primarily to allow using LibUnicode within LibJS and its REPL.

Note: this seems to be the first time that a Lagom dependency requires
generated source files. For this to work, some of Lagom's CMakeLists.txt
commands needed to be re-organized to include the CMake files that fetch
and parse UnicodeData.txt. The paths required to invoke the generator
also differ depending on what is currently building (SerenityOS vs.
Lagom as part of the Serenity build vs. a standalone Lagom build).
2021-07-26 17:03:55 +01:00
Jan de Visser
99dc3469e8 Lagom: Exclude LibSQL/SQLClient.cpp and the sql utility from Lagom
Lagom doesn't seem to like IPC services, and sql needs SQLClient.
2021-07-08 17:55:59 +04:30
Gunnar Beutner
5c3f781031 Meta: Fix building Lagom on Linux 2021-07-06 02:10:37 +02:00
Gunnar Beutner
01db5205ab LibThreading: Fix building the library on macOS 2021-07-06 00:06:32 +02:00
Jan de Visser
4198f7e1af LibSQL: Move Lexer and Parser machinery to AST directory
The SQL engine is expected to be a fairly sizeable piece of software.
Therefore we're starting to restructure the codebase for growth.
2021-06-24 00:36:53 +02:00
coderdreams
9ffb3e7e30 LibCore: Add unit test for File::read_line 2021-06-22 18:54:40 +04:30
Peter Bocan
4d5ffd364a LibCrypto+LibTLS: Split and move test suite into Tests directory
This change splits test-crypto.cpp from Userland into separate test
suites located in Tests/ directory.
2021-06-19 19:05:36 +04:30
sin-ack
e2215cc0e1 Lagom: Add an install target
This is used by libjs-test262-runner to be able to copy the libraries
within its own build directory in order to link with them.
2021-06-17 21:20:24 +01:00
Ali Mohammad Pur
1414c7b049 LibJS: Add a basic pass manager and add some basic passes
This commit adds a bunch of passes, the most interesting of which is a
pass that merges blocks together, and a pass that places blocks that
flow into each other next to each other, and a very simply pass that
removes duplicate basic blocks.
Note that this does not remove the jump at the end of each block in that
pass to avoid scope creep in the passes.
2021-06-15 22:06:33 +04:30
x-yl
ac712b07f9 Meta: Fuzz the LibIMAP Parser 2021-06-11 23:58:28 +04:30
Ali Mohammad Pur
0e4431af33 Meta: Run the Wasm spec tests in CI
Since LibWasm is still not capable of passing all of the spec tests,
ignore failing tests, only fail the build if some segfault/abort/etc
occurs.
2021-05-27 17:28:41 +04:30
Ali Mohammad Pur
b3c13c3e8a LibWasm+Meta: Add test-wasm and optionally test the conformance tests
This only tests "can it be parsed", but the goal of this commit is to
provide a test framework that can be built upon :)
The conformance tests are downloaded, compiled* and installed only if
the INCLUDE_WASM_SPEC_TESTS cmake option is enabled.
(*) Since we do not yet have a wast parser, the compilation is delegated
to an external tool from binaryen, `wasm-as`, which is required for the
test suite download/install to succeed.
This *does* run the tests in CI, but it currently does not include the
spec conformance tests.
2021-05-21 00:15:23 +01:00
Ali Mohammad Pur
f137c1bfaa LibJS+LibTest: Move out the test-js test runner into LibTest 2021-05-18 18:48:15 +01:00
Ali Mohammad Pur
dfb2178519 Meta: Build LibWasm and the wasm utility as part of Lagom 2021-05-17 23:25:30 +02:00
Andrew Kaster
d81f52e529 Lagom: Enable sanitizer builds with gcc
Previously the CMake options for -fsanitize=address, thread and
undefined were gated behind clang, which was unecessary. Only
-fsanitize=fuzzer is clang-only.
2021-05-14 08:34:00 +01:00
DexesTTP
5963f6f9ff LibCrypto: Split BigInteger operations into an Algorithms class
Since the operations are already complicated and will become even more
so soon, let's split them into their own files. We can also integrate
the NumberTheory operations that would better fit there into this class
as well.

This commit doesn't change behaviors, but moves the allocation of some
variables into caller classes.
2021-05-13 19:18:07 +01:00
Dylan Katz
984b6e08cf LibSQL: Add a syntax highlighter 2021-05-09 09:39:05 +02:00
Brian Gianforcaro
7e28ecc305 Meta + Lagom: Fix paths after movement of tests to Tests/
Not sure how this didn't cause a build break when I originally
moved them in #6891. Better fix them now.
2021-05-08 00:04:10 +01:00
Brian Gianforcaro
9720ad3901 Tests: Move Userland/Utilities/test-js to Tests/LibJS 2021-05-08 00:04:10 +01:00
Carlos César Neves Enumo
69e2431c64 Lagom/CMake: Add -fconstexpr-steps for AppleClang 2021-05-01 07:38:43 +02:00
Idan Horowitz
33fdd402b5 Meta: Increase fconstexpr-steps for Clang Lagom builds
Clang's default constexpr-steps limit is 1048576, which is not enough
for LibGfx's generation of the unicode bidirectional class lookup table
while GCC doesn't have any limit at all, so this patch increases the
limit to an arbitrarily larger value.
2021-04-29 10:36:23 +02:00
Andrew Kaster
35c0a6c54d AK+Userland: Move AK/TestSuite.h into LibTest and rework Tests' CMake
As many macros as possible are moved to Macros.h, while the
macros to create a test case are moved to TestCase.h. TestCase is now
the only user-facing header for creating a test case. TestSuite and its
helpers have moved into a .cpp file. Instead of requiring a TEST_MAIN
macro to be instantiated into the test file, a TestMain.cpp file is
provided instead that will be linked against each test. This has the
side effect that, if we wanted to have test cases split across multiple
files, it's as simple as adding them all to the same executable.

The test main should be portable to kernel mode as well, so if
there's a set of tests that should be run in self-test mode in kernel
space, we can accomodate that.

A new serenity_test CMake function streamlines adding a new test with
arguments for the test source file, subdirectory under /usr/Tests to
install the test application and an optional list of libraries to link
against the test application. To accomodate future test where the
provided TestMain.cpp is not suitable (e.g. test-js), a CUSTOM_MAIN
parameter can be passed to the function to not link against the
boilerplate main function.
2021-04-25 09:36:49 +02:00
Marco Biscaro
8124719c3d Tests: Reorganize LibCompress unit tests
Move LibCompress unit tests to LibCompress/Tests directory and register
them with CMake's add_test. This allows us to run these tests with
ninja test instead of running a separate executable.

Also split the existing tests in 3 test files that better follow the
source code structure (inspired by AK tests).
2021-04-21 08:00:32 +02:00
Timothy Flynn
110cd98c0a Lagom: Add sql utility and LibSQL unit tests 2021-04-20 18:28:34 +02:00
Luke
1ff6f2b4b2 Lagom/Fuzzers: Add fuzzer for zip file parser 2021-03-27 16:28:34 +01:00
Andrew Kaster
1aa605bc03 Meta: Run test-js and test-compress as CTest executables
Reduces the number of steps in the Actions workflow this way :^)
2021-03-02 09:00:21 +01:00
Andrew Kaster
669b6c43aa AK/Lagom: Modify TestSuite to return how many tests failed from main
This allows us to remove the FAIL_REGEX logic from the CTest invocation
of AK and LibRegex tests, as they will return a non-zero exit code on
failure :^).

Also means that running a failing TestSuite-enabled test with the
run-test-and-shutdown script will actually print that the test failed.
2021-03-01 11:12:36 +01:00
Luke
a66f96ff62 Lagom/Fuzzers: Add WAV fuzzer 2021-03-01 11:09:09 +01:00
Andrew Kaster
e787738c24 Meta: Build AK and LibRegex tests in Lagom and for Serenity
These tests were never built for the serenity target. Move their Lagom
build steps to the Lagom CMakeLists.txt, and add serenity build steps
for them. Also, fix the build errors when building them with the
serenity cross-compiler :^)
2021-02-28 18:19:37 +01:00
Linus Groh
24120d9dca Lagom: Build with -Wno-literal-suffix when using GCC
This is already set in the root CMakeLists.txt as well as here for Clang
(-Wno-user-defined-literals), but was forgotten for GCC which made an
Lagom-only build (cmake ../Meta/Lagom [...]) fail.
2021-02-27 16:38:21 +01:00
Linus Groh
590ae7a36d Lagom: CMakeLists.txt housekeeping
- Remove unused LIBM_SOURCES
- Some cosmetic and whitespace changes
2021-02-27 16:38:21 +01:00
Brian Gianforcaro
31e1b08e15 AK: Add support for AK::StringView literals with operator""sv
A new operator, operator""sv was added as of C++17 to support
string_view literals. This allows string_views to be constructed
from string literals and with no runtime cost to find the string
length.

See: https://en.cppreference.com/w/cpp/string/basic_string_view/operator%22%22sv

This change implements that functionality in AK::StringView.
We do have to suppress some warnings about implementing reserved
operators as we are essentially implementing STL functions in AK
as we have no STL :).
2021-02-24 14:38:31 +01:00
Andreas Kling
5625a1d608 Lagom: Build LibTextCodec into Lagom 2021-02-16 19:00:02 +01:00
Ben Wiederhake
1cbc01e288 Lagom: Don't include Shell/main.cpp
This caused some confusion: Apparently, clang has no trouble overriding Shell's
main, and this issue only surfaced when I tried to build the fuzzers with
wrong configuration (i.e., without the clang-injected 'main').

The diff is suggested by, and work of, @alimpfard.
2021-01-30 09:23:18 +01:00
Andreas Kling
7ec8f83a7f Lagom+AK: Remove remains of clang -Wconsumed usage
We stopped using that warning ages ago since it confused the compiler.
2021-01-28 09:14:49 +01:00
asynts
7cf0c7cc0d Meta: Split debug defines into multiple headers.
The following script was used to make these changes:

    #!/bin/bash
    set -e

    tmp=$(mktemp -d)

    echo "tmp=$tmp"

    find Kernel \( -name '*.cpp' -o -name '*.h' \) | sort > $tmp/Kernel.files
    find . \( -path ./Toolchain -prune -o -path ./Build -prune -o -path ./Kernel -prune \) -o \( -name '*.cpp' -o -name '*.h' \) -print | sort > $tmp/EverythingExceptKernel.files

    cat $tmp/Kernel.files | xargs grep -Eho '[A-Z0-9_]+_DEBUG' | sort | uniq > $tmp/Kernel.macros
    cat $tmp/EverythingExceptKernel.files | xargs grep -Eho '[A-Z0-9_]+_DEBUG' | sort | uniq > $tmp/EverythingExceptKernel.macros

    comm -23 $tmp/Kernel.macros $tmp/EverythingExceptKernel.macros > $tmp/Kernel.unique
    comm -1 $tmp/Kernel.macros $tmp/EverythingExceptKernel.macros > $tmp/EverythingExceptKernel.unique

    cat $tmp/Kernel.unique | awk '{ print "#cmakedefine01 "$1 }' > $tmp/Kernel.header
    cat $tmp/EverythingExceptKernel.unique | awk '{ print "#cmakedefine01 "$1 }' > $tmp/EverythingExceptKernel.header

    for macro in $(cat $tmp/Kernel.unique)
    do
        cat $tmp/Kernel.files | xargs grep -l $macro >> $tmp/Kernel.new-includes ||:
    done
    cat $tmp/Kernel.new-includes | sort > $tmp/Kernel.new-includes.sorted

    for macro in $(cat $tmp/EverythingExceptKernel.unique)
    do
        cat $tmp/Kernel.files | xargs grep -l $macro >> $tmp/Kernel.old-includes ||:
    done
    cat $tmp/Kernel.old-includes | sort > $tmp/Kernel.old-includes.sorted

    comm -23 $tmp/Kernel.new-includes.sorted $tmp/Kernel.old-includes.sorted > $tmp/Kernel.includes.new
    comm -13 $tmp/Kernel.new-includes.sorted $tmp/Kernel.old-includes.sorted > $tmp/Kernel.includes.old
    comm -12 $tmp/Kernel.new-includes.sorted $tmp/Kernel.old-includes.sorted > $tmp/Kernel.includes.mixed

    for file in $(cat $tmp/Kernel.includes.new)
    do
        sed -i -E 's/#include <AK\/Debug\.h>/#include <Kernel\/Debug\.h>/' $file
    done

    for file in $(cat $tmp/Kernel.includes.mixed)
    do
        echo "mixed include in $file, requires manual editing."
    done
2021-01-26 21:20:00 +01:00
asynts
1a3a0836c0 Everywhere: Use CMake to generate AK/Debug.h.
This was done with the help of several scripts, I dump them here to
easily find them later:

    awk '/#ifdef/ { print "#cmakedefine01 "$2 }' AK/Debug.h.in

    for debug_macro in $(awk '/#ifdef/ { print $2 }' AK/Debug.h.in)
    do
        find . \( -name '*.cpp' -o -name '*.h' -o -name '*.in' \) -not -path './Toolchain/*' -not -path './Build/*' -exec sed -i -E 's/#ifdef '$debug_macro'/#if '$debug_macro'/' {} \;
    done

    # Remember to remove WRAPPER_GERNERATOR_DEBUG from the list.
    awk '/#cmake/ { print "set("$2" ON)" }' AK/Debug.h.in
2021-01-25 09:47:36 +01:00
Luke
1c18d1380f Lagom/Fuzzers: Fix FuzzilliJs build and update patch for new Fuzzilli version
-fsanitize=fuzzer was being added to LINKER_FLAGS from Lagom/CMakeLists,
which we don't want with FuzzilliJs as we want to define the functions
it provides ourselves.
2021-01-24 00:40:49 +01:00
AnotherTest
86f50aa74e Shell: Make tests use PASS/FAIL instead of exit codes
There's no guarantee that the last executed command will have a zero
exit code, and so the shell exit code may or may not be zero, even if
all the tests pass.
Also changes the `test || echo fail && exit` to
`if not test { echo fail && exit }`, since that's nicer-looking.
2021-01-19 08:19:43 +01:00
Andreas Kling
affcdafc8d Lagom: Add Userland/ to include paths
This should make FuzzShell build.
2021-01-12 13:28:00 +01:00
Andreas Kling
13d7c09125 Libraries: Move to Userland/Libraries/ 2021-01-12 12:17:46 +01:00
Andreas Kling
ececac65c2 Userland: Move command-line utilities to Userland/Utilities/ 2021-01-12 12:04:09 +01:00
Andreas Kling
c4e2fd8123 Shell: Move to Userland/Shell/ 2021-01-12 12:04:07 +01:00
Linus Groh
b36f57e570 Lagom: Build gml-format 2021-01-03 22:12:08 +01:00
William Marlow
3e815ad5b1 Loader.so+LibELF: Move most of Loader.so's logic into ELF::DynamicLinker
Loader.so now just performs the initial self relocations and static
LibC initialisation before handing over to ELF::DynamicLinker::linker_main
to handle the rest of the process.

As a trade-off, ELF::DynamicLinker needs to be explicitly excluded from
Lagom unless we really want to try writing a cross platform dynamic loader
2021-01-03 17:15:55 +01:00
Luke
5dfcecd244 Lagom/Fuzzers: Add TTF fuzzer 2021-01-02 01:39:34 +01:00
Andreas Kling
3eca77bd86 Lagom: Build test-compress 2021-01-01 22:42:13 +01:00
Brian Gianforcaro
6d67c4cafc CMake: Consolidate all options to the root of the project 2021-01-01 14:37:04 +01:00
AnotherTest
eb7a943641 Meta: Set a 10 second timeout on Shell tests
Shell-backgrounding seems to be deadlocking at times, until that's
fixed, let's not take 5 hours to fail the build.
2020-12-15 20:58:32 +01:00
Luke
5c911ad4b1 Meta: Add GitHub Actions workflow for Lagom with Fuzzers
There are cases where Lagom will build with GCC but not Clang.
This often goes unnoticed for a while as we don't often build with
Clang.

However, this is now important to test in CI because of the
OSS-Fuzz integration.

Note that this only tests the build, it does not run any tests.
Note that it also only builds LagomCore, Lagom and the fuzzers.
It does not build the other programs that use Lagom.
2020-11-29 20:22:56 +01:00
AnotherTest
1ecea2f105 Lagom: Add a Shell parser fuzzer 2020-11-29 16:22:44 +01:00
DavidKorczynski
ff8ac60184
Lagom: Various fixes to make Lagom run on OSS-Fuzz (#4176) 2020-11-27 23:57:02 +01:00
Emanuel Sprung
55450055d8 LibRegex: Add a regular expression library
This commit is a mix of several commits, squashed into one because the
commits before 'Move regex to own Library and fix all the broken stuff'
were not fixable in any elegant way.
The commits are listed below for "historical" purposes:

- AK: Add options/flags and Errors for regular expressions

Flags can be provided for any possible flavour by adding a new scoped enum.
Handling of flags is done by templated Options class and the overloaded
'|' and '&' operators.

- AK: Add Lexer for regular expressions

The lexer parses the input and extracts tokens needed to parse a regular
expression.

- AK: Add regex Parser and PosixExtendedParser

This patchset adds a abstract parser class that can be derived to implement
different parsers. A parser produces bytecode to be executed within the
regex matcher.

- AK: Add regex matcher

This patchset adds an regex matcher based on the principles of the T-REX VM.
The bytecode pruduced by the respective Parser is put into the matcher and
the VM will recursively execute the bytecode according to the available OpCodes.
Possible improvement: the recursion could be replaced by multi threading capabilities.

To match a Regular expression, e.g. for the Posix standard regular expression matcher
use the following API:

```
Pattern<PosixExtendedParser> pattern("^.*$");
auto result = pattern.match("Well, hello friends!\nHello World!"); // Match whole needle

EXPECT(result.count == 1);
EXPECT(result.matches.at(0).view.starts_with("Well"));
EXPECT(result.matches.at(0).view.end() == "!");

result = pattern.match("Well, hello friends!\nHello World!", PosixFlags::Multiline); // Match line by line

EXPECT(result.count == 2);
EXPECT(result.matches.at(0).view == "Well, hello friends!");
EXPECT(result.matches.at(1).view == "Hello World!");

EXPECT(pattern.has_match("Well,....")); // Just check if match without a result, which saves some resources.
```

- AK: Rework regex to work with opcodes objects

This patchsets reworks the matcher to work on a more structured base.
For that an abstract OpCode class and derived classes for the specific
OpCodes have been added. The respective opcode logic is contained in
each respective execute() method.

- AK: Add benchmark for regex

- AK: Some optimization in regex for runtime and memory

- LibRegex: Move regex to own Library and fix all the broken stuff

Now regex works again and grep utility is also in place for testing.
This commit also fixes the use of regex.h in C by making `regex_t`
an opaque (-ish) type, which makes its behaviour consistent between
C and C++ compilers.
Previously, <regex.h> would've blown C compilers up, and even if it
didn't, would've caused a leak in C code, and not in C++ code (due to
the existence of `OwnPtr` inside the struct).

To make this whole ordeal easier to deal with (for now), this pulls the
definitions of `reg*()` into LibRegex.

pros:
- The circular dependency between LibC and LibRegex is broken
- Eaiser to test (without accidentally pulling in the host's libc!)

cons:
- Using any of the regex.h functions will require the user to link -lregex
- The symbols will be missing from libc, which will be a big surprise
  down the line (especially with shared libs).

Co-Authored-By: Ali Mohammad Pur <ali.mpfard@gmail.com>
2020-11-27 21:32:41 +01:00
Nico Weber
39ce7251f7 Lagom: Use -fsanitize=fuzzer, not -fsanitize=fuzzer-no-link
Fuzzers don't link for me without this change.
2020-11-14 10:09:03 +01:00
Nico Weber
5c2e8b6189 Lagom: Add ntpquery to lagom build 2020-11-12 21:21:33 +01:00
Nico Weber
5fcd34b810 Userland: Add an "adjtime" utility
It's a thin userland wrapper around adjtime(2). It can be used
to view current pending time adjustments, and root can use it to
smoothly adjust the system time.

As far as I can tell, other systems don't have a userland utility
for this, but it seems useful. Useful enough that I'm adding it to
the lagom build so I can use it on my linux box too :)
2020-11-10 19:03:08 +01:00
AnotherTest
37c089fb7b LibTLS: (Almost) verify certificate chain against root CA certificates
Also adds a very primitive systemwide ca_certs.ini file.
2020-10-30 23:42:03 +01:00
asynts
607931268e CMake: Use CONFIGURE_DEPENDS in existing globs. 2020-10-29 11:52:47 +01:00
Andreas Kling
e002cbb06b Build: Add some -Wno-unknown-warning-option flags to CXXFLAGS
Patch from Anonymous.
2020-09-01 12:00:53 +02:00
AnotherTest
465d46c665 Meta+Userland: Run the TLS test too
While this _does_ add a point of failure, it'll be a pretty bad day when
google goes down.
And this is unlikely to put a (positive) dent in their incoming
requests, so let's just roll with it until we have our own TLS server.
2020-08-24 09:29:39 +02:00
Andreas Kling
a37331e236 Lagom: Add LibHTTP to Lagom (it builds fine without changes) 2020-08-15 00:05:45 +02:00
Nico Weber
700f5cfc90 Lagom: Pass -Wno-deprecated-copy to clang too
Due to Serenity's use of concepts, it now requires clang 10 or newer,
and clang 10 or newer both understands and requires
-Wno-deprecated-copy.
2020-08-09 21:13:06 +02:00
Nico Weber
9c136be08b disasm: For ELF files, disassemble .text section
Since disasm is built in lagom, this requires adding LibELF to lagom.
2020-08-09 21:12:54 +02:00
Nico Weber
a619943001 Build: Make things build with clang without needing local changes
Useful for sanitizer fuzzer builds.

clang doesn't have a -fconcepts switch (I'm guessing it just enables
concepts automatically with -std=c++2a, but I haven't checked),
and at least the version on my system doesn't understand
-Wno-deprecated-move, so pass these two flags only to gcc.
In return, disable -Woverloaded-virtual which fires in many places.

The preceding commits fixed the handful of -Wunused-private-field
warnings that clang emitted.
2020-08-04 17:42:08 +02:00
stelar7
98e18d7339 LibCompress: Add LibCompress
For now this only contains DEFLATE, and a very simple Zlib
Eventually GZip, etc. can go here as well.
2020-08-04 11:27:07 +02:00
Nico Weber
c7652a2166 Lagom: Clean up glob after puff.c -> puff.cpp rename 2020-07-29 15:11:04 +02:00
Andreas Kling
96dce6893f Lagom: Build with -std=c++2a -Wno-deprecated-copy 2020-07-27 21:54:22 +02:00
Nico Weber
3f45e9ab1e Lagom: Add LibGemini, LibGfx
They are dependencies of LibWeb and might be useful for
running test-web on GitHub actions one day.
2020-07-23 23:02:28 +02:00
Nico Weber
c22790adb2 Lagom: Add a markdown fuzzer 2020-07-14 17:45:16 +02:00
Matthew Olsson
b9cf7a833f LibJS/test-js: Create test-js program, prepare for test suite refactor
This moves most of the work from run-tests.sh to test-js.cpp. This way,
we have a lot more control over how the test suite runs, as well as how
it outputs. This should result in some cool functionality!

This commit also refactors test-common.js to mimic the jest library.
This should allow tests to be much more expressive :)
2020-07-06 23:40:35 +02:00
AnotherTest
639c1a1737 Shell: Build as part of Lagom as well
Bringing the Serenity Shell to your very own host system :^)
2020-07-05 15:43:14 +02:00
AnotherTest
40829b849a Meta: Run Crypto tests in CI
We skip the tests that are not self-contained (TLS) to avoid adding
extra variables to the tests.
2020-06-07 19:29:40 +02:00
Andreas Kling
5f9de61bf2 Lagom: Build LibCrypto, LibTLS and test-crypto :^) 2020-05-27 12:28:39 +02:00
Sergey Bugaev
450a2a0f9c Build: Switch to CMake :^)
Closes https://github.com/SerenityOS/serenity/issues/2080
2020-05-14 20:15:18 +02:00
Andreas Kling
32d83fdee4 LibX86: Add an X86 instruction decoder library + basic disassembler
This will be very useful for developer tools like ProfileView, and also
for future tools like debuggers and such. :^)
2020-04-11 13:16:17 +02:00
Paul Redmond
7291d5c86f
Lagom: Add fuzz testing for LibJS using libFuzzer (#1692)
Note: clang only (see https://llvm.org/docs/LibFuzzer.html)

- add FuzzJs which will run the LibJS parser on random javascript inputs
- added a basic dictionary of javascript tokens

To use fuzzer:
CC=/usr/bin/clang CXX=/usr/bin/clang++ cmake -DENABLE_FUZZER_SANITIZER=1 ..
Fuzzers/FuzzJs -dict=../Fuzzers/FuzzJs.dict
2020-04-08 10:40:02 +02:00
Brian Gianforcaro
1e67efc5c1 Lagom: Add clang address/memory/undefined-behavior analyzer support
Adding the ability to turn on Clang analyzer support in the Lagom build.
Right now the following are working warning free on the LibJS test suite:

 -DENABLE_MEMORY_SANITIZER:BOOL=ON

 -DENABLE_ADDRESS_SANITIZER:BOOL=ON

The following analyzer produces errors when running the LibJS test suite:

 -DENABLE_UNDEFINED_SANITIZER:BOOL=ON
2020-04-05 12:49:06 +02:00
Andreas Kling
6529b78d08 Lagom: Build LibLine since "js" depends on it 2020-03-31 18:58:59 +02:00
Andreas Kling
e31dac3ba4 Lagom: Build LibJS + "js" test program
You can now get a full Linux build of the "js" test program by simply
building in Meta/Lagom :^)
2020-03-23 13:15:32 +01:00
Andreas Kling
5f4d81ff55 Lagom: Add a tiny JSON test app 2020-02-06 19:32:34 +01:00
Andreas Kling
be6a31dde1 Lagom: Remove the bitrotted SimpleIPC test program 2020-02-06 19:23:18 +01:00
Andreas Kling
0ddde627b1 Lagom: Fix build 2019-11-19 11:14:26 +01:00
Andreas Kling
dcd10149fe Lagom: Move this into Meta/
This is more of a meta thing, since it's not seeing active development,
but is just a way for me to build some Serenity parts and include them
in other projects. Move it out of the root to keep things tidy.
2019-11-18 09:07:05 +01:00
Renamed from Lagom/CMakeLists.txt (Browse further)