Commit graph

206 commits

Author SHA1 Message Date
Andreas Kling
4e99e013f0 Meta: Import the serenityos.org website
It doesn't seem right to not have the website in version control.
Also this way people can make changes to it. :^)
2020-12-28 00:12:12 +01:00
Liav A
72b1998f0d Kernel: Introduce a new partitioning subsystem
The partitioning code was very outdated, and required a full refactor.
The new subsystem removes duplicated code and uses more AK containers.

The most important change is that all implementations of the
PartitionTable class conform to one interface, which made it possible
to remove unnecessary code in the EBRPartitionTable class.

Finding partitions is now done in the StorageManagement singleton,
instead of doing so in init.cpp.

Also, now we don't try to find partitions on demand - the kernel will
try to detect if a StorageDevice is partitioned, and if so, will check
what is the partition table, which could be MBR, GUID or EBR.
Then, it will create DiskPartitionMetadata object for each partition
that is available in the partition table. This object will be used
by the partition enumeration code to create a DiskPartition with the
correct minor number.
2020-12-27 23:07:44 +01:00
Liav A
bc18712adf Build: Fix build of grub image when choosing EBR scheme 2020-12-27 23:07:44 +01:00
Liav A
0f208669af Build: Fix build of grub image when choosing GPT scheme 2020-12-27 23:07:44 +01:00
Liav A
247517cd4a Kernel: Introduce the DevFS
The DevFS along with DevPtsFS give a complete solution for populating
device nodes in /dev. The main purpose of DevFS is to eliminate the
need of device nodes generation when building the system.

Later on, DevFS will assist with exposing disk partition nodes.
2020-12-27 23:07:44 +01:00
Linus Groh
fec7501d1a Meta: Run lint-prettier.sh on CI 2020-12-27 21:25:27 +01:00
Linus Groh
ee719c23d4 Meta: Add lint-prettier.sh
This is a script similar to the clang-format one to ensure prettier
formatting of most JavaScript files.
2020-12-27 21:25:27 +01:00
Linus Groh
51bcfb5a44 Meta: Update lint-{clang-format,shell-scripts}.sh to take a list of files
This should speed up pre-commit a bit as only files that are staged will
be processed, and clang-format and shellcheck are only invoked once, not
for every file. When no arguments are given (e.g. on CI), it still uses
'git ls-files'.
2020-12-27 21:25:27 +01:00
Linus Groh
a56b3cbf7c Meta: Set 'pipefail' option correctly in shell scripts
This needs '-o' to work correctly. Also update the shebang to bash in
some scripts as shellcheck was complaining about pipefail not being a
POSIX shell thing otherwise.
2020-12-27 21:25:27 +01:00
Andreas Kling
00cfc25688 Base: Add some default desktop icons
I just picked three apps at random: Browser, TextEditor and Help.
This should make it a bit more visible that we have a working desktop.
2020-12-27 14:51:08 +01:00
AnotherTest
3dcdee75be Meta: Make lint-shell-scripts.sh happy
`${FAILURES}` -> `"${FAILURES}"`
2020-12-26 11:54:54 +01:00
Andreas Kling
1e4c010643 LibELF: Remove ELF::Loader and move everyone to ELF::Image
This commit gets rid of ELF::Loader entirely since its very ambiguous
purpose was actually to load executables for the kernel, and that is
now handled by the kernel itself.

This patch includes some drive-by cleanup in LibDebug and CrashDaemon
enabled by the fact that we no longer need to keep the ref-counted
ELF::Loader around.
2020-12-25 02:14:56 +01:00
Itamar
bbedd320b5 Toolchain+LibC: Fix usage of crt files
We now configure the gcc spec files to use a different crt files for
static & PIE binaries.

This relieves us from the need to explicitly specify the desired crt0
file in cmake scripts.
2020-12-24 21:46:35 +01:00
Lenny Maiorani
1db5276c05 Meta: Run all lint checks and report failures together
Problem:
- The first lint check that fails results in all subsequent checks not
  being run.

Solution:
- Run all the lint checks aggregating the number of failures.
- Return a non-0 exit code if any have failed.
2020-12-24 21:00:35 +01:00
Lenny Maiorani
d46de3aeb4 Meta: Verify all AK test files are listed in CMake
Problem:
- It is possible for a new test file to be added to the `AK/Tests`
  directory without being added to the corresponding
  `CMakeLists.txt`. This results in the tests not being run.

Solution:
- As part of CI linting, verify that all the `AK/Tests/Test*.cpp`
  files are mentioned in the `CMakeLists.txt`.
2020-12-24 21:00:10 +01:00
Lenny Maiorani
e4ce485309 CMake: Decouple cmake utility functions from top-level CMakeLists.txt
Problem:
- These utility functions are only used in `AK`, but are being defined
  in the top-level. This clutters the top-level.

Solution:
- Move the utility functions to `Meta/CMake/utils.cmake` and include
  where needed.
- Also, move `all_the_debug_macros.cmake` into `Meta/CMake` directory
  to consolidate the location of `*.cmake` script files.
2020-12-24 11:02:04 +01:00
Lenny Maiorani
cef6b7b2e4 CMake: Use built-in add_compile_definitions for *_DEBUG macros
Problem:
- Modifying CXXFLAGS directly is an old CMake style.
- The giant and ever-growing list of `*_DEBUG` macros clutters the
  top-level CMakeLists.txt.

Solution:
- Use the more current `add_compile_definitions` function.
- Sort all the debug options so that they are easy to view.
- Move the `*_DEBUG` macros to their own file which can be included
  directly.
2020-12-22 21:01:51 +01:00
Ben Wiederhake
32210802ac Meta: Use the correct clang-format
After all the work of determining the correct name for clang-format-10,
we forgot to actually use it. How silly!
2020-12-22 20:41:35 +01:00
Liav A
b781495e74 Build: Create device files according to the new major-minor numbers 2020-12-21 00:19:21 +01:00
William Marlow
39364bdda4 Build: Embed application icons directly in the executables.
New serenity_app() targets can be defined which allows application
icons to be emedded directly into the executable. The embedded
icons will then be used when creating an icon for that file in
LibGUI.
2020-12-21 00:12:59 +01:00
Lenny Maiorani
765936ebae
Everywhere: Switch from (void) to [[maybe_unused]] (#4473)
Problem:
- `(void)` simply casts the expression to void. This is understood to
  indicate that it is ignored, but this is really a compiler trick to
  get the compiler to not generate a warning.

Solution:
- Use the `[[maybe_unused]]` attribute to indicate the value is unused.

Note:
- Functions taking a `(void)` argument list have also been changed to
  `()` because this is not needed and shows up in the same grep
  command.
2020-12-21 00:09:48 +01:00
Andreas Kling
822dc56ef3 LibGUI: Introduce GML - a simple GUI Markup Language :^)
This patch replaces the UI-from-JSON mechanism with a more
human-friendly DSL.

The current implementation simply converts the GML into a JSON object
that can be consumed by GUI::Widget::load_from_json(). The parser is
not very helpful if you make a mistake.

The language offers a very simple way to instantiate any registered
Core::Object class by simply saying @ClassName

@GUI::Label {
    text: "Hello friends!"
    tooltip: ":^)"
}

Layouts are Core::Objects and can be assigned to the "layout" property:

@GUI::Widget {
    layout: @GUI::VerticalBoxLayout {
        spacing: 2
        margins: [8, 8, 8, 8]
    }
}

And finally, child objects are simply nested within their parent:

@GUI::Widget {
    layout: @GUI::HorizontalBoxLayout {
    }
    @GUI::Button {
        text: "OK"
    }
    @GUI::Button {
        text: "Cancel"
    }
}

This feels a *lot* more pleasant to write than the JSON we had. The fact
that no new code was being written with the JSON mechanism was pretty
telling, so let's approach this with developer convenience in mind. :^)
2020-12-20 11:59:40 +01:00
Andreas Kling
48d74c5356 LibHTTP: Make HTTPRequest::from_raw_request() take a ReadonlyBytes
This allows us to get rid of some ByteBuffer::wrap() usage.
2020-12-19 18:29:13 +01:00
AnotherTest
eb7a943641 Meta: Set a 10 second timeout on Shell tests
Shell-backgrounding seems to be deadlocking at times, until that's
fixed, let's not take 5 hours to fail the build.
2020-12-15 20:58:32 +01:00
Itamar
b1e01066fe Lagom/Fuzzers: Fix creation of ELF::Loader 2020-12-14 23:05:53 +01:00
Itamar
e2d695eabb Build: Build libgcc_s.so
This is the shared version of libgcc
2020-12-14 23:05:53 +01:00
Ben Wiederhake
4b4f23165f Meta+LibHTTP: Fuzz HTTP request parsing 2020-12-02 10:46:40 +01:00
Ben Wiederhake
e85aad6acc Meta: Always check completeness of ALL_THE_DEBUG_MACROS 2020-12-01 11:06:53 +01:00
Ben Wiederhake
59091e1861 Meta: Nicer wording in lint 2020-12-01 11:06:53 +01:00
Ben Wiederhake
a0f6db246e Meta: Document fuzz dump 2020-12-01 11:06:23 +01:00
Luke
030dac8c4d Lagom/Fuzzers: Add URL fuzzer 2020-11-30 11:35:16 +01:00
Luke
5c911ad4b1 Meta: Add GitHub Actions workflow for Lagom with Fuzzers
There are cases where Lagom will build with GCC but not Clang.
This often goes unnoticed for a while as we don't often build with
Clang.

However, this is now important to test in CI because of the
OSS-Fuzz integration.

Note that this only tests the build, it does not run any tests.
Note that it also only builds LagomCore, Lagom and the fuzzers.
It does not build the other programs that use Lagom.
2020-11-29 20:22:56 +01:00
Nico Weber
29e4dc7634 Lagom: Mention OSS-Fuzz in ReadMe
We added OSS-Fuzz integration in #4154, but documentation about it
is spread across several pull requests, IRC, and issues. Let's collect
the important bits in the ReadMe.
2020-11-29 20:22:09 +01:00
AnotherTest
1ecea2f105 Lagom: Add a Shell parser fuzzer 2020-11-29 16:22:44 +01:00
Linus Groh
6a37285d93 Lagom: Add Regex fuzzers 2020-11-29 09:29:26 +01:00
DavidKorczynski
ff8ac60184
Lagom: Various fixes to make Lagom run on OSS-Fuzz (#4176) 2020-11-27 23:57:02 +01:00
Emanuel Sprung
55450055d8 LibRegex: Add a regular expression library
This commit is a mix of several commits, squashed into one because the
commits before 'Move regex to own Library and fix all the broken stuff'
were not fixable in any elegant way.
The commits are listed below for "historical" purposes:

- AK: Add options/flags and Errors for regular expressions

Flags can be provided for any possible flavour by adding a new scoped enum.
Handling of flags is done by templated Options class and the overloaded
'|' and '&' operators.

- AK: Add Lexer for regular expressions

The lexer parses the input and extracts tokens needed to parse a regular
expression.

- AK: Add regex Parser and PosixExtendedParser

This patchset adds a abstract parser class that can be derived to implement
different parsers. A parser produces bytecode to be executed within the
regex matcher.

- AK: Add regex matcher

This patchset adds an regex matcher based on the principles of the T-REX VM.
The bytecode pruduced by the respective Parser is put into the matcher and
the VM will recursively execute the bytecode according to the available OpCodes.
Possible improvement: the recursion could be replaced by multi threading capabilities.

To match a Regular expression, e.g. for the Posix standard regular expression matcher
use the following API:

```
Pattern<PosixExtendedParser> pattern("^.*$");
auto result = pattern.match("Well, hello friends!\nHello World!"); // Match whole needle

EXPECT(result.count == 1);
EXPECT(result.matches.at(0).view.starts_with("Well"));
EXPECT(result.matches.at(0).view.end() == "!");

result = pattern.match("Well, hello friends!\nHello World!", PosixFlags::Multiline); // Match line by line

EXPECT(result.count == 2);
EXPECT(result.matches.at(0).view == "Well, hello friends!");
EXPECT(result.matches.at(1).view == "Hello World!");

EXPECT(pattern.has_match("Well,....")); // Just check if match without a result, which saves some resources.
```

- AK: Rework regex to work with opcodes objects

This patchsets reworks the matcher to work on a more structured base.
For that an abstract OpCode class and derived classes for the specific
OpCodes have been added. The respective opcode logic is contained in
each respective execute() method.

- AK: Add benchmark for regex

- AK: Some optimization in regex for runtime and memory

- LibRegex: Move regex to own Library and fix all the broken stuff

Now regex works again and grep utility is also in place for testing.
This commit also fixes the use of regex.h in C by making `regex_t`
an opaque (-ish) type, which makes its behaviour consistent between
C and C++ compilers.
Previously, <regex.h> would've blown C compilers up, and even if it
didn't, would've caused a leak in C code, and not in C++ code (due to
the existence of `OwnPtr` inside the struct).

To make this whole ordeal easier to deal with (for now), this pulls the
definitions of `reg*()` into LibRegex.

pros:
- The circular dependency between LibC and LibRegex is broken
- Eaiser to test (without accidentally pulling in the host's libc!)

cons:
- Using any of the regex.h functions will require the user to link -lregex
- The symbols will be missing from libc, which will be a big surprise
  down the line (especially with shared libs).

Co-Authored-By: Ali Mohammad Pur <ali.mpfard@gmail.com>
2020-11-27 21:32:41 +01:00
Nico Weber
f160df123b Lagom: Rename FuzzBMP to FuzzBMPLoader 2020-11-26 09:59:46 +01:00
Nico Weber
ec4c2f971b Lagom: Make BMP fuzzer look like the other image loader fuzzers 2020-11-26 09:59:46 +01:00
Nico Weber
4f12efe191 Lagom: Add fuzzers for remaining image loaders: ICO, PNG, PBM, PGM 2020-11-26 09:59:46 +01:00
Ben Wiederhake
ec36388f40 Meta: Reduce IRC spam
Fixes #4145.
2020-11-25 20:50:22 +01:00
Linus Groh
4e68f179d6 Lagom: Fix FuzzJs build
This was broken with the JS::Parser::Error position changes, but I don't
actually see a reason to do anything with the parser errors here, so
let's remove it and consider simply not crashing a success. :^)
2020-11-25 20:00:23 +01:00
Nico Weber
f13a38aa4d Lagom: Add a gif loader fuzzer 2020-11-20 21:17:58 +01:00
Nico Weber
812892ef86 Lagom: Add a jpg fuzzer 2020-11-19 21:21:45 +01:00
Nico Weber
3f3a7d128b Lagom: Make fuzzer cmake less repetitive 2020-11-19 21:21:45 +01:00
Nico Weber
f5967c4745 Lagom: Add a PPM fuzzer
It finds the problem fixed in 69518bd178 but nothing else.
2020-11-19 14:04:35 +01:00
Ben Wiederhake
c4933f9bd7 Meta: Fix IRC notification script
What a silly mistake. How did I manage to do that?
2020-11-14 17:14:45 +01:00
Ben Wiederhake
ef9ac8a8a2 Meta: Use SerenityBot for IRC notifications
This avoids "useless" join/part notifications.
2020-11-14 15:30:29 +01:00
Nico Weber
39ce7251f7 Lagom: Use -fsanitize=fuzzer, not -fsanitize=fuzzer-no-link
Fuzzers don't link for me without this change.
2020-11-14 10:09:03 +01:00
Nico Weber
d775dea13c Lagom: Add a gemini fuzzer
Didn't find anything interesting, but might as well check it in.
2020-11-14 10:09:03 +01:00