Commit graph

3863 commits

Author SHA1 Message Date
Andreas Kling
f2decb6665 Revert "Kernel: Fix Thread::relock_process leaving critical section"
This reverts commit e9e76b8074.

This was causing a noticeable slowdown, and we're not sure that it was
actually necessary.
2021-01-27 23:23:21 +01:00
Tom
db1448b21a Kernel: Add a compile-time switch to enable scheduling on all CPUs
This is meant to be temporary only and should be removed once scheduling
on all CPUs is stable.
2021-01-27 22:48:41 +01:00
Tom
e9e76b8074 Kernel: Fix Thread::relock_process leaving critical section
We don't want to explicitly enable interrupts when leaving the
critical section to trigger a context switch.
2021-01-27 22:48:41 +01:00
Tom
ccaeb47401 Kernel: Don't hold scheduler lock while setting up blocker in Thread::block
This fixes a deadlock when one processor is trying to block while another is
trying to unblock the same.
2021-01-27 22:48:41 +01:00
Tom
250a310454 Kernel: Release MM lock while yielding from inode page fault handler
We need to make sure other processors can grab the MM lock while we
wait, so release it when we might block. Reading the page from
disk may also block, so release it during that time as well.
2021-01-27 22:48:41 +01:00
Tom
ac3927086f Kernel: Keep a list of threads per Process
This allow us to iterate only the threads of the process.
2021-01-27 22:48:41 +01:00
Tom
03a9ee79fa Kernel: Implement thread priority queues
Rather than walking all Thread instances and putting them into
a vector to be sorted by priority, queue them into priority sorted
linked lists as soon as they become ready to be executed.
2021-01-27 22:48:41 +01:00
Tom
c531084873 Kernel: Track processor idle state and wake processors when waking threads
Attempt to wake idle processors to get threads to be scheduled more quickly.
We don't want to wait until the next timer tick if we have processors that
aren't doing anything.
2021-01-27 22:48:41 +01:00
Tom
39f408daa0 Kernel: Use the CPU# as logical apic id 2021-01-27 22:48:41 +01:00
Andreas Kling
e67402c702 Kernel: Remove Range "valid" state and use Optional<Range> instead
It's easier to understand VM ranges if they are always valid. We can
simply use an empty Optional<Range> to encode absence when needed.
2021-01-27 21:14:42 +01:00
Andreas Kling
5ab27e4bdc Kernel: sys$mmap() without MAP_FIXED should consider address a hint
If we can't use that specific address, it's still okay to put it
anywhere else in VM.
2021-01-27 21:14:42 +01:00
Tom
e2f9e557d3 Kernel: Make Processor::id a static function
This eliminates the window between calling Processor::current and
the member function where a thread could be moved to another
processor. This is generally not as big of a concern as with
Processor::current_thread, but also slightly more light weight.
2021-01-27 21:12:24 +01:00
Tom
21d288a10e Kernel: Make Thread::current smp-safe
Change Thread::current to be a static function and read using the fs
register, which eliminates a window between Processor::current()
returning and calling a function on it, which can trigger preemption
and a move to a different processor, which then causes operating
on the wrong object.
2021-01-27 21:12:24 +01:00
Tom
f88a8b16d7 Kernel: Make entering and leaving critical sections atomic
We also need to store m_in_critical in the Thread upon switching,
and we need to restore it. This solves a problem where threads
moving between different processors could end up with an unexpected
value.
2021-01-27 21:12:24 +01:00
Tom
33cdc1d2f1 Kernel: Use new Thread::previous_mode to track ticks 2021-01-27 21:12:24 +01:00
Tom
0bd558081e Kernel: Track previous mode when entering/exiting traps
This allows us to determine what the previous mode (user or kernel)
was, e.g. in the timer interrupt. This is used e.g. to determine
whether a signal handler should be set up.

Fixes #5096
2021-01-27 21:12:24 +01:00
Andreas Kling
76a69be217 Kernel: Assert in RangeAllocator that sizes are multiple of PAGE_SIZE 2021-01-27 19:45:53 +01:00
asynts
7cf0c7cc0d Meta: Split debug defines into multiple headers.
The following script was used to make these changes:

    #!/bin/bash
    set -e

    tmp=$(mktemp -d)

    echo "tmp=$tmp"

    find Kernel \( -name '*.cpp' -o -name '*.h' \) | sort > $tmp/Kernel.files
    find . \( -path ./Toolchain -prune -o -path ./Build -prune -o -path ./Kernel -prune \) -o \( -name '*.cpp' -o -name '*.h' \) -print | sort > $tmp/EverythingExceptKernel.files

    cat $tmp/Kernel.files | xargs grep -Eho '[A-Z0-9_]+_DEBUG' | sort | uniq > $tmp/Kernel.macros
    cat $tmp/EverythingExceptKernel.files | xargs grep -Eho '[A-Z0-9_]+_DEBUG' | sort | uniq > $tmp/EverythingExceptKernel.macros

    comm -23 $tmp/Kernel.macros $tmp/EverythingExceptKernel.macros > $tmp/Kernel.unique
    comm -1 $tmp/Kernel.macros $tmp/EverythingExceptKernel.macros > $tmp/EverythingExceptKernel.unique

    cat $tmp/Kernel.unique | awk '{ print "#cmakedefine01 "$1 }' > $tmp/Kernel.header
    cat $tmp/EverythingExceptKernel.unique | awk '{ print "#cmakedefine01 "$1 }' > $tmp/EverythingExceptKernel.header

    for macro in $(cat $tmp/Kernel.unique)
    do
        cat $tmp/Kernel.files | xargs grep -l $macro >> $tmp/Kernel.new-includes ||:
    done
    cat $tmp/Kernel.new-includes | sort > $tmp/Kernel.new-includes.sorted

    for macro in $(cat $tmp/EverythingExceptKernel.unique)
    do
        cat $tmp/Kernel.files | xargs grep -l $macro >> $tmp/Kernel.old-includes ||:
    done
    cat $tmp/Kernel.old-includes | sort > $tmp/Kernel.old-includes.sorted

    comm -23 $tmp/Kernel.new-includes.sorted $tmp/Kernel.old-includes.sorted > $tmp/Kernel.includes.new
    comm -13 $tmp/Kernel.new-includes.sorted $tmp/Kernel.old-includes.sorted > $tmp/Kernel.includes.old
    comm -12 $tmp/Kernel.new-includes.sorted $tmp/Kernel.old-includes.sorted > $tmp/Kernel.includes.mixed

    for file in $(cat $tmp/Kernel.includes.new)
    do
        sed -i -E 's/#include <AK\/Debug\.h>/#include <Kernel\/Debug\.h>/' $file
    done

    for file in $(cat $tmp/Kernel.includes.mixed)
    do
        echo "mixed include in $file, requires manual editing."
    done
2021-01-26 21:20:00 +01:00
Maciej Zygmanowski
645657865d Kernel: Add RTC as fallback entropy source if HPET is not found 2021-01-26 21:14:44 +01:00
Maciej Zygmanowski
0a61924727 Kernel: Use HPET as entropy source if CPU doesn't support RDRAND
We don't have anything better for these CPUs for now.
2021-01-26 21:14:44 +01:00
Maciej Zygmanowski
ceb5682b54 Kernel: Add HPET::read_main_counter() 2021-01-26 21:14:44 +01:00
Maciej Zygmanowski
67e93745cb Kernel: Ensure that HPET is initialized before using random the first time 2021-01-26 21:14:44 +01:00
Andreas Kling
3ff88a1d77 Kernel: Assert on attempt to map private region backed by shared inode
If we find ourselves with a user-accessible, non-shared Region backed by
a SharedInodeVMObject, that's pretty bad news, so let's just panic the
kernel instead of getting abused.

There might be a better place for this kind of check, so I've added a
FIXME about putting more thought into that.
2021-01-26 18:35:10 +01:00
Andreas Kling
a131927c75 Kernel: sys$munmap() region splitting did not preserve "shared" flag
This was exploitable since the shared flag determines whether inode
permission checks are applied in sys$mprotect().

The bug was pretty hard to spot due to default arguments being used
instead. This patch removes the default arguments to make explicit
at each call site what's being done.
2021-01-26 18:35:04 +01:00
Linus Groh
e7183cc762 Kernel: Don't drop pledge()'d promises/execpromises when passing nullptr
When passing nullptr for either promises or execpromises to pledge(),
the expected behaviour is to not change their current value at all - we
were accidentally resetting them to 0, effectively dropping previously
pledge()'d promises.
2021-01-26 18:18:01 +01:00
Andreas Kling
c7858622ec Kernel: Update process promise states on execve() and fork()
We now move the execpromises state into the regular promises, and clear
the execpromises state.

Also make sure to duplicate the promise state on fork.

This fixes an issue where "su" would launch a shell which immediately
crashed due to not having pledged "stdio".
2021-01-26 15:26:37 +01:00
Andreas Kling
1e25d2b734 Kernel: Remove allocate_region() functions that don't take a Range
Let's force callers to provide a VM range when allocating a region.
This makes ENOMEM error handling more visible and removes implicit
VM allocation which felt a bit magical.
2021-01-26 14:13:57 +01:00
Linus Groh
629180b7d8 Kernel: Support pledge() with empty promises
This tells the kernel that the process wants to use pledge, but without
pledging anything - effectively restricting it to syscalls that don't
require a certain promise. This is part of OpenBSD's pledge() as well,
which served as basis for Serenity's.
2021-01-25 23:22:21 +01:00
Tom
b580c005f1 Kernel: Fix possible context switch within first context switch of a thread
We were enabling interrupts too early, before the first context switch to
a thread was complete. This could then trigger another context switch
within the context switch, which lead to a crash.
2021-01-25 23:22:12 +01:00
Tom
bd73102513 Kernel: Fix some race conditions with Lock and waiting/waking threads
There is a window between acquiring/releasing the lock with the atomic
variables and subsequently waiting or waking threads. With a single
processor this window was closed by using a critical section, but
this doesn't prevent other processors from running these code paths.
To solve this, set a flag in the WaitQueue while holding m_lock which
determines if threads should be blocked at all.
2021-01-25 22:54:55 +01:00
Tom
4cf0859612 Kernel: Fix race condition in Lock::lock that may leave corrupted state 2021-01-25 22:54:55 +01:00
Andreas Kling
ab14b0ac64 Kernel: Hoist VM range allocation up to sys$mmap() itself
Instead of letting each File subclass do range allocation in their
mmap() override, do it up front in sys$mmap().

This makes us honor alignment requests for file-backed memory mappings
and simplifies the code somwhat.
2021-01-25 18:57:06 +01:00
asynts
8571209560 Kernel: Fix crash when booting with ACPI. 2021-01-25 13:00:49 +01:00
asynts
eea72b9b5c Everywhere: Hook up remaining debug macros to Debug.h. 2021-01-25 09:47:36 +01:00
asynts
8465683dcf Everywhere: Debug macros instead of constexpr.
This was done with the following script:

    find . \( -name '*.cpp' -o -name '*.h' -o -name '*.in' \) -not -path './Toolchain/*' -not -path './Build/*' -exec sed -i -E 's/dbgln<debug_([a-z_]+)>/dbgln<\U\1_DEBUG>/' {} \;

    find . \( -name '*.cpp' -o -name '*.h' -o -name '*.in' \) -not -path './Toolchain/*' -not -path './Build/*' -exec sed -i -E 's/if constexpr \(debug_([a-z0-9_]+)/if constexpr \(\U\1_DEBUG/' {} \;
2021-01-25 09:47:36 +01:00
asynts
acdcf59a33 Everywhere: Remove unnecessary debug comments.
It would be tempting to uncomment these statements, but that won't work
with the new changes.

This was done with the following commands:

    find . \( -name '*.cpp' -o -name '*.h' -o -name '*.in' \) -not -path './Toolchain/*' -not -path './Build/*' -exec awk -i inplace '$0 !~ /\/\/#define/ { if (!toggle) { print; } else { toggle = !toggle } } ; $0 ~/\/\/#define/ { toggle = 1 }' {} \;

    find . \( -name '*.cpp' -o -name '*.h' -o -name '*.in' \) -not -path './Toolchain/*' -not -path './Build/*' -exec awk -i inplace '$0 !~ /\/\/ #define/ { if (!toggle) { print; } else { toggle = !toggle } } ; $0 ~/\/\/ #define/ { toggle = 1 }' {} \;
2021-01-25 09:47:36 +01:00
asynts
1a3a0836c0 Everywhere: Use CMake to generate AK/Debug.h.
This was done with the help of several scripts, I dump them here to
easily find them later:

    awk '/#ifdef/ { print "#cmakedefine01 "$2 }' AK/Debug.h.in

    for debug_macro in $(awk '/#ifdef/ { print $2 }' AK/Debug.h.in)
    do
        find . \( -name '*.cpp' -o -name '*.h' -o -name '*.in' \) -not -path './Toolchain/*' -not -path './Build/*' -exec sed -i -E 's/#ifdef '$debug_macro'/#if '$debug_macro'/' {} \;
    done

    # Remember to remove WRAPPER_GERNERATOR_DEBUG from the list.
    awk '/#cmake/ { print "set("$2" ON)" }' AK/Debug.h.in
2021-01-25 09:47:36 +01:00
Andreas Kling
f5d916a881 Kernel: Make sys$anon_create() fail if size == 0
An empty anonymous file is useless since it cannot be resized anyway,
so let's not support creating it.
2021-01-25 09:36:42 +01:00
Jean-Baptiste Boric
4d755725bf Kernel: Allow disabling of IDE controllers with disable_ide
The kernel doesn't like the IDE controllers on an Asus A7N8X-E Deluxe
motherboard, so add an option to disable them.
2021-01-24 22:16:18 +01:00
Jean-Baptiste Boric
7eaefa5aa6 Kernel: Make use of interrupts as an entropy source
Booting old computers without RDRAND/RDSEED and without a disk makes
the system severely starved for entropy. Uses interrupts as a source
to side-step that issue.

Also warn whenever the system is starved of entropy, because that's
a non-obvious failure mode.
2021-01-24 22:16:18 +01:00
Andreas Kling
2112b79986 Revert "Kernel: Make VFS::create() fail with EINVAL on invalid file mode"
This reverts commit ca3489eec7.

Fixes #5087.
2021-01-24 08:32:09 +01:00
Luke
50a2cb38e5 Kernel: Fix two error codes being returned as positive in Process::exec
This made the assertion on line 921 think it was a successful exec, when it wasn't.

Fixes #5084
2021-01-24 01:06:24 +01:00
Andreas Kling
8a9853d5da Kernel: Create core dumps with S_IFREG set (regular file)
Otherwise, the VFS will refuse to create the file.
2021-01-23 17:59:42 +01:00
Jean-Baptiste Boric
adb2fae69c Kernel: Evaluate block conditions inside VirtualConsole::emit()
This makes text mode boot usable again, because the shell never
received keyboard input otherwise.
2021-01-23 16:53:03 +01:00
Jean-Baptiste Boric
8941d831bb Kernel: Allow "serial_debug" everywhere on the command line 2021-01-23 16:53:03 +01:00
asynts
1c1e577a5e Everywhere: Deprecate dbg(). 2021-01-23 16:46:26 +01:00
Andreas Kling
d7345cf560 Kernel: Use current EUID/EGID for LocalSocket prebind credentials 2021-01-23 16:45:05 +01:00
Andreas Kling
ca3489eec7 Kernel: Make VFS::create() fail with EINVAL on invalid file mode
Instead of trying to fix up the mode to look like a regular file,
just fail instead.
2021-01-23 16:45:05 +01:00
Andreas Kling
bfb254ed14 Ext2FS: Assert that create_directory() is called with valid mode 2021-01-23 16:45:05 +01:00
Andreas Kling
c32176db27 Kernel: Don't preserve set-uid bit in open() and bind() modes
For some reason we were keeping the bits 04777 in file modes. That
doesn't seem right and I can't think of a reason why the set-uid bit
should be allowed to slip through.
2021-01-23 16:45:05 +01:00
Andreas Kling
f2ea6c3d4c Ext2FS: Don't create a directory when asked to create a socket file
(mode & S_IFDIR) is not enough to check if "mode" is a directory,
we have to check all the bits in the S_IFMT mask.

Use the is_directory() helper to fix this bug.
2021-01-23 16:45:05 +01:00
Andreas Kling
54f421e170 Kernel: Clear coredump metadata on exec()
If for some reason a process wants to exec after saving some coredump
metadata, we should just throw away the data.
2021-01-23 09:41:11 +01:00
Andreas Kling
82c879c315 Kernel: Fix PATADiskDevice device names
This broke the regular QEMU boot.
2021-01-22 22:24:43 +01:00
Jean-Baptiste Boric
666936a06b Kernel: Find boot device by enumerating devices
Since devices are enumerable and can compute their own name inside the
/dev hierarchy, there is no need to try and parse "root=/dev/xxx" by
hand.

This also makes any block device a candidate for the boot device, which
now includes ramdisk devices, so SerenityOS can now boot diskless too.
The disk image generated for QEMU is suitable, as long as it fits in
memory with room to spare for the rest of the system.
2021-01-22 22:17:39 +01:00
Jean-Baptiste Boric
f64e287b82 Kernel: Make device generate their own names
Besides removing the monolithic DevFSDeviceInode::determine_name()
method, being able to determine a device's name inside the /dev
hierarchy outside of DevFS has its uses.
2021-01-22 22:17:39 +01:00
Jean-Baptiste Boric
a2601e1308 Kernel: Hook Ramdisk devices inside DevFS 2021-01-22 22:17:39 +01:00
Jean-Baptiste Boric
225957283e Kernel: Implement RamdiskDevice 2021-01-22 22:17:39 +01:00
Jean-Baptiste Boric
ec056f3bd1 Kernel: Parse boot modules from Multiboot specification 2021-01-22 22:17:39 +01:00
Jean-Baptiste Boric
3cbe805486 Kernel: Move kmalloc heaps and super pages inside .bss segment
The kernel ignored the first 8 MiB of RAM while parsing the memory map
because the kmalloc heaps and the super physical pages lived here. Move
all that stuff inside the .bss segment so that those memory regions are
accounted for, otherwise we risk overwriting boot modules placed next
to the kernel.
2021-01-22 22:17:39 +01:00
Jean-Baptiste Boric
5cd1217b6e Kernel: Remove trace log in MemoryManager::deallocate_user_physical_page() 2021-01-22 22:17:39 +01:00
Jean-Baptiste Boric
06b73eea94 Kernel: Do not assert if unable to load kernel symbols 2021-01-22 22:17:39 +01:00
Jean-Baptiste Boric
2f8b047339 Kernel: Untangle StorageController from PCI::DeviceController 2021-01-22 22:17:39 +01:00
asynts
ea7b7d8ceb Everywhere: Replace a bundle of dbg with dbgln.
These changes are arbitrarily divided into multiple commits to make it
easier to find potentially introduced bugs with git bisect.
2021-01-22 22:14:30 +01:00
asynts
27bc48e06c Everywhere: Replace a bundle of dbg with dbgln.
These changes are arbitrarily divided into multiple commits to make it
easier to find potentially introduced bugs with git bisect.
2021-01-22 22:14:30 +01:00
asynts
7b0a1a98d9 Everywhere: Replace a bundle of dbg with dbgln.
These changes are arbitrarily divided into multiple commits to make it
easier to find potentially introduced bugs with git bisect.
2021-01-22 22:14:30 +01:00
asynts
a348ab55b0 Everywhere: Replace a bundle of dbg with dbgln.
These changes are arbitrarily divided into multiple commits to make it
easier to find potentially introduced bugs with git bisect.
2021-01-22 22:14:30 +01:00
asynts
c6ebca5b45 Everywhere: Replace a bundle of dbg with dbgln.
These changes are arbitrarily divided into multiple commits to make it
easier to find potentially introduced bugs with git bisect.
2021-01-22 22:14:30 +01:00
asynts
dd727d1fec Everywhere: Replace a bundle of dbg with dbgln.
These changes are arbitrarily divided into multiple commits to make it
easier to find potentially introduced bugs with git bisect.
2021-01-22 22:14:30 +01:00
asynts
67583bc424 Everywhere: Replace a bundle of dbg with dbgln.
These changes are arbitrarily divided into multiple commits to make it
easier to find potentially introduced bugs with git bisect.
2021-01-22 22:14:30 +01:00
asynts
78b2be5a2a Everywhere: Replace a bundle of dbg with dbgln.
These changes are arbitrarily divided into multiple commits to make it
easier to find potentially introduced bugs with git bisect.
2021-01-22 22:14:30 +01:00
asynts
9d588cc9cc Everywhere: Replace a bundle of dbg with dbgln.
These changes are arbitrarily divided into multiple commits to make it
easier to find potentially introduced bugs with git bisect.
2021-01-22 22:14:30 +01:00
asynts
5356aae3cc Everywhere: Replace a bundle of dbg with dbgln.
These changes are arbitrarily divided into multiple commits to make it
easier to find potentially introduced bugs with git bisect.
2021-01-22 22:14:30 +01:00
Andreas Kling
2cd07c6212 Kernel+Userland: Remove "dns" pledge promise alias
This was just an alias for "unix" that I added early on back when there
was some belief that we might be compatible with OpenBSD. We're clearly
never going to be compatible with their pledges so just drop the alias.
2021-01-22 19:39:44 +01:00
Linus Groh
421587c15c Everywhere: Fix typos 2021-01-22 18:41:29 +01:00
Tom
2830ce5383 Kernel: Unsupported TTY ioctls should return EINVAL gracefully
Fixes #4971
2021-01-22 09:25:04 +01:00
Andreas Kling
c9a7f81dc3 Kernel: Create new files with the current process EUID/EGID
We were using the UID/GID and not the EUID/EGID, which didn't match
other systems.
2021-01-21 11:08:16 +01:00
Andreas Kling
928ee2c791 Kernel: Don't let signals unblock threads while handling a page fault
It was possible to signal a process while it was paging in an inode
backed VM object. This would cause the inode read to EINTR, and the
page fault handler would assert.

Solve this by simply not unblocking threads due to signals if they are
currently busy handling a page fault. This is probably not the best way
to solve this issue, so I've added a FIXME to that effect.
2021-01-21 00:14:56 +01:00
Andreas Kling
1f53dd0943 Ext2FS: Propagate I/O errors from Ext2FSInode::write_bytes() 2021-01-21 00:14:56 +01:00
Andreas Kling
57ca15f126 Kernel: Remove commented-out code from Thread::dispatch_signal() 2021-01-20 23:27:23 +01:00
Andreas Kling
19d3f8cab7 Kernel+LibC: Turn errno codes into a strongly typed enum
..and allow implicit creation of KResult and KResultOr from ErrnoCode.
This means that kernel functions that return those types can finally
do "return EINVAL;" and it will just work.

There's a handful of functions that still deal with signed integers
that should be converted to return KResults.
2021-01-20 23:20:02 +01:00
Andreas Kling
e279b45aed Kernel: Make BlockBasedFS read/write functions return a KResult
This way, if something goes wrong, we get to keep the actual error.
Also, KResults are nodiscard, so we have to deal with that in Ext2FS
instead of just silently ignoring I/O errors(!)
2021-01-20 22:57:36 +01:00
Linus Groh
678919e9c1 Kernel: Set "pledge_violation" coredump metadata in REQUIRE_PROMISE()
Similar to LibC storing an assertion message before aborting, process
death by pledge violation now sets a "pledge_violation" key with the
respective pledge name as value in its coredump metadata, which the
CrashReporter will then show.
2021-01-20 21:01:15 +01:00
Andreas Kling
b7248be251 Kernel: Allow sys$chmod() to change the sticky bit
We were incorrectly masking off the sticky bit when setting file modes.
2021-01-19 20:28:13 +01:00
Andreas Kling
8601108e21 Kernel: Implement the same symlink protection as Linux
Path resolution will now refuse to follow symlinks in some cases where
you don't own the symlink, or when it's in a sticky world-writable
directory and the link has a different owner than the directory.

The point of all this is to prevent classic TOCTOU bugs in /tmp etc.

Fixes #4934
2021-01-19 20:28:09 +01:00
Andreas Kling
9681e3eca0 Kernel: Implement the same hard link protection as Linux
sys$link() will now fail to create hard links in some cases where you
don't own or have write access to the link target.

Work towards #4934
2021-01-19 20:28:04 +01:00
Jean-Baptiste Boric
6677ab1ccd Boot: Fix undefined Multiboot behaviors
Both ESP and GDTR are left undefined by the Multiboot specification and
OS images must not rely on these values to be valid. Fix the undefined
behaviors so that booting with PXELINUX does not triple-fault the CPU.
2021-01-19 09:03:37 +01:00
Linus Groh
2cc3d68615 Kernel+LibC: Add _SC_TTY_NAME_MAX 2021-01-18 22:28:56 +01:00
Andreas Kling
a6917465d7 Kernel: Assert on attempt to mark inode metadata dirty on read-only FS 2021-01-17 21:32:59 +01:00
Andreas Kling
121594ace2 Kernel: Remove /proc/PID/vmobjects
This file was useful for debugging a long time ago, but has bitrotted
at this point. Instead of updating it, let's just remove it since
nothing is using it.
2021-01-17 21:16:13 +01:00
Andreas Kling
cfe54f86bd Kernel: Remove unused /proc/mm file
This was a file I used very early on to dump information about kernel
VM objects. It's long since superseded by other JSON-based files.
2021-01-17 21:14:20 +01:00
Andreas Kling
57a2394cb4 Kernel: Unbreak /proc/PID/root symlink
The generator callback for this file was mistakenly returning false
on success, which caused the kernel to fail sys$readlink() with ENOENT.
2021-01-17 21:11:21 +01:00
Tom
1d621ab172 Kernel: Some futex improvements
This adds support for FUTEX_WAKE_OP, FUTEX_WAIT_BITSET, FUTEX_WAKE_BITSET,
FUTEX_REQUEUE, and FUTEX_CMP_REQUEUE, as well well as global and private
futex and absolute/relative timeouts against the appropriate clock. This
also changes the implementation so that kernel resources are only used when
a thread is blocked on a futex.

Global futexes are implemented as offsets in VMObjects, so that different
processes can share a futex against the same VMObject despite potentially
being mapped at different virtual addresses.
2021-01-17 20:30:31 +01:00
Tom
b17a889320 Kernel: Add safe atomic functions
This allows us to perform atomic operations on potentially unsafe
user space pointers.
2021-01-17 20:30:31 +01:00
Andreas Kling
992f513ad2 Kernel: Limit exec arguments and environment to 1/8th of stack each
This sort-of matches what some other systems do and seems like a
generally sane thing to do instead of allowing programs to spawn a
child with a nearly full stack.
2021-01-17 18:29:56 +01:00
Andreas Kling
6613cef2f8 Ext2FS: Update block group directory count after directory removal
When freeing an inode, we were checking if it's a directory *after*
wiping the inode metadata. This caused us to forget updating the block
group descriptor with the new directory count.
2021-01-17 16:56:07 +01:00
Andreas Kling
1730c23775 Kernel: Remove a bunch of no-longer-necessary SmapDisablers
We forgot to remove the automatic SMAP disablers after fixing up all
this code to not access userspace memory directly. Let's lock things
down at last. :^)
2021-01-17 15:03:07 +01:00
Andreas Kling
647cfcb641 Kernel: Prune uninteresting kernel frames from profiling samples
Start capturing the sample stacks at the EIP/EBP of the pre-empted
thread instead of capturing EBP in the sampling function itself.
2021-01-17 14:36:53 +01:00
Andreas Kling
bf0719092f Kernel+Userland: Remove shared buffers (shbufs)
All users of this mechanism have been switched to anonymous files and
passing file descriptors with sendfd()/recvfd().

Shbufs got us where we are today, but it's time we say good-bye to them
and welcome a much more idiomatic replacement. :^)
2021-01-17 09:07:32 +01:00
Andreas Kling
05dbfe9ab6 Kernel: Remove sys$shbuf_seal() and userland wrappers
There are no remaining users of this syscall so let it go. :^)
2021-01-17 00:18:01 +01:00
Andreas Kling
0571125dfe Kernel: Remove some unused code in the SharedBuffer class 2021-01-16 22:43:03 +01:00
Andreas Kling
b818cf898e Kernel+Userland: Remove sys$shbuf_allow_all() and userland wrappers
Nobody is using globally shared shbufs anymore, so let's remove them.
2021-01-16 22:43:03 +01:00
Ben Wiederhake
ea5825f2c9 Kernel+LibC: Make sys$getcwd truncate the result silently
This gives us the superpower of knowing the ideal buffer length if it fails.
See also https://github.com/SerenityOS/serenity/discussions/4357
2021-01-16 22:40:53 +01:00
Ben Wiederhake
68416d7293 Kernel: Make realpath return silently truncated data
For context, see https://github.com/SerenityOS/serenity/discussions/4357
2021-01-16 22:40:53 +01:00
Ben Wiederhake
2a8baf9582 Kernel: Remove unused 'ImmutableBufferArgument' 2021-01-16 22:40:53 +01:00
Brendan Coles
1fa9d9dd68 Kernel: execve: find_elf_interpreter_for_executable: Fix dbgln 2021-01-16 22:36:46 +01:00
Andreas Kling
01c2480eb3 Kernel+LibC+WindowServer: Remove unused thread/process boost mechanism
The priority boosting mechanism has been broken for a very long time.
Let's remove it from the codebase and we can bring it back the day
someone feels like implementing it in a working way. :^)
2021-01-16 14:52:04 +01:00
Andreas Kling
43109f9614 Kernel: Remove unused syscall sys$minherit()
This is no longer used. We can bring it back the day we need it.
2021-01-16 14:52:04 +01:00
Andreas Kling
de31e82f97 Kernel: Remove sys$shbuf_set_volatile() and userland wrappers
There are no remaining users of this syscall so let's remove it! :^)
2021-01-16 14:52:04 +01:00
Ben Wiederhake
38c5b3f788 Kernel: Fix inverted logic in KResultOr
This silly inversion has survived so long because we don't exercise the
'unhappy paths' enough. :^)
2021-01-16 12:53:23 +01:00
asynts
94bb544c33 Everywhere: Replace a bundle of dbg with dbgln.
These changes are arbitrarily divided into multiple commits to make it
easier to find potentially introduced bugs with git bisect.

This commit touches some dbg() calls which are enclosed in macros. This
should be fine because with the new constexpr stuff, we ensure that the
stuff actually compiles.
2021-01-16 11:54:35 +01:00
Linus Groh
1ccc2e6482 Kernel: Store process arguments and environment in coredumps
Currently they're only pushed onto the stack but not easily accessible
from the Process class, so this adds a Vector<String> for both.
2021-01-15 23:26:47 +01:00
Linus Groh
057ae36e32 Kernel: Prevent threads from being destructed between die() and finalize()
Killing remaining threads already happens in Process::die(), but
coredumps are only written in Process::finalize(). We need to keep a
reference to each of those threads to prevent them from being destructed
between those two functions, otherwise coredumps will only ever contain
information about the last remaining thread.

Fixes the underlying problem of #4778, though the UI will need
refinements to not show every thread's backtrace mashed together.
2021-01-15 23:26:47 +01:00
Linus Groh
568cde5e23 Kernel+LibELF+LibCoreDump+CrashReporter: Use JSON for ProcessInfo
This is in preparation of adding (much) more process information to
coredumps. As we can only have one null-terminated char[] of arbitrary
length in each struct it's now a single JSON blob, which is a great fit:
easily extensible in the future and allows for key/value pairs and even
nested objects, which will be used e.g. for the process environment, for
example.
2021-01-15 23:26:47 +01:00
Andreas Kling
64b0d89335 Kernel: Make Process::allocate_region*() return KResultOr<Region*>
This allows region allocation to return specific errors and we don't
have to assume every failure is an ENOMEM.
2021-01-15 19:10:30 +01:00
Andreas Kling
7899e14e72 Kernel: Make sys$anon_create() require the "stdio" promise if pledged 2021-01-15 19:10:30 +01:00
Andreas Kling
a525d0271c Kernel: Fix bogus negation of alloc_fd() error in sys$anon_create()
Thanks to Idan for spotting this!
2021-01-15 15:13:48 +01:00
Andreas Kling
fb4993f067 Kernel: Add anonymous files, created with sys$anon_create()
This patch adds a new AnonymousFile class which is a File backed by
an AnonymousVMObject that can only be mmap'ed and nothing else, really.

I'm hoping that this can become a replacement for shbufs. :^)
2021-01-15 13:56:47 +01:00
Tom
a51fbb13e8 Kernel: Make Locker remember whether the lock is held
This allows temporarily unlocking a lock or re-locking it, and it will
only unlock if it is still being held.

Fixes #4352
2021-01-15 08:12:08 +01:00
Mart G
ff7dee49f6 ProcFS: Ignore directories in refresh_data(). 2021-01-14 08:35:18 +01:00
Andreas Kling
4fa8435310 Kernel: Use current process EUID in doing profiling access control 2021-01-12 23:34:01 +01:00
Andreas Kling
13d7c09125 Libraries: Move to Userland/Libraries/ 2021-01-12 12:17:46 +01:00
Lenny Maiorani
e6f907a155 AK: Simplify constructors and conversions from nullptr_t
Problem:
- Many constructors are defined as `{}` rather than using the ` =
  default` compiler-provided constructor.
- Some types provide an implicit conversion operator from `nullptr_t`
  instead of requiring the caller to default construct. This violates
  the C++ Core Guidelines suggestion to declare single-argument
  constructors explicit
  (https://isocpp.github.io/CppCoreGuidelines/CppCoreGuidelines#c46-by-default-declare-single-argument-constructors-explicit).

Solution:
- Change default constructors to use the compiler-provided default
  constructor.
- Remove implicit conversion operators from `nullptr_t` and change
  usage to enforce type consistency without conversion.
2021-01-12 09:11:45 +01:00
Andreas Kling
6412e7e8e3 Kernel: Remove /proc/mounts
Everyone was already using /proc/df which has all the info anyway.
2021-01-11 23:00:07 +01:00
Andreas Kling
9b907e27ce Kernel: Remove /proc/inodes
There was nothing interesting in this file.
2021-01-11 22:51:28 +01:00
Andreas Kling
a0506cb39e Kernel: Only send SIGTTOU if TTY termios has TOSTOP flag
Fixes #4909
2021-01-11 22:40:40 +01:00
Andreas Kling
f03800cee3 Kernel: Add dedicated "ptrace" pledge promise
The vast majority of programs don't ever need to use sys$ptrace(),
and it seems like a high-value system call to prevent a compromised
process from using.

This patch moves sys$ptrace() from the "proc" promise to its own,
new "ptrace" promise and updates the affected apps.
2021-01-11 22:32:59 +01:00
Andreas Kling
f7435dd95f Kernel: Remove MM_DEBUG debug spam code
This was too spammy to ever actually be used anyway.
2021-01-11 22:09:40 +01:00
Andreas Kling
7c4ddecacb Kernel: Convert a bunch of String::format() => String::formatted() 2021-01-11 22:07:01 +01:00
Sahan Fernando
9bf76a85c8 Everywhere: Fix incorrect uses of String::format and StringBuilder::appendf
These changes are arbitrarily divided into multiple commits to make it
easier to find potentially introduced bugs with git bisect.
2021-01-11 21:06:32 +01:00
Sahan Fernando
099b83fd28 Everywhere: Fix incorrect uses of String::format and StringBuilder::appendf
These changes are arbitrarily divided into multiple commits to make it
easier to find potentially introduced bugs with git bisect.
2021-01-11 21:06:32 +01:00
Lenny Maiorani
1b2364846f SinglyLinkedList: Implement find in terms of AK::find
Problem:
- The implementation of `find` is coupled to the implementation of
  `SinglyLinkedList`.

Solution:
- Decouple the implementation of `find` from the class by using a
  generic `find` algorithm.
2021-01-11 19:45:05 +01:00
Lenny Maiorani
853cb8af5c DoublyLinkedList: Implement find in terms of AK::find
Problem:
- The implementation of `find` is coupled to the implementation of
  `DoublyLinkedList`.
- `append` and `prepend` are implemented multiple times so that
  r-value references can be moved from into the new node. This is
  probably not called very often because a pr-value or x-value needs
  to be used here.

Solution:
- Decouple the implementation of `find` from the class by using a
  generic `find` algorithm.
- Make `append` and `prepend` be function templates so that they can
  have binding references which can be forwarded.
2021-01-11 19:45:05 +01:00
Andreas Kling
5c73c1bff8 Kernel: Don't dump perfcore for non-dumpable processes
Fixes #4904
2021-01-11 18:53:45 +01:00
asynts
723effd051 Everywhere: Replace a bundle of dbg with dbgln.
These changes are arbitrarily divided into multiple commits to make it
easier to find potentially introduced bugs with git bisect.Everything:
2021-01-11 11:55:47 +01:00
asynts
5931758dbc Everywhere: Replace a bundle of dbg with dbgln.
These changes are arbitrarily divided into multiple commits to make it
easier to find potentially introduced bugs with git bisect.Everything:
2021-01-11 11:55:47 +01:00
asynts
dca6f1f49b Everywhere: Replace a bundle of dbg with dbgln.
These changes are arbitrarily divided into multiple commits to make it
easier to find potentially introduced bugs with git bisect.Everything:
2021-01-11 11:55:47 +01:00
asynts
872f2a3b90 Everywhere: Replace a bundle of dbg with dbgln.
These changes are arbitrarily divided into multiple commits to make it
easier to find potentially introduced bugs with git bisect.Everything:
2021-01-11 11:55:47 +01:00
asynts
11d651d447 Everywhere: Replace a bundle of dbg with dbgln.
These changes are arbitrarily divided into multiple commits to make it
easier to find potentially introduced bugs with git bisect.Everything:
2021-01-11 11:55:47 +01:00
Andreas Kling
603147f47a Kernel: Fix perfcore filename generation build error 2021-01-11 11:37:14 +01:00
Andreas Kling
5dafb72370 Kernel+Profiler: Make profiling per-process and without core dumps
This patch merges the profiling functionality in the kernel with the
performance events mechanism. A profiler sample is now just another
perf event, rather than a dedicated thing.

Since perf events were already per-process, this now makes profiling
per-process as well.

Processes with perf events would already write out a perfcore.PID file
to the current directory on death, but since we may want to profile
a process and then let it continue running, recorded perf events can
now be accessed at any time via /proc/PID/perf_events.

This patch also adds information about process memory regions to the
perfcore JSON format. This removes the need to supply a core dump to
the Profiler app for symbolication, and so the "profiler coredump"
mechanism is removed entirely.

There's still a hard limit of 4MB worth of perf events per process,
so this is by no means a perfect final design, but it's a nice step
forward for both simplicity and stability.

Fixes #4848
Fixes #4849
2021-01-11 11:36:00 +01:00
Itamar
f259d96871 Kernel: Avoid collision between dynamic loader and main program
When loading non position-independent programs, we now take care not to
load the dynamic loader at an address that collides with the location
the main program wants to load at.

Fixes #4847.
2021-01-10 22:04:43 +01:00
Itamar
40a8159c62 Kernel: Plumb the elf header of the main program down to Process::load
This will enable us to take the desired load address of non-position
independent programs into account when randomizing the load address
of the dynamic loader.
2021-01-10 22:04:43 +01:00
Andrew Kaster
a5e557472c Kernel: Defer handling of key press events in VirtualConsole
Trying to pass these onto the Terminal while handling an IRQ is a recipe
for disaster. Use Processor::deferred_call_queue to create an ad-hoc
"second half" of the interrupt handler.

Fixes #4889
2021-01-10 21:50:08 +01:00
AnotherTest
44305ea214 LibVT: Respect the Negative attribute when drawing text
This makes the "reverse video" SGR actually work.
2021-01-10 16:58:08 +01:00
Andreas Kling
190e0e1551 Kernel+SystemServer+CrashDaemon: Better control where we put core dumps
SystemServer now creates the /tmp/coredump and /tmp/profiler_coredumps
directories at startup, ensuring that they are owned by root, and with
basic 0755 permissions.

The kernel will also now refuse to put core dumps in a directory that
doesn't fulfill the following criteria:

- Owned by 0:0
- Directory with sticky bit not set
- 0755 permissions

Fixes #4435
Fixes #4850
2021-01-10 11:38:28 +01:00
Andreas Kling
795bccbf69 Kernel: Don't allow non-root, non-owners to rmdir any child of sticky
We were not handling sticky parents properly in sys$rmdir(). Child
directories of a sticky parent should not be rmdir'able by just anyone.
Only the owner and root.

Fixes #4875.
2021-01-10 10:14:33 +01:00
Andreas Kling
77656aed8e Ext2FS: Zero out new space when growing an inode
Before this change, truncating an Ext2FS inode to a larger size than it
was before would give you uninitialized on-disk data.

Fix this by zeroing out all the new space when doing an inode resize.

This is pretty naively implemented via Inode::write_bytes() and there's
lots of room for cleverness here in the future.
2021-01-09 22:08:53 +01:00
Andreas Kling
f0093e5d59 Ext2FS: Convert dbg() to dbgln()
Also remove some dbg()'s that were printing incorrect information.
2021-01-09 21:54:27 +01:00
asynts
938e5c7719 Everywhere: Replace a bundle of dbg with dbgln.
These changes are arbitrarily divided into multiple commits to make it
easier to find potentially introduced bugs with git bisect.Everything:

The modifications in this commit were automatically made using the
following command:

    find . -name '*.cpp' -exec sed -i -E 's/dbg\(\) << ("[^"{]*");/dbgln\(\1\);/' {} \;
2021-01-09 21:11:09 +01:00
asynts
4e8fd0216b Everywhere: Replace a bundle of dbg with dbgln.
These changes are arbitrarily divided into multiple commits to make it
easier to find potentially introduced bugs with git bisect.
2021-01-09 21:11:09 +01:00
asynts
019c9eb749 Everywhere: Replace a bundle of dbg with dbgln.
These changes are arbitrarily divided into multiple commits to make it
easier to find potentially introduced bugs with git bisect.
2021-01-09 21:11:09 +01:00
asynts
1160817a9e AK: Add Formatter<FormatString> as helper class. 2021-01-09 21:11:09 +01:00
Andreas Kling
8ff0afd829 Kernel: Defer switching the paging scope in ptrace(PT_POKE) a little
If we can fail with EFAULT early, might as well avoid switching the
paging scope.
2021-01-09 15:42:03 +01:00
Andreas Kling
4714b04d32 Kernel: Don't put gratuitous cursing in kernel log 2021-01-09 14:03:06 +01:00
Jesse Buhagiar
6df15aca7e Kernel/USB: Implement test transfer
We can now test a _very_ basic transaction via `do_debug_transfer()`.
This function merely attaches some TDs to the LSCTRL queue head
and points some input and output buffers. We then sense an interrupt
with USBSTS value of 1, meaning Interrupt On Completion
(of the transaction). At this point, the input buffer is filled with
some data.
2021-01-09 13:48:15 +01:00
Jesse Buhagiar
3fb7e98e42 Kernel/USB: Simplify schedule
It seems like Haiku and Grub do this, so let's not bother
with any fancy timing stuff for now (to at least get
_something_ working...)
2021-01-09 13:48:15 +01:00
Jesse Buhagiar
ff4afe17be Kernel/USB: Ignore shared IRQs
According the USB spec/UHCI datasheet (as well as the Linux and
BSD source code), if we receive an IRQ and USBSTS is 0, then
the IRQ does not belong to us and we should immediately jump
out of the handler.
2021-01-09 13:48:15 +01:00
Jesse Buhagiar
770a729e59 Kernel/USB: Add basic root port detection/management
We can now read/write to the two root ports exposed to the
UHCI controller, and detect when a device is plugged in or
out via a kernel process that constantly scans the port
for any changes. This is very basic, but is a bit of fun to see
the kernel detecting hardware on the fly :^)
2021-01-09 13:48:15 +01:00
Jesse Buhagiar
a5f895d251 Kernel/USB: Implement UHCI Data structures
Implemented both Queue Heads and Transfer Descriptors. These
are required to actually perform USB transactions. The UHCI
driver sets up a pool of these that can be allocated when we
need them. It seems some drivers have these statically
allocated, so it might be worth looking into that, but
for now, the simple way seems to be to allocate them on
the fly as we need them, and then release them.
2021-01-09 13:48:15 +01:00
Jesse Buhagiar
375d269b21 Kernel/USB: Prevent system crash via correct UHCI inititilisation
It seems that not setting the framelist address register
was causing the entire system to lock up as it generated an insane
interrupt storm in the IRQ handler for the UHCI controller.
We now allocate a 4KiB aligned page via
`MemoryManager::allocate_supervisor_physical_page()` and set every
value to 1. In effect, this creates a framelist with each entry
being a "TERMINATE" entry in which the controller stalls until its'
1mS time slice is up.

Some more registers have also been set for consistency, though it
seems like this don't need to be set explicitly in software.
2021-01-09 13:48:15 +01:00
Jesse Buhagiar
33d73c600c Kernel/USB: Move USB classes into USB namespace
The entire USB spec involves more than just UHCI, so
let's put everything into it's own nice namespace :^)
2021-01-09 13:48:15 +01:00
Jesse Buhagiar
0dbb9361b8 Kernel/USB: Move USB related files to into specific directory
As the USB/UHCI driver grows in size, it'll be much cleaner to have
all of the USB related files in one folder where they can be easily
accessed :^)
2021-01-09 13:48:15 +01:00
Davide Carella
ca9e0a70f5 Syscall: Changed 'setkeymap' to take also the Shift+AltGr map. 2021-01-06 09:32:08 +01:00
Tom
d3e6cdf21f Kernel: Provide consistent memory stats in ProcFS
We should take the MM lock when gathering all the statistics that
we need so that the values are consistent.
2021-01-05 10:59:00 +01:00
Andreas Kling
d991658794 Kernel+LibC: Tidy up assertion failures with a dedicated syscall
This patch adds sys$abort() which immediately crashes the process with
SIGABRT. This makes assertion backtraces a lot nicer by removing all
the gunk that otherwise happens between __assertion_failed() and
actually crashing from the SIGABRT.
2021-01-04 21:57:30 +01:00
Tom
901ef3f1c8 Kernel: Specify default memory order for some non-synchronizing Atomics 2021-01-04 19:13:52 +01:00
Linus Groh
0571a17f57 Kernel+LibELF: Store termination signal in coredump ProcessInfo 2021-01-03 22:12:42 +01:00
Tom
f98ca35b83 Kernel: Improve ProcFS behavior in low memory conditions
When ProcFS could no longer allocate KBuffer objects to serve calls to
read, it would just return 0, indicating EOF. This then triggered
parsing errors because code assumed it read the file.

Because read isn't supposed to return ENOMEM, change ProcFS to populate
the file data upon file open or seek to the beginning. This also means
that calls to open can now return ENOMEM if needed. This allows the
caller to either be able to successfully open the file and read it, or
fail to open it in the first place.
2021-01-03 22:12:19 +01:00
William Marlow
747e8de96a Kernel+Loader.so: Allow dynamic executables without an interpreter
Commit a3a9016701 removed the PT_INTERP header
from Loader.so which cleaned up some kernel code in execve. Unfortunately
it prevents Loader.so from being run as an executable
2021-01-03 19:45:16 +01:00
Tom
c943ca95c4 Kernel: Fix ProcFS returning Inodes that are being deleted
There is a window between dropping the last reference and removing
a ProcFSInode from the lookup map. So, when looking up we need to
check if that Inode is being destructed.
2021-01-03 18:57:55 +01:00
Tom
0d44ee6f2b Kernel: Ignore TLB flush requests for user addresses of other processes
If a TLB flush request is broadcast to other processors and the addresses
to flush are user mode addresses, we can ignore such a request on the
target processor if the page directory currently in use doesn't match
the addresses to be flushed. We still need to broadcast to all processors
in that case because the other processors may switch to that same page
directory at any time.
2021-01-02 20:56:35 +01:00
Tom
c630669304 Kernel: If a VMObject is shared, broadcast page remappings
If we remap pages (e.g. lazy allocation) inside a VMObject that is
shared among more than one region, broadcast it to any other region
that may be mapping the same page.
2021-01-02 20:56:35 +01:00
Tom
e3190bd144 Revert "Kernel: Allocate shared memory regions immediately"
This reverts commit fe6b3f99d1.
2021-01-02 20:56:35 +01:00
Andreas Kling
fe6b3f99d1 Kernel: Allocate shared memory regions immediately
Lazily committed shared memory was not working in situations where one
process would write to the memory and another would only read from it.

Since the reading process would never cause a write fault in the shared
region, we'd never notice that the writing process had added real
physical pages to the VMObject. This happened because the lazily
committed pages were marked "present" in the page table.

This patch solves the issue by always allocating shared memory up front
and not trying to be clever about it.
2021-01-02 16:57:31 +01:00
Andreas Kling
5dae85afe7 Kernel: Pass "shared" flag to Region constructor
Before this change, we would sometimes map a region into the address
space with !is_shared(), and then moments later call set_shared(true).

I found this very confusing while debugging, so this patch makes us pass
the initial shared flag to the Region constructor, ensuring that it's in
the correct state by the time we first map the region.
2021-01-02 16:57:31 +01:00
Andreas Kling
9ec9d20e84 Kernel: Fix bad VMObject iteration in sys$purge()
We were fooling ourselves into thinking all VMObjects are anonymous and
then tried to call purge() on them as if they were.
2021-01-02 13:34:29 +01:00
Brian Gianforcaro
79328b2aba Kernel: Enable -fstack-protector-strong (again)
Insert stack canaries to find stack corruptions in the kernel.
It looks like this was enabled in the past (842716a) but appears to have been
lost during the CMake conversion.

The `-fstack-protector-strong` variant was chosen because it catches more issues
than `-fstack-protector`, but doesn't have substantial performance impact like
`-fstack-protector-all`.
2021-01-02 13:04:30 +01:00
Itamar
df5ea97c51 Kernel: Release scheduler_lock in ptrace once we know tracee is stopped
This fixes a kernel crash that occured when calling ptrace with PT_PEEK
on non paged-in memory.
The crash occurred because we were holding the scheduler lock while
trying to read from the disk's block device, which we do not allow.
Fixes #4740
2021-01-02 11:52:19 +01:00
Tom
60f5f48dd1 Kernel: Allocate profiling memory upfront
We need to allocate all pages for the profiler right away so that
we don't trigger page faults in the timer interrupt handler to
allocate them.

Fixes #4734
2021-01-02 11:36:48 +01:00
Brian Gianforcaro
84c2be604a
Kernel+LibELF: Use hex instead of decimal for stack offsets in back traces (#4728)
Hex is the de facto format for representing memory addresses, make backtraces
conform to that convention.
2021-01-02 01:38:43 +01:00
Andreas Kling
14493645e0 Kernel: Make Region::amount_shared() and amount_resident() lazy-aware
Don't count the lazy-committed page towards shared/resident amounts.
2021-01-02 00:47:55 +01:00
Tom
a1904b67e9 Kernel: Fix dirty page map bitmap
We also need to check against the new lazy allocation page
when generating the dirty page bitmap.
2021-01-02 00:10:21 +01:00
Tom
e87eaf5df0 Kernel: Fix memory corruption when rolling back regions in execve
We need to free the regions before reverting the paging scope to the
original one when rolling back changes due to an error. This fixes
silent memory corruption.
2021-01-01 23:43:44 +01:00
Tom
2f429bd2d5 Kernel: Pass new region owner to Region::clone 2021-01-01 23:43:44 +01:00
Tom
a0c91719d8 Kernel: Restore thread count if thread cannot be fully created 2021-01-01 23:43:44 +01:00
Tom
bf9be3ec01 Kernel: More gracefully handle out-of-memory when creating PageDirectory 2021-01-01 23:43:44 +01:00
Tom
ae956edf6e Kernel: Improve some low-memory situations with ext2 2021-01-01 23:43:44 +01:00
Tom
476f17b3f1 Kernel: Merge PurgeableVMObject into AnonymousVMObject
This implements memory commitments and lazy-allocation of committed
memory.
2021-01-01 23:43:44 +01:00
Tom
b2a52f6208 Kernel: Implement lazy committed page allocation
By designating a committed page pool we can guarantee to have physical
pages available for lazy allocation in mappings. However, when forking
we will overcommit. The assumption is that worst-case it's better for
the fork to die due to insufficient physical memory on COW access than
the parent that created the region. If a fork wants to ensure that all
memory is available (trigger a commit) then it can use madvise.

This also means that fork now can gracefully fail if we don't have
enough physical pages available.
2021-01-01 23:43:44 +01:00
Tom
e21cc4cff6 Kernel: Remove MAP_PURGEABLE from mmap
This brings mmap more in line with other operating systems. Prior to
this, it was impossible to request memory that was definitely committed,
instead MAP_PURGEABLE would provide a region that was not actually
purgeable, but also not fully committed, which meant that using such memory
still could cause crashes when the underlying pages could no longer be
allocated.

This fixes some random crashes in low-memory situations where non-volatile
memory is mapped (e.g. malloc, tls, Gfx::Bitmap, etc) but when a page in
these regions is first accessed, there is insufficient physical memory
available to commit a new page.
2021-01-01 23:43:44 +01:00
Tom
c3451899bc Kernel: Add MAP_NORESERVE support to mmap
Rather than lazily committing regions by default, we now commit
the entire region unless MAP_NORESERVE is specified.

This solves random crashes in low-memory situations where e.g. the
malloc heap allocated memory, but using pages that haven't been
used before triggers a crash when no more physical memory is available.

Use this flag to create large regions without actually committing
the backing memory. madvise() can be used to commit arbitrary areas
of such regions after creating them.
2021-01-01 23:43:44 +01:00
Tom
bc5d6992a4 Kernel: Memory purging improvements
This adds the ability for a Region to define volatile/nonvolatile
areas within mapped memory using madvise(). This also means that
memory purging takes into account all views of the PurgeableVMObject
and only purges memory that is not needed by all of them. When calling
madvise() to change an area to nonvolatile memory, return whether
memory from that area was purged. At that time also try to remap
all memory that is requested to be nonvolatile, and if insufficient
pages are available notify the caller of that fact.
2021-01-01 23:43:44 +01:00
Liav A
9dc8bea3e7 Kernel: Allow to boot from a partition with partition UUID
Instead of specifying the boot argument to be root=/dev/hdXY, now
one can write root=PARTUUID= with the right UUID, and if the partition
is found, the kernel will boot from it.

This feature is mainly used with GUID partitions, and is considered to
be the most reliable way for the kernel to identify partitions.
2021-01-01 22:59:48 +01:00
Andreas Kling
7c3b6b10e4 Kernel: Remove the limited use of AK::TypeTraits we had in the kernel
This was only used for VMObject and we can do without it there. This is
preparation for migrating to dynamic_cast-based helpers in userspace.
2021-01-01 15:32:44 +01:00
Andrew Kaster
350d4d3543 Meta: Enable RTTI for Userspace programs
RTTI is still disabled for the Kernel, and for the Dynamic Loader. This
allows for much less awkward navigation of class heirarchies in LibCore,
LibGUI, LibWeb, and LibJS (eventually). Measured RootFS size increase
was < 1%, and libgui.so binary size was ~3.3%. The small binary size
increase here seems worth it :^)
2021-01-01 14:45:09 +01:00
Brian Gianforcaro
ab6ee9f7b2 CMake: Remove some trailing whitespace from a few CMakeLists.txt files 2021-01-01 14:37:04 +01:00
Andrew Kaster
a3a9016701 DynamicLoader: Tell the linker to not add a PT_INTERP header
Use the GNU LD option --no-dynamic-linker. This allows uncommenting some
code in the Kernel that gets upset if your ELF interpreter has its own
interpreter.
2021-01-01 02:12:28 +01:00
Linus Groh
bbe787a0af Everywhere: Re-format with clang-format-11
Compared to version 10 this fixes a bunch of formatting issues, mostly
around structs/classes with attributes like [[gnu::packed]], and
incorrect insertion of spaces in parameter types ("T &"/"T &&").
I also removed a bunch of // clang-format off/on and FIXME comments that
are no longer relevant - on the other hand it tried to destroy a couple of
neatly formatted comments, so I had to add some as well.
2020-12-31 21:51:00 +01:00
Tom
72440d90fe Kernel: Fix BlockCondition::unblock return value
BlockCondition::unblock should return true if it unblocked at
least one thread, not if iterating the blockers had been stopped.
This is a regression introduced by 49a76164c.

Fixes #4670
2020-12-31 10:52:58 +01:00
Tom
82c4812730 Kernel: Remove flawed SharedInodeVMObject assertion
This assertion cannot be safely/reliably made in the
~SharedInodeVMObject destructor. The problem is that
Inode::is_shared_vmobject holds a weak reference to the instance
that is being destroyed (ref count 0). Checking the pointer using
WeakPtr::unsafe_ptr will produce nullptr depending on timing in
this case, and WeakPtr::safe_ref will reliably produce a nullptr
as soon as the reference count drops to 0. The only case where
this assertion could succeed is when WeakPtr::unsafe_ptr returned
the pointer because it won the race against revoking it. And
because WeakPtr::safe_ref will always return a nullptr, we cannot
reliably assert this from the ~SharedInodeVMObject destructor.

Fixes #4621
2020-12-31 10:52:45 +01:00
Andreas Kling
1fdd39ff14 Kernel: Sprinkle some lockers in Inode
It did look pretty suspicious the way we were accessing members in some
of these functions without taking the lock first.
2020-12-31 02:10:31 +01:00
Luke
0f66589007 Everywhere: Fix more typos 2020-12-31 01:47:41 +01:00
Tom
22250780ff Kernel: Fix heap expansions deadlock
If a heap expansion is triggered by allocating from e.g. the
RangeAllocator, which may be holding a spin lock, we cannot
immediately allocate another block of backup memory, which could
require the same locks to be acquired. So, defer allocating the
backup memory

Fixes #4675
2020-12-31 01:15:37 +01:00
asynts
7e62ffbc6e AK+Format: Remove TypeErasedFormatParams& from format function. 2020-12-30 20:33:53 +01:00
Luke
865f5ed4f6 Kernel: Prevent sign bit extension when creating a PDPTE
When doing the cast to u64 on the page directory physical address,
the sign bit was being extended. This only beomes an issue when
crossing the 2 GiB boundary. At >= 2 GiB, the physical address
has the sign bit set. For example, 0x80000000.

This set all the reserved bits in the PDPTE, causing a GPF
when loading the PDPT pointer into CR3. The reserved bits are
presumably there to stop you writing out a physical address that
the CPU physically cannot handle, as the size of the reserved bits
is determined by the physical address width of the CPU.

This fixes this by casting to FlatPtr instead. I believe the sign
extension only happens when casting to a bigger type. I'm also using
FlatPtr because it's a pointer we're writing into the PDPTE.
sizeof(FlatPtr) will always be the same size as sizeof(void*).

This also now asserts that the physical address in the PDPTE is
within the max physical address the CPU supports. This is better
than getting a GPF, because CPU::handle_crash tries to do the same
operation that caused the GPF in the first place. That would cause
an infinite loop of GPFs until the stack was exhausted, causing a
triple fault.

As far as I know and tested, I believe we can now use the full 32-bit
physical range without crashing.

Fixes #4584. See that issue for the full debugging story.
2020-12-30 20:33:15 +01:00
Linus Groh
d84b96bddc Kernel: Embed a Metadata notes entry in coredumps 2020-12-30 16:28:27 +01:00
Linus Groh
91332515a6 Kernel: Add sys$set_coredump_metadata() syscall
This can be used by applications to store information (key/value pairs)
likely useful for debugging, which will then be embedded in the coredump.
2020-12-30 16:28:27 +01:00
Linus Groh
6fe6e0a36a Kernel: Embed a ProcessInfo notes entry in coredumps 2020-12-30 15:00:17 +01:00
Tom
49a76164c8 Kernel: Consolidate the various BlockCondition::unblock variants
The unblock_all variant used to ASSERT if a blocker didn't unblock,
but it wasn't clear from the name that it would do that. Because
the BlockCondition already asserts that no blockers are left at
destruction time, it would still catch blockers that haven't been
unblocked for whatever reason.

Fixes #4496
2020-12-30 13:23:17 +01:00
asynts
50d24e4f98 AK: Make binary_search signature more generic. 2020-12-30 02:13:30 +01:00
Tom
c2332780ee Kernel: Fix HPET::update_time to set ticks within the valid range
ticks_this_second must be less than the ticks per second (frequency).
2020-12-30 02:11:06 +01:00
meme
23b23cee5a Build: Support non-i686 toolchains
* Add SERENITY_ARCH option to CMake for selecting the target toolchain
* Port all build scripts but continue to use i686
* Update GitHub Actions cache to include BuildIt.sh
2020-12-29 17:42:04 +01:00
Andreas Kling
af28a8ad11 Kernel: Hold InodeVMObject reference while inspecting it in sys$mmap() 2020-12-29 15:43:35 +01:00
Andreas Kling
b8db585a83 Kernel: Remove unnecessary non-const Inode::shared_vmobject() 2020-12-29 15:43:35 +01:00
Andreas Kling
30dbe9c78a Kernel+LibC: Add a very limited sys$mremap() implementation
This syscall can currently only remap a shared file-backed mapping into
a private file-backed mapping.
2020-12-29 02:20:43 +01:00
Luke
b980782343 Kernel/VM: Make local_offset in PhysicalRegion::find_one_free_page unsigned
An extension to #4613, as I didn't notice that it also happens here.
2020-12-29 02:20:26 +01:00
Luke
eb38fe4a82 Kernel/VM: Make local_offset in PhysicalRegion::free_page_at unsigned
Anything above or equal to the 2 GB mark has the left most bit set
(0x8000...), which was falsely interpreted as negative due to
local_offset being signed.

This makes it unsigned by using FlatPtr. To check for underflow as
was intended, lets use Checked instead.

Fixes #4585
2020-12-29 01:41:16 +01:00
Andreas Kling
43d9fe15f9 Revert "Kernel: Convert read_block method to get a reference instead of pointer"
This reverts commit 092a13211a.

Fixes #4611.
2020-12-29 00:06:52 +01:00
Liav A
72b1998f0d Kernel: Introduce a new partitioning subsystem
The partitioning code was very outdated, and required a full refactor.
The new subsystem removes duplicated code and uses more AK containers.

The most important change is that all implementations of the
PartitionTable class conform to one interface, which made it possible
to remove unnecessary code in the EBRPartitionTable class.

Finding partitions is now done in the StorageManagement singleton,
instead of doing so in init.cpp.

Also, now we don't try to find partitions on demand - the kernel will
try to detect if a StorageDevice is partitioned, and if so, will check
what is the partition table, which could be MBR, GUID or EBR.
Then, it will create DiskPartitionMetadata object for each partition
that is available in the partition table. This object will be used
by the partition enumeration code to create a DiskPartition with the
correct minor number.
2020-12-27 23:07:44 +01:00
Liav A
43d833d94f Kernel: Add DiskPartitionMetadata Class
This class will be used to describe a partition of a StorageDevice,
without creating a DiskPartition object.
2020-12-27 23:07:44 +01:00
Liav A
3a19e18d1e Kernel: Move Partition code files to the Storage folder
This folder is more appropriate for these files.
2020-12-27 23:07:44 +01:00
Liav A
247517cd4a Kernel: Introduce the DevFS
The DevFS along with DevPtsFS give a complete solution for populating
device nodes in /dev. The main purpose of DevFS is to eliminate the
need of device nodes generation when building the system.

Later on, DevFS will assist with exposing disk partition nodes.
2020-12-27 23:07:44 +01:00
Liav A
18e77aa285 Kernel: Add a method to determine the desired permissions of a Device
This method will be used later in DevFS, to set the appropriate
permissions for each device node.
2020-12-27 23:07:44 +01:00
Liav A
092a13211a Kernel: Convert read_block method to get a reference instead of pointer
BlockBasedFileSystem::read_block method should get a reference of
a UserOrKernelBuffer.

If we need to force caching a block, we will call other method to do so.
2020-12-27 23:07:44 +01:00
Nathan Lanza
d1891f67ac
AK: Use direct-list-initialization for Vector::empend() (#4564)
clang trunk with -std=c++20 doesn't seem to properly look for an
aggregate initializer here when the type being constructed is a simple
aggregate (e.g. `struct Thing { int a; int b; };`). This template fails
to compile in a usage added 12/16/2020 in `AK/Trie.h`.

Both forms of initialization are supposed to call the
aggregate-initializers but direct-list-initialization delegating to
aggregate initializers is a new addition in c++20 that might not be
implemented yet.
2020-12-27 23:06:37 +01:00
Brendan Coles
fae2304c67 Kernel: CoreDump::write_program_headers: set NOTE p_memsz to p_filesz 2020-12-27 22:45:25 +01:00
Andreas Kling
ddaedbca87 Kernel: Allow sys$rename() to rename symlinks
Previously, this syscall would try to rename the target of the link,
not the link itself.
2020-12-27 15:38:07 +01:00
Brian Gianforcaro
815d39886f Kernel: Tag more methods and types as [[nodiscard]]
Tag methods at where not obvserving the return value is an obvious error
with [[nodiscard]] to catch potential future bugs.
2020-12-27 11:09:30 +01:00
Tom
f1534ff36e Kernel: Take into account the time keeper's frequency (if no HPET)
The PIT is now also running at a rate of ~250 ticks/second, so rather
than assuming there are 1000 ticks/second we need to query the timer
being used for the actual frequency.

Fixes #4508
2020-12-27 01:17:50 +01:00
Andreas Kling
0e2b7f9c9a Kernel: Remove the per-process icon_id and sys$set_process_icon()
This was a goofy kernel API where you could assign an icon_id (int) to
a process which referred to a global shbuf with a 16x16 icon bitmap
inside it.

Instead of this, programs that want to display a process icon now
retrieve it from the process executable instead.
2020-12-27 01:16:56 +01:00
Andreas Kling
21ccbc2167 Kernel: Expose process executable paths in /proc/all 2020-12-27 01:16:56 +01:00
Andreas Kling
87492e723b Kernel: Lock target process when generating core dump
Dumping core can happen at the end of a profiling run, and in that case
we have to protect the target process and take the lock while iterating
over its region map.

Fixes #4509.
2020-12-27 01:16:56 +01:00
Tom
74fa894994 Kernel: Remove subheap from list before removing memory
When the ExpandableHeap calls the remove_memory function, the
subheap is assumed to be removed and freed entirely. remove_memory
may drop the underlying memory at any time, but it also may cause
further allocation requests. Not removing it from the list before
calling remove_memory could cause a memory allocation in that
subheap while remove_memory is executing. which then causes issues
once the underlying memory is actually freed.
2020-12-26 19:55:01 +01:00
AnotherTest
7b5aa06702 Kernel: Allow 'elevating' unveil permissions if implicitly inherited from '/'
This can happen when an unveil follows another with a path that is a
sub-path of the other one:
```c++
unveil("/home/anon/.config/whoa.ini", "rw");
unveil("/home/anon", "r"); // this would fail, as "/home/anon" inherits
                           // the permissions of "/", which is None.
```
2020-12-26 16:10:04 +01:00
AnotherTest
a9184fcb76 Kernel: Implement unveil() as a prefix-tree
Fixes #4530.
2020-12-26 11:54:54 +01:00
Lenny Maiorani
b2316701a8 Everywhere: void arguments to C functions
Problem:
- C functions with no arguments require a single `void` in the argument list.

Solution:
- Put the `void` in the argument list of functions in C header files.
2020-12-26 10:10:27 +01:00
Sahan Fernando
6b01d1cf14 LibC: Enable compiler warnings for printf format strings 2020-12-26 10:05:50 +01:00
Andreas Kling
1cfdaf96c4 Kernel: Reset the process dumpable flag on successful non-setid exec
Once we've committed to a new memory layout and non-setid credentials,
we can reset the dumpable flag.
2020-12-26 01:31:24 +01:00
Andreas Kling
82f86e35d6 Kernel+LibC: Introduce a "dumpable" flag for processes
This new flag controls two things:
- Whether the kernel will generate core dumps for the process
- Whether the EUID:EGID should own the process's files in /proc

Processes are automatically made non-dumpable when their EUID or EGID is
changed, either via syscalls that specifically modify those ID's, or via
sys$execve(), when a set-uid or set-gid program is executed.

A process can change its own dumpable flag at any time by calling the
new sys$prctl(PR_SET_DUMPABLE) syscall.

Fixes #4504.
2020-12-25 19:35:55 +01:00
Andreas Kling
3c9bd911b8 Kernel: Make /proc/PID directories owned by the EUID:EGID
This is instead of the UID:GID, since that was allowing some very bad
information leaks like spawning "su" as an unprivileged user and having
full /proc access to it.

Work towards #4504.
2020-12-25 19:35:55 +01:00
Andreas Kling
057c1d4798 Kernel: Fix build with E1000_DEBUG 2020-12-25 19:35:55 +01:00
Andreas Kling
ed5c26d698 AK: Remove custom %w format string specifier
This was a non-standard specifier alias for %04x. This patch replaces
all uses of it with new-style formatting functions instead.
2020-12-25 17:05:05 +01:00
Andreas Kling
cb2c8f71f4 AK: Remove custom %b format string specifier
This was a non-standard specifier alias for %02x. This patch replaces
all uses of it with new-style formatting functions instead.
2020-12-25 17:04:28 +01:00
Andreas Kling
89d3b09638 Kernel: Allocate new main thread stack before committing to exec
If the allocation fails (e.g ENOMEM) we want to simply return an error
from sys$execve() and continue executing the current executable.

This patch also moves make_userspace_stack_for_main_thread() out of the
Thread class since it had nothing in particular to do with Thread.
2020-12-25 16:22:01 +01:00
Andreas Kling
2f1712cc29 Kernel: Move ELF auxiliary vector building out of Process class
Process had a couple of members whose only purpose was holding on to
some temporary data while building the auxiliary vector. Remove those
members and move the vector building to a free function in execve.cpp
2020-12-25 15:23:35 +01:00
Andreas Kling
40e9edd798 LibELF: Move AuxiliaryValue into the ELF namespace 2020-12-25 14:48:30 +01:00
Andreas Kling
6c9a6bea1e Kernel+LibELF: Abort ELF executable load sooner when something fails
Make it possible to bail out of ELF::Image::for_each_program_header()
and then do exactly that if something goes wrong during executable
loading in the kernel.

Also make the errors we return slightly more nuanced than just ENOEXEC.
2020-12-25 14:42:42 +01:00
Andreas Kling
791b32e3c6 Kernel: Remove an unnecessary cast in sys$execve() 2020-12-25 14:16:35 +01:00
Andreas Kling
9c640e67ac Kernel: Don't fetch full inode metadata in sys$execve()
We only need the size, so let's not fetch all the metadata.
2020-12-25 14:15:33 +01:00
Andreas Kling
c3eddbcb49 Kernel: Add back missing ELF::Image validity check
If the image is not a valid ELF we should just fail ASAP.
2020-12-25 14:13:44 +01:00