beenull/ladybird
1
0
Fork 0
mirror of https://github.com/LadybirdBrowser/ladybird.git synced 2024-11-26 01:20:25 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 6
ladybird/Kernel/Locking/Mutex.cpp

378 lines
13 KiB
C++
Raw Normal View History

Meta: Add license header to source files As suggested by Joshua, this commit adds the 2-clause BSD license as a comment block to the top of every source file. For the first pass, I've just added myself for simplicity. I encourage everyone to add themselves as copyright holders of any file they've added or modified in some significant way. If I've added myself in error somewhere, feel free to replace it with the appropriate copyright holder instead. Going forward, all new source files should include a license header.
2020-01-18 08:38:21 +00:00
/*
* Copyright (c) 2018-2020, Andreas Kling <kling@serenityos.org>
Kernel: Stop using HashMap in Mutex This commit removes the usage of HashMap in Mutex, thereby making Mutex be allocation-free. In order to achieve this several simplifications were made to Mutex, removing unused code-paths and extra VERIFYs: * We no longer support 'upgrading' a shared lock holder to an exclusive holder when it is the only shared holder and it did not unlock the lock before relocking it as exclusive. NOTE: Unlike the rest of these changes, this scenario is not VERIFY-able in an allocation-free way, as a result the new LOCK_SHARED_UPGRADE_DEBUG debug flag was added, this flag lets Mutex allocate in order to detect such cases when debugging a deadlock. * We no longer support checking if a Mutex is locked by the current thread when the Mutex was not locked exclusively, the shared version of this check was not used anywhere. * We no longer support force unlocking/relocking a Mutex if the Mutex was not locked exclusively, the shared version of these functions was not used anywhere.
2022-01-28 23:47:18 +00:00
* Copyright (c) 2022, Idan Horowitz <idan.horowitz@serenityos.org>
Meta: Add license header to source files As suggested by Joshua, this commit adds the 2-clause BSD license as a comment block to the top of every source file. For the first pass, I've just added myself for simplicity. I encourage everyone to add themselves as copyright holders of any file they've added or modified in some significant way. If I've added myself in error somewhere, feel free to replace it with the appropriate copyright holder instead. Going forward, all new source files should include a license header.
2020-01-18 08:38:21 +00:00
*
Everything: Move to SPDX license identifiers in all files. SPDX License Identifiers are a more compact / standardized way of representing file license information. See: https://spdx.dev/resources/use/#identifiers This was done with the `ambr` search and replace tool. ambr --no-parent-ignore --key-from-file --rep-from-file key.txt rep.txt *
2021-04-22 08:24:48 +00:00
* SPDX-License-Identifier: BSD-2-Clause
Meta: Add license header to source files As suggested by Joshua, this commit adds the 2-clause BSD license as a comment block to the top of every source file. For the first pass, I've just added myself for simplicity. I encourage everyone to add themselves as copyright holders of any file they've added or modified in some significant way. If I've added myself in error somewhere, feel free to replace it with the appropriate copyright holder instead. Going forward, all new source files should include a license header.
2020-01-18 08:38:21 +00:00
*/
Meta: Split debug defines into multiple headers. The following script was used to make these changes: #!/bin/bash set -e tmp=$(mktemp -d) echo "tmp=$tmp" find Kernel \( -name '*.cpp' -o -name '*.h' \) | sort > $tmp/Kernel.files find . \( -path ./Toolchain -prune -o -path ./Build -prune -o -path ./Kernel -prune \) -o \( -name '*.cpp' -o -name '*.h' \) -print | sort > $tmp/EverythingExceptKernel.files cat $tmp/Kernel.files | xargs grep -Eho '[A-Z0-9_]+_DEBUG' | sort | uniq > $tmp/Kernel.macros cat $tmp/EverythingExceptKernel.files | xargs grep -Eho '[A-Z0-9_]+_DEBUG' | sort | uniq > $tmp/EverythingExceptKernel.macros comm -23 $tmp/Kernel.macros $tmp/EverythingExceptKernel.macros > $tmp/Kernel.unique comm -1 $tmp/Kernel.macros $tmp/EverythingExceptKernel.macros > $tmp/EverythingExceptKernel.unique cat $tmp/Kernel.unique | awk '{ print "#cmakedefine01 "$1 }' > $tmp/Kernel.header cat $tmp/EverythingExceptKernel.unique | awk '{ print "#cmakedefine01 "$1 }' > $tmp/EverythingExceptKernel.header for macro in $(cat $tmp/Kernel.unique) do cat $tmp/Kernel.files | xargs grep -l $macro >> $tmp/Kernel.new-includes ||: done cat $tmp/Kernel.new-includes | sort > $tmp/Kernel.new-includes.sorted for macro in $(cat $tmp/EverythingExceptKernel.unique) do cat $tmp/Kernel.files | xargs grep -l $macro >> $tmp/Kernel.old-includes ||: done cat $tmp/Kernel.old-includes | sort > $tmp/Kernel.old-includes.sorted comm -23 $tmp/Kernel.new-includes.sorted $tmp/Kernel.old-includes.sorted > $tmp/Kernel.includes.new comm -13 $tmp/Kernel.new-includes.sorted $tmp/Kernel.old-includes.sorted > $tmp/Kernel.includes.old comm -12 $tmp/Kernel.new-includes.sorted $tmp/Kernel.old-includes.sorted > $tmp/Kernel.includes.mixed for file in $(cat $tmp/Kernel.includes.new) do sed -i -E 's/#include <AK\/Debug\.h>/#include <Kernel\/Debug\.h>/' $file done for file in $(cat $tmp/Kernel.includes.mixed) do echo "mixed include in $file, requires manual editing." done
2021-01-25 15:07:10 +00:00
#include <Kernel/Debug.h>
Kernel: Move all code into the Kernel namespace
2020-02-16 00:27:42 +00:00
#include <Kernel/KSyms.h>
Kernel: Reduce LOCK_DEBUG ifdefs by utilizing Kernel::LockLocation The LOCK_DEBUG conditional code is pretty ugly for a feature that we only use rarely. We can remove a significant amount of this code by utilizing a zero sized fake type when not building in LOCK_DEBUG mode. This lets us keep the same API, but just let the compiler optimize it away when don't actually care about the location the caller came from.
2021-08-07 11:19:39 +00:00
#include <Kernel/Locking/LockLocation.h>
Kernel: Move Mutex into Locking/
2021-07-18 07:10:27 +00:00
#include <Kernel/Locking/Mutex.h>
Kernel: Rename SpinLock => Spinlock
2021-08-21 23:37:17 +00:00
#include <Kernel/Locking/Spinlock.h>
Kernel: Fix Lock race causing infinite spinning between two threads We need to account for how many shared lock instances the current thread owns, so that we can properly release such references when yielding execution. We also need to release the process lock when donating.
2020-12-14 23:36:22 +00:00
#include <Kernel/Thread.h>
Kernel: Don't check that interrupts are enabled during early boot The interrupts enabled check in the Kernel mutex is there so that we don't lock mutexes within a spinlock, because mutexes reenable interrupts and that will mess up the spinlock in more ways than one if the thread moves processors. This check is guarded behind a debug flag because it's too hard to fix all the problems at once, but we regressed and weren't even getting to init stage 2 with it enabled. With this commit, we get to stage 2 again. In early boot, there are no interrupts enabled and spinlocks used, so we can sort of kind of safely ignore the interrupt state. There might be a better solution with another boot state flag that checks whether APs are up (because they have interrupts enabled from the start) but that seems overkill.
2022-06-17 21:58:01 +00:00
extern bool g_in_early_boot;
Kernel: Move all code into the Kernel namespace
2020-02-16 00:27:42 +00:00
namespace Kernel {
Kernel: Convert lock debug APIs to east const
2021-08-07 11:58:07 +00:00
void Mutex::lock(Mode mode, [[maybe_unused]] LockLocation const& location)
Kernel: Introduce shared locking mode A Lock can now be held either in shared or exclusive mode. Multiple threads can hold the same lock in shared mode at one time, but if any thread holds the lock in exclusive mode, no other thread can hold it at the same time in either mode.
2020-04-18 09:21:00 +00:00
{
Kernel: Fix some problems with Thread::wait_on and Lock This changes the Thread::wait_on function to not enable interrupts upon leaving, which caused some problems with page fault handlers and in other situations. It may now be called from critical sections, with interrupts enabled or disabled, and returns to the same state. This also requires some fixes to Lock. To aid debugging, a new define LOCK_DEBUG is added that enables checking for Lock leaks upon finalization of a Thread.
2020-12-01 02:04:36 +00:00
// NOTE: This may be called from an interrupt handler (not an IRQ handler)
// and also from within critical sections!
Kernel: Convert Processor::in_irq() to static current_in_irq() This closes the race window between Processor::current() and a context switch happening before in_irq().
2021-08-22 10:21:31 +00:00
VERIFY(!Processor::current_in_irq());
Kernel: Don't check that interrupts are enabled during early boot The interrupts enabled check in the Kernel mutex is there so that we don't lock mutexes within a spinlock, because mutexes reenable interrupts and that will mess up the spinlock in more ways than one if the thread moves processors. This check is guarded behind a debug flag because it's too hard to fix all the problems at once, but we regressed and weren't even getting to init stage 2 with it enabled. With this commit, we get to stage 2 again. In early boot, there are no interrupts enabled and spinlocks used, so we can sort of kind of safely ignore the interrupt state. There might be a better solution with another boot state flag that checks whether APs are up (because they have interrupts enabled from the start) but that seems overkill.
2022-06-17 21:58:01 +00:00
if constexpr (LOCK_IN_CRITICAL_DEBUG) {
// There are no interrupts enabled in early boot.
if (!g_in_early_boot)
VERIFY_INTERRUPTS_ENABLED();
}
Everywhere: Rename ASSERT => VERIFY (...and ASSERT_NOT_REACHED => VERIFY_NOT_REACHED) Since all of these checks are done in release builds as well, let's rename them to VERIFY to prevent confusion, as everyone is used to assertions being compiled out in release. We can introduce a new ASSERT macro that is specifically for debug checks, but I'm doing this wholesale conversion first since we've accumulated thousands of these already, and it's not immediately obvious which ones are suitable for ASSERT.
2021-02-23 19:42:32 +00:00
VERIFY(mode != Mode::Unlocked);
Kernel: Add implied auto-specifiers in Locking As per clang-tidy.
2021-12-15 13:56:39 +00:00
auto* current_thread = Thread::current();
Kernel: Fix race condition in Lock::lock that may leave corrupted state
2021-01-23 17:43:52 +00:00
Kernel: Rename ScopedSpinlock => SpinlockLocker This matches MutexLocker, and doesn't sound like it's a lock itself.
2021-08-21 23:49:22 +00:00
SpinlockLocker lock(m_lock);
Kernel: Allow Lock to block from BlockCondition This enables the Lock class to block a thread even while the thread is working on a BlockCondition. A thread can still only be either blocked by a Lock or a BlockCondition. This also establishes a linked list of threads that are blocked by a Lock and unblocking directly unlocks threads and wakes them directly.
2021-07-10 16:23:16 +00:00
bool did_block = false;
Mode current_mode = m_mode;
switch (current_mode) {
case Mode::Unlocked: {
Kernel: Rename Lock to Mutex Let's be explicit about what kind of lock this is meant to be.
2021-07-17 19:09:51 +00:00
dbgln_if(LOCK_TRACE_DEBUG, "Mutex::lock @ ({}) {}: acquire {}, currently unlocked", this, m_name, mode_to_string(mode));
Kernel: Allow Lock to block from BlockCondition This enables the Lock class to block a thread even while the thread is working on a BlockCondition. A thread can still only be either blocked by a Lock or a BlockCondition. This also establishes a linked list of threads that are blocked by a Lock and unblocking directly unlocks threads and wakes them directly.
2021-07-10 16:23:16 +00:00
m_mode = mode;
VERIFY(!m_holder);
Kernel: Stop using HashMap in Mutex This commit removes the usage of HashMap in Mutex, thereby making Mutex be allocation-free. In order to achieve this several simplifications were made to Mutex, removing unused code-paths and extra VERIFYs: * We no longer support 'upgrading' a shared lock holder to an exclusive holder when it is the only shared holder and it did not unlock the lock before relocking it as exclusive. NOTE: Unlike the rest of these changes, this scenario is not VERIFY-able in an allocation-free way, as a result the new LOCK_SHARED_UPGRADE_DEBUG debug flag was added, this flag lets Mutex allocate in order to detect such cases when debugging a deadlock. * We no longer support checking if a Mutex is locked by the current thread when the Mutex was not locked exclusively, the shared version of this check was not used anywhere. * We no longer support force unlocking/relocking a Mutex if the Mutex was not locked exclusively, the shared version of these functions was not used anywhere.
2022-01-28 23:47:18 +00:00
VERIFY(m_shared_holders == 0);
Kernel: Allow Lock to block from BlockCondition This enables the Lock class to block a thread even while the thread is working on a BlockCondition. A thread can still only be either blocked by a Lock or a BlockCondition. This also establishes a linked list of threads that are blocked by a Lock and unblocking directly unlocks threads and wakes them directly.
2021-07-10 16:23:16 +00:00
if (mode == Mode::Exclusive) {
m_holder = current_thread;
} else {
VERIFY(mode == Mode::Shared);
Kernel: Stop using HashMap in Mutex This commit removes the usage of HashMap in Mutex, thereby making Mutex be allocation-free. In order to achieve this several simplifications were made to Mutex, removing unused code-paths and extra VERIFYs: * We no longer support 'upgrading' a shared lock holder to an exclusive holder when it is the only shared holder and it did not unlock the lock before relocking it as exclusive. NOTE: Unlike the rest of these changes, this scenario is not VERIFY-able in an allocation-free way, as a result the new LOCK_SHARED_UPGRADE_DEBUG debug flag was added, this flag lets Mutex allocate in order to detect such cases when debugging a deadlock. * We no longer support checking if a Mutex is locked by the current thread when the Mutex was not locked exclusively, the shared version of this check was not used anywhere. * We no longer support force unlocking/relocking a Mutex if the Mutex was not locked exclusively, the shared version of these functions was not used anywhere.
2022-01-28 23:47:18 +00:00
++m_shared_holders;
#if LOCK_SHARED_UPGRADE_DEBUG
m_shared_holders_map.set(current_thread, 1);
#endif
Kernel: Allow Lock to block from BlockCondition This enables the Lock class to block a thread even while the thread is working on a BlockCondition. A thread can still only be either blocked by a Lock or a BlockCondition. This also establishes a linked list of threads that are blocked by a Lock and unblocking directly unlocks threads and wakes them directly.
2021-07-10 16:23:16 +00:00
}
VERIFY(m_times_locked == 0);
m_times_locked++;
Kernel: Utilize AK::SourceLocation for LOCK_DEBUG instrumentation. The previous `LOCKER(..)` instrumentation only covered some of the cases where a lock is actually acquired. By utilizing the new `AK::SourceLocation` functionality we can now reliably instrument all calls to lock automatically. Other changes: - Tweak the message in `Thread::finalize()` which dumps leaked lock so it's more readable and includes the function information that is now available. - Make the `LOCKER(..)` define a no-op, it will be cleaned up in a follow up change.
2021-04-24 22:17:02 +00:00
Everywhere: Use CMake to generate AK/Debug.h. This was done with the help of several scripts, I dump them here to easily find them later: awk '/#ifdef/ { print "#cmakedefine01 "$2 }' AK/Debug.h.in for debug_macro in $(awk '/#ifdef/ { print $2 }' AK/Debug.h.in) do find . \( -name '*.cpp' -o -name '*.h' -o -name '*.in' \) -not -path './Toolchain/*' -not -path './Build/*' -exec sed -i -E 's/#ifdef '$debug_macro'/#if '$debug_macro'/' {} \; done # Remember to remove WRAPPER_GERNERATOR_DEBUG from the list. awk '/#cmake/ { print "set("$2" ON)" }' AK/Debug.h.in
2021-01-23 22:29:11 +00:00
#if LOCK_DEBUG
Kernel: Allow Lock to block from BlockCondition This enables the Lock class to block a thread even while the thread is working on a BlockCondition. A thread can still only be either blocked by a Lock or a BlockCondition. This also establishes a linked list of threads that are blocked by a Lock and unblocking directly unlocks threads and wakes them directly.
2021-07-10 16:23:16 +00:00
if (current_thread) {
current_thread->holding_lock(*this, 1, location);
}
Kernel: Fix Lock race causing infinite spinning between two threads We need to account for how many shared lock instances the current thread owns, so that we can properly release such references when yielding execution. We also need to release the process lock when donating.
2020-12-14 23:36:22 +00:00
#endif
Kernel: Allow Lock to block from BlockCondition This enables the Lock class to block a thread even while the thread is working on a BlockCondition. A thread can still only be either blocked by a Lock or a BlockCondition. This also establishes a linked list of threads that are blocked by a Lock and unblocking directly unlocks threads and wakes them directly.
2021-07-10 16:23:16 +00:00
return;
}
case Mode::Exclusive: {
VERIFY(m_holder);
if (m_holder != current_thread) {
block(*current_thread, mode, lock, 1);
did_block = true;
Kernel: Fix some Lock problems and VERIFY statements When a Lock blocks (e.g. due to a mode mismatch or because someone else holds it) the lock mode will be updated to what was requested. There were also some cases where restoring locks may have not worked as intended as it may have been held already by the same thread. Fixes #8787
2021-07-15 22:10:52 +00:00
// If we blocked then m_mode should have been updated to what we requested
VERIFY(m_mode == mode);
Kernel: Fix race condition in Lock::lock that may leave corrupted state
2021-01-23 17:43:52 +00:00
}
Everywhere: Replace a bundle of dbg with dbgln. These changes are arbitrarily divided into multiple commits to make it easier to find potentially introduced bugs with git bisect.
2021-01-18 16:25:44 +00:00
Kernel: Fix some Lock problems and VERIFY statements When a Lock blocks (e.g. due to a mode mismatch or because someone else holds it) the lock mode will be updated to what was requested. There were also some cases where restoring locks may have not worked as intended as it may have been held already by the same thread. Fixes #8787
2021-07-15 22:10:52 +00:00
if (m_mode == Mode::Exclusive) {
VERIFY(m_holder == current_thread);
Kernel: Stop using HashMap in Mutex This commit removes the usage of HashMap in Mutex, thereby making Mutex be allocation-free. In order to achieve this several simplifications were made to Mutex, removing unused code-paths and extra VERIFYs: * We no longer support 'upgrading' a shared lock holder to an exclusive holder when it is the only shared holder and it did not unlock the lock before relocking it as exclusive. NOTE: Unlike the rest of these changes, this scenario is not VERIFY-able in an allocation-free way, as a result the new LOCK_SHARED_UPGRADE_DEBUG debug flag was added, this flag lets Mutex allocate in order to detect such cases when debugging a deadlock. * We no longer support checking if a Mutex is locked by the current thread when the Mutex was not locked exclusively, the shared version of this check was not used anywhere. * We no longer support force unlocking/relocking a Mutex if the Mutex was not locked exclusively, the shared version of these functions was not used anywhere.
2022-01-28 23:47:18 +00:00
VERIFY(m_shared_holders == 0);
Kernel: Fix some Lock problems and VERIFY statements When a Lock blocks (e.g. due to a mode mismatch or because someone else holds it) the lock mode will be updated to what was requested. There were also some cases where restoring locks may have not worked as intended as it may have been held already by the same thread. Fixes #8787
2021-07-15 22:10:52 +00:00
} else if (did_block && mode == Mode::Shared) {
// Only if we blocked trying to acquire a shared lock the lock would have been converted
VERIFY(!m_holder);
Kernel: Stop using HashMap in Mutex This commit removes the usage of HashMap in Mutex, thereby making Mutex be allocation-free. In order to achieve this several simplifications were made to Mutex, removing unused code-paths and extra VERIFYs: * We no longer support 'upgrading' a shared lock holder to an exclusive holder when it is the only shared holder and it did not unlock the lock before relocking it as exclusive. NOTE: Unlike the rest of these changes, this scenario is not VERIFY-able in an allocation-free way, as a result the new LOCK_SHARED_UPGRADE_DEBUG debug flag was added, this flag lets Mutex allocate in order to detect such cases when debugging a deadlock. * We no longer support checking if a Mutex is locked by the current thread when the Mutex was not locked exclusively, the shared version of this check was not used anywhere. * We no longer support force unlocking/relocking a Mutex if the Mutex was not locked exclusively, the shared version of these functions was not used anywhere.
2022-01-28 23:47:18 +00:00
VERIFY(m_shared_holders > 0);
Kernel: Fix some Lock problems and VERIFY statements When a Lock blocks (e.g. due to a mode mismatch or because someone else holds it) the lock mode will be updated to what was requested. There were also some cases where restoring locks may have not worked as intended as it may have been held already by the same thread. Fixes #8787
2021-07-15 22:10:52 +00:00
}
Kernel: Allow Lock to block from BlockCondition This enables the Lock class to block a thread even while the thread is working on a BlockCondition. A thread can still only be either blocked by a Lock or a BlockCondition. This also establishes a linked list of threads that are blocked by a Lock and unblocking directly unlocks threads and wakes them directly.
2021-07-10 16:23:16 +00:00
if constexpr (LOCK_TRACE_DEBUG) {
if (mode == Mode::Exclusive)
Kernel: Rename Lock to Mutex Let's be explicit about what kind of lock this is meant to be.
2021-07-17 19:09:51 +00:00
dbgln("Mutex::lock @ {} ({}): acquire {}, currently exclusive, holding: {}", this, m_name, mode_to_string(mode), m_times_locked);
Kernel: Allow Lock to block from BlockCondition This enables the Lock class to block a thread even while the thread is working on a BlockCondition. A thread can still only be either blocked by a Lock or a BlockCondition. This also establishes a linked list of threads that are blocked by a Lock and unblocking directly unlocks threads and wakes them directly.
2021-07-10 16:23:16 +00:00
else
Kernel: Rename Lock to Mutex Let's be explicit about what kind of lock this is meant to be.
2021-07-17 19:09:51 +00:00
dbgln("Mutex::lock @ {} ({}): acquire exclusive (requested {}), currently exclusive, holding: {}", this, m_name, mode_to_string(mode), m_times_locked);
Kernel: Allow Lock to block from BlockCondition This enables the Lock class to block a thread even while the thread is working on a BlockCondition. A thread can still only be either blocked by a Lock or a BlockCondition. This also establishes a linked list of threads that are blocked by a Lock and unblocking directly unlocks threads and wakes them directly.
2021-07-10 16:23:16 +00:00
}
VERIFY(m_times_locked > 0);
Kernel: Fix some Lock problems and VERIFY statements When a Lock blocks (e.g. due to a mode mismatch or because someone else holds it) the lock mode will be updated to what was requested. There were also some cases where restoring locks may have not worked as intended as it may have been held already by the same thread. Fixes #8787
2021-07-15 22:10:52 +00:00
if (!did_block) {
// if we didn't block we must still be an exclusive lock
VERIFY(m_mode == Mode::Exclusive);
Kernel: Fix race condition in Lock::lock that may leave corrupted state
2021-01-23 17:43:52 +00:00
m_times_locked++;
Kernel: Fix some Lock problems and VERIFY statements When a Lock blocks (e.g. due to a mode mismatch or because someone else holds it) the lock mode will be updated to what was requested. There were also some cases where restoring locks may have not worked as intended as it may have been held already by the same thread. Fixes #8787
2021-07-15 22:10:52 +00:00
}
Kernel: Utilize AK::SourceLocation for LOCK_DEBUG instrumentation. The previous `LOCKER(..)` instrumentation only covered some of the cases where a lock is actually acquired. By utilizing the new `AK::SourceLocation` functionality we can now reliably instrument all calls to lock automatically. Other changes: - Tweak the message in `Thread::finalize()` which dumps leaked lock so it's more readable and includes the function information that is now available. - Make the `LOCKER(..)` define a no-op, it will be cleaned up in a follow up change.
2021-04-24 22:17:02 +00:00
Everywhere: Use CMake to generate AK/Debug.h. This was done with the help of several scripts, I dump them here to easily find them later: awk '/#ifdef/ { print "#cmakedefine01 "$2 }' AK/Debug.h.in for debug_macro in $(awk '/#ifdef/ { print $2 }' AK/Debug.h.in) do find . \( -name '*.cpp' -o -name '*.h' -o -name '*.in' \) -not -path './Toolchain/*' -not -path './Build/*' -exec sed -i -E 's/#ifdef '$debug_macro'/#if '$debug_macro'/' {} \; done # Remember to remove WRAPPER_GERNERATOR_DEBUG from the list. awk '/#cmake/ { print "set("$2" ON)" }' AK/Debug.h.in
2021-01-23 22:29:11 +00:00
#if LOCK_DEBUG
Kernel: Allow Lock to block from BlockCondition This enables the Lock class to block a thread even while the thread is working on a BlockCondition. A thread can still only be either blocked by a Lock or a BlockCondition. This also establishes a linked list of threads that are blocked by a Lock and unblocking directly unlocks threads and wakes them directly.
2021-07-10 16:23:16 +00:00
current_thread->holding_lock(*this, 1, location);
Kernel: Fix Lock race causing infinite spinning between two threads We need to account for how many shared lock instances the current thread owns, so that we can properly release such references when yielding execution. We also need to release the process lock when donating.
2020-12-14 23:36:22 +00:00
#endif
Kernel: Allow Lock to block from BlockCondition This enables the Lock class to block a thread even while the thread is working on a BlockCondition. A thread can still only be either blocked by a Lock or a BlockCondition. This also establishes a linked list of threads that are blocked by a Lock and unblocking directly unlocks threads and wakes them directly.
2021-07-10 16:23:16 +00:00
return;
}
case Mode::Shared: {
Kernel: Track big lock blocked threads in separate list When we lock a mutex, eventually `Thread::block` is invoked which could in turn invoke `Process::big_lock().restore_exclusive_lock()`. This would then try to add the current thread to a different blocked thread list then the one in use for the original mutex being locked, and because it's an intrusive list, the thread is removed from its original list during the `.append()`. When the original mutex eventually unblocks, we no longer have the thread in the intrusive blocked threads list and we panic. Solve this by making the big lock mutex special and giving it its own blocked thread list. Because the process big lock is temporary and is being actively removed from e.g. syscalls, it's a matter of time before we can also remove the fix introduced by this commit. Fixes issue #9401.
2022-04-05 23:49:30 +00:00
VERIFY(m_behavior == MutexBehavior::Regular);
Kernel: Allow Lock to block from BlockCondition This enables the Lock class to block a thread even while the thread is working on a BlockCondition. A thread can still only be either blocked by a Lock or a BlockCondition. This also establishes a linked list of threads that are blocked by a Lock and unblocking directly unlocks threads and wakes them directly.
2021-07-10 16:23:16 +00:00
VERIFY(!m_holder);
Kernel: Fix some Lock problems and VERIFY statements When a Lock blocks (e.g. due to a mode mismatch or because someone else holds it) the lock mode will be updated to what was requested. There were also some cases where restoring locks may have not worked as intended as it may have been held already by the same thread. Fixes #8787
2021-07-15 22:10:52 +00:00
if (mode == Mode::Exclusive) {
Kernel: Stop using HashMap in Mutex This commit removes the usage of HashMap in Mutex, thereby making Mutex be allocation-free. In order to achieve this several simplifications were made to Mutex, removing unused code-paths and extra VERIFYs: * We no longer support 'upgrading' a shared lock holder to an exclusive holder when it is the only shared holder and it did not unlock the lock before relocking it as exclusive. NOTE: Unlike the rest of these changes, this scenario is not VERIFY-able in an allocation-free way, as a result the new LOCK_SHARED_UPGRADE_DEBUG debug flag was added, this flag lets Mutex allocate in order to detect such cases when debugging a deadlock. * We no longer support checking if a Mutex is locked by the current thread when the Mutex was not locked exclusively, the shared version of this check was not used anywhere. * We no longer support force unlocking/relocking a Mutex if the Mutex was not locked exclusively, the shared version of these functions was not used anywhere.
2022-01-28 23:47:18 +00:00
dbgln_if(LOCK_TRACE_DEBUG, "Mutex::lock @ {} ({}): blocking for exclusive access, currently shared, locks held {}", this, m_name, m_times_locked);
#if LOCK_SHARED_UPGRADE_DEBUG
VERIFY(m_shared_holders_map.size() != 1 || m_shared_holders_map.begin()->key != current_thread);
#endif
// WARNING: The following block will deadlock if the current thread is the only shared locker of this Mutex
// and is asking to upgrade the lock to be exclusive without first releasing the shared lock. We have no
// allocation-free way to detect such a scenario, so if you suspect that this is the cause of your deadlock,
// try turning on LOCK_SHARED_UPGRADE_DEBUG.
Kernel: Allow Lock to block from BlockCondition This enables the Lock class to block a thread even while the thread is working on a BlockCondition. A thread can still only be either blocked by a Lock or a BlockCondition. This also establishes a linked list of threads that are blocked by a Lock and unblocking directly unlocks threads and wakes them directly.
2021-07-10 16:23:16 +00:00
block(*current_thread, mode, lock, 1);
did_block = true;
Kernel: Fix some Lock problems and VERIFY statements When a Lock blocks (e.g. due to a mode mismatch or because someone else holds it) the lock mode will be updated to what was requested. There were also some cases where restoring locks may have not worked as intended as it may have been held already by the same thread. Fixes #8787
2021-07-15 22:10:52 +00:00
VERIFY(m_mode == mode);
Kernel: Fix race condition in Lock::lock that may leave corrupted state
2021-01-23 17:43:52 +00:00
}
Everywhere: Replace a bundle of dbg with dbgln. These changes are arbitrarily divided into multiple commits to make it easier to find potentially introduced bugs with git bisect.
2021-01-18 16:25:44 +00:00
Kernel: Rename Lock to Mutex Let's be explicit about what kind of lock this is meant to be.
2021-07-17 19:09:51 +00:00
dbgln_if(LOCK_TRACE_DEBUG, "Mutex::lock @ {} ({}): acquire {}, currently shared, locks held {}", this, m_name, mode_to_string(mode), m_times_locked);
Everywhere: Replace a bundle of dbg with dbgln. These changes are arbitrarily divided into multiple commits to make it easier to find potentially introduced bugs with git bisect.
2021-01-18 16:25:44 +00:00
Kernel: Allow Lock to block from BlockCondition This enables the Lock class to block a thread even while the thread is working on a BlockCondition. A thread can still only be either blocked by a Lock or a BlockCondition. This also establishes a linked list of threads that are blocked by a Lock and unblocking directly unlocks threads and wakes them directly.
2021-07-10 16:23:16 +00:00
VERIFY(m_times_locked > 0);
Kernel: Fix some Lock problems and VERIFY statements When a Lock blocks (e.g. due to a mode mismatch or because someone else holds it) the lock mode will be updated to what was requested. There were also some cases where restoring locks may have not worked as intended as it may have been held already by the same thread. Fixes #8787
2021-07-15 22:10:52 +00:00
if (m_mode == Mode::Shared) {
VERIFY(!m_holder);
Kernel: Stop using HashMap in Mutex This commit removes the usage of HashMap in Mutex, thereby making Mutex be allocation-free. In order to achieve this several simplifications were made to Mutex, removing unused code-paths and extra VERIFYs: * We no longer support 'upgrading' a shared lock holder to an exclusive holder when it is the only shared holder and it did not unlock the lock before relocking it as exclusive. NOTE: Unlike the rest of these changes, this scenario is not VERIFY-able in an allocation-free way, as a result the new LOCK_SHARED_UPGRADE_DEBUG debug flag was added, this flag lets Mutex allocate in order to detect such cases when debugging a deadlock. * We no longer support checking if a Mutex is locked by the current thread when the Mutex was not locked exclusively, the shared version of this check was not used anywhere. * We no longer support force unlocking/relocking a Mutex if the Mutex was not locked exclusively, the shared version of these functions was not used anywhere.
2022-01-28 23:47:18 +00:00
VERIFY(!did_block);
Kernel: Fix some Lock problems and VERIFY statements When a Lock blocks (e.g. due to a mode mismatch or because someone else holds it) the lock mode will be updated to what was requested. There were also some cases where restoring locks may have not worked as intended as it may have been held already by the same thread. Fixes #8787
2021-07-15 22:10:52 +00:00
} else if (did_block) {
VERIFY(mode == Mode::Exclusive);
VERIFY(m_holder == current_thread);
Kernel: Stop using HashMap in Mutex This commit removes the usage of HashMap in Mutex, thereby making Mutex be allocation-free. In order to achieve this several simplifications were made to Mutex, removing unused code-paths and extra VERIFYs: * We no longer support 'upgrading' a shared lock holder to an exclusive holder when it is the only shared holder and it did not unlock the lock before relocking it as exclusive. NOTE: Unlike the rest of these changes, this scenario is not VERIFY-able in an allocation-free way, as a result the new LOCK_SHARED_UPGRADE_DEBUG debug flag was added, this flag lets Mutex allocate in order to detect such cases when debugging a deadlock. * We no longer support checking if a Mutex is locked by the current thread when the Mutex was not locked exclusively, the shared version of this check was not used anywhere. * We no longer support force unlocking/relocking a Mutex if the Mutex was not locked exclusively, the shared version of these functions was not used anywhere.
2022-01-28 23:47:18 +00:00
VERIFY(m_shared_holders == 0);
Kernel: Fix some Lock problems and VERIFY statements When a Lock blocks (e.g. due to a mode mismatch or because someone else holds it) the lock mode will be updated to what was requested. There were also some cases where restoring locks may have not worked as intended as it may have been held already by the same thread. Fixes #8787
2021-07-15 22:10:52 +00:00
}
if (!did_block) {
// if we didn't block we must still be a shared lock
VERIFY(m_mode == Mode::Shared);
Kernel: Fix race condition in Lock::lock that may leave corrupted state
2021-01-23 17:43:52 +00:00
m_times_locked++;
Kernel: Stop using HashMap in Mutex This commit removes the usage of HashMap in Mutex, thereby making Mutex be allocation-free. In order to achieve this several simplifications were made to Mutex, removing unused code-paths and extra VERIFYs: * We no longer support 'upgrading' a shared lock holder to an exclusive holder when it is the only shared holder and it did not unlock the lock before relocking it as exclusive. NOTE: Unlike the rest of these changes, this scenario is not VERIFY-able in an allocation-free way, as a result the new LOCK_SHARED_UPGRADE_DEBUG debug flag was added, this flag lets Mutex allocate in order to detect such cases when debugging a deadlock. * We no longer support checking if a Mutex is locked by the current thread when the Mutex was not locked exclusively, the shared version of this check was not used anywhere. * We no longer support force unlocking/relocking a Mutex if the Mutex was not locked exclusively, the shared version of these functions was not used anywhere.
2022-01-28 23:47:18 +00:00
VERIFY(m_shared_holders > 0);
++m_shared_holders;
#if LOCK_SHARED_UPGRADE_DEBUG
auto it = m_shared_holders_map.find(current_thread);
if (it != m_shared_holders_map.end())
Kernel: Fix race condition in Lock::lock that may leave corrupted state
2021-01-23 17:43:52 +00:00
it->value++;
else
Kernel: Stop using HashMap in Mutex This commit removes the usage of HashMap in Mutex, thereby making Mutex be allocation-free. In order to achieve this several simplifications were made to Mutex, removing unused code-paths and extra VERIFYs: * We no longer support 'upgrading' a shared lock holder to an exclusive holder when it is the only shared holder and it did not unlock the lock before relocking it as exclusive. NOTE: Unlike the rest of these changes, this scenario is not VERIFY-able in an allocation-free way, as a result the new LOCK_SHARED_UPGRADE_DEBUG debug flag was added, this flag lets Mutex allocate in order to detect such cases when debugging a deadlock. * We no longer support checking if a Mutex is locked by the current thread when the Mutex was not locked exclusively, the shared version of this check was not used anywhere. * We no longer support force unlocking/relocking a Mutex if the Mutex was not locked exclusively, the shared version of these functions was not used anywhere.
2022-01-28 23:47:18 +00:00
m_shared_holders_map.set(current_thread, 1);
#endif
Kernel: Allow Lock to block from BlockCondition This enables the Lock class to block a thread even while the thread is working on a BlockCondition. A thread can still only be either blocked by a Lock or a BlockCondition. This also establishes a linked list of threads that are blocked by a Lock and unblocking directly unlocks threads and wakes them directly.
2021-07-10 16:23:16 +00:00
}
Kernel: Utilize AK::SourceLocation for LOCK_DEBUG instrumentation. The previous `LOCKER(..)` instrumentation only covered some of the cases where a lock is actually acquired. By utilizing the new `AK::SourceLocation` functionality we can now reliably instrument all calls to lock automatically. Other changes: - Tweak the message in `Thread::finalize()` which dumps leaked lock so it's more readable and includes the function information that is now available. - Make the `LOCKER(..)` define a no-op, it will be cleaned up in a follow up change.
2021-04-24 22:17:02 +00:00
Everywhere: Use CMake to generate AK/Debug.h. This was done with the help of several scripts, I dump them here to easily find them later: awk '/#ifdef/ { print "#cmakedefine01 "$2 }' AK/Debug.h.in for debug_macro in $(awk '/#ifdef/ { print $2 }' AK/Debug.h.in) do find . \( -name '*.cpp' -o -name '*.h' -o -name '*.in' \) -not -path './Toolchain/*' -not -path './Build/*' -exec sed -i -E 's/#ifdef '$debug_macro'/#if '$debug_macro'/' {} \; done # Remember to remove WRAPPER_GERNERATOR_DEBUG from the list. awk '/#cmake/ { print "set("$2" ON)" }' AK/Debug.h.in
2021-01-23 22:29:11 +00:00
#if LOCK_DEBUG
Kernel: Allow Lock to block from BlockCondition This enables the Lock class to block a thread even while the thread is working on a BlockCondition. A thread can still only be either blocked by a Lock or a BlockCondition. This also establishes a linked list of threads that are blocked by a Lock and unblocking directly unlocks threads and wakes them directly.
2021-07-10 16:23:16 +00:00
current_thread->holding_lock(*this, 1, location);
Kernel: Fix Lock race causing infinite spinning between two threads We need to account for how many shared lock instances the current thread owns, so that we can properly release such references when yielding execution. We also need to release the process lock when donating.
2020-12-14 23:36:22 +00:00
#endif
Kernel: Allow Lock to block from BlockCondition This enables the Lock class to block a thread even while the thread is working on a BlockCondition. A thread can still only be either blocked by a Lock or a BlockCondition. This also establishes a linked list of threads that are blocked by a Lock and unblocking directly unlocks threads and wakes them directly.
2021-07-10 16:23:16 +00:00
return;
}
default:
VERIFY_NOT_REACHED();
Kernel: Oops, forgot to add Lock.cpp.
2019-07-29 10:00:14 +00:00
}
}
Kernel: Rename Lock to Mutex Let's be explicit about what kind of lock this is meant to be.
2021-07-17 19:09:51 +00:00
void Mutex::unlock()
Kernel: Oops, forgot to add Lock.cpp.
2019-07-29 10:00:14 +00:00
{
Kernel: Fix some problems with Thread::wait_on and Lock This changes the Thread::wait_on function to not enable interrupts upon leaving, which caused some problems with page fault handlers and in other situations. It may now be called from critical sections, with interrupts enabled or disabled, and returns to the same state. This also requires some fixes to Lock. To aid debugging, a new define LOCK_DEBUG is added that enables checking for Lock leaks upon finalization of a Thread.
2020-12-01 02:04:36 +00:00
// NOTE: This may be called from an interrupt handler (not an IRQ handler)
// and also from within critical sections!
Kernel: Convert Processor::in_irq() to static current_in_irq() This closes the race window between Processor::current() and a context switch happening before in_irq().
2021-08-22 10:21:31 +00:00
VERIFY(!Processor::current_in_irq());
Kernel: Don't check that interrupts are enabled during early boot The interrupts enabled check in the Kernel mutex is there so that we don't lock mutexes within a spinlock, because mutexes reenable interrupts and that will mess up the spinlock in more ways than one if the thread moves processors. This check is guarded behind a debug flag because it's too hard to fix all the problems at once, but we regressed and weren't even getting to init stage 2 with it enabled. With this commit, we get to stage 2 again. In early boot, there are no interrupts enabled and spinlocks used, so we can sort of kind of safely ignore the interrupt state. There might be a better solution with another boot state flag that checks whether APs are up (because they have interrupts enabled from the start) but that seems overkill.
2022-06-17 21:58:01 +00:00
if constexpr (LOCK_IN_CRITICAL_DEBUG) {
// There are no interrupts enabled in early boot.
if (!g_in_early_boot)
VERIFY_INTERRUPTS_ENABLED();
}
Kernel: Add implied auto-specifiers in Locking As per clang-tidy.
2021-12-15 13:56:39 +00:00
auto* current_thread = Thread::current();
Kernel: Rename ScopedSpinlock => SpinlockLocker This matches MutexLocker, and doesn't sound like it's a lock itself.
2021-08-21 23:49:22 +00:00
SpinlockLocker lock(m_lock);
Kernel: Allow Lock to block from BlockCondition This enables the Lock class to block a thread even while the thread is working on a BlockCondition. A thread can still only be either blocked by a Lock or a BlockCondition. This also establishes a linked list of threads that are blocked by a Lock and unblocking directly unlocks threads and wakes them directly.
2021-07-10 16:23:16 +00:00
Mode current_mode = m_mode;
if constexpr (LOCK_TRACE_DEBUG) {
if (current_mode == Mode::Shared)
Kernel: Rename Lock to Mutex Let's be explicit about what kind of lock this is meant to be.
2021-07-17 19:09:51 +00:00
dbgln("Mutex::unlock @ {} ({}): release {}, locks held: {}", this, m_name, mode_to_string(current_mode), m_times_locked);
Kernel: Allow Lock to block from BlockCondition This enables the Lock class to block a thread even while the thread is working on a BlockCondition. A thread can still only be either blocked by a Lock or a BlockCondition. This also establishes a linked list of threads that are blocked by a Lock and unblocking directly unlocks threads and wakes them directly.
2021-07-10 16:23:16 +00:00
else
Kernel: Rename Lock to Mutex Let's be explicit about what kind of lock this is meant to be.
2021-07-17 19:09:51 +00:00
dbgln("Mutex::unlock @ {} ({}): release {}, holding: {}", this, m_name, mode_to_string(current_mode), m_times_locked);
Kernel: Allow Lock to block from BlockCondition This enables the Lock class to block a thread even while the thread is working on a BlockCondition. A thread can still only be either blocked by a Lock or a BlockCondition. This also establishes a linked list of threads that are blocked by a Lock and unblocking directly unlocks threads and wakes them directly.
2021-07-10 16:23:16 +00:00
}
Everywhere: Replace a bundle of dbg with dbgln. These changes are arbitrarily divided into multiple commits to make it easier to find potentially introduced bugs with git bisect.
2021-01-18 16:25:44 +00:00
Kernel: Allow Lock to block from BlockCondition This enables the Lock class to block a thread even while the thread is working on a BlockCondition. A thread can still only be either blocked by a Lock or a BlockCondition. This also establishes a linked list of threads that are blocked by a Lock and unblocking directly unlocks threads and wakes them directly.
2021-07-10 16:23:16 +00:00
VERIFY(current_mode != Mode::Unlocked);
Kernel: Introduce shared locking mode A Lock can now be held either in shared or exclusive mode. Multiple threads can hold the same lock in shared mode at one time, but if any thread holds the lock in exclusive mode, no other thread can hold it at the same time in either mode.
2020-04-18 09:21:00 +00:00
Kernel: Allow Lock to block from BlockCondition This enables the Lock class to block a thread even while the thread is working on a BlockCondition. A thread can still only be either blocked by a Lock or a BlockCondition. This also establishes a linked list of threads that are blocked by a Lock and unblocking directly unlocks threads and wakes them directly.
2021-07-10 16:23:16 +00:00
VERIFY(m_times_locked > 0);
m_times_locked--;
Kernel: Fix Lock race causing infinite spinning between two threads We need to account for how many shared lock instances the current thread owns, so that we can properly release such references when yielding execution. We also need to release the process lock when donating.
2020-12-14 23:36:22 +00:00
Kernel: Allow Lock to block from BlockCondition This enables the Lock class to block a thread even while the thread is working on a BlockCondition. A thread can still only be either blocked by a Lock or a BlockCondition. This also establishes a linked list of threads that are blocked by a Lock and unblocking directly unlocks threads and wakes them directly.
2021-07-10 16:23:16 +00:00
switch (current_mode) {
case Mode::Exclusive:
VERIFY(m_holder == current_thread);
Kernel: Stop using HashMap in Mutex This commit removes the usage of HashMap in Mutex, thereby making Mutex be allocation-free. In order to achieve this several simplifications were made to Mutex, removing unused code-paths and extra VERIFYs: * We no longer support 'upgrading' a shared lock holder to an exclusive holder when it is the only shared holder and it did not unlock the lock before relocking it as exclusive. NOTE: Unlike the rest of these changes, this scenario is not VERIFY-able in an allocation-free way, as a result the new LOCK_SHARED_UPGRADE_DEBUG debug flag was added, this flag lets Mutex allocate in order to detect such cases when debugging a deadlock. * We no longer support checking if a Mutex is locked by the current thread when the Mutex was not locked exclusively, the shared version of this check was not used anywhere. * We no longer support force unlocking/relocking a Mutex if the Mutex was not locked exclusively, the shared version of these functions was not used anywhere.
2022-01-28 23:47:18 +00:00
VERIFY(m_shared_holders == 0);
Kernel: Allow Lock to block from BlockCondition This enables the Lock class to block a thread even while the thread is working on a BlockCondition. A thread can still only be either blocked by a Lock or a BlockCondition. This also establishes a linked list of threads that are blocked by a Lock and unblocking directly unlocks threads and wakes them directly.
2021-07-10 16:23:16 +00:00
if (m_times_locked == 0)
m_holder = nullptr;
break;
case Mode::Shared: {
VERIFY(!m_holder);
Kernel: Stop using HashMap in Mutex This commit removes the usage of HashMap in Mutex, thereby making Mutex be allocation-free. In order to achieve this several simplifications were made to Mutex, removing unused code-paths and extra VERIFYs: * We no longer support 'upgrading' a shared lock holder to an exclusive holder when it is the only shared holder and it did not unlock the lock before relocking it as exclusive. NOTE: Unlike the rest of these changes, this scenario is not VERIFY-able in an allocation-free way, as a result the new LOCK_SHARED_UPGRADE_DEBUG debug flag was added, this flag lets Mutex allocate in order to detect such cases when debugging a deadlock. * We no longer support checking if a Mutex is locked by the current thread when the Mutex was not locked exclusively, the shared version of this check was not used anywhere. * We no longer support force unlocking/relocking a Mutex if the Mutex was not locked exclusively, the shared version of these functions was not used anywhere.
2022-01-28 23:47:18 +00:00
VERIFY(m_shared_holders > 0);
--m_shared_holders;
#if LOCK_SHARED_UPGRADE_DEBUG
auto it = m_shared_holders_map.find(current_thread);
if (it->value > 1)
Kernel: Allow Lock to block from BlockCondition This enables the Lock class to block a thread even while the thread is working on a BlockCondition. A thread can still only be either blocked by a Lock or a BlockCondition. This also establishes a linked list of threads that are blocked by a Lock and unblocking directly unlocks threads and wakes them directly.
2021-07-10 16:23:16 +00:00
it->value--;
Kernel: Stop using HashMap in Mutex This commit removes the usage of HashMap in Mutex, thereby making Mutex be allocation-free. In order to achieve this several simplifications were made to Mutex, removing unused code-paths and extra VERIFYs: * We no longer support 'upgrading' a shared lock holder to an exclusive holder when it is the only shared holder and it did not unlock the lock before relocking it as exclusive. NOTE: Unlike the rest of these changes, this scenario is not VERIFY-able in an allocation-free way, as a result the new LOCK_SHARED_UPGRADE_DEBUG debug flag was added, this flag lets Mutex allocate in order to detect such cases when debugging a deadlock. * We no longer support checking if a Mutex is locked by the current thread when the Mutex was not locked exclusively, the shared version of this check was not used anywhere. * We no longer support force unlocking/relocking a Mutex if the Mutex was not locked exclusively, the shared version of these functions was not used anywhere.
2022-01-28 23:47:18 +00:00
else
m_shared_holders_map.remove(it);
#endif
Kernel: Allow Lock to block from BlockCondition This enables the Lock class to block a thread even while the thread is working on a BlockCondition. A thread can still only be either blocked by a Lock or a BlockCondition. This also establishes a linked list of threads that are blocked by a Lock and unblocking directly unlocks threads and wakes them directly.
2021-07-10 16:23:16 +00:00
break;
}
default:
VERIFY_NOT_REACHED();
}
Kernel: Fix Lock race causing infinite spinning between two threads We need to account for how many shared lock instances the current thread owns, so that we can properly release such references when yielding execution. We also need to release the process lock when donating.
2020-12-14 23:36:22 +00:00
Everywhere: Use CMake to generate AK/Debug.h. This was done with the help of several scripts, I dump them here to easily find them later: awk '/#ifdef/ { print "#cmakedefine01 "$2 }' AK/Debug.h.in for debug_macro in $(awk '/#ifdef/ { print $2 }' AK/Debug.h.in) do find . \( -name '*.cpp' -o -name '*.h' -o -name '*.in' \) -not -path './Toolchain/*' -not -path './Build/*' -exec sed -i -E 's/#ifdef '$debug_macro'/#if '$debug_macro'/' {} \; done # Remember to remove WRAPPER_GERNERATOR_DEBUG from the list. awk '/#cmake/ { print "set("$2" ON)" }' AK/Debug.h.in
2021-01-23 22:29:11 +00:00
#if LOCK_DEBUG
Kernel: Allow Lock to block from BlockCondition This enables the Lock class to block a thread even while the thread is working on a BlockCondition. A thread can still only be either blocked by a Lock or a BlockCondition. This also establishes a linked list of threads that are blocked by a Lock and unblocking directly unlocks threads and wakes them directly.
2021-07-10 16:23:16 +00:00
if (current_thread) {
current_thread->holding_lock(*this, -1, {});
}
Kernel: Fix some problems with Thread::wait_on and Lock This changes the Thread::wait_on function to not enable interrupts upon leaving, which caused some problems with page fault handlers and in other situations. It may now be called from critical sections, with interrupts enabled or disabled, and returns to the same state. This also requires some fixes to Lock. To aid debugging, a new define LOCK_DEBUG is added that enables checking for Lock leaks upon finalization of a Thread.
2020-12-01 02:04:36 +00:00
#endif
Kernel: Allow Lock to block from BlockCondition This enables the Lock class to block a thread even while the thread is working on a BlockCondition. A thread can still only be either blocked by a Lock or a BlockCondition. This also establishes a linked list of threads that are blocked by a Lock and unblocking directly unlocks threads and wakes them directly.
2021-07-10 16:23:16 +00:00
if (m_times_locked == 0) {
Kernel: Stop using HashMap in Mutex This commit removes the usage of HashMap in Mutex, thereby making Mutex be allocation-free. In order to achieve this several simplifications were made to Mutex, removing unused code-paths and extra VERIFYs: * We no longer support 'upgrading' a shared lock holder to an exclusive holder when it is the only shared holder and it did not unlock the lock before relocking it as exclusive. NOTE: Unlike the rest of these changes, this scenario is not VERIFY-able in an allocation-free way, as a result the new LOCK_SHARED_UPGRADE_DEBUG debug flag was added, this flag lets Mutex allocate in order to detect such cases when debugging a deadlock. * We no longer support checking if a Mutex is locked by the current thread when the Mutex was not locked exclusively, the shared version of this check was not used anywhere. * We no longer support force unlocking/relocking a Mutex if the Mutex was not locked exclusively, the shared version of these functions was not used anywhere.
2022-01-28 23:47:18 +00:00
VERIFY(current_mode == Mode::Exclusive ? !m_holder : m_shared_holders == 0);
Kernel: Allow Lock to block from BlockCondition This enables the Lock class to block a thread even while the thread is working on a BlockCondition. A thread can still only be either blocked by a Lock or a BlockCondition. This also establishes a linked list of threads that are blocked by a Lock and unblocking directly unlocks threads and wakes them directly.
2021-07-10 16:23:16 +00:00
m_mode = Mode::Unlocked;
unblock_waiters(current_mode);
}
}
Kernel: Make all Spinlocks use u8 for storage, remove template The default template argument is only used in one place, and it looks like it was probably just an oversight. The rest of the Kernel code all uses u8 as the type. So lets make that the default and remove the unused template argument, as there doesn't seem to be a reason to allow the size to be customizable.
2021-09-05 17:02:03 +00:00
void Mutex::block(Thread& current_thread, Mode mode, SpinlockLocker<Spinlock>& lock, u32 requested_locks)
Kernel: Allow Lock to block from BlockCondition This enables the Lock class to block a thread even while the thread is working on a BlockCondition. A thread can still only be either blocked by a Lock or a BlockCondition. This also establishes a linked list of threads that are blocked by a Lock and unblocking directly unlocks threads and wakes them directly.
2021-07-10 16:23:16 +00:00
{
Kernel: Don't check that interrupts are enabled during early boot The interrupts enabled check in the Kernel mutex is there so that we don't lock mutexes within a spinlock, because mutexes reenable interrupts and that will mess up the spinlock in more ways than one if the thread moves processors. This check is guarded behind a debug flag because it's too hard to fix all the problems at once, but we regressed and weren't even getting to init stage 2 with it enabled. With this commit, we get to stage 2 again. In early boot, there are no interrupts enabled and spinlocks used, so we can sort of kind of safely ignore the interrupt state. There might be a better solution with another boot state flag that checks whether APs are up (because they have interrupts enabled from the start) but that seems overkill.
2022-06-17 21:58:01 +00:00
if constexpr (LOCK_IN_CRITICAL_DEBUG) {
// There are no interrupts enabled in early boot.
if (!g_in_early_boot)
VERIFY_INTERRUPTS_ENABLED();
}
Kernel: Protect Mutex's thread lists with a spinlock
2022-04-05 12:44:50 +00:00
m_blocked_thread_lists.with([&](auto& lists) {
Kernel: Track big lock blocked threads in separate list When we lock a mutex, eventually `Thread::block` is invoked which could in turn invoke `Process::big_lock().restore_exclusive_lock()`. This would then try to add the current thread to a different blocked thread list then the one in use for the original mutex being locked, and because it's an intrusive list, the thread is removed from its original list during the `.append()`. When the original mutex eventually unblocks, we no longer have the thread in the intrusive blocked threads list and we panic. Solve this by making the big lock mutex special and giving it its own blocked thread list. Because the process big lock is temporary and is being actively removed from e.g. syscalls, it's a matter of time before we can also remove the fix introduced by this commit. Fixes issue #9401.
2022-04-05 23:49:30 +00:00
auto append_to_list = [&]<typename L>(L& list) {
VERIFY(!list.contains(current_thread));
list.append(current_thread);
};
if (m_behavior == MutexBehavior::BigLock)
append_to_list(lists.exclusive_big_lock);
else
append_to_list(lists.list_for_mode(mode));
Kernel: Protect Mutex's thread lists with a spinlock
2022-04-05 12:44:50 +00:00
});
Kernel: Allow Lock to block from BlockCondition This enables the Lock class to block a thread even while the thread is working on a BlockCondition. A thread can still only be either blocked by a Lock or a BlockCondition. This also establishes a linked list of threads that are blocked by a Lock and unblocking directly unlocks threads and wakes them directly.
2021-07-10 16:23:16 +00:00
Kernel: Rename Lock to Mutex Let's be explicit about what kind of lock this is meant to be.
2021-07-17 19:09:51 +00:00
dbgln_if(LOCK_TRACE_DEBUG, "Mutex::lock @ {} ({}) waiting...", this, m_name);
Kernel: Allow Lock to block from BlockCondition This enables the Lock class to block a thread even while the thread is working on a BlockCondition. A thread can still only be either blocked by a Lock or a BlockCondition. This also establishes a linked list of threads that are blocked by a Lock and unblocking directly unlocks threads and wakes them directly.
2021-07-10 16:23:16 +00:00
current_thread.block(*this, lock, requested_locks);
Kernel: Rename Lock to Mutex Let's be explicit about what kind of lock this is meant to be.
2021-07-17 19:09:51 +00:00
dbgln_if(LOCK_TRACE_DEBUG, "Mutex::lock @ {} ({}) waited", this, m_name);
Kernel: Allow Lock to block from BlockCondition This enables the Lock class to block a thread even while the thread is working on a BlockCondition. A thread can still only be either blocked by a Lock or a BlockCondition. This also establishes a linked list of threads that are blocked by a Lock and unblocking directly unlocks threads and wakes them directly.
2021-07-10 16:23:16 +00:00
Kernel: Protect Mutex's thread lists with a spinlock
2022-04-05 12:44:50 +00:00
m_blocked_thread_lists.with([&](auto& lists) {
Kernel: Track big lock blocked threads in separate list When we lock a mutex, eventually `Thread::block` is invoked which could in turn invoke `Process::big_lock().restore_exclusive_lock()`. This would then try to add the current thread to a different blocked thread list then the one in use for the original mutex being locked, and because it's an intrusive list, the thread is removed from its original list during the `.append()`. When the original mutex eventually unblocks, we no longer have the thread in the intrusive blocked threads list and we panic. Solve this by making the big lock mutex special and giving it its own blocked thread list. Because the process big lock is temporary and is being actively removed from e.g. syscalls, it's a matter of time before we can also remove the fix introduced by this commit. Fixes issue #9401.
2022-04-05 23:49:30 +00:00
auto remove_from_list = [&]<typename L>(L& list) {
VERIFY(list.contains(current_thread));
list.remove(current_thread);
};
if (m_behavior == MutexBehavior::BigLock)
remove_from_list(lists.exclusive_big_lock);
else
remove_from_list(lists.list_for_mode(mode));
Kernel: Protect Mutex's thread lists with a spinlock
2022-04-05 12:44:50 +00:00
});
Kernel: Allow Lock to block from BlockCondition This enables the Lock class to block a thread even while the thread is working on a BlockCondition. A thread can still only be either blocked by a Lock or a BlockCondition. This also establishes a linked list of threads that are blocked by a Lock and unblocking directly unlocks threads and wakes them directly.
2021-07-10 16:23:16 +00:00
}
Kernel: Rename Lock to Mutex Let's be explicit about what kind of lock this is meant to be.
2021-07-17 19:09:51 +00:00
void Mutex::unblock_waiters(Mode previous_mode)
Kernel: Allow Lock to block from BlockCondition This enables the Lock class to block a thread even while the thread is working on a BlockCondition. A thread can still only be either blocked by a Lock or a BlockCondition. This also establishes a linked list of threads that are blocked by a Lock and unblocking directly unlocks threads and wakes them directly.
2021-07-10 16:23:16 +00:00
{
VERIFY(m_times_locked == 0);
VERIFY(m_mode == Mode::Unlocked);
Kernel: Protect Mutex's thread lists with a spinlock
2022-04-05 12:44:50 +00:00
m_blocked_thread_lists.with([&](auto& lists) {
auto unblock_shared = [&]() {
if (lists.shared.is_empty())
return false;
Kernel: Track big lock blocked threads in separate list When we lock a mutex, eventually `Thread::block` is invoked which could in turn invoke `Process::big_lock().restore_exclusive_lock()`. This would then try to add the current thread to a different blocked thread list then the one in use for the original mutex being locked, and because it's an intrusive list, the thread is removed from its original list during the `.append()`. When the original mutex eventually unblocks, we no longer have the thread in the intrusive blocked threads list and we panic. Solve this by making the big lock mutex special and giving it its own blocked thread list. Because the process big lock is temporary and is being actively removed from e.g. syscalls, it's a matter of time before we can also remove the fix introduced by this commit. Fixes issue #9401.
2022-04-05 23:49:30 +00:00
VERIFY(m_behavior == MutexBehavior::Regular);
Kernel: Protect Mutex's thread lists with a spinlock
2022-04-05 12:44:50 +00:00
m_mode = Mode::Shared;
for (auto& thread : lists.shared) {
auto requested_locks = thread.unblock_from_mutex(*this);
m_shared_holders += requested_locks;
Kernel: Stop using HashMap in Mutex This commit removes the usage of HashMap in Mutex, thereby making Mutex be allocation-free. In order to achieve this several simplifications were made to Mutex, removing unused code-paths and extra VERIFYs: * We no longer support 'upgrading' a shared lock holder to an exclusive holder when it is the only shared holder and it did not unlock the lock before relocking it as exclusive. NOTE: Unlike the rest of these changes, this scenario is not VERIFY-able in an allocation-free way, as a result the new LOCK_SHARED_UPGRADE_DEBUG debug flag was added, this flag lets Mutex allocate in order to detect such cases when debugging a deadlock. * We no longer support checking if a Mutex is locked by the current thread when the Mutex was not locked exclusively, the shared version of this check was not used anywhere. * We no longer support force unlocking/relocking a Mutex if the Mutex was not locked exclusively, the shared version of these functions was not used anywhere.
2022-01-28 23:47:18 +00:00
#if LOCK_SHARED_UPGRADE_DEBUG
Kernel: Protect Mutex's thread lists with a spinlock
2022-04-05 12:44:50 +00:00
auto set_result = m_shared_holders_map.set(&thread, requested_locks);
VERIFY(set_result == AK::HashSetResult::InsertedNewEntry);
Kernel: Stop using HashMap in Mutex This commit removes the usage of HashMap in Mutex, thereby making Mutex be allocation-free. In order to achieve this several simplifications were made to Mutex, removing unused code-paths and extra VERIFYs: * We no longer support 'upgrading' a shared lock holder to an exclusive holder when it is the only shared holder and it did not unlock the lock before relocking it as exclusive. NOTE: Unlike the rest of these changes, this scenario is not VERIFY-able in an allocation-free way, as a result the new LOCK_SHARED_UPGRADE_DEBUG debug flag was added, this flag lets Mutex allocate in order to detect such cases when debugging a deadlock. * We no longer support checking if a Mutex is locked by the current thread when the Mutex was not locked exclusively, the shared version of this check was not used anywhere. * We no longer support force unlocking/relocking a Mutex if the Mutex was not locked exclusively, the shared version of these functions was not used anywhere.
2022-01-28 23:47:18 +00:00
#endif
Kernel: Protect Mutex's thread lists with a spinlock
2022-04-05 12:44:50 +00:00
m_times_locked += requested_locks;
}
Kernel: Allow Lock to block from BlockCondition This enables the Lock class to block a thread even while the thread is working on a BlockCondition. A thread can still only be either blocked by a Lock or a BlockCondition. This also establishes a linked list of threads that are blocked by a Lock and unblocking directly unlocks threads and wakes them directly.
2021-07-10 16:23:16 +00:00
return true;
Kernel: Protect Mutex's thread lists with a spinlock
2022-04-05 12:44:50 +00:00
};
Kernel: Track big lock blocked threads in separate list When we lock a mutex, eventually `Thread::block` is invoked which could in turn invoke `Process::big_lock().restore_exclusive_lock()`. This would then try to add the current thread to a different blocked thread list then the one in use for the original mutex being locked, and because it's an intrusive list, the thread is removed from its original list during the `.append()`. When the original mutex eventually unblocks, we no longer have the thread in the intrusive blocked threads list and we panic. Solve this by making the big lock mutex special and giving it its own blocked thread list. Because the process big lock is temporary and is being actively removed from e.g. syscalls, it's a matter of time before we can also remove the fix introduced by this commit. Fixes issue #9401.
2022-04-05 23:49:30 +00:00
auto unblock_exclusive = [&]<typename L>(L& list) {
if (auto* next_exclusive_thread = list.first()) {
Kernel: Protect Mutex's thread lists with a spinlock
2022-04-05 12:44:50 +00:00
m_mode = Mode::Exclusive;
m_times_locked = next_exclusive_thread->unblock_from_mutex(*this);
m_holder = next_exclusive_thread;
return true;
}
return false;
};
Kernel: Allow Lock to block from BlockCondition This enables the Lock class to block a thread even while the thread is working on a BlockCondition. A thread can still only be either blocked by a Lock or a BlockCondition. This also establishes a linked list of threads that are blocked by a Lock and unblocking directly unlocks threads and wakes them directly.
2021-07-10 16:23:16 +00:00
Kernel: Track big lock blocked threads in separate list When we lock a mutex, eventually `Thread::block` is invoked which could in turn invoke `Process::big_lock().restore_exclusive_lock()`. This would then try to add the current thread to a different blocked thread list then the one in use for the original mutex being locked, and because it's an intrusive list, the thread is removed from its original list during the `.append()`. When the original mutex eventually unblocks, we no longer have the thread in the intrusive blocked threads list and we panic. Solve this by making the big lock mutex special and giving it its own blocked thread list. Because the process big lock is temporary and is being actively removed from e.g. syscalls, it's a matter of time before we can also remove the fix introduced by this commit. Fixes issue #9401.
2022-04-05 23:49:30 +00:00
if (m_behavior == MutexBehavior::BigLock) {
unblock_exclusive(lists.exclusive_big_lock);
} else if (previous_mode == Mode::Exclusive) {
Kernel: Protect Mutex's thread lists with a spinlock
2022-04-05 12:44:50 +00:00
if (!unblock_shared())
Kernel: Track big lock blocked threads in separate list When we lock a mutex, eventually `Thread::block` is invoked which could in turn invoke `Process::big_lock().restore_exclusive_lock()`. This would then try to add the current thread to a different blocked thread list then the one in use for the original mutex being locked, and because it's an intrusive list, the thread is removed from its original list during the `.append()`. When the original mutex eventually unblocks, we no longer have the thread in the intrusive blocked threads list and we panic. Solve this by making the big lock mutex special and giving it its own blocked thread list. Because the process big lock is temporary and is being actively removed from e.g. syscalls, it's a matter of time before we can also remove the fix introduced by this commit. Fixes issue #9401.
2022-04-05 23:49:30 +00:00
unblock_exclusive(lists.exclusive);
Kernel: Protect Mutex's thread lists with a spinlock
2022-04-05 12:44:50 +00:00
} else {
Kernel: Track big lock blocked threads in separate list When we lock a mutex, eventually `Thread::block` is invoked which could in turn invoke `Process::big_lock().restore_exclusive_lock()`. This would then try to add the current thread to a different blocked thread list then the one in use for the original mutex being locked, and because it's an intrusive list, the thread is removed from its original list during the `.append()`. When the original mutex eventually unblocks, we no longer have the thread in the intrusive blocked threads list and we panic. Solve this by making the big lock mutex special and giving it its own blocked thread list. Because the process big lock is temporary and is being actively removed from e.g. syscalls, it's a matter of time before we can also remove the fix introduced by this commit. Fixes issue #9401.
2022-04-05 23:49:30 +00:00
if (!unblock_exclusive(lists.exclusive))
Kernel: Protect Mutex's thread lists with a spinlock
2022-04-05 12:44:50 +00:00
unblock_shared();
}
});
Kernel: Oops, forgot to add Lock.cpp.
2019-07-29 10:00:14 +00:00
}
Kernel: Stop using HashMap in Mutex This commit removes the usage of HashMap in Mutex, thereby making Mutex be allocation-free. In order to achieve this several simplifications were made to Mutex, removing unused code-paths and extra VERIFYs: * We no longer support 'upgrading' a shared lock holder to an exclusive holder when it is the only shared holder and it did not unlock the lock before relocking it as exclusive. NOTE: Unlike the rest of these changes, this scenario is not VERIFY-able in an allocation-free way, as a result the new LOCK_SHARED_UPGRADE_DEBUG debug flag was added, this flag lets Mutex allocate in order to detect such cases when debugging a deadlock. * We no longer support checking if a Mutex is locked by the current thread when the Mutex was not locked exclusively, the shared version of this check was not used anywhere. * We no longer support force unlocking/relocking a Mutex if the Mutex was not locked exclusively, the shared version of these functions was not used anywhere.
2022-01-28 23:47:18 +00:00
auto Mutex::force_unlock_exclusive_if_locked(u32& lock_count_to_restore) -> Mode
Kernel: Oops, forgot to add Lock.cpp.
2019-07-29 10:00:14 +00:00
{
Kernel: Verify mutex big lock behavior These two methods are big lock specific, so verify our mutex' behavior.
2022-04-09 13:30:34 +00:00
VERIFY(m_behavior == MutexBehavior::BigLock);
Kernel: Fix some problems with Thread::wait_on and Lock This changes the Thread::wait_on function to not enable interrupts upon leaving, which caused some problems with page fault handlers and in other situations. It may now be called from critical sections, with interrupts enabled or disabled, and returns to the same state. This also requires some fixes to Lock. To aid debugging, a new define LOCK_DEBUG is added that enables checking for Lock leaks upon finalization of a Thread.
2020-12-01 02:04:36 +00:00
// NOTE: This may be called from an interrupt handler (not an IRQ handler)
// and also from within critical sections!
Kernel: Convert Processor::in_irq() to static current_in_irq() This closes the race window between Processor::current() and a context switch happening before in_irq().
2021-08-22 10:21:31 +00:00
VERIFY(!Processor::current_in_irq());
Kernel: Verify mutex big lock behavior These two methods are big lock specific, so verify our mutex' behavior.
2022-04-09 13:30:34 +00:00
Kernel: Add implied auto-specifiers in Locking As per clang-tidy.
2021-12-15 13:56:39 +00:00
auto* current_thread = Thread::current();
Kernel: Rename ScopedSpinlock => SpinlockLocker This matches MutexLocker, and doesn't sound like it's a lock itself.
2021-08-21 23:49:22 +00:00
SpinlockLocker lock(m_lock);
Kernel: Allow Lock to block from BlockCondition This enables the Lock class to block a thread even while the thread is working on a BlockCondition. A thread can still only be either blocked by a Lock or a BlockCondition. This also establishes a linked list of threads that are blocked by a Lock and unblocking directly unlocks threads and wakes them directly.
2021-07-10 16:23:16 +00:00
auto current_mode = m_mode;
switch (current_mode) {
case Mode::Exclusive: {
if (m_holder != current_thread) {
lock_count_to_restore = 0;
return Mode::Unlocked;
}
Kernel: Stop using HashMap in Mutex This commit removes the usage of HashMap in Mutex, thereby making Mutex be allocation-free. In order to achieve this several simplifications were made to Mutex, removing unused code-paths and extra VERIFYs: * We no longer support 'upgrading' a shared lock holder to an exclusive holder when it is the only shared holder and it did not unlock the lock before relocking it as exclusive. NOTE: Unlike the rest of these changes, this scenario is not VERIFY-able in an allocation-free way, as a result the new LOCK_SHARED_UPGRADE_DEBUG debug flag was added, this flag lets Mutex allocate in order to detect such cases when debugging a deadlock. * We no longer support checking if a Mutex is locked by the current thread when the Mutex was not locked exclusively, the shared version of this check was not used anywhere. * We no longer support force unlocking/relocking a Mutex if the Mutex was not locked exclusively, the shared version of these functions was not used anywhere.
2022-01-28 23:47:18 +00:00
dbgln_if(LOCK_RESTORE_DEBUG, "Mutex::force_unlock_exclusive_if_locked @ {}: unlocking exclusive with lock count: {}", this, m_times_locked);
Kernel: Fix some race conditions with Lock and waiting/waking threads There is a window between acquiring/releasing the lock with the atomic variables and subsequently waiting or waking threads. With a single processor this window was closed by using a critical section, but this doesn't prevent other processors from running these code paths. To solve this, set a flag in the WaitQueue while holding m_lock which determines if threads should be blocked at all.
2021-01-24 05:30:10 +00:00
#if LOCK_DEBUG
Kernel: Allow Lock to block from BlockCondition This enables the Lock class to block a thread even while the thread is working on a BlockCondition. A thread can still only be either blocked by a Lock or a BlockCondition. This also establishes a linked list of threads that are blocked by a Lock and unblocking directly unlocks threads and wakes them directly.
2021-07-10 16:23:16 +00:00
m_holder->holding_lock(*this, -(int)m_times_locked, {});
Kernel: Fix some race conditions with Lock and waiting/waking threads There is a window between acquiring/releasing the lock with the atomic variables and subsequently waiting or waking threads. With a single processor this window was closed by using a critical section, but this doesn't prevent other processors from running these code paths. To solve this, set a flag in the WaitQueue while holding m_lock which determines if threads should be blocked at all.
2021-01-24 05:30:10 +00:00
#endif
Kernel: Allow Lock to block from BlockCondition This enables the Lock class to block a thread even while the thread is working on a BlockCondition. A thread can still only be either blocked by a Lock or a BlockCondition. This also establishes a linked list of threads that are blocked by a Lock and unblocking directly unlocks threads and wakes them directly.
2021-07-10 16:23:16 +00:00
m_holder = nullptr;
VERIFY(m_times_locked > 0);
lock_count_to_restore = m_times_locked;
m_times_locked = 0;
m_mode = Mode::Unlocked;
unblock_waiters(Mode::Exclusive);
break;
}
case Mode::Unlocked: {
lock_count_to_restore = 0;
break;
}
default:
VERIFY_NOT_REACHED();
}
return current_mode;
Kernel: Fix Lock race causing infinite spinning between two threads We need to account for how many shared lock instances the current thread owns, so that we can properly release such references when yielding execution. We also need to release the process lock when donating.
2020-12-14 23:36:22 +00:00
}
Kernel: Stop using HashMap in Mutex This commit removes the usage of HashMap in Mutex, thereby making Mutex be allocation-free. In order to achieve this several simplifications were made to Mutex, removing unused code-paths and extra VERIFYs: * We no longer support 'upgrading' a shared lock holder to an exclusive holder when it is the only shared holder and it did not unlock the lock before relocking it as exclusive. NOTE: Unlike the rest of these changes, this scenario is not VERIFY-able in an allocation-free way, as a result the new LOCK_SHARED_UPGRADE_DEBUG debug flag was added, this flag lets Mutex allocate in order to detect such cases when debugging a deadlock. * We no longer support checking if a Mutex is locked by the current thread when the Mutex was not locked exclusively, the shared version of this check was not used anywhere. * We no longer support force unlocking/relocking a Mutex if the Mutex was not locked exclusively, the shared version of these functions was not used anywhere.
2022-01-28 23:47:18 +00:00
void Mutex::restore_exclusive_lock(u32 lock_count, [[maybe_unused]] LockLocation const& location)
Kernel: Fix Lock race causing infinite spinning between two threads We need to account for how many shared lock instances the current thread owns, so that we can properly release such references when yielding execution. We also need to release the process lock when donating.
2020-12-14 23:36:22 +00:00
{
Kernel: Verify mutex big lock behavior These two methods are big lock specific, so verify our mutex' behavior.
2022-04-09 13:30:34 +00:00
VERIFY(m_behavior == MutexBehavior::BigLock);
Everywhere: Rename ASSERT => VERIFY (...and ASSERT_NOT_REACHED => VERIFY_NOT_REACHED) Since all of these checks are done in release builds as well, let's rename them to VERIFY to prevent confusion, as everyone is used to assertions being compiled out in release. We can introduce a new ASSERT macro that is specifically for debug checks, but I'm doing this wholesale conversion first since we've accumulated thousands of these already, and it's not immediately obvious which ones are suitable for ASSERT.
2021-02-23 19:42:32 +00:00
VERIFY(lock_count > 0);
Kernel: Convert Processor::in_irq() to static current_in_irq() This closes the race window between Processor::current() and a context switch happening before in_irq().
2021-08-22 10:21:31 +00:00
VERIFY(!Processor::current_in_irq());
Kernel: Verify mutex big lock behavior These two methods are big lock specific, so verify our mutex' behavior.
2022-04-09 13:30:34 +00:00
Kernel: Add implied auto-specifiers in Locking As per clang-tidy.
2021-12-15 13:56:39 +00:00
auto* current_thread = Thread::current();
Kernel: Allow Lock to block from BlockCondition This enables the Lock class to block a thread even while the thread is working on a BlockCondition. A thread can still only be either blocked by a Lock or a BlockCondition. This also establishes a linked list of threads that are blocked by a Lock and unblocking directly unlocks threads and wakes them directly.
2021-07-10 16:23:16 +00:00
bool did_block = false;
Kernel: Rename ScopedSpinlock => SpinlockLocker This matches MutexLocker, and doesn't sound like it's a lock itself.
2021-08-21 23:49:22 +00:00
SpinlockLocker lock(m_lock);
Kernel: Stop using HashMap in Mutex This commit removes the usage of HashMap in Mutex, thereby making Mutex be allocation-free. In order to achieve this several simplifications were made to Mutex, removing unused code-paths and extra VERIFYs: * We no longer support 'upgrading' a shared lock holder to an exclusive holder when it is the only shared holder and it did not unlock the lock before relocking it as exclusive. NOTE: Unlike the rest of these changes, this scenario is not VERIFY-able in an allocation-free way, as a result the new LOCK_SHARED_UPGRADE_DEBUG debug flag was added, this flag lets Mutex allocate in order to detect such cases when debugging a deadlock. * We no longer support checking if a Mutex is locked by the current thread when the Mutex was not locked exclusively, the shared version of this check was not used anywhere. * We no longer support force unlocking/relocking a Mutex if the Mutex was not locked exclusively, the shared version of these functions was not used anywhere.
2022-01-28 23:47:18 +00:00
[[maybe_unused]] auto previous_mode = m_mode;
if (m_mode == Mode::Exclusive && m_holder != current_thread) {
block(*current_thread, Mode::Exclusive, lock, lock_count);
did_block = true;
// If we blocked then m_mode should have been updated to what we requested
VERIFY(m_mode == Mode::Exclusive);
Kernel: Allow Lock to block from BlockCondition This enables the Lock class to block a thread even while the thread is working on a BlockCondition. A thread can still only be either blocked by a Lock or a BlockCondition. This also establishes a linked list of threads that are blocked by a Lock and unblocking directly unlocks threads and wakes them directly.
2021-07-10 16:23:16 +00:00
}
Kernel: Fix Lock race causing infinite spinning between two threads We need to account for how many shared lock instances the current thread owns, so that we can properly release such references when yielding execution. We also need to release the process lock when donating.
2020-12-14 23:36:22 +00:00
Kernel: Stop using HashMap in Mutex This commit removes the usage of HashMap in Mutex, thereby making Mutex be allocation-free. In order to achieve this several simplifications were made to Mutex, removing unused code-paths and extra VERIFYs: * We no longer support 'upgrading' a shared lock holder to an exclusive holder when it is the only shared holder and it did not unlock the lock before relocking it as exclusive. NOTE: Unlike the rest of these changes, this scenario is not VERIFY-able in an allocation-free way, as a result the new LOCK_SHARED_UPGRADE_DEBUG debug flag was added, this flag lets Mutex allocate in order to detect such cases when debugging a deadlock. * We no longer support checking if a Mutex is locked by the current thread when the Mutex was not locked exclusively, the shared version of this check was not used anywhere. * We no longer support force unlocking/relocking a Mutex if the Mutex was not locked exclusively, the shared version of these functions was not used anywhere.
2022-01-28 23:47:18 +00:00
dbgln_if(LOCK_RESTORE_DEBUG, "Mutex::restore_exclusive_lock @ {}: restoring exclusive with lock count {}, was {}", this, lock_count, mode_to_string(previous_mode));
Kernel: Allow Lock to block from BlockCondition This enables the Lock class to block a thread even while the thread is working on a BlockCondition. A thread can still only be either blocked by a Lock or a BlockCondition. This also establishes a linked list of threads that are blocked by a Lock and unblocking directly unlocks threads and wakes them directly.
2021-07-10 16:23:16 +00:00
Kernel: Stop using HashMap in Mutex This commit removes the usage of HashMap in Mutex, thereby making Mutex be allocation-free. In order to achieve this several simplifications were made to Mutex, removing unused code-paths and extra VERIFYs: * We no longer support 'upgrading' a shared lock holder to an exclusive holder when it is the only shared holder and it did not unlock the lock before relocking it as exclusive. NOTE: Unlike the rest of these changes, this scenario is not VERIFY-able in an allocation-free way, as a result the new LOCK_SHARED_UPGRADE_DEBUG debug flag was added, this flag lets Mutex allocate in order to detect such cases when debugging a deadlock. * We no longer support checking if a Mutex is locked by the current thread when the Mutex was not locked exclusively, the shared version of this check was not used anywhere. * We no longer support force unlocking/relocking a Mutex if the Mutex was not locked exclusively, the shared version of these functions was not used anywhere.
2022-01-28 23:47:18 +00:00
VERIFY(m_mode != Mode::Shared);
VERIFY(m_shared_holders == 0);
if (did_block) {
VERIFY(m_times_locked > 0);
VERIFY(m_holder == current_thread);
} else {
if (m_mode == Mode::Unlocked) {
m_mode = Mode::Exclusive;
VERIFY(m_times_locked == 0);
m_times_locked = lock_count;
VERIFY(!m_holder);
m_holder = current_thread;
Kernel: Allow Lock to block from BlockCondition This enables the Lock class to block a thread even while the thread is working on a BlockCondition. A thread can still only be either blocked by a Lock or a BlockCondition. This also establishes a linked list of threads that are blocked by a Lock and unblocking directly unlocks threads and wakes them directly.
2021-07-10 16:23:16 +00:00
} else {
Kernel: Stop using HashMap in Mutex This commit removes the usage of HashMap in Mutex, thereby making Mutex be allocation-free. In order to achieve this several simplifications were made to Mutex, removing unused code-paths and extra VERIFYs: * We no longer support 'upgrading' a shared lock holder to an exclusive holder when it is the only shared holder and it did not unlock the lock before relocking it as exclusive. NOTE: Unlike the rest of these changes, this scenario is not VERIFY-able in an allocation-free way, as a result the new LOCK_SHARED_UPGRADE_DEBUG debug flag was added, this flag lets Mutex allocate in order to detect such cases when debugging a deadlock. * We no longer support checking if a Mutex is locked by the current thread when the Mutex was not locked exclusively, the shared version of this check was not used anywhere. * We no longer support force unlocking/relocking a Mutex if the Mutex was not locked exclusively, the shared version of these functions was not used anywhere.
2022-01-28 23:47:18 +00:00
VERIFY(m_mode == Mode::Exclusive);
VERIFY(m_holder == current_thread);
VERIFY(m_times_locked > 0);
m_times_locked += lock_count;
Kernel: Fix some problems with Thread::wait_on and Lock This changes the Thread::wait_on function to not enable interrupts upon leaving, which caused some problems with page fault handlers and in other situations. It may now be called from critical sections, with interrupts enabled or disabled, and returns to the same state. This also requires some fixes to Lock. To aid debugging, a new define LOCK_DEBUG is added that enables checking for Lock leaks upon finalization of a Thread.
2020-12-01 02:04:36 +00:00
}
Kernel: Stop using HashMap in Mutex This commit removes the usage of HashMap in Mutex, thereby making Mutex be allocation-free. In order to achieve this several simplifications were made to Mutex, removing unused code-paths and extra VERIFYs: * We no longer support 'upgrading' a shared lock holder to an exclusive holder when it is the only shared holder and it did not unlock the lock before relocking it as exclusive. NOTE: Unlike the rest of these changes, this scenario is not VERIFY-able in an allocation-free way, as a result the new LOCK_SHARED_UPGRADE_DEBUG debug flag was added, this flag lets Mutex allocate in order to detect such cases when debugging a deadlock. * We no longer support checking if a Mutex is locked by the current thread when the Mutex was not locked exclusively, the shared version of this check was not used anywhere. * We no longer support force unlocking/relocking a Mutex if the Mutex was not locked exclusively, the shared version of these functions was not used anywhere.
2022-01-28 23:47:18 +00:00
}
Kernel: Move all code into the Kernel namespace
2020-02-16 00:27:42 +00:00
Kernel: Allow Lock to block from BlockCondition This enables the Lock class to block a thread even while the thread is working on a BlockCondition. A thread can still only be either blocked by a Lock or a BlockCondition. This also establishes a linked list of threads that are blocked by a Lock and unblocking directly unlocks threads and wakes them directly.
2021-07-10 16:23:16 +00:00
#if LOCK_DEBUG
Kernel: Stop using HashMap in Mutex This commit removes the usage of HashMap in Mutex, thereby making Mutex be allocation-free. In order to achieve this several simplifications were made to Mutex, removing unused code-paths and extra VERIFYs: * We no longer support 'upgrading' a shared lock holder to an exclusive holder when it is the only shared holder and it did not unlock the lock before relocking it as exclusive. NOTE: Unlike the rest of these changes, this scenario is not VERIFY-able in an allocation-free way, as a result the new LOCK_SHARED_UPGRADE_DEBUG debug flag was added, this flag lets Mutex allocate in order to detect such cases when debugging a deadlock. * We no longer support checking if a Mutex is locked by the current thread when the Mutex was not locked exclusively, the shared version of this check was not used anywhere. * We no longer support force unlocking/relocking a Mutex if the Mutex was not locked exclusively, the shared version of these functions was not used anywhere.
2022-01-28 23:47:18 +00:00
m_holder->holding_lock(*this, (int)lock_count, location);
Kernel: Allow Lock to block from BlockCondition This enables the Lock class to block a thread even while the thread is working on a BlockCondition. A thread can still only be either blocked by a Lock or a BlockCondition. This also establishes a linked list of threads that are blocked by a Lock and unblocking directly unlocks threads and wakes them directly.
2021-07-10 16:23:16 +00:00
#endif
Kernel: Allow process with multiple threads to call exec and exit This allows a process wich has more than 1 thread to call exec, even from a thread. This kills all the other threads, but it won't wait for them to finish, just makes sure that they are not in a running/runable state. In the case where a thread does exec, the new program PID will be the thread TID, to keep the PID == TID in the new process. This introduces a new function inside the Process class, kill_threads_except_self which is called on exit() too (exit with multiple threads wasn't properly working either). Inside the Lock class, there is the need for a new function, clear_waiters, which removes all the waiters from the Process::big_lock. This is needed since after a exit/exec, there should be no other threads waiting for this lock, the threads should be simply killed. Only queued threads should wait for this lock at this point, since blocked threads are handled in set_should_die.
2020-02-18 12:28:28 +00:00
}
Kernel: Move all code into the Kernel namespace
2020-02-16 00:27:42 +00:00
}
Reference in a new issue Copy permalink