This is in preparation for adding MSI(x) support to the NVMe device.
NVMeInterruptQueue needs access to the PCI device to deal with MSI(x)
interrupts. It is ok to pass the NVMeController as a reference to the
NVMeQueue as NVMeController is the one that owns the NVMeQueue.
This is very similar to how AHCIController passes its reference to its
interrupt handler.
Add an explicit QueueType enum which could be used to create a poll or
an interrupt queue. This is better than passing an Optional<irq>.
This refactoring is in preparation for adding MSIx support to NVMe.
PCIIRQHandler is a generic IRQ handler that the device driver can
inherit to use either Pin or MSI(x) based interrupt mechanism.
The PCIIRQHandler can do what the existing IRQHandler can do for pin
based interrupts but also deal with MSI based interrupts. We can
hopefully convert all the PCI based devices to use this handler so that
MSI(x) can be used.
Add reserve_irqs, allocate_irq, enable_interrupt and disable_interrupt
API to a PCI device.
reserve_irqs() can be used by a device driver that would like to
reserve irqs for MSI(x) interrupts. The API returns the type of IRQ
that was reserved by the PCI device. If the PCI device does not support
MSI(x), then it is a noop.
allocate_irq() API can be used to allocate an IRQ at an index. For
MSIx the driver needs to map the vector table into the memory and add
the corresponding IRQ at the given index. This API will return the
actual IRQ that was used so that the driver can use it create interrupt
handler for that IRQ.
{enable, disable}_interrupt API is used to enable or disable a
particular IRQ at the given index. It is a noop for pin-based
interrupts. This could be used by IRQHandler to enable or disable an
interrupt.
MSIx table entry is used to program interrupt vectors and it is
architecture specific. Add helper functions declaration in
Arch/PCIMSI.h. The definition of the function is placed in the
respective arch specific code.
Add a struct named MSIxInfo that stores all the relevant MSIx
information as a part of PCI DeviceIdentifier struct.
Populate the MSIx struct during the PCI device init. As the
DeviceIdentifier struct need to populate MSIx info, don't mark
DeviceIdentifier as const in the PCI::Device class.
MSI(x) interrupts need to reserve IRQs so that it can be programmed by
the device. Add an API to reserve contiguous ranges of interrupt
handlers so that it can used by PCI devices that use MSI(x) mechanism.
This API needs to be implemented by aarch64 architecture.
Set pin-based interrupt handler as reserved during PCI bus init.
This is required so that MSI(x) based interrupts can avoid sharing the
IRQ which has been marked as reserved.
Pin-based PCI device are allocated an IRQ, and it could be shared with
multiple devices. An interrupt handler with an IRQ for a PCI device
will get registered only during the driver initialization.
For MSI(x) interrupts, the driver has to allocate IRQs and this field
can be used to skip IRQs that have already been reserved by pin-based
interrupts so that we don't have to share IRQs, which generally will
reduce the performance.
"The official project language is American English […]."
5d2e915623/CONTRIBUTING.md (L30)
Here's a short statistic of the occurrences of the word "behavio(u)r":
$ git grep -IPioh 'behaviou?r' | sort | uniq -c | sort -n
2 BEHAVIOR
24 Behaviour
32 behaviour
407 Behavior
992 behavior
Therefore, it is clear that "behaviour" (56 occurrences) should be
regarded a typo, and "behavior" (1401 occurrences) should be preferred.
Note that The occurrences in LibJS are intentionally NOT changed,
because there are taken verbatim from the specification. Hence:
$ git grep -IPioh 'behaviou?r' | sort | uniq -c | sort -n
2 BEHAVIOR
10 behaviour
24 Behaviour
407 Behavior
1014 behavior
This was discovered by me during a work on USB keyboard patches, so it
triggered this bug.
The printing format for the VirtualAddress part is incorrect, leading to
another crash when handling page fault after accessing UNMAP_AFTER_INIT
code section.
Whenever an entry is added to the cache, the last element is removed to
make space for the new entry(if the cache is full). To make this an LRU
cache, the entry needs to be moved to the front of the list when there
is a cache hit so that the least recently used entry moves to the end
to be evicted first.
Rename the initialize method to initialize_virtio_resources so it's
clear what this method is intended for.
To ensure healthier device initialization, we could also return the type
of ErrorOr<void> from this method, so in all overriden instances and in
the original method code, we could leverage TRY() pattern which also
does simplify the code a bit.
Since multiboot_modules_count is set to 0, we can safely set the
multiboot_modules pointer to 0 (null pointer), as we don't use multiboot
on aarch64 anyway.
This reuses the existing `RPi::Mailbox` interface to read the command
line via a VideoCore-specific mailbox message. This will have to be
replaced if that interface starts being smarter, as this is needed very
early, and nothing guarantees that a smarter Mailbox interface wouldn't
need to allocate or log, which is a no-no during early boot.
As the response string can be arbitrarily long, it's the caller's job to
provide a long enough buffer for `Mailbox::query_kernel_command_line`.
This commit chose 512 bytes, as it provides a large enough headroom over
the 150-200 characters implicitly added by the VC firmware.
The portable way would be to parse the `/chosen/bootargs` property of
the device tree, but we currently lack the scaffolding for doing that.
Support for this in QEMU relies on a patch that has not yet been
accepted upstream, but is available via our `Toolchain/BuildQEMU.sh`
script. It should, however, work on bare metal.
Tested-By: Timon Kruiper <timonkruiper@gmail.com>
The Raspberry Pi's mailbox interface does not guarantee that the
returned command line is null-terminated. This commit removes that
assumption from the current code, allowing the next commit to add
support for reading it on the Pi.
This also lets us eliminate a few manual `strlen()` calls :^)
Some hardware/software configurations crash KVM as soon as we try to
start Serenity. The exact cause is currently unknown, so just fully
revert it for now.
This reverts commit 897c4e5145.
Process created performance events for kernel processes are only ever
emitted for the kernel processes that exist when profiling is enabled.
Any new kernel processes created after profiling is enabled will not
have corresponding process created performance events, so all kernel
processes should be created before enabling profiling.
NetworkTask was the only kernel process being created after enabling
profiling, so we now just create it before enabling profiling. This
fixes an issue where Profiler was failing to parse boot profiles as a
result of NetworkTask not having a process created event.
The new baked image is a Prekernel and a Kernel baked together now, so
essentially we no longer need to pass the Prekernel as -kernel and the
actual kernel image as -initrd to QEMU, leaving the option to pass an
actual initrd or initramfs module later on with multiboot.
is_sharing_with_others API was never really put to use properly since
it was introduced. The only place where it is used in Interrupts.cpp is
in conjuction with is_shared_handler() which is only true for
SharedIRQHandler and is_sharing_with_others will always return false.
Remove that API.
If IRQHandler's IRQ is shared, then disable_irq() should not call the
controller to disable that IRQ as some other device might be using it.
IRQHandler had a private variable to indicate if it is being shared:
m_shared_with_others but it was never modified even if the IRQ was
shared.
Add a new member function set_shared_with_others() to enable/disable
m_shared_with_others member of IRQHandler class. This function is
called when an IRQHandler is being added/removed as a part of
SharedIRQHandler.
This is done mainly by implementing safe locking on the data structure
keeping the pointers to the PerContextState objects. Therefore, this now
eliminates the need for using LockRefPtr, as SpinlockProtected is enough
for the whole list.
The usage of HashMap in this class was questionable, and according to
Sahan Fernando (the original contributor to the VirGL work also known as
ccapitalK) there was no deep research on which data structure to use for
keeping all pointers to PerContextState objects.
Therefore, this structure is changed to IntrusiveList as the main reason
and advantage to use it is that handling OOM conditions is much more
simple, because if we succeeded to create a PerContextState object, we
can be sure now that inserting it to the list will not cause OOM error
condition.
This is quite useful for userspace applications that can't cope with the
restriction, but it's still useful to impose other non-configurable
restrictions by using jails.
Instead of storing x86_64 register names in `SC_create_thread_params`,
let the Kernel figure out how to pass the parameters to
`pthread_create_helper`.
This was the last change that was needed to be able boot with the flag
of LOCK_IN_CRITICAL_DEBUG. That flag is not always enabled because there
are still other issues in which we hold a spinlock and still try to lock
a mutex.
Instead of using one global mutex we can protect internal structures of
the InodeWatcher class with SpinlockProtected wrappers. This in turn
allows the InodeWatcher code to be called from other parts in the kernel
while holding a prior spinlock properly.
`process.fds()` is protected by a Mutex, which causes issues when we try
to acquire it while holding a Spinlock. Since nothing seems to use this
value, let's just remove it entirely for now.
This should not affect boot times on qemu, as that does not use
dynamic transfer delays in its adma code path.
On real hardware this could potentially double the data throughput,
decreasing load times.
This has been broken on x86_64 since its introduction, as it features
more registers to be saved, and we never held up the "rbp has to be the
last pushed register" there.
Instead, just copy rbp from the thread structure, which is now properly
updated since the last commit.
The existing `read_entire` is quite slow due to allocating and copying
multiple times, but it is simultaneously quite hard to get rid of in a
single step. As a replacement, add a new function that reads as much as
possible directly into a user-provided buffer.
Currently we do not use interrupts for the SD driver, yet we
had enabled the signaling of all of them.
Since we were never acknowledging them, we were getting spammed by
unnecessary interrupts, causing the system to slow down to a crawl.
This commit makes the system boot in less than 1 minute with PIO,
compared to the old 30+ minute boot.
There is a big mix of LockRefPtrs all over the Networking subsystem, as
well as lots of room for improvements with our locking patterns, which
this commit will not pursue, but will give a good start for such work.
To deal with this situation, we change the following things:
- Creating instances of NetworkAdapter should always yield a non-locking
NonnullRefPtr. Acquiring an instance from the NetworkingManagement
should give a simple RefPtr,as giving LockRefPtr does not really
protect from concurrency problems in such case.
- Since NetworkingManagement works with normal RefPtrs we should
protect all instances of RefPtr<NetworkAdapter> with SpinlockProtected
to ensure references are gone unexpectedly.
- Protect the so_error class member with a proper spinlock. This happens
to be important because the clear_so_error() method lacked any proper
locking measures. It also helps preventing a possible TOCTOU when we
might do a more fine-grained locking in the Socket code, so this could
be definitely a start for this.
- Change unnecessary LockRefPtr<PacketWithTimestamp> in the structure
of OutgoingPacket to a simple RefPtr<PacketWithTimestamp> as the whole
list should be MutexProtected.
To do this we also need to get rid of LockRefPtrs in the USB code as
well.
Most of the SysFS nodes are statically generated during boot and are not
mutated afterwards.
The same goes for general device code - once we generate the appropriate
SysFS nodes, we almost never mutate the node pointers afterwards, making
locking unnecessary.
While doing this, we can also just return a normal RefPtr instead of a
LockRefPtr, because we create these channels when initializing an audio
controller, and never change the pointer in AudioController instances
after their initialization, hence no locking is necessary.
Instead of enumerating all available controllers and then ask each to
find its audio channels, we change the initialization sequence to match
what happens in the Networking subsystem and Graphics subsystem - we
essentially probe for a matching driver on a PCI device, create a device
instance, and immediately initialize it.
This in fact allows us to immediately find any hardware initialization
issues and report it, and then dropping the created instance, as usually
being done in other initialization paths in the Kernel.
This also opens the opportunity to propagate errors when failed to
initialize an AudioChannel instance, and it will be addressed in a
future commit.
This is done by 2 ways which both fit very well together:
- We stop use LockRefPtrs. We also don't allow expansion of the
m_channels member, by setting it to be a fixed Array of 2
IDEChannels.
- More error propagation through the code, in the construction point of
IDEChannel(s). This means that in the future we could technically do
something meaningful with OOM conditions when initializing an IDE
controller.
We were crashing on the VERIFY_INTERRUPTS_DISABLED() in
RecursiveSpinlock::unlock, which was caused by the compiler reordering
instructions in `sys$get_root_session_id`. In this function, a SpinLock
is locked and quickly unlocked again, and since the lock and unlock
functions were inlined into `sys$get_root_session_id` and the DAIF::read
was missing the `volatile` keyword, the compiler was free to reorder the
reads from the DAIF register to the top of this function. This caused
the CPU to read the interrupts state at the beginning of the function,
and storing the result on the stack, which in turn caused the
VERIFY_INTERRUPTS_DISABLED() assertion to fail. By adding the `volatile`
modifier to the inline assembly, the compiler will not reorder the
instructions.
In aa40cef2b7, I mistakenly assumed that the crash was related to the
initial interrupts state of the kernel threads, but it turns out that
the missing `volatile` keyword was the actual problem. This commit also
removes that code again.
We have a problem with the original utimensat syscall because when we
do call LibC futimens function, internally we provide an empty path,
and the Kernel get_syscall_path_argument method will detect this as an
invalid path.
This happens to spit an error for example in the touch utility, so if a
user is running "touch non_existing_file", it will create that file, but
the user will still see an error coming from LibC futimens function.
This new syscall gets an open file description and it provides the same
functionality as utimensat, on the specified open file description.
The new syscall will be used later by LibC to properly implement LibC
futimens function so the situation described with relation to the
"touch" utility could be fixed.
We were detaching from the jail process list too early. To ensure we
detach properly, leverage the remove_from_secondary_lists method
so the possibly jailed parent process can still see the dying process
and therefore clean it properly.
For a very long time, the kernel had only support for basic PS/2 devices
such as the PS2 AT keyboard and regular PS2 mouse (with a scroll wheel).
To adapt to this, we had very simple abstractions in place, essentially,
the PS2 devices were registered as IRQ handlers (IRQ 1 and 12), and when
an interrupt was triggered, we simply had to tell the I8042Controller to
fetch a byte for us, then send it back to the appropriate device for
further processing and queueing of either a key event, or a mouse packet
so userspace can do something meaningful about it.
When we added the VMWare mouse integration feature it was easily adapted
to this paradigm, requiring small changes across the handling code for
these devices.
This patch is a major cleanup for any future advancements in the HID
subsystem.
It ensures we do things in a much more sane manner:
- We stop using LockRefPtrs. Currently, after the initialization of the
i8042 controller, we never have to change RefPtrs in that class, as we
simply don't support PS2 hotplugging currently.
Also, we remove the unnecessary getters for keyboard and mouse devices
which also returned a LockRefPtr.
- There's a clear separation between PS2 devices and the actual device
nodes that normally exist in /dev. PS2 devices are not polled, because
when the user uses these devices, they will trigger an IRQ which when
is handled, could produce either a MousePacket or KeyEvent, depending
on the device state.
The separation is crucial for buses that are polled, for example - USB
is a polled bus and will not generate an IRQ for HID devices.
- There's a clear separation in roles of each structure. The PS2 devices
which are attached to a I8042Controller object are managing the device
state, while the generic MouseDevice and KeyboardDevice manage all
related tasks of a CharacterDevice, as well as interpreting scan code
events and mouse relative/absolute coordinates.
It happens to be that only PS/2 devices that are connected via the i8042
controller can generate interrupt events, so it makes much more sense to
have those devices to implement the enable_interrupts method because of
the I8042Device class and not the HIDDevice class.
Use the new class in HID code, because all other HID device controllers
will be using this class as their parent class.
Hence, we no longer keep a reference to any PS/2 device in HIDManagement
and rely on HIDController derived classes to do this for us.
It also means that we removed another instance of a LockRefPtr, which
is designated to be removed and is replaced by the better pattern of
SpinlockProtected<RefPtr<>> instead.
The definitions were being defined already by `BootInfo.h` and that was
being included here via transitive includes. The extern definitions of
the variables do not have the `READONLY_AFTER_INIT` attribute in
`BootInfo.h`. This causes conflicting definitions of the same variable.
The `READONLY_AFTER_INIT` specifier is not needed for extern variables
as it only effects their linkage, not their actual use, so just use the
versions in `BootInfo.h` instead of re-declaring.
These were easy to pick-up as these pointers are assigned during the
construction point and are never changed afterwards.
This small change to these pointers will ensure that our code will not
accidentally assign these pointers with a new object which is always a
kind of bug we will want to prevent.
When switching to the new address space, we also have to switch the
Process::m_master_tls_* variables as they may refer to a region in
the old address space.
This was causing `su` to not run correctly.
Regression from 65641187ff.
Specifically this commit implements two setters set_userspace_sp and
set_ip in RegisterState.h, and also adds a stack pointer getter (sp) in
ThreadRegisters.h. Contributed by konrad, thanks for that.
Setting the page table base register (ttbr0_el1) is not enough, and will
not flush the TLB caches, in contrary with x86_64 where setting the CR3
register will actually flush the caches. This commit adds the necessary
code to properly flush the TLB caches when context switching. This
commit also changes Processor::flush_tlb_local to use the vmalle1
variant, as previously we would be flushing the tlb's of all the cores
in the inner-shareable domain.
Previously we had a race condition in the page fault handling: We were
relying on the affected Region staying alive while handling the page
fault, but this was not actually guaranteed, as an munmap from another
thread could result in the region being removed concurrently.
This commit closes that hole by extending the lifetime of the region
affected by the page fault until the handling of the page fault is
complete. This is achieved by maintaing a psuedo-reference count on the
region which counts the number of in-progress page faults being handled
on this region, and extending the lifetime of the region while this
counter is non zero.
Since both the increment of the counter by the page fault handler and
the spin loop waiting for it to reach 0 during Region destruction are
serialized using the appropriate AddressSpace spinlock, eventual
progress is guaranteed: As soon as the region is removed from the tree
no more page faults on the region can start.
And similarly correctness is ensured: The counter is incremented under
the same lock, so any page faults that are being handled will have
already incremented the counter before the region is deallocated.
This replaces the previous owning address space pointer. This commit
should not change any of the existing functionality, but it lays down
the groundwork needed to let us properly access the region table under
the address space spinlock during page fault handling.
Instead of setting up the new address space on it's own, and only swap
to the new address space at the end, we now immediately swap to the new
address space (while still keeping the old one alive) and only revert
back to the old one if we fail at any point.
This is done to ensure that the process' active address space (aka the
contents of m_space) always matches actual address space in use by it.
That should allow us to eventually make the page fault handler process-
aware, which will let us properly lock the process address space lock.
