Instead of returning HeapBlock memory to the kernel (or a non-type
specific shared cache), we now keep a BlockAllocator per CellAllocator
and implement "deallocation" by basically informing the kernel that we
don't need the physical memory right now.
This is done with MADV_FREE or MADV_DONTNEED if available, but for other
platforms (including SerenityOS) we munmap and then re-mmap the memory
to achieve the same effect. It's definitely clunky, so I've added a
FIXME about implementing the madvise options on SerenityOS too.
The important outcome of this change is that GC types that use a
type-specific allocator become immune to use-after-free type confusion
attacks, since their virtual addresses will only ever be re-used for
the same exact type again and again.
Fixes#22274
Cell::heap() and Cell::vm() needed to access member functions from
HeapBlock, and wanted to be inline, so they were moved to VM.h.
That approach will no longer work with VM.h not being included in every
file (starting from the next commit), so this commit fixes that circular
import issue by introducing secondary base classes to host the
references to Heap and VM, respectively.
Mark the entirety of a heap block's storage poisoned at construction.
Unpoison all of a Cell's memory before allocating it, and re-poison as
much as possible on deallocation. Unfortunately, the entirety of the
FreelistEntry must be kept unpoisoned in order for reallocation to work
correctly.
Decreasing the size of FreelistEntry or adding a larger redzone to Cells
would make the instrumentation even better.
This patch adds a BlockAllocator to the GC heap where we now cache up to
64 HeapBlock-sized mmap's that get recycled when allocating HeapBlocks.
This improves test-js runtime performance by ~35%, pretty cool! :^)
So far we only have two states: Live and Dead. In the future, we can
add additional states to support incremental sweeping and/or multi-
stage cell destruction.
HeapBlock now implements the same lazy freelist as LibC malloc() does,
where new blocks start out in a "bump allocator" mode that gets used
until we've bump-allocated all the way to the end of the block.
Then we fall back to the old freelist style as before.
This means we don't have to pre-initialize the freelist on HeapBlock
construction. This defers page faults and reduces memory usage for
blocks where all cells don't get used. :^)
SPDX License Identifiers are a more compact / standardized
way of representing file license information.
See: https://spdx.dev/resources/use/#identifiers
This was done with the `ambr` search and replace tool.
ambr --no-parent-ignore --key-from-file --rep-from-file key.txt rep.txt *
(...and ASSERT_NOT_REACHED => VERIFY_NOT_REACHED)
Since all of these checks are done in release builds as well,
let's rename them to VERIFY to prevent confusion, as everyone is
used to assertions being compiled out in release.
We can introduce a new ASSERT macro that is specifically for debug
checks, but I'm doing this wholesale conversion first since we've
accumulated thousands of these already, and it's not immediately
obvious which ones are suitable for ASSERT.
Allocate GC heap blocks with mmap(MAP_RANDOMIZED) for ASLR.
This may very well be too aggressive in terms of fragmentation, and we
can figure out ways to scale that back once it becomes a big problem.
For now, this makes the GC heap a lot less predictable for an attacker.
