beenull/ladybird
1
0
Fork 0
mirror of https://github.com/LadybirdBrowser/ladybird.git synced 2024-11-21 23:20:20 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 6
ladybird/Libraries/LibGC/CellAllocator.cpp

59 lines
1.6 KiB
C++
Raw Normal View History

LibJS: Split Heap into per-cell-size allocators Instead of keeping all the HeapBlocks in one big list, we now split it into two levels: - Heap has a set of Allocators, each with a specific cell size. - Allocators have two lists of blocks, "full" and "usable". Allocating a new cell no longer has to scan the entire set of blocks, but instead just needs to find the right allocator and then pop a cell from its freelist. If all the blocks in the allocator are full, a new block will be created. Blocks are moved from the "full" to "usable" list after sweeping has determined that they are not completely empty and not completely full. There are certainly many ways we can improve on this. This patch is mostly about getting the new allocator architecture in place. :^)
2020-10-06 16:50:47 +00:00
/*
Meta: Update my e-mail address everywhere
2024-10-04 11:19:50 +00:00
* Copyright (c) 2020-2023, Andreas Kling <andreas@ladybird.org>
LibJS: Split Heap into per-cell-size allocators Instead of keeping all the HeapBlocks in one big list, we now split it into two levels: - Heap has a set of Allocators, each with a specific cell size. - Allocators have two lists of blocks, "full" and "usable". Allocating a new cell no longer has to scan the entire set of blocks, but instead just needs to find the right allocator and then pop a cell from its freelist. If all the blocks in the allocator are full, a new block will be created. Blocks are moved from the "full" to "usable" list after sweeping has determined that they are not completely empty and not completely full. There are certainly many ways we can improve on this. This patch is mostly about getting the new allocator architecture in place. :^)
2020-10-06 16:50:47 +00:00
*
Everything: Move to SPDX license identifiers in all files. SPDX License Identifiers are a more compact / standardized way of representing file license information. See: https://spdx.dev/resources/use/#identifiers This was done with the `ambr` search and replace tool. ambr --no-parent-ignore --key-from-file --rep-from-file key.txt rep.txt *
2021-04-22 08:24:48 +00:00
* SPDX-License-Identifier: BSD-2-Clause
LibJS: Split Heap into per-cell-size allocators Instead of keeping all the HeapBlocks in one big list, we now split it into two levels: - Heap has a set of Allocators, each with a specific cell size. - Allocators have two lists of blocks, "full" and "usable". Allocating a new cell no longer has to scan the entire set of blocks, but instead just needs to find the right allocator and then pop a cell from its freelist. If all the blocks in the allocator are full, a new block will be created. Blocks are moved from the "full" to "usable" list after sweeping has determined that they are not completely empty and not completely full. There are certainly many ways we can improve on this. This patch is mostly about getting the new allocator architecture in place. :^)
2020-10-06 16:50:47 +00:00
*/
#include <AK/Badge.h>
LibGC+Everywhere: Factor out a LibGC from LibJS Resulting in a massive rename across almost everywhere! Alongside the namespace change, we now have the following names: * JS::NonnullGCPtr -> GC::Ref * JS::GCPtr -> GC::Ptr * JS::HeapFunction -> GC::Function * JS::CellImpl -> GC::Cell * JS::Handle -> GC::Root
2024-11-14 15:01:23 +00:00
#include <LibGC/BlockAllocator.h>
#include <LibGC/CellAllocator.h>
#include <LibGC/Heap.h>
#include <LibGC/HeapBlock.h>
LibJS: Split Heap into per-cell-size allocators Instead of keeping all the HeapBlocks in one big list, we now split it into two levels: - Heap has a set of Allocators, each with a specific cell size. - Allocators have two lists of blocks, "full" and "usable". Allocating a new cell no longer has to scan the entire set of blocks, but instead just needs to find the right allocator and then pop a cell from its freelist. If all the blocks in the allocator are full, a new block will be created. Blocks are moved from the "full" to "usable" list after sweeping has determined that they are not completely empty and not completely full. There are certainly many ways we can improve on this. This patch is mostly about getting the new allocator architecture in place. :^)
2020-10-06 16:50:47 +00:00
LibGC+Everywhere: Factor out a LibGC from LibJS Resulting in a massive rename across almost everywhere! Alongside the namespace change, we now have the following names: * JS::NonnullGCPtr -> GC::Ref * JS::GCPtr -> GC::Ptr * JS::HeapFunction -> GC::Function * JS::CellImpl -> GC::Cell * JS::Handle -> GC::Root
2024-11-14 15:01:23 +00:00
namespace GC {
LibJS: Split Heap into per-cell-size allocators Instead of keeping all the HeapBlocks in one big list, we now split it into two levels: - Heap has a set of Allocators, each with a specific cell size. - Allocators have two lists of blocks, "full" and "usable". Allocating a new cell no longer has to scan the entire set of blocks, but instead just needs to find the right allocator and then pop a cell from its freelist. If all the blocks in the allocator are full, a new block will be created. Blocks are moved from the "full" to "usable" list after sweeping has determined that they are not completely empty and not completely full. There are certainly many ways we can improve on this. This patch is mostly about getting the new allocator architecture in place. :^)
2020-10-06 16:50:47 +00:00
LibJS: Show class in SerenityOS mmap name for type-specific allocators
2023-12-31 11:39:46 +00:00
CellAllocator::CellAllocator(size_t cell_size, char const* class_name)
: m_class_name(class_name)
, m_cell_size(cell_size)
LibJS: Split Heap into per-cell-size allocators Instead of keeping all the HeapBlocks in one big list, we now split it into two levels: - Heap has a set of Allocators, each with a specific cell size. - Allocators have two lists of blocks, "full" and "usable". Allocating a new cell no longer has to scan the entire set of blocks, but instead just needs to find the right allocator and then pop a cell from its freelist. If all the blocks in the allocator are full, a new block will be created. Blocks are moved from the "full" to "usable" list after sweeping has determined that they are not completely empty and not completely full. There are certainly many ways we can improve on this. This patch is mostly about getting the new allocator architecture in place. :^)
2020-10-06 16:50:47 +00:00
{
}
LibGC+Everywhere: Factor out a LibGC from LibJS Resulting in a massive rename across almost everywhere! Alongside the namespace change, we now have the following names: * JS::NonnullGCPtr -> GC::Ref * JS::GCPtr -> GC::Ptr * JS::HeapFunction -> GC::Function * JS::CellImpl -> GC::Cell * JS::Handle -> GC::Root
2024-11-14 15:01:23 +00:00
Cell* CellAllocator::allocate_cell(Heap& heap)
LibJS: Split Heap into per-cell-size allocators Instead of keeping all the HeapBlocks in one big list, we now split it into two levels: - Heap has a set of Allocators, each with a specific cell size. - Allocators have two lists of blocks, "full" and "usable". Allocating a new cell no longer has to scan the entire set of blocks, but instead just needs to find the right allocator and then pop a cell from its freelist. If all the blocks in the allocator are full, a new block will be created. Blocks are moved from the "full" to "usable" list after sweeping has determined that they are not completely empty and not completely full. There are certainly many ways we can improve on this. This patch is mostly about getting the new allocator architecture in place. :^)
2020-10-06 16:50:47 +00:00
{
LibJS: Make Heap aware of all CellAllocators Also add a link from HeapBlock to their owning CellAllocator. This fixes an issue where the Heap would skip over non-size-based cell allocators.
2023-12-23 14:13:51 +00:00
if (!m_list_node.is_in_list())
heap.register_cell_allocator({}, *this);
LibJS: Split Heap into per-cell-size allocators Instead of keeping all the HeapBlocks in one big list, we now split it into two levels: - Heap has a set of Allocators, each with a specific cell size. - Allocators have two lists of blocks, "full" and "usable". Allocating a new cell no longer has to scan the entire set of blocks, but instead just needs to find the right allocator and then pop a cell from its freelist. If all the blocks in the allocator are full, a new block will be created. Blocks are moved from the "full" to "usable" list after sweeping has determined that they are not completely empty and not completely full. There are certainly many ways we can improve on this. This patch is mostly about getting the new allocator architecture in place. :^)
2020-10-06 16:50:47 +00:00
if (m_usable_blocks.is_empty()) {
LibJS: Show class in SerenityOS mmap name for type-specific allocators
2023-12-31 11:39:46 +00:00
auto block = HeapBlock::create_with_cell_size(heap, *this, m_cell_size, m_class_name);
LibJS: Save min and max block addresses in CellAllocator This allows to skip iterating through all allocated blocks in `find_min_and_max_block_addresses()`. With this change `collect_garbage()` in profiles of Discord goes down from 17% to 8%.
2024-04-23 14:53:59 +00:00
auto block_ptr = reinterpret_cast<FlatPtr>(block.ptr());
if (m_min_block_address > block_ptr)
m_min_block_address = block_ptr;
if (m_max_block_address < block_ptr)
m_max_block_address = block_ptr;
LibJS: Use IntrusiveList for Allocator's block lists This way we don't need to deal with shifting vector storage, and most operations are upgraded from O(n) to O(1) :^)
2020-10-07 12:04:52 +00:00
m_usable_blocks.append(*block.leak_ptr());
LibJS: Split Heap into per-cell-size allocators Instead of keeping all the HeapBlocks in one big list, we now split it into two levels: - Heap has a set of Allocators, each with a specific cell size. - Allocators have two lists of blocks, "full" and "usable". Allocating a new cell no longer has to scan the entire set of blocks, but instead just needs to find the right allocator and then pop a cell from its freelist. If all the blocks in the allocator are full, a new block will be created. Blocks are moved from the "full" to "usable" list after sweeping has determined that they are not completely empty and not completely full. There are certainly many ways we can improve on this. This patch is mostly about getting the new allocator architecture in place. :^)
2020-10-06 16:50:47 +00:00
}
auto& block = *m_usable_blocks.last();
auto* cell = block.allocate();
Everywhere: Rename ASSERT => VERIFY (...and ASSERT_NOT_REACHED => VERIFY_NOT_REACHED) Since all of these checks are done in release builds as well, let's rename them to VERIFY to prevent confusion, as everyone is used to assertions being compiled out in release. We can introduce a new ASSERT macro that is specifically for debug checks, but I'm doing this wholesale conversion first since we've accumulated thousands of these already, and it's not immediately obvious which ones are suitable for ASSERT.
2021-02-23 19:42:32 +00:00
VERIFY(cell);
LibJS: Use IntrusiveList for Allocator's block lists This way we don't need to deal with shifting vector storage, and most operations are upgraded from O(n) to O(1) :^)
2020-10-07 12:04:52 +00:00
if (block.is_full())
m_full_blocks.append(*m_usable_blocks.last());
LibJS: Split Heap into per-cell-size allocators Instead of keeping all the HeapBlocks in one big list, we now split it into two levels: - Heap has a set of Allocators, each with a specific cell size. - Allocators have two lists of blocks, "full" and "usable". Allocating a new cell no longer has to scan the entire set of blocks, but instead just needs to find the right allocator and then pop a cell from its freelist. If all the blocks in the allocator are full, a new block will be created. Blocks are moved from the "full" to "usable" list after sweeping has determined that they are not completely empty and not completely full. There are certainly many ways we can improve on this. This patch is mostly about getting the new allocator architecture in place. :^)
2020-10-06 16:50:47 +00:00
return cell;
}
LibJS: Rename Allocator => CellAllocator Now that we have a BlockAllocator as well, it seems appropriate to name the allocator-that-allocates-cells something more specific to match.
2021-05-27 17:03:41 +00:00
void CellAllocator::block_did_become_empty(Badge<Heap>, HeapBlock& block)
LibJS: Split Heap into per-cell-size allocators Instead of keeping all the HeapBlocks in one big list, we now split it into two levels: - Heap has a set of Allocators, each with a specific cell size. - Allocators have two lists of blocks, "full" and "usable". Allocating a new cell no longer has to scan the entire set of blocks, but instead just needs to find the right allocator and then pop a cell from its freelist. If all the blocks in the allocator are full, a new block will be created. Blocks are moved from the "full" to "usable" list after sweeping has determined that they are not completely empty and not completely full. There are certainly many ways we can improve on this. This patch is mostly about getting the new allocator architecture in place. :^)
2020-10-06 16:50:47 +00:00
{
LibJS: Use IntrusiveList for Allocator's block lists This way we don't need to deal with shifting vector storage, and most operations are upgraded from O(n) to O(1) :^)
2020-10-07 12:04:52 +00:00
block.m_list_node.remove();
LibJS: Recycle up to 64 HeapBlocks to improve performance :^) This patch adds a BlockAllocator to the GC heap where we now cache up to 64 HeapBlock-sized mmap's that get recycled when allocating HeapBlocks. This improves test-js runtime performance by ~35%, pretty cool! :^)
2021-05-27 17:01:26 +00:00
// NOTE: HeapBlocks are managed by the BlockAllocator, so we don't want to `delete` the block here.
block.~HeapBlock();
LibJS: Never give back virtual memory once it belongs to a cell type Instead of returning HeapBlock memory to the kernel (or a non-type specific shared cache), we now keep a BlockAllocator per CellAllocator and implement "deallocation" by basically informing the kernel that we don't need the physical memory right now. This is done with MADV_FREE or MADV_DONTNEED if available, but for other platforms (including SerenityOS) we munmap and then re-mmap the memory to achieve the same effect. It's definitely clunky, so I've added a FIXME about implementing the madvise options on SerenityOS too. The important outcome of this change is that GC types that use a type-specific allocator become immune to use-after-free type confusion attacks, since their virtual addresses will only ever be re-used for the same exact type again and again. Fixes #22274
2023-12-31 10:36:18 +00:00
m_block_allocator.deallocate_block(&block);
LibJS: Split Heap into per-cell-size allocators Instead of keeping all the HeapBlocks in one big list, we now split it into two levels: - Heap has a set of Allocators, each with a specific cell size. - Allocators have two lists of blocks, "full" and "usable". Allocating a new cell no longer has to scan the entire set of blocks, but instead just needs to find the right allocator and then pop a cell from its freelist. If all the blocks in the allocator are full, a new block will be created. Blocks are moved from the "full" to "usable" list after sweeping has determined that they are not completely empty and not completely full. There are certainly many ways we can improve on this. This patch is mostly about getting the new allocator architecture in place. :^)
2020-10-06 16:50:47 +00:00
}
LibJS: Rename Allocator => CellAllocator Now that we have a BlockAllocator as well, it seems appropriate to name the allocator-that-allocates-cells something more specific to match.
2021-05-27 17:03:41 +00:00
void CellAllocator::block_did_become_usable(Badge<Heap>, HeapBlock& block)
LibJS: Split Heap into per-cell-size allocators Instead of keeping all the HeapBlocks in one big list, we now split it into two levels: - Heap has a set of Allocators, each with a specific cell size. - Allocators have two lists of blocks, "full" and "usable". Allocating a new cell no longer has to scan the entire set of blocks, but instead just needs to find the right allocator and then pop a cell from its freelist. If all the blocks in the allocator are full, a new block will be created. Blocks are moved from the "full" to "usable" list after sweeping has determined that they are not completely empty and not completely full. There are certainly many ways we can improve on this. This patch is mostly about getting the new allocator architecture in place. :^)
2020-10-06 16:50:47 +00:00
{
Everywhere: Rename ASSERT => VERIFY (...and ASSERT_NOT_REACHED => VERIFY_NOT_REACHED) Since all of these checks are done in release builds as well, let's rename them to VERIFY to prevent confusion, as everyone is used to assertions being compiled out in release. We can introduce a new ASSERT macro that is specifically for debug checks, but I'm doing this wholesale conversion first since we've accumulated thousands of these already, and it's not immediately obvious which ones are suitable for ASSERT.
2021-02-23 19:42:32 +00:00
VERIFY(!block.is_full());
LibJS: Use IntrusiveList for Allocator's block lists This way we don't need to deal with shifting vector storage, and most operations are upgraded from O(n) to O(1) :^)
2020-10-07 12:04:52 +00:00
m_usable_blocks.append(block);
LibJS: Split Heap into per-cell-size allocators Instead of keeping all the HeapBlocks in one big list, we now split it into two levels: - Heap has a set of Allocators, each with a specific cell size. - Allocators have two lists of blocks, "full" and "usable". Allocating a new cell no longer has to scan the entire set of blocks, but instead just needs to find the right allocator and then pop a cell from its freelist. If all the blocks in the allocator are full, a new block will be created. Blocks are moved from the "full" to "usable" list after sweeping has determined that they are not completely empty and not completely full. There are certainly many ways we can improve on this. This patch is mostly about getting the new allocator architecture in place. :^)
2020-10-06 16:50:47 +00:00
}
}
Reference in a new issue Copy permalink