Commit graph

222 commits

Author SHA1 Message Date
Andreas Kling
a1d1a3b50b Kernel: Use BitmapView instead of Bitmap::wrap() 2021-03-04 11:25:45 +01:00
Ben Wiederhake
336303bda4 Kernel: Make kgettimeofday use AK::Time 2021-03-02 08:36:08 +01:00
Andreas Kling
69a30f95cc Ext2FS: Make block list flushing a bit less aggressive
We don't need to flush the on-disk inode struct multiple times while
writing out its block list. Just mark the in-memory Inode as having
dirty metadata and the SyncTask will flush it eventually.
2021-02-26 18:24:40 +01:00
Andreas Kling
c3a0fd4b7a Ext2FS: Move block list computation from Ext2FS to Ext2FSInode
Since the inode is the logical owner of its block list, let's move the
code that computes the block list there, and also stop hogging the FS
lock while we compute the block list, as there is no need for it.
2021-02-26 18:14:02 +01:00
Andreas Kling
c09921b9be Ext2FS: Don't hog FS lock while reading/writing inodes
There are two locks in the Ext2FS implementation:

* The FS lock (Ext2FS::m_lock)
  This governs access to the superblock, block group descriptors,
  and the block & inode bitmap blocks. It's held while allocating
  or freeing blocks/inodes.

* The inode lock (Ext2FSInode::m_lock)
  This governs access to the inode metadata, including the block
  list, and to the content data as well. It's held while doing
  basically anything with the inode.

Once an on-disk block/inode is allocated, it logically belongs
to the in-memory Inode object, so there's no need for the FS lock
to be taken while manipulating them, the inode lock is all you need.

This dramatically reduces the impact of disk I/O on path resolution
and various other things that look at individual inodes.
2021-02-26 17:57:38 +01:00
Andreas Kling
c7c63727bf Ext2FS: Remove unnecessary locking in find_block_containing_inode()
This is just a bunch of index math based on immutable values in the
super block and block group descriptor. No need to lock here!
2021-02-26 17:24:39 +01:00
Andreas Kling
81e3ea29c3 Ext2FS: Remove unnecessary lock in Ext2FS::write_ext2_node()
Now that writing to the underlying storage is serialized, we don't
need to take the FS lock when writing out an inode struct.
2021-02-26 17:23:46 +01:00
Andreas Kling
65e083ed36 Revert "Ext2FS: Don't reload already-cached block list when freeing inode"
This reverts commit 1e737a5c50.

The cached block list does not include meta-blocks, so we'd end up
leaking those. There's definitely a nice way to avoid work here, but it
turns out it wasn't quite this trivial. Reverting for now.
2021-02-26 14:57:00 +01:00
Andreas Kling
1e737a5c50 Ext2FS: Don't reload already-cached block list when freeing inode
If we already have a cached copy of the inode's block list, we can use
that to free the blocks. No need to reload the list.
2021-02-26 14:05:18 +01:00
Andreas Kling
1f9409a658 Ext2FS: Inode allocation improvements
This patch combines inode the scan for an available inode with the
updating of the bit in the inode bitmap into a single operation.

We also exit the scan immediately when we find an inode, instead of
continuing until we've scanned all the eligible groups(!)

Finally, we stop holding the filesystem lock throughout the entire
operation, and instead only take it while actually necessary
(during inode allocation, flush, and inode cache update.)
2021-02-26 14:05:18 +01:00
Andreas Kling
19083fd760 Ext2FS: Propagate errors from more places
Improve a bunch of situations where we'd previously panic the kernel
on failure. We now propagate whatever error we had instead. Usually
that'll be EIO.
2021-02-26 14:05:18 +01:00
Andreas Kling
6352b4fd74 Ext2FS: Share some bitmap code between inode and block allocation
Both inode and block allocation operate on bitmap blocks and update
counters in the superblock and group descriptor.

Since we're here, also add some error propagation around this code.
2021-02-26 14:05:18 +01:00
Andreas Kling
5d180d1f99 Everywhere: Rename ASSERT => VERIFY
(...and ASSERT_NOT_REACHED => VERIFY_NOT_REACHED)

Since all of these checks are done in release builds as well,
let's rename them to VERIFY to prevent confusion, as everyone is
used to assertions being compiled out in release.

We can introduce a new ASSERT macro that is specifically for debug
checks, but I'm doing this wholesale conversion first since we've
accumulated thousands of these already, and it's not immediately
obvious which ones are suitable for ASSERT.
2021-02-23 20:56:54 +01:00
Brian Gianforcaro
cbd8f78cce Kernel: Use uniform initialization instead of memset for a few stack buffer.
Raw memset is relatively easy to mess up, avoid it when there are
better alternatives provided by the compiler in modern C++.
2021-02-21 11:52:47 +01:00
Brian Gianforcaro
7c950c2d01 Kernel: Use ByteBuffer::zero_fill() instead of raw memset in Ext2
There was a typo in one of the memsets, use the type safe wrapper instead.

Fix EXt
2021-02-21 11:52:47 +01:00
Jean-Baptiste Boric
9ce0639383 Kernel: Use divide_rounded_up inside write_block_list_for_inode 2021-02-13 19:56:49 +01:00
Jean-Baptiste Boric
869b33d6dd Kernel: Support triply indirect blocks for BlockListShape computation 2021-02-13 19:56:49 +01:00
Andreas Kling
c62c00e7db Ext2FS: Make Ext2FS::GroupIndex a distinct integer type 2021-02-12 13:33:58 +01:00
Andreas Kling
489317e573 Kernel: Make BlockBasedFS::BlockIndex a distinct integer type 2021-02-12 11:59:27 +01:00
Andreas Kling
e44c1792a7 Kernel: Add distinct InodeIndex type
Use the DistinctNumeric mechanism to make InodeIndex a strongly typed
integer type.
2021-02-12 10:26:29 +01:00
Andreas Kling
95064f8b58 Ext2FS: Convert #if EXT2_DEBUG => dbgln_if() and constexpr-if 2021-02-11 23:05:16 +01:00
Andreas Kling
a280cdf9ba Ext2FS: Shrink Ext2FSDirectoryEntry from 16 to 12 bytes
The way we read/write directories is very inefficient, and this doesn't
solve any of that. It does however reduce memory usage of directory
entry vectors by 25% which has nice immediate benefits.
2021-02-11 22:45:50 +01:00
Andreas Kling
4c0707e56c Kernel: Don't create a zero-length VLA in Ext2FS block list walk
Found by KUBSAN :^)
2021-02-05 21:23:11 +01:00
Andreas Kling
d4f40241f1 Ext2FS: Avoid unnecessary parent inode lookup during inode creation
Creation of new inodes is always driven by the parent inode, so we can
just refer directly to it instead of looking up the parent by ID.
2021-02-02 18:58:26 +01:00
Andreas Kling
9e4dd834ab Ext2FS: Simplify inode creation by always starting empty
We had two ways of creating a new Ext2FS inode. Either they were empty,
or they started with some pre-allocated size.

In practice, the pre-sizing code path was only used for new directories
and it didn't actually improve anything as far as I can tell.

This patch simplifies inode creation by simply always allocating empty
inodes. Block allocation and block list generation now always happens
on the same code path.
2021-02-02 18:58:26 +01:00
Andreas Kling
dbb668ddd3 Ext2FS: Propagate error codes from write_directory() 2021-02-02 18:58:26 +01:00
Peter Elliott
c0e88b9710 Kernel: Add FIBMAP ioctl to Ext2FileSystem
FIBMAP is a linux ioctl that gives the location on disk of a specific
block of a file
2021-01-30 22:54:51 +01:00
asynts
7cf0c7cc0d Meta: Split debug defines into multiple headers.
The following script was used to make these changes:

    #!/bin/bash
    set -e

    tmp=$(mktemp -d)

    echo "tmp=$tmp"

    find Kernel \( -name '*.cpp' -o -name '*.h' \) | sort > $tmp/Kernel.files
    find . \( -path ./Toolchain -prune -o -path ./Build -prune -o -path ./Kernel -prune \) -o \( -name '*.cpp' -o -name '*.h' \) -print | sort > $tmp/EverythingExceptKernel.files

    cat $tmp/Kernel.files | xargs grep -Eho '[A-Z0-9_]+_DEBUG' | sort | uniq > $tmp/Kernel.macros
    cat $tmp/EverythingExceptKernel.files | xargs grep -Eho '[A-Z0-9_]+_DEBUG' | sort | uniq > $tmp/EverythingExceptKernel.macros

    comm -23 $tmp/Kernel.macros $tmp/EverythingExceptKernel.macros > $tmp/Kernel.unique
    comm -1 $tmp/Kernel.macros $tmp/EverythingExceptKernel.macros > $tmp/EverythingExceptKernel.unique

    cat $tmp/Kernel.unique | awk '{ print "#cmakedefine01 "$1 }' > $tmp/Kernel.header
    cat $tmp/EverythingExceptKernel.unique | awk '{ print "#cmakedefine01 "$1 }' > $tmp/EverythingExceptKernel.header

    for macro in $(cat $tmp/Kernel.unique)
    do
        cat $tmp/Kernel.files | xargs grep -l $macro >> $tmp/Kernel.new-includes ||:
    done
    cat $tmp/Kernel.new-includes | sort > $tmp/Kernel.new-includes.sorted

    for macro in $(cat $tmp/EverythingExceptKernel.unique)
    do
        cat $tmp/Kernel.files | xargs grep -l $macro >> $tmp/Kernel.old-includes ||:
    done
    cat $tmp/Kernel.old-includes | sort > $tmp/Kernel.old-includes.sorted

    comm -23 $tmp/Kernel.new-includes.sorted $tmp/Kernel.old-includes.sorted > $tmp/Kernel.includes.new
    comm -13 $tmp/Kernel.new-includes.sorted $tmp/Kernel.old-includes.sorted > $tmp/Kernel.includes.old
    comm -12 $tmp/Kernel.new-includes.sorted $tmp/Kernel.old-includes.sorted > $tmp/Kernel.includes.mixed

    for file in $(cat $tmp/Kernel.includes.new)
    do
        sed -i -E 's/#include <AK\/Debug\.h>/#include <Kernel\/Debug\.h>/' $file
    done

    for file in $(cat $tmp/Kernel.includes.mixed)
    do
        echo "mixed include in $file, requires manual editing."
    done
2021-01-26 21:20:00 +01:00
asynts
eea72b9b5c Everywhere: Hook up remaining debug macros to Debug.h. 2021-01-25 09:47:36 +01:00
asynts
acdcf59a33 Everywhere: Remove unnecessary debug comments.
It would be tempting to uncomment these statements, but that won't work
with the new changes.

This was done with the following commands:

    find . \( -name '*.cpp' -o -name '*.h' -o -name '*.in' \) -not -path './Toolchain/*' -not -path './Build/*' -exec awk -i inplace '$0 !~ /\/\/#define/ { if (!toggle) { print; } else { toggle = !toggle } } ; $0 ~/\/\/#define/ { toggle = 1 }' {} \;

    find . \( -name '*.cpp' -o -name '*.h' -o -name '*.in' \) -not -path './Toolchain/*' -not -path './Build/*' -exec awk -i inplace '$0 !~ /\/\/ #define/ { if (!toggle) { print; } else { toggle = !toggle } } ; $0 ~/\/\/ #define/ { toggle = 1 }' {} \;
2021-01-25 09:47:36 +01:00
Andreas Kling
bfb254ed14 Ext2FS: Assert that create_directory() is called with valid mode 2021-01-23 16:45:05 +01:00
Andreas Kling
f2ea6c3d4c Ext2FS: Don't create a directory when asked to create a socket file
(mode & S_IFDIR) is not enough to check if "mode" is a directory,
we have to check all the bits in the S_IFMT mask.

Use the is_directory() helper to fix this bug.
2021-01-23 16:45:05 +01:00
Andreas Kling
1f53dd0943 Ext2FS: Propagate I/O errors from Ext2FSInode::write_bytes() 2021-01-21 00:14:56 +01:00
Andreas Kling
19d3f8cab7 Kernel+LibC: Turn errno codes into a strongly typed enum
..and allow implicit creation of KResult and KResultOr from ErrnoCode.
This means that kernel functions that return those types can finally
do "return EINVAL;" and it will just work.

There's a handful of functions that still deal with signed integers
that should be converted to return KResults.
2021-01-20 23:20:02 +01:00
Andreas Kling
e279b45aed Kernel: Make BlockBasedFS read/write functions return a KResult
This way, if something goes wrong, we get to keep the actual error.
Also, KResults are nodiscard, so we have to deal with that in Ext2FS
instead of just silently ignoring I/O errors(!)
2021-01-20 22:57:36 +01:00
Andreas Kling
6613cef2f8 Ext2FS: Update block group directory count after directory removal
When freeing an inode, we were checking if it's a directory *after*
wiping the inode metadata. This caused us to forget updating the block
group descriptor with the new directory count.
2021-01-17 16:56:07 +01:00
Andreas Kling
77656aed8e Ext2FS: Zero out new space when growing an inode
Before this change, truncating an Ext2FS inode to a larger size than it
was before would give you uninitialized on-disk data.

Fix this by zeroing out all the new space when doing an inode resize.

This is pretty naively implemented via Inode::write_bytes() and there's
lots of room for cleverness here in the future.
2021-01-09 22:08:53 +01:00
Andreas Kling
f0093e5d59 Ext2FS: Convert dbg() to dbgln()
Also remove some dbg()'s that were printing incorrect information.
2021-01-09 21:54:27 +01:00
asynts
938e5c7719 Everywhere: Replace a bundle of dbg with dbgln.
These changes are arbitrarily divided into multiple commits to make it
easier to find potentially introduced bugs with git bisect.Everything:

The modifications in this commit were automatically made using the
following command:

    find . -name '*.cpp' -exec sed -i -E 's/dbg\(\) << ("[^"{]*");/dbgln\(\1\);/' {} \;
2021-01-09 21:11:09 +01:00
Tom
ae956edf6e Kernel: Improve some low-memory situations with ext2 2021-01-01 23:43:44 +01:00
Andreas Kling
43d9fe15f9 Revert "Kernel: Convert read_block method to get a reference instead of pointer"
This reverts commit 092a13211a.

Fixes #4611.
2020-12-29 00:06:52 +01:00
Liav A
092a13211a Kernel: Convert read_block method to get a reference instead of pointer
BlockBasedFileSystem::read_block method should get a reference of
a UserOrKernelBuffer.

If we need to force caching a block, we will call other method to do so.
2020-12-27 23:07:44 +01:00
Nathan Lanza
d1891f67ac
AK: Use direct-list-initialization for Vector::empend() (#4564)
clang trunk with -std=c++20 doesn't seem to properly look for an
aggregate initializer here when the type being constructed is a simple
aggregate (e.g. `struct Thing { int a; int b; };`). This template fails
to compile in a usage added 12/16/2020 in `AK/Trie.h`.

Both forms of initialization are supposed to call the
aggregate-initializers but direct-list-initialization delegating to
aggregate initializers is a new addition in c++20 that might not be
implemented yet.
2020-12-27 23:06:37 +01:00
Andreas Kling
8e79bde2b7 Kernel: Move KBufferBuilder to the fallible KBuffer API
KBufferBuilder::build() now returns an OwnPtr<KBuffer> and can fail.
Clients of the API have been updated to handle that situation.
2020-12-18 19:22:26 +01:00
Andreas Kling
47da86d136 Ext2FS: Fail the mount if BGD table cache allocation fails
Instead of asserting if we can't allocate enough memory for a BGD table
cache, just fail the mount instead.
2020-12-18 19:22:26 +01:00
Andreas Kling
76308c2e1f Kernel: Reduce ByteBuffer thrashing in inode block list generation
Instead of creating and destroying a new ByteBuffer for every block we
process during block list generation, just use stack memory instead.
2020-11-24 21:29:08 +01:00
Andreas Kling
3e3a72f2a2 Ext2FS: Oops, fix forgotten assignment in Ext2FSInode::resize()
If the inode's block list cache is empty, we forgot to assign the
result of computing the block list. The fact that this worked anyway
makes me wonder when we actually don't have a cache..

Thanks to szyszkienty for spotting this! :^)
2020-11-24 16:16:09 +01:00
Andreas Kling
20205708b9 Ext2FS: Use cached inode block list in resize() if available
If we have already cached the block list of an Ext2FSInode, we can save
a lot of time by not regenerating it.
2020-11-24 13:40:45 +01:00
Andreas Kling
bb9c705fc2 Ext2FS: Move some EXT2_DEBUG logging behind EXT2_VERY_DEBUG
This makes the build actually somewhat usable with EXT2_DEBUG. :^)
2020-11-23 16:08:42 +01:00
Andreas Kling
df758a5a51 Ext2FS: Clear out the direct block list when an inode is resized to 0
e2fsck was complaining about blocks being allocated in an inode's list
of direct blocks while at the same time being free in the block bitmap.

It was easy to reproduce by creating a file with non-zero length and
then truncating it. This fixes the issue by clearing out the direct
block list when resizing a file to 0.
2020-11-23 14:08:50 +01:00