Commit graph

14237 commits

Author SHA1 Message Date
AnotherTest
4c343c5f26 AK: Fix OOB access in DuplexMemoryStream::offset_of()
This fixes an OOB access when the last read/written chunk is empty (as we _just_
started on a new chunk).
Also adds a test case to TestMemoryStream.
Found via human fuzzing in the shell:
```sh
for $(cat /dev/urandom) {
    clear
    match $it {
        ?* as (x) {
            echo $x
            sleep 1
        }
    }
}
```
would assert at some point.
2020-11-17 17:07:39 +01:00
ry755
b1fb8e3741 LibM: Define some floating point classification macros
This adds a few macros used to determine the category of a floating
point number. This fixes a build error with the jq port due to the
previously missing isnormal() macro.

Co-authored-by: Lua MacDougall <luawhat@gmail.com>
2020-11-17 09:57:06 +01:00
Zac
7ef8835e5a
ClipboardHistoryModel: Prevent duplicate rows (#4073)
Prevents the adding of items to the ClipboardHistoryModel if the raw
data and mime_type of the item being added is the same as another item
already in the list.
2020-11-17 09:50:39 +01:00
Linus Groh
d6a4c0c79e AK: Trim whitespace in StringUtils::convert_to_{int,uint,uint_from_hex}()
Personally I found this unintuitive at first, but it is in line with
strtol(), Python's int() or JavaScript's parseInt(), so I guess it makes
sense.

Fixes #4097.
2020-11-17 09:48:35 +01:00
Lenny Maiorani
0cb16ffe08 Shell: Remove unused private member and includes
Problem:
- Clang reports unused private member warning in the `Shell::Formatter`.
- Vector is not used in the `Shell::Formatter`.

Solution:
- Remove unused private member variable.
- Remove unused includes.
2020-11-17 09:48:02 +01:00
Lenny Maiorani
fd97f23cef MACAddress: Unit testing for basic functionality
Problem:
- There are no unit tests for `MACAddress` class. This makes it
  difficult to refactor and ensure the same behavior.
- `m_data` private member variable is uninitialized leading to undefined
  behavior of `is_zero()`.

Solution:
- Add unit tests to cover basic functionality.
- Initialize `m_data`.
2020-11-17 09:47:50 +01:00
Spencer Dixon
f23d9a73aa Userland: Add -v verbose flag to 'rm' 2020-11-17 09:40:03 +01:00
Spencer Dixon
2dab9d4bac Userland: Add -v verbose flag to 'mv' 2020-11-17 09:40:03 +01:00
Spencer Dixon
7ba28b5b0b Userland: Add -v verbose flag to 'cp' 2020-11-17 09:40:03 +01:00
Andreas Kling
fe79b9ff94 LibC: Notify UE at the start of free() instead of at the end
This way, if we end up deallocating an entire ChunkedBlock, UE doesn't
get confused thinking the freed pointer has never been allocated.
2020-11-16 15:11:02 +01:00
Andreas Kling
1965fc5b98 UserspaceEmulator: Keep Emulator& closer to the action in some places
This avoids the cost of calling Emulator::the() in some very hot paths.
2020-11-16 15:11:02 +01:00
Andreas Kling
d14695f823 UserspaceEmulator: Cache the region we're executing code from
Instead of caching a raw pointer to the next instruction, cache the
region we're fetching instructions from, and a pointer to its base.
This way we don't need to keep invalidating and reloading the cache
whenever the CPU jumps.
2020-11-16 15:11:02 +01:00
Andreas Kling
b4ff85f138 UserspaceEmulator: Reduce malloc thrashing in backtrace capture 2020-11-16 15:11:02 +01:00
Andreas Kling
e1f617950e UserspaceEmulator: Make big malloc block lookup O(1) as well
By passing the Region& to the auditing functions, we know exactly which
block we are hitting. This allows us to track big mallocations the same
way we already do chunked ones.

This gets rid of the O(n) scan in find_mallocation() for allocations
larger than the maximum malloc chunk size. :^)
2020-11-16 15:11:02 +01:00
Andreas Kling
8d9dd4c518 UserspaceEmulator: Make Region a top-level class 2020-11-16 15:11:02 +01:00
AnotherTest
de4061ff94 LibTLS: Count the mac size towards the packet length in CBC mode
This is a regression introduced in 1172746, where the padding would be
done without accounting for the added MAC bytes.
Fixes #4098.
2020-11-16 13:21:18 +01:00
Lenny Maiorani
2a06b026ef Vector: C++20 equality operators
Problem:
- C++20 changes the way equality operators are generated. This results
  in overload ambiguity as reported by clang.

Solution:
- Remove `AK::Vector::operator!=` because it will be automatically
  generated in terms of `AK::Vector::operator==`.
- Change `AK::Vector::operator==` to be a function template so that
  overload resolution is not confused about `a == b` vs `b == a`.
- Add tests to ensure the behavior works.

Notes:
- There is more info available at
  https://brevzin.github.io/c++/2019/07/28/comparisons-cpp20/ for
  deeper discussion about overload resolution, operator rewriting, and
  generated functions.
2020-11-16 10:06:23 +01:00
Andreas Kling
3c64cec4d7 UserspaceEmulator: Devirtualize read/write/execute region permissions
These are getting quite hot (~4% of general emulation profile combined)
so let's just devirtualize them and turn the function calls into simple
boolean checks.
2020-11-16 09:44:30 +01:00
Andreas Kling
f41b9946e2 UserspaceEmulator: Hang malloc metadata on malloc block MmapRegions
Instead of tracking known malloc blocks in a separate hash table,
add an optional malloc metadata pointer to MmapRegion.

This makes finding the malloc metadata for a given pointer extremely
fast since it can piggyback on the page table array. :^)
2020-11-16 09:33:30 +01:00
Andreas Kling
395313039d LibDebug: Use FlyString for file paths
The same file path occurs over and over in debug info, so let's store
them as FlyString to ensure we only have one of each in memory.
2020-11-16 09:10:49 +01:00
Andreas Kling
4eb3cf68b7 LibDebug: Avoid creating the String("serenity/") a bazillion times
We were creating and destroying this string twice for every LineInfo.
That's a lot of malloc() and free() during UE startup.
2020-11-16 09:10:49 +01:00
Andreas Kling
fe7036d8f4 UserspaceEmulator: Improve error text on jump to non-executable memory
The memory is non-executable, not non-readable. :^)
2020-11-16 09:10:49 +01:00
Nico Weber
6252e5b1f6 UserspaceEmulator: Implement 32-bit FIDIV, 16-bit FIADD/FISUB/FISUBR/FIDIV/FIDIVR/FISTP, and fix 32-bit FIADD
Not motivated by anything in particular, they just looked easy to fill
in. With this, all arithmetic FI* FPU instructions are implemented.

Switch to the mXXint style in a few more functions, this part is no-op.
2020-11-15 20:16:55 +01:00
Andreas Kling
6dab0af9af UserspaceEmulator: Add a fast path for forward REP STOSD
Same as REP STOSB, except for 32-bit fills instead.
2020-11-15 18:09:15 +01:00
Andreas Kling
102e1d330c UserspaceEmulator: Add a fast path for forward REP STOSB
This is used by memset() so we get a lot of mileage out of optimizing
this instruction.

Note that we currently audit every individual byte accessed separately.
This could be greatly improved by adding a range auditing mechanism to
MallocTracer.
2020-11-15 18:09:08 +01:00
Andreas Kling
92e152f11d UserspaceEmulator: Add virtual data()/shadow_data() accessors to Region 2020-11-15 18:09:04 +01:00
Andreas Kling
59b4874443 UserspaceEmulator: Add a page-address-to-MMU-region lookup map
To make SoftMMU::find_region() O(1), this patch invests 3MiB into a
lookup table where we track each possible page base address and map
them to the SoftMMU::Region corresponding to that address.

This is another large improvement to general emulation performance. :^)
2020-11-15 18:08:59 +01:00
Andreas Kling
a4a389156d UserspaceEmulator: Make sure the (crappy) VM allocator is page-aligned
We don't want the next_address pointer losing its alignment somehow.
This whole thing should be replaced at some point, since UE hosted
programs won't be able to run forever with this allocation strategy.
2020-11-15 18:08:56 +01:00
Linus Groh
df9fe8fa7b Base: Add filetype-json icons 2020-11-15 16:49:40 +01:00
Linus Groh
6b9ff8d6e2 LibGUI/FileIconProvider: Add config file and use patterns
This moves file extension to icon mappings from compile time macros to an
INI config file (/etc/FileIconProvider.ini), so file icons can easily be
customized and extended :^)

I also switched the format from a static file extension (".foo") to
glob-like patterns ("*.foo", using StringUtils::matches()), which allows
us to assign icons to specific exactly matching file names, like many
IDEs do - e.g. "CMakeLists.txt" or ".prettierrc".
2020-11-15 16:49:40 +01:00
Andreas Kling
42bc518d70 LibDebug: Oops, file:line info should have '/' between directory & file 2020-11-15 15:12:13 +01:00
Andreas Kling
485d28298a LibDebug: Use StringBuilder in append_to_line_info()
This avoids a lot of temporary allocations and speeds up launching
UE on a large executable by quite a bit.
2020-11-15 13:12:07 +01:00
Andreas Kling
adabcf24ec Everywhere: Add missing <AK/ByteBuffer.h> includes
All of these files were getting ByteBuffer.h from someone else and then
using it. Let's include it explicitly.
2020-11-15 13:11:21 +01:00
Andreas Kling
9ffdb71111 LibGfx: Don't leak malloc memory in cloned fonts
Fixes #4083.
2020-11-15 11:35:07 +01:00
Brendan Coles
b31b666301 LibC: Ensure mkstemp generates a pattern 6 characters in length 2020-11-15 09:38:35 +01:00
Nico Weber
be73f9f544 UserspaceEmulator: Implement FISUBR_RM32
Fixes crash when playing `ue Breakout` with the mouse.
2020-11-15 09:36:37 +01:00
Brendan Coles
d739483ee8 Userland: Tests: Use mkstemp temporary files in tests 2020-11-15 00:50:57 +01:00
Andreas Kling
12d923bb7e UserspaceEmulator: Fix some FPU instructions' handling of RM32/RM64
m32int is a 32-bit integer stored in memory, and should not be mistaken
for a floating point number. :^)

Also add missing handling of 64-bit FPU register operands to some of
the RM64 instructions.
2020-11-14 23:47:50 +01:00
Andreas Kling
647e92b74f UserspaceEmulator: Skip destroying the Emulator object on shutdown
There are some destruction order races that can cause hangs while
shutting down UE. Since there's no particular value right now in
destroying the Emulator object properly, just avoid destruction and
add a FIXME about looking into it later.
2020-11-14 23:39:44 +01:00
Andreas Kling
b5b535aa81 UserspaceEmulator: Untaint flags in FCOMI and FUCOMI for now
This makes UE logging bearable until we can get proper shadow data
support for the FPU stack.
2020-11-14 23:11:07 +01:00
Andreas Kling
8fd97bee7f UserspaceEmulator: Forget ChunkedBlocks after they are munmap()'ed
This is not ideal since we lose free() backtraces, but it will require
some thinking to get this right.
2020-11-14 23:07:07 +01:00
Andreas Kling
2066f48b87 UserspaceEmulator: Avoid one hash lookup in target_did_malloc() 2020-11-14 22:52:07 +01:00
Andreas Kling
d88b36448b UserspaceEmulator: Add some helpers to tidy up TrackedChunkedBlock 2020-11-14 22:52:04 +01:00
Andreas Kling
2fceffff6f UserspaceEmulator: Track malloc ChunkedBlocks for faster auditing
Instead of doing an O(n) scan over all the mallocations whenever we're
doing a read/write audit, UE now keeps track of ChunkedBlocks and their
chunks. Both the block lookup and the chunk lookup is O(1).

We know what ChunkedBlocks look like via mallocdefs.h from LibC.

Note that the old linear scan is still in use for big mallocations,
but the vast majority of mallocations are chunked, so this helps a lot.

This makes malloc auditing significantly faster! :^)
2020-11-14 22:51:58 +01:00
Andreas Kling
b494cfea38 LibC: Move some of malloc's data structures into mallocdefs.h
This allows UE to see what the heap metadata looks like.
2020-11-14 22:51:49 +01:00
Tom
53cffb5ad9 Kernel: Fix mouse lag when VMWareBackdoor absolute mode is enabled
We won't be receiving full PS/2 mouse packets when the VMWareBackdoor
absolute mouse mode is enabled. So, read just one byte every time
and retrieve the latest mouse packet from VMWareBackdoor immediately.

Fixes #4086
2020-11-14 22:09:48 +01:00
Tom
13383f3267 Revert "Kernel: Keep reading from i8042 until the buffer is empty"
This reverts commit 467f6c74a4.
2020-11-14 22:09:48 +01:00
Andreas Kling
467f6c74a4 Kernel: Keep reading from i8042 until the buffer is empty
Otherwise we might not drain the mouse buffer until the next IRQ.
2020-11-14 17:20:17 +01:00
Andreas Kling
a5982f8605 AK: Mark SimpleIterator::operator*() as ALWAYS_INLINE
This gives the compiler enough information to optimize index validation
when using range-for to iterate over a Vector, drastically reducing the
cost of such loops.
2020-11-14 17:20:17 +01:00
Ben Wiederhake
c4933f9bd7 Meta: Fix IRC notification script
What a silly mistake. How did I manage to do that?
2020-11-14 17:14:45 +01:00