Commit graph

783 commits

Author SHA1 Message Date
Zaggy1024
074f771b59 LibVideo: Add VideoFrame class for decoded video frames
The class is virtual and has one subclass, SubsampledYUVFrame, which
is used by the VP9 decoder to return a single frame. The
output_to_bitmap(Bitmap&) function can be used to set pixels on an
existing bitmap of the correct size to the RGB values that
should be displayed. The to_bitmap() function will allocate a new bitmap
and fill it using output_to_bitmap.

This new class also implements bilinear scaling of the subsampled U and
V planes so that subsampled videos' colors will appear smoother.
2022-10-31 14:47:13 +01:00
kleines Filmröllchen
2477409680 Tests: Add pthread scheduler priority tests 2022-10-27 11:30:19 +01:00
Liav A
ec96aee8e3 Tests: Use new global variables at /sys/kernel/ directory 2022-10-25 15:33:34 -06:00
demostanis
7c33f8f7df AK: Add SplitBehavior::KeepTrailingSeparator with tests 2022-10-24 23:29:18 +01:00
demostanis
3e8b5ac920 AK+Everywhere: Turn bool keep_empty to an enum in split* functions 2022-10-24 23:29:18 +01:00
davidot
c9aa664eb0 AK: Make the JsonParser use the new double parser for numbers
Because we still support u64 and i64 (on top of i32 and u32) we do still
have to parse the number ourself first. Then if we determine that the
number is a floating point or is outside of the range of i64 and u64 we
fallback and parse it as a double.

Before JsonParser had ifdefs guarding the double computation, but it
just build when we error on ifdef KERNEL so JsonParser is no longer
usable in the Kernel. This can be remedied fairly easily but since
it is not needed we #error on that for now.
2022-10-23 15:48:45 +02:00
davidot
35e52f7bfd LibC+Tests: Add extra tests for special values for strtod 2022-10-23 15:48:45 +02:00
davidot
2334cd85a2 AK: Add an exact and fast hex float parsing algorithm
Similar to decimal floating point parsing the current strtod hex float
parsing gives a lot of incorrect results. We can use a similar technique
as with decimal parsing however hex floats are much simpler as we don't
need to scale with a power of 5.

For hex floats we just provide the parse_first_hexfloat API as there is
currently no need for a parse_hexfloat_completely API.

Again the accepted input for parse_first_hexfloat is very lenient and
any validation should be done before calling this method.
2022-10-23 15:48:45 +02:00
davidot
53b7f5e6a1 AK: Add an exact and fast floating point parsing algorithm
This is based on the paper by Daniel Lemire called
"Number parsing at a Gigabyte per second", currently available at
https://arxiv.org/abs/2101.11408
An implementation can be found at
https://github.com/fastfloat/fast_float

To support both strtod like methods and String::to_double we have two
different APIs. The parse_first_floating_point gives back both the
result, next character to read and the error/out of range status.
Out of range here means we rounded to infinity 0.

The other API, parse_floating_point_completely, will return a floating
point only if the given character range contains just the floating point
and nothing else. This can be much faster as we can skip actually
computing the value if we notice we did not parse the whole range.

Both of these APIs support a very lenient format to be usable in as many
places as possible. Also it does not check for "named" values like
"nan", "inf", "NAN" etc. Because this can be different for every usage.

For integers and small values this new method is not faster and often
even a tiny bit slower than the current strtod implementation. However
the strtod implementation is wrong for a lot of values and has a much
less predictable running time.

For correctness this method was tested against known string -> double
datasets from https://github.com/nigeltao/parse-number-fxx-test-data
This method gives 100% accuracy.
The old strtod gave an incorrect value in over 50% of the numbers
tested.
2022-10-23 15:48:45 +02:00
davidot
bf6d4a5cbf AK: Make truncating UFixedBigInts constexpr
Also add some tests and shift tests while we're at it.
2022-10-23 15:48:45 +02:00
Timothy Flynn
b9dc0b7d1b AK: Do not append string bytes as code points when title-casing a string
By appending individual bytes as code points, we were "breaking apart"
multi-byte UTF-8 code points. This now behaves the same way as the
invert_case() helper in StringUtils.
2022-10-20 18:55:43 +02:00
Jelle Raaijmakers
bdb2be8f9e LibGL: Remove context initialization from tests
We are either not using these defaults or they are already the
`GLContext`'s defaults to begin with.
2022-10-19 22:22:58 +02:00
Timothy Flynn
27737f613c LibTimeZone+LibJS: Update to TZDB version 2022e
https://mm.icann.org/pipermail/tz-announce/2022-October/000074.html

This version changes America/Chicago's transtion from LMT to CST from
1883 Nov 18 12:09:24 to 1883 Nov 18 18:00.
2022-10-18 16:01:44 +02:00
Timothy Flynn
ed612d835d LibTimeZone: Default to UTC if parsing the TZ environment variable fails
Commit c3fd455 changed LibTimeZone to fall back to the system time zone
when we fail to parse the TZ environment variable. This behavior differs
from both our LibC and glibc; they abort parsing and default to UTC.

This changes LibTimeZone to behave the same way to avoid a very awkward
situation where some parts of the codebase thinks the timezone is UTC,
and others think the timezone is whatever /etc/timezone indicates.
2022-10-17 21:50:55 +02:00
Jelle Raaijmakers
0cf3cb6279 LibGL: Immediately dereference vertex attribute data in display lists
According to the spec, pointers to client data need to be dereferenced
immediately when adding calls such as `glDrawElements` or
`glArrayElement` to a display list. We were trying to support display
lists for these calls but since they only invoke _other_ calls that also
support display lists, we can simply defer the display list
functionality to them.

This fixes the rendering of the ClassiCube port by cflip.
2022-10-16 21:12:15 +02:00
Liav A
b9dca3300e Kernel: Use more fine-grained content data block granularity in TmpFS
Instead of just having a giant KBuffer that is not resizeable easily, we
use multiple AnonymousVMObjects in one Vector to store them.
The idea is to not have to do giant memcpy or memset each time we need
to allocate or de-allocate memory for TmpFS inodes, but instead, we can
allocate only the desired block range when trying to write to it.
Therefore, it is also possible to have data holes in the inode content
in case of skipping an entire set of one data block or more when writing
to the inode content, thus, making memory usage much more efficient.

To ensure we don't run out of virtual memory range, don't allocate a
Region in advance to each TmpFSInode, but instead try to allocate a
Region on IO operation, and then use that Region to map the VMObjects
in IO loop.
2022-10-16 17:46:40 +02:00
Timothy Flynn
1524288127 LibSQL: Rewrite the SQL::Value type to be contained within one class
Currently, the Value class is essentially a "pImpl" wrapper around the
ValueImpl hierarchy of classes. This is a bit difficult to follow and
reason about, as methods jump between the Value class and its impl
classes.

This changes the Variant held by Value to instead store the specified
types (String, int, etc.) directly. In doing so, the ValueImpl classes
are removed, and all methods are now just concise Variant visitors.

As part of this rewrite, support for the "array" type is dropped (or
rather, just not re-implemented) as it was unused. If it's needed in the
future, support can be re-added.

This does retain the ability for non-NULL types to store NULL values
(i.e. an empty Optional). I tried dropping this support as well, but it
is depended upon by the on-disk storage classes in non-trivial ways.
2022-10-14 17:47:44 +03:30
Timothy Flynn
7d41b46a7d LibSQL: Remove infallible type conversions from SQL::Value
Force the callers to either know that the type is convertible, or to
handle the conversion failure.
2022-10-14 17:47:44 +03:30
Gunnar Beutner
808c43312d Tests+Userland: Implement AARCH64 support for some inline assembly blobs 2022-10-14 13:01:13 +02:00
Gunnar Beutner
dadf656dc9 Tests+Userland: Prefer using __builtin_trap() instead of UD2
This way we don't have to hard-code per-architecture instructions.
2022-10-14 13:01:13 +02:00
Gunnar Beutner
a650c74b27 AK+Toolchain: Make char and wchar_t behave on AARCH64
By default char and wchar_t are unsigned on AARCH64. This fixes a
bunch of related compiler errors.
2022-10-14 13:01:13 +02:00
Gunnar Beutner
31bd5b1a02 AK+Userland: Stub out code that isn't currently implemented on AARCH64
Even though this almost certainly wouldn't run properly even if we had
a working kernel for AARCH64 this at least lets us build all the
userland binaries.
2022-10-14 13:01:13 +02:00
davidot
340a2a96a4 test262-runner: Use names for the different exit codes
These are not used by test-test262 but can be used to quickly
distinguish the type of problem if the runner fails when running
manually.
2022-10-12 12:00:21 -06:00
davidot
1f54259922 test262-runner: Port to Core::Stream 2022-10-12 12:00:21 -06:00
davidot
302593ded5 test-test262: Port to Core::Stream and use TRY more
The only complication here is that Core::Stream::File is not RefCounted
meaning we have to use OwnPtr instead of RefPtr.
Unfortunately we cannot propagate errors as some errors must be caught
and dealt with as the runner can do anything (like stop at any moment
or close pipes).
2022-10-12 12:00:21 -06:00
davidot
49e3b387ac test-test262: Close the output file stream after writing
Without this the runner is waiting for new tests which will never come
and test-test262 is waiting for output which never comes since the
runner is blocked.
Also finish off a comment, and make the variables follow serenity style.
2022-10-12 12:00:21 -06:00
Zaggy1024
41cb705b47 LibVideo: Allow the VP9 decoder to decode ultra high resolution video
Previously, some integer overflows and truncations were causing parsing
errors for 4K videos, with those fixed it can fully decode 8K video.

This adds a test to ensure that 4K video will continue to be decoded.

Note: There seems to be unexpectedly high memory usage while decoding
them, causing 8K video to require more than a gigabyte of RAM. (!!!)
2022-10-12 00:54:31 -06:00
Tommy Murphy
0ee98c69c1 LibCompress: Brotli support metadata of skip_length=0
The relevant RFC section from
https://www.rfc-editor.org/rfc/rfc7932#section-9.2

MSKIPBYTES * 8 bits: MSKIPLEN - 1, where MSKIPLEN is
       the number of metadata bytes; this field is
       only present if MSKIPBYTES is positive;
       otherwise, MSKIPLEN is 0 (if MSKIPBYTES is
       greater than 1, and the last byte is all
       zeros, then the stream should be rejected as
       invalid)

So when skip_bytes is zero we need to break and
re-align bytes.

Added the relevant test case that demonstrates this from:
https://github.com/google/brotli/blob/master/tests/testdata/x.compressed
2022-10-11 22:49:33 -06:00
Sam Atkins
a0d44026fc AK+Tests: Correct off-by-one error when right-trimming text
If the entire string you want to right-trim consists of characters you
want to remove, we previously would incorrectly leave the first
character there.

For example: `trim("aaaaa", "a")` would return "a" instead of "".

We can't use `i >= 0` in the loop since that would fail to detect
underflow, so instead we keep `i` in the range `size .. 1` and then
subtract 1 from it when reading the character.

Added some trim() tests while I was at it. (And to confirm that this was
the issue.)
2022-10-11 17:49:32 +02:00
Andrew Kaster
828441852f Everywhere: Replace uses of __serenity__ with AK_OS_SERENITY
Now that we have OS macros for essentially every supported OS, let's try
to use them everywhere.
2022-10-10 12:23:12 +02:00
Andrew Kaster
51ebf20200 Tests: Remove LibRegex benchmark test file that has become stale
This test file had #ifdef macros at the top that caused none of the
content to be compiled unless a developer manually wanted to run the
specific benchmarks within. As such, it has become stale. Remove it for
now, if someone wants to restore it in an always-runnable state, we can
restore the specific tests it's trying to benchmark.
2022-10-10 12:23:12 +02:00
Zaggy1024
d67231312e LibVideo: Add test to ensure that a VP9 WebM file will decode
This will test decoding of one second of video, to ensure that it can
fully decode the entire file.
2022-10-09 20:32:40 -06:00
Ben Wiederhake
ff8f3814cc AK+Tests: Avoid creating invalid code points from malformed UTF-8
Instead of doing anything reasonable, Utf8CodePointIterator returned
invalid code points, for example U+123456. However, many callers of this
iterator assume that a code point is always at most 0x10FFFF.

In fact, this is one of two reasons for the following OSS Fuzz issue:
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=49184
This is probably a very old bug.

In the particular case of URLParser, AK::is_url_code_point got confused:
    return /* ... */ || code_point >= 0xA0;
If code_point is a "code point" beyond 0x10FFFF, this violates the
condition given in the preceding comment, but satisfies the given
condition, which eventually causes URLParser to crash.

This commit fixes *only* the erroneous UTF-8 decoding, and does not
fully resolve OSS-Fuzz#49184.
2022-10-09 10:37:20 -06:00
Timothy Flynn
f38c68177b LibUnicode: Update code point ideographic replacements for Unicode 15 2022-10-07 18:17:40 +01:00
matcool
104b51b912 LibUnicode: Fix Hangul syllable composition for specific cases
This fixes `combine_hangul_code_points` which would try to combine
a LVT syllable with a trailing consonant, resulting in a wrong
character.

Also added a test for this specific case.
2022-10-07 07:53:27 -04:00
EWouters
7c93eabffe LibC: Implement mkstemps() in stdlib and add a test
`mkstemps` generates a unique temporary file name from a pattern like
`prefixXXXXXXsuffix` where `prefix` and `suffix` can be any string with
only characters that are valid in a filename. The second parameter is
the length of the suffix.

`mkstemp` is `mkstemps` with suffix length 0, so to avoid code
duplication it calls `mkstemps`. It is unlikely this has any
significant performance impact on SerenityOS.

`generate_unique_filename` now takes the suffix length as a `size_t`.
The original behavior of this function is preserved when specifying a
suffix length of 0. All original uses of this function have been
adapted.

`mkstemps()` was added because it is required by version 4.6.3 of the
ccache port.
2022-10-06 16:14:40 +01:00
matcool
c8d7b0a33a Tests: Add tests for LibUnicode's normalize 2022-10-06 08:24:39 -04:00
Nico Weber
2af028132a AK+Everywhere: Add AK_COMPILER_{GCC,CLANG} and use them most places
Doesn't use them in libc headers so that those don't have to pull in
AK/Platform.h.

AK_COMPILER_GCC is set _only_ for gcc, not for clang too. (__GNUC__ is
defined in clang builds as well.) Using AK_COMPILER_GCC simplifies
things some.

AK_COMPILER_CLANG isn't as much of a win, other than that it's
consistent with AK_COMPILER_GCC.
2022-10-04 23:35:07 +01:00
Timothy Flynn
9a1b24d5be LibTimeZone: Use the last DST rule in the TZDB if a match isn't found
Some time zones, like "Asia/Shanghai", use a set of DST rules that end
before present day. In these cases, we should fall back to last possible
RULE entry from the TZDB. The time zone compiler published by IANA (zic)
performs the same fallback starting with version 2 of the time zone file
format.
2022-09-28 23:52:51 +01:00
Liav A
7d91724dd2 Tests/Kernel: Add tests to ensure we don't regress InodeVMObjects 2022-09-26 20:00:34 +03:00
Liav A
e181269da7 Revert "Tests/Kernel: Add tests to ensure we don't regress InodeVMObjects"
This reverts commit b0d555163bc724542ea6560a2bc82bc3b620c31d.
2022-09-24 13:49:40 +02:00
Daniel Bertalan
2b69af2dfe AK+LibJS: Handle NaN-boxing pointers on AArch64
JS::Value stores 48 bit pointers to separately allocated objects in its
payload. On x86-64, canonical addresses have their top 16 bits set to
the same value as bit 47, effectively meaning that the value has to be
sign-extended to get the pointer. AArch64, however, expects the topmost
bits to be all zeros.

This commit gates sign extension behind `#if ARCH(X86_64)`, and adds an
`#error` for unsupported architectures, so that we do not forget to
think about pointer handling when porting to a new architecture.

Fixes #15290
Fixes SerenityOS/ladybird#56
2022-09-21 11:55:57 +02:00
Liav A
a308331bb7 Kernel/Tests: Don't compile x86 IO for non-x86 builds
x86 IO space is not relevant for non-x86 builds so it doesn't make sense
to include it in such build as it will not compile anyway.
2022-09-20 18:43:05 +01:00
Andreas Kling
287a9b552a AK: Fix bad parsing of some file:/// URLs with base URL
We were dropping the base URL path components in the resulting URL due
to mistakenly determining the input URL to start with a Windows drive
letter. Fix this, add a spec link, and a test.
2022-09-20 15:38:53 +02:00
Ben Wiederhake
8de7a91571 LibCompress+Tests: Demonstrate and fix faulty metadata length
The test-case is heavily inspired by:
https://github.com/google/brotli/blob/master/tests/testdata/x.compressed.01
Or in words: A metadata meta-block containing `Y` (which should be
ignored), and then the actual data (a single `Z`). The bug used to skip
one metadata byte too few, and thus read garbage.
2022-09-17 04:10:11 +00:00
Tim Schumacher
bbcdd0dca0 Tests: Move the former LibM test into the LibC namespace 2022-09-16 16:09:19 +00:00
Andrew Kaster
8ed5ed3ec0 LibGL: Make GL::create_context fallible
Propagate errors in places that are already set up to handle them, like
WebGLRenderingContext and the Tubes demo, and convert other callers
to using MUST.
2022-09-16 15:32:38 +02:00
Liav A
e6306d459a Tests/Kernel: Add tests to ensure we don't regress InodeVMObjects 2022-09-16 14:55:45 +03:00
Hendiadyoin1
6b6510b577 AK+Tests: Don't double-destroy NoAllocationGuard in TestFixedArray
This caused the m_allocation_enabled_previously member to be technically
uninitialized when the compiler emits the implicit destructor call for
stack allocated classes.
This was pointed out by gcc on lagom builds, no clue how this was flying
under the radar for so long and is not triggering CI.
2022-09-15 23:04:46 +00:00
Brian Gianforcaro
d0a1775369 Everywhere: Fix a variety of typos
Spelling fixes found by `codespell`.
2022-09-14 04:46:49 +00:00