Since we already have the type information in the Userspace template,
it was a bit silly to cast manually everywhere. Just add a sufficiently
scary-sounding getter for a typed pointer.
Thanks @alimpfard for pointing out that I was being silly with tossing
out the type.
In the future we may want to make this API non-public as well.
This will be used in the kernel to wrap pointers into userspace memory
without convenient direct access. The idea is to use the compiler to
enforce that we don't dereference userspace pointers.
I accidently wrote `Span<RemoveConst<T>>` when I meant
`Span<RemoveConst<T>::Type>`.
Changing that wouldn't be enough though, this constructor can only be
defined if T is not const, otherwise it would redefine the copy
constructor. This can be avoided by overloading the cast operator.
There's no great advantage to using MMX instructions here on modern
processors, since REP MOVSB/STOSB are optimized in microcode anyway
and tend to run much faster than MMX/SSE/AVX variants.
This also makes it much easier to implement high-level emulation of
memcpy/memset in UserspaceEmulator once we get there. :^)
Fixes#2776.
This fixes, among other things, JSON serialization.
The underlying bug was that 'print_double' defined fraction_length
as a function argument with a default value, whereas
printf_internal *always* provided a value, even if nothing was read.
The 'use 6 by default' logic has been moved to printf_internal instead.
I totally forgot about the C++ basics here. There are three distinct
types: "char", "signed char" and "unsigned char". Whether "char" is
signed or unsigned is implementation specific.
This allows performing an action based on whether something
was actually added or removed without having to look it up
prior to calling set() or remove().
The fact that JsonValues can contain 64-bit values isn't a JavaScript
compatible behavior in the first place, but as long as we're supporting
this, we should make sure it works correctly.
Prior to this, we wrote to the log every time the << operator
was used, which meant that only these parts of the log statement
were serialized. If the thread was preempted, or especially with
multiple CPUs the debug output was hard to decipher. Instead, we
buffer up the log statements. To avoid allocations we'll attempt
to use stack space, which covers most log statements.
This was showing up in Browser profiles, which is silly, so write a new
version that doesn't create a temporary String object.
There are a whole bunch of these and long-term it would be nice to find
a way to share all the very similar logic instead of duplicating it.
- Parsing invalid JSON no longer asserts
Instead of asserting when coming across malformed JSON,
JsonParser::parse now returns an Optional<JsonValue>.
- Disallow trailing commas in JSON objects and arrays
- No longer parse 'undefined', as that is a purely JS thing
- No longer allow non-whitespace after anything consumed by the initial
parse() call. Examples of things that were valid and no longer are:
- undefineddfz
- {"foo": 1}abcd
- [1,2,3]4
- JsonObject.for_each_member now iterates in original insertion order
Get rid of the weird old signature:
- int StringType::to_int(bool& ok) const
And replace it with sensible new signature:
- Optional<int> StringType::to_int() const
Before this, it has been possible to assign a RefCounted object to another
RefCounted object. Hilariosly (or sadly), that copied the refcount among
the other fields, meaning the target value ended up with a wrong refcount.
Ensure this never happens by disallowing copies and moves for RefCounted types.
This fixes all sorts of race conditions, primarily in the kernel, where till
now it's been possible to obtain either double free or use-after-free by
exploiting refcounting races.
I've been using this in the new HTML parser and it makes it much easier
to understand the state of unfinished code branches.
TODO() is for places where it's okay to end up but we need to implement
something there.
ASSERT_NOT_REACHED() is for places where it's not okay to end up, and
something has gone wrong.
The SDL port failed to build because the CMake toolchain filed pointed
to the old root. Now the toolchain file assumes that the Root is in
Build/Root.
Additionally, the AK/ and Kernel/ headers need to be installed in the
root too.
.. and make travis run it.
I renamed check-license-headers.sh to check-style.sh and expanded it so
that it now also checks for the presence of "#pragma once" in .h files.
It also checks the presence of a (single) blank line above and below the
"#pragma once" line.
I also added "#pragma once" to all the files that need it: even the ones
we are not check.
I also added/removed blank lines in order to make the script not fail.
I also ran clang-format on the files I modified.
And move canonicalized_path() to a static method on LexicalPath.
This is to make it clear that FileSystemPath/canonicalized_path() only
perform *lexical* canonicalization.
The CMake runner looks at the return code if you don't set
the pattern. Since the AK test suite setup doesn't use return
codes, we were missing test failures.
FileSystemPath::has_extension was jumping through hoops and allocating
memory to do a case insensitive comparison needlessly. Extend the
existing String::ends_with method to allow the caller to specify the
case sensitivity required.
Previously, passing a fragment string ("#section3") to the complete_url
method would result in a URL that looked like
"file:///home/anon/www/#section3" which was obviously incorrect. Now the
result looks like "file:///home/anon/www/afrag.html#section3".
We shouldn't just drop leading ..-s for relative paths. At the same time,
we should handle paths like
../foo/../../bar
correctly: the first .. after the foo cancels out the foo, but the second
one should get treated as a leading one and not get dropped.
Note that since this path resolution is purely lexical, it's never going to be
completely correct with respect to symlinks and other filesystem magic. Better
don't use it when dealing with files.
If these methods get inlined, the compiler is able to statically eliminate most
of the assertions. Alas, it doesn't realize this, and believes inlining them to
be too expensive. So give it a strong hint that it's not the case.
This *decreases* the kernel binary size.
In order to remove libstdc++ completely, we need to give up on their
implementation of abi::__cxa_demangle. The demangler logic will actually
have to be quite complex, and included in both the kernel and userspace.
A definite fixme for the future, to parse the mangled names into real
deal names.
This allows operator new and operator delete to be available to anyone
that links -lc (everyone) rather than just people that include
kmalloc.h (almost no one).
Use the AK version of std::initializer_list in AK::Vector, but only
when in serenity. When building AK for a non-serenity target, the header
<initializer_list> should be always available.
This was supposed to be the foundation for some kind of pre-kernel
environment, but nobody is working on it right now, so let's move
everything back into the kernel and remove all the confusion.
This stopped working quite some time ago due to Clang losing track of
typestates for some reason and everything becoming "unknown".
Since we're primarily using GCC anyway, it doesn't seem worth it to try
and maintain this non-working experiment for a secondary compiler.
Also it doesn't look like the Clang team is actively maintaining this
flag anyway. So good-bye, -Wconsumed. :/
With 0 initial capacity, we don't allocate an underlying ByteBuffer
for the StringBuilder, which would then lead to a null String() being
returned from to_string().
This patch makes sure we always build a valid String.
StringUtils::equals_ignoring_case() already operates on a StringView&,
so StringView should have the method directly without having to go
through a temporary String (which also has the method).
As suggested by @awesomekling in a code review and (initially) ignored
by me :^)
Implementation is roughly based on LibJS's trim_string(), but with a fix
for trimming all-whitespace strings.
When we switched the Bitmap code to operating 32 bits at a time,
we neglected to look in the trailing remainder bits after the last
full 32-bit word.
This patch fixes that and adds a couple of tests for Bitmap that I
hacked up while tracking down this bug.
I found this bug when noticing that the kernel would OOM while there
were still some pages left in the physical page allocator.
This allows easy creation of a new string from an existing StringView.
Can be used e.g. for output with printf(..., view.to_string().characters())
instead of writing printf(..., String{view}.characters()).
zeroPad => zero_pad
leftPad => left_pad
fieldWidth => field_width
These were the only variables with names in camelCase.
We were not consistent with the naming of these variables: some times we
called them zeroPad, leftPad, fieldWidth; other times we called them
zero_pad, left_pad, field_width.
These inconsistencies made the code hard to read, so I changed their
names to snake_case.
Also rename width => field_width in AK::print_hex()
These methods search from the beginning or end of a string for the
first character in the input StringView and returns the position in
the string of the first match. Note that this is not a substring match.
Each comes with single char overloads for efficiency.
Due to us using size_t for the length, the actual value will always be positive.
If, for example, we calculate the length as "0 - 1", we'll get SIZE_T_MAX. What
we can do is check that adding the characters pointer and the length together
doesn't overflow.
And switch the two-argument version of Checked::multiplication_would_overflow()
to use __builtin_mul_overflow_p(). This helps GCC optimize the code better.
There were some ideas about how to use this class but we never actually
started using it, so let's just simplify it and get it ready for use.
The basic idea is: a function returns a Result<ValueType, ErrorType>.
Callers check if the result object is_error(). If so, an ErrorType can
be fetched with the error() getter. Otherwise, a ValueType is fetched
with the value() getter. That's it. :^)
DWARF line number information, if generated, is stored in the
.debug_line section of an object file.
The information is encoded as instructions for a VM that is defined in
the DWARF specification.
By executing these instructions, we can extract the encoded line number
information.
We allow the ref-counting parts of an object to be mutated even when the
object itself is a const.
An important detail is that we allow invoking 'will_be_destroyed' and
'one_ref_left', which are not required to be const qualified, on const
objects.
This is an utility to create a URL from a given string, which may be either a
URL such as http://example.com (which will be used as-is), or a file path such
as /etc/fstab (which will be transformed into file:///etc/fstab).
Since the FlyString deduplication mechanism uses a HashTable, we know
that any StringImpl inside a non-null FlyString will already have its
lazily computed hash.
This turns into much less code in the most common cases, here's why:
The normal Optional usage pattern is something like:
auto foo = get_me_an_optional();
if (foo.has_value())
do_stuff_with(foo.value());
In this typical scenario, we check has_value() before calling value().
Without inlining, value() will double-check has_value() itself and
assert if it fails. Inlining allows the compiler to optimize all of
this away.
Clang keeps whining that NonnullFooPtrs are in "unknown" state and I'm
not sure how to resolve that right now. Disable the checking until we
can figure it out.
Our C++ code generator tools have been relying on host-side dbg() being
forwarded to stdout until now. Now they use out() instead.
Hopefully this will make it easier and more enticing to use streams in
userspace programs as well. :^)
AK::Bitmap is extended with find_next_range_of_unset_bits().
The function is implemented using count_trailing_zeroes_32(), which is
optimized on many platforms, that gives a huge performance boost.
Functions find_longest_range_of_unset_bits() and find_first_fit() are
implemented with find_next_range_of_unset_bits(). According to
benchmarks, they are 60-100% faster.
Same issue here as we had with RefPtr and NonnullRefPtr.
Since we can't make copies of an owning pointer, we don't get quite the
same static_ptr_cast<T> here. Instead I've only added a new templated
version of OwnPtr::release_nonnull() in this patch, to solve the only
issue that popped up.
I'm not sure what the best solution here is, but this works for now.
We were allowing this dangerous kind of thing:
RefPtr<Base> base;
RefPtr<Derived> derived = base;
This patch changes the {Nonnull,}RefPtr constructors so this is no
longer possible.
To downcast one of these pointers, there is now static_ptr_cast<T>:
RefPtr<Derived> derived = static_ptr_cast<Derived>(base);
Fixing this exposed a ton of cowboy-downcasts in various places,
which we're now forced to fix. :^)
Since ByteBuffer is a Buffer, it should allow us to overwrite parts of
it that we have allocated.
This comes in useful in handling unsequenced writes like handling
fragmented ip packets :^)
With relative filenames in the executable code, the executable is basically not
relocatable. This makes out-of-source builds unneccesery hard. This patchset moves
the relative link into the filesystem: that can be handled much easier :^)
This patchsets adds the semantic check of two values. One first approach
was to compare the (generated) json strings of the two values. This works
out in the most cases, but not with numbers, where "1.0" and "1" in JSON
format are semantically the same. Therefore, this patch adds deep (recursive)
check of two JsonValues.
This adds a replace functionality that replaces a string that contains
occurences of a "needle" by a "replacement" value. With "all_occurences"
enabled, all occurences are being replaced, otherwise only the first
occurence is being replaced.
* Add double number to object serializer
* Handle negative double numbers correctly
* Handle \r and \n in quoted strings independently
This improves the situation when keys contain \r or \n that currently
has the effect that "a\rkey" and "a\nkey" in an JSON object are the
same key value.
This patchset allows double numbers to be printed with the printf function.
The fraction will always be printed as 6 digit number. This can be improved :^)
This commit adds a basic implementation of
the ptrace syscall, which allows one process
(the tracer) to control another process (the tracee).
While a process is being traced, it is stopped whenever a signal is
received (other than SIGCONT).
The tracer can start tracing another thread with PT_ATTACH,
which causes the tracee to stop.
From there, the tracer can use PT_CONTINUE
to continue the execution of the tracee,
or use other request codes (which haven't been implemented yet)
to modify the state of the tracee.
Additional request codes are PT_SYSCALL, which causes the tracee to
continue exection but stop at the next entry or exit from a syscall,
and PT_GETREGS which fethces the last saved register set of the tracee
(can be used to inspect syscall arguments and return value).
A special request code is PT_TRACE_ME, which is issued by the tracee
and causes it to stop when it calls execve and wait for the
tracer to attach.
This patch adds the parsing of double values to the JSON parser.
There is another char buffer that get's filled when a "." is present
in the number parsing. When number finished, a divider is calculated
to transform the number behind the "." to the actual fraction value.
FlyString is a flyweight string class that wraps a RefPtr<StringImpl>
known to be unique among the set of FlyStrings. The class is very
unoptimized at the moment.
When to use FlyString:
- When you want O(1) string comparison
- When you want to deduplicate a lot of identical strings
When not to use FlyString:
- For strings that don't need either of the above features
- For strings that are likely to be unique
Move the "fast memcpy" stuff out of StdLibExtras.h and into Memory.h.
This will break a ton of things that were relying on StdLibExtras.h
to include a bunch of other headers. Fix will follow immediately after.
This makes it possible to include StdLibExtras.h from Types.h, which is
the main point of this exercise.
We can use __builtin_memset() without including <string.h>.
This is pretty neat, as it will allow us to reduce the header deps
of AK templates a bit, if applied consistently.
Note that this is an enabling change for an upcoming #include removal.
This was causing some obvious-in-hindsight but hard to spot bugs where
we'd implicitly convert the bool to an integer type and carry on with
the number 1 instead of the actual value().
Instead of set(const JsonValue&) and set(JsonValue&&), just do
set(JsonValue) and let callers move() if they want. This removes some
ambiguity and the compiler is smart enough to optimize it anyway.
Now it actually defaults to "a < b" comparison, instead of forcing you
to provide a trivial less-than comparator. Also you can pass in any
collection type that has .begin() and .end() and we'll sort it for you.
Add an extra out-parameter to shbuf_get() that receives the size of the
shared buffer. That way we don't need to make a separate syscall to
get the size, which we always did immediately after.
This feels a lot more consistent and Unixy:
create_shared_buffer() => shbuf_create()
share_buffer_with() => shbuf_allow_pid()
share_buffer_globally() => shbuf_allow_all()
get_shared_buffer() => shbuf_get()
release_shared_buffer() => shbuf_release()
seal_shared_buffer() => shbuf_seal()
get_shared_buffer_size() => shbuf_get_size()
Also, "shared_buffer_id" is shortened to "shbuf_id" all around.
This commit replaces SinglyLinkedListIterator::universal_end() with an
empty SinglyLinkedListIterator(). Piano needs this in order to
initialize a member array of iterators without 84 lines of
universal_end().
This should make stuff like placement new work correctly when building
outside of Serenity. This stuff is a bit delicate due to the weirdly
staged toolchain build at the moment. Hopefully we can unify this stuff
in the future.
