AK: Introduce the new String, replacement for DeprecatedString
DeprecatedString (formerly String) has been with us since the start,
and it has served us well. However, it has a number of shortcomings
that I'd like to address.
Some of these issues are hard if not impossible to solve incrementally
inside of DeprecatedString, so instead of doing that, let's build a new
String class and then incrementally move over to it instead.
Problems in DeprecatedString:
- It assumes string allocation never fails. This makes it impossible
to use in allocation-sensitive contexts, and is the reason we had to
ban DeprecatedString from the kernel entirely.
- The awkward null state. DeprecatedString can be null. It's different
from the empty state, although null strings are considered empty.
All code is immediately nicer when using Optional<DeprecatedString>
but DeprecatedString came before Optional, which is how we ended up
like this.
- The encoding of the underlying data is ambiguous. For the most part,
we use it as if it's always UTF-8, but there have been cases where
we pass around strings in other encodings (e.g ISO8859-1)
- operator[] and length() are used to iterate over DeprecatedString one
byte at a time. This is done all over the codebase, and will *not*
give the right results unless the string is all ASCII.
How we solve these issues in the new String:
- Functions that may allocate now return ErrorOr<String> so that ENOMEM
errors can be passed to the caller.
- String has no null state. Use Optional<String> when needed.
- String is always UTF-8. This is validated when constructing a String.
We may need to add a bypass for this in the future, for cases where
you have a known-good string, but for now: validate all the things!
- There is no operator[] or length(). You can get the underlying data
with bytes(), but for iterating over code points, you should be using
an UTF-8 iterator.
Furthermore, it has two nifty new features:
- String implements a small string optimization (SSO) for strings that
can fit entirely within a pointer. This means up to 3 bytes on 32-bit
platforms, and 7 bytes on 64-bit platforms. Such small strings will
not be heap-allocated.
- String can create substrings without making a deep copy of the
substring. Instead, the superstring gets +1 refcount from the
substring, and it acts like a view into the superstring. To make
substrings like this, use the substring_with_shared_superstring() API.
One caveat:
- String does not guarantee that the underlying data is null-terminated
like DeprecatedString does today. While this was nifty in a handful of
places where we were calling C functions, it did stand in the way of
shared-superstring substrings.
2022-12-01 12:27:43 +00:00
|
|
|
/*
|
|
|
|
* Copyright (c) 2018-2022, Andreas Kling <kling@serenityos.org>
|
|
|
|
*
|
|
|
|
* SPDX-License-Identifier: BSD-2-Clause
|
|
|
|
*/
|
|
|
|
|
|
|
|
#pragma once
|
|
|
|
|
2022-11-14 18:20:59 +00:00
|
|
|
#include <AK/Concepts.h>
|
AK: Introduce the new String, replacement for DeprecatedString
DeprecatedString (formerly String) has been with us since the start,
and it has served us well. However, it has a number of shortcomings
that I'd like to address.
Some of these issues are hard if not impossible to solve incrementally
inside of DeprecatedString, so instead of doing that, let's build a new
String class and then incrementally move over to it instead.
Problems in DeprecatedString:
- It assumes string allocation never fails. This makes it impossible
to use in allocation-sensitive contexts, and is the reason we had to
ban DeprecatedString from the kernel entirely.
- The awkward null state. DeprecatedString can be null. It's different
from the empty state, although null strings are considered empty.
All code is immediately nicer when using Optional<DeprecatedString>
but DeprecatedString came before Optional, which is how we ended up
like this.
- The encoding of the underlying data is ambiguous. For the most part,
we use it as if it's always UTF-8, but there have been cases where
we pass around strings in other encodings (e.g ISO8859-1)
- operator[] and length() are used to iterate over DeprecatedString one
byte at a time. This is done all over the codebase, and will *not*
give the right results unless the string is all ASCII.
How we solve these issues in the new String:
- Functions that may allocate now return ErrorOr<String> so that ENOMEM
errors can be passed to the caller.
- String has no null state. Use Optional<String> when needed.
- String is always UTF-8. This is validated when constructing a String.
We may need to add a bypass for this in the future, for cases where
you have a known-good string, but for now: validate all the things!
- There is no operator[] or length(). You can get the underlying data
with bytes(), but for iterating over code points, you should be using
an UTF-8 iterator.
Furthermore, it has two nifty new features:
- String implements a small string optimization (SSO) for strings that
can fit entirely within a pointer. This means up to 3 bytes on 32-bit
platforms, and 7 bytes on 64-bit platforms. Such small strings will
not be heap-allocated.
- String can create substrings without making a deep copy of the
substring. Instead, the superstring gets +1 refcount from the
substring, and it acts like a view into the superstring. To make
substrings like this, use the substring_with_shared_superstring() API.
One caveat:
- String does not guarantee that the underlying data is null-terminated
like DeprecatedString does today. While this was nifty in a handful of
places where we were calling C functions, it did stand in the way of
shared-superstring substrings.
2022-12-01 12:27:43 +00:00
|
|
|
#include <AK/Format.h>
|
|
|
|
#include <AK/Forward.h>
|
2023-01-08 21:33:30 +00:00
|
|
|
#include <AK/Optional.h>
|
AK: Introduce the new String, replacement for DeprecatedString
DeprecatedString (formerly String) has been with us since the start,
and it has served us well. However, it has a number of shortcomings
that I'd like to address.
Some of these issues are hard if not impossible to solve incrementally
inside of DeprecatedString, so instead of doing that, let's build a new
String class and then incrementally move over to it instead.
Problems in DeprecatedString:
- It assumes string allocation never fails. This makes it impossible
to use in allocation-sensitive contexts, and is the reason we had to
ban DeprecatedString from the kernel entirely.
- The awkward null state. DeprecatedString can be null. It's different
from the empty state, although null strings are considered empty.
All code is immediately nicer when using Optional<DeprecatedString>
but DeprecatedString came before Optional, which is how we ended up
like this.
- The encoding of the underlying data is ambiguous. For the most part,
we use it as if it's always UTF-8, but there have been cases where
we pass around strings in other encodings (e.g ISO8859-1)
- operator[] and length() are used to iterate over DeprecatedString one
byte at a time. This is done all over the codebase, and will *not*
give the right results unless the string is all ASCII.
How we solve these issues in the new String:
- Functions that may allocate now return ErrorOr<String> so that ENOMEM
errors can be passed to the caller.
- String has no null state. Use Optional<String> when needed.
- String is always UTF-8. This is validated when constructing a String.
We may need to add a bypass for this in the future, for cases where
you have a known-good string, but for now: validate all the things!
- There is no operator[] or length(). You can get the underlying data
with bytes(), but for iterating over code points, you should be using
an UTF-8 iterator.
Furthermore, it has two nifty new features:
- String implements a small string optimization (SSO) for strings that
can fit entirely within a pointer. This means up to 3 bytes on 32-bit
platforms, and 7 bytes on 64-bit platforms. Such small strings will
not be heap-allocated.
- String can create substrings without making a deep copy of the
substring. Instead, the superstring gets +1 refcount from the
substring, and it acts like a view into the superstring. To make
substrings like this, use the substring_with_shared_superstring() API.
One caveat:
- String does not guarantee that the underlying data is null-terminated
like DeprecatedString does today. While this was nifty in a handful of
places where we were calling C functions, it did stand in the way of
shared-superstring substrings.
2022-12-01 12:27:43 +00:00
|
|
|
#include <AK/RefCounted.h>
|
|
|
|
#include <AK/Span.h>
|
|
|
|
#include <AK/StringView.h>
|
|
|
|
#include <AK/Traits.h>
|
|
|
|
#include <AK/Types.h>
|
|
|
|
|
|
|
|
namespace AK {
|
|
|
|
|
|
|
|
namespace Detail {
|
|
|
|
class StringData;
|
|
|
|
}
|
|
|
|
|
|
|
|
// String is a strongly owned sequence of Unicode code points encoded as UTF-8.
|
|
|
|
// The data may or may not be heap-allocated, and may or may not be reference counted.
|
|
|
|
// There is no guarantee that the underlying bytes are null-terminated.
|
|
|
|
class String {
|
|
|
|
public:
|
|
|
|
// NOTE: For short strings, we avoid heap allocations by storing them in the data pointer slot.
|
|
|
|
static constexpr size_t MAX_SHORT_STRING_BYTE_COUNT = sizeof(Detail::StringData*) - 1;
|
|
|
|
|
|
|
|
String(String const&);
|
|
|
|
String(String&&);
|
|
|
|
|
|
|
|
String& operator=(String&&);
|
|
|
|
String& operator=(String const&);
|
|
|
|
|
|
|
|
~String();
|
|
|
|
|
|
|
|
// Creates an empty (zero-length) String.
|
|
|
|
String();
|
|
|
|
|
|
|
|
// Creates a new String from a sequence of UTF-8 encoded code points.
|
|
|
|
static ErrorOr<String> from_utf8(StringView);
|
|
|
|
|
2023-01-08 21:33:30 +00:00
|
|
|
// Creates a new String by transforming this String to lower- or uppercase. Using these methods
|
|
|
|
// require linking LibUnicode into your application.
|
|
|
|
ErrorOr<String> to_lowercase(Optional<StringView> const& locale = {}) const;
|
|
|
|
ErrorOr<String> to_uppercase(Optional<StringView> const& locale = {}) const;
|
|
|
|
|
AK: Introduce the new String, replacement for DeprecatedString
DeprecatedString (formerly String) has been with us since the start,
and it has served us well. However, it has a number of shortcomings
that I'd like to address.
Some of these issues are hard if not impossible to solve incrementally
inside of DeprecatedString, so instead of doing that, let's build a new
String class and then incrementally move over to it instead.
Problems in DeprecatedString:
- It assumes string allocation never fails. This makes it impossible
to use in allocation-sensitive contexts, and is the reason we had to
ban DeprecatedString from the kernel entirely.
- The awkward null state. DeprecatedString can be null. It's different
from the empty state, although null strings are considered empty.
All code is immediately nicer when using Optional<DeprecatedString>
but DeprecatedString came before Optional, which is how we ended up
like this.
- The encoding of the underlying data is ambiguous. For the most part,
we use it as if it's always UTF-8, but there have been cases where
we pass around strings in other encodings (e.g ISO8859-1)
- operator[] and length() are used to iterate over DeprecatedString one
byte at a time. This is done all over the codebase, and will *not*
give the right results unless the string is all ASCII.
How we solve these issues in the new String:
- Functions that may allocate now return ErrorOr<String> so that ENOMEM
errors can be passed to the caller.
- String has no null state. Use Optional<String> when needed.
- String is always UTF-8. This is validated when constructing a String.
We may need to add a bypass for this in the future, for cases where
you have a known-good string, but for now: validate all the things!
- There is no operator[] or length(). You can get the underlying data
with bytes(), but for iterating over code points, you should be using
an UTF-8 iterator.
Furthermore, it has two nifty new features:
- String implements a small string optimization (SSO) for strings that
can fit entirely within a pointer. This means up to 3 bytes on 32-bit
platforms, and 7 bytes on 64-bit platforms. Such small strings will
not be heap-allocated.
- String can create substrings without making a deep copy of the
substring. Instead, the superstring gets +1 refcount from the
substring, and it acts like a view into the superstring. To make
substrings like this, use the substring_with_shared_superstring() API.
One caveat:
- String does not guarantee that the underlying data is null-terminated
like DeprecatedString does today. While this was nifty in a handful of
places where we were calling C functions, it did stand in the way of
shared-superstring substrings.
2022-12-01 12:27:43 +00:00
|
|
|
// Creates a substring with a deep copy of the specified data window.
|
|
|
|
ErrorOr<String> substring_from_byte_offset(size_t start, size_t byte_count) const;
|
|
|
|
|
|
|
|
// Creates a substring that strongly references the origin superstring instead of making a deep copy of the data.
|
|
|
|
ErrorOr<String> substring_from_byte_offset_with_shared_superstring(size_t start, size_t byte_count) const;
|
|
|
|
|
|
|
|
// Returns an iterable view over the Unicode code points.
|
|
|
|
[[nodiscard]] Utf8View code_points() const;
|
|
|
|
|
|
|
|
// Returns the underlying UTF-8 encoded bytes.
|
|
|
|
// NOTE: There is no guarantee about null-termination.
|
|
|
|
[[nodiscard]] ReadonlyBytes bytes() const;
|
|
|
|
|
|
|
|
// Returns true if the String is zero-length.
|
|
|
|
[[nodiscard]] bool is_empty() const;
|
|
|
|
|
|
|
|
// Returns a StringView covering the full length of the string. Note that iterating this will go byte-at-a-time, not code-point-at-a-time.
|
|
|
|
[[nodiscard]] StringView bytes_as_string_view() const;
|
|
|
|
|
|
|
|
ErrorOr<String> replace(StringView needle, StringView replacement, ReplaceMode replace_mode) const;
|
|
|
|
|
|
|
|
[[nodiscard]] bool operator==(String const&) const;
|
|
|
|
[[nodiscard]] bool operator!=(String const& other) const { return !(*this == other); }
|
|
|
|
|
|
|
|
[[nodiscard]] bool operator==(StringView) const;
|
|
|
|
[[nodiscard]] bool operator!=(StringView other) const { return !(*this == other); }
|
|
|
|
|
|
|
|
[[nodiscard]] bool operator==(char const* cstring) const;
|
|
|
|
[[nodiscard]] bool operator!=(char const* cstring) const { return !(*this == cstring); }
|
|
|
|
|
2022-12-11 13:31:41 +00:00
|
|
|
// NOTE: UTF-8 is defined in a way that lexicographic ordering of code points is equivalent to lexicographic ordering of bytes.
|
|
|
|
[[nodiscard]] int operator<=>(String const& other) const { return this->bytes_as_string_view().compare(other.bytes_as_string_view()); }
|
|
|
|
|
AK: Introduce the new String, replacement for DeprecatedString
DeprecatedString (formerly String) has been with us since the start,
and it has served us well. However, it has a number of shortcomings
that I'd like to address.
Some of these issues are hard if not impossible to solve incrementally
inside of DeprecatedString, so instead of doing that, let's build a new
String class and then incrementally move over to it instead.
Problems in DeprecatedString:
- It assumes string allocation never fails. This makes it impossible
to use in allocation-sensitive contexts, and is the reason we had to
ban DeprecatedString from the kernel entirely.
- The awkward null state. DeprecatedString can be null. It's different
from the empty state, although null strings are considered empty.
All code is immediately nicer when using Optional<DeprecatedString>
but DeprecatedString came before Optional, which is how we ended up
like this.
- The encoding of the underlying data is ambiguous. For the most part,
we use it as if it's always UTF-8, but there have been cases where
we pass around strings in other encodings (e.g ISO8859-1)
- operator[] and length() are used to iterate over DeprecatedString one
byte at a time. This is done all over the codebase, and will *not*
give the right results unless the string is all ASCII.
How we solve these issues in the new String:
- Functions that may allocate now return ErrorOr<String> so that ENOMEM
errors can be passed to the caller.
- String has no null state. Use Optional<String> when needed.
- String is always UTF-8. This is validated when constructing a String.
We may need to add a bypass for this in the future, for cases where
you have a known-good string, but for now: validate all the things!
- There is no operator[] or length(). You can get the underlying data
with bytes(), but for iterating over code points, you should be using
an UTF-8 iterator.
Furthermore, it has two nifty new features:
- String implements a small string optimization (SSO) for strings that
can fit entirely within a pointer. This means up to 3 bytes on 32-bit
platforms, and 7 bytes on 64-bit platforms. Such small strings will
not be heap-allocated.
- String can create substrings without making a deep copy of the
substring. Instead, the superstring gets +1 refcount from the
substring, and it acts like a view into the superstring. To make
substrings like this, use the substring_with_shared_superstring() API.
One caveat:
- String does not guarantee that the underlying data is null-terminated
like DeprecatedString does today. While this was nifty in a handful of
places where we were calling C functions, it did stand in the way of
shared-superstring substrings.
2022-12-01 12:27:43 +00:00
|
|
|
[[nodiscard]] u32 hash() const;
|
|
|
|
|
2022-11-14 18:20:59 +00:00
|
|
|
template<Arithmetic T>
|
AK: Introduce the new String, replacement for DeprecatedString
DeprecatedString (formerly String) has been with us since the start,
and it has served us well. However, it has a number of shortcomings
that I'd like to address.
Some of these issues are hard if not impossible to solve incrementally
inside of DeprecatedString, so instead of doing that, let's build a new
String class and then incrementally move over to it instead.
Problems in DeprecatedString:
- It assumes string allocation never fails. This makes it impossible
to use in allocation-sensitive contexts, and is the reason we had to
ban DeprecatedString from the kernel entirely.
- The awkward null state. DeprecatedString can be null. It's different
from the empty state, although null strings are considered empty.
All code is immediately nicer when using Optional<DeprecatedString>
but DeprecatedString came before Optional, which is how we ended up
like this.
- The encoding of the underlying data is ambiguous. For the most part,
we use it as if it's always UTF-8, but there have been cases where
we pass around strings in other encodings (e.g ISO8859-1)
- operator[] and length() are used to iterate over DeprecatedString one
byte at a time. This is done all over the codebase, and will *not*
give the right results unless the string is all ASCII.
How we solve these issues in the new String:
- Functions that may allocate now return ErrorOr<String> so that ENOMEM
errors can be passed to the caller.
- String has no null state. Use Optional<String> when needed.
- String is always UTF-8. This is validated when constructing a String.
We may need to add a bypass for this in the future, for cases where
you have a known-good string, but for now: validate all the things!
- There is no operator[] or length(). You can get the underlying data
with bytes(), but for iterating over code points, you should be using
an UTF-8 iterator.
Furthermore, it has two nifty new features:
- String implements a small string optimization (SSO) for strings that
can fit entirely within a pointer. This means up to 3 bytes on 32-bit
platforms, and 7 bytes on 64-bit platforms. Such small strings will
not be heap-allocated.
- String can create substrings without making a deep copy of the
substring. Instead, the superstring gets +1 refcount from the
substring, and it acts like a view into the superstring. To make
substrings like this, use the substring_with_shared_superstring() API.
One caveat:
- String does not guarantee that the underlying data is null-terminated
like DeprecatedString does today. While this was nifty in a handful of
places where we were calling C functions, it did stand in the way of
shared-superstring substrings.
2022-12-01 12:27:43 +00:00
|
|
|
static ErrorOr<String> number(T value)
|
|
|
|
{
|
|
|
|
return formatted("{}", value);
|
|
|
|
}
|
|
|
|
|
|
|
|
static ErrorOr<String> vformatted(StringView fmtstr, TypeErasedFormatParams&);
|
|
|
|
|
|
|
|
template<typename... Parameters>
|
|
|
|
static ErrorOr<String> formatted(CheckedFormatString<Parameters...>&& fmtstr, Parameters const&... parameters)
|
|
|
|
{
|
|
|
|
VariadicFormatParams variadic_format_parameters { parameters... };
|
|
|
|
return vformatted(fmtstr.view(), variadic_format_parameters);
|
|
|
|
}
|
|
|
|
|
|
|
|
// NOTE: This is primarily interesting to unit tests.
|
|
|
|
[[nodiscard]] bool is_short_string() const;
|
|
|
|
|
|
|
|
// FIXME: Remove these once all code has been ported to String
|
|
|
|
[[nodiscard]] DeprecatedString to_deprecated_string() const;
|
|
|
|
static ErrorOr<String> from_deprecated_string(DeprecatedString const&);
|
|
|
|
|
|
|
|
private:
|
|
|
|
// NOTE: If the least significant bit of the pointer is set, this is a short string.
|
|
|
|
static constexpr uintptr_t SHORT_STRING_FLAG = 1;
|
|
|
|
|
|
|
|
struct ShortString {
|
|
|
|
ReadonlyBytes bytes() const;
|
|
|
|
size_t byte_count() const;
|
|
|
|
|
|
|
|
// NOTE: This is the byte count shifted left 1 step and or'ed with a 1 (the SHORT_STRING_FLAG)
|
|
|
|
u8 byte_count_and_short_string_flag { 0 };
|
|
|
|
u8 storage[MAX_SHORT_STRING_BYTE_COUNT] = { 0 };
|
|
|
|
};
|
|
|
|
|
|
|
|
explicit String(NonnullRefPtr<Detail::StringData>);
|
|
|
|
explicit String(ShortString);
|
|
|
|
|
|
|
|
union {
|
|
|
|
ShortString m_short_string;
|
|
|
|
Detail::StringData* m_data { nullptr };
|
|
|
|
};
|
|
|
|
};
|
|
|
|
|
|
|
|
template<>
|
|
|
|
struct Traits<String> : public GenericTraits<String> {
|
|
|
|
static unsigned hash(String const&);
|
|
|
|
};
|
|
|
|
|
|
|
|
template<>
|
|
|
|
struct Formatter<String> : Formatter<StringView> {
|
|
|
|
ErrorOr<void> format(FormatBuilder&, String const&);
|
|
|
|
};
|
|
|
|
|
|
|
|
}
|