beenull/ladybird
1
0
Fork 0
mirror of https://github.com/LadybirdBrowser/ladybird.git synced 2024-11-21 23:20:20 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 6
ladybird/AK/String.h

281 lines
11 KiB
C
Raw Normal View History

AK: Introduce the new String, replacement for DeprecatedString DeprecatedString (formerly String) has been with us since the start, and it has served us well. However, it has a number of shortcomings that I'd like to address. Some of these issues are hard if not impossible to solve incrementally inside of DeprecatedString, so instead of doing that, let's build a new String class and then incrementally move over to it instead. Problems in DeprecatedString: - It assumes string allocation never fails. This makes it impossible to use in allocation-sensitive contexts, and is the reason we had to ban DeprecatedString from the kernel entirely. - The awkward null state. DeprecatedString can be null. It's different from the empty state, although null strings are considered empty. All code is immediately nicer when using Optional<DeprecatedString> but DeprecatedString came before Optional, which is how we ended up like this. - The encoding of the underlying data is ambiguous. For the most part, we use it as if it's always UTF-8, but there have been cases where we pass around strings in other encodings (e.g ISO8859-1) - operator[] and length() are used to iterate over DeprecatedString one byte at a time. This is done all over the codebase, and will *not* give the right results unless the string is all ASCII. How we solve these issues in the new String: - Functions that may allocate now return ErrorOr<String> so that ENOMEM errors can be passed to the caller. - String has no null state. Use Optional<String> when needed. - String is always UTF-8. This is validated when constructing a String. We may need to add a bypass for this in the future, for cases where you have a known-good string, but for now: validate all the things! - There is no operator[] or length(). You can get the underlying data with bytes(), but for iterating over code points, you should be using an UTF-8 iterator. Furthermore, it has two nifty new features: - String implements a small string optimization (SSO) for strings that can fit entirely within a pointer. This means up to 3 bytes on 32-bit platforms, and 7 bytes on 64-bit platforms. Such small strings will not be heap-allocated. - String can create substrings without making a deep copy of the substring. Instead, the superstring gets +1 refcount from the substring, and it acts like a view into the superstring. To make substrings like this, use the substring_with_shared_superstring() API. One caveat: - String does not guarantee that the underlying data is null-terminated like DeprecatedString does today. While this was nifty in a handful of places where we were calling C functions, it did stand in the way of shared-superstring substrings.
2022-12-01 12:27:43 +00:00
/*
* Copyright (c) 2018-2022, Andreas Kling <kling@serenityos.org>
AK: Add a String factory to create a string from a single code point
2023-01-21 17:34:01 +00:00
* Copyright (c) 2023, Tim Flynn <trflynn89@serenityos.org>
AK: Introduce the new String, replacement for DeprecatedString DeprecatedString (formerly String) has been with us since the start, and it has served us well. However, it has a number of shortcomings that I'd like to address. Some of these issues are hard if not impossible to solve incrementally inside of DeprecatedString, so instead of doing that, let's build a new String class and then incrementally move over to it instead. Problems in DeprecatedString: - It assumes string allocation never fails. This makes it impossible to use in allocation-sensitive contexts, and is the reason we had to ban DeprecatedString from the kernel entirely. - The awkward null state. DeprecatedString can be null. It's different from the empty state, although null strings are considered empty. All code is immediately nicer when using Optional<DeprecatedString> but DeprecatedString came before Optional, which is how we ended up like this. - The encoding of the underlying data is ambiguous. For the most part, we use it as if it's always UTF-8, but there have been cases where we pass around strings in other encodings (e.g ISO8859-1) - operator[] and length() are used to iterate over DeprecatedString one byte at a time. This is done all over the codebase, and will *not* give the right results unless the string is all ASCII. How we solve these issues in the new String: - Functions that may allocate now return ErrorOr<String> so that ENOMEM errors can be passed to the caller. - String has no null state. Use Optional<String> when needed. - String is always UTF-8. This is validated when constructing a String. We may need to add a bypass for this in the future, for cases where you have a known-good string, but for now: validate all the things! - There is no operator[] or length(). You can get the underlying data with bytes(), but for iterating over code points, you should be using an UTF-8 iterator. Furthermore, it has two nifty new features: - String implements a small string optimization (SSO) for strings that can fit entirely within a pointer. This means up to 3 bytes on 32-bit platforms, and 7 bytes on 64-bit platforms. Such small strings will not be heap-allocated. - String can create substrings without making a deep copy of the substring. Instead, the superstring gets +1 refcount from the substring, and it acts like a view into the superstring. To make substrings like this, use the substring_with_shared_superstring() API. One caveat: - String does not guarantee that the underlying data is null-terminated like DeprecatedString does today. While this was nifty in a handful of places where we were calling C functions, it did stand in the way of shared-superstring substrings.
2022-12-01 12:27:43 +00:00
*
* SPDX-License-Identifier: BSD-2-Clause
*/
#pragma once
AK: Add a String factory to create a string from a single code point
2023-01-21 17:34:01 +00:00
#include <AK/CharacterTypes.h>
Everywhere: Use C++ concepts instead of requires clauses
2022-11-14 18:20:59 +00:00
#include <AK/Concepts.h>
AK: Introduce the new String, replacement for DeprecatedString DeprecatedString (formerly String) has been with us since the start, and it has served us well. However, it has a number of shortcomings that I'd like to address. Some of these issues are hard if not impossible to solve incrementally inside of DeprecatedString, so instead of doing that, let's build a new String class and then incrementally move over to it instead. Problems in DeprecatedString: - It assumes string allocation never fails. This makes it impossible to use in allocation-sensitive contexts, and is the reason we had to ban DeprecatedString from the kernel entirely. - The awkward null state. DeprecatedString can be null. It's different from the empty state, although null strings are considered empty. All code is immediately nicer when using Optional<DeprecatedString> but DeprecatedString came before Optional, which is how we ended up like this. - The encoding of the underlying data is ambiguous. For the most part, we use it as if it's always UTF-8, but there have been cases where we pass around strings in other encodings (e.g ISO8859-1) - operator[] and length() are used to iterate over DeprecatedString one byte at a time. This is done all over the codebase, and will *not* give the right results unless the string is all ASCII. How we solve these issues in the new String: - Functions that may allocate now return ErrorOr<String> so that ENOMEM errors can be passed to the caller. - String has no null state. Use Optional<String> when needed. - String is always UTF-8. This is validated when constructing a String. We may need to add a bypass for this in the future, for cases where you have a known-good string, but for now: validate all the things! - There is no operator[] or length(). You can get the underlying data with bytes(), but for iterating over code points, you should be using an UTF-8 iterator. Furthermore, it has two nifty new features: - String implements a small string optimization (SSO) for strings that can fit entirely within a pointer. This means up to 3 bytes on 32-bit platforms, and 7 bytes on 64-bit platforms. Such small strings will not be heap-allocated. - String can create substrings without making a deep copy of the substring. Instead, the superstring gets +1 refcount from the substring, and it acts like a view into the superstring. To make substrings like this, use the substring_with_shared_superstring() API. One caveat: - String does not guarantee that the underlying data is null-terminated like DeprecatedString does today. While this was nifty in a handful of places where we were calling C functions, it did stand in the way of shared-superstring substrings.
2022-12-01 12:27:43 +00:00
#include <AK/Format.h>
#include <AK/Forward.h>
AK+LibUnicode: Provide Unicode-aware String case transformations Since AK can't refer to LibUnicode directly, the strategy here is that if you need case transformations, you can link LibUnicode and receive them. If you try to use either of these methods without linking it, then you'll of course get a linker error (note we don't do any fallbacks to e.g. ASCII case transformations). If you don't need these methods, you don't have to link LibUnicode.
2023-01-08 21:33:30 +00:00
#include <AK/Optional.h>
AK: Introduce the new String, replacement for DeprecatedString DeprecatedString (formerly String) has been with us since the start, and it has served us well. However, it has a number of shortcomings that I'd like to address. Some of these issues are hard if not impossible to solve incrementally inside of DeprecatedString, so instead of doing that, let's build a new String class and then incrementally move over to it instead. Problems in DeprecatedString: - It assumes string allocation never fails. This makes it impossible to use in allocation-sensitive contexts, and is the reason we had to ban DeprecatedString from the kernel entirely. - The awkward null state. DeprecatedString can be null. It's different from the empty state, although null strings are considered empty. All code is immediately nicer when using Optional<DeprecatedString> but DeprecatedString came before Optional, which is how we ended up like this. - The encoding of the underlying data is ambiguous. For the most part, we use it as if it's always UTF-8, but there have been cases where we pass around strings in other encodings (e.g ISO8859-1) - operator[] and length() are used to iterate over DeprecatedString one byte at a time. This is done all over the codebase, and will *not* give the right results unless the string is all ASCII. How we solve these issues in the new String: - Functions that may allocate now return ErrorOr<String> so that ENOMEM errors can be passed to the caller. - String has no null state. Use Optional<String> when needed. - String is always UTF-8. This is validated when constructing a String. We may need to add a bypass for this in the future, for cases where you have a known-good string, but for now: validate all the things! - There is no operator[] or length(). You can get the underlying data with bytes(), but for iterating over code points, you should be using an UTF-8 iterator. Furthermore, it has two nifty new features: - String implements a small string optimization (SSO) for strings that can fit entirely within a pointer. This means up to 3 bytes on 32-bit platforms, and 7 bytes on 64-bit platforms. Such small strings will not be heap-allocated. - String can create substrings without making a deep copy of the substring. Instead, the superstring gets +1 refcount from the substring, and it acts like a view into the superstring. To make substrings like this, use the substring_with_shared_superstring() API. One caveat: - String does not guarantee that the underlying data is null-terminated like DeprecatedString does today. While this was nifty in a handful of places where we were calling C functions, it did stand in the way of shared-superstring substrings.
2022-12-01 12:27:43 +00:00
#include <AK/RefCounted.h>
#include <AK/Span.h>
AK: Add String::join
2023-01-27 19:06:05 +00:00
#include <AK/StringBuilder.h>
AK: Add String::contains
2023-01-14 15:17:32 +00:00
#include <AK/StringUtils.h>
AK: Introduce the new String, replacement for DeprecatedString DeprecatedString (formerly String) has been with us since the start, and it has served us well. However, it has a number of shortcomings that I'd like to address. Some of these issues are hard if not impossible to solve incrementally inside of DeprecatedString, so instead of doing that, let's build a new String class and then incrementally move over to it instead. Problems in DeprecatedString: - It assumes string allocation never fails. This makes it impossible to use in allocation-sensitive contexts, and is the reason we had to ban DeprecatedString from the kernel entirely. - The awkward null state. DeprecatedString can be null. It's different from the empty state, although null strings are considered empty. All code is immediately nicer when using Optional<DeprecatedString> but DeprecatedString came before Optional, which is how we ended up like this. - The encoding of the underlying data is ambiguous. For the most part, we use it as if it's always UTF-8, but there have been cases where we pass around strings in other encodings (e.g ISO8859-1) - operator[] and length() are used to iterate over DeprecatedString one byte at a time. This is done all over the codebase, and will *not* give the right results unless the string is all ASCII. How we solve these issues in the new String: - Functions that may allocate now return ErrorOr<String> so that ENOMEM errors can be passed to the caller. - String has no null state. Use Optional<String> when needed. - String is always UTF-8. This is validated when constructing a String. We may need to add a bypass for this in the future, for cases where you have a known-good string, but for now: validate all the things! - There is no operator[] or length(). You can get the underlying data with bytes(), but for iterating over code points, you should be using an UTF-8 iterator. Furthermore, it has two nifty new features: - String implements a small string optimization (SSO) for strings that can fit entirely within a pointer. This means up to 3 bytes on 32-bit platforms, and 7 bytes on 64-bit platforms. Such small strings will not be heap-allocated. - String can create substrings without making a deep copy of the substring. Instead, the superstring gets +1 refcount from the substring, and it acts like a view into the superstring. To make substrings like this, use the substring_with_shared_superstring() API. One caveat: - String does not guarantee that the underlying data is null-terminated like DeprecatedString does today. While this was nifty in a handful of places where we were calling C functions, it did stand in the way of shared-superstring substrings.
2022-12-01 12:27:43 +00:00
#include <AK/StringView.h>
#include <AK/Traits.h>
#include <AK/Types.h>
AK: Add a String factory to create a string from a single code point
2023-01-21 17:34:01 +00:00
#include <AK/UnicodeUtils.h>
AK: Ensure short String instances are valid UTF-8 We are currently only validating long strings.
2023-03-03 14:03:45 +00:00
#include <AK/Utf8View.h>
AK: Add `split()` for `String`
2023-01-16 16:12:53 +00:00
#include <AK/Vector.h>
AK: Introduce the new String, replacement for DeprecatedString DeprecatedString (formerly String) has been with us since the start, and it has served us well. However, it has a number of shortcomings that I'd like to address. Some of these issues are hard if not impossible to solve incrementally inside of DeprecatedString, so instead of doing that, let's build a new String class and then incrementally move over to it instead. Problems in DeprecatedString: - It assumes string allocation never fails. This makes it impossible to use in allocation-sensitive contexts, and is the reason we had to ban DeprecatedString from the kernel entirely. - The awkward null state. DeprecatedString can be null. It's different from the empty state, although null strings are considered empty. All code is immediately nicer when using Optional<DeprecatedString> but DeprecatedString came before Optional, which is how we ended up like this. - The encoding of the underlying data is ambiguous. For the most part, we use it as if it's always UTF-8, but there have been cases where we pass around strings in other encodings (e.g ISO8859-1) - operator[] and length() are used to iterate over DeprecatedString one byte at a time. This is done all over the codebase, and will *not* give the right results unless the string is all ASCII. How we solve these issues in the new String: - Functions that may allocate now return ErrorOr<String> so that ENOMEM errors can be passed to the caller. - String has no null state. Use Optional<String> when needed. - String is always UTF-8. This is validated when constructing a String. We may need to add a bypass for this in the future, for cases where you have a known-good string, but for now: validate all the things! - There is no operator[] or length(). You can get the underlying data with bytes(), but for iterating over code points, you should be using an UTF-8 iterator. Furthermore, it has two nifty new features: - String implements a small string optimization (SSO) for strings that can fit entirely within a pointer. This means up to 3 bytes on 32-bit platforms, and 7 bytes on 64-bit platforms. Such small strings will not be heap-allocated. - String can create substrings without making a deep copy of the substring. Instead, the superstring gets +1 refcount from the substring, and it acts like a view into the superstring. To make substrings like this, use the substring_with_shared_superstring() API. One caveat: - String does not guarantee that the underlying data is null-terminated like DeprecatedString does today. While this was nifty in a handful of places where we were calling C functions, it did stand in the way of shared-superstring substrings.
2022-12-01 12:27:43 +00:00
namespace AK {
namespace Detail {
class StringData;
}
AK: Use constexpr instead of consteval on OpenBSD
2023-01-25 23:25:36 +00:00
// FIXME: Remove this when Apple Clang and OpenBSD Clang fully supports consteval.
#if defined(AK_OS_MACOS) || defined(AK_OS_OPENBSD)
AK: Disable use of consteval in String::from_utf8_short_string for Apple This causes an ICE on older versions of clang, and Apple's clang is currently based on such a version.
2023-01-20 20:03:04 +00:00
# define AK_SHORT_STRING_CONSTEVAL constexpr
#else
# define AK_SHORT_STRING_CONSTEVAL consteval
#endif
AK: Introduce the new String, replacement for DeprecatedString DeprecatedString (formerly String) has been with us since the start, and it has served us well. However, it has a number of shortcomings that I'd like to address. Some of these issues are hard if not impossible to solve incrementally inside of DeprecatedString, so instead of doing that, let's build a new String class and then incrementally move over to it instead. Problems in DeprecatedString: - It assumes string allocation never fails. This makes it impossible to use in allocation-sensitive contexts, and is the reason we had to ban DeprecatedString from the kernel entirely. - The awkward null state. DeprecatedString can be null. It's different from the empty state, although null strings are considered empty. All code is immediately nicer when using Optional<DeprecatedString> but DeprecatedString came before Optional, which is how we ended up like this. - The encoding of the underlying data is ambiguous. For the most part, we use it as if it's always UTF-8, but there have been cases where we pass around strings in other encodings (e.g ISO8859-1) - operator[] and length() are used to iterate over DeprecatedString one byte at a time. This is done all over the codebase, and will *not* give the right results unless the string is all ASCII. How we solve these issues in the new String: - Functions that may allocate now return ErrorOr<String> so that ENOMEM errors can be passed to the caller. - String has no null state. Use Optional<String> when needed. - String is always UTF-8. This is validated when constructing a String. We may need to add a bypass for this in the future, for cases where you have a known-good string, but for now: validate all the things! - There is no operator[] or length(). You can get the underlying data with bytes(), but for iterating over code points, you should be using an UTF-8 iterator. Furthermore, it has two nifty new features: - String implements a small string optimization (SSO) for strings that can fit entirely within a pointer. This means up to 3 bytes on 32-bit platforms, and 7 bytes on 64-bit platforms. Such small strings will not be heap-allocated. - String can create substrings without making a deep copy of the substring. Instead, the superstring gets +1 refcount from the substring, and it acts like a view into the superstring. To make substrings like this, use the substring_with_shared_superstring() API. One caveat: - String does not guarantee that the underlying data is null-terminated like DeprecatedString does today. While this was nifty in a handful of places where we were calling C functions, it did stand in the way of shared-superstring substrings.
2022-12-01 12:27:43 +00:00
// String is a strongly owned sequence of Unicode code points encoded as UTF-8.
// The data may or may not be heap-allocated, and may or may not be reference counted.
// There is no guarantee that the underlying bytes are null-terminated.
class String {
public:
// NOTE: For short strings, we avoid heap allocations by storing them in the data pointer slot.
static constexpr size_t MAX_SHORT_STRING_BYTE_COUNT = sizeof(Detail::StringData*) - 1;
String(String const&);
String(String&&);
String& operator=(String&&);
String& operator=(String const&);
AK: Support creating known short string literals at compile time In cases where we know a string literal will fit in the short string storage, we can do so at compile time without needing to handle error propagation. If the provided string literal is too long, a compilation error will be emitted due to the failed VERIFY statement being a non- constant expression.
2023-01-20 12:20:01 +00:00
constexpr ~String()
{
if (!is_constant_evaluated())
destroy_string();
}
AK: Introduce the new String, replacement for DeprecatedString DeprecatedString (formerly String) has been with us since the start, and it has served us well. However, it has a number of shortcomings that I'd like to address. Some of these issues are hard if not impossible to solve incrementally inside of DeprecatedString, so instead of doing that, let's build a new String class and then incrementally move over to it instead. Problems in DeprecatedString: - It assumes string allocation never fails. This makes it impossible to use in allocation-sensitive contexts, and is the reason we had to ban DeprecatedString from the kernel entirely. - The awkward null state. DeprecatedString can be null. It's different from the empty state, although null strings are considered empty. All code is immediately nicer when using Optional<DeprecatedString> but DeprecatedString came before Optional, which is how we ended up like this. - The encoding of the underlying data is ambiguous. For the most part, we use it as if it's always UTF-8, but there have been cases where we pass around strings in other encodings (e.g ISO8859-1) - operator[] and length() are used to iterate over DeprecatedString one byte at a time. This is done all over the codebase, and will *not* give the right results unless the string is all ASCII. How we solve these issues in the new String: - Functions that may allocate now return ErrorOr<String> so that ENOMEM errors can be passed to the caller. - String has no null state. Use Optional<String> when needed. - String is always UTF-8. This is validated when constructing a String. We may need to add a bypass for this in the future, for cases where you have a known-good string, but for now: validate all the things! - There is no operator[] or length(). You can get the underlying data with bytes(), but for iterating over code points, you should be using an UTF-8 iterator. Furthermore, it has two nifty new features: - String implements a small string optimization (SSO) for strings that can fit entirely within a pointer. This means up to 3 bytes on 32-bit platforms, and 7 bytes on 64-bit platforms. Such small strings will not be heap-allocated. - String can create substrings without making a deep copy of the substring. Instead, the superstring gets +1 refcount from the substring, and it acts like a view into the superstring. To make substrings like this, use the substring_with_shared_superstring() API. One caveat: - String does not guarantee that the underlying data is null-terminated like DeprecatedString does today. While this was nifty in a handful of places where we were calling C functions, it did stand in the way of shared-superstring substrings.
2022-12-01 12:27:43 +00:00
// Creates an empty (zero-length) String.
AK: Change String's default constructor to be constant This allows creating expressions such as: constexpr Array<String, 10> {};
2023-01-21 17:02:49 +00:00
constexpr String()
: String(ShortString { SHORT_STRING_FLAG, {} })
{
}
AK: Introduce the new String, replacement for DeprecatedString DeprecatedString (formerly String) has been with us since the start, and it has served us well. However, it has a number of shortcomings that I'd like to address. Some of these issues are hard if not impossible to solve incrementally inside of DeprecatedString, so instead of doing that, let's build a new String class and then incrementally move over to it instead. Problems in DeprecatedString: - It assumes string allocation never fails. This makes it impossible to use in allocation-sensitive contexts, and is the reason we had to ban DeprecatedString from the kernel entirely. - The awkward null state. DeprecatedString can be null. It's different from the empty state, although null strings are considered empty. All code is immediately nicer when using Optional<DeprecatedString> but DeprecatedString came before Optional, which is how we ended up like this. - The encoding of the underlying data is ambiguous. For the most part, we use it as if it's always UTF-8, but there have been cases where we pass around strings in other encodings (e.g ISO8859-1) - operator[] and length() are used to iterate over DeprecatedString one byte at a time. This is done all over the codebase, and will *not* give the right results unless the string is all ASCII. How we solve these issues in the new String: - Functions that may allocate now return ErrorOr<String> so that ENOMEM errors can be passed to the caller. - String has no null state. Use Optional<String> when needed. - String is always UTF-8. This is validated when constructing a String. We may need to add a bypass for this in the future, for cases where you have a known-good string, but for now: validate all the things! - There is no operator[] or length(). You can get the underlying data with bytes(), but for iterating over code points, you should be using an UTF-8 iterator. Furthermore, it has two nifty new features: - String implements a small string optimization (SSO) for strings that can fit entirely within a pointer. This means up to 3 bytes on 32-bit platforms, and 7 bytes on 64-bit platforms. Such small strings will not be heap-allocated. - String can create substrings without making a deep copy of the substring. Instead, the superstring gets +1 refcount from the substring, and it acts like a view into the superstring. To make substrings like this, use the substring_with_shared_superstring() API. One caveat: - String does not guarantee that the underlying data is null-terminated like DeprecatedString does today. While this was nifty in a handful of places where we were calling C functions, it did stand in the way of shared-superstring substrings.
2022-12-01 12:27:43 +00:00
// Creates a new String from a sequence of UTF-8 encoded code points.
static ErrorOr<String> from_utf8(StringView);
AK: Add String::from_stream method The caller is responsible for determining how long the string is that they want to read.
2023-02-20 01:34:29 +00:00
// Creates a new String by reading byte_count bytes from a UTF-8 encoded Stream.
static ErrorOr<String> from_stream(Stream&, size_t byte_count);
AK: Support creating known short string literals at compile time In cases where we know a string literal will fit in the short string storage, we can do so at compile time without needing to handle error propagation. If the provided string literal is too long, a compilation error will be emitted due to the failed VERIFY statement being a non- constant expression.
2023-01-20 12:20:01 +00:00
// Creates a new String from a short sequence of UTF-8 encoded code points. If the provided string
// does not fit in the short string storage, a compilation error will be emitted.
AK: Disable use of consteval in String::from_utf8_short_string for Apple This causes an ICE on older versions of clang, and Apple's clang is currently based on such a version.
2023-01-20 20:03:04 +00:00
static AK_SHORT_STRING_CONSTEVAL String from_utf8_short_string(StringView string)
AK: Support creating known short string literals at compile time In cases where we know a string literal will fit in the short string storage, we can do so at compile time without needing to handle error propagation. If the provided string literal is too long, a compilation error will be emitted due to the failed VERIFY statement being a non- constant expression.
2023-01-20 12:20:01 +00:00
{
VERIFY(string.length() <= MAX_SHORT_STRING_BYTE_COUNT);
AK: Ensure short String instances are valid UTF-8 We are currently only validating long strings.
2023-03-03 14:03:45 +00:00
VERIFY(Utf8View { string }.validate());
AK: Support creating known short string literals at compile time In cases where we know a string literal will fit in the short string storage, we can do so at compile time without needing to handle error propagation. If the provided string literal is too long, a compilation error will be emitted due to the failed VERIFY statement being a non- constant expression.
2023-01-20 12:20:01 +00:00
ShortString short_string;
for (size_t i = 0; i < string.length(); ++i)
short_string.storage[i] = string.characters_without_null_termination()[i];
short_string.byte_count_and_short_string_flag = (string.length() << 1) | SHORT_STRING_FLAG;
AK: Add a String factory to create a string from a single code point
2023-01-21 17:34:01 +00:00
return String { short_string };
}
// Creates a new String from a single code point.
static constexpr String from_code_point(u32 code_point)
{
VERIFY(is_unicode(code_point));
ShortString short_string;
size_t i = 0;
auto length = UnicodeUtils::code_point_to_utf8(code_point, [&](auto byte) {
short_string.storage[i++] = static_cast<u8>(byte);
});
short_string.byte_count_and_short_string_flag = (length << 1) | SHORT_STRING_FLAG;
AK: Support creating known short string literals at compile time In cases where we know a string literal will fit in the short string storage, we can do so at compile time without needing to handle error propagation. If the provided string literal is too long, a compilation error will be emitted due to the failed VERIFY statement being a non- constant expression.
2023-01-20 12:20:01 +00:00
return String { short_string };
}
AK: Add a method to create a String with a repeated code point
2023-01-22 15:17:48 +00:00
// Creates a new String with a single code point repeated N times.
static ErrorOr<String> repeated(u32 code_point, size_t count);
AK+LibUnicode: Provide Unicode-aware String titlecase transformation
2023-01-16 16:28:27 +00:00
// Creates a new String by case-transforming this String. Using these methods require linking LibUnicode into your application.
AK+LibUnicode: Provide Unicode-aware String case transformations Since AK can't refer to LibUnicode directly, the strategy here is that if you need case transformations, you can link LibUnicode and receive them. If you try to use either of these methods without linking it, then you'll of course get a linker error (note we don't do any fallbacks to e.g. ASCII case transformations). If you don't need these methods, you don't have to link LibUnicode.
2023-01-08 21:33:30 +00:00
ErrorOr<String> to_lowercase(Optional<StringView> const& locale = {}) const;
ErrorOr<String> to_uppercase(Optional<StringView> const& locale = {}) const;
AK+LibUnicode: Provide Unicode-aware String titlecase transformation
2023-01-16 16:28:27 +00:00
ErrorOr<String> to_titlecase(Optional<StringView> const& locale = {}) const;
AK+LibUnicode: Provide Unicode-aware caseless String matching The Unicode spec defines much more complicated caseless matching algorithms in its Collation spec. This implements the "basic" case folding comparison.
2023-01-17 16:30:10 +00:00
ErrorOr<String> to_casefold() const;
// Compare this String against another string with caseless matching. Using this method requires linking LibUnicode into your application.
AK+LibUnicode: Implement String::equals_ignoring_case without allocating We currently fully casefold the left- and right-hand sides to compare two strings with case-insensitivity. Now, we casefold one code point at a time, storing the result in a view for comparison, until we exhaust both strings.
2023-03-08 18:03:04 +00:00
[[nodiscard]] bool equals_ignoring_case(String const&) const;
AK+LibUnicode: Provide Unicode-aware String case transformations Since AK can't refer to LibUnicode directly, the strategy here is that if you need case transformations, you can link LibUnicode and receive them. If you try to use either of these methods without linking it, then you'll of course get a linker error (note we don't do any fallbacks to e.g. ASCII case transformations). If you don't need these methods, you don't have to link LibUnicode.
2023-01-08 21:33:30 +00:00
AK: Make String::{starts,ends}_with(code_point) handle non-ASCII We currently pass the code point to StringView::{starts,ends}_with, which actually accepts a single char, thus cannot handle non-ASCII code points.
2023-03-08 13:56:02 +00:00
[[nodiscard]] bool starts_with(u32 code_point) const;
[[nodiscard]] bool starts_with_bytes(StringView) const;
AK: Add two starts_with{bytes,}() APIs to String
2023-02-18 06:34:37 +00:00
AK: Make String::{starts,ends}_with(code_point) handle non-ASCII We currently pass the code point to StringView::{starts,ends}_with, which actually accepts a single char, thus cannot handle non-ASCII code points.
2023-03-08 13:56:02 +00:00
[[nodiscard]] bool ends_with(u32 code_point) const;
[[nodiscard]] bool ends_with_bytes(StringView) const;
AK: Add String::ends_with{,_bytes}()
2023-03-03 09:27:50 +00:00
AK: Introduce the new String, replacement for DeprecatedString DeprecatedString (formerly String) has been with us since the start, and it has served us well. However, it has a number of shortcomings that I'd like to address. Some of these issues are hard if not impossible to solve incrementally inside of DeprecatedString, so instead of doing that, let's build a new String class and then incrementally move over to it instead. Problems in DeprecatedString: - It assumes string allocation never fails. This makes it impossible to use in allocation-sensitive contexts, and is the reason we had to ban DeprecatedString from the kernel entirely. - The awkward null state. DeprecatedString can be null. It's different from the empty state, although null strings are considered empty. All code is immediately nicer when using Optional<DeprecatedString> but DeprecatedString came before Optional, which is how we ended up like this. - The encoding of the underlying data is ambiguous. For the most part, we use it as if it's always UTF-8, but there have been cases where we pass around strings in other encodings (e.g ISO8859-1) - operator[] and length() are used to iterate over DeprecatedString one byte at a time. This is done all over the codebase, and will *not* give the right results unless the string is all ASCII. How we solve these issues in the new String: - Functions that may allocate now return ErrorOr<String> so that ENOMEM errors can be passed to the caller. - String has no null state. Use Optional<String> when needed. - String is always UTF-8. This is validated when constructing a String. We may need to add a bypass for this in the future, for cases where you have a known-good string, but for now: validate all the things! - There is no operator[] or length(). You can get the underlying data with bytes(), but for iterating over code points, you should be using an UTF-8 iterator. Furthermore, it has two nifty new features: - String implements a small string optimization (SSO) for strings that can fit entirely within a pointer. This means up to 3 bytes on 32-bit platforms, and 7 bytes on 64-bit platforms. Such small strings will not be heap-allocated. - String can create substrings without making a deep copy of the substring. Instead, the superstring gets +1 refcount from the substring, and it acts like a view into the superstring. To make substrings like this, use the substring_with_shared_superstring() API. One caveat: - String does not guarantee that the underlying data is null-terminated like DeprecatedString does today. While this was nifty in a handful of places where we were calling C functions, it did stand in the way of shared-superstring substrings.
2022-12-01 12:27:43 +00:00
// Creates a substring with a deep copy of the specified data window.
ErrorOr<String> substring_from_byte_offset(size_t start, size_t byte_count) const;
AK: Add convenience substring wrappers to String to exclude a length These overloads exist on other string classes and are used throughout the code base.
2023-01-22 16:40:57 +00:00
ErrorOr<String> substring_from_byte_offset(size_t start) const;
AK: Introduce the new String, replacement for DeprecatedString DeprecatedString (formerly String) has been with us since the start, and it has served us well. However, it has a number of shortcomings that I'd like to address. Some of these issues are hard if not impossible to solve incrementally inside of DeprecatedString, so instead of doing that, let's build a new String class and then incrementally move over to it instead. Problems in DeprecatedString: - It assumes string allocation never fails. This makes it impossible to use in allocation-sensitive contexts, and is the reason we had to ban DeprecatedString from the kernel entirely. - The awkward null state. DeprecatedString can be null. It's different from the empty state, although null strings are considered empty. All code is immediately nicer when using Optional<DeprecatedString> but DeprecatedString came before Optional, which is how we ended up like this. - The encoding of the underlying data is ambiguous. For the most part, we use it as if it's always UTF-8, but there have been cases where we pass around strings in other encodings (e.g ISO8859-1) - operator[] and length() are used to iterate over DeprecatedString one byte at a time. This is done all over the codebase, and will *not* give the right results unless the string is all ASCII. How we solve these issues in the new String: - Functions that may allocate now return ErrorOr<String> so that ENOMEM errors can be passed to the caller. - String has no null state. Use Optional<String> when needed. - String is always UTF-8. This is validated when constructing a String. We may need to add a bypass for this in the future, for cases where you have a known-good string, but for now: validate all the things! - There is no operator[] or length(). You can get the underlying data with bytes(), but for iterating over code points, you should be using an UTF-8 iterator. Furthermore, it has two nifty new features: - String implements a small string optimization (SSO) for strings that can fit entirely within a pointer. This means up to 3 bytes on 32-bit platforms, and 7 bytes on 64-bit platforms. Such small strings will not be heap-allocated. - String can create substrings without making a deep copy of the substring. Instead, the superstring gets +1 refcount from the substring, and it acts like a view into the superstring. To make substrings like this, use the substring_with_shared_superstring() API. One caveat: - String does not guarantee that the underlying data is null-terminated like DeprecatedString does today. While this was nifty in a handful of places where we were calling C functions, it did stand in the way of shared-superstring substrings.
2022-12-01 12:27:43 +00:00
// Creates a substring that strongly references the origin superstring instead of making a deep copy of the data.
ErrorOr<String> substring_from_byte_offset_with_shared_superstring(size_t start, size_t byte_count) const;
AK: Add convenience substring wrappers to String to exclude a length These overloads exist on other string classes and are used throughout the code base.
2023-01-22 16:40:57 +00:00
ErrorOr<String> substring_from_byte_offset_with_shared_superstring(size_t start) const;
AK: Introduce the new String, replacement for DeprecatedString DeprecatedString (formerly String) has been with us since the start, and it has served us well. However, it has a number of shortcomings that I'd like to address. Some of these issues are hard if not impossible to solve incrementally inside of DeprecatedString, so instead of doing that, let's build a new String class and then incrementally move over to it instead. Problems in DeprecatedString: - It assumes string allocation never fails. This makes it impossible to use in allocation-sensitive contexts, and is the reason we had to ban DeprecatedString from the kernel entirely. - The awkward null state. DeprecatedString can be null. It's different from the empty state, although null strings are considered empty. All code is immediately nicer when using Optional<DeprecatedString> but DeprecatedString came before Optional, which is how we ended up like this. - The encoding of the underlying data is ambiguous. For the most part, we use it as if it's always UTF-8, but there have been cases where we pass around strings in other encodings (e.g ISO8859-1) - operator[] and length() are used to iterate over DeprecatedString one byte at a time. This is done all over the codebase, and will *not* give the right results unless the string is all ASCII. How we solve these issues in the new String: - Functions that may allocate now return ErrorOr<String> so that ENOMEM errors can be passed to the caller. - String has no null state. Use Optional<String> when needed. - String is always UTF-8. This is validated when constructing a String. We may need to add a bypass for this in the future, for cases where you have a known-good string, but for now: validate all the things! - There is no operator[] or length(). You can get the underlying data with bytes(), but for iterating over code points, you should be using an UTF-8 iterator. Furthermore, it has two nifty new features: - String implements a small string optimization (SSO) for strings that can fit entirely within a pointer. This means up to 3 bytes on 32-bit platforms, and 7 bytes on 64-bit platforms. Such small strings will not be heap-allocated. - String can create substrings without making a deep copy of the substring. Instead, the superstring gets +1 refcount from the substring, and it acts like a view into the superstring. To make substrings like this, use the substring_with_shared_superstring() API. One caveat: - String does not guarantee that the underlying data is null-terminated like DeprecatedString does today. While this was nifty in a handful of places where we were calling C functions, it did stand in the way of shared-superstring substrings.
2022-12-01 12:27:43 +00:00
// Returns an iterable view over the Unicode code points.
[[nodiscard]] Utf8View code_points() const;
// Returns the underlying UTF-8 encoded bytes.
// NOTE: There is no guarantee about null-termination.
[[nodiscard]] ReadonlyBytes bytes() const;
// Returns true if the String is zero-length.
[[nodiscard]] bool is_empty() const;
// Returns a StringView covering the full length of the string. Note that iterating this will go byte-at-a-time, not code-point-at-a-time.
[[nodiscard]] StringView bytes_as_string_view() const;
ErrorOr<String> replace(StringView needle, StringView replacement, ReplaceMode replace_mode) const;
AK: Add a somewhat naive implementation of String::reverse This will reverse the String's code points (i.e. not just its bytes), but is not aware of grapheme clusters.
2023-01-13 16:34:00 +00:00
ErrorOr<String> reverse() const;
AK: Introduce the new String, replacement for DeprecatedString DeprecatedString (formerly String) has been with us since the start, and it has served us well. However, it has a number of shortcomings that I'd like to address. Some of these issues are hard if not impossible to solve incrementally inside of DeprecatedString, so instead of doing that, let's build a new String class and then incrementally move over to it instead. Problems in DeprecatedString: - It assumes string allocation never fails. This makes it impossible to use in allocation-sensitive contexts, and is the reason we had to ban DeprecatedString from the kernel entirely. - The awkward null state. DeprecatedString can be null. It's different from the empty state, although null strings are considered empty. All code is immediately nicer when using Optional<DeprecatedString> but DeprecatedString came before Optional, which is how we ended up like this. - The encoding of the underlying data is ambiguous. For the most part, we use it as if it's always UTF-8, but there have been cases where we pass around strings in other encodings (e.g ISO8859-1) - operator[] and length() are used to iterate over DeprecatedString one byte at a time. This is done all over the codebase, and will *not* give the right results unless the string is all ASCII. How we solve these issues in the new String: - Functions that may allocate now return ErrorOr<String> so that ENOMEM errors can be passed to the caller. - String has no null state. Use Optional<String> when needed. - String is always UTF-8. This is validated when constructing a String. We may need to add a bypass for this in the future, for cases where you have a known-good string, but for now: validate all the things! - There is no operator[] or length(). You can get the underlying data with bytes(), but for iterating over code points, you should be using an UTF-8 iterator. Furthermore, it has two nifty new features: - String implements a small string optimization (SSO) for strings that can fit entirely within a pointer. This means up to 3 bytes on 32-bit platforms, and 7 bytes on 64-bit platforms. Such small strings will not be heap-allocated. - String can create substrings without making a deep copy of the substring. Instead, the superstring gets +1 refcount from the substring, and it acts like a view into the superstring. To make substrings like this, use the substring_with_shared_superstring() API. One caveat: - String does not guarantee that the underlying data is null-terminated like DeprecatedString does today. While this was nifty in a handful of places where we were calling C functions, it did stand in the way of shared-superstring substrings.
2022-12-01 12:27:43 +00:00
AK: Add String::trim
2023-01-27 19:37:40 +00:00
ErrorOr<String> trim(Utf8View const& code_points_to_trim, TrimMode mode = TrimMode::Both) const;
ErrorOr<String> trim(StringView code_points_to_trim, TrimMode mode = TrimMode::Both) const;
AK: Remove [[nodiscard]] attribute from String methods returning ErrorOr
2023-01-22 14:04:02 +00:00
ErrorOr<Vector<String>> split_limit(u32 separator, size_t limit, SplitBehavior = SplitBehavior::Nothing) const;
ErrorOr<Vector<String>> split(u32 separator, SplitBehavior = SplitBehavior::Nothing) const;
AK: Add `split()` for `String`
2023-01-16 16:12:53 +00:00
AK: Add a method to find the byte offset of a code point
2023-01-22 14:24:12 +00:00
Optional<size_t> find_byte_offset(u32 code_point, size_t from_byte_offset = 0) const;
AK: Add an overload of String::find_byte_offset for StringView
2023-01-27 15:17:34 +00:00
Optional<size_t> find_byte_offset(StringView substring, size_t from_byte_offset = 0) const;
AK: Add a method to find the byte offset of a code point
2023-01-22 14:24:12 +00:00
AK: Introduce the new String, replacement for DeprecatedString DeprecatedString (formerly String) has been with us since the start, and it has served us well. However, it has a number of shortcomings that I'd like to address. Some of these issues are hard if not impossible to solve incrementally inside of DeprecatedString, so instead of doing that, let's build a new String class and then incrementally move over to it instead. Problems in DeprecatedString: - It assumes string allocation never fails. This makes it impossible to use in allocation-sensitive contexts, and is the reason we had to ban DeprecatedString from the kernel entirely. - The awkward null state. DeprecatedString can be null. It's different from the empty state, although null strings are considered empty. All code is immediately nicer when using Optional<DeprecatedString> but DeprecatedString came before Optional, which is how we ended up like this. - The encoding of the underlying data is ambiguous. For the most part, we use it as if it's always UTF-8, but there have been cases where we pass around strings in other encodings (e.g ISO8859-1) - operator[] and length() are used to iterate over DeprecatedString one byte at a time. This is done all over the codebase, and will *not* give the right results unless the string is all ASCII. How we solve these issues in the new String: - Functions that may allocate now return ErrorOr<String> so that ENOMEM errors can be passed to the caller. - String has no null state. Use Optional<String> when needed. - String is always UTF-8. This is validated when constructing a String. We may need to add a bypass for this in the future, for cases where you have a known-good string, but for now: validate all the things! - There is no operator[] or length(). You can get the underlying data with bytes(), but for iterating over code points, you should be using an UTF-8 iterator. Furthermore, it has two nifty new features: - String implements a small string optimization (SSO) for strings that can fit entirely within a pointer. This means up to 3 bytes on 32-bit platforms, and 7 bytes on 64-bit platforms. Such small strings will not be heap-allocated. - String can create substrings without making a deep copy of the substring. Instead, the superstring gets +1 refcount from the substring, and it acts like a view into the superstring. To make substrings like this, use the substring_with_shared_superstring() API. One caveat: - String does not guarantee that the underlying data is null-terminated like DeprecatedString does today. While this was nifty in a handful of places where we were calling C functions, it did stand in the way of shared-superstring substrings.
2022-12-01 12:27:43 +00:00
[[nodiscard]] bool operator==(String const&) const;
[[nodiscard]] bool operator!=(String const& other) const { return !(*this == other); }
AK: Implement FlyString for the new String class This implements a FlyString that will de-duplicate String instances. The FlyString will store the raw encoded data of the String instance: If the String is a short string, FlyString holds the String::ShortString bytes; otherwise FlyString holds a pointer to the Detail::StringData. FlyString itself does not know about String's storage or how to refcount its Detail::StringData. It defers to String to implement these details.
2023-01-11 13:26:49 +00:00
[[nodiscard]] bool operator==(FlyString const&) const;
[[nodiscard]] bool operator!=(FlyString const& other) const { return !(*this == other); }
AK: Introduce the new String, replacement for DeprecatedString DeprecatedString (formerly String) has been with us since the start, and it has served us well. However, it has a number of shortcomings that I'd like to address. Some of these issues are hard if not impossible to solve incrementally inside of DeprecatedString, so instead of doing that, let's build a new String class and then incrementally move over to it instead. Problems in DeprecatedString: - It assumes string allocation never fails. This makes it impossible to use in allocation-sensitive contexts, and is the reason we had to ban DeprecatedString from the kernel entirely. - The awkward null state. DeprecatedString can be null. It's different from the empty state, although null strings are considered empty. All code is immediately nicer when using Optional<DeprecatedString> but DeprecatedString came before Optional, which is how we ended up like this. - The encoding of the underlying data is ambiguous. For the most part, we use it as if it's always UTF-8, but there have been cases where we pass around strings in other encodings (e.g ISO8859-1) - operator[] and length() are used to iterate over DeprecatedString one byte at a time. This is done all over the codebase, and will *not* give the right results unless the string is all ASCII. How we solve these issues in the new String: - Functions that may allocate now return ErrorOr<String> so that ENOMEM errors can be passed to the caller. - String has no null state. Use Optional<String> when needed. - String is always UTF-8. This is validated when constructing a String. We may need to add a bypass for this in the future, for cases where you have a known-good string, but for now: validate all the things! - There is no operator[] or length(). You can get the underlying data with bytes(), but for iterating over code points, you should be using an UTF-8 iterator. Furthermore, it has two nifty new features: - String implements a small string optimization (SSO) for strings that can fit entirely within a pointer. This means up to 3 bytes on 32-bit platforms, and 7 bytes on 64-bit platforms. Such small strings will not be heap-allocated. - String can create substrings without making a deep copy of the substring. Instead, the superstring gets +1 refcount from the substring, and it acts like a view into the superstring. To make substrings like this, use the substring_with_shared_superstring() API. One caveat: - String does not guarantee that the underlying data is null-terminated like DeprecatedString does today. While this was nifty in a handful of places where we were calling C functions, it did stand in the way of shared-superstring substrings.
2022-12-01 12:27:43 +00:00
[[nodiscard]] bool operator==(StringView) const;
[[nodiscard]] bool operator!=(StringView other) const { return !(*this == other); }
[[nodiscard]] bool operator==(char const* cstring) const;
[[nodiscard]] bool operator!=(char const* cstring) const { return !(*this == cstring); }
AK: Add comparison operator s p a c e s h i p o p e r a t o r Comparing UTF-8 can be done by simple byte lexicographic comparison per definition, so we just piggy-back on StringView's high-performance comparator.
2022-12-11 13:31:41 +00:00
// NOTE: UTF-8 is defined in a way that lexicographic ordering of code points is equivalent to lexicographic ordering of bytes.
[[nodiscard]] int operator<=>(String const& other) const { return this->bytes_as_string_view().compare(other.bytes_as_string_view()); }
AK: Add String::is_one_of for variadic string comparison
2023-01-14 14:59:18 +00:00
template<typename... Ts>
[[nodiscard]] ALWAYS_INLINE constexpr bool is_one_of(Ts&&... strings) const
{
return (this->operator==(forward<Ts>(strings)) || ...);
}
AK: Add String::contains
2023-01-14 15:17:32 +00:00
[[nodiscard]] bool contains(StringView, CaseSensitivity = CaseSensitivity::CaseSensitive) const;
AK: Make String::contains(code_point) handle non-ASCII We currently only accept a char, instead of a full code point.
2023-03-08 14:06:59 +00:00
[[nodiscard]] bool contains(u32, CaseSensitivity = CaseSensitivity::CaseSensitive) const;
AK: Add String::contains
2023-01-14 15:17:32 +00:00
AK: Introduce the new String, replacement for DeprecatedString DeprecatedString (formerly String) has been with us since the start, and it has served us well. However, it has a number of shortcomings that I'd like to address. Some of these issues are hard if not impossible to solve incrementally inside of DeprecatedString, so instead of doing that, let's build a new String class and then incrementally move over to it instead. Problems in DeprecatedString: - It assumes string allocation never fails. This makes it impossible to use in allocation-sensitive contexts, and is the reason we had to ban DeprecatedString from the kernel entirely. - The awkward null state. DeprecatedString can be null. It's different from the empty state, although null strings are considered empty. All code is immediately nicer when using Optional<DeprecatedString> but DeprecatedString came before Optional, which is how we ended up like this. - The encoding of the underlying data is ambiguous. For the most part, we use it as if it's always UTF-8, but there have been cases where we pass around strings in other encodings (e.g ISO8859-1) - operator[] and length() are used to iterate over DeprecatedString one byte at a time. This is done all over the codebase, and will *not* give the right results unless the string is all ASCII. How we solve these issues in the new String: - Functions that may allocate now return ErrorOr<String> so that ENOMEM errors can be passed to the caller. - String has no null state. Use Optional<String> when needed. - String is always UTF-8. This is validated when constructing a String. We may need to add a bypass for this in the future, for cases where you have a known-good string, but for now: validate all the things! - There is no operator[] or length(). You can get the underlying data with bytes(), but for iterating over code points, you should be using an UTF-8 iterator. Furthermore, it has two nifty new features: - String implements a small string optimization (SSO) for strings that can fit entirely within a pointer. This means up to 3 bytes on 32-bit platforms, and 7 bytes on 64-bit platforms. Such small strings will not be heap-allocated. - String can create substrings without making a deep copy of the substring. Instead, the superstring gets +1 refcount from the substring, and it acts like a view into the superstring. To make substrings like this, use the substring_with_shared_superstring() API. One caveat: - String does not guarantee that the underlying data is null-terminated like DeprecatedString does today. While this was nifty in a handful of places where we were calling C functions, it did stand in the way of shared-superstring substrings.
2022-12-01 12:27:43 +00:00
[[nodiscard]] u32 hash() const;
Everywhere: Use C++ concepts instead of requires clauses
2022-11-14 18:20:59 +00:00
template<Arithmetic T>
AK: Introduce the new String, replacement for DeprecatedString DeprecatedString (formerly String) has been with us since the start, and it has served us well. However, it has a number of shortcomings that I'd like to address. Some of these issues are hard if not impossible to solve incrementally inside of DeprecatedString, so instead of doing that, let's build a new String class and then incrementally move over to it instead. Problems in DeprecatedString: - It assumes string allocation never fails. This makes it impossible to use in allocation-sensitive contexts, and is the reason we had to ban DeprecatedString from the kernel entirely. - The awkward null state. DeprecatedString can be null. It's different from the empty state, although null strings are considered empty. All code is immediately nicer when using Optional<DeprecatedString> but DeprecatedString came before Optional, which is how we ended up like this. - The encoding of the underlying data is ambiguous. For the most part, we use it as if it's always UTF-8, but there have been cases where we pass around strings in other encodings (e.g ISO8859-1) - operator[] and length() are used to iterate over DeprecatedString one byte at a time. This is done all over the codebase, and will *not* give the right results unless the string is all ASCII. How we solve these issues in the new String: - Functions that may allocate now return ErrorOr<String> so that ENOMEM errors can be passed to the caller. - String has no null state. Use Optional<String> when needed. - String is always UTF-8. This is validated when constructing a String. We may need to add a bypass for this in the future, for cases where you have a known-good string, but for now: validate all the things! - There is no operator[] or length(). You can get the underlying data with bytes(), but for iterating over code points, you should be using an UTF-8 iterator. Furthermore, it has two nifty new features: - String implements a small string optimization (SSO) for strings that can fit entirely within a pointer. This means up to 3 bytes on 32-bit platforms, and 7 bytes on 64-bit platforms. Such small strings will not be heap-allocated. - String can create substrings without making a deep copy of the substring. Instead, the superstring gets +1 refcount from the substring, and it acts like a view into the superstring. To make substrings like this, use the substring_with_shared_superstring() API. One caveat: - String does not guarantee that the underlying data is null-terminated like DeprecatedString does today. While this was nifty in a handful of places where we were calling C functions, it did stand in the way of shared-superstring substrings.
2022-12-01 12:27:43 +00:00
static ErrorOr<String> number(T value)
{
return formatted("{}", value);
}
AK: Add String::to_number
2023-01-14 15:23:44 +00:00
template<Arithmetic T>
Optional<T> to_number(TrimWhitespace trim_whitespace = TrimWhitespace::Yes) const
{
if constexpr (IsSigned<T>)
return StringUtils::convert_to_int<T>(bytes_as_string_view(), trim_whitespace);
else
return StringUtils::convert_to_uint<T>(bytes_as_string_view(), trim_whitespace);
}
AK: Introduce the new String, replacement for DeprecatedString DeprecatedString (formerly String) has been with us since the start, and it has served us well. However, it has a number of shortcomings that I'd like to address. Some of these issues are hard if not impossible to solve incrementally inside of DeprecatedString, so instead of doing that, let's build a new String class and then incrementally move over to it instead. Problems in DeprecatedString: - It assumes string allocation never fails. This makes it impossible to use in allocation-sensitive contexts, and is the reason we had to ban DeprecatedString from the kernel entirely. - The awkward null state. DeprecatedString can be null. It's different from the empty state, although null strings are considered empty. All code is immediately nicer when using Optional<DeprecatedString> but DeprecatedString came before Optional, which is how we ended up like this. - The encoding of the underlying data is ambiguous. For the most part, we use it as if it's always UTF-8, but there have been cases where we pass around strings in other encodings (e.g ISO8859-1) - operator[] and length() are used to iterate over DeprecatedString one byte at a time. This is done all over the codebase, and will *not* give the right results unless the string is all ASCII. How we solve these issues in the new String: - Functions that may allocate now return ErrorOr<String> so that ENOMEM errors can be passed to the caller. - String has no null state. Use Optional<String> when needed. - String is always UTF-8. This is validated when constructing a String. We may need to add a bypass for this in the future, for cases where you have a known-good string, but for now: validate all the things! - There is no operator[] or length(). You can get the underlying data with bytes(), but for iterating over code points, you should be using an UTF-8 iterator. Furthermore, it has two nifty new features: - String implements a small string optimization (SSO) for strings that can fit entirely within a pointer. This means up to 3 bytes on 32-bit platforms, and 7 bytes on 64-bit platforms. Such small strings will not be heap-allocated. - String can create substrings without making a deep copy of the substring. Instead, the superstring gets +1 refcount from the substring, and it acts like a view into the superstring. To make substrings like this, use the substring_with_shared_superstring() API. One caveat: - String does not guarantee that the underlying data is null-terminated like DeprecatedString does today. While this was nifty in a handful of places where we were calling C functions, it did stand in the way of shared-superstring substrings.
2022-12-01 12:27:43 +00:00
static ErrorOr<String> vformatted(StringView fmtstr, TypeErasedFormatParams&);
template<typename... Parameters>
static ErrorOr<String> formatted(CheckedFormatString<Parameters...>&& fmtstr, Parameters const&... parameters)
{
AK: Add support for "debug only" formatters These are formatters that can only be used with debug print functions, such as dbgln(). Currently this is limited to Formatter<ErrorOr<T>>. With this you can still debug log ErrorOr values (good for debugging), but trying to use them in any String::formatted() call will fail (which prevents .to_string() errors with the new failable strings being ignored). You make a formatter debug only by adding a constexpr method like: static constexpr bool is_debug_only() { return true; }
2023-01-13 02:19:40 +00:00
VariadicFormatParams<AllowDebugOnlyFormatters::No, Parameters...> variadic_format_parameters { parameters... };
AK: Introduce the new String, replacement for DeprecatedString DeprecatedString (formerly String) has been with us since the start, and it has served us well. However, it has a number of shortcomings that I'd like to address. Some of these issues are hard if not impossible to solve incrementally inside of DeprecatedString, so instead of doing that, let's build a new String class and then incrementally move over to it instead. Problems in DeprecatedString: - It assumes string allocation never fails. This makes it impossible to use in allocation-sensitive contexts, and is the reason we had to ban DeprecatedString from the kernel entirely. - The awkward null state. DeprecatedString can be null. It's different from the empty state, although null strings are considered empty. All code is immediately nicer when using Optional<DeprecatedString> but DeprecatedString came before Optional, which is how we ended up like this. - The encoding of the underlying data is ambiguous. For the most part, we use it as if it's always UTF-8, but there have been cases where we pass around strings in other encodings (e.g ISO8859-1) - operator[] and length() are used to iterate over DeprecatedString one byte at a time. This is done all over the codebase, and will *not* give the right results unless the string is all ASCII. How we solve these issues in the new String: - Functions that may allocate now return ErrorOr<String> so that ENOMEM errors can be passed to the caller. - String has no null state. Use Optional<String> when needed. - String is always UTF-8. This is validated when constructing a String. We may need to add a bypass for this in the future, for cases where you have a known-good string, but for now: validate all the things! - There is no operator[] or length(). You can get the underlying data with bytes(), but for iterating over code points, you should be using an UTF-8 iterator. Furthermore, it has two nifty new features: - String implements a small string optimization (SSO) for strings that can fit entirely within a pointer. This means up to 3 bytes on 32-bit platforms, and 7 bytes on 64-bit platforms. Such small strings will not be heap-allocated. - String can create substrings without making a deep copy of the substring. Instead, the superstring gets +1 refcount from the substring, and it acts like a view into the superstring. To make substrings like this, use the substring_with_shared_superstring() API. One caveat: - String does not guarantee that the underlying data is null-terminated like DeprecatedString does today. While this was nifty in a handful of places where we were calling C functions, it did stand in the way of shared-superstring substrings.
2022-12-01 12:27:43 +00:00
return vformatted(fmtstr.view(), variadic_format_parameters);
}
AK: Add String::join
2023-01-27 19:06:05 +00:00
template<class SeparatorType, class CollectionType>
static ErrorOr<String> join(SeparatorType const& separator, CollectionType const& collection, StringView fmtstr = "{}"sv)
{
StringBuilder builder;
TRY(builder.try_join(separator, collection, fmtstr));
return builder.to_string();
}
AK: Introduce the new String, replacement for DeprecatedString DeprecatedString (formerly String) has been with us since the start, and it has served us well. However, it has a number of shortcomings that I'd like to address. Some of these issues are hard if not impossible to solve incrementally inside of DeprecatedString, so instead of doing that, let's build a new String class and then incrementally move over to it instead. Problems in DeprecatedString: - It assumes string allocation never fails. This makes it impossible to use in allocation-sensitive contexts, and is the reason we had to ban DeprecatedString from the kernel entirely. - The awkward null state. DeprecatedString can be null. It's different from the empty state, although null strings are considered empty. All code is immediately nicer when using Optional<DeprecatedString> but DeprecatedString came before Optional, which is how we ended up like this. - The encoding of the underlying data is ambiguous. For the most part, we use it as if it's always UTF-8, but there have been cases where we pass around strings in other encodings (e.g ISO8859-1) - operator[] and length() are used to iterate over DeprecatedString one byte at a time. This is done all over the codebase, and will *not* give the right results unless the string is all ASCII. How we solve these issues in the new String: - Functions that may allocate now return ErrorOr<String> so that ENOMEM errors can be passed to the caller. - String has no null state. Use Optional<String> when needed. - String is always UTF-8. This is validated when constructing a String. We may need to add a bypass for this in the future, for cases where you have a known-good string, but for now: validate all the things! - There is no operator[] or length(). You can get the underlying data with bytes(), but for iterating over code points, you should be using an UTF-8 iterator. Furthermore, it has two nifty new features: - String implements a small string optimization (SSO) for strings that can fit entirely within a pointer. This means up to 3 bytes on 32-bit platforms, and 7 bytes on 64-bit platforms. Such small strings will not be heap-allocated. - String can create substrings without making a deep copy of the substring. Instead, the superstring gets +1 refcount from the substring, and it acts like a view into the superstring. To make substrings like this, use the substring_with_shared_superstring() API. One caveat: - String does not guarantee that the underlying data is null-terminated like DeprecatedString does today. While this was nifty in a handful of places where we were calling C functions, it did stand in the way of shared-superstring substrings.
2022-12-01 12:27:43 +00:00
// NOTE: This is primarily interesting to unit tests.
[[nodiscard]] bool is_short_string() const;
AK: Implement FlyString for the new String class This implements a FlyString that will de-duplicate String instances. The FlyString will store the raw encoded data of the String instance: If the String is a short string, FlyString holds the String::ShortString bytes; otherwise FlyString holds a pointer to the Detail::StringData. FlyString itself does not know about String's storage or how to refcount its Detail::StringData. It defers to String to implement these details.
2023-01-11 13:26:49 +00:00
[[nodiscard]] static String fly_string_data_to_string(Badge<FlyString>, uintptr_t const&);
[[nodiscard]] static StringView fly_string_data_to_string_view(Badge<FlyString>, uintptr_t const&);
AK: Make FlyString::hash() use the cached hash in StringData if possible This avoids rehashing the string every time.
2023-03-08 22:11:59 +00:00
[[nodiscard]] static u32 fly_string_data_to_hash(Badge<FlyString>, uintptr_t const&);
AK: Implement FlyString for the new String class This implements a FlyString that will de-duplicate String instances. The FlyString will store the raw encoded data of the String instance: If the String is a short string, FlyString holds the String::ShortString bytes; otherwise FlyString holds a pointer to the Detail::StringData. FlyString itself does not know about String's storage or how to refcount its Detail::StringData. It defers to String to implement these details.
2023-01-11 13:26:49 +00:00
[[nodiscard]] uintptr_t to_fly_string_data(Badge<FlyString>) const;
static void ref_fly_string_data(Badge<FlyString>, uintptr_t);
static void unref_fly_string_data(Badge<FlyString>, uintptr_t);
void did_create_fly_string(Badge<FlyString>) const;
AK: Introduce the new String, replacement for DeprecatedString DeprecatedString (formerly String) has been with us since the start, and it has served us well. However, it has a number of shortcomings that I'd like to address. Some of these issues are hard if not impossible to solve incrementally inside of DeprecatedString, so instead of doing that, let's build a new String class and then incrementally move over to it instead. Problems in DeprecatedString: - It assumes string allocation never fails. This makes it impossible to use in allocation-sensitive contexts, and is the reason we had to ban DeprecatedString from the kernel entirely. - The awkward null state. DeprecatedString can be null. It's different from the empty state, although null strings are considered empty. All code is immediately nicer when using Optional<DeprecatedString> but DeprecatedString came before Optional, which is how we ended up like this. - The encoding of the underlying data is ambiguous. For the most part, we use it as if it's always UTF-8, but there have been cases where we pass around strings in other encodings (e.g ISO8859-1) - operator[] and length() are used to iterate over DeprecatedString one byte at a time. This is done all over the codebase, and will *not* give the right results unless the string is all ASCII. How we solve these issues in the new String: - Functions that may allocate now return ErrorOr<String> so that ENOMEM errors can be passed to the caller. - String has no null state. Use Optional<String> when needed. - String is always UTF-8. This is validated when constructing a String. We may need to add a bypass for this in the future, for cases where you have a known-good string, but for now: validate all the things! - There is no operator[] or length(). You can get the underlying data with bytes(), but for iterating over code points, you should be using an UTF-8 iterator. Furthermore, it has two nifty new features: - String implements a small string optimization (SSO) for strings that can fit entirely within a pointer. This means up to 3 bytes on 32-bit platforms, and 7 bytes on 64-bit platforms. Such small strings will not be heap-allocated. - String can create substrings without making a deep copy of the substring. Instead, the superstring gets +1 refcount from the substring, and it acts like a view into the superstring. To make substrings like this, use the substring_with_shared_superstring() API. One caveat: - String does not guarantee that the underlying data is null-terminated like DeprecatedString does today. While this was nifty in a handful of places where we were calling C functions, it did stand in the way of shared-superstring substrings.
2022-12-01 12:27:43 +00:00
// FIXME: Remove these once all code has been ported to String
[[nodiscard]] DeprecatedString to_deprecated_string() const;
static ErrorOr<String> from_deprecated_string(DeprecatedString const&);
private:
// NOTE: If the least significant bit of the pointer is set, this is a short string.
static constexpr uintptr_t SHORT_STRING_FLAG = 1;
AK: Implement FlyString for the new String class This implements a FlyString that will de-duplicate String instances. The FlyString will store the raw encoded data of the String instance: If the String is a short string, FlyString holds the String::ShortString bytes; otherwise FlyString holds a pointer to the Detail::StringData. FlyString itself does not know about String's storage or how to refcount its Detail::StringData. It defers to String to implement these details.
2023-01-11 13:26:49 +00:00
static constexpr bool has_short_string_bit(uintptr_t data)
{
return (data & SHORT_STRING_FLAG) != 0;
}
AK: Introduce the new String, replacement for DeprecatedString DeprecatedString (formerly String) has been with us since the start, and it has served us well. However, it has a number of shortcomings that I'd like to address. Some of these issues are hard if not impossible to solve incrementally inside of DeprecatedString, so instead of doing that, let's build a new String class and then incrementally move over to it instead. Problems in DeprecatedString: - It assumes string allocation never fails. This makes it impossible to use in allocation-sensitive contexts, and is the reason we had to ban DeprecatedString from the kernel entirely. - The awkward null state. DeprecatedString can be null. It's different from the empty state, although null strings are considered empty. All code is immediately nicer when using Optional<DeprecatedString> but DeprecatedString came before Optional, which is how we ended up like this. - The encoding of the underlying data is ambiguous. For the most part, we use it as if it's always UTF-8, but there have been cases where we pass around strings in other encodings (e.g ISO8859-1) - operator[] and length() are used to iterate over DeprecatedString one byte at a time. This is done all over the codebase, and will *not* give the right results unless the string is all ASCII. How we solve these issues in the new String: - Functions that may allocate now return ErrorOr<String> so that ENOMEM errors can be passed to the caller. - String has no null state. Use Optional<String> when needed. - String is always UTF-8. This is validated when constructing a String. We may need to add a bypass for this in the future, for cases where you have a known-good string, but for now: validate all the things! - There is no operator[] or length(). You can get the underlying data with bytes(), but for iterating over code points, you should be using an UTF-8 iterator. Furthermore, it has two nifty new features: - String implements a small string optimization (SSO) for strings that can fit entirely within a pointer. This means up to 3 bytes on 32-bit platforms, and 7 bytes on 64-bit platforms. Such small strings will not be heap-allocated. - String can create substrings without making a deep copy of the substring. Instead, the superstring gets +1 refcount from the substring, and it acts like a view into the superstring. To make substrings like this, use the substring_with_shared_superstring() API. One caveat: - String does not guarantee that the underlying data is null-terminated like DeprecatedString does today. While this was nifty in a handful of places where we were calling C functions, it did stand in the way of shared-superstring substrings.
2022-12-01 12:27:43 +00:00
struct ShortString {
ReadonlyBytes bytes() const;
size_t byte_count() const;
// NOTE: This is the byte count shifted left 1 step and or'ed with a 1 (the SHORT_STRING_FLAG)
u8 byte_count_and_short_string_flag { 0 };
u8 storage[MAX_SHORT_STRING_BYTE_COUNT] = { 0 };
};
AK: Make String const-correct internally
2023-02-19 22:00:24 +00:00
explicit String(NonnullRefPtr<Detail::StringData const>);
AK: Support creating known short string literals at compile time In cases where we know a string literal will fit in the short string storage, we can do so at compile time without needing to handle error propagation. If the provided string literal is too long, a compilation error will be emitted due to the failed VERIFY statement being a non- constant expression.
2023-01-20 12:20:01 +00:00
explicit constexpr String(ShortString short_string)
: m_short_string(short_string)
{
}
void destroy_string();
AK: Introduce the new String, replacement for DeprecatedString DeprecatedString (formerly String) has been with us since the start, and it has served us well. However, it has a number of shortcomings that I'd like to address. Some of these issues are hard if not impossible to solve incrementally inside of DeprecatedString, so instead of doing that, let's build a new String class and then incrementally move over to it instead. Problems in DeprecatedString: - It assumes string allocation never fails. This makes it impossible to use in allocation-sensitive contexts, and is the reason we had to ban DeprecatedString from the kernel entirely. - The awkward null state. DeprecatedString can be null. It's different from the empty state, although null strings are considered empty. All code is immediately nicer when using Optional<DeprecatedString> but DeprecatedString came before Optional, which is how we ended up like this. - The encoding of the underlying data is ambiguous. For the most part, we use it as if it's always UTF-8, but there have been cases where we pass around strings in other encodings (e.g ISO8859-1) - operator[] and length() are used to iterate over DeprecatedString one byte at a time. This is done all over the codebase, and will *not* give the right results unless the string is all ASCII. How we solve these issues in the new String: - Functions that may allocate now return ErrorOr<String> so that ENOMEM errors can be passed to the caller. - String has no null state. Use Optional<String> when needed. - String is always UTF-8. This is validated when constructing a String. We may need to add a bypass for this in the future, for cases where you have a known-good string, but for now: validate all the things! - There is no operator[] or length(). You can get the underlying data with bytes(), but for iterating over code points, you should be using an UTF-8 iterator. Furthermore, it has two nifty new features: - String implements a small string optimization (SSO) for strings that can fit entirely within a pointer. This means up to 3 bytes on 32-bit platforms, and 7 bytes on 64-bit platforms. Such small strings will not be heap-allocated. - String can create substrings without making a deep copy of the substring. Instead, the superstring gets +1 refcount from the substring, and it acts like a view into the superstring. To make substrings like this, use the substring_with_shared_superstring() API. One caveat: - String does not guarantee that the underlying data is null-terminated like DeprecatedString does today. While this was nifty in a handful of places where we were calling C functions, it did stand in the way of shared-superstring substrings.
2022-12-01 12:27:43 +00:00
union {
ShortString m_short_string;
AK: Make String const-correct internally
2023-02-19 22:00:24 +00:00
Detail::StringData const* m_data { nullptr };
AK: Introduce the new String, replacement for DeprecatedString DeprecatedString (formerly String) has been with us since the start, and it has served us well. However, it has a number of shortcomings that I'd like to address. Some of these issues are hard if not impossible to solve incrementally inside of DeprecatedString, so instead of doing that, let's build a new String class and then incrementally move over to it instead. Problems in DeprecatedString: - It assumes string allocation never fails. This makes it impossible to use in allocation-sensitive contexts, and is the reason we had to ban DeprecatedString from the kernel entirely. - The awkward null state. DeprecatedString can be null. It's different from the empty state, although null strings are considered empty. All code is immediately nicer when using Optional<DeprecatedString> but DeprecatedString came before Optional, which is how we ended up like this. - The encoding of the underlying data is ambiguous. For the most part, we use it as if it's always UTF-8, but there have been cases where we pass around strings in other encodings (e.g ISO8859-1) - operator[] and length() are used to iterate over DeprecatedString one byte at a time. This is done all over the codebase, and will *not* give the right results unless the string is all ASCII. How we solve these issues in the new String: - Functions that may allocate now return ErrorOr<String> so that ENOMEM errors can be passed to the caller. - String has no null state. Use Optional<String> when needed. - String is always UTF-8. This is validated when constructing a String. We may need to add a bypass for this in the future, for cases where you have a known-good string, but for now: validate all the things! - There is no operator[] or length(). You can get the underlying data with bytes(), but for iterating over code points, you should be using an UTF-8 iterator. Furthermore, it has two nifty new features: - String implements a small string optimization (SSO) for strings that can fit entirely within a pointer. This means up to 3 bytes on 32-bit platforms, and 7 bytes on 64-bit platforms. Such small strings will not be heap-allocated. - String can create substrings without making a deep copy of the substring. Instead, the superstring gets +1 refcount from the substring, and it acts like a view into the superstring. To make substrings like this, use the substring_with_shared_superstring() API. One caveat: - String does not guarantee that the underlying data is null-terminated like DeprecatedString does today. While this was nifty in a handful of places where we were calling C functions, it did stand in the way of shared-superstring substrings.
2022-12-01 12:27:43 +00:00
};
};
template<>
struct Traits<String> : public GenericTraits<String> {
static unsigned hash(String const&);
};
template<>
struct Formatter<String> : Formatter<StringView> {
ErrorOr<void> format(FormatBuilder&, String const&);
};
}
AK: Add operator""_{short_,}string to create a String from a literal We briefly discussed this when adding the new String type but couldn't settle on a name. However, having to use String::from_utf8() on every literal string is a bit unwieldy, so let's have these options available! Naming-wise '_string' is not as short as 'sv' but should be relatively clear; it also matches '_bigint' and '_ubigint' in length. '_short_string' may be longer than the actual string itself, but it's still an improvement over the static function :^) Since our C++ source files are UTF-8 encoded anyway, it should be impossible to create a string literal with invalid UTF-8, so including that in the name is not as important as in the function that can receive arbitrary data.
2023-02-25 15:14:37 +00:00
[[nodiscard]] ALWAYS_INLINE AK::ErrorOr<AK::String> operator""_string(char const* cstring, size_t length)
{
return AK::String::from_utf8(AK::StringView(cstring, length));
}
AK: Use the same consteval condition on _short_string as its factory This fixes the build with Apple Clang.
2023-02-25 20:01:44 +00:00
[[nodiscard]] ALWAYS_INLINE AK_SHORT_STRING_CONSTEVAL AK::String operator""_short_string(char const* cstring, size_t length)
AK: Add operator""_{short_,}string to create a String from a literal We briefly discussed this when adding the new String type but couldn't settle on a name. However, having to use String::from_utf8() on every literal string is a bit unwieldy, so let's have these options available! Naming-wise '_string' is not as short as 'sv' but should be relatively clear; it also matches '_bigint' and '_ubigint' in length. '_short_string' may be longer than the actual string itself, but it's still an improvement over the static function :^) Since our C++ source files are UTF-8 encoded anyway, it should be impossible to create a string literal with invalid UTF-8, so including that in the name is not as important as in the function that can receive arbitrary data.
2023-02-25 15:14:37 +00:00
{
return AK::String::from_utf8_short_string(AK::StringView(cstring, length));
}
Reference in a new issue Copy permalink