beenull/ladybird
1
0
Fork 0
mirror of https://github.com/LadybirdBrowser/ladybird.git synced 2024-11-22 15:40:19 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 6
ladybird/Libraries/LibUnicode/CharacterTypes.h

76 lines
2.6 KiB
C
Raw Normal View History

LibUnicode: Introduce a Unicode library for interacting with UCD files The Unicode standard publishes the Unicode Character Database (UCD) with information about every code point, such as each code point's upper case mapping. LibUnicode exists to download and parse UCD files at build time and to provide accessors to that data. As a start, LibUnicode includes upper- and lower-case code point converters.
2021-07-25 19:10:51 +00:00
/*
LibUnicode: Replace case transformations and comparison with ICUs There are a couple of differences here due to using ICU: 1. Titlecasing behaves slightly differently. We previously transformed "123dollars" to "123Dollars", as we would use word segmentation to split a string into words, then transform the first cased character to titlecase. ICU doesn't go quite that far, and leaves the string as "123dollars". While this is a behavior change, the only user of this API is the `text-transform: capitalize;` CSS rule, and we now match the behavior of other browsers. 2. There isn't an API to compare strings with case insensitivity without allocating case-folded strings for both the left- and right-hand-side strings. Our implementation was previously allocation-free; however, in a benchmark, ICU is still ~1.4x faster.
2024-06-19 20:39:30 +00:00
* Copyright (c) 2021-2024, Tim Flynn <trflynn89@serenityos.org>
LibUnicode: Introduce a Unicode library for interacting with UCD files The Unicode standard publishes the Unicode Character Database (UCD) with information about every code point, such as each code point's upper case mapping. LibUnicode exists to download and parse UCD files at build time and to provide accessors to that data. As a start, LibUnicode includes upper- and lower-case code point converters.
2021-07-25 19:10:51 +00:00
*
* SPDX-License-Identifier: BSD-2-Clause
*/
#pragma once
LibUnicode: Begin implementing special Unicode case folding This implements unconditional special case folding, and conditional folding for non-locale cases. Worth noting that the only conditional, non-locale special case is for converting an uppercase sigma to lowercase.
2021-07-27 17:46:08 +00:00
#include <AK/Forward.h>
LibUnicode: Implement locale-aware AFTER_I special casing
2021-09-05 18:39:20 +00:00
#include <AK/Optional.h>
LibUnicode: Replace case transformations and comparison with ICUs There are a couple of differences here due to using ICU: 1. Titlecasing behaves slightly differently. We previously transformed "123dollars" to "123Dollars", as we would use word segmentation to split a string into words, then transform the first cased character to titlecase. ICU doesn't go quite that far, and leaves the string as "123dollars". While this is a behavior change, the only user of this API is the `text-transform: capitalize;` CSS rule, and we now match the behavior of other browsers. 2. There isn't an API to compare strings with case insensitivity without allocating case-folded strings for both the left- and right-hand-side strings. Our implementation was previously allocation-free; however, in a benchmark, ICU is still ~1.4x faster.
2024-06-19 20:39:30 +00:00
#include <AK/StringView.h>
LibUnicode: Introduce a Unicode library for interacting with UCD files The Unicode standard publishes the Unicode Character Database (UCD) with information about every code point, such as each code point's upper case mapping. LibUnicode exists to download and parse UCD files at build time and to provide accessors to that data. As a start, LibUnicode includes upper- and lower-case code point converters.
2021-07-25 19:10:51 +00:00
#include <AK/Types.h>
LibUnicode: Add public methods to compare and lookup Unicode properties Adds methods to retrieve a Unicode property from a string and to check if a code point matches a Unicode property. Also adds a <LibUnicode/Forward.h> header.
2021-07-29 01:45:09 +00:00
#include <LibUnicode/Forward.h>
LibUnicode: Introduce a Unicode library for interacting with UCD files The Unicode standard publishes the Unicode Character Database (UCD) with information about every code point, such as each code point's upper case mapping. LibUnicode exists to download and parse UCD files at build time and to provide accessors to that data. As a start, LibUnicode includes upper- and lower-case code point converters.
2021-07-25 19:10:51 +00:00
namespace Unicode {
Everywhere: Pass AK::StringView by value
2021-11-10 23:55:02 +00:00
Optional<GeneralCategory> general_category_from_string(StringView);
LibUnicode: Add public methods to compare and lookup General Categories Adds methods to retrieve a General Category from a string and to check if a code point matches a General Category.
2021-07-31 20:38:24 +00:00
bool code_point_has_general_category(u32 code_point, GeneralCategory general_category);
LibUnicode: Add a helper to determine if a code point is printable
2024-10-21 20:29:16 +00:00
bool code_point_is_printable(u32 code_point);
LibUnicode: Replace code point general categories with ICU
2024-06-21 15:24:00 +00:00
bool code_point_has_control_general_category(u32 code_point);
LibWeb: Only fire keypress events if the key press produced a character For example, pressing just the shift key should not producde a keypress event.
2024-10-08 21:15:55 +00:00
bool code_point_has_letter_general_category(u32 code_point);
bool code_point_has_number_general_category(u32 code_point);
LibUnicode: Add code point GC queries for Punctuation and Separator
2024-09-05 16:07:59 +00:00
bool code_point_has_punctuation_general_category(u32 code_point);
bool code_point_has_separator_general_category(u32 code_point);
LibUnicode: Replace code point general categories with ICU
2024-06-21 15:24:00 +00:00
bool code_point_has_space_separator_general_category(u32 code_point);
LibWeb: Only fire keypress events if the key press produced a character For example, pressing just the shift key should not producde a keypress event.
2024-10-08 21:15:55 +00:00
bool code_point_has_symbol_general_category(u32 code_point);
LibUnicode: Replace code point general categories with ICU
2024-06-21 15:24:00 +00:00
Everywhere: Pass AK::StringView by value
2021-11-10 23:55:02 +00:00
Optional<Property> property_from_string(StringView);
LibUnicode: Add public methods to compare and lookup Unicode properties Adds methods to retrieve a Unicode property from a string and to check if a code point matches a Unicode property. Also adds a <LibUnicode/Forward.h> header.
2021-07-29 01:45:09 +00:00
bool code_point_has_property(u32 code_point, Property property);
LibUnicode: Replace code point properties with ICU
2024-06-21 14:39:40 +00:00
bool code_point_has_emoji_property(u32 code_point);
bool code_point_has_emoji_modifier_base_property(u32 code_point);
bool code_point_has_emoji_presentation_property(u32 code_point);
bool code_point_has_identifier_start_property(u32 code_point);
bool code_point_has_identifier_continue_property(u32 code_point);
bool code_point_has_regional_indicator_property(u32 code_point);
bool code_point_has_variation_selector_property(u32 code_point);
AK+LibUnicode: Add a method to trim non-ASCII whitespace from a String Required by WebDriver.
2024-11-03 22:13:56 +00:00
bool code_point_has_white_space_property(u32 code_point);
LibUnicode: Replace code point properties with ICU
2024-06-21 14:39:40 +00:00
LibRegex+LibUnicode: Begin implementing Unicode property escapes This supports some binary property matching. It does not support any properties not yet parsed by LibUnicode, nor does it support value matching (such as Script_Extensions=Latin).
2021-07-29 18:18:51 +00:00
bool is_ecma262_property(Property);
LibUnicode: Add public methods to compare and lookup Unicode properties Adds methods to retrieve a Unicode property from a string and to check if a code point matches a Unicode property. Also adds a <LibUnicode/Forward.h> header.
2021-07-29 01:45:09 +00:00
Everywhere: Pass AK::StringView by value
2021-11-10 23:55:02 +00:00
Optional<Script> script_from_string(StringView);
LibUnicode: Parse UCD Scripts.txt and generate as a Unicode property There are a couple of minor nuances with parsing script values, compared to other properties. In Scripts.txt, the UCD file lists the full name of each script; other properties, like General Category, list the shorter name in their primary files. This means that the aliases listed in PropertyValueAliases.txt are reversed for script values.
2021-08-03 21:11:19 +00:00
bool code_point_has_script(u32 code_point, Script script);
LibUnicode: Parse UCD ScriptExtensions.txt and generate property
2021-08-04 11:05:30 +00:00
bool code_point_has_script_extension(u32 code_point, Script script);
LibUnicode: Parse UCD Scripts.txt and generate as a Unicode property There are a couple of minor nuances with parsing script values, compared to other properties. In Scripts.txt, the UCD file lists the full name of each script; other properties, like General Category, list the shorter name in their primary files. This means that the aliases listed in PropertyValueAliases.txt are reversed for script values.
2021-08-03 21:11:19 +00:00
LibUnicode: Replace code point bidirectional classes with ICU
2024-06-21 19:28:54 +00:00
enum class BidiClass {
ArabicNumber, // AN
BlockSeparator, // B
BoundaryNeutral, // BN
CommonNumberSeparator, // CS
DirNonSpacingMark, // NSM
EuropeanNumber, // EN
EuropeanNumberSeparator, // ES
EuropeanNumberTerminator, // ET
FirstStrongIsolate, // FSI
LeftToRight, // L
LeftToRightEmbedding, // LRE
LeftToRightIsolate, // LRI
LeftToRightOverride, // LRO
OtherNeutral, // ON
PopDirectionalFormat, // PDF
PopDirectionalIsolate, // PDI
RightToLeft, // R
RightToLeftArabic, // AL
RightToLeftEmbedding, // RLE
RightToLeftIsolate, // RLI
RightToLeftOverride, // RLO
SegmentSeparator, // S
WhiteSpaceNeutral, // WS
};
BidiClass bidirectional_class(u32 code_point);
LibUnicode: Generate data for bidirectional character types This will let us examine code points to determine the rtl/ltr direction of a piece of text.
2023-08-12 20:00:58 +00:00
LibUnicode: Introduce a Unicode library for interacting with UCD files The Unicode standard publishes the Unicode Character Database (UCD) with information about every code point, such as each code point's upper case mapping. LibUnicode exists to download and parse UCD files at build time and to provide accessors to that data. As a start, LibUnicode includes upper- and lower-case code point converters.
2021-07-25 19:10:51 +00:00
}
Reference in a new issue Copy permalink