beenull/ladybird
1
0
Fork 0
mirror of https://github.com/LadybirdBrowser/ladybird.git synced 2024-11-22 07:30:19 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 6

LibWeb/CSS: Tokenize comments as whitespace tokens

Browse source 
This is in a weird position where the spec tells us to discard the
comments, but then we have to preserve the original source text which
may include comments. As a compromise, I'm treating each comment as a
whitespace token - comments are functionally equivalent to whitespace
so this should not have any behaviour changes beyond preserving the
original text.
This commit is contained in:
Sam Atkins 2024-10-14 16:18:57 +01:00 committed by Andreas Kling
parent ea164124de
commit f8995d37a2
Notes: github-actions[bot] 2024-10-16 12:23:41 +00:00 
Author: https://github.com/AtkinsSJ
Commit: https://github.com/LadybirdBrowser/ladybird/commit/f8995d37a2b
Pull-request: https://github.com/LadybirdBrowser/ladybird/pull/1795
1 changed files with 10 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

11
Userland/Libraries/LibWeb/CSS/Parser/Tokenizer.cpp
View file

@ -1064,11 +1064,20 @@ Token Tokenizer::consume_a_token()
// This section describes how to consume a token from a stream of code points.
// It will return a single token of any type.
auto start_byte_offset = current_byte_offset();
// Consume comments.
consume_comments();
// AD-HOC: Preserve comments as whitespace tokens, for serializing custom properties.
auto after_comments_byte_offset = current_byte_offset();
if (after_comments_byte_offset != start_byte_offset) {
auto token = create_new_token(Token::Type::Whitespace);
token.m_original_source_text = input_since(start_byte_offset);
return token;
}
// Consume the next input code point.
auto start_byte_offset = current_byte_offset();
auto input = next_code_point();
// whitespace