浏览代码

LibWeb/CSS: Tokenize comments as whitespace tokens

This is in a weird position where the spec tells us to discard the
comments, but then we have to preserve the original source text which
may include comments. As a compromise, I'm treating each comment as a
whitespace token - comments are functionally equivalent to whitespace
so this should not have any behaviour changes beyond preserving the
original text.
Sam Atkins 9 月之前
父节点
当前提交
f8995d37a2
共有 1 个文件被更改,包括 10 次插入1 次删除
  1. 10 1
      Userland/Libraries/LibWeb/CSS/Parser/Tokenizer.cpp

+ 10 - 1
Userland/Libraries/LibWeb/CSS/Parser/Tokenizer.cpp

@@ -1064,11 +1064,20 @@ Token Tokenizer::consume_a_token()
     // This section describes how to consume a token from a stream of code points.
     // It will return a single token of any type.
 
+    auto start_byte_offset = current_byte_offset();
+
     // Consume comments.
     consume_comments();
 
+    // AD-HOC: Preserve comments as whitespace tokens, for serializing custom properties.
+    auto after_comments_byte_offset = current_byte_offset();
+    if (after_comments_byte_offset != start_byte_offset) {
+        auto token = create_new_token(Token::Type::Whitespace);
+        token.m_original_source_text = input_since(start_byte_offset);
+        return token;
+    }
+
     // Consume the next input code point.
-    auto start_byte_offset = current_byte_offset();
     auto input = next_code_point();
 
     // whitespace