beenull/ladybird
1
0
Fork 0
mirror of https://github.com/LadybirdBrowser/ladybird.git synced 2024-11-21 23:20:20 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 6
ladybird/Libraries/LibCompress/Gzip.h

110 lines
3 KiB
C
Raw Normal View History

LibCompress: Implement gzip.
2020-08-28 15:53:57 +00:00
/*
Libraries: Use default constructors/destructors in LibCompress https://isocpp.github.io/CppCoreGuidelines/CppCoreGuidelines#cother-other-default-operation-rules "The compiler is more likely to get the default semantics right and you cannot implement these functions better than the compiler."
2022-02-26 16:06:40 +00:00
* Copyright (c) 2020-2022, the SerenityOS developers.
AK+Userland: Use idan.horowitz@serenityos.org for my copyright headers
2021-04-22 20:40:43 +00:00
* Copyright (c) 2021, Idan Horowitz <idan.horowitz@serenityos.org>
LibCompress: Implement gzip.
2020-08-28 15:53:57 +00:00
*
Everything: Move to SPDX license identifiers in all files. SPDX License Identifiers are a more compact / standardized way of representing file license information. See: https://spdx.dev/resources/use/#identifiers This was done with the `ambr` search and replace tool. ambr --no-parent-ignore --key-from-file --rep-from-file key.txt rep.txt *
2021-04-22 08:24:48 +00:00
* SPDX-License-Identifier: BSD-2-Clause
LibCompress: Implement gzip.
2020-08-28 15:53:57 +00:00
*/
#pragma once
LibCore: Remove `Stream.h`
2023-02-09 02:11:50 +00:00
#include <AK/NonnullOwnPtr.h>
#include <AK/OwnPtr.h>
#include <AK/Stream.h>
LibCompress: Implement gzip.
2020-08-28 15:53:57 +00:00
#include <LibCompress/Deflate.h>
#include <LibCrypto/Checksum/CRC32.h>
namespace Compress {
LibCore+LibHTTP+LibGfx: Switch to LibCompress This commit removes the only 3rd party library (and its usages) in serenity: puff, which is used for deflate decompression. and replaces it with the existing original serenity implementation in LibCompress. :^)
2021-03-03 21:54:07 +00:00
constexpr u8 gzip_magic_1 = 0x1f;
constexpr u8 gzip_magic_2 = 0x8b;
LibCompress: Implement GZip compression This commit implements a stream compressor for the gzip specification (RFC 1952), which is essentially a thin wrapper around the DEFLATE compression format.
2021-03-12 23:26:44 +00:00
struct [[gnu::packed]] BlockHeader {
u8 identification_1;
u8 identification_2;
u8 compression_method;
u8 flags;
LittleEndian<u32> modification_time;
u8 extra_flags;
u8 operating_system;
bool valid_magic_number() const;
bool supported_by_implementation() const;
};
struct Flags {
static constexpr u8 FTEXT = 1 << 0;
static constexpr u8 FHCRC = 1 << 1;
static constexpr u8 FEXTRA = 1 << 2;
static constexpr u8 FNAME = 1 << 3;
static constexpr u8 FCOMMENT = 1 << 4;
static constexpr u8 MAX = FTEXT | FHCRC | FEXTRA | FNAME | FCOMMENT;
};
LibCore+LibHTTP+LibGfx: Switch to LibCompress This commit removes the only 3rd party library (and its usages) in serenity: puff, which is used for deflate decompression. and replaces it with the existing original serenity implementation in LibCompress. :^)
2021-03-03 21:54:07 +00:00
Everywhere: Remove the `AK::` qualifier from Stream usages
2023-02-10 00:00:18 +00:00
class GzipDecompressor final : public Stream {
LibCompress: Implement gzip.
2020-08-28 15:53:57 +00:00
public:
LibCompress: Let `GzipDecompressor` take a `MaybeOwned<Stream>`
2023-06-08 16:25:36 +00:00
GzipDecompressor(MaybeOwned<Stream>);
LibCompress: Implement gzip.
2020-08-28 15:53:57 +00:00
~GzipDecompressor();
AK: Rename Stream::{read,write} to Stream::{read_some,write_some} Similar to POSIX read, the basic read and write functions of AK::Stream do not have a lower limit of how much data they read or write (apart from "none at all"). Rename the functions to "read some [data]" and "write some [data]" (with "data" being omitted, since everything here is reading and writing data) to make them sufficiently distinct from the functions that ensure to use the entire buffer (which should be the go-to function for most usages). No functional changes, just a lot of new FIXMEs.
2023-02-24 21:38:01 +00:00
virtual ErrorOr<Bytes> read_some(Bytes) override;
virtual ErrorOr<size_t> write_some(ReadonlyBytes) override;
LibCompress: Port GzipDecompressor to `Core::Stream`
2022-11-29 14:43:31 +00:00
virtual bool is_eof() const override;
virtual bool is_open() const override { return true; }
LibCompress: Allow using GzipCompressor in a streaming fashion GzipCompressor is currently written assuming that it's write_some method is only called once. When we use this class for LibWeb, we may very well receive data to compress in small chunks. So this patch makes us write the gzip header and footer only once, which now resembles the zlib and deflate compressors.
2024-11-15 20:39:02 +00:00
virtual void close() override { }
AK: Lower the requirements for InputStream::eof and rename it. Consider the following snippet: void foo(InputStream& stream) { if(!stream.eof()) { u8 byte; stream >> byte; } } There is a very subtle bug in this snippet, for some input streams eof() might return false even if no more data can be read. In this case an error flag would be set on the stream. Until now I've always ensured that this is not the case, but this made the implementation of eof() unnecessarily complicated. InputFileStream::eof had to keep a ByteBuffer around just to make this possible. That meant a ton of unnecessary copies just to get a reliable eof(). In most cases it isn't actually necessary to have a reliable eof() implementation. In most other cases a reliable eof() is avaliable anyways because in some cases like InputMemoryStream it is very easy to implement.
2020-09-13 10:24:17 +00:00
LibCompress: Port GzipDecompressor to `Core::Stream`
2022-11-29 14:43:31 +00:00
static ErrorOr<ByteBuffer> decompress_all(ReadonlyBytes);
gunzip+LibCompress: Move utility to decompress files to GzipDecompressor This is to allow re-using this method (and any optimization it receives) by other utilities, like gzip.
2023-03-31 23:01:39 +00:00
LibCompress/Gzip: Replace usage of DeprecatedString
2023-06-16 03:35:33 +00:00
static ErrorOr<Optional<String>> describe_header(ReadonlyBytes);
LibCore+LibHTTP+LibGfx: Switch to LibCompress This commit removes the only 3rd party library (and its usages) in serenity: puff, which is used for deflate decompression. and replaces it with the existing original serenity implementation in LibCompress. :^)
2021-03-03 21:54:07 +00:00
static bool is_likely_compressed(ReadonlyBytes bytes);
LibCompress: Implement gzip.
2020-08-28 15:53:57 +00:00
private:
class Member {
public:
LibCompress: Use a bit stream for the entire GZIP decompression process We currently mix normal and bit streams during GZIP decompression, where the latter is a wrapper around the former. This isn't causing issues now as the underlying bit stream buffer is a byte, so the normal stream can pick up where the bit stream left off. In order to increase the size of that buffer though, the normal stream will not be able to assume it can resume reading after the bit stream. The buffer can easily contain more bits than it was meant to read, so when the normal stream resumes, there may be N bits leftover in the bit stream that the normal stream was meant to read. To avoid weird behavior when mixing streams, this changes the GZIP decompressor to always read from a bit stream.
2023-03-28 15:28:04 +00:00
static ErrorOr<NonnullOwnPtr<Member>> construct(BlockHeader header, LittleEndianInputBitStream&);
LibCompress: Implement gzip.
2020-08-28 15:53:57 +00:00
BlockHeader m_header;
LibCompress: Switch `DeflateDecompressor` to a fallible constructor We don't have anything fallible in there yet, but we will soon switch the seekback buffer to the new `CircularBuffer`, which has a fallible constructor. We have to do the same for the internal `GzipDecompressor::Member` class, as it needs to construct a `DeflateCompressor` from its received stream.
2022-12-31 23:38:05 +00:00
NonnullOwnPtr<DeflateDecompressor> m_stream;
LibCompress: Implement gzip.
2020-08-28 15:53:57 +00:00
Crypto::Checksum::CRC32 m_checksum;
size_t m_nread { 0 };
LibCompress: Switch `DeflateDecompressor` to a fallible constructor We don't have anything fallible in there yet, but we will soon switch the seekback buffer to the new `CircularBuffer`, which has a fallible constructor. We have to do the same for the internal `GzipDecompressor::Member` class, as it needs to construct a `DeflateCompressor` from its received stream.
2022-12-31 23:38:05 +00:00
private:
Member(BlockHeader, NonnullOwnPtr<DeflateDecompressor>);
LibCompress: Implement gzip.
2020-08-28 15:53:57 +00:00
};
LibCompress: Switch `DeflateDecompressor` to a fallible constructor We don't have anything fallible in there yet, but we will soon switch the seekback buffer to the new `CircularBuffer`, which has a fallible constructor. We have to do the same for the internal `GzipDecompressor::Member` class, as it needs to construct a `DeflateCompressor` from its received stream.
2022-12-31 23:38:05 +00:00
Member const& current_member() const { return *m_current_member; }
Member& current_member() { return *m_current_member; }
LibCompress: Implement gzip.
2020-08-28 15:53:57 +00:00
LibCompress: Use a bit stream for the entire GZIP decompression process We currently mix normal and bit streams during GZIP decompression, where the latter is a wrapper around the former. This isn't causing issues now as the underlying bit stream buffer is a byte, so the normal stream can pick up where the bit stream left off. In order to increase the size of that buffer though, the normal stream will not be able to assume it can resume reading after the bit stream. The buffer can easily contain more bits than it was meant to read, so when the normal stream resumes, there may be N bits leftover in the bit stream that the normal stream was meant to read. To avoid weird behavior when mixing streams, this changes the GZIP decompressor to always read from a bit stream.
2023-03-28 15:28:04 +00:00
NonnullOwnPtr<LittleEndianInputBitStream> m_input_stream;
LibCompress: Allow partial header reads in GzipDecompressor We now read the header into a temporary header byte array that is used as the header once its filled up by the input stream, instead of just ending the stream if we are out of bytes mid header.
2021-03-16 15:39:50 +00:00
u8 m_partial_header[sizeof(BlockHeader)];
size_t m_partial_header_offset { 0 };
LibCompress: Switch `DeflateDecompressor` to a fallible constructor We don't have anything fallible in there yet, but we will soon switch the seekback buffer to the new `CircularBuffer`, which has a fallible constructor. We have to do the same for the internal `GzipDecompressor::Member` class, as it needs to construct a `DeflateCompressor` from its received stream.
2022-12-31 23:38:05 +00:00
OwnPtr<Member> m_current_member {};
AK: Lower the requirements for InputStream::eof and rename it. Consider the following snippet: void foo(InputStream& stream) { if(!stream.eof()) { u8 byte; stream >> byte; } } There is a very subtle bug in this snippet, for some input streams eof() might return false even if no more data can be read. In this case an error flag would be set on the stream. Until now I've always ensured that this is not the case, but this made the implementation of eof() unnecessarily complicated. InputFileStream::eof had to keep a ByteBuffer around just to make this possible. That meant a ton of unnecessary copies just to get a reliable eof(). In most cases it isn't actually necessary to have a reliable eof() implementation. In most other cases a reliable eof() is avaliable anyways because in some cases like InputMemoryStream it is very easy to implement.
2020-09-13 10:24:17 +00:00
bool m_eof { false };
LibCompress: Implement gzip.
2020-08-28 15:53:57 +00:00
};
Everywhere: Remove the `AK::` qualifier from Stream usages
2023-02-10 00:00:18 +00:00
class GzipCompressor final : public Stream {
LibCompress: Implement GZip compression This commit implements a stream compressor for the gzip specification (RFC 1952), which is essentially a thin wrapper around the DEFLATE compression format.
2021-03-12 23:26:44 +00:00
public:
LibCompress: Allow using GzipCompressor in a streaming fashion GzipCompressor is currently written assuming that it's write_some method is only called once. When we use this class for LibWeb, we may very well receive data to compress in small chunks. So this patch makes us write the gzip header and footer only once, which now resembles the zlib and deflate compressors.
2024-11-15 20:39:02 +00:00
static ErrorOr<NonnullOwnPtr<GzipCompressor>> create(MaybeOwned<Stream>);
LibCompress: Implement GZip compression This commit implements a stream compressor for the gzip specification (RFC 1952), which is essentially a thin wrapper around the DEFLATE compression format.
2021-03-12 23:26:44 +00:00
AK: Rename Stream::{read,write} to Stream::{read_some,write_some} Similar to POSIX read, the basic read and write functions of AK::Stream do not have a lower limit of how much data they read or write (apart from "none at all"). Rename the functions to "read some [data]" and "write some [data]" (with "data" being omitted, since everything here is reading and writing data) to make them sufficiently distinct from the functions that ensure to use the entire buffer (which should be the go-to function for most usages). No functional changes, just a lot of new FIXMEs.
2023-02-24 21:38:01 +00:00
virtual ErrorOr<Bytes> read_some(Bytes) override;
virtual ErrorOr<size_t> write_some(ReadonlyBytes) override;
LibCompress: Port `GzipCompressor` to `Core::Stream`
2022-12-27 10:40:23 +00:00
virtual bool is_eof() const override;
virtual bool is_open() const override;
virtual void close() override;
LibCompress: Implement GZip compression This commit implements a stream compressor for the gzip specification (RFC 1952), which is essentially a thin wrapper around the DEFLATE compression format.
2021-03-12 23:26:44 +00:00
LibCompress: Port `GzipCompressor` to `Core::Stream`
2022-12-27 10:40:23 +00:00
static ErrorOr<ByteBuffer> compress_all(ReadonlyBytes bytes);
LibCompress: Implement GZip compression This commit implements a stream compressor for the gzip specification (RFC 1952), which is essentially a thin wrapper around the DEFLATE compression format.
2021-03-12 23:26:44 +00:00
LibCompress: Allow using GzipCompressor in a streaming fashion GzipCompressor is currently written assuming that it's write_some method is only called once. When we use this class for LibWeb, we may very well receive data to compress in small chunks. So this patch makes us write the gzip header and footer only once, which now resembles the zlib and deflate compressors.
2024-11-15 20:39:02 +00:00
ErrorOr<void> finish();
LibCompress: Implement GZip compression This commit implements a stream compressor for the gzip specification (RFC 1952), which is essentially a thin wrapper around the DEFLATE compression format.
2021-03-12 23:26:44 +00:00
private:
LibCompress: Allow using GzipCompressor in a streaming fashion GzipCompressor is currently written assuming that it's write_some method is only called once. When we use this class for LibWeb, we may very well receive data to compress in small chunks. So this patch makes us write the gzip header and footer only once, which now resembles the zlib and deflate compressors.
2024-11-15 20:39:02 +00:00
GzipCompressor(MaybeOwned<Stream>, NonnullOwnPtr<DeflateCompressor>);
Everywhere: Remove the `AK::` qualifier from Stream usages
2023-02-10 00:00:18 +00:00
MaybeOwned<Stream> m_output_stream;
LibCompress: Allow using GzipCompressor in a streaming fashion GzipCompressor is currently written assuming that it's write_some method is only called once. When we use this class for LibWeb, we may very well receive data to compress in small chunks. So this patch makes us write the gzip header and footer only once, which now resembles the zlib and deflate compressors.
2024-11-15 20:39:02 +00:00
NonnullOwnPtr<DeflateCompressor> m_deflate_compressor;
Crypto::Checksum::CRC32 m_crc32;
size_t m_total_bytes { 0 };
bool m_finished { false };
LibCompress: Implement GZip compression This commit implements a stream compressor for the gzip specification (RFC 1952), which is essentially a thin wrapper around the DEFLATE compression format.
2021-03-12 23:26:44 +00:00
};
LibCompress: Implement gzip.
2020-08-28 15:53:57 +00:00
}
Reference in a new issue Copy permalink