beenull/ladybird
1
0
Fork 0
mirror of https://github.com/LadybirdBrowser/ladybird.git synced 2024-11-26 01:20:25 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 6
ladybird/Services/ImageDecoder/ConnectionFromClient.cpp

172 lines
6.5 KiB
C++
Raw Normal View History

ImageDecoder: Add a new service for out-of-process image decoding :^) The new ImageDecoder service (available for members of "image" via /tmp/portal/image) allows you to decode images in a separate process. This will allow programs to confidently load untrusted images, since the bulk of the security concerns are sandboxed to a separate process. The only API right now is a synchronous IPC DecodeImage() call that takes a shbuf with encoded image data and returns a shared buffer and metadata for the decoded image. It also comes with a very simple library for interfacing with the ImageDecoder service: LibImageDecoderClient. The name is a bit of a mouthful but I guess we can rename it later if we think of something nicer to call it. There's obviously a bit of overhead to spawning a separate process for every image decode, so this is mostly only appropriate for untrusted images (e.g stuff downloaded from the web) and not necessary for trusted local images (e.g stuff in /res)
2020-06-22 19:35:22 +00:00
/*
Meta: Update my e-mail address everywhere
2024-10-04 11:19:50 +00:00
* Copyright (c) 2020, Andreas Kling <andreas@ladybird.org>
ImageDecoder: Add a new service for out-of-process image decoding :^) The new ImageDecoder service (available for members of "image" via /tmp/portal/image) allows you to decode images in a separate process. This will allow programs to confidently load untrusted images, since the bulk of the security concerns are sandboxed to a separate process. The only API right now is a synchronous IPC DecodeImage() call that takes a shbuf with encoded image data and returns a shared buffer and metadata for the decoded image. It also comes with a very simple library for interfacing with the ImageDecoder service: LibImageDecoderClient. The name is a bit of a mouthful but I guess we can rename it later if we think of something nicer to call it. There's obviously a bit of overhead to spawning a separate process for every image decode, so this is mostly only appropriate for untrusted images (e.g stuff downloaded from the web) and not necessary for trusted local images (e.g stuff in /res)
2020-06-22 19:35:22 +00:00
*
Everything: Move to SPDX license identifiers in all files. SPDX License Identifiers are a more compact / standardized way of representing file license information. See: https://spdx.dev/resources/use/#identifiers This was done with the `ambr` search and replace tool. ambr --no-parent-ignore --key-from-file --rep-from-file key.txt rep.txt *
2021-04-22 08:24:48 +00:00
* SPDX-License-Identifier: BSD-2-Clause
ImageDecoder: Add a new service for out-of-process image decoding :^) The new ImageDecoder service (available for members of "image" via /tmp/portal/image) allows you to decode images in a separate process. This will allow programs to confidently load untrusted images, since the bulk of the security concerns are sandboxed to a separate process. The only API right now is a synchronous IPC DecodeImage() call that takes a shbuf with encoded image data and returns a shared buffer and metadata for the decoded image. It also comes with a very simple library for interfacing with the ImageDecoder service: LibImageDecoderClient. The name is a bit of a mouthful but I guess we can rename it later if we think of something nicer to call it. There's obviously a bit of overhead to spawning a separate process for every image decode, so this is mostly only appropriate for untrusted images (e.g stuff downloaded from the web) and not necessary for trusted local images (e.g stuff in /res)
2020-06-22 19:35:22 +00:00
*/
ImageDecoder: Add missing <AK/Debug.h> include and use dbgln_if()
2021-04-21 18:08:03 +00:00
#include <AK/Debug.h>
Everywhere: Transition ImageDecoder to be single-instance, owned by UI This is the same behavior as RequestServer, with the added benefit that we know how to gracefully reconnect ImageDecoder to all WebContent processes on restart.
2024-06-26 19:44:42 +00:00
#include <AK/IDAllocator.h>
Userland: Rename IPC ClientConnection => ConnectionFromClient This was done with CLion's automatic rename feature and with: find . -name ClientConnection.h | rename 's/ClientConnection\.h/ConnectionFromClient.h/' find . -name ClientConnection.cpp | rename 's/ClientConnection\.cpp/ConnectionFromClient.cpp/'
2022-02-25 10:18:30 +00:00
#include <ImageDecoder/ConnectionFromClient.h>
ImageDecoder: Add a new service for out-of-process image decoding :^) The new ImageDecoder service (available for members of "image" via /tmp/portal/image) allows you to decode images in a separate process. This will allow programs to confidently load untrusted images, since the bulk of the security concerns are sandboxed to a separate process. The only API right now is a synchronous IPC DecodeImage() call that takes a shbuf with encoded image data and returns a shared buffer and metadata for the decoded image. It also comes with a very simple library for interfacing with the ImageDecoder service: LibImageDecoderClient. The name is a bit of a mouthful but I guess we can rename it later if we think of something nicer to call it. There's obviously a bit of overhead to spawning a separate process for every image decode, so this is mostly only appropriate for untrusted images (e.g stuff downloaded from the web) and not necessary for trusted local images (e.g stuff in /res)
2020-06-22 19:35:22 +00:00
#include <ImageDecoder/ImageDecoderClientEndpoint.h>
#include <LibGfx/Bitmap.h>
LibGfx: Move all image loaders and writers to a subdirectory
2023-03-21 18:58:06 +00:00
#include <LibGfx/ImageFormats/ImageDecoder.h>
ImageViewer: Transform the image's dimension accordingly to the metadata Exif metadata have two tags to store the pixel density along each axis. If both values are different and no action is taken, the resulting image will appear deformed. This commit scales the displayed bitmap accordingly to these tags in order to show the image in its intended shape. This unfortunately includes a lot of plumbing to get this information through IPC.
2024-02-14 06:15:06 +00:00
#include <LibGfx/ImageFormats/TIFFMetadata.h>
ImageDecoder: Add a new service for out-of-process image decoding :^) The new ImageDecoder service (available for members of "image" via /tmp/portal/image) allows you to decode images in a separate process. This will allow programs to confidently load untrusted images, since the bulk of the security concerns are sandboxed to a separate process. The only API right now is a synchronous IPC DecodeImage() call that takes a shbuf with encoded image data and returns a shared buffer and metadata for the decoded image. It also comes with a very simple library for interfacing with the ImageDecoder service: LibImageDecoderClient. The name is a bit of a mouthful but I guess we can rename it later if we think of something nicer to call it. There's obviously a bit of overhead to spawning a separate process for every image decode, so this is mostly only appropriate for untrusted images (e.g stuff downloaded from the web) and not necessary for trusted local images (e.g stuff in /res)
2020-06-22 19:35:22 +00:00
namespace ImageDecoder {
Everywhere: Transition ImageDecoder to be single-instance, owned by UI This is the same behavior as RequestServer, with the added benefit that we know how to gracefully reconnect ImageDecoder to all WebContent processes on restart.
2024-06-26 19:44:42 +00:00
static HashMap<int, RefPtr<ConnectionFromClient>> s_connections;
static IDAllocator s_client_ids;
LibIPC+Everywhere: Introduce an IPC Transport abstraction This abstraction will help us to support multiple IPC transport mechanisms going forward. For now, we only have a TransportSocket that implements the same behavior we previously had, using Unix Sockets for IPC.
2024-10-22 21:47:33 +00:00
ConnectionFromClient::ConnectionFromClient(IPC::Transport transport)
: IPC::ConnectionFromClient<ImageDecoderClientEndpoint, ImageDecoderServerEndpoint>(*this, move(transport), s_client_ids.allocate())
ImageDecoder: Add a new service for out-of-process image decoding :^) The new ImageDecoder service (available for members of "image" via /tmp/portal/image) allows you to decode images in a separate process. This will allow programs to confidently load untrusted images, since the bulk of the security concerns are sandboxed to a separate process. The only API right now is a synchronous IPC DecodeImage() call that takes a shbuf with encoded image data and returns a shared buffer and metadata for the decoded image. It also comes with a very simple library for interfacing with the ImageDecoder service: LibImageDecoderClient. The name is a bit of a mouthful but I guess we can rename it later if we think of something nicer to call it. There's obviously a bit of overhead to spawning a separate process for every image decode, so this is mostly only appropriate for untrusted images (e.g stuff downloaded from the web) and not necessary for trusted local images (e.g stuff in /res)
2020-06-22 19:35:22 +00:00
{
Everywhere: Transition ImageDecoder to be single-instance, owned by UI This is the same behavior as RequestServer, with the added benefit that we know how to gracefully reconnect ImageDecoder to all WebContent processes on restart.
2024-06-26 19:44:42 +00:00
s_connections.set(client_id(), *this);
ImageDecoder: Add a new service for out-of-process image decoding :^) The new ImageDecoder service (available for members of "image" via /tmp/portal/image) allows you to decode images in a separate process. This will allow programs to confidently load untrusted images, since the bulk of the security concerns are sandboxed to a separate process. The only API right now is a synchronous IPC DecodeImage() call that takes a shbuf with encoded image data and returns a shared buffer and metadata for the decoded image. It also comes with a very simple library for interfacing with the ImageDecoder service: LibImageDecoderClient. The name is a bit of a mouthful but I guess we can rename it later if we think of something nicer to call it. There's obviously a bit of overhead to spawning a separate process for every image decode, so this is mostly only appropriate for untrusted images (e.g stuff downloaded from the web) and not necessary for trusted local images (e.g stuff in /res)
2020-06-22 19:35:22 +00:00
}
Userland: Rename IPC ClientConnection => ConnectionFromClient This was done with CLion's automatic rename feature and with: find . -name ClientConnection.h | rename 's/ClientConnection\.h/ConnectionFromClient.h/' find . -name ClientConnection.cpp | rename 's/ClientConnection\.cpp/ConnectionFromClient.cpp/'
2022-02-25 10:18:30 +00:00
void ConnectionFromClient::die()
ImageDecoder: Add a new service for out-of-process image decoding :^) The new ImageDecoder service (available for members of "image" via /tmp/portal/image) allows you to decode images in a separate process. This will allow programs to confidently load untrusted images, since the bulk of the security concerns are sandboxed to a separate process. The only API right now is a synchronous IPC DecodeImage() call that takes a shbuf with encoded image data and returns a shared buffer and metadata for the decoded image. It also comes with a very simple library for interfacing with the ImageDecoder service: LibImageDecoderClient. The name is a bit of a mouthful but I guess we can rename it later if we think of something nicer to call it. There's obviously a bit of overhead to spawning a separate process for every image decode, so this is mostly only appropriate for untrusted images (e.g stuff downloaded from the web) and not necessary for trusted local images (e.g stuff in /res)
2020-06-22 19:35:22 +00:00
{
ImageDecoder: Schedule decode jobs on the LibThreading background thread This allows the ImageDecoder service to handle new IPC requests while decoding in parallel.
2024-04-24 00:58:52 +00:00
for (auto& [_, job] : m_pending_jobs) {
job->cancel();
}
m_pending_jobs.clear();
Everywhere: Transition ImageDecoder to be single-instance, owned by UI This is the same behavior as RequestServer, with the added benefit that we know how to gracefully reconnect ImageDecoder to all WebContent processes on restart.
2024-06-26 19:44:42 +00:00
auto client_id = this->client_id();
s_connections.remove(client_id);
s_client_ids.deallocate(client_id);
if (s_connections.is_empty()) {
Threading::quit_background_thread();
Core::EventLoop::current().quit(0);
}
}
ErrorOr<IPC::File> ConnectionFromClient::connect_new_client()
{
int socket_fds[2] {};
if (auto err = Core::System::socketpair(AF_LOCAL, SOCK_STREAM, 0, socket_fds); err.is_error())
return err.release_error();
auto client_socket_or_error = Core::LocalSocket::adopt_fd(socket_fds[0]);
if (client_socket_or_error.is_error()) {
close(socket_fds[0]);
close(socket_fds[1]);
return client_socket_or_error.release_error();
}
auto client_socket = client_socket_or_error.release_value();
// Note: A ref is stored in the static s_connections map
LibIPC+Everywhere: Introduce an IPC Transport abstraction This abstraction will help us to support multiple IPC transport mechanisms going forward. For now, we only have a TransportSocket that implements the same behavior we previously had, using Unix Sockets for IPC.
2024-10-22 21:47:33 +00:00
auto client = adopt_ref(*new ConnectionFromClient(IPC::Transport(move(client_socket))));
Everywhere: Transition ImageDecoder to be single-instance, owned by UI This is the same behavior as RequestServer, with the added benefit that we know how to gracefully reconnect ImageDecoder to all WebContent processes on restart.
2024-06-26 19:44:42 +00:00
return IPC::File::adopt_fd(socket_fds[1]);
}
Messages::ImageDecoderServer::ConnectNewClientsResponse ConnectionFromClient::connect_new_clients(size_t count)
{
Vector<IPC::File> files;
files.ensure_capacity(count);
for (size_t i = 0; i < count; ++i) {
auto file_or_error = connect_new_client();
if (file_or_error.is_error()) {
dbgln("Failed to connect new client: {}", file_or_error.error());
return Vector<IPC::File> {};
}
files.unchecked_append(file_or_error.release_value());
}
return files;
ImageDecoder: Add a new service for out-of-process image decoding :^) The new ImageDecoder service (available for members of "image" via /tmp/portal/image) allows you to decode images in a separate process. This will allow programs to confidently load untrusted images, since the bulk of the security concerns are sandboxed to a separate process. The only API right now is a synchronous IPC DecodeImage() call that takes a shbuf with encoded image data and returns a shared buffer and metadata for the decoded image. It also comes with a very simple library for interfacing with the ImageDecoder service: LibImageDecoderClient. The name is a bit of a mouthful but I guess we can rename it later if we think of something nicer to call it. There's obviously a bit of overhead to spawning a separate process for every image decode, so this is mostly only appropriate for untrusted images (e.g stuff downloaded from the web) and not necessary for trusted local images (e.g stuff in /res)
2020-06-22 19:35:22 +00:00
}
ImageDecoder: Pass decoded images as Gfx::Bitmap over IPC Before this change, we were passing them as Gfx::ShareableBitmap. The problem is that shareable bitmaps keep their underlying file descriptor open, so that they can be shared again with someone else. When a Gfx::Bitmap is decoded from an IPC message, the file descriptor is closed and recovered immediately. This fixes an issue where we'd accumulate one file descriptor for every image decoded. This eventually led to descriptor starvation after enough images were loaded and still referenced at the same time.
2024-07-17 17:14:44 +00:00
static void decode_image_to_bitmaps_and_durations_with_decoder(Gfx::ImageDecoder const& decoder, Optional<Gfx::IntSize> ideal_size, Vector<Optional<NonnullRefPtr<Gfx::Bitmap>>>& bitmaps, Vector<u32>& durations)
ImageDecoder: Add a new service for out-of-process image decoding :^) The new ImageDecoder service (available for members of "image" via /tmp/portal/image) allows you to decode images in a separate process. This will allow programs to confidently load untrusted images, since the bulk of the security concerns are sandboxed to a separate process. The only API right now is a synchronous IPC DecodeImage() call that takes a shbuf with encoded image data and returns a shared buffer and metadata for the decoded image. It also comes with a very simple library for interfacing with the ImageDecoder service: LibImageDecoderClient. The name is a bit of a mouthful but I guess we can rename it later if we think of something nicer to call it. There's obviously a bit of overhead to spawning a separate process for every image decode, so this is mostly only appropriate for untrusted images (e.g stuff downloaded from the web) and not necessary for trusted local images (e.g stuff in /res)
2020-06-22 19:35:22 +00:00
{
LibGfx+Ladybird+Userland: Don't sniff for TGA images with only raw bytes Because TGA images don't have magic bytes as a signature to be detected, instead assume a sequence of ReadonlyBytes is a possible TGA image only if we are given a path so we could check the extension of the file and see if it's a TGA image. When we know the path of the file being loaded, we will try to first check its extension, and only if there's no match to a known decoder, based on simple extension lookup, then we would probe for other formats as usual with the normal sniffing method.
2023-01-15 19:50:32 +00:00
for (size_t i = 0; i < decoder.frame_count(); ++i) {
ImageDecoder: Add plumbing to allow a client to request an ideal size This is only used for vector graphics format where requesting the true displayed size leads to visual enhancement.
2023-12-23 18:56:23 +00:00
auto frame_or_error = decoder.frame(i, ideal_size);
LibGfx+Ladybird+Userland: Don't sniff for TGA images with only raw bytes Because TGA images don't have magic bytes as a signature to be detected, instead assume a sequence of ReadonlyBytes is a possible TGA image only if we are given a path so we could check the extension of the file and see if it's a TGA image. When we know the path of the file being loaded, we will try to first check its extension, and only if there's no match to a known decoder, based on simple extension lookup, then we would probe for other formats as usual with the normal sniffing method.
2023-01-15 19:50:32 +00:00
if (frame_or_error.is_error()) {
ImageDecoder: Pass decoded images as Gfx::Bitmap over IPC Before this change, we were passing them as Gfx::ShareableBitmap. The problem is that shareable bitmaps keep their underlying file descriptor open, so that they can be shared again with someone else. When a Gfx::Bitmap is decoded from an IPC message, the file descriptor is closed and recovered immediately. This fixes an issue where we'd accumulate one file descriptor for every image decoded. This eventually led to descriptor starvation after enough images were loaded and still referenced at the same time.
2024-07-17 17:14:44 +00:00
bitmaps.append({});
LibGfx+Ladybird+Userland: Don't sniff for TGA images with only raw bytes Because TGA images don't have magic bytes as a signature to be detected, instead assume a sequence of ReadonlyBytes is a possible TGA image only if we are given a path so we could check the extension of the file and see if it's a TGA image. When we know the path of the file being loaded, we will try to first check its extension, and only if there's no match to a known decoder, based on simple extension lookup, then we would probe for other formats as usual with the normal sniffing method.
2023-01-15 19:50:32 +00:00
durations.append(0);
} else {
auto frame = frame_or_error.release_value();
ImageDecoder: Pass decoded images as Gfx::Bitmap over IPC Before this change, we were passing them as Gfx::ShareableBitmap. The problem is that shareable bitmaps keep their underlying file descriptor open, so that they can be shared again with someone else. When a Gfx::Bitmap is decoded from an IPC message, the file descriptor is closed and recovered immediately. This fixes an issue where we'd accumulate one file descriptor for every image decoded. This eventually led to descriptor starvation after enough images were loaded and still referenced at the same time.
2024-07-17 17:14:44 +00:00
bitmaps.append(frame.image.release_nonnull());
LibGfx+Ladybird+Userland: Don't sniff for TGA images with only raw bytes Because TGA images don't have magic bytes as a signature to be detected, instead assume a sequence of ReadonlyBytes is a possible TGA image only if we are given a path so we could check the extension of the file and see if it's a TGA image. When we know the path of the file being loaded, we will try to first check its extension, and only if there's no match to a known decoder, based on simple extension lookup, then we would probe for other formats as usual with the normal sniffing method.
2023-01-15 19:50:32 +00:00
durations.append(frame.duration);
}
ImageDecoder: Add a new service for out-of-process image decoding :^) The new ImageDecoder service (available for members of "image" via /tmp/portal/image) allows you to decode images in a separate process. This will allow programs to confidently load untrusted images, since the bulk of the security concerns are sandboxed to a separate process. The only API right now is a synchronous IPC DecodeImage() call that takes a shbuf with encoded image data and returns a shared buffer and metadata for the decoded image. It also comes with a very simple library for interfacing with the ImageDecoder service: LibImageDecoderClient. The name is a bit of a mouthful but I guess we can rename it later if we think of something nicer to call it. There's obviously a bit of overhead to spawning a separate process for every image decode, so this is mostly only appropriate for untrusted images (e.g stuff downloaded from the web) and not necessary for trusted local images (e.g stuff in /res)
2020-06-22 19:35:22 +00:00
}
LibGfx+Ladybird+Userland: Don't sniff for TGA images with only raw bytes Because TGA images don't have magic bytes as a signature to be detected, instead assume a sequence of ReadonlyBytes is a possible TGA image only if we are given a path so we could check the extension of the file and see if it's a TGA image. When we know the path of the file being loaded, we will try to first check its extension, and only if there's no match to a known decoder, based on simple extension lookup, then we would probe for other formats as usual with the normal sniffing method.
2023-01-15 19:50:32 +00:00
}
ImageDecoder: Schedule decode jobs on the LibThreading background thread This allows the ImageDecoder service to handle new IPC requests while decoding in parallel.
2024-04-24 00:58:52 +00:00
static ErrorOr<ConnectionFromClient::DecodeResult> decode_image_to_details(Core::AnonymousBuffer const& encoded_buffer, Optional<Gfx::IntSize> ideal_size, Optional<ByteString> const& known_mime_type)
LibGfx+Ladybird+Userland: Don't sniff for TGA images with only raw bytes Because TGA images don't have magic bytes as a signature to be detected, instead assume a sequence of ReadonlyBytes is a possible TGA image only if we are given a path so we could check the extension of the file and see if it's a TGA image. When we know the path of the file being loaded, we will try to first check its extension, and only if there's no match to a known decoder, based on simple extension lookup, then we would probe for other formats as usual with the normal sniffing method.
2023-01-15 19:50:32 +00:00
{
ImageDecoder: Schedule decode jobs on the LibThreading background thread This allows the ImageDecoder service to handle new IPC requests while decoding in parallel.
2024-04-24 00:58:52 +00:00
auto decoder = TRY(Gfx::ImageDecoder::try_create_for_raw_bytes(ReadonlyBytes { encoded_buffer.data<u8>(), encoded_buffer.size() }, known_mime_type));
ImageDecoder: Add a new service for out-of-process image decoding :^) The new ImageDecoder service (available for members of "image" via /tmp/portal/image) allows you to decode images in a separate process. This will allow programs to confidently load untrusted images, since the bulk of the security concerns are sandboxed to a separate process. The only API right now is a synchronous IPC DecodeImage() call that takes a shbuf with encoded image data and returns a shared buffer and metadata for the decoded image. It also comes with a very simple library for interfacing with the ImageDecoder service: LibImageDecoderClient. The name is a bit of a mouthful but I guess we can rename it later if we think of something nicer to call it. There's obviously a bit of overhead to spawning a separate process for every image decode, so this is mostly only appropriate for untrusted images (e.g stuff downloaded from the web) and not necessary for trusted local images (e.g stuff in /res)
2020-06-22 19:35:22 +00:00
ImageDecoder: Schedule decode jobs on the LibThreading background thread This allows the ImageDecoder service to handle new IPC requests while decoding in parallel.
2024-04-24 00:58:52 +00:00
if (!decoder)
return Error::from_string_literal("Could not find suitable image decoder plugin for data");
ImageDecoder: Add a new service for out-of-process image decoding :^) The new ImageDecoder service (available for members of "image" via /tmp/portal/image) allows you to decode images in a separate process. This will allow programs to confidently load untrusted images, since the bulk of the security concerns are sandboxed to a separate process. The only API right now is a synchronous IPC DecodeImage() call that takes a shbuf with encoded image data and returns a shared buffer and metadata for the decoded image. It also comes with a very simple library for interfacing with the ImageDecoder service: LibImageDecoderClient. The name is a bit of a mouthful but I guess we can rename it later if we think of something nicer to call it. There's obviously a bit of overhead to spawning a separate process for every image decode, so this is mostly only appropriate for untrusted images (e.g stuff downloaded from the web) and not necessary for trusted local images (e.g stuff in /res)
2020-06-22 19:35:22 +00:00
ImageDecoder: Schedule decode jobs on the LibThreading background thread This allows the ImageDecoder service to handle new IPC requests while decoding in parallel.
2024-04-24 00:58:52 +00:00
if (!decoder->frame_count())
return Error::from_string_literal("Could not decode image from encoded data");
ConnectionFromClient::DecodeResult result;
result.is_animated = decoder->is_animated();
result.loop_count = decoder->loop_count();
ImageViewer: Transform the image's dimension accordingly to the metadata Exif metadata have two tags to store the pixel density along each axis. If both values are different and no action is taken, the resulting image will appear deformed. This commit scales the displayed bitmap accordingly to these tags in order to show the image in its intended shape. This unfortunately includes a lot of plumbing to get this information through IPC.
2024-02-14 06:15:06 +00:00
ImageDecoder+LibGfx: Collate decoded bitmaps before sending over IPC There is an issue where gifs with many frames cannot be loaded, as each bitmap is sent over IPC using a separate file descriptor, and there is limit on the maximum number of descriptors per IPC message. Thus, trying to load gifs with more than 64 frames (the current limit) causes the image decoder process to die. This commit introduces the BitmapSequence class, which is a thin wrapper around the type Vector<Optional<NonnullRefPtr<Gfx::Bitmap>>> and provides an IPC encode/decode routine that collates all bitmap data into a single buffer so that only a single file descriptor is required per IPC transfer, even if multiple frames are being sent.
2024-09-18 21:37:44 +00:00
Vector<Optional<NonnullRefPtr<Gfx::Bitmap>>> bitmaps;
ImageViewer: Transform the image's dimension accordingly to the metadata Exif metadata have two tags to store the pixel density along each axis. If both values are different and no action is taken, the resulting image will appear deformed. This commit scales the displayed bitmap accordingly to these tags in order to show the image in its intended shape. This unfortunately includes a lot of plumbing to get this information through IPC.
2024-02-14 06:15:06 +00:00
if (auto maybe_metadata = decoder->metadata(); maybe_metadata.has_value() && is<Gfx::ExifMetadata>(*maybe_metadata)) {
auto const& exif = static_cast<Gfx::ExifMetadata const&>(maybe_metadata.value());
if (exif.x_resolution().has_value() && exif.y_resolution().has_value()) {
auto const x_resolution = exif.x_resolution()->as_double();
auto const y_resolution = exif.y_resolution()->as_double();
if (x_resolution < y_resolution)
ImageDecoder: Schedule decode jobs on the LibThreading background thread This allows the ImageDecoder service to handle new IPC requests while decoding in parallel.
2024-04-24 00:58:52 +00:00
result.scale.set_y(x_resolution / y_resolution);
ImageViewer: Transform the image's dimension accordingly to the metadata Exif metadata have two tags to store the pixel density along each axis. If both values are different and no action is taken, the resulting image will appear deformed. This commit scales the displayed bitmap accordingly to these tags in order to show the image in its intended shape. This unfortunately includes a lot of plumbing to get this information through IPC.
2024-02-14 06:15:06 +00:00
else
ImageDecoder: Schedule decode jobs on the LibThreading background thread This allows the ImageDecoder service to handle new IPC requests while decoding in parallel.
2024-04-24 00:58:52 +00:00
result.scale.set_x(y_resolution / x_resolution);
ImageViewer: Transform the image's dimension accordingly to the metadata Exif metadata have two tags to store the pixel density along each axis. If both values are different and no action is taken, the resulting image will appear deformed. This commit scales the displayed bitmap accordingly to these tags in order to show the image in its intended shape. This unfortunately includes a lot of plumbing to get this information through IPC.
2024-02-14 06:15:06 +00:00
}
}
ImageDecoder+LibGfx: Collate decoded bitmaps before sending over IPC There is an issue where gifs with many frames cannot be loaded, as each bitmap is sent over IPC using a separate file descriptor, and there is limit on the maximum number of descriptors per IPC message. Thus, trying to load gifs with more than 64 frames (the current limit) causes the image decoder process to die. This commit introduces the BitmapSequence class, which is a thin wrapper around the type Vector<Optional<NonnullRefPtr<Gfx::Bitmap>>> and provides an IPC encode/decode routine that collates all bitmap data into a single buffer so that only a single file descriptor is required per IPC transfer, even if multiple frames are being sent.
2024-09-18 21:37:44 +00:00
decode_image_to_bitmaps_and_durations_with_decoder(*decoder, move(ideal_size), bitmaps, result.durations);
ImageDecoder: Schedule decode jobs on the LibThreading background thread This allows the ImageDecoder service to handle new IPC requests while decoding in parallel.
2024-04-24 00:58:52 +00:00
ImageDecoder+LibGfx: Collate decoded bitmaps before sending over IPC There is an issue where gifs with many frames cannot be loaded, as each bitmap is sent over IPC using a separate file descriptor, and there is limit on the maximum number of descriptors per IPC message. Thus, trying to load gifs with more than 64 frames (the current limit) causes the image decoder process to die. This commit introduces the BitmapSequence class, which is a thin wrapper around the type Vector<Optional<NonnullRefPtr<Gfx::Bitmap>>> and provides an IPC encode/decode routine that collates all bitmap data into a single buffer so that only a single file descriptor is required per IPC transfer, even if multiple frames are being sent.
2024-09-18 21:37:44 +00:00
if (bitmaps.is_empty())
ImageDecoder: Schedule decode jobs on the LibThreading background thread This allows the ImageDecoder service to handle new IPC requests while decoding in parallel.
2024-04-24 00:58:52 +00:00
return Error::from_string_literal("Could not decode image");
ImageDecoder+LibGfx: Collate decoded bitmaps before sending over IPC There is an issue where gifs with many frames cannot be loaded, as each bitmap is sent over IPC using a separate file descriptor, and there is limit on the maximum number of descriptors per IPC message. Thus, trying to load gifs with more than 64 frames (the current limit) causes the image decoder process to die. This commit introduces the BitmapSequence class, which is a thin wrapper around the type Vector<Optional<NonnullRefPtr<Gfx::Bitmap>>> and provides an IPC encode/decode routine that collates all bitmap data into a single buffer so that only a single file descriptor is required per IPC transfer, even if multiple frames are being sent.
2024-09-18 21:37:44 +00:00
result.bitmaps = Gfx::BitmapSequence { bitmaps };
ImageDecoder: Schedule decode jobs on the LibThreading background thread This allows the ImageDecoder service to handle new IPC requests while decoding in parallel.
2024-04-24 00:58:52 +00:00
return result;
LibGfx+Ladybird+Userland: Don't sniff for TGA images with only raw bytes Because TGA images don't have magic bytes as a signature to be detected, instead assume a sequence of ReadonlyBytes is a possible TGA image only if we are given a path so we could check the extension of the file and see if it's a TGA image. When we know the path of the file being loaded, we will try to first check its extension, and only if there's no match to a known decoder, based on simple extension lookup, then we would probe for other formats as usual with the normal sniffing method.
2023-01-15 19:50:32 +00:00
}
ImageDecoder: Add a new service for out-of-process image decoding :^) The new ImageDecoder service (available for members of "image" via /tmp/portal/image) allows you to decode images in a separate process. This will allow programs to confidently load untrusted images, since the bulk of the security concerns are sandboxed to a separate process. The only API right now is a synchronous IPC DecodeImage() call that takes a shbuf with encoded image data and returns a shared buffer and metadata for the decoded image. It also comes with a very simple library for interfacing with the ImageDecoder service: LibImageDecoderClient. The name is a bit of a mouthful but I guess we can rename it later if we think of something nicer to call it. There's obviously a bit of overhead to spawning a separate process for every image decode, so this is mostly only appropriate for untrusted images (e.g stuff downloaded from the web) and not necessary for trusted local images (e.g stuff in /res)
2020-06-22 19:35:22 +00:00
ImageDecoder: Schedule decode jobs on the LibThreading background thread This allows the ImageDecoder service to handle new IPC requests while decoding in parallel.
2024-04-24 00:58:52 +00:00
NonnullRefPtr<ConnectionFromClient::Job> ConnectionFromClient::make_decode_image_job(i64 image_id, Core::AnonymousBuffer encoded_buffer, Optional<Gfx::IntSize> ideal_size, Optional<ByteString> mime_type)
ImageDecoder: Handle decoding of images in an asynchronous manner ImageDecoder now queues up image decoding requests and returns the images back to the caller later. ImageDecoderClient has a new promise-based API that allows callers to attach their own resolve/reject handlers to the responses from ImageDecoder.
2024-04-19 20:23:16 +00:00
{
ImageDecoder: Schedule decode jobs on the LibThreading background thread This allows the ImageDecoder service to handle new IPC requests while decoding in parallel.
2024-04-24 00:58:52 +00:00
return Job::construct(
[encoded_buffer = move(encoded_buffer), ideal_size = move(ideal_size), mime_type = move(mime_type)](auto&) -> ErrorOr<DecodeResult> {
return TRY(decode_image_to_details(encoded_buffer, ideal_size, mime_type));
},
[strong_this = NonnullRefPtr(*this), image_id](DecodeResult result) -> ErrorOr<void> {
strong_this->async_did_decode_image(image_id, result.is_animated, result.loop_count, result.bitmaps, result.durations, result.scale);
strong_this->m_pending_jobs.remove(image_id);
return {};
},
[strong_this = NonnullRefPtr(*this), image_id](Error error) -> void {
if (strong_this->is_open())
strong_this->async_did_fail_to_decode_image(image_id, MUST(String::formatted("Decoding failed: {}", error)));
strong_this->m_pending_jobs.remove(image_id);
});
ImageDecoder: Handle decoding of images in an asynchronous manner ImageDecoder now queues up image decoding requests and returns the images back to the caller later. ImageDecoderClient has a new promise-based API that allows callers to attach their own resolve/reject handlers to the responses from ImageDecoder.
2024-04-19 20:23:16 +00:00
}
ImageDecoder: Add plumbing to allow a client to request an ideal size This is only used for vector graphics format where requesting the true displayed size leads to visual enhancement.
2023-12-23 18:56:23 +00:00
Messages::ImageDecoderServer::DecodeImageResponse ConnectionFromClient::decode_image(Core::AnonymousBuffer const& encoded_buffer, Optional<Gfx::IntSize> const& ideal_size, Optional<ByteString> const& mime_type)
LibGfx+Ladybird+Userland: Don't sniff for TGA images with only raw bytes Because TGA images don't have magic bytes as a signature to be detected, instead assume a sequence of ReadonlyBytes is a possible TGA image only if we are given a path so we could check the extension of the file and see if it's a TGA image. When we know the path of the file being loaded, we will try to first check its extension, and only if there's no match to a known decoder, based on simple extension lookup, then we would probe for other formats as usual with the normal sniffing method.
2023-01-15 19:50:32 +00:00
{
ImageDecoder: Handle decoding of images in an asynchronous manner ImageDecoder now queues up image decoding requests and returns the images back to the caller later. ImageDecoderClient has a new promise-based API that allows callers to attach their own resolve/reject handlers to the responses from ImageDecoder.
2024-04-19 20:23:16 +00:00
auto image_id = m_next_image_id++;
LibGfx+Ladybird+Userland: Don't sniff for TGA images with only raw bytes Because TGA images don't have magic bytes as a signature to be detected, instead assume a sequence of ReadonlyBytes is a possible TGA image only if we are given a path so we could check the extension of the file and see if it's a TGA image. When we know the path of the file being loaded, we will try to first check its extension, and only if there's no match to a known decoder, based on simple extension lookup, then we would probe for other formats as usual with the normal sniffing method.
2023-01-15 19:50:32 +00:00
if (!encoded_buffer.is_valid()) {
dbgln_if(IMAGE_DECODER_DEBUG, "Encoded data is invalid");
ImageDecoder: Handle decoding of images in an asynchronous manner ImageDecoder now queues up image decoding requests and returns the images back to the caller later. ImageDecoderClient has a new promise-based API that allows callers to attach their own resolve/reject handlers to the responses from ImageDecoder.
2024-04-19 20:23:16 +00:00
async_did_fail_to_decode_image(image_id, "Encoded data is invalid"_string);
return image_id;
ImageDecoder+LibImageDecoder+LibWeb: Support animations in ImageDecoder The ImageDecoder service now returns a list of image frames, each with a duration value. The code for in-process image decoding is removed from LibWeb, an all image decode requests are sent out-of-process to ImageDecoder. :^) This won't scale super well to very long and/or large animations, but we can work on improving that separately. The main goal here is simply to stop doing any image decoding inside LibWeb. Fixes #5165.
2021-01-29 21:30:48 +00:00
}
ImageDecoder: Handle decoding of images in an asynchronous manner ImageDecoder now queues up image decoding requests and returns the images back to the caller later. ImageDecoderClient has a new promise-based API that allows callers to attach their own resolve/reject handlers to the responses from ImageDecoder.
2024-04-19 20:23:16 +00:00
m_pending_jobs.set(image_id, make_decode_image_job(image_id, encoded_buffer, ideal_size, mime_type));
return image_id;
}
void ConnectionFromClient::cancel_decoding(i64 image_id)
{
if (auto job = m_pending_jobs.take(image_id); job.has_value()) {
ImageDecoder: Schedule decode jobs on the LibThreading background thread This allows the ImageDecoder service to handle new IPC requests while decoding in parallel.
2024-04-24 00:58:52 +00:00
job.value()->cancel();
ImageDecoder: Handle decoding of images in an asynchronous manner ImageDecoder now queues up image decoding requests and returns the images back to the caller later. ImageDecoderClient has a new promise-based API that allows callers to attach their own resolve/reject handlers to the responses from ImageDecoder.
2024-04-19 20:23:16 +00:00
}
ImageDecoder: Add a new service for out-of-process image decoding :^) The new ImageDecoder service (available for members of "image" via /tmp/portal/image) allows you to decode images in a separate process. This will allow programs to confidently load untrusted images, since the bulk of the security concerns are sandboxed to a separate process. The only API right now is a synchronous IPC DecodeImage() call that takes a shbuf with encoded image data and returns a shared buffer and metadata for the decoded image. It also comes with a very simple library for interfacing with the ImageDecoder service: LibImageDecoderClient. The name is a bit of a mouthful but I guess we can rename it later if we think of something nicer to call it. There's obviously a bit of overhead to spawning a separate process for every image decode, so this is mostly only appropriate for untrusted images (e.g stuff downloaded from the web) and not necessary for trusted local images (e.g stuff in /res)
2020-06-22 19:35:22 +00:00
}
}
Reference in a new issue Copy permalink