Commit graph

46 commits

Author SHA1 Message Date
Timothy Flynn
22e0eeada2 Everywhere: Hoist the Services folder to the top-level 2024-11-10 12:50:45 +01:00
rmg-x
c755a09376 RequestServer/ConnectionFromClient: Remove unused header directives 2024-11-03 20:02:52 +01:00
rmg-x
fc53422e1c RequestServer: Implement preconnect for CreateConnection cache level 2024-11-03 20:02:52 +01:00
rmg-x
79046c73da RequestServer: Extract HTTP reason phrase from headers 2024-11-02 21:05:07 +01:00
Andrew Kaster
7372b2af48 LibIPC+Everywhere: Introduce an IPC Transport abstraction
This abstraction will help us to support multiple IPC transport
mechanisms going forward. For now, we only have a TransportSocket that
implements the same behavior we previously had, using Unix Sockets for
IPC.
2024-10-23 12:29:01 -06:00
rmg-x
10acf3f6c7 RequestServer: Set CURLOPT_CONNECTTIMEOUT instead of CURLOPT_TIMEOUT
Set the connection timeout which only limits the connection phase of the
request.

Previously, CURLOPT_TIMEOUT would apply to all transfer operations which
could result in legitimate upload or download operations being
cancelled.
2024-10-11 13:18:05 +02:00
rmg-x
db0dbb384e RequestServer: Set default timeout to 90 seconds for all requests 2024-10-10 19:56:11 +01:00
rmg-x
ff18114ae7 LibWeb+LibRequests+RequestServer: Report network error on request finish
This allows us to bubble up network errors to API consumers after
finishing a request.
2024-10-10 19:56:11 +01:00
Andreas Kling
504c80a202 RequestServer: Remove now-unused HTTP related code 2024-09-22 06:43:12 +02:00
marcin mikołajczak
fc2141c852 RequestServer: Set request body for other methods that can have a body 2024-09-21 18:13:31 +02:00
Andreas Kling
3ef0fc89b3 RequestServer: Fix assertion on exit when curl had pending timeouts
Some checks are pending
CI / Lagom (false, FUZZ, ubuntu-22.04, Linux, Clang) (push) Waiting to run
CI / Lagom (false, NO_FUZZ, macos-14, macOS, Clang) (push) Waiting to run
CI / Lagom (false, NO_FUZZ, ubuntu-22.04, Linux, GNU) (push) Waiting to run
CI / Lagom (true, NO_FUZZ, ubuntu-22.04, Linux, Clang) (push) Waiting to run
Package the js repl as a binary artifact / build-and-package (macos-14, macOS, macOS-universal2) (push) Waiting to run
Package the js repl as a binary artifact / build-and-package (ubuntu-22.04, Linux, Linux-x86_64) (push) Waiting to run
Run test262 and test-wasm / run_and_update_results (push) Waiting to run
Lint Code / lint (push) Waiting to run
Push notes / build (push) Waiting to run
If we already destroyed our timer during destruction, and then curl
tries to flush its timeouts when we tear down the multi, we can just
ignore the timer callbacks.
2024-09-19 11:45:15 +02:00
Andreas Kling
3e6448efcf RequestServer: Use CURLOPT_PRIVATE to store pointer to our own data
This is a lot nicer than scanning through our list of requests. :^)
2024-09-19 11:45:15 +02:00
Andreas Kling
e205723b95 RequestServer: Make WebSocket IPC APIs asynchronous
This fixes deadlocking when interacting with WebSockets while
RequestServer is trying to stream downloaded data to WebContent.
2024-09-19 07:37:46 +02:00
Andreas Kling
853a75c4ab RequestServer: Remove download progress updates that nobody used 2024-09-19 07:37:46 +02:00
Andreas Kling
6d91c42214 RequestServer: Remove "dump connection info" mechanism and UI
This is not relevant anymore when we use CURL.
2024-09-19 07:37:46 +02:00
Andreas Kling
e483bb70f0 RequestServer: Use libcurl for HTTP and HTTPS downloads
This replaces the use of our home-grown implementations inherited
from SerenityOS.
2024-09-19 07:37:46 +02:00
Timothy Flynn
e75ba44f37 LibProtocol+RequestServer: Add IPC to dump RequestServer connection info
This was available on Serenity by sending SIGINFO to RequestServer. Add
an IPC so we may see this info while running the browser.
2024-08-03 15:53:07 -04:00
Ali Mohammad Pur
93216a875e RequestServer: Actually hold on to the preload job until it's done
This was some nonsense code left in by mistake, ref commit:
4211639e45

Fixes #904.
2024-07-30 12:03:30 -04:00
Ali Mohammad Pur
4211639e45 RequestServer+LibHTTP: Cancel requests on client exit
That usually happens in "exceptional" states when the client exits
unexpectedly (crash, force quit mid-load, etc), leading to the job flush
timer firing and attempting to write to a nonexistent object (the
client).

This commit makes RS simply cancel such jobs; cancelled jobs in this
state simply go away instead of sending notifications around.
2024-07-28 13:02:49 +02:00
Ali Mohammad Pur
18499c4eac RequestServer: Make the ThreadPool global
Previously we made one thread pool per ipc client, which is excessive
and can eat up all the available fds very fast.
Fixes #760.
2024-07-23 08:56:59 +02:00
Andreas Kling
2c918b540d LibWeb+RequestServer: Let RequestServer set HTTP Accept-Encoding header
Ultimately it's RequestServer who knows which kind of encodings it can
handle and decompress, so let's have it set the Accept-Encoding.
2024-07-16 11:01:01 +02:00
Andreas Kling
0d22e0703f LibWebSocket: Use HTTP::HeaderMap in WebSocket code 2024-06-22 18:53:24 +02:00
Andreas Kling
260c5c50ad LibHTTP+RequestServer: Use HTTP::HeaderMap for request headers
No longer just for response headers! The same type is obviously useful
and ergonomic when making requests as well.
2024-06-09 15:34:02 +02:00
Andreas Kling
e636851481 LibHTTP+RequestServer: Add HTTP::HeaderMap and use for response headers
Instead of using a HashMap<ByteString, ByteString, CaseInsensitive...>
everywhere, we now encapsulate this in a class.

Even better, the new class also allows keeping track of multiple headers
with the same name! This will make it possible for HTTP responses to
actually retain all their headers on the perilous journey from
RequestServer to LibWeb.
2024-06-09 15:34:02 +02:00
Ali Mohammad Pur
ddc622233f RequestServer: Serialise accesses to IPC::Connection
Most of IPC::Connection is thread-safe, but the responsiveness timer is
very much not so, this commit makes sure only one thread can send stuff
through IPC to avoid having threads racing in IPC::Connection.
This is simply less work than making sure everything IPC::Connection
uses (now and later) is thread-safe.
2024-05-20 08:03:35 +02:00
Ali Mohammad Pur
57714fbb38 RequestServer: Handle IPC requests on multiple threads concurrently
Previously RS handled all the requests in an event loop, leading to
issues with connections being started in the middle of other connections
being started (and potentially blowing up the stack), ultimately causing
requests to be delayed because of other requests.
This commit reworks the way we handle these (specifically starting
connections) by first serialising the requests, and then performing them
in multiple threads concurrently; which yields a significant loading
performance and reliability increase.
2024-05-20 08:03:35 +02:00
Andrew Kaster
5e1d678bae Ladybird+Userland: Remove use of unnecessary fd passing socket concept
Now that LibIPC is using SCM_RIGHTS properly, we can go back to only
having one socket laying around when needing to transfer fds to peers.
2024-04-19 16:38:55 -04:00
Andrew Kaster
6d4ba21832 LibIPC+Userland: Make IPC::File always own its file descriptor
Add factory functions to distinguish between when the owner of the File
wants to transfer ownership to the new IPC object (adopt) or to send a
copy of the same fd to the IPC peer (clone).

This behavior is more intuitive than the previous behavior. Previously,
an IPC::File would default to a shallow clone of the file descriptor,
only *actually* calling dup(2) for the fd when encoding or it into an
IPC MessageBuffer. Now the dup(2) for the fd is explicit in the clone_fd
factory function.
2024-04-19 06:34:07 -04:00
Andrew Kaster
5e18d157d0 RequestServer: Add IPC to create a new client and return the sockets
We'll need this to implement single-instance RS for the Browser.
2024-04-17 10:09:49 -04:00
Shannon Booth
e800605ad3 AK+LibURL: Move AK::URL into a new URL library
This URL library ends up being a relatively fundamental base library of
the system, as LibCore depends on LibURL.

This change has two main benefits:
 * Moving AK back more towards being an agnostic library that can
   be used between the kernel and userspace. URL has never really fit
   that description - and is not used in the kernel.
 * URL _should_ depend on LibUnicode, as it needs punnycode support.
   However, it's not really possible to do this inside of AK as it can't
   depend on any external library. This change brings us a little closer
   to being able to do that, but unfortunately we aren't there quite
   yet, as the code generators depend on LibCore.
2024-03-18 14:06:28 -04:00
Ali Mohammad Pur
6dfb2f9dc8 Everywhere: Merge the WebSocket service into RequestServer
This keeps the APIs separate as they are wildly different, a future
improvement could be to somehow unify the APIs (if possible).

Closes #23080.
2024-03-06 10:07:27 +01:00
Ali Mohammad Pur
5232afa13d RequestServer+LibProtocol: Make starting requests fully async
This makes it so the clients don't have to wait for RS to become
responsive, potentially allowing them to do other things while RS
handles the connections.
Fixes #23306.
2024-02-26 14:13:37 +01:00
mobounya
7326d00baa RequestServer: Avoid using gethostbyname
Replace gethostbyname with Core::Socket::resolve_host in
RequestServer::ConnectionFromClient::ensure_connection.

Resolved #22199.
2024-02-06 21:43:23 +01:00
MacDue
a1d669fe63 RequestServer: Make pre-connection job refcounted
Fixes #22582

Previously, the job and the (cache of them) would lead to a UAF, as
after `.start()` was called on the job it'd be immediately destroyed.

Example of previous bug:

```
// Note due to the cache &jobA == &jobB
auto& jobA = Job::ensure("https://r.bing.com/");
auto& jobB = Job::ensure("https://r.bing.com/");
// Previously, the first .start() free'd the job
jobA.start();
// So the second .start() was a UAF
jobB.start();
```
2024-01-05 04:03:32 +03:30
Ali Mohammad Pur
5e1499d104 Everywhere: Rename {Deprecated => Byte}String
This commit un-deprecates DeprecatedString, and repurposes it as a byte
string.
As the null state has already been removed, there are no other
particularly hairy blockers in repurposing this type as a byte string
(what it _really_ is).

This commit is auto-generated:
  $ xs=$(ack -l \bDeprecatedString\b\|deprecated_string AK Userland \
    Meta Ports Ladybird Tests Kernel)
  $ perl -pie 's/\bDeprecatedString\b/ByteString/g;
    s/deprecated_string/byte_string/g' $xs
  $ clang-format --style=file -i \
    $(git diff --name-only | grep \.cpp\|\.h)
  $ gn format $(git ls-files '*.gn' '*.gni')
2023-12-17 18:25:10 +03:30
Shannon Booth
c25485700a AK: Port URL scheme from DeprecatedString to String 2023-08-13 15:03:53 -06:00
Shannon Booth
8751be09f9 AK: Serialize URL hosts with 'concept-host-serializer'
In order to follow spec text to achieve this, we need to change the
underlying representation of a host in AK::URL to deserialized format.
Before this, we were parsing the host and then immediately serializing
it again.

Making that change resulted in a whole bunch of fallout.

After this change, callers can access the serialized data through
this concept-host-serializer. The functional end result of this
change is that IPv6 hosts are now correctly serialized to be
surrounded with '[' and ']'.
2023-07-31 05:18:51 +02:00
Ben Wiederhake
9b9a38ec81 RequestServer: Avoid unnecessary IPC::Dictionary wrapper
We already have and use HashMap<DeprecatedString, DeprecatedString>
nearly everywhere, which is equivalent.
2023-05-21 07:53:45 +02:00
Tim Schumacher
a96339b72b LibCore: Move Stream-based sockets into the Core namespace 2023-02-13 00:50:07 +00:00
Linus Groh
6e19ab2bbc AK+Everywhere: Rename String to DeprecatedString
We have a new, improved string type coming up in AK (OOM aware, no null
state), and while it's going to use UTF-8, the name UTF8String is a
mouthful - so let's free up the String name by renaming the existing
class.
Making the old one have an annoying name will hopefully also help with
quick adoption :^)
2022-12-06 08:54:33 +01:00
networkException
4230dbbb21 AK+Everywhere: Replace "protocol" with "scheme" url helpers
URL had properly named replacements for protocol(), set_protocol() and
create_with_file_protocol() already. This patch removes these function
and updates all call sites to use the functions named according to the
specification.

See https://url.spec.whatwg.org/#concept-url-scheme
2022-09-29 09:39:04 +01:00
Kenneth Myhra
1e1d59cc25 RequestServer: Do not VERIFY(request.total_size.has_value())
If the request is stopped RequestServer::did_finish_request() will crash
on the VERIFY() call since request.total_size.has_value() returns false.

Let us instead use a conditional expression to verify if it has a value
and then call async_request_finished().
2022-06-21 10:29:14 +01:00
Ali Mohammad Pur
45867435c4 RequestServer+LibProtocol: Allow users to specify a per-request proxy 2022-04-09 12:21:43 +02:00
Lenny Maiorani
0b7baa7e5a Services: Use default constructors/destructors
https://isocpp.github.io/CppCoreGuidelines/CppCoreGuidelines#cother-other-default-operation-rules

"The compiler is more likely to get the default semantics right and
you cannot implement these functions better than the compiler."
2022-03-24 20:09:26 -07:00
Ali Mohammad Pur
dff8344336 RequestServer: Keep the EnsureConnection job alive until it finishes
Fixes #12906.
2022-03-19 22:04:35 +01:00
Itamar
3a71748e5d Userland: Rename IPC ClientConnection => ConnectionFromClient
This was done with CLion's automatic rename feature and with:
find . -name ClientConnection.h
    | rename 's/ClientConnection\.h/ConnectionFromClient.h/'

find . -name ClientConnection.cpp
    | rename 's/ClientConnection\.cpp/ConnectionFromClient.cpp/'
2022-02-25 22:35:12 +01:00
Renamed from Userland/Services/RequestServer/ClientConnection.cpp (Browse further)