Commit graph

52 commits

Author SHA1 Message Date
Andreas Kling
c964a6b548 LibWeb: Paper over a VERIFY() crash in ResourceLoader for now 2022-07-17 14:11:36 +02:00
sin-ack
3f3f45580a Everywhere: Add sv suffix to strings relying on StringView(char const*)
Each of these strings would previously rely on StringView's char const*
constructor overload, which would call __builtin_strlen on the string.
Since we now have operator ""sv, we can replace these with much simpler
versions. This opens the door to being able to remove
StringView(char const*).

No functional changes.
2022-07-12 23:11:35 +02:00
Kenneth Myhra
92a3803066 LibWeb: Add timeout_callback to ResourceLoader::load() 2022-07-03 13:26:32 +02:00
Kenneth Myhra
07b6c7114b LibWeb: Use a single shot timer instead of an ordinary repetitive timer 2022-07-03 13:26:32 +02:00
Lucas CHOLLET
662711fa26 Browser+LibWeb+WebContent: Allow Browser to load local files
To achieve this goal:
 - The Browser unveils "/tmp/portal/filesystemaccess"
 - Pass the page through LoadRequest => ResourceLoader
 - ResourceLoader requests a file to the FileSystemAccessServer via IPC
 - OutOfProcessWebView handles it and sends a file descriptor back to
 the Page.
2022-06-27 20:22:15 +01:00
Andreas Kling
c03a0e7260 LibWeb: Fix unsafe capture of ref-to-local when setting up load timeout
We were capturing a reference to a stack local and then persisting the
closure, causing it to dereference a long-gone object when invoked.
2022-06-23 20:37:29 +02:00
Kenneth Myhra
c805987329 LibWeb: Add timeout functionality to ResourceLoader
Add timeout functionality to ResourceLoader and use it from
XMLHttpRequest.
2022-06-21 10:29:14 +01:00
Luke Wilde
210c3795f9 LibWeb: Apply content filter to DNS prefetch and pre-connect
Performing DNS prefetch or pre-connect on filtered URLs is wasteful,
as we would block any actual use further down the line.

A bunch of websites perform DNS prefetch and/or pre-connect to trackers
as well, for example:
```
prefetch DNS for 'https://adserver-us.adtech.advertising.com/'
prefetch DNS for 'https://secure.adnxs.com/'
prefetch DNS for 'https://bidder.criteo.com/'
prefetch DNS for 'https://static.criteo.net/'
prefetch DNS for 'https://cdn.krxd.net/'
prefetch DNS for 'https://widgets.outbrain.com/'
prefetch DNS for 'https://images.outbrain.com/'
prefetch DNS for 'https://log.outbrain.com/
prefetch DNS for 'https://amplifypixel.outbrain.com/'
prefetch DNS for 'https://odb.outbrain.com/'
prefetch DNS for 'https://js-sec.indexww.com/'
prefetch DNS for 'https://as-sec.casalemedia.com/'
prefetch DNS for 'https://as.casalemedia.com/'
prefetch DNS for 'https://sofia.trustx.org/'
prefetch DNS for 'https://c.amazon-adsystem.com/'
prefetch DNS for 'https://s.amazon-adsystem.com/'
prefetch DNS for 'https://aax.amazon-adsystem.com/'
prefetch DNS for 'https://t.teads.tv/'
prefetch DNS for 'https://beacon.krxd.net/'
pre-connect to 'https://www.google-analytics.com/'
pre-connect to 'https://www.googletagmanager.com/'
```
2022-06-10 12:15:37 +01:00
Michiel Visser
7278ad761e LibHTTP+LibWeb: Accept Brotli encoded responses 2022-05-21 22:41:40 +02:00
DexesTTP
c00ae53b66 LibWeb: Abstract the LibProtocol ResourceLoader connection
This is the final component that required LibProtocol as a dependency
of LibWeb. With this, we can now remove the dependency, and LibWeb no
longer requires IPC to work :^)
2022-05-15 12:17:36 +02:00
Sam Atkins
1f82beded3 LibWeb: Make about:blank load correctly
- Don't treat an empty `about:blank` resource as an error.
- Give `about:` urls a content-type so `FrameLoader::parse_document()`
  won't reject them.
2022-05-13 16:25:33 +02:00
DexesTTP
6027ab9e12 LibWeb: Only generate ResourceLoader signposts while on Serenity 2022-05-06 14:11:03 +02:00
Andreas Kling
9ed5a14af2 LibWeb: Remove unused ResourceLoader::load_sync()
There are no remaining users of this API, everyone has been migrated
to asynchronous resource loading.
2022-04-10 01:37:45 +02:00
Andreas Kling
f21eb90294 LibWeb: Remove debug spam about proxy configuration lookups 2022-04-09 14:50:05 +02:00
Ali Mohammad Pur
a42e03b01a Browser+LibWeb+WebContent: Implement per-URL-pattern proxies
...at least for SOCKS5.
2022-04-09 12:21:43 +02:00
Idan Horowitz
086969277e Everywhere: Run clang-format 2022-04-01 21:24:45 +01:00
Simon Wanner
3d80d38954 LibWeb: Emit signposts for resource loads 2022-03-24 14:35:47 +01:00
Timothy Flynn
20eb441cba LibWeb: Evict replaced Resource objects from cache
When a Resource is converted to an ImageResource, evict the original
resource from cache. The original resource's data has been moved, so on
a warm reload of a page, when that resource is loaded from cache, it
would not have any data to actually show.
2022-03-23 21:26:35 +01:00
Andreas Kling
252ed8ad18 LibWeb: Fail resource loads on HTTP 4xx or 5xx error
This fixes an issue on ACID3 where failing image loads with body content
would still get displayed.
2022-03-09 16:43:00 +01:00
Andreas Kling
378bca8b0c LibWeb: Make debug logging of resource load errors red instead of green
Red is a bit more suspicious than green, after all. :^)
2022-02-04 00:16:25 +01:00
Sam Atkins
c388a879d7 AK+Userland: Make AK::decode_base64 return ErrorOr 2022-01-24 22:36:09 +01:00
sin-ack
2e1bbcb0fa LibCore+LibIPC+Everywhere: Return Stream::LocalSocket from LocalServer
This change unfortunately cannot be atomically made without a single
commit changing everything.

Most of the important changes are in LibIPC/Connection.cpp,
LibIPC/ServerConnection.cpp and LibCore/LocalServer.cpp.

The notable changes are:
- IPCCompiler now generates the decode and decode_message functions such
  that they take a Core::Stream::LocalSocket instead of the socket fd.
- IPC::Decoder now uses the receive_fd method of LocalSocket instead of
  doing system calls directly on the fd.
- IPC::ConnectionBase and related classes now use the Stream API
  functions.
- IPC::ServerConnection no longer constructs the socket itself; instead,
  a convenience macro, IPC_CLIENT_CONNECTION, is used in place of
  C_OBJECT and will generate a static try_create factory function for
  the ServerConnection subclass. The subclass is now responsible for
  passing the socket constructed in this function to its
  ServerConnection base; the socket is passed as the first argument to
  the constructor (as a NonnullOwnPtr<Core::Stream::LocalServer>) before
  any other arguments.
- The functionality regarding taking over sockets from SystemServer has
  been moved to LibIPC/SystemServerTakeover.cpp. The Core::LocalSocket
  implementation of this functionality hasn't been deleted due to my
  intention of removing this class in the near future and to reduce
  noise on this (already quite noisy) PR.
2022-01-15 13:29:48 +03:30
Andreas Kling
a7f1f1c34b LibCore: Use ErrorOr<T> for Core::File::open() 2021-11-08 00:35:27 +01:00
Ben Wiederhake
cb868cfa41 AK+Everywhere: Make Base64 decoding fallible 2021-10-23 19:16:40 +01:00
Ali Mohammad Pur
f0e2c517fc LibWeb: Implement the dns-prefetch and preconnect link relationships 2021-09-28 22:32:31 +02:00
Andreas Kling
fccf0a466b LibWeb: Log resource load success before invoking success callback
The success callback may trigger JavaScript execution, causing resource
load times to appear much longer than they actually are. :^)
2021-09-22 14:35:13 +02:00
Ali Mohammad Pur
e780ee2832 LibWeb: Avoid introducing a reference cycle in ResourceLoader::load()
Previously we were kinda sorta resolving the reference cycle, but let's
just keep the requests in a hashtable instead of relying on hard to
track refcount tricks.
Fixes #7314.
2021-09-19 21:10:23 +04:30
Andreas Kling
d69e4f9ca7 LibWeb: Don't dump full data URLs in ResourceLoader logging
Some pages use *really* large data URLs. :^)
2021-09-16 01:39:47 +02:00
Idan Horowitz
d6cfa34667 AK: Make URL::m_port an Optional<u16>, Expose raw port getter
Our current way of signalling a missing port with m_port == 0 was
lacking, as 0 is a valid port number in URLs.
2021-09-14 00:14:45 +02:00
Idan Horowitz
4629f2e4ad LibWeb: Add the Web::URL namespace and move URLEncoder to it
This namespace will be used for all interfaces defined in the URL
specification, like URL and URLSearchParams.

This has the unfortunate side-effect of requiring us to use the fully
qualified AK::URL name whenever we want to refer to the AK class, so
this commit also fixes all such references.
2021-09-13 01:43:10 +02:00
Brian Gianforcaro
189066d91d LibWeb: Log resource loading success, failure, and duration
When debugging why your website isn't loading in LibWeb the
resource loader is a blind spot as we don't have much logging
except on certain error paths. This can lead to confusing situations
where the browser just appears to hang.

This changes attempts to fix that by adding common success and
failure logging handlers so all resource loading outcomes can
are logged.
2021-09-12 16:39:23 +02:00
Brian Gianforcaro
bca8707e15 LibWeb: Start tracking elapsed time when a resource is loaded 2021-09-12 16:39:23 +02:00
sin-ack
8ea22121ac Userland: Migrate to argument-less deferred_invoke
Only one place used this argument and it was to hold on to a strong ref
for the object. Since we already do that now, there's no need to keep
this argument around since this can be easily captured.

This commit contains no changes.
2021-09-02 03:47:47 +04:30
TheFightingCatfish
652fa54495 Browser+LibWeb: Silence some debug spams 2021-08-08 00:33:14 +02:00
Max Wipfli
30cdebfa9e LibProtocol: Use URL class in RequestClient::start_request argument
This changes the RequestClient::start_request() method to take a URL
object instead of a URL string as argument. All callers of the method
already had a URL object anyway, and start_request() in turn parses the
URL string back into a URL object. This removes this unnecessary
conversion.
2021-06-06 16:00:11 +02:00
Max Wipfli
33396494f6 AK+LibWeb: Remove URL::to_string_encoded()
This replaces URL::to_string_encoded() with to_string() and removes the
former, since they are now equivalent.
2021-06-01 12:23:16 +02:00
Ali Mohammad Pur
a91a49337c LibCore+Everywhere: Move OpenMode out of IODevice
...and make it an enum class so people don't omit "OpenMode".
2021-05-12 11:00:45 +01:00
DexesTTP
71d27abb97 Services: Rename ProtocolServer to RequestServer
The current ProtocolServer was really only used for requests, and with
the recent introduction of the WebSocket service, long-lasting
connections with another server are not part of it. To better reflect
this, this commit renames it to RequestServer.

This commit also changes the existing 'protocol' portal to 'request',
the existing 'protocol' user and group to 'request', and most mentions
of the 'download' aspect of the request to 'request' when relevant, to
make everything consistent across the system.

Note that LibProtocol still exists as-is, but the more generic Client
class and the more specific Download class have both been renamed to a
more accurate RequestClient and Request to match the new names.

This commit only change names, not behaviors.
2021-04-25 19:04:34 +02:00
Simon Danner
7ab8be9e0b
LibWeb: Encode URL in ResourceLoader::load()
Encode URLs before requesting them - this fixes loading of resources
that have spaces in their URL, for example.
2021-04-22 20:45:59 +02:00
Brian Gianforcaro
1682f0b760 Everything: Move to SPDX license identifiers in all files.
SPDX License Identifiers are a more compact / standardized
way of representing file license information.

See: https://spdx.dev/resources/use/#identifiers

This was done with the `ambr` search and replace tool.

 ambr --no-parent-ignore --key-from-file --rep-from-file key.txt rep.txt *
2021-04-22 11:22:27 +02:00
Linus Groh
7400e3d8fc LibWeb: Don't call ResourceLoader error callback on 4xx status code
A website with a 4xx status code is still a valid website, we should not
artificially hide it. In fact, many websites implement custom 404 error
pages for example, often containing search functionality or links back
to the homepage.
This might have implications regarding the loading of stylesheets where
the request 404s, but since we were not handling 5xx status codes as
errors either, I think that's fine for now (but might need additional
work later). Worst case, the parser rejects to load some error page HTML
as CSS :^)
2021-04-20 19:47:23 +02:00
Timothy Flynn
347838a240 LibWeb: Set Cookie header on <script> resource requests
This required changing the load_sync API to take a LoadRequest instead
of just a URL. Since HTMLScriptElement was the only (non-test) user of
this API, it didn't seem useful to instead add an overload of load_sync
for this.
2021-04-15 09:46:49 +02:00
Idan Horowitz
bc9cd55da4 Browser+LibWeb: Add support for spoofing the browser user agent
This is helpful when testing certain sites like twitter.com which
display differently based on the user agent.
2021-04-14 23:06:32 +02:00
Linus Groh
000ef96613 LibWeb: Pass optional status code to ResourceLoader callbacks
This is needed for XMLHttpRequest, and will certainly be useful for
other things, too.
2021-04-03 16:34:34 +02:00
Timothy Flynn
855920fe13 Browser+LibWeb+WebContent: Add action to clear resource cache 2021-03-30 10:27:49 +02:00
Luke
dddaa529b2 LibWeb: Advertise that we support the deflate content encoding 2021-03-22 12:45:34 +01:00
AnotherTest
09a43969ba Everywhere: Replace dbgln<flag>(...) with dbgln_if(flag, ...)
Replacement made by `find Kernel Userland -name '*.h' -o -name '*.cpp' | sed -i -Ee 's/dbgln\b<(\w+)>\(/dbgln_if(\1, /g'`
2021-02-08 18:08:55 +01:00
asynts
8465683dcf Everywhere: Debug macros instead of constexpr.
This was done with the following script:

    find . \( -name '*.cpp' -o -name '*.h' -o -name '*.in' \) -not -path './Toolchain/*' -not -path './Build/*' -exec sed -i -E 's/dbgln<debug_([a-z_]+)>/dbgln<\U\1_DEBUG>/' {} \;

    find . \( -name '*.cpp' -o -name '*.h' -o -name '*.in' \) -not -path './Toolchain/*' -not -path './Build/*' -exec sed -i -E 's/if constexpr \(debug_([a-z0-9_]+)/if constexpr \(\U\1_DEBUG/' {} \;
2021-01-25 09:47:36 +01:00
Andreas Kling
ae0be7797f LibWeb: Disable resource cache for file:// URLs
This makes the browser a bit less annoying when testing local files,
since you no longer have to restart it for changes to take effect.

Longer-term we should have a proper way to decide which resources
are cacheable.
2021-01-24 12:11:08 +01:00
asynts
3f23a58fa1 Everywhere: Replace a bundle of dbg with dbgln.
These changes are arbitrarily divided into multiple commits to make it
easier to find potentially introduced bugs with git bisect.
2021-01-22 22:14:30 +01:00