beenull/ladybird
1
0
Fork 0
mirror of https://github.com/LadybirdBrowser/ladybird.git synced 2024-11-22 15:40:19 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 6
ladybird/Documentation/HighDPI.md

190 lines
8.3 KiB
Markdown
Raw Normal View History

Docs: Add design doc and implementation plan for highdpi I have a local branch that gets us past implementation stage 1 in this doc, and it seems like a useful enough checkpoint to upstream it and then iterate in tree.
2021-01-12 19:41:36 +00:00
HighDPI design
==============
Background
----------
- macOS: Only integer scale factors at app level. Can stretch final composited framebuffer at very end for non-integer display scales.
- advantages: simple programming model, still fairly flexible; scaling at end produces coherent final image
- disadvantages: needs 4x memory even for 1.5x scale; scaling at very makes final image less detailed than it could be
- Android: Many (but discrete) scale levels (ldpi (0.75x), mdpi, hdpi (1.5x), xhdpi (2x), xxhdpi (3x), xxhdpi (4x))
- Windows: has "not recommended" free form text entry for scale factor between 100% and 500%.
Integer scale factors are needed in any case so let's get that working first. Actually, let's focus on just 2x for now.
Desired end state
-----------------
LibGfx: Give Bitmap a scale factor Gfx::Bitmap can now store its scale factor. Normally it's 1, but in high dpi mode it can be 2. If a Bitmap with a scale factor of 2 is blitted to a Painter with scale factor of 2, the pixels can be copied over without any resampling. (When blitting a Bitmap with a scale factor of 1 to a Painter with scale factor of 2, the Bitmap is painted at twice its width and height at paint time. Blitting a Bitmap with a scale factor of 2 to a Painter with scale factor 1 is not supported.) A Bitmap with scale factor of 2 reports the same width() and height() as one with scale factor 1. That's important because many places in the codebase use a bitmap's width() and height() to layout Widgets, and all widget coordinates are in logical coordinates as well, per Documentation/HighDPI.md. Bitmap grows physical_width() / physical_height() to access the actual pixel size. Update a few callers that work with pixels to call this instead. Make Painter's constructor take its scale factor from the target bitmap that's passed in, and update its various blit() methods to handle blitting a 2x bitmap to a 2x painter. This allows removing some gnarly code in Compositor. (In return, put some new gnarly code in LibGfxScaleDemo to preserve behavior there.) No intended behavior change.
2021-01-19 17:10:47 +00:00
- All rects (Window and Widget rects, mouse cursor, even bitmap sizes) are in "logical" coordinates, which is the same as pixels at 1x scale, as much as possible.
WindowServer: Make HighDPI aware Almost all logic stays in "logical" (unscaled coordinates), which means the patch is small and things like DnD, window moving and resizing, menu handling, menuapplets, etc all work without changes. Screen knows about phyiscal coordinates and mouse handling internally is in physical coordinates (so that two 1 pixel movements in succession can translate to one 1 logical coordinate mouse movement -- only a single event is sent in this case, on the 2nd moved pixel). Compositor also knows about physical pixels for its backbuffers. This is a temporary state -- in a follow-up, I'll try to let Bitmaps know about their intrinsic scale, then Compositor won't have to know about pixels any longer. Most of Compositor's logic stays in view units, just blitting to and from back buffers and the cursor save buffer has to be done in pixels. The back buffer Painter gets a scale applied which transparently handles all drawing. (But since the backbuffer and cursor save buffer are also HighDPI, they currently need to be drawn using a hack temporary unscaled Painter object. This will also go away once Bitmaps know about their intrinsic scale.) With this, editing WindowServer.ini to say Width=800 Height=600 ScaleFactor=2 and booting brings up a fully-functional HighDPI UI. (Except for minimizing windows, which will crash the window server until #4932 is merged. And I didn't test the window switcher since the win-tab shortcut doesn't work on my system.) It's all pixel-scaled, but it looks pretty decent :^)
2021-01-15 15:58:55 +00:00
- If something needs to be in pixels, its name starts with `physical_`. Physical coordinates should as much as possible not cross API boundaries.
Docs: Add design doc and implementation plan for highdpi I have a local branch that gets us past implementation stage 1 in this doc, and it seems like a useful enough checkpoint to upstream it and then iterate in tree.
2021-01-12 19:41:36 +00:00
- Jury's still out if logical coordinates should stay ints. Probably, but it means mouse cursor etc only have point resolution, not pixel resolution
- We should have something that can store a collection of (lazily-loaded?) bitmaps and fonts that each represent a single image / font at different scale levels, and at paint time the right representation is picked for the current scale
Docs: Start outlining options for highdpi resource handling
2021-01-17 18:58:25 +00:00
Resource loading
----------------
Resources such as icons, cursors, bitmap fonts are scale-dependent: In HighDPI modes, different resources need to be loaded.
### Art direction
A 2x resource should look like a 1x resource, just with less jagged edges. A horizontal or vertical line that's 1 pixel wide in 1x should be 2 pixels wide in 2x.
A good guideline for black-and-white images: start with a 1x bitmap, resize it to 200% using nearest-neighbor filtering, and then move black pixels around to smooth diagonal edges -- but the number of black pixels shouldn't change relative to the 200% nearest-neighbor-resampled image.
While a 1x 32x32 bitmap and a 2x 16x16 bitmap both have 32x32 pixels, they don't have to look the same: The 2x 16x16 should look exactly like the corresonding 1x 16x16, just with smoother edges. The 1x 32x32 pixel resource could instead pick a different crop. As a concrete example, the 1x 7x10 ladybug emoji image currently just has the ladybug's shell (for space reasons), and so should the 2x version of that emoji. On the other hand, if we add a higher-res 1x 14x20 ladybug emoji at some point, we might want show the ladybug's legs on it, instead of just a smoother rendition of just the shell. (The 2x version of that 14x20 emoji would then have legs and shell in less jagged.)
### Directory structure
currently:
res/
cursors/
arrowx2y2.png
...
emoji/ (currently all 7x10 px)
U+1F346.png
...
fonts/
CsillaRegular10.font
...
graphics/
brand-banner.png
...
icons/
16x16/
small app icons, small filetype icons, toolbar icons, window buttons, ...
32x32/
large app icons, large filetype icons, message box icons
various per-app folders with in-app UI images (XXX: maybe move into "apps" subdir?)
themes/
Coffee/
16x16/
custom window buttons
(more themes)
...
wallpapers/
desktop wallpapers
Every one of these could grow a 2x variant (and if we do more scale factors later, even more variants).
Possible new structures:
1. Have "1x", "2x" folders right inside res and then mirror folder structures inside them:
res/
1x/
cursors/
16x16/
emoji/
...
2x/
cursors/
16x16/
emoji/
...
2. Instead of having the 1x/2x fork at the root, have it at each leaf:
res/
cursors/
1x/
arrowx2y2.png
...
2x/
arrowx2y2.png
...
emoji/
1/
U+1F346.png
...
2/
U+1F346.png
...
...
3. Use filename suffixes instead of directories (similar to macOS):
res/
cursors/
arrowx2y2.png
arrowx2y2@2x.png
...
emoji/
U+1F346.png
U+1F346@2x.png
...
4. Use suffixes on directory instead of subdirectory:
res/
cursors/
arrowx2y2.png
...
cursors-2x/
arrowx2y2.png
...
Root-level split makes it easy to see which scale factors exist and is subjectively aesthetically pleasing.
Filename suffixes make it easy to see which icons don't have high-res versions (but in return clutter up an icon directory), and it makes it easy to get the intrinsic scale factor of a bitmap (just need to look at the image's basename, not at any directory).
Android has additional modifiers in additon to scale factors in its resource system (UI language, light/dark mode, screen size in addition to resolution, etc). If we ever add more factors to the resource system, a suffix-based system would probably extend more nicely than a nesting-based one.
In the end probably doesn't matter all that much which version to pick.
### Resource loading strategy tradeoffs
- eagerly load one scale, reload at new scale on scale factor change events
- needs explicit code
- random code in LibGfx currently loads icons, and scale factor change events would be more a LibGUI level concept, not clear how to plumb the event to there
+ memory efficient -- only have one copy of each resource in memory
+ easy to understand: Bitmap stays Bitmap, Font stays Font, no need for collections
- have BitmapCollection that stores high-res and low-res path and load lazily when needed
- need to do synchronous disk access at first paint on UI thread
- or load compressed data at each scale eagerly and decompress lazily. still a blocking decode on UI thread then, and needs more memory -- 2x compressed resources in memory even if they might never be needed
+ puts complexity in framework, app doesn't have to care
+ can transparently paint UI at both 1x and 2x into different backbuffers (eg for multiple screens that have different scale factors)
- eagerly load both and use the right one at paint time
- similar to GUI::Icon
- 400% memory overhead in 1x mode (but most icons are small)
+ conceptually easy to understand, but still need some collection class
+ puts (less) complexity in framework, app doesn't have to care
+ can transparently paint UI at both 1x and 2x into different backbuffers (eg for multiple screens that have different scale factors)
### Resource loading API
Currently:
auto app_icon = GUI::Icon::default_icon("app-playground");
or
s_unfilled_circle_bitmap = Bitmap::load_from_file("/res/icons/serenity/unfilled-radio-circle.png");
or
header.set_font(Gfx::Font::load_from_file("/res/fonts/PebbletonBold14.font"));
Going forward:
FIXME (depends on loading strategy decison a bit?)
Docs: Add design doc and implementation plan for highdpi I have a local branch that gets us past implementation stage 1 in this doc, and it seems like a useful enough checkpoint to upstream it and then iterate in tree.
2021-01-12 19:41:36 +00:00
Implementation plan
-------------------
The plan is to have all applications use highdpi backbuffers eventually. It'll take some time to get there though, so here's a plan for getting there incrementally.
0. Add some scaling support to Painter. Make it do 2x nearest neighbor scaling of everything at paint time for now.
WindowServer: Make HighDPI aware Almost all logic stays in "logical" (unscaled coordinates), which means the patch is small and things like DnD, window moving and resizing, menu handling, menuapplets, etc all work without changes. Screen knows about phyiscal coordinates and mouse handling internally is in physical coordinates (so that two 1 pixel movements in succession can translate to one 1 logical coordinate mouse movement -- only a single event is sent in this case, on the 2nd moved pixel). Compositor also knows about physical pixels for its backbuffers. This is a temporary state -- in a follow-up, I'll try to let Bitmaps know about their intrinsic scale, then Compositor won't have to know about pixels any longer. Most of Compositor's logic stays in view units, just blitting to and from back buffers and the cursor save buffer has to be done in pixels. The back buffer Painter gets a scale applied which transparently handles all drawing. (But since the backbuffer and cursor save buffer are also HighDPI, they currently need to be drawn using a hack temporary unscaled Painter object. This will also go away once Bitmaps know about their intrinsic scale.) With this, editing WindowServer.ini to say Width=800 Height=600 ScaleFactor=2 and booting brings up a fully-functional HighDPI UI. (Except for minimizing windows, which will crash the window server until #4932 is merged. And I didn't test the window switcher since the win-tab shortcut doesn't work on my system.) It's all pixel-scaled, but it looks pretty decent :^)
2021-01-15 15:58:55 +00:00
1. Add scale factor concept to WindowServer. WindowServer has a scaled framebuffer/backbuffer. All other bitmaps (both other bitmaps in WindowServer, as well as everything WindowServer-client-side) are always stored at 1x and scaled up when they're painted to the framebuffer. Things will look fine at 2x, but pixely (but window gradients will be smooth already).
2. Let DisplaySettings toggle it WindowServer scale. Now it's possible to switch to and from HighDPI dynamically, using UI.
DisplaySettings: Add UI for switching the scale factor For now, only support 1x and 2x scale. I tried doing something "smarter" first where the UI would try to keep the physical resolution constant when toggling between 1x and 2x, but many of the smaller 1x resolutions map to 2x logical resolutions that Compositor rejects (e.g. 1024x768 becomes 512x384, which is less than the minimum 640x480 that Compositor wants) and it felt complicated and overly magical. So this instead just gives you a 1x/2x toggle and a dropdown with logical (!) resolutions. That is, 800x600 @ 2x gives you a physical resolution of 1600x1200. If we don't like this after trying it for a while, we can change the UI then.
2021-01-16 00:02:17 +00:00
3. Come up with a system to have scale-dependent bitmap and font resources. Use that to use a high-res cursor bitmaps and high-res menu bar text painting in window server. Menu text and cursor will look less pixely. (And window frames too, I suppose.)
4. Let apps opt in to high-res window framebuffers, and convert all apps one-by-one
5. Remove high-res window framebuffer opt-in since all apps have it now.
Docs: Add design doc and implementation plan for highdpi I have a local branch that gets us past implementation stage 1 in this doc, and it seems like a useful enough checkpoint to upstream it and then iterate in tree.
2021-01-12 19:41:36 +00:00
DisplaySettings: Add UI for switching the scale factor For now, only support 1x and 2x scale. I tried doing something "smarter" first where the UI would try to keep the physical resolution constant when toggling between 1x and 2x, but many of the smaller 1x resolutions map to 2x logical resolutions that Compositor rejects (e.g. 1024x768 becomes 512x384, which is less than the minimum 640x480 that Compositor wants) and it felt complicated and overly magical. So this instead just gives you a 1x/2x toggle and a dropdown with logical (!) resolutions. That is, 800x600 @ 2x gives you a physical resolution of 1600x1200. If we don't like this after trying it for a while, we can change the UI then.
2021-01-16 00:02:17 +00:00
We're currently before point 3.
Reference in a new issue Copy permalink