WindowServer: Make HighDPI aware

Almost all logic stays in "logical" (unscaled coordinates), which
means the patch is small and things like DnD, window moving and
resizing, menu handling, menuapplets, etc all work without changes.

Screen knows about phyiscal coordinates and mouse handling internally is
in physical coordinates (so that two 1 pixel movements in succession can
translate to one 1 logical coordinate mouse movement -- only a single
event is sent in this case, on the 2nd moved pixel).

Compositor also knows about physical pixels for its backbuffers. This is
a temporary state -- in a follow-up, I'll try to let Bitmaps know about
their intrinsic scale, then Compositor won't have to know about pixels
any longer. Most of Compositor's logic stays in view units, just
blitting to and from back buffers and the cursor save buffer has to be
done in pixels. The back buffer Painter gets a scale applied which
transparently handles all drawing. (But since the backbuffer and cursor
save buffer are also HighDPI, they currently need to be drawn using a
hack temporary unscaled Painter object. This will also go away once
Bitmaps know about their intrinsic scale.)

With this, editing WindowServer.ini to say

  Width=800
  Height=600
  ScaleFactor=2

and booting brings up a fully-functional HighDPI UI.
(Except for minimizing windows, which will crash the window server
until #4932 is merged. And I didn't test the window switcher since the
win-tab shortcut doesn't work on my system.) It's all pixel-scaled,
but it looks pretty decent :^)

Documentation/HighDPI.md

@@ -18,8 +18,8 @@ Integer scale factors are needed in any case so let's get that working first. Ac
 Desired end state
 -----------------
 
-- Window and Widget rects are in "logical" coordinates, which is the same as pixels at 1x scale
-- Same for mouse cursor
+- All rects  (Window and Widget rects, mouse cursor) are in "logical" coordinates, which is the same as pixels at 1x scale, as much as possible.
+- If something needs to be in pixels, its name starts with `physical_`. Physical coordinates should as much as possible not cross API boundaries.
 - Jury's still out if logical coordinates should stay ints. Probably, but it means mouse cursor etc only have point resolution, not pixel resolution
 - We should have something that can store a collection of (lazily-loaded?) bitmaps and fonts that each represent a single image / font at different scale levels, and at paint time the right representation is picked for the current scale
 
@@ -29,9 +29,10 @@ Implementation plan
 The plan is to have all applications use highdpi backbuffers eventually. It'll take some time to get there though, so here's a plan for getting there incrementally.
 
 0. Add some scaling support to Painter. Make it do 2x nearest neighbor scaling of everything at paint time for now.
-1. Add scale factor concept to WindowServer and let DisplaySettings toggle it. WindowServer has a scaled framebuffer/backbuffer. All other bitmaps (both other bitmaps in WindowServer, as well as everything WindowServer-client-side) are always stored at 1x and scaled up when they're painted to the framebuffer. Things will look fine at 2x, but pixely (but window gradients will be smooth already).
+1. Add scale factor concept to WindowServer. WindowServer has a scaled framebuffer/backbuffer. All other bitmaps (both other bitmaps in WindowServer, as well as everything WindowServer-client-side) are always stored at 1x and scaled up when they're painted to the framebuffer. Things will look fine at 2x, but pixely (but window gradients will be smooth already).
+2. Let DisplaySettings toggle it WindowServer scale. Now it's possible to switch to and from HighDPI dynamically, using UI.
 2. Come up with a system to have scale-dependent bitmap and font resources. Use that to use a high-res cursor bitmaps and high-res menu bar text painting in window server. Menu text and cursor will look less pixely. (And window frames too, I suppose.)
 3. Let apps opt in to high-res window framebuffers, and convert all apps one-by-one
 4. Remove high-res window framebuffer opt-in since all apps have it now.
 
-We're currently before point 1.
+We're currently before point 2.

Userland/Services/WindowServer/Compositor.cpp

@@ -92,20 +92,22 @@ Compositor::Compositor()
 void Compositor::init_bitmaps()
 {
     auto& screen = Screen::the();
-    auto size = screen.size();
+    auto physical_size = screen.physical_size();
 
-    m_front_bitmap = Gfx::Bitmap::create_wrapper(Gfx::BitmapFormat::RGB32, size, screen.pitch(), screen.scanline(0));
+    m_front_bitmap = Gfx::Bitmap::create_wrapper(Gfx::BitmapFormat::RGB32, physical_size, screen.pitch(), screen.scanline(0));
+    m_front_painter = make<Gfx::Painter>(*m_front_bitmap);
+    m_front_painter->scale(screen.scale_factor());
 
     if (m_screen_can_set_buffer)
-        m_back_bitmap = Gfx::Bitmap::create_wrapper(Gfx::BitmapFormat::RGB32, size, screen.pitch(), screen.scanline(size.height()));
+        m_back_bitmap = Gfx::Bitmap::create_wrapper(Gfx::BitmapFormat::RGB32, physical_size, screen.pitch(), screen.scanline(physical_size.height()));
     else
-        m_back_bitmap = Gfx::Bitmap::create(Gfx::BitmapFormat::RGB32, size);
-
-    m_temp_bitmap = Gfx::Bitmap::create(Gfx::BitmapFormat::RGB32, size);
-
-    m_front_painter = make<Gfx::Painter>(*m_front_bitmap);
+        m_back_bitmap = Gfx::Bitmap::create(Gfx::BitmapFormat::RGB32, physical_size);
     m_back_painter = make<Gfx::Painter>(*m_back_bitmap);
+    m_back_painter->scale(screen.scale_factor());
+
+    m_temp_bitmap = Gfx::Bitmap::create(Gfx::BitmapFormat::RGB32, physical_size);
     m_temp_painter = make<Gfx::Painter>(*m_temp_bitmap);
+    m_temp_painter->scale(screen.scale_factor());
 
     m_buffers_are_flipped = false;
 
@@ -451,8 +453,14 @@ void Compositor::compose()
         }());
 
         // Copy anything rendered to the temporary buffer to the back buffer
-        for (auto& rect : flush_transparent_rects.rects())
-            back_painter.blit(rect.location(), *m_temp_bitmap, rect);
+        {
+            // FIXME: Give Bitmap an intrinsic scale factor and make Painter::blit() do the right thing if both it and the passed bitmap have scale factors:
+            // If a 2x scaled bitmap is blitted on a 2x scaled painter, it should be blitted without scale.
+            Gfx::Painter unscaled_back_painter(*m_back_bitmap);
+            auto scale = Screen::the().scale_factor();
+            for (auto& rect : flush_transparent_rects.rects())
+                unscaled_back_painter.blit(rect.location() * scale, *m_temp_bitmap, rect * scale);
+        }
 
         Gfx::IntRect geometry_label_damage_rect;
         if (draw_geometry_label(geometry_label_damage_rect))
@@ -530,6 +538,10 @@ void Compositor::flush(const Gfx::IntRect& a_rect)
 {
     auto rect = Gfx::IntRect::intersection(a_rect, Screen::the().rect());
 
+    // Almost everything in Compositor is in logical coordintes, with the painters having
+    // a scale applied. But this routine accesses the backbuffer pixels directly, so it
+    // must work in physical coordinates.
+    rect = rect * Screen::the().scale_factor();
     Gfx::RGBA32* front_ptr = m_front_bitmap->scanline(rect.y()) + rect.x();
     Gfx::RGBA32* back_ptr = m_back_bitmap->scanline(rect.y()) + rect.x();
     size_t pitch = m_back_bitmap->pitch();
@@ -709,7 +721,8 @@ bool Compositor::set_resolution(int desired_width, int desired_height)
         dbg() << "Compositor: Tried to set invalid resolution: " << desired_width << "x" << desired_height;
         return false;
     }
-    bool success = Screen::the().set_resolution(desired_width, desired_height);
+
+    bool success = Screen::the().set_resolution(desired_width, desired_height, 1);
     init_bitmaps();
     invalidate_occlusions();
     compose();
@@ -790,14 +803,16 @@ void Compositor::draw_cursor(const Gfx::IntRect& cursor_rect)
     auto& wm = WindowManager::the();
 
     if (!m_cursor_back_bitmap || m_cursor_back_bitmap->size() != cursor_rect.size()) {
-        m_cursor_back_bitmap = Gfx::Bitmap::create(Gfx::BitmapFormat::RGB32, cursor_rect.size());
+        m_cursor_back_bitmap = Gfx::Bitmap::create(Gfx::BitmapFormat::RGB32, cursor_rect.size() * Screen::the().scale_factor());
         m_cursor_back_painter = make<Gfx::Painter>(*m_cursor_back_bitmap);
     }
 
     auto& current_cursor = m_current_cursor ? *m_current_cursor : wm.active_cursor();
-    m_cursor_back_painter->blit({ 0, 0 }, *m_back_bitmap, current_cursor.rect().translated(cursor_rect.location()).intersected(Screen::the().rect()));
-    auto& back_painter = *m_back_painter;
-    back_painter.blit(cursor_rect.location(), current_cursor.bitmap(), current_cursor.source_rect(m_current_cursor_frame));
+    // FIXME: Give Bitmap an intrinsic scale factor and make Painter::blit() do the right thing if both it and the passed bitmap have scale factors:
+    // If a 2x scaled bitmap is blitted on a 2x scaled painter, it should be blitted without scale.
+    m_cursor_back_painter->blit({ 0, 0 }, *m_back_bitmap, (current_cursor.rect().translated(cursor_rect.location()) * Screen::the().scale_factor()).intersected(Screen::the().physical_rect()));
+
+    m_back_painter->blit(cursor_rect.location(), current_cursor.bitmap(), current_cursor.source_rect(m_current_cursor_frame));
 
     m_last_cursor_rect = cursor_rect;
 }
@@ -807,7 +822,11 @@ void Compositor::restore_cursor_back()
     if (!m_cursor_back_bitmap)
         return;
 
-    m_back_painter->blit(m_last_cursor_rect.location().constrained(Screen::the().rect()), *m_cursor_back_bitmap, { { 0, 0 }, m_last_cursor_rect.intersected(Screen::the().rect()).size() });
+    // FIXME: Give Bitmap an intrinsic scale factor and make Painter::blit() do the right thing if both it and the passed bitmap have scale factors:
+    // If a 2x scaled bitmap is blitted on a 2x scaled painter, it should be blitted without scale.
+    Gfx::Painter unscaled_back_painter(*m_back_bitmap);
+    auto last_physical_cursor_rect = (m_last_cursor_rect * Screen::the().scale_factor()).intersected(Screen::the().physical_rect());
+    unscaled_back_painter.blit(last_physical_cursor_rect.location(), *m_cursor_back_bitmap, { { 0, 0 }, last_physical_cursor_rect.size() });
 }
 
 void Compositor::notify_display_links()

Userland/Services/WindowServer/Screen.cpp

@@ -46,7 +46,7 @@ Screen& Screen::the()
     return *s_the;
 }
 
-Screen::Screen(unsigned desired_width, unsigned desired_height)
+Screen::Screen(unsigned desired_width, unsigned desired_height, int scale_factor)
 {
     ASSERT(!s_the);
     s_the = this;
@@ -60,8 +60,8 @@ Screen::Screen(unsigned desired_width, unsigned desired_height)
         m_can_set_buffer = true;
     }
 
-    set_resolution(desired_width, desired_height);
-    m_cursor_location = rect().center();
+    set_resolution(desired_width, desired_height, scale_factor);
+    m_physical_cursor_location = physical_rect().center();
 }
 
 Screen::~Screen()
@@ -69,26 +69,26 @@ Screen::~Screen()
     close(m_framebuffer_fd);
 }
 
-bool Screen::set_resolution(int width, int height)
+bool Screen::set_resolution(int width, int height, int scale_factor)
 {
-    FBResolution resolution { 0, (unsigned)width, (unsigned)height };
-    int rc = fb_set_resolution(m_framebuffer_fd, &resolution);
+    FBResolution physical_resolution { 0, (unsigned)(width * scale_factor), (unsigned)(height * scale_factor) };
+    int rc = fb_set_resolution(m_framebuffer_fd, &physical_resolution);
 #ifdef WSSCREEN_DEBUG
     dbg() << "fb_set_resolution() - return code " << rc;
 #endif
     if (rc == 0) {
-        on_change_resolution(resolution.pitch, resolution.width, resolution.height);
+        on_change_resolution(physical_resolution.pitch, physical_resolution.width, physical_resolution.height, scale_factor);
         return true;
     }
     if (rc == -1) {
         dbg() << "Invalid resolution " << width << "x" << height;
-        on_change_resolution(resolution.pitch, resolution.width, resolution.height);
+        on_change_resolution(physical_resolution.pitch, physical_resolution.width, physical_resolution.height, scale_factor);
         return false;
     }
     ASSERT_NOT_REACHED();
 }
 
-void Screen::on_change_resolution(int pitch, int width, int height)
+void Screen::on_change_resolution(int pitch, int physical_width, int physical_height, int scale_factor)
 {
     if (m_framebuffer) {
         size_t previous_size_in_bytes = m_size_in_bytes;
@@ -103,10 +103,11 @@ void Screen::on_change_resolution(int pitch, int width, int height)
     ASSERT(m_framebuffer && m_framebuffer != (void*)-1);
 
     m_pitch = pitch;
-    m_width = width;
-    m_height = height;
+    m_width = physical_width / scale_factor;
+    m_height = physical_height / scale_factor;
+    m_scale_factor = scale_factor;
 
-    m_cursor_location.constrain(rect());
+    m_physical_cursor_location.constrain(physical_rect());
 }
 
 void Screen::set_buffer(int index)
@@ -130,20 +131,21 @@ void Screen::set_scroll_step_size(unsigned step_size)
 
 void Screen::on_receive_mouse_data(const MousePacket& packet)
 {
-    auto prev_location = m_cursor_location;
+    auto prev_location = m_physical_cursor_location / m_scale_factor;
     if (packet.is_relative) {
-        m_cursor_location.move_by(packet.x * m_acceleration_factor, packet.y * m_acceleration_factor);
+        m_physical_cursor_location.move_by(packet.x * m_acceleration_factor, packet.y * m_acceleration_factor);
 #ifdef WSSCREEN_DEBUG
-        dbgln("Screen: New Relative mouse point @ {}", m_cursor_location);
+        dbgln("Screen: New Relative mouse point @ {}", m_physical_cursor_location);
 #endif
     } else {
-        m_cursor_location = { packet.x * m_width / 0xffff, packet.y * m_height / 0xffff };
+        m_physical_cursor_location = { packet.x * physical_width() / 0xffff, packet.y * physical_height() / 0xffff };
 #ifdef WSSCREEN_DEBUG
-        dbgln("Screen: New Absolute mouse point @ {}", m_cursor_location);
+        dbgln("Screen: New Absolute mouse point @ {}", m_physical_cursor_location);
 #endif
     }
 
-    m_cursor_location.constrain(rect());
+    m_physical_cursor_location.constrain(physical_rect());
+    auto new_location = m_physical_cursor_location / m_scale_factor;
 
     unsigned buttons = packet.buttons;
     unsigned prev_buttons = m_mouse_button_state;
@@ -152,7 +154,7 @@ void Screen::on_receive_mouse_data(const MousePacket& packet)
     auto post_mousedown_or_mouseup_if_needed = [&](MouseButton button) {
         if (!(changed_buttons & (unsigned)button))
             return;
-        auto message = make<MouseEvent>(buttons & (unsigned)button ? Event::MouseDown : Event::MouseUp, m_cursor_location, buttons, button, m_modifiers);
+        auto message = make<MouseEvent>(buttons & (unsigned)button ? Event::MouseDown : Event::MouseUp, new_location, buttons, button, m_modifiers);
         Core::EventLoop::current().post_event(WindowManager::the(), move(message));
     };
     post_mousedown_or_mouseup_if_needed(MouseButton::Left);
@@ -160,19 +162,19 @@ void Screen::on_receive_mouse_data(const MousePacket& packet)
     post_mousedown_or_mouseup_if_needed(MouseButton::Middle);
     post_mousedown_or_mouseup_if_needed(MouseButton::Back);
     post_mousedown_or_mouseup_if_needed(MouseButton::Forward);
-    if (m_cursor_location != prev_location) {
-        auto message = make<MouseEvent>(Event::MouseMove, m_cursor_location, buttons, MouseButton::None, m_modifiers);
+    if (new_location != prev_location) {
+        auto message = make<MouseEvent>(Event::MouseMove, new_location, buttons, MouseButton::None, m_modifiers);
         if (WindowManager::the().dnd_client())
             message->set_mime_data(WindowManager::the().dnd_mime_data());
         Core::EventLoop::current().post_event(WindowManager::the(), move(message));
     }
 
     if (packet.z) {
-        auto message = make<MouseEvent>(Event::MouseWheel, m_cursor_location, buttons, MouseButton::None, m_modifiers, packet.z * m_scroll_step_size);
+        auto message = make<MouseEvent>(Event::MouseWheel, new_location, buttons, MouseButton::None, m_modifiers, packet.z * m_scroll_step_size);
         Core::EventLoop::current().post_event(WindowManager::the(), move(message));
     }
 
-    if (m_cursor_location != prev_location)
+    if (new_location != prev_location)
         Compositor::the().invalidate_cursor();
 }
 

Userland/Services/WindowServer/Screen.h

@@ -41,24 +41,32 @@ const unsigned scroll_step_size_min = 1;
 
 class Screen {
 public:
-    Screen(unsigned width, unsigned height);
+    Screen(unsigned width, unsigned height, int scale_factor);
     ~Screen();
 
-    bool set_resolution(int width, int height);
+    bool set_resolution(int width, int height, int scale_factor);
     bool can_set_buffer() { return m_can_set_buffer; }
     void set_buffer(int index);
 
+    int physical_width() const { return width() * scale_factor(); }
+    int physical_height() const { return height() * scale_factor(); }
+    size_t pitch() const { return m_pitch; }
+
     int width() const { return m_width; }
     int height() const { return m_height; }
-    size_t pitch() const { return m_pitch; }
+    int scale_factor() const { return m_scale_factor; }
+
     Gfx::RGBA32* scanline(int y);
 
     static Screen& the();
 
+    Gfx::IntSize physical_size() const { return { physical_width(), physical_height() }; }
+    Gfx::IntRect physical_rect() const { return { { 0, 0 }, physical_size() }; }
+
     Gfx::IntSize size() const { return { width(), height() }; }
-    Gfx::IntRect rect() const { return { 0, 0, width(), height() }; }
+    Gfx::IntRect rect() const { return { { 0, 0 }, size() }; }
 
-    Gfx::IntPoint cursor_location() const { return m_cursor_location; }
+    Gfx::IntPoint cursor_location() const { return m_physical_cursor_location / m_scale_factor; }
     unsigned mouse_button_state() const { return m_mouse_button_state; }
 
     double acceleration_factor() const { return m_acceleration_factor; }
@@ -71,7 +79,7 @@ public:
     void on_receive_keyboard_data(::KeyEvent);
 
 private:
-    void on_change_resolution(int pitch, int width, int height);
+    void on_change_resolution(int pitch, int physical_width, int physical_height, int scale_factor);
 
     size_t m_size_in_bytes;
 
@@ -82,8 +90,9 @@ private:
     int m_width { 0 };
     int m_height { 0 };
     int m_framebuffer_fd { -1 };
+    int m_scale_factor { 1 };
 
-    Gfx::IntPoint m_cursor_location;
+    Gfx::IntPoint m_physical_cursor_location;
     unsigned m_mouse_button_state { 0 };
     unsigned m_modifiers { 0 };
     double m_acceleration_factor { 1.0 };

Userland/Services/WindowServer/main.cpp

@@ -89,8 +89,8 @@ int main(int, char**)
         return 1;
     }
 
-    WindowServer::Screen screen(wm_config->read_num_entry("Screen", "Width", 1024),
-        wm_config->read_num_entry("Screen", "Height", 768));
+    int scale = wm_config->read_num_entry("Screen", "ScaleFactor", 1);
+    WindowServer::Screen screen(wm_config->read_num_entry("Screen", "Width", 1024 / scale), wm_config->read_num_entry("Screen", "Height", 768 / scale), scale);
     screen.set_acceleration_factor(atof(wm_config->read_entry("Mouse", "AccelerationFactor", "1.0").characters()));
     screen.set_scroll_step_size(wm_config->read_num_entry("Mouse", "ScrollStepSize", 4));
     WindowServer::Compositor::the();