LibGfx: Fix a slight mistake in AA ellipse error calculation

The initial signs were wrong for the deltas of f(x), the ellipse
equation. This seemed to be fine for larger circles/ellipses but
broke things at a small scale, this was previously fixed with a
horrible "error = error / 4" hack. With this change, all ellipses
look a little better :^)

This also fixed a signed integer overflow Andreas found with UBSAN,
which happened for circles with a 1px radius.
This commit is contained in:
MacDue 2023-02-19 16:08:11 +00:00 committed by Andreas Kling
parent 2db168acc1
commit bb8c8a67dc
Notes: sideshowbarker 2024-07-17 03:19:14 +09:00

View file

@ -443,8 +443,8 @@ FLATTEN AntiAliasingPainter::Range AntiAliasingPainter::draw_ellipse_part(
int f_squared = y * y;
// 1st and 2nd order differences of f(i)*f(i)
int delta_f_squared = -(static_cast<int64_t>(b_squared) * subpixel_resolution * subpixel_resolution) / a_squared;
int delta2_f_squared = 2 * delta_f_squared;
int delta_f_squared = (static_cast<int64_t>(b_squared) * subpixel_resolution * subpixel_resolution) / a_squared;
int delta2_f_squared = -delta_f_squared - delta_f_squared;
// edge_intersection_area/subpixel_resolution = percentage of pixel intersected by circle
// (aka the alpha for the pixel)
@ -487,11 +487,6 @@ FLATTEN AntiAliasingPainter::Range AntiAliasingPainter::draw_ellipse_part(
auto correct = [&] {
int error = y - y_hat;
// FIXME: The alpha values seem too low, which makes things look
// overly pointy. This fixes that, though there's probably a better
// solution to be found. (This issue seems to exist in the base algorithm)
error /= 4;
delta2_y += error;
delta_y += error;
};