If `Document::resolve()` was called during parsing, it'd change the
reader's current position, so the parsing code that called it would
then end up at an unexpected position in the file.
Parser.cpp already had special-case recovery when a stream's length
was stored in an indirect reference.
Commit ead02da98ac70c ("/JBIG2Globals") in #23503 added another case
where we could resolve indirect reference during parsing, but wasn't
aware of having to save and restore the reader position for that.
Put the save/restore code in `DocumentParser::parse_object_with_index`
instead, right before the place that ultimately changes the reader's
position during `Document::resolve`. This fixes `/JBIG2Globals` and
lets us remove the special-case code for `/Length` handling.
Since this is kind of subtle, include a test.
Rather than make path segments virtual and refcounted let's store
`Gfx::Path`s as a list of `FloatPoints` and a separate list of commands.
This reduces the size of paths, for example, a `MoveTo` goes from 24
bytes to 9 bytes (one point + a single byte command), and removes a
layer of indirection when accessing segments. A nice little bonus is
transforming a path can now be done by applying the transform to all
points in the path (without looking at the commands).
Alongside this there's been a few minor API changes:
- `path.segments()` has been removed
* All current uses could be replaced by a new `path.is_empty()` API
* There's also now an iterator for looping over `Gfx::Path` segments
- `path.add_path(other_path)` has been removed
* This was a duplicate of `path.append_path(other_path)`
- `path.ensure_subpath(point)` has been removed
* Had one use and is equivalent to an `is_empty()` check + `move_to()`
- `path.close()` and `path.close_all_subpaths()` assume an implicit
`moveto 0,0` if there's no `moveto` at the start of a path (for
consistency with `path.segmentize_path()`).
Only the last point could change behaviour (though in LibWeb/SVGs all
paths start with a `moveto` as per the spec, it's only possible to
construct a path without a starting `moveto` via LibGfx APIs).
Text can be rendered in various ways in PDFs: Filled, stroked,
both filled and stroked, set as clipping path, hidden, or
some combinations thereof.
We don't implement any of this at the moment except "filled".
Hidden text is used in scanned documents: The image of the scan is
drawn in the background, and then OCRd text is "drawn" as hidden
on top of the scanned bitmap. That way, the (hidden) text can be
selected and copied, and it looks like you're selecting text from
the scanned bitmap. Find-in-page also works similarly. (We currently
have neither text selection nor find-in-page, but one day we will.)
Now that we have pretty good support for CCITT and are growing some
support for JBIG2, we now draw both the scanned background image
as well as the foreground text. They're not always perfectly aligned.
This change makes it so that we don't render text that's marked as
hidden. (We still do most of the coordinate math, which will probably
come in handy at some point when we implement text selection.)
This makes these scanned documents appear as they're supposed to
appear (at least in documents where we manage to decode the background
bitmap).
This also adds a debug option to force rendering of hidden text.
Several ramifications:
* /JBIG2Globals is an indirect reference, which means we now need
a Document for unfiltering. (Technically, other decode parameters
can also be indirect objects and we should use the Document to
resolve() those too, but in practice it only seems to be needed
for /JBIG2Globals.)
* Since /JBIG2Globals are so rare, we just parse once for each
image that use them, and decode_embedded() now receives a
Vector<ReadonlyBytes> with all sections of sequences of
segments.
* Internally, decode_segment_headers() is now called several times
for embedded JBIG2s with multiple such sections (e.g. PDFs with
/JBIG2Globals).
* That means `data` is now no longer part of JBIG2LoadingContext
and things get slightly reshuffled due to this.
This completes the LibPDF part of JBIG2 support. Once LibGfx
implements actual decoding of JBIG2s, things should start to
Just Work in PDFs.
Except for /JBIG2Globals, which we bail out on for now. In my 1000
files, 13 use JBIG2, and of those, 2 use JBIG2Globals (0000372.pdf e.g.
page 11 and 0000857.pdf e.g. page 1), and only one (the latter) of the
two uses the same JBIG2Globals stream for more than a single image.
JBIG2ImageDecoderPlugin cannot decode the data yet, so no behavior
change, but with `#define JBIG2_DEBUG 1` at the top of that file,
it now prints segment header info for PDFs containing JBIG2 data :^)
Fixes pages 17-19 on
https://www.iro.umontreal.ca/~feeley/papers/ChevalierBoisvertFeeleyECOOP15.pdf
Calling the fill handler after painting the stroke as previously doesn't
work, since we need to set up the clip before both stroke and fill, and
unset it after both. The duplication is a bit unfortunate, but also
minor.
Turns out the spec didn't mean that the whole range is populated,
but that one of these ranges is populated. So take the argmax.
As fallout, explicitly mark the Liberation fonts as nonsymbolic
when we use them for the 14 standard fonts. Else, we'd regress
"PostScrõpt", since the Liberation fonts would otherwise go down
the "is symbolic or doesn't have explicit encoding" codepath,
since the standard fonts usually don't have an explicit encoding.
As a fallout from _that_, since the 14 standard fonts now go down
the regular truetype rendering path, and since we don't implement
lookup by postscript name yet, glyphs not present in Liberation
now cause text to stop rendering with a diag, instead of rendering
a "glyph not found" symbol. That isn't super common, only an
additional 4 files appear for the "'post' table not yet implemented"
diag. Since we'll implement that soon, this seems fine until then.
An array image mask contains a min/max range for each channel,
and if each channel of a given pixel is in that channel's range,
that pixel is masked out (i.e. transparent). (It's similar to
having a single color or palette index be transparent, but it
supports a range of transparent colors if desired.)
What makes this a bit awkward is that the range is relative to the
origin bits per pixel and the inputs to the image's color space.
So an indexed (palettized) image with 4bpp has a 2-element mask
array where both entries are between 0 and 15.
We currently apply masks after converting images to a Gfx::Bitmap,
that is after converting to 8bpp sRGB. And we do this by mapping
everything to 8bpp very early on in load_image().
This leaves us with a bunch of options that are all a bit awkward:
1. Make load_image() store the up- (or for 16bpp inputs, down-)
sampled-to-8bpp pixel data. And also return if we expanded the
pixel range while resampling (for color values) or not (for
palettized images). Then, when applying the image filter,
resample the array bounds in exactly the same way. This requires
passing around more stuff.
2. Like 1, but pass in the mask array to load_image() and apply
the mask right there and then. This means we'd apply mask arrays
at a different time than other masks.
3. Make the function that computes the mask from the mask array
work from the original, unprocessed image data. This is the most
local change, but probably also requires the largest amount of
code (in return, the color mask for 16bpp images is precise, in
addition that it separates concerns the most nicely).
This goes with 3 for now.
Introduces CIDIterator, an iterator type for iterating over CIDs.
Also introduces Type0CMap which can return a CIDIterator given some
bytes.
The existing code of treating the bytes as an identity map of
big-endian u16s is now implemented in IdentityType0CMap.
No behavior change.
This will allow us to get at the font's glyphs as paths, which will
eventually enable us to implement glyph rotation. We'll have to do our
own caching then, but we can then hopefully share the caching across the
Type0 / Type1 / TrueType codepaths.
It also gives us access to a font's glyphs by glyph id, which will help
us implementing looking up glyph ids by postscript name. (Else we'd
have to plumb through a whole Painter::draw_glyph_by_postscript_name()
API just for LibPDF.)
No behavior change.
Liberation Sans still doesn't have the vast majority of the
Zapf Dingbats glyphs, but now we map the Zapf Dingbats names to good
unicode values. So we only need to use a different font and all should
work. (And Liberation Sans has _some_ of the glyphs, like 13 of the
223.) And we now render empty squares instead of wrong glyphs for the
ones we don't have.
I haven't seen any PDFs using ZapfDingbats in the wild, but they
probably exist somewhere.
(Tests/LibPDF/standard-14-fonts.pdf is a synthetic PDF using it.)
Turns out there's a spec that goes with the table.
The big change here is that we can now map `uni1234` to 0x1234 and
`u123456` to 0x123456.
The parts where we split a name on `_` and map each component
and the part where we're supposed to allow multiple groups of 4
after `uni` aren't implemented yet.
The ZapfDingbats lookup is also still missing.
I haven't seen this have an effect in practice, but it's easy to
construct a PDF with a custom encoding where it would make a
difference.
We use Liberation Sans for the actual glyph for these, and that's
missing some (Symbol) / all (ZapfDingbats) of the glyphs we need
for these two standard fonts (...or at least the mapping from
name to glyph, not sure). But still, better rendering squares than
completely incorrect glpyhs.
Our code deciding what to do when a value isn't found in an encoding,
or when the name doesn't map to a glpyh, also needs work, but that's
mostly independent of this change. I think this is a nice small
standalone progression.
Makes text show up on 0000646.pdf pages 87-92, which for some reason
renders all text using 2x2 images with huge masks that contain
rendered text outlines.
This will need further thought once we implement support for the
truetype 'post' table, but for now it's correct most of the time,
and better than not doing it.
...and for fallback fonts too.
We use Liberation Sans (a truetype font) for standard and fallback
fonts. So we should use the standard PDF algorithm for mapping bytes
to truetype glyphs. TrueTypePainter knows how to do this.
Makes the "fi" ligature in the title on page 1 of 5014.CIDFont_Spec.pdf
or the dotless-i in the title of page 2 of ThinkingInPostScript.pdf
show up. They use Helvetica and TImes, and Helvetica and Symbol
respecitively (with -Bold variants).
Since ScaledFont bakes the size of the font into the font type, we
do the same for Type1 fonts, and then have to divide by the font height
when figuring out what to scale by. For a target width of 0, chances are
the source width is also 0, and we end up with NaN due to dividing
0 by 0. This then triggered the `VERIFY(isfinite(error))` in
can_approximate_bezier_curve() in Painter.cpp.
Check for this case and scale by 0 instead of dividing.
It could happen that the denominator is 0 without the numerator being 0,
but it's not clear what that's supposed to mean. In this case we'd end
up with +inf/-inf, which would also trigger the assert. I haven't seen
this case in practice, so let's not worry about that for now.
(A nicer longer-term fix is probably to make LibPDF use VectorFont
instead of ScaledFont, so that we don't have to bake the font size into
the font type. Then we won't need this division at all. In the meantime,
this fixes the crash.)
Fixes a crash on page 66 of
https://developer.apple.com/library/archive/documentation/mac/pdf/Text.pdf
Fixes a crash on page 37 of
https://open-std.org/jtc1/sc22/wg14/www/docs/n3220.pdf
Fixes crashes in `0000310.pdf`, `0000430.pdf`, `0000229.pdf`.
Brings down the number of crashes on my 1000 file test set from
5 with 3 distinct stacks to 2 with 1 distinct stack.
(The number went up from 3 crashes with 2 distinct stacks to 5/3 when we
started rendering much more text when Type0 font support was added.
This fixes the crashes we had before Type0 support.)
Non-CID-keyed fonts in PDFs have 8-bit codepoints which are mapped from
bytes to character names via encoding.
TrueType fonts don't index glyphs by name (Type1 fonts do), so the fix
(codified in the spec) was to make a list of all possible glyph names
and map those to (16-bit) unicode values, and then pass those into the
truetype cmap.
(As a fallback, we're supposed to look at the optional names in the
font's "post" table. That part isn't implemented here yet.)
(Note that this affects the behavior of fallback fonts for TrueType
fonts, but not yet fallback fonts for Type1 fonts, and neither the
behavior of the 14 built-in Type1 fonts (which we implement as
fallback fonts), since the TrueType fallback in Type1Font.cpp does
not use this algorithm yet. This will be fixed in a future patch.)
For `:#xx` in names, we now also handle lower-case hex digits.
The spec is silent on the case of these hex digits.
Our previous check (isxdigit(), and now is_ascii_hex_digit()) lets
through lower-case hex digits, so it seems better to handle them
rather than computing e.g. `'a' - 'A' + 10` (== 42 -- off by 32!).
I don't know if this has any visible effect on any files, but it's
more correct, and less code, and the code looks more like the code
in Filter::decode_ascii_hex().
Both type 1 and type 2 spec tell us to do this.
I haven't observed a difference from this, but I noticed it in the
spec while I was touching this code. Probably good to do what the
spec tells us to do.
With this, a character can be defined that uses two existing glyphs.
This is useful for umlauts and the like, which then just need to
reference e.g. the glyphs named "a" and "dieresis" and provide a
translation.
Makes umlauts appear on some PDFs using CFF type2 data in Type 1
fonts.
It is sometimes truncated in fonts embedded in PDFs, and the data
is not needed to render PDFs. 2 of my 1000 test PDFs used to
complain "Could not load OS2 v1: Not enough data" and 1
"Could not load OS2 v2: Not enough data" before.
Increases number of PDFs that render without diagnostics from
764 to 765 (and decreases the number of distinct error messages
from 27 to 25).
It is sometimes truncated in fonts embedded in PDFs, and the data
is not needed to render PDFs. 26 of my 1000 test files complained
"Could not load Hmtx: Not enough data" before.
Increases number of PDFs that render without diagnostics from
743 to 764.
