We had numerous NiH-based implementations of audio formats and metadata
that we now no longer need because we either don't make use of the code,
or we replaced its implementation by FFmpeg.
This loader supports whatever format libavformat and libavcodec can
handle. Currently only seekable streams are supported, and we still have
some limitations as to the number of channels and sample format.
Plays all non-streaming audio files at:
https://tools.woolyss.com/html5-audio-video-tester/
Rather than checking the avcodec version in CMake, check it using the
avcodec version macros in the only source file that needs to know about
the AVFrame API/ABI change in version 59.24.100. This is friendlier to
other build systems that would rather avoid configure time checks.
Any data that sticks around in a decoder, especially frames that
haven't been retrieved, may cause issues for playback.
This is especially the case with H.264, since its arbitrary frame
ordering to allow reference frames to precede B-frames causes it to
hold onto frames, and causes the playback manager to get back a frame
at a completely wrong timestamp after seeking.
The timestamp offset of a block was being converted from i16 to u64, so
negative values would overflow and cause timestamps that fall before
the cluster's timestamp from being close to the minimum representable
i64.
The math is also now done using saturating operations to prevent any
other similar issues from occurring.
H.264 in Matroska can have blocks with unordered timestamps. Without
passing these as the presentation timestamp into the FFmpeg decoder,
the frames will not be returned in chronological order.
VideoFrame will now include a timestamp that is used by the
PlaybackManager, rather than assuming that it is the same timestamp
returned by the demuxer.
VP9 continues to function, but this also allows AV1 to be decoded. With
this commit, H.264 is still non-functional, as the decoder requires
some extra initial data from the track definition in the Matroska file.
These aren't particularly small objects, but we were still copying them
around all over the place. When TrackEntry contains data buffers, they
won't need to be copied as well.
This should halve the size of frames in memory for frames with 8-bit
color components, which is the majority of videos.
Calculation of the size of subsampled planes has also been consolidated
into a struct. There are likely some places that will still need to
change over to this, but it should prevent issues due to differing
handling of rounding/ceiling.
BT.2020 will mainly be used with bit depths greater than 8, so having
this specialization is mostly pointless until we use fixed-point math
for higher bit depths.
