This allows the logic for keeping track of whether to resume to the
paused or the playing state when exiting these states. The new
StartingStateHandler also uses the class, since it can also be paused
and unpaused while waiting for samples.
The pause/play actions on the handlers inheriting from the resuming
handler will also now notify the owner that the state has changed so
that it can change icons, etc.
The PlaybackStateChangeEvent wasn't connected up anymore, so the player
wouldn't change icons when stopping playback due to reaching the end of
the stream or encountering an error.
This new state handler will retrieve and display the first frame, while
ensuring that playback can start as soon as possible by buffering two
frames on top of the first frame for the PlayingStateHandler to set its
next frame timer by.
Previously, we assumed the timer was hitting at the correct time. This
meant that if we changed states and the previous state handler had
prepared a next frame, we would immediately display it without checking
the timestamp.
If we encounter an error in the video decoder, we can set a timestamp
for the error item in the queue so that it will display the error only
when the frame that caused the error would have been displayed.
Previously we had dispatch_decoder_error and on_decoder_error serving
the same function, with one not handling the end of stream properly.
There is also a new function to dispatch either an error or a frame to
the owner of this playback manager, so that PlaybackStateHandlers don't
have to duplicate this logic.
For example, consider cases where we want to propagate errors only in
specific instances:
auto result = read_data(); // something like ErrorOr<ByteBuffer>
if (result.is_error() && result.error().code() != EINTR)
continue;
auto bytes = TRY(result);
The TRY invocation will currently copy the byte buffer when the
expression (in this case, just a local variable) is stored into
_temporary_result.
This patch binds the expression to a reference to prevent such copies.
In less trival invocations (such as TRY(some_function()), this will
incur only temporary lifetime extensions, i.e. no functional change.
That matches the terminology used in ITU-T Rec. H.273,
PNG's cICP chunk, and the ICC cicpTag.
Also change the enum values to match the values in the spec --
0 means "not full range" and 1 means "full range".
(For now, keep the "Unspecified" entry around, and give it value 2.
This value is not in the spec.)
No intended behavior change.
Fast seeking does not work correctly when seeking in small increments,
so it is necessary to use accurate seeking when using certain actions.
The PlaybackManager has been changed to accept the seek mode as a
parameter to `seek_to_timestamp` to facilitate this. This now means
that it no longer has to track a seek mode preference.
In cases where the PlaybackManager's earliest buffered or displayed
sample is closer to the seek target than the demuxer's chosen keyframe,
we don't want to seek at all. To enable this, demuxers now receive an
optional parameter with the earliest timestamp that the caller can
still access.
The demuxer in turn returns an optional to indicate when a seek was not
needed, which allows PlaybackManager to avoid clearing its queue and
re-decoding frames.
Storing playback states in virtual classes allows the behavior to be
much more clearly written. Each `PlaybackStateHandler` subclass can
implement some event-handling functions to model their behavior, and
has functions to change its parent PlaybackManager's state to any other
state.
This will allow expanding the functionality of playback in the future,
for example to allow skipping a single frame forward/backward.
A bit of a bikeshed, but status sounds more like a result of an action,
and state sounds more accurate to what the `PlaybackManager` does.
The previous and current state fields of the `PlaybackStateChangeEvent`
are now removed, since they were unused (for now).
This is a lead-up to the refactoring of VideoPlaybackManager to make
that diff more legible.
If USING_AK_GLOBALLY is not defined, the name IsLvalueReference might
not be available in the global namespace. Follow the pattern established
in LibTest to fully qualify AK types in macros to avoid this problem.
This will make it easier to support both string types at the same time
while we convert code, and tracking down remaining uses.
One big exception is Value::to_string() in LibJS, where the name is
dictated by the ToString AO.
When errors are encountered by PlaybackManager, it attempts to switch
states to either Stopped or Corrupted. However, that causes it to set
the last presentation media time to the current playback time while the
last presentation time is unexpectedly negative because the seek never
ended.
Ending the seek before the state changes to Stopped or Corrupted
prevents this situation from happening.
This implements the fastest seeking mode available for tracks with cues
using an array of cue points for each track. It approximates the index
based on the seeking timestamp and then finds the earliest cue point
before the timestamp. The approximation assumes that cues will be on
a regular interval, which I don't believe is always the case, but it
should at least be faster than iterating the whole set of cue points
each time.
Cues are stored per track, but most videos will only have cue points
for the video track(s) that are present. For now, this assumes that it
should only seek based on the cue points for the selected track. To
seek audio in a video file, we should copy the seeked iterator over to
the audio track's iterator after seeking is complete. The iterator will
then skip to the next audio block.
Now that we're able to find the nearest keyframe, we can have a fast
seeking mode that only seeks to keyframes, so that it doesn't have to
also decode inter frames until it reaches the timestamp.
The default is still accurate seeking, so that the entire seeking
implementation can be tested.
This implements the PlaybackManager portion of seeking, so that it can
seek to any frame in the video by dropping all preceding frames until
it reaches the seek point.
MatroskaDemuxer currently will only seek to the start of the video.
That means that every seek has to drop all the frames until it comes
across the target timestamp.
The PlaybackManager::update_presented_frame function was getting out of
hand and adding seeking was making it illegible. This rewrites it to be
(hopefully) quite a bit more readable, and adds a few comments to help
future readers of the code.
In addition, some helpful debugging prints were added that should help
debug any future issues with the player.
With these changes, the seek bar can be used, but only to seek to the
start of the file. Seeking to anywhere else in the file will cause an
error in the demuxer.
The timestamp label that was previously invisible now has its text set
according to either the playback or seek slider's position.
The Demuxer class was changed to return errors for more functions so
that all of the underlying reading can be done lazily. Other than that,
the demuxer interface is unchanged, and only the underlying reader was
modified.
The MatroskaDocument class is no more, and MatroskaReader's getter
functions replace it. Every MatroskaReader getter beyond the Segment
element's position is parsed lazily from the file as needed. This means
that all getter functions can return DecoderErrors which must be
handled by callers.
As new demuxers are added, this will get quite full of files, so it'll
be good to have a separate folder for these.
To avoid too many chained namespaces, the Containers subdirectory is
not also a namespace, but the Matroska folder is for the sake of
separating the multiple classes for parsed information entering the
Video namespace.
Timers keep their previously set interval even for single-shot mode.
We want all timers to fire immediately if they don't have a delay set
in the start() call.
VideoPlayerWidget was keeping a reference to PlaybackManager when
changing files, so the old and new managers would both send frames to
be presented at the same time, causing it to flicker back and forth
between the two videos. However, PlaybackManager no longer relies on
event bubbling to pass events to its parent. By changing it to send
events directly to an Object, it can avoid being ref counted, so that
it will get destroyed with its containing object and stop sending
events.
Frames will now be queued for retrieval by the user of the decoder.
When the end of the current queue is reached, a DecoderError of
category NeedsMoreInput will be emitted, allowing the caller to react
by displaying what was previously retrieved for sending more samples.
I've realized that it probably makes more sense to change the input
transfer characteristics to treat these as sRGB since color conversion
in linear converted from BT.709 doesn't really make sense. If content
creation applications expect media players to display BT.709 without
conversions, this means they expect applications to treat it as sRGB,
since that's what most displays use. That most likely also means they
process it as sRGB internally, meaning we should do the same for our
color primaries conversion.
No longer will the video player explode with error dialogs that then
lock the user out of closing them.
To avoid issues where the playback state becomes invalid when an error
occurs, I've made all decoder errors pass through the frame queue.
This way, when a video is corrupted, there should be no chance that the
playback state becomes invalid due to setting the state to Corrupted
in the event handler while a presentation event is still pending.
Or at least I think that was what caused some issues I was seeing :^)
This system should be a lot more robust if any future errors need to be
handled.
This file will be the basis for abstracting away the out-of-thread or
later out-of-process decoding from applications displaying videos. For
now, the demuxer is hardcoded to be MatroskaParser, since that is all
we support so far. The demuxer should later be selected based on the
file header.
The playback and decoding are currently all done on one thread using
timers. The design of the code is such that adding threading should
be trivial, at least based on an earlier version of the code. For now,
though, it's better that this runs in one thread, as the multithreaded
approach causes the Video Player to lock up permanently after a few
frames are decoded.
