Panvox is a cross-platform player with a Flutter UI and an open-source C++ engine, avox. This article is about playback speed — a feature that looks like “multiply the clock by a factor” and actually has to be split into two render chains.
Drawing the dividing line first
From the user's side, speed is one slider from 0.25× to 64×. From the implementation side it is not: 4× is a line, and the two sides run differently.
- Up to 4×: audio decodes and plays as usual, only the timeline is compressed (speed changes, pitch does not); video decodes as usual too, each frame's display duration is scaled by the factor, and frames are dropped to catch up when it falls behind. There is sound.
- Above 4×: video keeps only keyframes (I-frames — complete pictures that decode independently) and discards everything else at packet intake; audio exits entirely — not muted, the packets never enter the pipeline. There is no sound.
Why is the line at 4×? Directly: 4× is roughly the ceiling at which the decoder can run in real time. Decoding is the heaviest stage in the chain and, at full load, lands around four times real time — so up to 4× every frame still has to be decoded, and the pacing is controlled by dropping frames at the render end (in this stretch the decoder runs flat out, not even yielding time slices: decode the next packet as soon as the previous one is done); beyond that, decoding every frame can no longer keep up, so non-keyframes simply are not decoded. Above 4× the load on the decoder actually drops — the code comment says it outright: “above 4× we only decode I-frames, so it can slow down instead.” Screen refresh rate, the audibility of pitch-shifted audio and what the user wants at high speed are secondary reasons that help the story hang together, not where the line came from.
The steps themselves are product configuration: the whitelist is 0.25 / 0.5 / 0.75 / 1 / 1.25 / 1.5 / 1.75 / 2 / 3 / 4 / 8 / 16 / 32 / 64; whichever steps are checked in settings are the ones the playback menu offers, with 0.5 / 1 / 1.5 / 2 / 4 checked by default.
Up to 4×: pitch-preserving audio, catch-up frame drops
The audio half relies on a plugin: the engine has an audio time-stretch interface whose implementation is SoundTouch (using the WSOLA algorithm — windowing, overlapping, re-stitching at the target rate, hence speed without pitch change). Step changes take effect smoothly on the render thread without flushing already-queued audio frames; coming back to 1× bypasses the stretcher and drops its internal residual window, so the switch does not leak a short tail of the old rate. When the plugin is absent it degrades to resampling — the old behaviour that changes pitch.
The video half is catch-up frame dropping. Each frame's display duration is scaled by the factor (frame duration ÷ speed), and when the render baseline falls behind the master clock far enough, the head-of-queue frame is dropped. Two details look simple and are very easy to get wrong.
- The lag criterion must first require that a wait was intended. When the picture is already behind the master clock the wait duration is zero, and at that point “drop another frame” judges every frame expired — the criterion is almost always true. Measured, about three quarters of frames get dropped and the picture becomes sluggish.
- Dropping a frame must advance the render baseline in step. Otherwise the baseline neither moves forward nor passes the correction condition above, time runs further and further away, every frame is judged expired, and the picture freezes completely — only a seek that resets the clock brings it back briefly.
One more protection: the deterministic initialisation at startup (a 4K HDR hardware-decoded first-frame cluster, for example) drags out a run of “late” frames, so the render side keeps a protection window and does not turn that initialisation cost into user-visible skipped frames.
Above 4×: how I-frame mode renders
Beyond 4×, the player switches to a different way of running. The trigger is one atomic truth assembled from “switch on and speed above 4” (the switch defaults on), and that truth is shared by three places: packet intake, render pacing, and the audio decode thread — sharing it is a discipline, and the reason is below.
Packet intake: drop. Non-key video packets (P/B frames) are discarded outright, audio packets are discarded too, and configuration frames (decoder parameters) pass as usual — they pass so that dropping back to 4× can recover immediately.
Why discard audio entirely rather than just mute it? If it were only muted, the audio pipeline would still consume at normal rate and the packet queue (a hundred-odd packets) would fill up; once enqueue blocks, the demuxer can no longer feed video packets — the video I-frame pipeline starves and the picture gets slower; conversely, if consumption ran too fast the demuxer would read wildly and the progress bar would flicker. Dropping audio packets at intake leaves the demuxer self-throttled by the video I-frame pipeline at the current speed, with no throttling parameters to tune on the audio side. The cost is that A/V sync is switched off as well: above 4× no audio is rendered, so sync is meaningless.
Render pacing: each frame shows for its scaled duration and does not chase the clock. In normal mode the picture follows the master clock (audio by default) with per-frame correction; in I-frame mode each frame shows for exactly “its own duration ÷ speed” — for a source with one keyframe every 2 seconds at 8×, that is 250 milliseconds per frame, advancing like a slideshow. Three corresponding special cases: long gaps are not treated as timestamp jumps (keyframes are far apart by nature); no frame dropping (there are few frames, and dropping one is a real freeze); no clock chasing (there is no audio clock to chase).
Clock and progress: the bar shows where rendering is, not where reading is. This one was changed specifically for I-frame mode: above 4× the disk or network reads ahead, the video queue can hold hundreds of keyframes — tens of seconds to minutes of content — and using the read position as progress would make the bar lie.
Two more: recording is unaffected (recording takes the complete original stream; dropping frames is a playback policy only); and the buffering-recovery criterion changes — up to 4× it waits for the queue to reach the configured duration, above 4× it recovers as soon as there is a video frame.
Source-side I-frame mode. Everything above is the playback side. There is also a source side: some live sources (streams whose server can genuinely switch to keyframe-only output at speed) send only keyframes at higher speeds, and the engine enters I-frame mode to match, rendering with the same pacing. It has a premise — only sources with a real “speed semantics” are predicted this way; for local/VOD sources whose packet stream does not change with speed, the prediction would only mute audio for nothing and disturb the clock. Another route is automatic detection: when consecutive keyframes with different timestamps arrive, the stream itself is judged to be keyframe-only (some sliced and monitoring streams are), rendered in I-frame mode directly, and exited automatically when P/B frames appear.
One easily overlooked discipline: the drop criterion at packet intake and the timeout exemption on the audio decode thread must use the same truth. If the audio side takes the shortcut of “speed greater than 4”, then when the switch is off and audio packets flow normally the timeout exemption still applies, and a real decoding failure is swallowed forever.
Two other kinds of speed
Besides the speed the user picks, two other places in the engine move speed by themselves.
One is live catch-up. In low-latency mode, if both audio and video queues pile beyond twice the buffering target (meaning playback has fallen behind the live edge), the engine raises speed to about 1.2× to burn off the backlog, returning to 1.0 once consumption catches up. It only applies when the user's speed is 1 and never overrides the user's choice — once the user has picked a speed, automatic catch-up stops.
The other is the symmetric treatment at the low end. Below 0.2×, audio is likewise not rendered, the same logic as above 4×: neither end does per-sample time-stretching — one is too slow to be meaningful, the other too fast to carry information.
Both sides of the line: switching is the pitfall
Now the switching. This is where the chain breaks most easily, and the two pitfalls sit symmetrically on either side of the line.
Speeding up (from within 4× to above 4×): dropping only catches new packets. In full mode the packet queue already holds tens of seconds of P/B frames; dropping at intake only affects packets that arrive afterwards, and the old packets already queued have to be decoded and drained before I-frame mode truly begins — that drain takes seconds to tens of seconds. What the user sees: tap 8×, the picture keeps playing slowly for a while, then suddenly starts jumping. The fix treats it as a seek: for sources that can re-position, re-seek to the current render position — flush the old packets, let everything after the restart pass through the already-open drop, and run keyframe-only immediately. Changing speed within the above-4× range (8× to 6×, say) has no backlog and does not hit this path.
Slowing down (from above 4× back to within 4×): the two axes are out of step. Above 4× audio packets were dropped and the demuxer read ahead, so the position where audio “resumes playing” (out ahead) does not match the video render position, and a progress bar following the audio clock jumps back and forth. The fix is again to re-seek to the render position and realign, and again only holds for sources that can re-position; for sources that cannot (pure live), the error can only converge through buffering and subsequent sync.
There is one more inconspicuous lag: once the time-stretch plugin's “playing state” is calibrated it does not reset immediately on returning to 1×. The reason is that the A/V decode-axis divergence accumulated at 0.5× and 2× only converges after the queues drain, and before that the alignment watchdog would misreport the divergence as an anomaly — so the latch stays until the film changes or the player closes. That is not an oversight; it is a trade forced by measurement.
One last experience-level note: above 4× there is no audio at all, so crossing the line feels like “sound ↔ no sound”, not like the volume getting smaller — pitch change only happens within 4×, and only on the degraded path where the time-stretch plugin is absent.
For people building players
Playback speed is not “multiply the clock by a factor”; it is a policy question of which render chain runs at which step. Where the 4× line sits is decided jointly by the audible ceiling of audio time-stretching, the screen refresh rate and what the user is trying to do; crossing the line costs one re-position, so switching between the two sides is best designed as a seek, not as a parameter update. I-frame mode saves a great deal of compute and bandwidth — decoding only keyframes, and a keyframe-only stream can drop decode cost by an order of magnitude — at the price of a slideshow picture and a pile of timing problems that only appear at the moment of switching.
The avox engine is open source on GitHub. The rest of the series: the product introduction, From the open-source engine avox to a product: the cross-platform player Panvox.