Subtitles are the most underrated module in a player. The common kinds fall into three groups, and they are handled in completely different ways:
- Effects subtitles (ASS/SSA): they carry a layout language of their own — positioning, animation, fonts and karaoke per-character colouring are all instructions — so a layout engine must reproduce them; they cannot be flattened into plain text;
- Bitmap subtitles (PGS): the subtitle format on Blu-ray discs, essentially a series of timestamped little bitmaps plus a palette, with no notion of text, font or style; they can only be overlaid at their coordinates;
- Text subtitles (SRT/mov_text): plain text, rasterized by us.
All three paths have to converge in one place to reach the screen.
One canvas, one winner
Inside the engine all three kinds finally render into the same “subtitle canvas”: an RGBA bitmap with a transparent background, composited onto the picture with sourceOver. There are three internal slots — embedded track, external file, AI output — and the last activated wins, with no fallback on an empty window; at most one subtitle run exists at any moment. The menu line “effects subtitles are fully rendered by the engine and mutually exclusive with external subtitles” comes from exactly this single-winner semantics: it is not a switch, it is arbitration.
There are two lanes to the screen (lanes in the engine), split by output method, not by decode method: the processing chain composites in Vulkan; in HDR passthrough the canvas switches to the platform's native render leg (DX11 / Metal / GLES), where it only overlays and does not touch the video signal. All four backends have canvas implementations, and even the colour domain in the passthrough state is handled deliberately: the Vulkan passthrough canvas works in 16-bit float linear where 1.0 is 80 nits; the Metal EDR leg defines 1.0 as 100 nits. HDR-passthrough subtitles are implemented; human acceptance on real HDR screens is still pending.
The bigger value of this design is that it leaves the platform render leg with a contract that only mixes. Whatever the subtitle came from — ASS layout, PGS bitmap or SRT text — it leaves the engine as an RGBA canvas, and the platform render leg only has to do one thing: composite that canvas onto the picture with sourceOver. It does not need to know any subtitle format, nor which kind is playing. The direct benefit: bringing up a new platform means writing a canvas layer, not a subtitle rendering system; the four backends are four implementations of one contract, each handling only its own colour domain and blend.
Canvas size is an engineering trade: above 1080p, subtitles always render at 1920×1080 and are stretched by the GPU — a 4K canvas costs 33 MB per frame, brought down to 8.3 MB, with fewer texture-upload syncs, and the difference is nearly invisible to the eye.
Style belongs to the source; what you can adjust is global
ASS styling (font, colour, position) belongs to the source. The engine's interface for “change style via a single parameter” is silently ignored — changing it has to go through the script layer: font size uses the layout engine's scaling (immediate reflow, no subtitle reload), plus three global transforms: overall scale, offset and opacity. External SRT renders with the built-in font, its size scaling with picture height.
Two numbers in the UI: subtitle delay ±60 seconds in 100-millisecond steps; the font-size slider maps to 0.55–1.45×, defaulting to exactly 1.0. Both take effect immediately and persist for the next playback.
Not done yet: style enhancement (colour, background, font name) is queued; online subtitle search is not in the UI. Bilingual same-screen was removed, and the second render leg built for it retired with it — subtitles now have exactly one engine path.
Three real incidents
Incident one: switching tracks crashes. On one class of sources, switching to the embedded subtitle and back to the external one always crashed; worse, the second switch froze everything. A test host that replays the application call sequence frame by frame reproduced it reliably, and two independent mechanisms fell out. First, when switching tracks the layout engine's overlay was destroyed outside the lock while the render and IO threads were still dereferencing it inside the lock — a use-after-free. Second, the canvas layer “detached and re-attached” on every switch, triggering a full rebuild, and the second Vulkan resource rebuild hung inside the graphics driver (the on-site dump showed a COM message wait that never returned). Two fixes: detach the overlay state inside the lock and destroy it outside, copying the canvas content into a stable snapshot; and make the canvas layer sticky — attach once, never detach during playback, just clear the content when there are no subtitles. Six repeated switching rounds passed afterwards.
Incident two: yellow became cyan. PGS subtitles from a 4K Blu-ray appeared offset and too small. Two layers of cause: PGS composition coordinates are written in the 1080p graphics plane, and the old implementation pasted them 1:1 onto 4K pixels, so they were naturally misaligned; and the palette channel order is 0xAARRGGBB while the code read the low byte as red, rendering yellow (0,255,255) as cyan (255,255,0). The second bug hid for a long time because white has equal channels. The fix: resample on the decode side against the composite canvas (no resampling at equal size, bilinear otherwise), and correct the channel order. Verification: a probe-geometry A/B put the subtitle at (691,863) before and (1382,1726) after; end-to-end frame capture plus an offline matrix passed all 77 checks.
Incident three: a whole line of boxes. On iPhone, Chinese subtitles were all hollow boxes while English was fine in the same frame. I suspected encoding first — but if it were a decode problem the number of boxes would track bytes; measured, 11 boxes matched 11 characters, so it was the font layer. The cause: the layout engine on iOS asks the system text framework for glyphs, and its fallback chain points at a system-private framework that cannot be opened inside the sandbox, so every character became a “no glyph” placeholder. The fix: feed the bundled fonts to the layout engine before plugin initialisation. Verified on device: the font-selection log pointed at the bundled font, and both the missing-glyph and font-open-failure counters dropped to zero.
Encoding: the boundaries of automatic repair
Garbled SRT is almost always an encoding problem: GBK read as UTF-8 produces replacement characters. The chain's policy: strict UTF-8 validation first (including overlong, truncated and surrogate checks) → transcode through GBK/GB18030 → strip the UTF-8 BOM. UTF-16 SRT gets a guard on the application side: if a BOM is detected, normalise to UTF-8 without a BOM before handing it to the engine.
.sub / VobSub and similar bitmap formats are not supported; the encoding healing covers the common combinations.
Numbers and support surface
- Text rasterization costs about 0.06 ms per frame (1080p steady state); the ASS chain's CPU increment on active frames targets under 3 ms, and idle stretches cost nothing;
- Queue limits: 512 blocks per ASS track, 8 frames per PGS track — subtitles never stall playback;
- Supported formats: ASS / SSA / SRT / mov_text / PGS; Blu-ray PGS and UHD Blu-ray included.
Afterword
The value of this chain is not any single feature but that three completely different things converge on one canvas and two lanes — the platform renderer only mixes — and that, after that, every pitfall that only shows up with real sources got filled in. All three fixes above are in the avox engine. The rest of the series: the product introduction, From the open-source engine avox to a product: the cross-platform player Panvox.