Before the fork: one RGBA conversion, shared by both paths
The decoder gives back a planar texture — NV12 for 8-bit sources, P010 for 10-bit. The window accepts RGBA, and the Vulkan side's interface is RGBA too, so this texture has to be converted once first.
The conversion happens in the platform pixel-processing layer, and on Windows it is a compute shader: planar texture in, RGBA8 out, and the product does not return to memory but becomes a shared texture created here. That shared texture is also the GPU context handed outward; both consumers take from it, in different ways.
The fork comes after it: an already-converted RGBA shared texture goes to Vulkan, or to the window.
Destination one: import into the Vulkan pipeline
With the switch on, the window rendering layer hands this shared texture to the VK path. What the VK input layer receives is an NT handle.
The handle is prepared when the shared texture is created: the creation flags include shared and NT handle, and the handle is exported after creation. On the VK side, the two entry points for obtaining the handle are loaded first, and a set of external-memory device extensions enabled; then the image is created, declaring at creation that it is an external image and the handle type is a D3D11 texture; then the memory's properties are queried, the memory imported by handle, and bound to the image. At this point the same video memory has a view in each of the two APIs.
The VK pipeline also has a D3D11 device of its own, used to open this handle into a texture and view on its own device; every frame the input layer copies the imported image into the pipeline's own image, and what happens after that in processing and presentation is the ground covered in the rendering layers article.
This step is not done unconditionally. The platform pixel layer's context is handed over only when the current frame really is a hardware-decoded frame (buffer type is not cpu). For a software frame VK uploads the planes itself and does not need it; and when a track is reused across media, that context may still hold the old texture left from the previous hardware-decoded session, and feeding it in unconditionally turns the new session's input slot into the old size and old format, the pipeline negotiation fails, and the whole session goes black. This was actually hit when switching from hardware to software decoding.
Waiting across APIs: the reader waits on the CPU
A texture being accessible does not mean it is readable. Writes on the GPU are asynchronous; after producing the image, the writing side (the compute shader step) signals the shared fence, and the signal value increases monotonically.
The reading side is the VK input layer, which waits for this after importing. It waits for the value the writer last submitted: if it has already been reached, it proceeds without introducing a stall; only if not does it actually wait, and that wait happens on the CPU — the fence is attached to an event, then it waits on the event with a timeout as a backstop, so a device fault does not hang the render thread.
Of the two destinations in this article, only this one waits.
Destination two: skip Vulkan and present through the window's chain
With the switch off, the window is a different class: it creates its own D3D11 device and swapchain, draws the shared texture as a fullscreen quad onto the back buffer, and submits for presentation.
It obtains the texture the same way as the VK side: first set the shared texture's interop device to its own, then open the handle on its own device to get a local texture and its view. After that each frame's actions are short, in the order shown in the figure; the viewport is computed from the picture's real aspect ratio, filling the window only in fullscreen. The presentation call carries vertical sync.
This path is kept to remove one more thing that can fail: the VK path has to create a surface, build a swapchain and negotiate formats, and the whole chain has to be rebuilt whenever the window size changes; the window path has only three steps — fetch the texture, draw the quad, present.
It does not wait on the writer's fence. The rhythm comes from vertical sync at submit time and swapchain rotation; there is no wait before fetching the texture.
A window size change only touches the swapchain: replace the back buffers and that is it, the device is not rebuilt. Rebuilding the device would split it from the one the decode implementation reuses (see the hardware decode article).
There is one more check before the first frame: when the shared texture has not arrived yet, the window returns early for this frame instead of presenting a blank picture.
Software frames take this path too. The window path accepts both kinds: a hardware frame goes through the platform pixel layer, which produces the shared texture; a software frame is consumed by the platform renderer on the spot, and the converted texture is sent down to the window. Without the second half, after switching to software decoding the window would keep returning early because it has no shared texture in hand, and the screen would stay black.
Switching media and resolution: where leftover patches come from
When two images have different sizes, a whole-image copy on the GPU does not error; it covers only the top-left block. Copy directly when switching media or resolution and the screen shows the new picture squeezed into the top-left corner, with the previous frame left everywhere else.
The VK output side handles this in two ways: copy directly only when the sizes match; otherwise take the scaling path and spread the picture over the target. The corresponding action on the input side comes earlier: read the external texture's size every frame, compare it with the previous one, and rebuild the whole image when it changed — the old code did not re-check, so the new session came into an input slot of the old size and the picture was wrong from the first frame. The window path uses the back-buffer rebuild from the previous section.
Interop capability is probed once up front too
The formats on the shared path are a small, finite set: the compute shader's input side accepts NV12, P010 and YUV422P10; across the API boundary the interface is always RGBA8, with RGBA10 only in the tier related to HDR passthrough. The interop capability itself has to be probed: at startup an RGBA8 probe texture is used to test once whether “import by D3D11 texture handle type” works, and if not this import is skipped and it falls back to the software frame path.
Probing at startup rather than waiting for some frame makes format negotiation failure happen at a moment when it can be seen, instead of a video suddenly going black in the middle of playback.
At this point both paths work: one sends hardware frames into Vulkan's unified processing pipeline, the other bypasses VK and draws straight to the window. They share one RGBA conversion and the shared texture; what differs is who does the presentation step.
On Android and Apple, these two paths swap in a different set of GPU memory objects — OES textures and AHardwareBuffer on one side, CVPixelBuffer and IOSurface on the other, each written up in the next two chapters.
The avox engine is open source on GitHub: github.com/xxxzhou/avox, and the shared texture, that fence and the window path are all in there.