Four adapters, one on each side
The interfaces the project already has are fixed: sources have the capture-callback family, encoders have input, release and encoded-packet callbacks. What the WebRTC side wants is also fixed: a track source, an audio source interface, and video and audio encoder interfaces. The two sides do not line up, so each gets its own adapter:
Video source and audio source bridge to its track source and audio source interfaces; video encoder and audio encoder bridge to its encoder interfaces. Above that, a single entry point brings signaling, the two data streams and rendering together, and inside WebRTC everything running is the project's own sources and codecs.
Video source: capture once, take one copy on each side
The video source adapter carries a broadcaster inside: every frame received goes to all consumers at the same time — one copy for the local preview window, one for the encoder. Data is captured once, preview and encoding each take a copy, which saves both CPU and GPU.
Resources follow the consumers: when the first consumer registers, the underlying capture source is opened and the local preview window starts; when the last one leaves, both close automatically. The image processing attached is the existing set — the local preview and the picture sent to the encoder go through the same render pass, with crop and scale, watermark and filters all inside.
The output form is decided by what the encoder eats. Hardware encoders that can take GPU textures directly, such as Apple's VideoToolbox, get the texture as is, without landing on the CPU (a reference count is added to keep the texture from being released early); those that want CPU data get RGBA converted to the needed YUV format with a compute shader — the same path used for rendering output in ordinary playback. Windows needs a separate note: its hardware encoders currently have no input that takes D3D11 textures directly, so the frame first lands on the CPU as NV12 and is then handed to the matching hardware encoder per graphics card — and when falling back to software encoding, it is converted to yuv420p. Which one it lands on is decided by an output-form switch on the source side, set before starting the encoder, and the chain follows it. This is the same conclusion as the hardware encoding on the ordinary playback side: the encoder is hardware, but the direct-texture-feed path is not wired up yet.
Audio source: normalization and slicing
The audio source adapter wears three hats: WebRTC's audio source interface, the player side's capture callback, and the reshaper that normalizes and slices. The normalization rules are the same as in the audio pre-processing article — 44100 is rewritten to 48000, converted to S16, channels folded down to two — then sliced into 10 ms segments and fed to WebRTC frame by frame by bit depth, sample rate and channel count. The consumer on/off logic is the same as the video source: open when someone is there, close when nobody is.
Video encoder: one conversion on the way in, one on the way out
The encoder adapter implements WebRTC's encoder interface and wraps the project's own encoder, covering H264 and H265 hardware encoding on each platform. When it is created it takes a default name from the registry based on platform and hardware-encoder configuration — Apple gets VideoToolbox's H264, and the other platforms each have their own default; when the encoder starts, it hands down the initial bitrate, which is set into the underlying encoder, and later adjustments all go through the bitrate callback. If no default name is found it reports an error and does not degrade at runtime — the fallback machinery of the candidate chain hangs off the record-and-stream leg, and that path is not wired here. The audio encoder side is isomorphic to video: the declared format list matches the decode side, and when AAC is negotiated it uses the project's own encoder.
Frames coming in are converted once: WebRTC's frame is carried by a dual-mode buffer — CPU mode is a YUV frame, GPU mode carries a texture; a native frame goes straight into the hardware encoder's texture path. On the way out it is wrapped once more: in the encoded-packet callback the project's packet is wrapped as its encoded frame — the RTP timestamp comes from the correspondence recorded when the frame came in, the capture time uses the packet's own pts, the frame type is judged from the NAL units to see whether it is a keyframe, and the encoding type is marked in the header. Apple's hardware encoder emits the length-header layout; after being judged once here it is converted uniformly to start codes, and the RTP packetization afterwards goes smoothly.
The bitrate is WebRTC's call: it computes a target bitrate from network conditions, the callback takes it out and sets it into the underlying encoder. The encoder just complies and does not run a rate-control scheme of its own. Keyframes are also its request: when the network jitters or a new endpoint joins, it sends a "next frame must be a keyframe" request, and the adapter passes it to the encoder as is — the picture recovering quickly after packet loss depends on this.
The top layer puts the two sides together
The unified entry implements the player interface and is at the same time the command executor and event observer — it plays those roles from the framework in the previous chapter, only with the sources and rendering inside swapped for the WebRTC set: signaling and data streams, local picture, remote picture, remote audio all integrated. There are two roles: the offerer actively calls CreateOffer, suited to streaming that exchanges SDP over an HTTP interface; the answerer passively waits for an offer and then answers, suited to calls with a persistent connection channel. Choosing the role is one call, and it adjusts the order in which local and remote descriptions are set: the offerer sends the offer and sets it locally first, then sets the remote once the other side's answer arrives; the answerer does the reverse — sets the received offer as remote first, then answers and sets it locally — get the order wrong and negotiation dies halfway.
Rendering is all reuse: local preview, remote picture and remote audio all use the project's existing renderers, with the same Vulkan image processing and hardware rendering underneath. To use it, set the data source and call open and it is up; in the bundled example, audio comes from the Windows sound card device, video from the screen capture device, and the address uses a WebRTC interface with the streaming flag — a dozen lines and one streaming session runs.
This chapter is complete here: how the library is wired into the project, echo and noise in recording, the pull side's own decoding and rendering, the push side's own capture and hardware encoding — each has its own article; WebRTC is only responsible for the network segment, getting the data across on a weak network.
The avox engine is open source on GitHub: github.com/xxxzhou/avox, with the four adapters and the unified entry inside.