OpenAI announced three audio models for the Realtime API in 2026: GPT-Realtime-2 for conversational reasoning, GPT-Realtime-Translate for live speech translation (70+ input languages to 13 output languages), and GPT-Realtime-Whisper for streaming transcription. We focused on Translate + Whisper — and the pairing reveals something important about usable live translation.
Whisper as a text floor, Translate as voice output
Whisper transcribes continuously while Translate produces the translated voice. Translate does not rush phrase-by-phrase — it often waits for a meaningful syntactic chunk (sometimes until the next verb or semantic unit) before opening a translated sentence. That avoids the classic dubbing effect where translation starts too early and then self-corrects.
Before vs after
- Before: voice translation felt packetized — visible latency, overlaps, translation starting before intent was complete
- After: streaming Whisper + Translate creates a more human sync — you read what you said while hearing the translation, which helps adjust pacing naturally
Our test protocol
Using an internal WebRTC harness, we ran five minutes of natural speech with hesitations and pauses, then accelerated delivery, then long explanatory sentences. Translate performed best on oral alignment — less machine-subtitle racing ahead of the speaker. Instability remains: cuts and erratic behavior under background noise or rapid chaining. TTS voice is still somewhat synthetic; premium livestream use may need voice cloning on top.
Production angles
For livestream, international client calls, or on-set multilingual workflows, this API class is finally usable in situ — airport, street, improvised meeting — because it respects tempo, not just word fidelity. An OBS plugin for real-time interpretation is an obvious next step. Not perfect today, but the foundation for live video production tooling is real.