OpenAI Rebuilds ChatGPT Voice With GPT-Live to Talk and Listen at Once

OpenAI published a detailed engineering post on how they rebuilt ChatGPT Voice to power GPT-Live, their third-generation voice system. The headline feature is full-duplex audio — the model can listen and speak simultaneously — but the deeper story is a complete rearchitecture of every layer between the user's microphone and the model, built over six months to serve 150 million weekly voice users.

Why the old approach broke down

Every previous ChatGPT voice system was turn-based. Advanced Voice Mode processed and generated audio within a single model, which reduced latency and smoothed conversations, but the model still had to wait for the user to stop speaking before it could respond. A small "turn detector" model decided when the user had finished talking. Because that detection relied on silence, a brief pause or background noise could trigger a premature response, cutting the user off at unnatural moments.

Two architectural bets

GPT-Live is built on two structural changes that work together:

• Full-duplex continuous inference: Instead of processing a sequence of separate messages, GPT-Live continuously processes input while generating output. The model makes interaction decisions many times per second: whether to speak, keep listening, pause, interrupt, or invoke a tool.
• Asynchronous delegation: The voice interaction layer is decoupled from the reasoning layer. Straightforward questions are handled directly by GPT-Live. Queries that require web search, deeper reasoning, or agentic work are delegated to a frontier model running in the background — GPT-5.5 at launch — while GPT-Live keeps the conversation going.

That continuous-processing design enables live translation and lets the model acknowledge a speaker mid-sentence with cues like "mhmm," rather than waiting for a full turn to end.

Rebuilding the media pipeline

Keeping audio flowing without gaps required rethinking every layer of the stack. OpenAI made a hard separation between the media path and everything else, then rewrote the media frontend and inference logic in Go, replacing a previous Python asyncio implementation. According to the engineering post, this significantly improved frame delivery smoothness: the new system's p95 latency matches the previous system's p50, meaning worst-case performance now equals what used to be the median. WebRTC handles the transport layer, managing packet loss, clock drift, and connection changes without audible gaps.

Stateful inference introduced its own problem. Voice sessions can run long, but the model's context window fills up. Rather than pausing the conversation to compact the context, OpenAI built a seamless handoff:

1. While the current model instance keeps talking, the system compacts the context in the background.
2. A replacement instance is warmed and prefilled with the compacted context.
3. Inference runs on both instances in parallel, then cuts over when the new one is ready.
4. The user hears nothing — the conversation never pauses.

WARP: one round trip instead of six

Standard WebRTC session startup requires six network round trips before audio can flow, a legacy of protocols designed independently, each with its own handshake overhead. OpenAI analyzed the stack and developed WARP (WebRTC Abridged Roundtrip Protocol), which reduces that to a single round trip by bundling several backward-compatible improvements: piggybacking the DTLS security handshake over the ICE connectivity check, using the faster DTLS 1.3 handshake, and pre-negotiating the data channel setup.

On top of WARP, OpenAI built "Instant Connect," which pre-negotiates session parameters before the user taps the voice button, so the server can materialize a session the moment the first audio packet arrives.

WARP is designed as an open specification. OpenAI worked with contributors from the WebRTC community and is advancing the proposals through the IETF's TSVWG working group. Support has already been added to both libwebrtc and Pion. Any real-time audio application could benefit, not just OpenAI's products. The WARP draft is publicly available at the IETF.

Making delegation fast enough to be invisible

Delegating to GPT-5.5 only works if the result returns before the voice model runs out of things to say. OpenAI treated the entire delegation loop — routing, prompt processing, inference, and tool calls — as part of the responsiveness budget. Key optimizations:

• Pre-warming: When a voice session starts, the application server creates an inference session for the frontier model and prefills it with the initial conversation context before any delegation is requested.
• Session affinity: Successive delegation requests go to the same inference session, combined with prompt caching, to avoid reprocessing the same context.
• Reasoning tiers: GPT-Live routes responses through three levels — Instant, Medium, and High — trading latency for depth depending on the query.

Bridging continuous audio and discrete messages

The rest of ChatGPT — the conversation UI, analytics, safety infrastructure — still expects discrete user and assistant turns. GPT-Live produces a continuous, overlapping audio stream where both speakers can talk at once. The application server maintains two views of the conversation simultaneously: a speculative view that updates in near-real-time for the UI, and an authoritative record for logging and safety pipelines. Brief backchannels like "mm hmm" from the assistant don't become their own messages, but substantive interjections do. The system commits to a final transcript only when speaker attribution is reliable.

Where it performs best

In head-to-head comparisons, GPT-Live-1 and GPT-Live-1 mini are strongly preferred over Advanced Voice Mode in matched 5–10 minute conversations measuring overall preference, turn-taking, interruptions, conversational flow, and naturalness. The model also shows strong gains on GPQA (expert-level scientific reasoning) and BrowseComp (agentic web search), reflecting the benefit of delegating to GPT-5.5 in the background.

The use cases where turn-based voice was most painful are where GPT-Live gains the most:

• Language practice and tutoring, where natural back-and-forth matters
• Hands-free workflows during commutes or physical tasks
• Live translation within a continuous conversation
• Agentic tasks like computer control and agent coordination in the ChatGPT desktop app
• Exploring ideas and thinking through complex problems in real time

At launch, GPT-Live does not support voice with video or screen sharing, and the API is not yet publicly available. Developers can sign up for API access notification on OpenAI's site.

Availability and pricing

GPT-Live-1 mini is included on the Free plan ($0). The full GPT-Live-1 is available on Go ($8/mo), Plus ($20/mo), and Pro (from $100/mo). There is no per-message voice charge or credit pack for consumers. API pricing for GPT-Live-1 and mini is listed as coming soon, with no published rates.

What shifts for the field

The architectural pattern OpenAI landed on — a lightweight, always-on voice model that delegates to a heavier reasoning model asynchronously — breaks from the assumption that one model should handle everything. Speed and depth pull against each other computationally, and OpenAI's answer is to decouple them rather than optimize a single model for both.

For teams building voice agents today, the bottleneck has shifted. Understanding speech is a solved problem. The hard parts are now session startup latency, context management across long calls, and running heavy reasoning without stalling the audio stream. GPT-Live's engineering post is a rare detailed look at what solving those problems actually requires.