Mistral's Voxtral 3 Bundles Speech and Reasoning Into One Streaming Agent

Mistral has released Voxtral 3, a family of speech-to-text and text-to-speech models paired with a voice agent SDK. The SDK connects listening, reasoning, tool calling, and speech generation in one streaming pipeline. Available through Mistral's EU-hosted API or for self-deployment on your own infrastructure, it gives organizations control over where audio is processed.

Building a voice agent usually requires separate services for transcription, turn detection, reasoning, tools, and speech synthesis. Each handoff can add latency, discard context, or mishandle interruptions. Voxtral 3 packages those components around a common API and SDK while keeping the transcription, language, and speech models independently configurable.

Four models cover the voice loop

Transcription tuned for names and turns

Voxtral Small Transcribe 3 supports 50 languages, including all 24 official EU languages and several lower-resource Asian and Indic languages. On the 45-language FLEURS benchmark, Mistral reports a 4.0% word error rate in forced-language mode, where the system receives the spoken language in advance. Word error rate measures incorrect, missing, or inserted words, so lower scores indicate more accurate transcription.

Mistral’s published comparison places Voxtral Small ahead of Microsoft AI Transcribe 2, Gemini 3.5 Transcribe, Scribe v2, GPT Transcribe, and Whisper large-v3, which recorded 14.9% on the same test. The company also reports leading results for multilingual transcription and diarization, the process of identifying which speaker said each segment. These figures are vendor-reported and should be validated against recordings from the intended workload.

Context biasing lets developers supply product names, technical terms, acronyms, and other relevant text at request time. This steers recognition toward domain-specific vocabulary without fine-tuning the model. On the Earnings 22 keyphrase recall test, Mistral reports that Voxtral Small improves from 81.4% without context to 94.6% with it.

Native end-of-turn prediction combines acoustic cues with the meaning of the transcript to estimate when a speaker has finished. That approach can handle pauses more accurately than a fixed silence timer, which may interrupt a thoughtful speaker or leave an agent waiting after a short answer.

Speech with controllable delivery

Voxtral’s speech model supports more than 35 languages and 50 dialects. It can infer emotion, pace, and emphasis from the surrounding conversation, or follow inline instructions supplied by the developer:

[emotion=sad][pace=slow] I'm sorry to hear that.
[emotion=excited][emphasis=high] Your flight is confirmed!

Pro Voice adaptation uses 30 minutes of reference audio to reproduce a speaker’s vocal character, emotional range, and delivery across languages. Mistral says its dialect evaluations cover pronunciation, formal and informal registers, spelling, verbosity, and local conventions for dates, times, temperatures, addresses, and currencies.

Voice adaptation requires operational safeguards because it reproduces identifiable vocal traits. Teams evaluating Pro Voice should confirm speaker-consent requirements, access controls, audit logging, disclosure policies, and the storage and deletion rules applied to reference recordings.

The whole turn stays in motion

Built on LiveKit, the voice agent SDK connects three independently configurable stages:

1. Listen: Voxtral Mini Transcribe Realtime 3 streams the transcript and detects the end of the caller’s turn.
2. Reason and act: An LLM such as Mistral Small interprets the request and invokes configured tools.
3. Reply: Voxtral Speech 3 streams a spoken response with delivery informed by the conversation.

Partial outputs move across the pipeline as they become available, reducing idle time between stages. Mistral reports turn latency below one second and supports barge-in handling, which stops the agent’s speech when the caller interrupts. Actual latency will depend on network conditions, model selection, tool execution, deployment region, and telephony infrastructure.

Configured tools can verify a caller’s identity, retrieve account data, check a bill, or book an appointment through function calling, web search, and retrieval over internal systems. The SDK supports SIP for phone calls and WebRTC for browsers and applications, providing integration paths for telephony, interactive voice response, and contact-center platforms.

Swap the pieces, choose the region

Keeping transcription, reasoning, and synthesis as separate components lets developers change the LLM, voice, or supported language without rebuilding the entire agent. The same structure also supports independent testing and scaling for each stage, which can help teams isolate transcription errors, slow tools, and weak responses.

Mistral offers EU, US, and global API endpoints alongside self-deployment. Those choices give teams more control over where audio is processed and how the service connects to private data. Residency, retention, security, and regulatory compliance still depend on the selected configuration, contract, logging policy, and surrounding infrastructure.

Workloads that match the design

• Contact centers and service desks that need multilingual transcription, speaker diarization, and vocabulary biasing for product names, stock-keeping units, or medical terms.
• Phone assistants that require SIP or WebRTC connectivity, interruption handling, and low-latency turns.
• Branded voice products that need a consistent speaker identity across languages and dialects.
• Residency-sensitive workloads that require EU-hosted inference or deployment on privately controlled infrastructure.

What to test before launch

• Vocal information loss: A text intermediate can discard laughter, sighs, hesitation, and other cues unless the transcription layer represents them explicitly.
• Composite quality: The agent’s reasoning and tool use depend on the selected LLM, prompts, retrieval system, and function definitions.
• End-to-end latency: Reported model latency does not include every network hop, tool call, database query, or telephony delay.
• Benchmark fit: Forced-language FLEURS results may not predict performance on noisy calls, code-switching, specialized terminology, or unfamiliar accents.
• Self-hosting costs: Private deployment adds hardware, scaling, monitoring, updates, and incident-response work.

A production evaluation should measure word error rate on real calls, keyphrase recall, diarization accuracy, end-of-turn errors, interruption recovery, tool-call completion, tail latency, and synthesized-voice quality for each target locale. The Voxtral docs provide API and deployment details.