Skip to main content
Back

OpenAI Releases Speech Model APIs: Real-Time AI Voices & Transcription

TRY NOW

March 24, 2025 5 min read

Hey,

Welcome to AlphaSignal – the most read newsletter by AI developers. 


We bring you the top 1% of news, papers, models, and repos, all summarized to keep you updated on the latest in AI.

IN TODAY'S SIGNAL

Read time: 5 min 08 sec

🎖️ Top News

⚡️ Trending Signals

📌  Assembly AI


💻  Top Tutorials

🧠  How To

If you're enjoying AlphaSignal please forward this email to a colleague. 

It helps us keep this content free.

TOP NEWS

Speech AI

OpenAI releases three audio model APIs and updates Agents SDK to support voice AI for building agents

⇧ 10,358 Likes

What's New

OpenAI has launched two new speech-to-text and one text-to-speech models, improving accuracy and customization. The models outperform Whisper and Gemini 2.0 Flash, especially in English transcription accuracy. You can now instruct AI-generated voices with text commands, modifying tone and style.


Speech-to-Text Models

OpenAI introduced GPT-4o Transcribe and GPT-4o Mini Transcribe, improving transcription speed and reliability.

  • Models handle accents, noisy environments, and variable speech speeds better than previous versions.

  • Voice activity detection and voice cancellation improve transcription in multi-speaker conversations.

  • These models process speech faster than Whisper but provides no specific latency benchmarks.

Text-to-Speech Model

The GPT-4o Mini TTS model lets you modify AI-generated speech characteristics using text input.

  • Developers can prompt different tones, such as “speak like a pirate” or “bedtime story voice.”

  • The model does not require fine-tuning to adjust speaking style.


Integration and Use Cases

The updated Agents SDK now supports voice-based AI interactions.

  • You can build real-time or batch-processing voice agents.

  • Use cases include customer service automation, meeting transcription, and AI assistants.

  • Streaming support details remain undisclosed.

  • OpenAI released OpenAI.fm for testing different voice styles.

Community Feedback

cedric
"Voice agents are cool! For the first time, developers can instruct the model on both what to say and how to say it. However, these text-to-speech models are still limited to preset artificial voices. OpenAI is going after ElevenLabs's TTS and AI voice generation?"


Boris Zubarev
"As an LLM specialist who's built 15+ projects in this space, the instructable TTS is a game-changer for agent development. The ability to modulate tone/emphasis opens new doors for emotional intelligence in voice agents. Perfect timing for those building conversational systems with nuanced delivery."

Igor

"It’s great news but can’t help noticing that voices are still somewhat robotic sounding. If you compare to Suno, difference is obvious. Not sure it’s practical to compare to Suno though."

Try now

TRENDING SIGNALS

AI Tools

Anthropic rolls out web search to Claude 3.7 Sonnet for up-to-date data, improving accuracy on time-sensitive queries

⇧ 10,057 Likes

Image Generation

xAI introduces image generation to its API; model refines prompts via chat before producing images in API responses

⇧ 10,148 Likes

AI Model

Tencent unveils Hunyuan-T1, a reasoning model that matches DeepSeek R1 in performance and pricing, excelling in math

⇧ 3,593 Likes

Audio Models

NVIDIA presents Canary-1B-Flash, a faster, more accurate open-source ASR model which ranks 2nd on HF Open ASR Leaderboard

⇧ 2,485 Likes

AI Evaluation

Hugging Face updates its analytics dashboard with real-time metrics for AI inference endpoint monitoring and debugging

⇧ 739 Like

Transcribe and Understand Speech with Unmatched Accuracy

AssemblyAI just released new updates for transcription, speaker diarization, and speech understanding. Reduce transcript errors, extract insights, and improve downstream processes.

Scale with enterprise-grade Speech AI built for performance and reliability.

Try it now ↗️

TOP TUTORIALS

Agents

Building AI Agents with MongoDB Atlas: 18 Hands-On Notebooks

⇧ 3,023 Likes

Learn how to build AI agents using MongoDB Atlas as a vector store and memory provider. Explore 18 Jupyter Notebooks covering frameworks like LangChain, LlamaIndex, Haystack, SmoLAgents, and Gemini 2.0. Implement retrieval pipelines, AI workflows, and model evaluations with OpenAI, Claude, AWS Bedrock, and more. Run production-ready Python and JavaScript applications.

MCP

Automating Cursor Workflows with MCPs

⇧ 2,538 Likes

Learn how to use MCPs to automate workflows in Cursor. Install common MCPs, read browser console and network logs, and generate UI assets with Replicate. Access Supabase and Figma data. Build a custom MCP using Cloudflare’s MCP-worker.

MCP

Creating a UI with Figma to Cursor MCP Server

⇧ 839 Likes

Use a Figma MCP server to implement designs directly in Cursor. Import auto-layout structures, apply Google Fonts, and refine sections with targeted updates. Copy Figma node links, process designs in agent mode, and debug layout issues. Improve UI generation accuracy while maintaining manual control when needed. Streamline design-to-code workflows efficiently.

HOW TO

Agents

Build Voice Agents Using OpenAI API and Agents SDK

Learn how to create voice agents that process audio and respond in natural language. Use OpenAI’s API and Agents SDK to build context-aware voice agents for customer support, language tutoring, and more.


The guide explains how to:

  • Implement speech-to-speech (multimodal) or chained (speech-to-text → LLM → text-to-speech) architectures.

  • Use the gpt-4o-realtime-preview model for low-latency, interactive voice agents.

  • Chain gpt-4o-transcribe → gpt-4o → gpt-4o-mini-tts for structured workflows.

For real-time speech-to-speech processing, review the Realtime API guide and Realtime API reference.

Start building now

At Alpha Signal, our mission is to build a sharp, engaged community focused on AI, machine learning, and cutting-edge language models, helping over 200,000 developers stay informed and ahead. We’re passionate about curating the best in AI, from top research and trending technical blogs to expert insights and tailored job opportunities. We keep you connected to the breakthroughs and discussions that matter, so you can stay in the loop without endless searching. We also work closely with partners who value the future of AI, including employers and advertisers who want to reach an audience as passionate about AI as we are.


Our partnerships are based on shared values of ethics, responsibility, and a commitment to building a better world through technology.Privacy is a priority at Alpha Signal. Our Privacy Policy clearly explains how we collect, store, and use your personal and non-personal information. By using our website, you accept these terms, which you can review on our website. This policy applies across all Alpha Signal pages, outlining your rights and how to contact us if you want to adjust the use of your information. We’re based in the United States. By using our site, you agree to be governed by U.S. laws.

Looking to promote your company, product, service, or event to 200,000+ AI developers? Let's work together.

Work with us