CMU and Idiap Build Synth-DoPaCo, a 1,329-Hour Medical AI Benchmark
A new open dataset of 8,800 synthetic doctor-patient conversations with 1,329 hours of audio tests whether audio language models can summarize long clinical visits.
- New open dataset Synth-DoPaCo releases 8,800 synthetic doctor-patient conversations with 1,329 hours of audio.
- Three-stage pipeline: persona-driven dialogue, multi-speaker TTS with room acoustics, and LLM-generated SOAP note references.
- Built entirely on open-weight models including Gemma3-27B, Qwen3-TTS-1.7B, and Kimi K2 Thinking.
- Cascaded ASR-plus-LLM systems substantially outperform end-to-end audio models like Qwen3-Omni on faithfulness.
- UTMOS perceptual score of 1.27 is nearly identical to real mock clinical recordings at 1.28.
- Train and dev splits release around Interspeech 2026; test set withheld to prevent contamination.
Synth-DoPaCo builds a 1,329-hour testbed for clinical audio models
Long clinical visits expose a gap in public audio-language-model research: most benchmarks use clips under five minutes, while privacy, consent, and deidentification requirements restrict access to recorded medical encounters. Synth-DoPaCo addresses that gap with 8,800 synthetic doctor-patient visits spanning 1,329 hours. Each encounter includes dialogue, speech, simulated room acoustics, non-speech sounds, structured facts, and a reference clinical note.
Why public benchmarks run short
The paper, written by researchers from Idiap, Carnegie Mellon, Johns Hopkins, and other institutions, targets long-context audio reasoning. Audio language models can consume speech and generate text, but their performance on conversations lasting more than five minutes remains poorly measured. Clinical documentation provides a demanding test because a model must track symptoms, history, medications, examination details, and treatment plans across many turns.
| Resource | Encounters | Audio | Main constraint |
|---|---|---|---|
| PriMock57 | 57 | Yes | Brief, actor-performed consultations |
| ACI-Bench | 187 | No | Text-only encounters |
Limited public data has left reproducible medical-scribing research with far less training material than proprietary systems can obtain from private clinical recordings. A large synthetic corpus gives outside teams a common dataset for training and comparison without exposing patient conversations.
Personas, room noise, and cited facts
The authors built the corpus with open-weight model components in three stages: persona-driven dialogue generation, multi-speaker audio synthesis, and reference-note production. Gemma3-27B-IT generates the conversations, while SDialog manages turn-taking and pipeline orchestration.
Personas prevent dialogue collapse
Direct dialogue prompts produced little variation, so the pipeline samples structured attributes for each speaker, including age, race, formality, hurriedness, insurance, occupation, and English fluency. A clinically trained co-author compiled 724 chief complaints covering common and atypical primary-care presentations. The model generates one turn at a time, an approach the authors report improved persona adherence and produced more challenging conversational dynamics.
This story is for Pro members
You've reached the end of the free preview. Upgrade to AlphaSignal Pro to read the full article - and everything else behind the paywall.
Comments
