---
title: "Interactive TTS Teaches Voice AI to Whisper, Shout, and Grieve on Cue"
subtitle: "A new research paper proposes decoupling style decisions from waveform generation, using explicit instructions and preference optimization to keep voices consistent across turns."
url: "https://alphasignal.ai/news/interactive-tts-teaches-voice-ai-to-whisper-shout-and-grieve-on-cue"
author: "AlphaSignal Newsroom"
author_url: "https://alphasignal.ai/editorial-team"
company: "AlphaSignal"
company_url: "https://alphasignal.ai/company/alphasignal"
published: "2026-09-22T05:25:02.000Z"
updated: "2026-09-26T02:00:14.796Z"
type: "Paper"
topics:
  - "Audio"
  - "Post Training"
subtopics:
  - "Text To Speech"
  - "Dpo"
  - "Fine Tuning"
upvotes: 12
source_url: "https://arxiv.org/abs/2609.25707"
comments: 0
truncated: true
---

# Interactive TTS Teaches Voice AI to Whisper, Shout, and Grieve on Cue

> A new research paper proposes decoupling style decisions from waveform generation, using explicit instructions and preference optimization to keep voices consistent across turns.

# Interactive TTS gives voice models a written delivery plan

Wenjie Tian and collaborators propose Interactive TTS, a framework that converts conversational context into a written speaking-style instruction before synthesizing audio. The design aims to vary pace, volume, and emotion while preserving the speaker’s identity and following user cues. The [research paper](https://arxiv.org/abs/2609.25707) reports stronger results than previous leading systems on two expressive-speech benchmarks.

The team’s [project demos](https://wjtian-wonderful.github.io/InteractiveTTS/) present Chinese-language responses to situations including a kitchen fire, a sleeping baby in the next room, and a user grieving a pet. These scenarios test whether one reference voice can adapt its delivery to conversational and visual context.

## Why a voice drifts with the dialogue

Context-aware text-to-speech systems commonly feed dialogue history into an end-to-end model and leave it to infer whether the response should be whispered, shouted, slowed down, or softened. Style, linguistic content, and timbre, the acoustic qualities that identify a speaker, can become entangled inside the model. That makes style decisions difficult to supervise and can cause inconsistent voices or missed instructions across turns.

Interactive TTS assigns style selection and audio rendering to separate components. One module determines how the response should sound, while another generates the waveform from the reply text, style instruction, and voice reference.

## A planner sits in front of the synthesizer

__Interactive TTS components__
| Component                      | Inputs                                                                        | Output                    | Purpose                                                  |
| ------------------------------ | ----------------------------------------------------------------------------- | ------------------------- | -------------------------------------------------------- |
| Context-to-Instruction planner | Dialogue history and available multimodal cues, including video in some demos | A short style instruction | Select an appropriate delivery for the current situation |
| TTS generator                  | Reply text, style instruction, and fixed voice reference                      |                           |                                                          |

_Preview only. Full text requires an AlphaSignal Pro account._

## Sources

- [Interactive TTS: Dynamic Speaking Style Adaptation for Expressive Speech Synthesis (arXiv)](https://arxiv.org/abs/2609.25707)
- [Interactive TTS Demo Page](https://wjtian-wonderful.github.io/InteractiveTTS/)
- [arXiv PDF](https://arxiv.org/pdf/2609.25707)