Google's AMIE Video AI Matches Board-Certified Doctors in Live Consultations
Google's AMIE now conducts live video consultations using a three-agent architecture, matching primary care physicians across all core clinical metrics in a 300-consultation randomized trial.
- AMIE (Video) is Google's new real-time video medical consultation AI, built on Gemini and Project Astra, matching board-certified PCPs across all core clinical metrics.
- A 300-consultation randomized OSCE study across 100 clinical scenarios and 30 PCPs is the first to demonstrate expert-level AI performance in live video medical consultations.
- A three-agent architecture (Talker, Planner, Perception) runs in parallel to maintain natural conversation speed while doing deep clinical reasoning and real-time audio-visual analysis.
- AMIE (Video) outperformed PCPs at guiding virtual physical examinations and eliciting physical signs; PCPs were still preferred for rapport and partnership building.
- Key limitations: tested only with patient actors in controlled scenarios; struggles with fine anatomical precision, subtle affect, and high-frequency movements; not yet validated with real patients.
- AMIE remains a research prototype with no public access; real-world validation studies are underway with Beth Israel Deaconess Medical Center and Included Health.
A physician's job is more than asking questions. They watch how a patient walks in, notice the tremor in their hand, hear the catch in their breath. For years, medical AI has been stuck in a text box, blind to all of that. Google Research just changed that with a new version of AMIE (Video), a real-time clinical video consultation system that sees, hears, and reasons about patients the way a doctor would.
The results are striking: in a randomized controlled study, AMIE (Video) was rated on par with board-certified primary care physicians (PCPs) across every core clinical competency. This is the first time an AI system has demonstrated expert-level performance in live, end-to-end video medical consultations.
The problem text-only AI could never solve
Previous versions of AMIE operated entirely through text chat. That approach has a fundamental ceiling. Patients must translate physical symptoms into words, losing diagnostic signal in the process. A doctor watching someone hold their arm still to avoid pain, or noticing the asymmetry of a facial expression, gets information that simply cannot be typed. Text-only systems also can't guide a patient through a physical exam maneuver, like asking them to look up and to the left, or to press on their abdomen.
AMIE (Video) was built to close that gap. Built on Gemini and Project Astra, it conducts synchronous clinical video consultations, perceiving non-verbal clinical cues, guiding patient actors through virtual physical examinations, and reasoning diagnostically, all in real time.
Three agents, one conversation
The core engineering challenge is a real tension: deep clinical reasoning takes time, but conversational pauses destroy patient trust. A single model can't do both at once. Google's solution is an asynchronous multi-agent architecture that splits the work across three specialized agents running in parallel: