Google's AMIE Video AI Matches Board-Certified Doctors in Live Consultations

A physician's job is more than asking questions. They watch how a patient walks in, notice the tremor in their hand, hear the catch in their breath. For years, medical AI has been stuck in a text box, blind to all of that. Google Research just changed that with a new version of AMIE (Video), a real-time clinical video consultation system that sees, hears, and reasons about patients the way a doctor would.

The results are striking: in a randomized controlled study, AMIE (Video) was rated on par with board-certified primary care physicians (PCPs) across every core clinical competency. This is the first time an AI system has demonstrated expert-level performance in live, end-to-end video medical consultations.

The problem text-only AI could never solve

Previous versions of AMIE operated entirely through text chat. That approach has a fundamental ceiling. Patients must translate physical symptoms into words, losing diagnostic signal in the process. A doctor watching someone hold their arm still to avoid pain, or noticing the asymmetry of a facial expression, gets information that simply cannot be typed. Text-only systems also can't guide a patient through a physical exam maneuver, like asking them to look up and to the left, or to press on their abdomen.

AMIE (Video) was built to close that gap. Built on Gemini and Project Astra, it conducts synchronous clinical video consultations, perceiving non-verbal clinical cues, guiding patient actors through virtual physical examinations, and reasoning diagnostically, all in real time.

Three agents, one conversation

The core engineering challenge is a real tension: deep clinical reasoning takes time, but conversational pauses destroy patient trust. A single model can't do both at once. Google's solution is an asynchronous multi-agent architecture that splits the work across three specialized agents running in parallel:

• Talker agent: The patient-facing voice. It keeps the conversation flowing at natural speed, incorporating guidance from the other two agents without making the patient wait.
• Planner agent: Runs in the background, continuously refining the differential diagnosis, updating management plans, and identifying what information is still missing.
• Perception agent: Watches and listens to the video stream continuously, flagging clinically relevant non-verbal signals, like visible signs of respiratory distress or abnormal posture, and feeding them into the ongoing clinical picture.

This decoupled design is what makes low-latency conversation and high-quality reasoning compatible. Automated evaluations confirmed that each agent contributes meaningfully to clinical metrics, including history-taking competency, reasoning quality, and response latency.

How the study was run

To evaluate the system rigorously, Google ran a large-scale Objective Structured Clinical Examination (OSCE) study. An OSCE is the gold-standard format used to assess medical students and residents: trained patient actors portray specific conditions according to standardized scripts, and independent evaluators score the consultation against detailed clinical rubrics.

The multi-arm randomized study covered 100 scenarios, 300 live consultations, and a group of 30 board-certified primary care physicians. The 100 scenarios spanned five body systems: cardiopulmonary, abdominal, HEENT (head, eyes, ears, nose, throat), neurological/psychiatric, and musculoskeletal. Three study arms ran in parallel:

1. AMIE (Video): The full real-time video system.
2. AMIE (Text): A text-only version of AMIE, used as a baseline to isolate the contribution of audio-visual capabilities.
3. PCP (Video): Ten board-certified primary care physicians using the same video interface.

An independent panel of 20 experienced PCPs evaluated all 300 consultations using both general clinical competency scales and scenario-specific rubrics.

What the results actually show

Clinical evaluators rated AMIE (Video) on par or better than PCPs in history-taking, diagnosis, management, and physical observation and examination. The physical exam finding is especially notable: AMIE (Video) was rated significantly higher than both PCPs and AMIE (Text) at eliciting physical signs and guiding patients through virtual examination maneuvers. An AI proactively directing a patient to demonstrate their gait or show their handwriting is genuinely new territory.

Patient actors also had strong opinions. Patient actors preferred AMIE's approach to assessing and explaining conditions, while PCPs were preferred for rapport and partnership building. In the modality comparison, patient actors preferred AMIE (Video)'s interface over text chat for communicative effectiveness, convenience, and feeling understood. The video format itself, not just the AI, was a meaningful upgrade for patients.

Where it still falls short

The team is direct about the limitations. Limitations remain in fine anatomical precision, subtle affective nuances, and high-frequency movements. The system also exhibits intermittent technical issues that can disrupt conversational naturalness, partly inherited from the prototype nature of Project Astra.

More fundamentally, the entire study used professional patient actors in controlled scenarios, not real patients with real conditions. The system was tested in simulated environments with controlled variables, which do not fully reflect the complexity, unpredictability, and emotional nuance of actual healthcare settings. Conditions that can't be convincingly acted out, like acute pain or certain neurological presentations, were excluded entirely. Real-world validation is the essential next step before any clinical deployment.

The bigger picture: what this actually unlocks

AMIE has been on a steady march. It started as a text-based diagnostic reasoner, then gained vision for reading medical images and documents, then expanded to longitudinal disease management. The video capability is the piece that finally makes it look like a telehealth visit rather than a chat window.

The practical implications are significant for healthcare access. Telehealth already serves millions of patients who can't easily reach a physician. An AI system that can conduct a competent video consultation, guide a physical exam remotely, and pick up on non-verbal cues could meaningfully extend that reach, especially for patients with limited health literacy who struggle to describe symptoms in text.

AMIE remains a research system and more research is needed before responsible real-world clinical deployment , but Google has already started building the bridge. A real-world feasibility study with Beth Israel Deaconess Medical Center validated the text-based version in a live clinical setting, and an ongoing nationwide randomized study with Included Health is evaluating AI in real-world virtual care. The video system is the research milestone; those partnerships are where it becomes medicine.

The full paper is available on arXiv, and a demonstration video has been published alongside the research blog. For now, AMIE (Video) is not publicly accessible. It is a research prototype, and the team is explicit that real-patient studies must come before any deployment decision.