Skip to main content
Back

Weill Cornell's VERITAS Turns Raw MRI Data Into Statistically Validated Verdicts

A multi-agent framework turns natural-language clinical questions into fully auditable statistical studies on MRI data, hitting 81.4% verdict accuracy.

Pro

  • VERITAS is a multi-agent system that autonomously tests natural-language hypotheses on medical imaging datasets with full audit trails.
  • Reaches 81.4% verdict accuracy with frontier models and 71.2% with 8-30B open-weight models, beating all single-model baselines.
  • Introduces an Evidence Classification Operator that mechanically labels results as Supported, Refuted, Underpowered, or Invalid.
  • Benchmark of 64 hypotheses spans cardiac MRI (ACDC) and brain glioma MRI (UCSF-PDGM) across six complexity tiers.
  • Produces highest verifiability rate (86.6%), so even failed runs remain diagnosable via inspectable artifacts.
  • Code, hypothesis bank, and evaluation pipeline released at github.com/LucZot/veritas.

Testing a clinical hypothesis from medical images usually requires a small committee: a clinician who knows the disease, a radiologist who can segment the anatomy, a programmer who can extract measurements, and a statistician who picks the right test. Researchers from Weill Cornell Medicine, Cornell Tech, and TU Munich have released VERITAS, a multi-agent system that compresses that pipeline into an autonomous workflow, taking a natural-language hypothesis and a raw MRI dataset and returning a statistically validated verdict along with every intermediate artifact.

The system decomposes the workflow into four phases handled by role-specialized agents, and introduces an epistemic evidence label framework that mechanically classifies outcomes as Supported, Refuted, Underpowered, or Invalid by jointly evaluating significance, effect direction, and study power. That last label matters more than it sounds, and it is the paper's most consequential contribution.

Why non-significant does not mean nothing

In medical imaging, cohorts are frequently tiny. A subgroup of 13 patients with a rare mutation can produce a non-significant p-value not because the effect is absent, but because the study lacks the statistical power to detect it. Binary evaluation schemes reward a system that conservatively labels every null result as "refuted," which inflates accuracy without demonstrating any real epistemic awareness.

VERITAS handles this with an Evidence Classification Operator (ECO), a deterministic function that takes the p-value, effect size, and power at a pre-specified smallest effect size of interest (SESOI) and outputs a label. No agent judgment is involved. A non-significant result with power of 0.38 becomes UNDERPOWERED rather than REFUTED, forcing the system to earn its verdicts through adequately powered analyses.