Skip to main content
Back

University of Utah's MedConcept Cracks Open Medical AI's Hidden Clinical Reasoning

A new framework uses sparse autoencoders to extract clinically named concepts from frozen medical vision-language models and grades them against radiology reports.

Pro

Paper, Full Access
  • MedConcept extracts named clinical concepts from frozen medical VLMs using sparse autoencoders trained without labels or reports.
  • A candidate vocabulary is built from UMLS terminology and expanded with ChatGPT to cover clinical phrasing.
  • Each SAE neuron is named by cosine similarity between its decoder vector and Merlin text embeddings of candidate terms.
  • MedGemma serves as a frozen judge, scoring concepts as Aligned, Unaligned, or Uncertain versus paired radiology reports.
  • On MerlinPlus, top-25 Aligned scores reach 0.30 to 0.35 and remain stable across LLM decoding temperatures from 0.0 to 0.5.
  • Code, prompts, and data are promised on acceptance, and reports never touch training.

Medical vision-language models can spot tumors, segment organs, and predict diagnoses with impressive accuracy, yet they remain frustratingly opaque about how they reach those conclusions. Researchers from the University of Utah have proposed MedConcept, a framework that peels open pretrained medical VLMs to expose the clinical concepts hiding inside their latent representations, then grades those concepts against radiology reports using a second, independent medical language model as judge.

The trust gap in clinical AI

Medical VLMs achieve strong performance on tasks like tumor segmentation and diagnosis prediction, but their opaque internal representations limit clinical trust and the ability to explain predictions. Existing tools for peering inside these models are thin. Gradient- and attention-based visualizations tend to be tied to specific tasks such as classification, producing heatmaps bound to a single output rather than reusable, concept-level explanations that survive across downstream tasks.

That gap matters because a heatmap telling a radiologist that a model "looked at the liver" says nothing about whether the model internally represents hepatobiliary structure, ascites, or vascular congestion. MedConcept aims for the second kind of explanation, one a physician could verify against a written report.

Sparse autoencoders as a scalpel

The core trick is borrowed from the mechanistic interpretability work reshaping how researchers probe large language models. Sparse autoencoders (SAEs) have become a powerful tool for unsupervised concept discovery in deep networks, first proving their worth by decomposing LLM representations into semantically meaningful, disentangled features. Dense activation vectors are polysemantic: a single neuron fires for a jumble of unrelated concepts. An SAE re-expresses those activations in a much larger, sparse basis where each dimension ideally responds to one crisp concept.