Skip to main content
Back

Evo-MedAgent Teaches Frozen AI Doctors to Learn From Past Mistakes

A training-free memory module lets frozen chest X-ray agents accumulate lessons across cases, lifting MCQ accuracy up to 11 points without any fine-tuning.

  • Evo-MedAgent adds a training-free memory layer that lifts chest X-ray MCQ accuracy by up to 11 points.
  • Three stores: episodic case traces, priority-tagged procedural rules, and per-tool trust labels updated by reflection.
  • On ChestAgentBench, Gemini-3 Flash jumps from 0.76 to 0.87 with memory enabled.
  • Tool-free memory beats tool-enabled baselines on qualitative tasks when the base VLM is strong enough.
  • Per-case overhead is one retrieval pass plus one reflection call, deployable on any frozen model.
  • Limitations: needs ground-truth feedback, order-sensitive, quantitative measurements still require specialist tools.

A radiologist who misreads a subtle pneumothorax on Monday is unlikely to make the same mistake by Friday. Today's tool-augmented medical agents don't have that luxury. They can call classifiers, segmenters, and vision-language models in sequence, but every patient arrives as a blank slate. A new paper called Evo-MedAgent tries to close that gap with a plug-in memory system that lets a frozen model learn from its own mistakes at test time.

The headline numbers are unusually clean for an agent paper. On ChestAgentBench, Evo-MedAgent raises multiple-choice accuracy from 0.68 to 0.79 on GPT-5-mini, and from 0.76 to 0.87 on Gemini-3 Flash. No fine-tuning, no reinforcement learning, no gradient updates anywhere in the stack.

Why static agents keep tripping on the same rock

Recent systems like MedRAX, CheXagent, and MDAgents orchestrate specialist tools through a ReAct-style reasoning loop. They work, but they forget. Each case is solved in isolation, experience never accumulates, and recurrent reasoning mistakes cannot be corrected without expensive reinforcement learning.

The authors argue the errors are often procedural rather than perceptual: wrong tool ordering, badly chosen hyperparameters, over-trust in a noisy classifier. Retraining the underlying VLM to fix any single one of these is absurdly expensive, and there is no obvious way to encode a lesson like "when you see a unilateral effusion in a teenager, also check the ribs for a spur."

Three memory stores, each doing a different job

Evo-MedAgent factors memory into three components that get read before inference and written after ground-truth feedback: