HKU's Dental-TriageBench Exposes Dangerous Gaps in 19 Frontier AI Models
A new expert-annotated benchmark of 246 real dental cases exposes how top MLLMs fail at safety-critical clinical routing when compared to junior dentists.
ProTakeaways
- New benchmark Dental-TriageBench tests multimodal reasoning on 246 real dental cases with expert-annotated referrals.
- Each case pairs a patient chief complaint with a panoramic X-ray (OPG), annotated by two senior dentists.
- Taxonomy is hierarchical: 8 coarse triage domains and 22 fine-grained treatment labels.
- Across 19 MLLMs tested, all trailed three junior dentists on fine-grained treatment-level triage.
- Ablation shows both complaint text and OPG image are necessary; either alone hurts fine-grained MacroF1.
- Dominant failure mode is omission on multi-domain cases: models return overly narrow referral sets.
Dental triage looks deceptively simple: read the patient's complaint, glance at the panoramic X-ray, decide who they need to see. In practice it is a safety-critical routing problem where missing a single referral can delay urgent care. A new benchmark from researchers at the University of Hong Kong shows that even the strongest multimodal models still bungle this task in ways that would be clinically unsafe.
Dental-TriageBench is described by its authors as the first expert-annotated benchmark for reasoning-driven multimodal dental triage. Built from authentic outpatient workflows, it contains 246 de-identified cases annotated with expert-authored golden reasoning trajectories, together with hierarchical triage labels. The team benchmarks 19 frontier models against three junior dentists and the gap is wide.
Why routing a toothache is harder than it looks
Dental triage is a multi-label decision rather than a single-department pick. A patient with pain in a lower molar may simultaneously need endodontics for the pulp, periodontics for gum disease, and oral surgery for an impacted wisdom tooth on the same side. Getting one of those right and missing the others is a failure of care, not a partial success.
The benchmark encodes this reality with a clinically hierarchical taxonomy of 8 coarse-grained triage domains and 22 fine-grained triage labels that cover major referral destinations and treatment types in routine dental triage. Coarse domains include specialties like oral and maxillofacial surgery, periodontics, endodontics, and prosthodontics; the fine-grained labels drill into specific treatments such as crowns, bridges, dentures, and implants.