Skip to main content
Back

SJTU's ChemDFM-R Beats GPT-4o on Chemistry With a 14B Open Model

A 14B open-source model built on functional-group reasoning beats GPT-4o and o4-mini on the toughest chemistry benchmark, ChemEval.

  • ChemDFM-R is a 14B open-source chemistry reasoning LLM built on Qwen2.5-14B.
  • It scores 73.8% on ChemEval, beating GPT-4o (63.3%), o4-mini (67.3%), and DeepSeek-R1 (57.6%).
  • Training centers on ChemFG, a dataset annotating functional groups in molecules and their transformations in reactions.
  • A four-stage pipeline combines domain pretraining, instruction tuning, mix-sourced distillation, and reinforcement learning.
  • Weights are available on Hugging Face and code at OpenDFM/ChemDFM.
  • Weak on numerical property prediction; strong on molecule- and reaction-centric reasoning with interpretable rationales.

Chemistry has been an awkward fit for general-purpose language models. They can recite textbook definitions, but ask them to trace how a molecule behaves under a specific reaction, and the reasoning tends to fall apart. A team from Shanghai Jiao Tong University's OpenDFM group thinks the missing ingredient is what chemists actually use when they reason: functional groups. Their new model, ChemDFM-R, is a 14B chemical reasoning LLM built around this idea, and it outperforms much larger commercial systems on hard chemistry benchmarks.

The functional group gap

Functional groups (things like hydroxyl, carbonyl, or phenyl substructures) are the intermediate abstraction chemists use to connect a molecule's structure to how it will react or what properties it will have. Their presence and transformations link molecular structures to high-level phenomena such as reactions and physicochemical behavior. Phenol, for example, consists of a phenyl group bonded to a hydroxy group and is water-soluble, while toluene, which swaps that hydroxy for a methyl, is not.

The authors argue that most chemistry LLMs skip this middle layer entirely. Existing chemical LLMs lean on literature text and high-level phenomena to enhance their chemical knowledge, without decomposing that knowledge into underlying mechanisms and atomized units. Their understanding of chemistry stays superficial, and performance on chemical tasks suffers. Feed a model a pile of journal articles and it learns to talk about reactions, but not to mechanistically reason about why they happen.