Shanghai AI Lab's CauTion Cuts Causal Discovery Tokens 100x While Beating Every Baseline
CauTion is a new academic framework that teaches LLMs when to stay quiet in causal discovery, using algorithm consensus to gate LLM input and calibrate trust without any labeled data.
Takeaways
- New framework: CauTion integrates LLM domain knowledge into causal discovery using calibrated trust, not blind reliance.
- Key mechanism: Multiple statistical algorithms vote first; LLMs only arbitrate on genuinely disputed edges, cutting token usage by ~100x vs. competing methods.
- Annotation-free calibration: Algorithm consensus edges serve as free pseudo-labels to estimate LLM reliability on the fly — no human labels needed.
- Strong benchmark results: On Win95pts (76 variables), CauTion achieves SHD of 27 vs. 63 for the next-best LLM-augmented baseline, with F1 of 85.2%.
- LLM-agnostic: Tested across 6 LLMs (Claude, GPT-5, Llama, DeepSeek, Qwen); trust calibration keeps performance stable even with weak models.
- Open source: Code available on GitHub; currently validated on discrete-variable benchmarks only.
Figuring out which variables cause which , from data alone , is one of the hardest problems in science. A new paper from researchers at Shanghai AI Laboratory, Shanghai Jiao Tong University, Nanjing University, and Tongji University introduces CauTion, a framework that takes a fundamentally different approach to combining statistical algorithms with LLM domain knowledge for causal discovery. The core insight: an LLM should only get a vote when the statistics genuinely can't decide.
Why causal discovery is still broken
Causal discovery , the task of recovering a directed acyclic graph (DAG) that encodes cause-and-effect relationships from observational data , has been an active research area for decades. Constraint-based methods rely on sufficient sample sizes and can only recover a Markov equivalence class (MEC), leaving some edge directions unresolved. Score-based and continuous optimization methods face local minimum problems and are sensitive to score functions under specific assumptions and hyperparameter choices.
LLMs seemed like a natural fix. They encode rich domain knowledge about the world and can reason about causal relationships from variable names alone. LLM-augmented methods use such semantic priors as an additional source of evidence to compensate for the limitations of purely data-centric causal discovery algorithms. But the existing approaches have a critical flaw: they query the LLM on every possible variable pair, regardless of whether the statistics already have a confident answer. This leads to high token costs and, worse, exposes the entire graph to LLM errors.
Global methods query the LLM across all O(n²) variable pairs, incurring high token cost and unstable pairwise judgments. Undirected-only methods reduce the query space by applying LLMs only to undirected edges in a pre-computed equivalence class, but suffer from accumulated orientation errors and incomplete orientation. Both categories also rely on a single statistical algorithm, making results fragile to that algorithm's specific failure modes.