Cornell's MASH Teaches AI to Say 'I Don't Know' Without Extra Training
A new reinforcement learning recipe teaches models to search only when they don't know, then reuses that same signal to decide when to abstain.
ProTakeaways
- MASH trains LLMs to selectively call search tools using RL with a pay-per-search reward, no knowledge boundary labels needed
- Removing search access at inference turns any search attempt into an abstention signal, giving two behaviors for the cost of one
- Delivers 7.6% accuracy improvement on multi-hop QA and matches specialized DPO abstention methods that require oracle knowledge boundaries
- Warm-start SFT with random search counts is critical; harsher reward penalties collapse to degenerate policies without it
- Oracle helpers (perfect retrievers) break training by making "always search" the trivial optimum, so retriever noise is essential
- Code, checkpoints and full pipeline available at github.com/momergul/mash
Getting a language model to say I don't know is surprisingly hard. Most abstention training pipelines require pre-computing which questions a model can and cannot answer, then hand-building a dataset that pairs unanswerable questions with refusals. A new paper from researchers at Cornell proposes a cleaner trick: teach the model to use a search tool efficiently, and its search behavior becomes an abstention signal for free.
The method, called MASH (Modeling Abstention via Selective Help-Seeking), reframes abstention as a side effect of frugal tool use. Charge a model every time it hits a search API, reward it for correct answers, and the optimal policy will only search when it genuinely lacks the answer. Strip away the search tool at inference, and any attempt to search becomes a proxy for I don't know.
Why existing abstention methods hit a wall
The core issue with LLM abstention is circular: to train a model to refuse questions it can't answer, you first have to figure out which ones those are. Past work leans on pipelined approaches that estimate a model's knowledge boundaries and then use this information either to construct datasets for SFT and DPO training, or to train model-specific reward functions for RLHF. That estimation step is expensive, model-specific, and has to be redone every time you fine-tune.
MASH sidesteps this entirely. The framework assumes no privileged information about knowledge boundaries and requires no structured multi-agent interactions. The model figures out its own limits during RL training, guided only by whether searching was worth the penalty.