Technion and Nvidia Catch LLMs Lying About Their Own Reasoning
A new benchmark and DPO recipe from Technion and Nvidia Research measure and fix the gap between what LLMs decide and what they say.
ProTakeaways
- Technion and Nvidia introduce PSCB, a benchmark linking 85K decisions with 428K explanations and attribution vectors.
- Spearman rank correlation separates good from bad explanations far better than cosine similarity for attribution alignment.
- DPO on attribution-ranked preference pairs boosts worst-case self-consistency by up to 92.6% on ARC-Easy.
- Accuracy stays within 1.5% of baseline, and TruthfulQA plus human plausibility scores are preserved.
- Supervised fine-tuning on the same data degrades alignment by roughly 40%; the contrastive DPO signal is essential.
- Gains transfer across domains but not across attribution methods, since LIME, LIG, and SHAP encode different notions of importance. Code available here.
When you ask a large language model why it picked answer B, the response sounds convincing. Researchers keep finding, however, that the words the model actually leaned on to make the choice are often not the ones it highlights in its justification. A new paper from the Technion and Nvidia Research calls this the self-consistency gap, and it introduces both a large-scale benchmark to measure it and a fine-tuning recipe to close it.
The trust problem hiding in plain sight
Post-hoc explanations, the kind you get when you prompt a model to justify its answer after the fact, are the default interpretability tool for most LLM deployments. The trouble is that these rationales often draw on different input features than those that determined the model's answer, meaning a plausible-sounding justification can quietly misrepresent what actually shaped the output.
The paper distinguishes this specific property, called self-consistency, from the broader idea of faithfulness. Self-consistency measures the agreement between the factors influencing a model's prediction and those invoked in its explanation. It is a mechanical property with a concrete test: do the input tokens that mattered for the answer also matter for the explanation?

Why nobody had tackled this at scale
Measuring self-consistency requires computing feature attributions, which assign an importance score to each input token for a given output. Methods like SHAP are rigorous but painfully slow. Theoretically principled approaches such as CC-SHAP provide strong guarantees but require more than 4 minutes per example, rendering large-scale benchmarking and preference-based training impractical. Earlier work in this area was capped at around 100 test examples, which is not enough to draw firm conclusions or to train anything.
A benchmark 850 times larger than what came before
The authors' first contribution is the Post-hoc Self-Consistency Bank (PSCB), which links model decisions to explanations and attribution vectors at a scale that had not been feasible. It covers:
- Over 85,000 decisions with 428,000 explanations and attribution vectors across four QA datasets and two LLMs
- Four multiple-choice reasoning datasets: ECQA, ARC-Easy, ARC-Challenge, and CODAH