Skip to main content
Back

SHARS Stops AI Hallucinations Mid-Sentence Before They Snowball

SHARS intercepts and resamples hallucinated text segments mid-generation, cutting factual errors without needing web search or external knowledge bases

  • SHARS is a new inference-time framework that intercepts and resamples hallucinated text segments during LLM generation, requiring no fine-tuning or external knowledge.
  • It targets hallucination snowballing, where early factual errors compound into larger chains of fabricated content in long-form outputs.
  • The method uses semantic uncertainty as a detector: sampling multiple completions per segment and flagging those where answers diverge significantly.
  • SHARS achieves 62-78% factual precision on long-form benchmarks, versus ~52% for standard greedy decoding, while preserving output informativeness.
  • The framework is detector-agnostic and modular -- any hallucination detector can be plugged in, and it works with closed-source models via API sampling.
  • Code is publicly available on GitHub; the main tradeoff is increased inference latency from multiple sampling passes.

Hallucinations in long-form LLM output are not just a nuisance -- they are a compounding disaster. A new paper by Lin Li, G. Channing, Suhaas Bhat, G. Jones, and Y. Gal introduces SHARS (Segment-wise HAllucination Rejection Sampling), an inference-time framework that intercepts hallucinated segments as they are generated and resamples them until the model produces faithful content. The result: substantially higher factual precision on long-form benchmarks, with no need for web search or external knowledge bases.

The snowball nobody wants

The core problem SHARS targets is what researchers call hallucination snowballing. An LM over-commits to early mistakes, leading to more mistakes that it otherwise would not make. In other words, one fabricated claim early in a paragraph becomes the foundation for the next sentence, and the next -- each error making the overall output harder to recover from.

Exposure bias can cause the model to hallucinate if an early token is incorrect or contextually inappropriate, and this deviation is further exacerbated by the cascade effect, where an early mistake leads to a chain reaction of subsequent errors. For short-form QA, this is manageable. For long-form generation -- think biographies, research summaries, technical explanations -- it is a fundamental reliability problem.

Example showing hallucinated vs. factual claims in a long-form biography of Ramesses IV, with green boxes for correct facts and red boxes for hallucinated content

Detect, reject, resample

SHARS works at inference time, meaning it requires no fine-tuning or retraining. The key idea is to break generation into segments, check each segment for hallucination, and throw out any segment that fails the check -- then regenerate from the last verified anchor point. The framework is deliberately detector-agnostic: any hallucination detector can be plugged in.