Floating-Point Rounding Errors Make Llama-3.1-8B Chaotically Unpredictable
A new arXiv paper dissects how floating-point rounding errors turn LLM outputs into chaos, with three regimes and a cheap fix.
Takeaways
- New arXiv paper traces LLM unpredictability to floating-point rounding errors amplified through Transformer layers.
- Identifies three regimes: stable (bitwise constant), chaotic (rounding dominates), and signal-dominated (input overrides noise).
- Directional condition numbers exceed 10^6 in early layers; median instability is 0 but mean hits 8.84 x 10^6.
- Singular value theory fails: stability boundary stays near 10^-10 across all 4096 singular vectors, varying only 3x.
- Higher precision (FP32 vs BFloat16 vs FP64) shifts thresholds but does not remove chaos; architecture matters too.
- Simple fix: averaging 10-100 noisy forward passes converges to the true sensitivity, cancelling rounding artifacts.
A new arXiv paper argues that a big chunk of the flakiness developers see in LLM pipelines (non-reproducible outputs, agents that disagree on identical inputs, safety guardrails that suddenly flip) is chaos in the mathematical sense, seeded by floating-point rounding errors that get amplified through Transformer depth.
The paper, Numerical Instability and Chaos by Chashi Mahiul Islam, Alan Villarreal, Mao Nishino, Shaeke Salman, and Xiuwen Liu, sets out to explain why identical prompts on identical hardware can still produce different outputs, and it puts numbers on how bad the problem gets inside Llama-3.1-8B and GPT-OSS-20B.
The bug lives in the arithmetic
LLMs are increasingly wired into agentic workflows, and their unpredictability has become a reliability issue rather than a curiosity. Prior work catalogued the symptoms: Kim et al. reported 13.78% of correct answers flip due to persona changes, while Vijayaraj found perturbations of magnitude 10^-3 cause 12.4% diagnostic flips in clinical LLMs. Multi-agent frameworks like AutoGen and MetaGPT report failure and non-reproducibility rates in the 23 to 31 percent range on identical hardware.
What has been missing is the mechanism. The authors trace it to something concrete: floating-point arithmetic is neither associative nor deterministic across GPUs, and tiny rounding errors at the level of machine epsilon (roughly 10^-14 in FP32) either explode or vanish as they pass through each Transformer layer.
Three regimes and an avalanche
The core claim is that LLMs, under perturbation, operate in one of three regimes:
- Stable (constant) regime