NVIDIA's Spatial-IQ Exposes Why Top AI Models Score 17.7% Where Humans Hit 82%
NVIDIA Research's Spatial-IQ benchmark reveals that top AI models hit 3D object-counting scores through shortcuts, not real spatial reasoning — and shows how to fix it.
- NVIDIA Research released Spatial-IQ, a benchmark decomposing 3D object counting into 9 hierarchical sub-tasks.
- Humans score 82.1% on the target task; the best AI model (Qwen) scores only 17.7%.
- Top models often reach correct final counts via shortcuts, bypassing the underlying reasoning chain entirely.
- Fine-tuning Qwen2.5-VL-32B with hierarchical chain-of-thought + RL lifted accuracy from 2.9% to 62.6%.
- Dataset (~80K scenes from NVIDIA Isaac Sim), code, and paper are all freely available on Hugging Face and GitHub.
- The framework doubles as a training signal: models forced to walk the sub-task hierarchy learn genuine composition, not shortcuts.
Ask a person to count every box in a stack of crates, including the ones hidden underneath, and they will score around 82%. Ask the best multimodal AI model available today, and it will score 17.7%. That gap is the central finding behind Spatial-IQ, a new diagnostic benchmark from NVIDIA Research that, unlike most AI benchmarks, opens the black box and shows you exactly where a model's reasoning breaks down.
Why a single score hides the real problem
Most spatial reasoning benchmarks hand a model an image and ask for a final answer. If the model gets it wrong, you have no idea why. Was it a perception failure, where the model could not tell where one object ended and another began? Or was it a reasoning failure, where the model could not infer that a hidden block must exist to physically support the ones above it? These are very different problems, and a single accuracy score cannot distinguish them. Spatial-IQ was built specifically to answer that question.
Nine sub-tasks, one target
The benchmark is grounded in developmental psychology. Humans counting a stack of boxes decompose the problem: identify clusters, parse columns and layers, count what is visible, then infer what must be hidden underneath for the structure to be physically stable. Spatial-IQ mirrors that decomposition, breaking 3D object counting into nine perceptual and cognitive sub-tasks organized by the developmental stages of human spatial cognition. NVIDIA Isaac Sim procedurally generates a diverse dataset of roughly 80,000 stacked 3D structures with per-task ground truth labels.
The nine sub-tasks form a clear hierarchy:
- S1–S4 (Perceptual grouping):