NVIDIA's Spatial-IQ Exposes Why Top AI Models Score 17.7% Where Humans Hit 82%
NVIDIA Research's Spatial-IQ benchmark reveals that top AI models hit 3D object-counting scores through shortcuts, not real spatial reasoning — and shows how to fix it.
- NVIDIA Research released Spatial-IQ, a benchmark decomposing 3D object counting into 9 hierarchical sub-tasks.
- Humans score 82.1% on the target task; the best AI model (Qwen) scores only 17.7%.
- Top models often reach correct final counts via shortcuts, bypassing the underlying reasoning chain entirely.
- Fine-tuning Qwen2.5-VL-32B with hierarchical chain-of-thought + RL lifted accuracy from 2.9% to 62.6%.
- Dataset (~80K scenes from NVIDIA Isaac Sim), code, and paper are all freely available on Hugging Face and GitHub.
- The framework doubles as a training signal: models forced to walk the sub-task hierarchy learn genuine composition, not shortcuts.
Ask a person to count every box in a stack of crates , including the ones hidden underneath , and they will score around 82%. Ask the best multimodal AI model available today, and it will score 17.7%. That gap is not just a number. It is the central finding behind Spatial-IQ, a new diagnostic benchmark from NVIDIA Research that does something most AI benchmarks refuse to do: it opens the black box.
The problem with "just count the blocks"
Most spatial reasoning benchmarks hand a model an image and ask for a final answer. If the model gets it right, great. If it gets it wrong, you have no idea why. Was it a perception failure , the model could not tell where one object ended and another began? Or was it a reasoning failure , the model could not infer that a hidden block must exist to physically support the ones above it? These are very different problems, and a single accuracy score cannot distinguish them.
Existing benchmarks evaluate models as black boxes, making it difficult to ascertain whether a failure is perceptual, such as recognizing object boundaries, or cognitive, such as reasoning about occlusion to infer hidden geometry. Spatial-IQ was built specifically to answer that question.
Nine sub-tasks, one target
The benchmark is grounded in developmental psychology. Humans do not count a stack of boxes in one shot , they decompose the problem: identify clusters, parse columns and layers, count what is visible, then infer what must be hidden underneath for the structure to be physically stable. Spatial-IQ decomposes 3D object counting into 9 perceptual and cognitive sub-tasks organized by the developmental stages of human spatial cognition, using NVIDIA Isaac Sim to procedurally generate a diverse dataset of roughly 80,000 stacked 3D structures with per-task ground truth.