---
title: "Sun Yat-sen's SIRI-Bench Exposes How Badly VLMs Fail at 3D Geometry"
subtitle: "A new 9,000-question benchmark hides geometry conditions inside 3D scenes, exposing that top vision-language models fail more than half the problems."
url: "https://alphasignal.ai/news/sun-yat-sen-s-siri-bench-exposes-how-badly-vlms-fail-at-3d-geometry"
author: "AlphaSignal Newsroom"
author_url: "https://alphasignal.ai/editorial-team"
company: "AlphaSignal"
company_url: "https://alphasignal.ai/company/alphasignal"
published: "2026-04-14T00:00:00.000Z"
updated: "2026-08-23T22:07:36.691Z"
type: "Paper"
topics:
  - "Image"
  - "Benchmarks"
subtopics:
  - "Vision Language"
  - "Object Detection"
upvotes: 1296
source_url: "https://arxiv.org/abs/2506.14512"
comments: 0
truncated: true
---

# Sun Yat-sen's SIRI-Bench Exposes How Badly VLMs Fail at 3D Geometry

> A new 9,000-question benchmark hides geometry conditions inside 3D scenes, exposing that top vision-language models fail more than half the problems.

Vision-language models have been racking up impressive scores on math and reasoning benchmarks, but those tests almost always hand the model every fact it needs in clean text. A new benchmark from Sun Yat-sen University buries the important conditions inside a rendered 3D scene, and the results are ugly: even OpenAI o3 and Doubao-1.6-Vision get more than half the questions wrong by a wide margin.

The benchmark, called [SIRI-Bench](https://arxiv.org/abs/2506.14512) (Spatial Intelligence ReasonIng Benchmark), takes a simple but pointed approach. Instead of describing a solid geometry problem in words or drawing a 2D diagram, the authors render the actual 3D object as an 8-second video and strip the numerical dimensions out of the question text. To answer, a model has to look at the scene, figure out what shape it is, estimate its measurements, and then do the multi-step math on top of that.

## Why textual math benchmarks were letting VLMs off easy

Spatial intelligence refers to the capacity to interpret spatial information, encompassing skills such as perceiving shapes, understanding transformations, and applying spatial knowledge. Prior spatial benchmarks mostly probed what the authors call intuitive intelligence, things like distance estimation or layout understanding, without forcing models to chain that perception into multi-step reasoning. Math-heavy benchmarks like MathVista and MATH-Vision target symbolic and geometric reasoning through plane geometry, but remain abstract and detached from real-world contexts.

SIRI-Bench sits in the gap. Each of the 9,078 video-question-answer triplets is a solid geometry problem where the geometric conditions have to be inferred from a rendered video rather than read from the prompt.

_Preview only. Full text requires an AlphaSignal Pro account._

## Sources

- [SIRI-Bench: Challenging VLMs' Spatial Intelligence through Complex Reasoning Tasks](https://arxiv.org/abs/2506.14512)
- [SIRI-Bench full paper (HTML)](https://arxiv.org/html/2506.14512)
- [Blender - 3D modelling and rendering package](http://www.blender.org)