---
title: "UW Researchers Force Vision AI to Mentally Sketch Unseen Spaces Before Answering"
subtitle: "A new training technique teaches vision-language models to mentally simulate unseen viewpoints before answering spatial questions, beating text-based chain-of-thought reasoning."
url: "https://alphasignal.ai/news/uw-researchers-force-vision-ai-to-mentally-sketch-unseen-spaces-before-answering"
author: "AlphaSignal Newsroom"
author_url: "https://alphasignal.ai/editorial-team"
company: "AlphaSignal"
company_url: "https://alphasignal.ai/company/alphasignal"
published: "2026-06-03T00:00:00.000Z"
updated: "2026-06-15T01:04:15.791Z"
type: "Paper"
topics:
  - "Image"
  - "Llms"
subtopics:
  - "Vision Language"
  - "Vision Language"
  - "Structured Output"
  - "Hallucinations"
upvotes: 70
source_url: "https://arxiv.org/abs/2606.03988"
comments: 0
truncated: true
---

# UW Researchers Force Vision AI to Mentally Sketch Unseen Spaces Before Answering

> A new training technique teaches vision-language models to mentally simulate unseen viewpoints before answering spatial questions, beating text-based chain-of-thought reasoning.

Ask a vision-language model (VLM) where an object is hiding behind a wall, or what a room looks like from the opposite corner, and it will almost certainly struggle. These are not perception failures -- the model sees the image just fine. They are **spatial imagination** failures: the model cannot reason about what it cannot directly see. A new paper from researchers at the University of Washington and AI2 proposes a surprisingly clean fix.

The work introduces [**Imaginative Perception Tokens (IPT)**](https://arxiv.org/abs/2606.03988), a training-time supervision technique that forces a VLM to generate an intermediate visual representation of an unobserved viewpoint before producing its final answer. Think of it as teaching the model to mentally sketch what it would see from another angle, then use that sketch to reason.

## Why spatial reasoning is still broken

The problem runs deep. LLMs remain fundamentally limited in spatial reasoning because their learning is confined to text-based representations -- they learn spatial concepts as statistical patterns of words, not as grounded geometric representations. VLMs inherit this bias even after adding vision. Spatial reasoning remains a universal challenge: across 32 tested models, average accuracy sits at 28.4%, only marginally above random chance (25%), and far short of the human baseline of 76.8%.

The go-to fix has been chain-of-thought (CoT) prompting -- asking the model to reason step by step in text before answering. But for spatial tasks, this backfires. Introducing CoT prompting leads to a decline in spatial reasoning performance, suggesting that explicit CoT reasoning steps may not always be beneficial for VLMs. The IPT paper makes this point even more forcefully: textual chain-of-thought can **substantially degrade** performance on spatial tasks, which the authors attribute to a _modality mismatch_ \-- spatial computation is fundamentally visual, and forcing it through language loses critical geometric structure.

_Preview only. Full text requires an AlphaSignal Pro account._

## Sources

- [Imaginative Perception Tokens Enhance Spatial Reasoning in Multimodal Language Models (arXiv)](https://arxiv.org/abs/2606.03988)
- [IPT paper page - Jaemin Cho (CVPR 2026 MUSI Workshop)](https://j-min.io/publication/ipt_cvprw2026/)
- [BAGEL: Open-Source Unified Multimodal Model (ByteDance Seed)](https://seed.bytedance.com/en/blog/seed-research-bagel-the-open-source-unified-multimodal-model-an-all-in-one-model)
- [Spatial Reasoning in Multimodal Large Language Models: A Survey](https://arxiv.org/html/2511.15722v1)
- [NuScenes-SpatialQA: Effect of Chain-of-Thought on Spatial Reasoning](https://arxiv.org/pdf/2504.03164)