---
title: "Pacific Northwest Lab's Granite Vision 3.3 Struggles Where Physics Meets Diagrams"
subtitle: "A systematic reward ablation on physics problems shows attention-based supervision doubles spatial reasoning accuracy but breaks symbolic math domains."
url: "https://alphasignal.ai/news/pacific-northwest-lab-s-granite-vision-3-3-struggles-where-physics-meets"
author: "AlphaSignal Newsroom"
author_url: "https://alphasignal.ai/editorial-team"
company: "AlphaSignal"
company_url: "https://alphasignal.ai/company/alphasignal"
published: "2026-04-15T00:00:00.000Z"
updated: "2026-08-23T21:09:20.680Z"
type: "Paper"
topics:
  - "Post Training"
  - "Image"
subtopics:
  - "Rlhf"
  - "Vision Language"
upvotes: 1162
source_url: "https://arxiv.org/abs/2604.13993"
comments: 0
truncated: true
---

# Pacific Northwest Lab's Granite Vision 3.3 Struggles Where Physics Meets Diagrams

> A systematic reward ablation on physics problems shows attention-based supervision doubles spatial reasoning accuracy but breaks symbolic math domains.

Reinforcement learning has become the default recipe for teaching language models to reason, but almost all the recent breakthroughs (DeepSeek-R1 style GRPO training, verifiable rewards, rubric feedback) have been tested on text-only problems. A new study from Pacific Northwest National Laboratory [runs the same playbook on a vision-language model](https://arxiv.org/abs/2604.13993) tackling physics, and the results are messier than the language-only literature would suggest. State-of-the-art VLMs fall far short of human performance on physics benchmarks, and how reward design shapes their physical reasoning behavior remains poorly understood.

## Why physics breaks vision-language models

Physics problems with diagrams are uniquely hostile to current VLMs. Solving one requires reading a schematic, recalling the right physical law, doing symbolic multi-step math, and producing an answer with correct units. The [PhyX benchmark](https://phyx-bench.github.io/) was built to test this combination, and the gap it exposes is brutal: GPT-4o, Claude 3.7 Sonnet, and GPT-o4-mini score 32.5%, 42.2%, and 45.8% respectively, trailing human experts by more than 29 points.

PhyX is deliberately hard because de-redundancy in the benchmark forces models to perceive concepts from one modality only. You cannot cheat by reading the answer out of the caption. The dataset covers six domains (Mechanics, Electromagnetism, Thermodynamics, Wave/Acoustics, Optics, and Modern Physics) in both multiple-choice and open-ended formats.

## Four rewards, one small model

_Preview only. Full text requires an AlphaSignal Pro account._

## Sources

- [Reward Design for Physical Reasoning in Vision-Language Models](https://arxiv.org/abs/2604.13993)
- [PhyX Benchmark](https://phyx-bench.github.io/)
- [PhyX: Does Your Model Have the Wits for Physical Reasoning?](https://arxiv.org/pdf/2505.15929)
- [VLMEvalKit PhyX Support](https://github.com/open-compass/vlmevalkit)