Skip to main content
Back

Politehnica Bucharest's GEST Framework Beats GPT-5 With Budget Claude Haiku at 80%

Researchers flipped the video generation stack: LLMs plan stories, a game engine executes them, and every scene comes with pixel-perfect ground truth.

Pro

Paper, Full Access
  • LLM agents author formal event graphs that a game engine executes deterministically, replacing neural video generation.
  • Staged pipeline with GPT-5 produced 0 executable graphs in 50 attempts; tool-constrained Haiku 4.5 hit 20 of 25.
  • Architecture uses a Director and Scene Builder over 30+ validated tools with transactional state backend.
  • Engine videos rated 69% physically valid versus VEO 3.1 at 18% and WAN 2.2 at 13%.
  • Reconstruction fidelity reaches 0.83 event F1 against source graphs, versus 0.55 for a random baseline.
  • Authoring a full multi-scene story costs approximately $0.25 in API calls with a budget model.

Most agentic video generation systems today hand the final word to a neural generator. An LLM writes a script, another agent casts characters, and a diffusion model paints the frames. The output looks impressive until you watch closely: objects vanish, actors morph mid-motion, and temporal order quietly breaks. A team from the Institute of Mathematics of the Romanian Academy and Politehnica Bucharest inverts that pipeline. Rather than asking an LLM to describe pixels, they ask it to author a formal event graph that a 3D game engine executes deterministically, with per-frame ground truth for every actor, object, and interaction.

The paper, Authoring for Living Worlds, reports a jarring result: a standard staged pipeline driving GPT-5 produced zero executable specifications out of 50 attempts. A different architecture, driving the budget Claude Haiku 4.5, produced 20 executable specifications out of 25. The model is not the trick. Where the constraints live is.

System diagram showing LLM Director and Scene Builder authoring a GEST through a constraint-enforcing tool layer, which the engine executes into video frames

The graph that replaces the pixel

The core object is a Graph of Events in Space and Time (GEST). Nodes are events (actions with multiple entities, performed by actors in specific locations) and edges carry temporal constraints from Allen's interval algebra, with support for logical and semantic relation types. Think of it as a screenplay compiled to a directed graph: who does what, to whom, where, and in what temporal relationship to everything else happening in the scene.

The execution target is unusual. After verifying the graph, the engine executes it deterministically through the Multi Theft Auto framework, in GTA San Andreas: a shipped game world with 70+ environments, 733 objects, 2,500+ animations, and 312 character skins. Because the world is a real simulator with state, executing a valid graph produces a film plus a full annotation pack. Two to six synchronized actors play a multi-scene story under automatic camera tracking, and the engine records per-frame entity and camera state, all pairwise spatial relations, instance segmentation, exact event-to-frame boundaries, and a graph-derived description.

That last part is what neural video generators cannot give you at any price: ground truth. No bounding box, no segmentation mask, no event boundary requires human labeling, because the engine knew all of it before it rendered the first frame.