WorldCoder-Bench Exposes How Top AI Models Fail 3D Interactive Web Apps
WorldCoder-Bench exposes a hard ceiling for LLMs building interactive 3D worlds in Three.js, with the best model scoring just 27.8%
- WorldCoder-Bench is a new benchmark of 2,026 expert-curated tasks for evaluating LLMs building interactive 3D worlds in Three.js.
- The best of nine frontier models scores only 27.8% on the standard split and 19.9% on the harder robust split.
- A new evaluation protocol called StateProbe injects JavaScript probes into a sandboxed browser to verify hidden runtime state, not just visual output.
- Most failures are caused by state-schema drift and broken interaction chains, not missing visual elements, making screenshot-based evaluators blind to them.
- Two new utility metrics, Return on Automation and Time Efficiency Multiplier, show cheaper models can still deliver value on simpler rendering tasks.
- The benchmark and code are publicly available for running evaluations on any model.
Building a static webpage is one thing. Building an interactive 3D physics simulation that responds correctly to user input, loads external 3D assets, and maintains consistent internal state across every interaction is something else entirely. A new benchmark called WorldCoder-Bench puts nine frontier LLMs through exactly that challenge, and the results are humbling: even the best model passes fewer than 3 in 10 tests.
The canvas problem nobody was measuring
Most web-generation benchmarks evaluate models by looking at screenshots or inspecting the DOM (the structured tree of HTML elements on a page). That works fine for static layouts and forms. But when a model generates a Three.js application, the interesting stuff happens inside a <canvas> element, which is essentially a black box to traditional evaluators. You can see the pixels, but you cannot query the scene graph, the physics state, or whether a button click actually mutated the right internal variable.
This is the gap WorldCoder-Bench was built to close. LLMs are increasingly asked not only to write static interfaces, but to construct executable interactive worlds from natural language. Three.js, the dominant library for browser-native 3D, is a natural test bed because generated programs must integrate assets, obey spatial and physical constraints, and keep user-facing controls synchronized with hidden runtime state.
What the benchmark actually tests
WorldCoder-Bench contains 2,026 expert-curated tasks across Simulation, Rendering, and Application scenarios, with optional .glb assets and hidden behavioral contracts. The .glb format is the binary version of glTF, the standard container for 3D models and scenes. Tasks span a wide range of real-world use cases:
This story is for Pro members
You've reached the end of the free preview. Upgrade to AlphaSignal Pro to read the full article - and everything else behind the paywall.
Comments
