Reka's WMGym Judges World Models on Decision-Making, Not Pretty Pixels
Reka's WorldModelGym shifts world model evaluation from visual realism to decision-making accuracy, launching an open leaderboard with a simple API submission protocol
- New leaderboard: Reka launched WorldModelGym (WMGym), evaluating world models on decision fidelity rather than visual realism.
- Core metric: Decision fidelity measures how often a world model correctly ranks which action leads to the best outcome.
- Four benchmarks: Classical Control, DeepMind Control, Meta-World, and Atari form the four evaluation families.
- Baseline results: Reka's Dreamer-v3 leads three of four families, hitting 84% decision fidelity on Classical Control.
- Open submission: Anyone can enter by hosting a
/scoreendpoint; Reka calls it and computes the score — no weight uploads needed. - Free to use: The leaderboard is open and free; the benchmark targets robotics, game AI, and simulation-based planning use cases.
Most world model benchmarks ask a deceptively simple question: does the generated video look real? WorldModelGym (WMGym) from Reka asks something fundamentally different: given a set of possible actions, can the model correctly predict which one leads to the best outcome? That shift in framing is not cosmetic. It targets the gap between a model that generates convincing pixels and one that actually understands how the world works.
The leaderboard is now open. Reka seeded it with their own Dreamer-v3 implementation as the first entry, and they are inviting the community to submit models via a lightweight API protocol.
The problem with looking good
The world model field has been dominated by video generation quality metrics: does the output look photorealistic, does motion flow naturally, does physics seem plausible to a human eye? Existing benchmarks largely treat world models as video generators and do not assess their functional role in decision-making or interaction. The result is a leaderboard ecosystem that rewards perceptual polish but says nothing about whether a model can be used as a simulator for an RL agent or a planning system.
While these benchmarks are effective for standardizing generative evaluation, they largely treat world models as video generators and do not assess their functional roles in decision-making or interaction. WMGym is a direct response to that gap. The core metric is