Robo-bench tests whether coding agents can control robots in real time

Vals AI has released Robo-bench, a benchmark that places frontier language models in real-time MuJoCo physics simulations and asks them to control two robots through cameras, sensors, and low-level motor commands. The top model averaged 50.8 points on the arm tasks, while every humanoid average remained below 2 points.

MuJoCo simulates the mechanics and contacts of a 7-degree-of-freedom Franka Emika Panda arm and a 29-degree-of-freedom Unitree G1 humanoid. Each episode lasts three minutes of wall-clock time, so the simulation continues while the model inspects files, reasons, writes Python, and runs its controller.

Robots stripped to the hardware

For each task, the agent receives the robot’s datasheet, kinematic model, sensor interfaces, motors, and cameras. It receives no motion planner, grasping library, walking controller, or object coordinates. A humanoid controller must establish and maintain balance using code written during the episode.

The Panda’s internal servos track position targets, with joint commands limited to the speeds in its datasheet. A protective stop halts motion when a link applies more than 45 N or the hand applies more than 80 N. The G1 follows a torque-speed curve that limits how much force its motors can produce at a given speed, preventing controllers from requesting physically impossible power.

Models infer the task scene from an overhead camera and wrist camera for the arm, plus a head camera for the humanoid. They can request depth images, but the benchmark does not expose object positions directly.

Arm control carries the leaderboard

Vals AI scores 24 tasks, comprising 13 arm tasks and 11 humanoid tasks. Each task is worth 100 points, with ordered milestones providing partial credit. GPT-6.1 Sol ranks first overall at 28.33%, followed by GPT-6 Astra at 23.33% and Claude Opus 5.5 at 14.58%.

Each model brings its coding harness

Every model runs through its vendor’s coding agent at the highest reasoning setting available. The benchmark uses the following pairings:

• OpenAI models run in Codex.
• Anthropic models run in Claude Code.
• Gemini runs in Antigravity.
• Grok runs in Grok Build.
• Muse runs in Muse Code.

The workspace contains a GOAL.md task description, a ROBOT.md datasheet, the MuJoCo robot model with the task scene excluded, and a small Python client. That client exposes live sensor state and allows the agent to register a function that runs at every simulator control step.

Because each entry includes a vendor-specific coding harness, the leaderboard reflects the combined performance of the model, its agent software, and its tool-use strategy. It does not isolate the underlying language model.

Reasoning consumes control time

While a model reasons or edits code, physics continues to run and gravity continues to act. A humanoid left on stiff joints can fall before the first command arrives. Stable balance also requires corrections roughly once per second, faster than a typical model turn, so the generated program must repeatedly read sensor state and adjust motor commands on its own.

The measured latency accounts for a substantial share of the ranking. GPT-6.1 Sol and GPT-6 Astra moved a robot in every episode, with time to first move of 79.89 seconds and 74.88 seconds respectively. The other five models averaged more than two minutes. Gemini 4 Argon and Grok 4.7 failed to move either robot in most episodes, leaving little time to earn milestones within the three-minute limit.

Controllers emerge during the episode

During the balance-scale task, GPT-6.1 Sol inspected the camera feeds nine times, wrote a controller, and issued 2,204 joint commands. It carried labeled weights to the empty pan until the beam reached level and remained there.

The Lego-manual task produced narrower progress. Sol, Astra, and Opus each placed one brick before declaring the build complete, while the other four models placed none.

Humanoid balance remains unsolved

Across the 13 arm tasks, GPT-6.1 Sol averaged 50.8 points and fully solved four tasks. Across the 11 humanoid tasks, no model completed a task, and no average exceeded 2 points.

On the public ninja course, Sol’s generated controller pitched the G1 onto the floor at the starting line. The robot then kicked and twisted for the remainder of the episode. The run illustrates the control burden: the agent must coordinate 29 joints, keep the center of mass over the feet, respond to contact, and still pursue the task objective.

Interpreting the rankings requires accounting for several benchmark design choices and sources of variance:

• Only the balance scale, Lego manual, and ninja course are public. The remaining 21 tasks are held out to reduce exposure through training data.
• Each task uses one simulation seed. Repeated runs shifted a model’s total by as much as roughly 10 points, so small leaderboard gaps are inconclusive.
• A full 24-task run costs from under $5 for Sol to nearly $30 for Astra at listed API prices.
• Milestones must be reached in order, which limits partial credit when an agent skips an intermediate step.

For developers building robotic agents, Robo-bench measures three capabilities together: rapid code generation, visual interpretation, and autonomous feedback control. Its current results show useful arm behavior within a three-minute budget, while whole-body humanoid stabilization remains beyond the tested systems.