Stanford's SteerAct Fixes Broken Robots Without Touching Their Weights

SteerAct, from researchers at Stanford, MIT, and Harvard, adapts a deployed vision-language-action model without updating its weights. The method identifies one internal feature associated with task success, monitors that feature during execution, and adjusts it when the rollout begins to resemble previous failures. Each task requires as few as 24 rollouts labeled by outcome.

On three physical manipulation tasks using a frozen π0.5 backbone, the authors report mean success rising from 31.1% to 52.2%. Each task and method was evaluated over 30 trials, producing a 21.1 percentage-point gain. Across simulation benchmarks, SteerAct improved success by roughly 10% relative to the unmodified policies.

Why deployment failures are hard to patch

Vision-language-action models convert camera observations and language instructions into robot actions. When one fails after deployment, conventional fine-tuning requires training data, gradient updates, and enough validation to check that the update has not damaged behavior the policy already performs well.

Activation steering changes behavior while leaving model weights frozen. At inference time, the system adds a signed vector to an internal hidden state, shifting the model toward or away from a learned feature. This approach avoids test-time gradient updates, although it still requires offline analysis of the model’s activations.

Effective steering depends on three decisions: which feature to modify, when to intervene during a rollout, and how strongly to shift the activation. SteerAct makes all three decisions through a shared representation called a temporal descriptor.