COSMIK-MPPI Lets Robot Arms Dodge Humans at 22ms With 100% Success
A new sampling based controller keeps robot arms from hitting people by treating near collisions as trajectory terminations, running at 22ms.
ProTakeaways
- COSMIK-MPPI hits 100% task success at a fixed 22ms per control step for manipulator collision avoidance.
- Constraints as Terminations replaces additive collision penalties, so unsafe rollouts lose their weight in the MPPI average.
- Gradient-based MPC blows past 1800ms and fails in non-convex scenes where sampling-based MPPI stays constant.
- Human pose comes from RT-COSMIK, a two-webcam markerless pipeline running at 20Hz.
- Real robot handles physical perturbations, concurrent human activity, and a human forming a loop around the arm.
- Trajectories are less smooth than GB-MPC and safety is statistical, not formally guaranteed, per the paper.
A team from LAAS-CNRS, CIIRC at Czech Technical University, and LIRMM has released COSMIK-MPPI, a predictive controller that lets a torque-controlled robot arm share a workspace with a moving human without hitting them. The headline number is a 100% task success rate in simulation and a fixed 22ms compute budget per control step, which is what you need if a manipulator is going to react to a person in the loop.
The setup is realistic in a way most collision-avoidance papers are not. Human pose comes from two cheap RGB webcams instead of a marker suit, joint torques are commanded directly, and the person is allowed to be adversarial, poking the arm with a stick or looping their arms around it to form a narrow passage.
Why the classic solvers keep breaking
Model Predictive Control (MPC) is the default framework for this problem because it can look ahead over a short horizon and enforce hard constraints. The dominant flavor, Gradient-Based MPC, formulates collision avoidance as a nonlinear program and solves it with sequential quadratic programming. That works fine on toy problems, but it has two well known failure modes. Collision constraints are non-convex, so the solver gets trapped in local minima and is very sensitive to how you warm-start it. More damaging, the solve time explodes as you add collision pairs. In the benchmark reported here, GB-MPC goes from 15ms in an empty workspace to 415ms with four obstacle primitives and 1812ms in a dense circular arrangement. At those numbers the controller is no longer real-time and the task success rate drops to zero.
The sampling-based alternative, Model Predictive Path Integral (MPPI), sidesteps gradients entirely. It draws thousands of random control sequences, rolls each one forward through the robot dynamics, scores them, and takes a softmin-weighted average. That scales cleanly, but vanilla MPPI encodes safety as an additive penalty in the cost function. Tune the weight too low and the arm happily grazes obstacles. Tune it too high and the numerics blow up.