Skip to main content
Back

Hybrid RL-MPC Planner Hits 72% Better Success and 2.1x Faster Learning

A new paper accepted at L4DC couples reinforcement learning with sampling-based MPC, boosting success rates up to 72% while cutting training time.

  • New paper fuses RL with MPPI sampling-based MPC in a hierarchical adaptive planner (arXiv:2512.17091).
  • RL policy biases the MPPI sampler; MPPI rollouts feed back into value learning based on critic uncertainty.
  • Reports up to 72% higher success rate and 2.1x faster convergence than baselines.
  • Evaluated on race driving, modified Acrobot, and obstacle-laden Lunar Lander environments.
  • Accepted at the 8th Learning for Dynamics and Control Conference (L4DC).
  • Treats MPPI as a structured data generator that regularizes high-level policy learning.

A team of researchers has proposed a hybrid planner that tightens the loop between reinforcement learning and model predictive control, and the numbers are notable: up to a 72% jump in task success and 2.1x faster convergence over comparable baselines. The paper, titled Learning to Plan, Planning to Learn, is authored by Toshiaki Hori, Jonathan DeCastro, Deepak Gopinath, Avinash Balachandran, and Guy Rosman, and is slated for the 8th Annual Learning for Dynamics and Control Conference (L4DC).

The two planners that never quite got along

Reinforcement learning agents excel at absorbing long-horizon reward signals into a fast reactive policy, but they are notoriously sample-hungry and can wander into dangerous states while exploring. Model Predictive Control (MPC), and its sampling variant MPPI, do the opposite: they roll out many candidate trajectories through a dynamics model at each timestep and pick the best one, which is safer and more interpretable but expensive and myopic. Combining the two is an obvious idea, and prior work has tried, though such methods usually focus on specific cost formulations, couple the components in rigid ways, or fail to exploit the problem structure on either side.

To ground the acronym: MPPI, or Model Predictive Path Integral control, is a stochastic sampling-based controller. It derives an optimal control law as a noise-weighted average over sampled trajectories, where the optimization is posed to push the controlled distribution as close as possible to the optimal one, corresponding to minimizing a KL divergence. In plain terms: perturb your current plan many times, score each rollout, and take a reward-weighted average as your next action.