Skip to main content
Back

PLAN-S Cuts Self-Driving Collision Rates 42% Without Retraining Any Model

PLAN-S adds a style-conditioned cost map between a frozen world model and its planner, cutting collision rates by 42% and enabling controllable driving behavior without retraining

Paper, Full Access
  • PLAN-S inserts a style-conditioned semantic cost map between a frozen latent world model and its trajectory planner, without retraining the backbone.
  • The cost map explicitly encodes drivability, collision risk, and lane boundaries, conditioned on ego state and a driving style token (aggressive/normal/conservative).
  • On nuScenes, PLAN-S achieves 0.55 m average L2 and a 42% relative reduction in 3-second collision rate over the ResWorld baseline.
  • On NAVSIM, the rule-cost variant reaches 89.4 PDMS, competitive with frontier models like DriveFuture (90.7 PDMS).
  • Two planner interfaces are supported: attention-level fusion for regression planners and reward-level fusion for anchor-score planners.
  • No code or model weights have been released; the paper is available at arxiv.org/abs/2606.06014.

End-to-end autonomous driving has made a quiet but significant bet: compress the entire world into a compact latent representation, then plan directly from it. PLAN-S (PLANning with latent Style dynamics) is a new paper that challenges whether that direct latent-to-trajectory pipeline is actually the right design. The answer, backed by strong benchmark numbers, is: not quite.

The black box problem inside world models

Latent world models (LWMs) work by encoding sensor inputs -- cameras, LiDAR -- into a compressed feature space that captures scene dynamics. World models aim to internalize the physical structure and dynamics of the real world into a predictive latent representation for imagination, control, and planning, and recent studies have introduced them into autonomous driving for scene generation, simulation evaluation, and decision-making. The planning pipeline that follows is where things get tricky.

The standard approach feeds these latent features directly into a trajectory planner. It works, but it creates a fundamental problem: the latent space bundles together everything -- road geometry, other agents, risk zones, drivability -- with no explicit structure. There is no way to ask the model "what does it think is dangerous here?" or "can you drive more conservatively?" before a trajectory is selected. Existing latent world models suffer from misalignment between reconstruction-oriented learning and planning requirements. PLAN-S calls this the compactness-controllability dilemma.

PLAN-S architecture: LWM-based environment prediction feeding into style-fused planning with dual AdaFiLM and costmap decoder

A semantic bridge between perception and decision

Rather than redesigning the world model or the planner, PLAN-S inserts a new module between them: a style-conditioned semantic cost map. Think of it as a top-down spatial map of the scene that explicitly marks four things -- drivable area, collision risk, lane boundaries, and free space -- all rendered as a function of the current driving style preference (aggressive, normal, conservative) and ego vehicle state.