Fast-JEPA Hits 96.5% Success While Cutting AI Planning Time by 2.41x
Fast-JEPA rearranges how latent world models search over action plans, cutting planning time to 10.88 ms while hitting 96.50% average success.
Takeaways
- Fast-JEPA hits 96.50% average success at 10.88 ms, a 2.41x speedup over INTACT.
- Builds a goal-directed proposal, then scores complete candidates via direct terminal predictions instead of recursive rollout.
- Candidate evaluation scales as H+NI world-model calls rather than NIH, cutting the dominant online cost.
- New DynaGoal benchmark tests planning under moving goals, deadlines, and 0 to 800 ms response delays.
- Scores 73.83% on DynaGoal-Shift and 87.67% on DynaGoal-Deadline, beating INTACT on both.
- Repository currently has docs and figures only; code release promised with no date.
Fast-JEPA reduces the cost of planning with a latent world model by changing how candidate action sequences are evaluated. Across four control benchmarks, the framework reports 96.50% average success with 10.88 ms planning latency, compared with 95.58% and 26.19 ms for the search-based INTACT controller. The reported 2.41× speedup addresses a central scaling problem: conventional planners predict every intermediate state for every candidate, refinement round, and time step.
A latent world model predicts compact representations of future observations, avoiding the cost of generating full images. In a joint-embedding predictive architecture, or JEPA, the planner encodes the current observation and goal, predicts future latent states under proposed actions, and scores candidates by their distance from the goal embedding.
One proposal, direct endpoint scores
Fast-JEPA first constructs a goal-directed proposal one action chunk at a time. An action generator chooses each chunk from the current latent state and goal. The shared forward model then receives the original state z_t and the accumulated action sequence. Its prediction guides construction of the next chunk. Anchoring every query to z_t limits error accumulation from recursively predicted states.
Cross-entropy method refinement samples alternative plans around the completed proposal. CEM repeatedly scores a population of candidates and concentrates later samples around the strongest candidates. Fast-JEPA evaluates each complete sequence with a direct prediction of its terminal latent state, then compares that state with the goal embedding. Each refinement round batches its candidates, and the pool retains both the original proposal and the current incumbent. This retention guarantees that refinement cannot worsen the world model's predicted objective, though execution in the environment can still produce a different outcome.