Skip to main content
Back

Seoul National University's Deep QP Safety Filter Trains Robots Without Crashing

Seoul National University researchers built a safety filter for reinforcement learning that works on black-box systems, using two neural networks and quadratic programming.

Pro

  • New safety filter learns from data alone, no dynamics model needed for black-box systems.
  • Combines Hamilton-Jacobi reachability with model-free learning via two neural networks.
  • Contraction-based Bellman losses guarantee convergence to the correct viscosity solution.
  • Zero QP infeasibility on smooth benchmarks, 0.23% on hybrid Hopper system.
  • Task-agnostic: one trained filter reusable across RL tasks via scalar aggressiveness knob.
  • Code available at GitHub, paper at arXiv 2601.21297.

Training a reinforcement learning agent on physical hardware has an ugly reality: before the policy learns to keep a robot upright or stay inside a lane, it crashes. A lot. The standard fix is a safety filter that intercepts dangerous commands and nudges them into safe ones, but nearly every existing filter needs an accurate mathematical model of the system, which is rarely available in practice.

Two researchers at Seoul National University are trying to remove that assumption entirely. Their new method, called Deep QP Safety Filter, learns a reachability-based safety layer directly from transition data with zero prior knowledge of the dynamics. According to the paper, the filter converges to the correct analytical solution in the exact setting and substantially drops training failures when paired with PPO on unstable benchmark tasks.

Why current safety filters break on real hardware

Quadratic-Program (QP) based safety filters, especially those built on Control Barrier Functions, have become the dominant approach for safety-critical control. They produce smoother interventions than switching or bang-bang filters and have been deployed on hard platforms like bipedal robots. Reliability deteriorates as model uncertainty grows, and recent learning-based variants still presume partial access to the dynamics, which is unavailable for black-box systems. That leaves a practical gap between elegant theory and deployment on real platforms with unknown or changing dynamics.