Eindhoven Researchers Fix AI Controllers That Break When Leaving the Simulator
A new framework yields controllers with finite-sample deployment guarantees under distribution shift, tying both concentration and robustness to a single closed-loop operator norm.
ProTakeaways
- New framework combines PAC-Bayes bounds with Wasserstein distributionally robust optimization for learning-based controllers.
- Handles unbounded losses and sim-to-real distribution shift, both blind spots in prior PAC-Bayes control work.
- Uses System Level Synthesis to tie concentration and robustness to one operator norm of the closed-loop map.
- For LTI systems, yields a tractable, differentiable posterior optimization problem with high-probability safety certificates.
- Double-integrator experiments show vanilla PAC-Bayes bound is violated under shift while robust version holds.
- Limitations: linear time-invariant only, norm losses only, no model uncertainty yet, no public code released.
Learning-based controllers are notoriously brittle when the training environment does not perfectly match the real world. A new paper from researchers at Eindhoven University of Technology attacks this sim-to-real gap by combining PAC-Bayesian generalization bounds with Wasserstein distributionally robust optimization to certify controller performance even when the deployment distribution drifts from the training one.
Why existing PAC-Bayes control breaks on hardware
PAC-Bayes gives high-probability generalization guarantees for a distribution over hypotheses rather than a single one. It has produced the first non-vacuous generalization bounds for deep neural networks, and researchers have started porting the machinery to control problems like robot policies that must generalize to unseen environments.
The trouble is that the standard formulation makes two assumptions that rarely hold for dynamical systems:
- Losses are bounded, which is natural in classification but forces awkward saturation tricks in control.
- Training and deployment distributions match, ignoring the disturbances and modeling errors that show up on real hardware.
Prior PAC-Bayes control work that addresses distribution mismatch has used information-theoretic f-divergences, which inflate the complexity term with a penalty independent of the controller. Every policy pays the same tax for robustness, whether it deserves it or not.
One operator norm to rule them all
The authors swap the f-divergence for a type-1 Wasserstein distance and combine it with a recent PAC-Bayes bound for unbounded, hypothesis-dependent sub-Gaussian losses. Through Kantorovich-Rubinstein duality, the Wasserstein robustness penalty gets tied explicitly to the controller-dependent Lipschitz constant of the closed-loop system. Controllers whose closed-loop behavior is smoother with respect to disturbance perturbations pay a smaller robustness tax.