Transformer Neural Operators Finally Get Math Guarantees for Chaotic Forecasting

Data-driven weather models and learned forecasters have posted competitive results against physics-based systems, yet their mathematical guarantees remain limited. In a new preprint, Edoardo Calvello, Elizabeth Carlson, Nikola Kovachki, Michael N. Manta, and Andrew M. Stuart present what they describe as the first universal approximation theorems for purely data-driven smoothing and forecasting of continuous-time dynamical systems. Experiments on three chaotic benchmarks connect the theory to working Transformer Neural Operator models.

From time steps to trajectories

Physics-based data assimilation methods, including the Kalman filter and its variants, have decades of convergence and stability analysis. Learned alternatives are usually evaluated through benchmark accuracy. The new work asks a foundational question: under which conditions can a neural architecture represent the maps required to reconstruct hidden states and predict future dynamics?

Smoothing reconstructs a system’s hidden state trajectory from incomplete or noisy observations collected over an interval. Forecasting maps available state or observation histories to future trajectories. The paper formulates both tasks in continuous time, treating inputs and outputs as functions rather than fixed-length vectors.

This formulation turns smoothing and forecasting into operator-learning problems. An operator maps one function to another, such as an observed trajectory to a complete state trajectory. Neural operators approximate these function-space mappings and can represent solutions across temporal or spatial coordinates without reducing the problem to one finite-dimensional regression.