TU Munich's Frozen-PINN Solves Equations 10,000x Faster Without Gradient Descent
A new physics-informed neural network throws out gradient descent entirely, delivering PDE solutions up to 5 orders of magnitude faster on plain CPUs.
Takeaways
- Frozen-PINN replaces gradient descent with random feature sampling plus classical ODE solvers, accepted as ICLR 2026 oral
- Achieves 4-5 orders of magnitude faster training and often higher accuracy on nine PDE benchmarks
- Enforces temporal causality by construction via space-time separation, unlike standard PINNs
- Two sampling variants: ELM for smooth solutions, SWIM for shocks and steep gradients
- SVD layer compresses last hidden layer up to 20x and accelerates training up to 75x
- Open-source code at GitLab repository; CPU-only training beats GPU-trained baselines
Physics-informed neural networks have long promised a mesh-free way to solve partial differential equations, but the reality has been messier. Training them is slow, unstable, and often fails on problems that classical solvers handle in seconds. A new paper from researchers at TU Munich and collaborators, accepted as an oral at ICLR 2026, argues the fix is to stop training with gradient descent altogether.
Their method, called Frozen-PINN, freezes the hidden layer weights after random sampling, then evolves the last-layer coefficients through time using a classical ODE solver. The approach achieves up to 4-5 orders of magnitude faster training and several orders of magnitude higher accuracy across diverse PDE benchmarks, all on CPUs.
Why PINNs Have Been Stuck
The authors identify two root causes behind chronic underperformance. The loss function combines PDE residuals, boundary conditions, and initial conditions into a single high-dimensional non-convex objective with conflicting terms. More subtly, most PINNs treat time as just another spatial dimension in the input layer.
That second point matters more than it sounds. Time-dependent PDEs are inherently Markovian, meaning the solution at each step depends only on the previous one. When you flatten time into the input, the neural bases span the entire space-time domain and the network has to learn everything at once. This is why standard PINNs famously collapse on the linear advection equation once the wave speed gets above about 40.
The Space-Time Separation Trick
Frozen-PINN parameterizes the solution as a sum of spatial basis functions multiplied by time-dependent coefficients: