Oxford's BSA-TNP Runs 1M-Point Weather Inference in Under a Minute
A new Neural Process architecture from Oxford and Imperial ditches inducing points, keeps translation invariance, and runs 1M test points in under a minute.
Takeaways
- BSA-TNP is a new Transformer Neural Process for scalable spatiotemporal inference from Oxford, Imperial, and Bristol.
- Runs inference on 1M test points and 100K context points in under a minute on a single 24GB GPU.
- Introduces Kernel Regression Blocks, group-invariant attention biases, and memory-efficient Biased Scan Attention in JAX.
- Beats ConvCNP, TNP-D, and PT-TE-TNP on 2D GPs, SIR epidemics, ERA5 weather, and Beijing air quality.
- Translation invariance lets it train on small domains and generalize to shifted, scaled, or multi-resolution targets.
- Code released in the
dl4bipackage; paper to appear at AISTATS 2026.
A team from the University of Oxford, Imperial College London, and the University of Bristol has released BSA-TNP, a Transformer Neural Process built for spatiotemporal problems like weather forecasting, disease modeling, and satellite-scale inference. The headline claim: the model matches or beats state-of-the-art neural processes while running inference on over 1M test points and 100K context points in under a minute on a single 24GB GPU.
Why Neural Processes Keep Hitting a Wall
Neural Processes (NPs) are meta-learned models that predict a full posterior distribution over functions given a set of observed context points. Think of them as a neural drop-in for Gaussian Processes: feed in some measurements at known locations, ask for predictions plus uncertainty at new ones. They now power foundation models like Aardvark for weather forecasting and are being deployed in geology, epidemiology, and robotics.
Scale is the sticking point. Standard Transformer Neural Processes have O(n²) time and space complexity, made worse by variants that use attention bias, which requires a second matrix of the same size. To cope, most recent NPs rely on inducing points, a small set of learned latent tokens that summarize context. This works, but it forces you to pick the number and location of those latents, and it tends to oversmooth predictions or bottleneck accuracy.
BSA-TNP takes the opposite bet: keep full attention over context points, but make the attention itself memory-cheap and inject strong priors through bias functions.

Three Ingredients That Make It Work
The architecture is a stack of embedding layers, Kernel Regression Blocks, and a prediction head. Three components do the heavy lifting:
- Kernel Regression Blocks (KRBlocks): a simple, stackable transformer block that shares weights for query and key updates and supports an arbitrary number of attention biases derived from subsets of input features, e.g. space, time, and fixed effects (observed covariates) such as elevation, weather, or categorical metadata. Context points attend only to other context points, and test points attend only to context points.