USI Lugano Reveals Why Longer Context Windows Actually Help Forecasting Models
A new paper proves that long context windows in time series models aren't just about memory -- they're solving a hidden identification problem that changes how we should design forecasters.
Takeaways
- New theory: Long context windows in global time series models serve two roles -- identifying the data-generating process (GPI) and making the actual forecast (CF).
- Proven result: Even if a process has memory length P, a window strictly larger than P is mathematically necessary to reach minimum error.
- Foundation model implication: More diverse training data means more context is needed at inference, creating a hidden compute cost for generalist models vs. domain-specific ones.
- Architecture insight: Decoupling GPI (a context encoder) from CF (a forecasting head) improves scalability without sacrificing accuracy, validated on PatchTST, TSMixer, ModernTCN, and LRU.
- Benchmarks: Experiments run on LargeST-SD (traffic), CEREn (energy), ClmUS-T/R (climate), and synthetic NAR processes of orders 1-6.
- No code yet: The paper is theoretical and empirical; no public repository has been released alongside it.
Every practitioner who has tuned a time series model knows the ritual: increase the lookback window, watch the validation loss drop, and accept it as a win. The standard explanation is that longer windows capture long-range dependencies. But a new paper from researchers at USI Lugano and Politecnico di Milano argues this explanation is incomplete -- and the real reason has direct consequences for how you should build and scale forecasting systems.
Modern deep learning models for forecasting groups of time series rely on increasingly longer observation windows, yet the benefit of increasing the window size is often simply attributed to capturing long-range dependencies, and broader discussion on how global forecasting models leverage input observations has been limited. The paper sets out to fix that.
Two jobs, one model
The core argument is that forecasting groups of time series involves two distinct objectives: generative process identification (GPI), i.e., inferring the specific process generating the input sequence, and conditional forecasting (CF), i.e., predicting future values given input observations. Most architectures blend these two tasks together without ever naming them -- but the distinction turns out to matter enormously.
Think of it this way: a global model trained on thousands of sensor streams, retail SKUs, or weather stations is not just predicting the future. It is first asking which kind of series am I looking at right now? -- and then making a forecast conditional on that answer. GPI is the identification step; CF is the prediction step.
Optimal predictions can be interpreted as an average over plausible data-generating processes, weighted by their likelihood given the input window. This suggests another explanation for the benefits of long context windows: they reduce the uncertainty about which specific process is generating the input time series during operation. In other words, you need more data not because the signal is far away in time, but because you need enough observations to confidently identify