Alibaba's Baguan-TS Bridges Two AI Forecasting Camps With 22M Parameters
Alibaba's Baguan-TS introduces a 3D Transformer that unifies raw-sequence learning with in-context adaptation, beating Chronos, TimesFM and TabPFN-TS on covariate-aware forecasting.
ProTakeaways
- Alibaba DAMO Academy released Baguan-TS, a 22.4M-param 3D Transformer for covariate-aware time series forecasting.
- Attends jointly over temporal, variable, and context axes for gradient-free in-context adaptation on raw sequences.
- Beats TabPFN-TS, Chronos-Bolt, TimesFM-2.5 on 30-task fev-bench-cov, cutting SQL by 4.8%.
- Y-space retrieval finds nearest historical windows by target similarity, no hand-crafted features needed.
- Context-overfitting strategy fixes oversmoothing by forcing the model to match template motifs from context.
- 2D+3D ensemble mode balances under- and over-confident predictions for near-perfect probabilistic calibration.
Time series forecasting has been split between two camps that rarely talk to each other. End-to-end sequence models like Chronos and TimesFM chew through raw histories but cannot adapt at inference. In-context learning approaches like TabPFN-TS adapt on the fly, but only after you flatten each series into hand-crafted tabular features. A team from Alibaba's DAMO Academy proposes a bridge with Baguan-TS, a 22.4M-parameter foundation model that runs in-context learning directly on raw multivariate sequences.
Baguan-TS treats forecasting as an episodic task where the model receives a small support set of similar historical windows alongside the query and adapts gradient-free. A 3D Transformer attends jointly over temporal, variable, and context axes, working out which historical episodes matter for the current forecast without any feature engineering.

Attention across three axes at once
Where standard time series transformers flatten variables and time into a single sequence, Baguan-TS keeps a structured tensor of shape (contexts x temporal patches x variables x embedding). Each transformer block runs three specialized attention passes in sequence:
- Temporal attention with Rotary Position Embeddings, capturing relative distances and periodicity within each series.
- Variable attention augmented with learnable per-variable embeddings, modeling cross-covariate correlations at each time step.
- Context attention without positional encodings, since the support set has no natural ordering, purely aggregating information across episodes.