Skip to main content
Back

Alibaba's Baguan-TS Bridges Two AI Forecasting Camps With 22M Parameters

Alibaba's Baguan-TS introduces a 3D Transformer that unifies raw-sequence learning with in-context adaptation, beating Chronos, TimesFM and TabPFN-TS on covariate-aware forecasting.

Pro

Paper, Full Access
  • Alibaba DAMO Academy released Baguan-TS, a 22.4M-param 3D Transformer for covariate-aware time series forecasting.
  • Attends jointly over temporal, variable, and context axes for gradient-free in-context adaptation on raw sequences.
  • Beats TabPFN-TS, Chronos-Bolt, TimesFM-2.5 on 30-task fev-bench-cov, cutting SQL by 4.8%.
  • Y-space retrieval finds nearest historical windows by target similarity, no hand-crafted features needed.
  • Context-overfitting strategy fixes oversmoothing by forcing the model to match template motifs from context.
  • 2D+3D ensemble mode balances under- and over-confident predictions for near-perfect probabilistic calibration.

Time series forecasting has been split between two camps that rarely talk to each other. End-to-end sequence models like Chronos and TimesFM chew through raw histories but cannot adapt at inference. In-context learning approaches like TabPFN-TS adapt on the fly, but only after you flatten each series into hand-crafted tabular features. A team from Alibaba's DAMO Academy proposes a bridge with Baguan-TS, a 22.4M-parameter foundation model that runs in-context learning directly on raw multivariate sequences.

Baguan-TS treats forecasting as an episodic task where the model receives a small support set of similar historical windows alongside the query and adapts gradient-free. A 3D Transformer attends jointly over temporal, variable, and context axes, working out which historical episodes matter for the current forecast without any feature engineering.

Comparison of three forecasting paradigms: end-to-end sequence models, tabular ICL, and Baguan-TS unified approach

Attention across three axes at once

Where standard time series transformers flatten variables and time into a single sequence, Baguan-TS keeps a structured tensor of shape (contexts x temporal patches x variables x embedding). Each transformer block runs three specialized attention passes in sequence:

  • Temporal attention with Rotary Position Embeddings, capturing relative distances and periodicity within each series.
  • Variable attention augmented with learnable per-variable embeddings, modeling cross-covariate correlations at each time step.
  • Context attention without positional encodings, since the support set has no natural ordering, purely aggregating information across episodes.