Skip to main content
Back

Google's TabFM Beats XGBoost on Tabular Data Without Training

Google's TabFM does zero-shot tabular classification and regression in one forward pass, no training or hyperparameter tuning required.

Model, Full Access
  • Google releases TabFM 1.0.0 -- a zero-shot foundation model for tabular classification and regression, no training or tuning required.
  • Single forward pass inference: Training rows are passed as context; the model predicts on new rows without updating any weights.
  • Beats tuned tree ensembles: Evaluated on 51 datasets via TabArena Elo scores, TabFM outperforms heavily-tuned XGBoost and similar baselines in zero-shot mode.
  • Trained on hundreds of millions of synthetic datasets generated by structural causal models -- no real-world proprietary data used.
  • Hard limits to know: Max 10 output classes, ~500 feature ceiling, non-commercial weights license, no built-in explainability.
  • BigQuery integration coming: Google announced an AI.PREDICT SQL command for BigQuery; weights available now on Hugging Face.

For the past fifteen years, if you needed to predict something from a spreadsheet, the answer was almost always a gradient-boosted tree. XGBoost, LightGBM, CatBoost -- these dominated Kaggle leaderboards and production pipelines alike. The catch: you paid for that performance with hours of feature engineering, hyperparameter search, and a fresh training run every time you touched a new dataset. That reign came with an endless cycle of feature engineering, hyperparameter search, and per-dataset retraining. Google Research just shipped something that challenges that assumption directly.

TabFM 1.0.0 is a zero-shot foundation model for tabular classification and regression, with public code on GitHub and pretrained weights on Hugging Face. The core idea: instead of training a model on your dataset, you hand the model your labeled rows as context, and it predicts on new rows in a single forward pass -- no weight updates, no tuning loops.

The trick that makes it work

The key concept here is in-context learning (ICL) -- a technique borrowed from large language models where a pretrained model learns a new task purely from examples provided in its input, without any gradient updates. TabFM uses in-context learning so teams can pass training examples and target rows as one prompt, avoiding per-dataset fine-tuning and hyperparameter search. The challenge is that tables aren't sequences: swapping two rows or two columns doesn't change the meaning of the data, so you can't just tokenize them like text.

The architecture combines TabPFN-style row/column attention with TabICL-style in-context learning. Concretely, the model has three stages:

  1. Column attention (Set Transformer): Each cell is embedded using Fourier features and a per-group linear projection, then the model attends across both rows and columns simultaneously to learn feature interactions -- the work that would otherwise require manual feature crafting.