Google's TabFM Beats XGBoost on Tabular Data Without Training
Google's TabFM does zero-shot tabular classification and regression in one forward pass, no training or hyperparameter tuning required.
Takeaways
- Google releases TabFM 1.0.0 -- a zero-shot foundation model for tabular classification and regression, no training or tuning required.
- Single forward pass inference: Training rows are passed as context; the model predicts on new rows without updating any weights.
- Beats tuned tree ensembles: Evaluated on 51 datasets via TabArena Elo scores, TabFM outperforms heavily-tuned XGBoost and similar baselines in zero-shot mode.
- Trained on hundreds of millions of synthetic datasets generated by structural causal models -- no real-world proprietary data used.
- Hard limits to know: Max 10 output classes, ~500 feature ceiling, non-commercial weights license, no built-in explainability.
- BigQuery integration coming: Google announced an
AI.PREDICTSQL command for BigQuery; weights available now on Hugging Face.
For the past fifteen years, if you needed to predict something from a spreadsheet, the answer was almost always a gradient-boosted tree. XGBoost, LightGBM, CatBoost -- these dominated Kaggle leaderboards and production pipelines alike. The catch: you paid for that performance with hours of feature engineering, hyperparameter search, and a fresh training run every time you touched a new dataset. That reign came with an endless cycle of feature engineering, hyperparameter search, and per-dataset retraining. Google Research just shipped something that challenges that assumption directly.
TabFM 1.0.0 is a zero-shot foundation model for tabular classification and regression, with public code on GitHub and pretrained weights on Hugging Face. The core idea: instead of training a model on your dataset, you hand the model your labeled rows as context, and it predicts on new rows in a single forward pass -- no weight updates, no tuning loops.
The trick that makes it work
The key concept here is in-context learning (ICL) -- a technique borrowed from large language models where a pretrained model learns a new task purely from examples provided in its input, without any gradient updates. TabFM uses in-context learning so teams can pass training examples and target rows as one prompt, avoiding per-dataset fine-tuning and hyperparameter search. The challenge is that tables aren't sequences: swapping two rows or two columns doesn't change the meaning of the data, so you can't just tokenize them like text.
The architecture combines TabPFN-style row/column attention with TabICL-style in-context learning. Concretely, the model has three stages:
- Column attention (Set Transformer): Each cell is embedded using Fourier features and a per-group linear projection, then the model attends across both rows and columns simultaneously to learn feature interactions -- the work that would otherwise require manual feature crafting.