Google Research's TabFM Beats XGBoost on 51 Datasets Without Any Training

For decades, the default playbook for tabular ML has been the same: pick XGBoost or a random forest, spend days on feature engineering and hyperparameter tuning, and repeat for every new dataset. Google Research just shipped a model that skips all of that. TabFM is a foundation model for tabular classification and regression that produces predictions on a brand-new table in a single forward pass, with zero training required.

The problem with the old playbook

The lifecycle of deploying traditional models presents a significant bottleneck. Fitting an XGBoost model to a new dataset is not merely a matter of a single .fit() step; it invariably requires tedious manual effort. Data scientists must invest hours into hyperparameter optimization and domain-specific feature engineering just to get reliable signal from raw data. That cost multiplies across every new dataset, every new project, every new team.

Recent advances in large language models changed how we interact with novel tasks, demonstrating the remarkable power of zero-shot prediction through in-context learning (ICL). This technique lets a pretrained model learn a new task by providing examples and instructions in the input context, without updating any underlying model weights. TabFM applies that same idea to structured, tabular data.

How it actually works

Instead of undergoing a traditional training phase for each new task, TabFM takes the entire dataset , comprising both the historical training examples and the target testing rows , as a single unified prompt. The model learns to interpret the relationships between columns and rows directly from this context at inference time.

Tables are fundamentally different from text: they are two-dimensional and orderless. Swapping two rows or two columns does not change the meaning of the data, but standard transformers would treat that as a completely different input. TabFM synthesizes the strengths of architectures like TabPFN and TabICL into a novel hybrid design built around three mechanisms: