TabFM: Zero-shot tabular ML without the boilerplate

Google Research has released TabFM, a foundation model that can predict on any tabular dataset without training, hyperparameter tuning, or feature engineering. It's available now on Hugging Face and GitHub, and will be integrated into BigQuery in the coming weeks.

How it works

TabFM reframes tabular prediction as an in-context learning (ICL) problem. Instead of training on each new dataset, the model takes the entire table—both training rows and test rows—as a single prompt. It learns column relationships and row patterns directly from that context at inference time.

To handle the two-dimensional, orderless nature of tables, TabFM uses a hybrid architecture combining ideas from TabPFN and TabICL:

  1. Alternating row and column attention: The model attends across both features (columns) and examples (rows) in multiple layers, capturing complex interactions without manual feature engineering.
  2. Row compression: Each row's contextualized representation is compressed into a single dense vector.
  3. In-context learning: A Transformer operates on the sequence of compressed embeddings, keeping computation efficient even for large datasets.

This design avoids the quadratic cost of full attention over the raw table, making it scalable.

Training on synthetic data at scale

TabFM was trained entirely on hundreds of millions of synthetic datasets generated using structural causal models (SCMs) with random functions. This approach was necessary because high-quality tabular datasets are scarce—industrial tables are proprietary and sensitive. The synthetic data captures diverse distributions and feature relationships, enabling the model to generalize to real-world tables.

Benchmarks: Outperforming XGBoost without tuning

Google evaluated TabFM on TabArena, a living benchmark that uses Elo scores from head-to-head matchups across 38 classification and 13 regression datasets (700 to 150,000 samples).

Two configurations were tested:

  • TabFM: Out-of-the-box, single forward pass, no tuning.
  • TabFM-Ensemble: Adds cross features, SVD features, and a 32-way ensemble with non-negative least squares weighting. Classification tasks also use Platt scaling.

Results show TabFM consistently beats heavily tuned baselines like XGBoost, AdaBoost, and random forests. The ensemble variant pushes performance further. Full per-fold metrics and win rates are on GitHub.

Code example: Using TabFM

from tabfm import TabFM
import pandas as pd

# Load your data
train_df = pd.read_csv('train.csv')
test_df = pd.read_csv('test.csv')

# Initialize model (zero-shot, no training)
model = TabFM()

# Predict in one forward pass
predictions = model.predict(train_df, test_df, target_column='label')

No .fit(), no grid search, no feature engineering. The model handles everything.

Why it matters

Traditional tabular ML workflows are a bottleneck. Data scientists spend hours on hyperparameter tuning and feature engineering for each new dataset. TabFM eliminates that entirely. For teams that frequently work with new tables, this could save days per project.

The integration with BigQuery via AI.PREDICT SQL command means even non-ML engineers can run advanced classification and regression directly in their data warehouse.

Limitations

TabFM is trained on synthetic data, which may not perfectly match all real-world distributions. The paper doesn't specify model size or inference latency on large tables. Also, the zero-shot approach may struggle with datasets that have unusual feature types or missing data patterns not seen during training.

Next steps

Try TabFM on your own datasets. The model is available on Hugging Face and GitHub. If you use BigQuery, watch for the AI.PREDICT integration in the coming weeks. For teams stuck with XGBoost tuning, this could be a genuine time-saver.