GOTabPFN Unlocks Biomedical AI for Datasets With 20,000 Features and 100 Samples
GOTabPFN uses graph-based feature ordering and neural compression to squeeze high-dimensional biomedical data into TabPFN's token budget without retraining the model.
- GOTabPFN extends TabPFN to handle high-dimensional, low-sample-size (HDLSS) tabular data without retraining the model.
- The core problem: TabPFN's attention mechanism scales quadratically with features, making it impractical beyond ~500 features.
- GO-LR reorders features using a graph-based algorithm so that correlated features sit adjacent, enabling better compression.
- NSC (Neuro-Inspired Subunit Compression) pools groups of adjacent ordered features into meta-features, shrinking the input to fit TabPFN's token budget.
- Tested on 8 biomedical HDLSS benchmarks (Arcene, Colon, GLI-85, Lung, etc.), GOTabPFN improves both accuracy and stability over baseline TabPFN variants.
- No code repository released yet; the method works as an inference-time preprocessing layer on top of the existing TabPFN library.
GOTabPFN is a new wrapper around TabPFN, the popular tabular foundation model, that finally makes it practical for the kind of data that shows up most often in biology and medicine: datasets with thousands of features and only a handful of samples. The paper introduces two key ideas -- a graph-guided feature ordering algorithm and a neural compression unit -- that together compress wide, sparse tables into the compact token representations TabPFN was designed to consume.
The wall TabPFN hits in the real world
TabPFN is a prior-fitted network designed for tabular data that approximates Bayesian inference via in-context learning. Unlike traditional models, it does not require any additional training when presented with a new dataset -- it directly takes a set of labeled training samples along with test samples and produces predictions in a single forward pass. That zero-retraining property is what makes it so appealing.
But there is a hard constraint baked into the architecture. Attention that is both row-wise and feature-wise dense introduces unfavorable scaling with table width, as the computational cost is quadratic with respect to the number of samples and the number of features. In practice, this means TabPFN has a strict token budget. TabPFN v2 significantly outperforms tree-based and neural network competitors on small-to-medium datasets with up to 10K samples and 500 features. Go beyond those limits and performance degrades fast.
This matters enormously for a class of problems called HDLSS -- High-Dimensional, Low-Sample Size. HDLSS data presents unique challenges in statistical modeling and machine learning, particularly in fields such as healthcare and genomics. The number of features, or variables, far exceeds the number of available samples or observations, making traditional statistical techniques less effective and often leading to overfitting and poor generalizability. A classic example: a study might involve expression levels of 20,000 genes across only 100 patients. TabPFN, as designed, simply cannot handle that ratio.
This story is for Pro members
You've reached the end of the free preview. Upgrade to AlphaSignal Pro to read the full article - and everything else behind the paywall.
Comments
