Skip to main content
Back

Meta's TRIBE v2 Builds a Digital Twin of the Human Brain

Meta FAIR open-sources a transformer that maps video, audio, and text onto whole-brain fMRI responses across 720 subjects with several-fold accuracy gains.

Pro

Repo, Full Access
  • Meta FAIR released TRIBE v2, a multimodal transformer that predicts whole-brain fMRI responses to video, audio, and text.
  • Trained on 1,000+ hours of fMRI from 720 subjects, spanning eight datasets and naturalistic plus controlled paradigms.
  • Combines frozen LLaMA 3.2, V-JEPA2, and Wav2Vec-BERT with an 8-layer temporal transformer over a 100-second window.
  • Delivers zero-shot predictions for new subjects, tasks, and languages, several-fold better than linear encoders.
  • Weights, code, and paper on Hugging Face and arXiv, under CC BY-NC 4.0.
  • Interactive demo lets you feed clips and view predicted cortical activation.

Meta's FAIR team has released TRIBE v2, a foundation model that predicts how the human brain responds to arbitrary video, audio, and text stimuli. Most brain-encoding work trains one narrow model per cognitive task. This one treats the cortex as a single prediction target and ships weights, code, a paper, and an interactive demo in one drop.

Built by researchers at Meta FAIR and École Normale Supérieure, TRIBE v2 is a transformer that ingests multimodal stimuli (videos, audio tracks, text transcripts) and predicts the corresponding fMRI responses across the whole brain. The team pitches it as a foundation model that predicts how a human brain responds to almost any sight or sound, aimed at reducing the need for fresh scans in neuroscience and clinical research.

The digital twin pitch

The core problem TRIBE v2 attacks is fragmentation. Researchers typically map specific cognitive functions to isolated brain regions, like motion to area V5 or faces to the fusiform gyrus, using models tailored to narrow experimental paradigms. The resulting landscape lacks a unified framework for how the brain integrates multisensory information. TRIBE v2 is one model that generalizes across dozens of paradigms instead.

Building on Meta's Algonauts 2025 award-winning architecture, TRIBE v2 draws on 500+ hours of fMRI recordings from 700+ people to create a digital twin of neural activity. It enables zero-shot predictions for new subjects, languages, and tasks, consistently outperforming standard modeling approaches. The accompanying paper reports an even larger unified corpus of more than 1,000 hours of fMRI across 720 subjects.

Under the hood

The architecture is a three-stage pipeline: modality-specific encoding, temporal integration, then projection onto the cortex. The Hugging Face model card describes TRIBE v2 as combining LLaMA 3.2 for text, V-JEPA2 for video, and Wav2Vec-BERT for audio inside a single transformer. Three specialized encoders handle each sensory modality independently, and their weights are frozen. TRIBE v2 never retrains them.