Skip to main content
Back

Hugging Face's Carbon-X Teaches a 14.8B Chat Model to Read DNA

Hugging Face's Carbon-X teaches a 14B Qwen3 model to speak DNA, matching GPT-6.1 Sol on genomics benchmarks at 1/1,800 the cost.

Model, Full Access
  • Hugging Face released Carbon-X, a 14.8B open DNA chat model built on Qwen3-14B under Apache 2.0.
  • Scores 66.7% average on five DNA benchmarks, matching GPT-6.1 Sol (67.5%) at roughly 1/1,800 the cost.
  • Uses a hybrid tokenizer: Qwen3 BPE for text, plus 5,515 DNA tokens where one token covers 6 base pairs.
  • Trained via 302B-token mid-training on 55% DNA, 20% English, 15% biology text, 10% instructions, then SFT and GRPO.
  • Fits on a single 80 GB GPU in bfloat16; served via vLLM or Transformers v5+ with trust_remote_code.
  • Weak on splice sites (29.8%), long DNA generation, and exact coordinates; not clinically validated.

Hugging Face has adapted a general-purpose chat model to process raw genetic sequences alongside natural language. Carbon-X model card describes a 14.8-billion-parameter, decoder-only model built from Qwen3-14B. Given DNA in a chat prompt, it can attempt variant-effect prediction, explain sequence function, classify regions, and predict subsequent bases.

Hugging Face Bio developed the model with the University of Naples Federico II and Zhongguancun Academy. The weights use the Apache 2.0 license and require roughly 30 GB in bfloat16 before inference overhead, allowing deployment on a single 80 GB accelerator.

DNA enters the token stream

Carbon-X processes DNA and text through the same decoder, the component that predicts each successive token. Earlier multimodal genomics systems commonly paired a dedicated sequence encoder with a language model; Carbon-X extends the language model's tokenizer and continues training the full network.

The tokenizer preserves Qwen3's byte-pair encoding for text and adds a 5,515-entry DNA extension. The documented entries include 4,096 possible six-base combinations, 1,364 shorter tail tokens, and three boundary markers. Full-length DNA tokens represent six base pairs, while tail tokens preserve sequences whose lengths are not divisible by six.

Runs containing at least six consecutive A, C, G, or T bases are automatically enclosed in <dna>...</dna> markers. This preprocessing lets developers submit ordinary chat messages while preserving exact sequence reconstruction after tokenization.

Strong scores, with caveats

Across five team-reported benchmarks covering variant effects, sequence understanding, and next-base prediction, Carbon-X reaches a simple average of 66.7% with Qwen's thinking mode disabled. Qwen3-14B scores 39.7% under the same aggregate calculation, a gap of about 27 percentage points. Carbon-X also records the highest reported scores in this comparison on BioReason Coding and NextBase.

Team-reported benchmark scores, in percent
Benchmark Carbon-X GPT-6.1 Sol Opus 5.5