Skip to main content
Back

HYDRA Beats FLUX With a Single ViT That Paints and Understands

A new pure vision transformer tokenizer bridges the split between image understanding and generation, setting fresh state of the art scores across benchmarks.

Pro

Paper, Full Access
  • HYDRA introduces a pure ViT tokenizer that transitions from generation features to semantic features via a bottleneck
  • Achieves state of the art reconstruction with rFID 0.08 on ImageNet-1K, beating FLUX-VAE and specialized tokenizers
  • 7B model hits GenEval 0.86 and DPG-Bench 86.4, surpassing the 12B FLUX.1 [Dev] on compositional generation
  • Outperforms other native unified multimodal models by roughly 10 points on eight understanding benchmarks
  • Joint training of generation and understanding shown to be mutually reinforcing rather than a zero sum tradeoff
  • Paper available on arXiv; no public code or model weights released yet

Unified multimodal models keep hitting the same wall. The features you need to understand an image, high level semantic abstractions, are almost the opposite of the features you need to generate one, dense pixel level primitives. Most systems paper over this by bolting together two encoders, stacking a semantic model on top of a VAE, or quantizing everything into discrete tokens. Each workaround pays a tax somewhere. HYDRA proposes a cleaner answer: a single Vision Transformer that transitions gradually from generation features to understanding features, mediated by a bottleneck that squeezes out the noise between them.

Why one encoder cannot serve two masters

Native unified multimodal models (UMMs) try to do text prediction and image synthesis inside one shared parameter space. Rational unification runs into a representational divergence between two inverse tasks with conflicting demands. Understanding wants high-level semantic abstractions. Generation wants compact structural primitives for fine-grained synthesis. Every existing design trades something off:

  • Decoupled encoders (Janus, BAGEL) run a VAE and a semantic encoder in parallel, but the two feature streams never really talk.
  • Sequential encoders (Show-o2) stack a representation encoder on top of a VAE, so the VAE's compression bottleneck destroys detail before semantics can be extracted.
  • Shared encoders (UniTok, Harmon) use one representation for both jobs, but the objectives fight each other during training.

The result across the field is an unavoidable trade-off: sacrifice generative fidelity, lose semantic alignment, or fail to converge on a shared representation.

A ViT that grows up from painter to critic

The core insight is worth stating directly: a compact feature space capable of reconstructing input data can serve as a robust foundation for semantic understanding. If you can rebuild the pixels from a compressed feature, that feature already captures the structural vocabulary that semantics can be built on top of.