Skip to main content
Back

USTC and Alibaba's FAR Generates Images in 10 Steps by Ditching Raster-Scan

A new autoregressive image model builds pictures from low to high frequencies using continuous tokens, generating full images in just 10 steps.

Pro

  • FAR predicts images from low to high frequencies instead of raster-scan token order, generating full images in 10 steps.
  • Uses continuous tokens with a diffusion MLP head, avoiding lossy vector quantization used by VQGAN and VAR.
  • FAR-H hits FID 3.21 and IS 300.6 on ImageNet 256 with 812M parameters and 10 inference steps.
  • Text-to-image variant trained on 7.8M pairs in 24 A100-days beats DALL-E and CogView2 on MS-COCO FID.
  • Key tricks: sine-curve loss reweighting, 70 percent masking at low frequencies, adaptive diffusion sampling steps.
  • Code and checkpoints available at github.com/yuhuUSTC/FAR; project page at yuhuustc.github.io.

Image generation models borrowed a lot from language models, and it shows. The dominant recipe for autoregressive image generation quantizes an image into discrete tokens and predicts them one by one in raster-scan order, exactly the way GPT-style models predict text. A new paper from researchers at USTC and Alibaba's DAMO Academy argues that this ordering mismatches the actual structure of images, and proposes a different regression direction to fix it.

The method, called FAR (Frequency progressive AutoRegressive), generates images by predicting successively higher frequency bands instead of successive tokens. Code and ImageNet checkpoints are already released, and the paper reports competitive quality against MAR and VAR while needing only 10 autoregressive steps.

Why raster-scan is the wrong shape for images

Text is discrete, causal, and one-dimensional, while images are continuous, non-causal, and two-dimensional. Flattening a 2D image into a 1D sequence and predicting left-to-right, top-to-bottom breaks the natural spatial locality: the first token of a new row ends up conditioned on the far end of the previous row rather than on the pixels right above it. Vector quantization, the other half of the standard recipe, also throws away information because it forces a continuous latent onto a finite codebook.

The authors frame the issue as two design choices worth rethinking: the tokenizer format (discrete vs continuous) and the regression direction (what order the model generates in). Recent work like MAR predicts random sets of masked tokens, and VAR predicts progressively larger scales using residual quantization, but both still inherit awkward assumptions from the language-modeling playbook.