Baidu's Unlimited-OCR Parses 200-Page PDFs With Flat Memory Usage

OCR models have a dirty secret: the longer the document, the slower and more memory-hungry they get. Every token the decoder generates adds to the KV cache (the running memory of keys and values that attention uses to look back at prior context), which means parsing a 40-page PDF is not just harder than a single page -- it is fundamentally different in cost. Baidu just shipped a model that breaks that assumption entirely.

A new model in the DeepSeek-OCR lineage

Unlimited-OCR is Baidu's open-source document-parsing model, built on top of DeepSeek-OCR and designed specifically to eliminate the memory wall that makes long-document OCR impractical. It is now officially supported in vLLM, with a dedicated recipe and Docker image. The model is free, MIT-licensed, and the weights are on Hugging Face.

Unlimited OCR is a 3B-parameter Mixture-of-Experts model, with only 500M parameters active at any given time. It builds on DeepSeek OCR via continue-training, not a from-scratch run. The research team continue-trained from the DeepSeek OCR checkpoint for 4,000 steps, froze the DeepEncoder, and trained only the decoder on about 2M document samples using 8x16 A800 GPUs.

The core problem: attention that never forgets

As the output sequence lengthens, the accumulated KV cache drives up memory consumption and progressively slows down generation -- in stark contrast to humans, who exhibit no such decline in efficiency during long-horizon copying tasks.

Standard attention stores a key and value for every token ever generated. Standard Multi-Head Attention stores a key and value for every token. As output length T grows, the cache grows with it -- memory and latency climb without bound. For a model trying to transcribe a 200-page book, this becomes a hard wall.

Reference Sliding Window Attention: the fix

The key innovation is Reference Sliding Window Attention (R-SWA) -- a new attention design that keeps the KV cache at a fixed size regardless of how long the output gets. Think of it like how a human copyist works: you glance at the source document, write a few words, and only need to remember the last sentence you wrote -- not every word since you started.

Each generated token attends to all reference tokens (visual tokens in OCR) and the preceding n output tokens (128 by default). Everything older is evicted. The cache becomes a fixed queue of size m + n, bounded by a constant. As T grows far beyond n, the cache ratio trends toward zero -- so memory stays flat and per-step latency stays flat.

The critical distinction from plain sliding window attention (SWA, which also limits the window but applies it uniformly) is what happens to the image tokens. Compared to standard full attention, R-SWA maintains a constant KV cache throughout decoding. Compared to vanilla SWA, it preserves visual token fidelity by excluding them from state transitions, thereby avoiding progressive blurring. The model always has full, unmodified access to the original document image -- only the generated text side gets the sliding window treatment.

Architecture under the hood

The DeepEncoder is the compression engine. It cascades a SAM-ViT under window attention with a CLIP-ViT under global attention. At the bridge, it applies 16x token compression -- a 1024x1024 PDF image becomes just 256 visual tokens. That tiny reference footprint is what makes the constant-cache design practical: the fixed part of the KV cache is small.

The model supports two inference modes:

• Gundam mode -- dynamic resolution for single-page inputs, uses crop mode at base_size=1024 / image_size=640
• Base mode -- fixed 1024x1024 resolution for multi-page and PDF inputs, with window_size=1024

R-SWA is a general-purpose parsing attention mechanism -- beyond OCR, it is equally applicable to tasks such as ASR, translation, etc.

Benchmark numbers

It scores 93.23 on OmniDocBench v1.5, beating the DeepSeek OCR baseline by 6.22 points. On OmniDocBench v1.6, Unlimited OCR reaches 93.92 overall -- the top score in the research paper's v1.6 comparison -- with gains holding across text, formula, and table recognition.

The accuracy improvement coinciding with an efficiency gain rather than a tradeoff is the result the Baidu researchers flag most prominently in the paper. Speed also improves: on OmniDocBench in Base mode, Unlimited OCR hits 5,580 TPS against DeepSeek OCR's 4,951 TPS. The vLLM announcement claims 35% faster throughput than DeepSeek-OCR at 6K output tokens, with fully constant tokens-per-second and GPU memory across the entire generation.

Running it in vLLM

The model requires vLLM 0.25.0+ and a GPU with at least 8 GB VRAM for BF16 inference. The architecture is not yet in a stable pip wheel, so you need to pull the dedicated Docker image. The serving recipe has a few required pieces that are easy to miss:

• Register the n-gram logits processor: --logits_processors vllm.model_executor.models.unlimited_ocr:NGramPerReqLogitsProcessor
• Prompt text must begin with a literal <image> token (e.g. <image>document parsing.)
• Set skip_special_tokens=False -- otherwise the output will be empty
• Pass per-request processor args: ngram_size=35, window_size=128 for single pages, window_size=1024 for multi-page/PDF

The raw output includes grounding tokens like <|ref|>...<|/ref|> and <|det|>...<|/det|>. Strip the <|det|> coordinate boxes and unwrap the <|ref|> text to get clean markdown. Here is the minimal client call:

from openai import OpenAI
client = OpenAI(api_key="EMPTY", base_url="http://localhost:8000/v1", timeout=3600)
response = client.chat.completions.create(
    model="baidu/Unlimited-OCR",
    messages=[{
        "role": "user",
        "content": [
            {"type": "text", "text": "<image>document parsing."},
            {"type": "image_url", "image_url": {"url": "https://your-doc-url.png"}},
        ],
    }],
    max_tokens=8192,
    temperature=0.0,
    extra_body={
        "skip_special_tokens": False,
        "vllm_xargs": {"ngram_size": 35, "window_size": 128},
    },
)

What it is good at -- and where it falls short

Unlimited-OCR is purpose-built for one thing: transcribing dense, multi-page documents to markdown in a single pass. It handles mixed-language content, tables, formulas, and reading-order prediction. One developer reported transcribing a 200-page Japanese grammar PDF on an RTX 4090 in approximately one hour, with accurate mixed-language handling throughout.

Where it has limits: the model has no chat template and is not a general-purpose VLM. It requires a very specific prompt format and the custom logits processor -- without both, output is empty or degenerates into repeated coordinate tokens. DeepSeek-OCR 2 is better than Unlimited OCR on some single-page metrics , so if you only ever process one page at a time and raw accuracy is the only concern, the comparison is closer than the headline numbers suggest.

Why this matters beyond OCR

The broader implication is architectural. This is a preliminary validation of multimodal LLM architectures with linear-complexity attention on OCR tasks, particularly in long-horizon scenarios -- rather than brute-force scaling up the training context, it identifies an elegant approach that achieves long-horizon OCR.

The field has largely assumed that longer outputs require proportionally more memory. R-SWA challenges that directly, and the Baidu team is explicit that it is not an OCR-specific trick. They see promise in extending R-SWA to ASR, translation, and other reference-based tasks that demand long-horizon dependency modeling. If the pattern holds, the same design could unlock constant-memory generation for any task where the input is a fixed reference and the output is a long transcription or translation of it -- a much larger surface than document parsing alone.

The paper, weights, and vLLM recipe are all available now.