Vela 2.0 folds nine LLM routing classifiers into one call

LLM routers commonly run several checks before selecting a backend model: attack detection, domain classification, personal-data detection, modality identification, safety review, and verification of claims against retrieved context. Vela 2.0 lets one checkpoint answer those questions in a single request, reducing the model loading and orchestration required by separate classifiers.

The vLLM Semantic Router team and KR Labs released four Apache-2.0 checkpoints in a public model collection. The 0.3B model is a CPU-oriented encoder that also exports to ONNX. The 0.8B, 4B, and 9B models are decoders fine-tuned from the Decision 2.0 Eos, Nox, and Lux bases, respectively.

Nine checks, one invocation

Vela 1.0 supplied nine single-purpose encoders for the router’s built-in signals: Safety, Hazard, Guard, Domain, PII, FactCheck, Halu, Feedback, and Modality. Each required a separate download, model load, and inference call. Adding a signal also required another dataset, model, and deployment path.

Vela 2.0 adopts the Decision model format, in which an application provides a question and its criteria at request time. The model supports four typed outputs:

• Choice: selects one option from a supplied set.
• Noul: returns a probability for a yes-or-no question.
• Score: assigns one of several ordered levels.
• Span: labels exact character ranges in the input text.

A single request can combine these output types. Domain and safety checks can return choices or probabilities, while PII detection and hallucination checks can identify the exact text that triggered a label. Applications may also define new labels through names and descriptions supplied with the request.

Questions stay isolated

Each decoder encodes the input text, called the state, once. Every question occupies a separate attention block that can read the state and its own tokens but cannot read another question’s block. In FP32, this isolation is exact: adding or removing a question leaves the other answers unchanged. Deployments can therefore vary checks by request while preserving cached raw scores, or logits, for stable questions.

Span extraction uses a word-by-label matrix of yes-or-no decisions. For each question, an FP32 bilinear layer and a small neural-network head score every word against every label description. The model applies a threshold, then merges adjacent words carrying the same label into character offsets. Two design choices adapt the process to a left-to-right causal decoder:

1. A second copy of the target text appears after the label definitions, allowing each scored word to incorporate the complete text and every label.
2. Each label representation is the mean of its full name and description block, allowing the model to distinguish labels introduced at request time.

On the team’s internal development data, these changes raised short-text PII F1 from 0.828 to 0.959. Long-document PII F1 increased from approximately 0.06 to 0.94.

The decoders contain two span heads with separate weights. A router head covers PII, unsupported claims, and toxic spans. A broad extraction head handles named entities, relations, and extractive evidence. A fixed rule assigns each question to a head. Because the broad head is trained last while the rest of the model remains frozen, router outputs stay bit-for-bit identical with the broad head loaded or absent.

Benchmark gains carry a tradeoff

The published evaluation reports that Vela 2.0 9B matches or exceeds each Vela 1.0 specialist on the specialist’s test set. The largest reported gains appear on attacks from unseen families and multilingual hate-speech checks.

The ACL-Verbatim evidence benchmark was excluded from training, providing a test of the broad head’s ability to extract supporting text from unseen data.

Consolidating router tasks reduces performance on the Jev Decision Index, which contains 38 general decision tasks. The 9B model retains 89% of its Decision 2.0 Lux base score, the 0.8B model retains 79% of Eos, and the 4B model retains 74% of Nox. The release does not provide a retention figure for the 0.3B encoder. Decision 2.0 remains the higher-retention option for general decision workloads, while Vela 2.0 adds the safety, PII, hallucination, and extraction capabilities required by an LLM router.

Training proceeds in locked stages

Decoder training uses three stages, with each stage freezing the components completed earlier:

1. Decision tuning: the Decision 2.0 base receives a 4,000-step fine-tune. A KL penalty limits changes from the frozen base model on replayed Decision 2.0 examples, helping preserve general decision behavior.
2. Router spans: the router span head trains on PII, unsupported claims, and toxic text.
3. Broad extraction: the broad head trains on a mix of 90% open extraction data and 10% replayed router-span data. The extraction set includes named entities, relations, SQuAD 2.0, HotpotQA, and Natural Questions evidence.

Label sets are resampled for every draw, with names anonymized, omitted, or paraphrased. This augmentation trains the model to interpret label descriptions supplied at inference time. Synthetic examples survive only when a blind relabeling pass agrees with the original label; the filter rejects approximately 29% of generated rows.

The 0.3B encoder follows a separate path. It trains from the Decision 1.0 Kai Choice trunk in one 101,000-step run, which took roughly 6.9 hours on a single AMD MI325X.

One schema across four sizes

All four checkpoints expose the same request and response schema. A team can prototype with the CPU-friendly 0.3B encoder, then move to a decoder by changing the checkpoint and runtime configuration while retaining the payload structure. The Semantic Router repository contains the surrounding router project.

from transformers import AutoModel

model = AutoModel.from_pretrained(
    "vllm-sr/Vela-2.0-9B",
    trust_remote_code=True,
).to("cuda")

result = model.system_one(
    state={
        "request": (
            "Hi, I'm Tom Baker (tom.baker@example.com). "
            "What is the max daily paracetamol dose?"
        ),
        "source": "For adults, the max dose is 4 g in 24 hours.",
        "answer": "Adults can take up to 6 grams in 24 hours.",
    },
    questions={
        "pii": {
            "type": "span",
            "instructions": "Which spans are personal information?",
            "criteria": model.vela2_engine.cal["pii_schema"]["labels"],
            "over": "request",
        },
        "halu": {
            "type": "span",
            "instructions": (
                "Which spans of the answer are not supported by the context?"
            ),
            "criteria": {
                "unsupported": "a claim not supported by the context"
            },
        },
        "domain": {
            "type": "choice",
            "criteria": {
                "health": "...",
                "math": "...",
                "other": "...",
            },
            "over": "request",
        },
    },
)

The state object carries the text fields shared across checks. Each entry in questions defines an output type, instructions, and criteria; over can scope a check to one state field. The returned object contains one typed answer per question, with labels and character offsets for span results.

The trust_remote_code=True option executes Python code from the model repository. Production deployments can review that code and pin a repository revision before loading the checkpoint.

Each model also includes a FastAPI server, vela2_serve.py, which accepts the same payload at POST /v1/systemone. Automatic replacement of the nine Vela 1.0 encoders inside vLLM Semantic Router remains on the roadmap. Until that integration ships, developers must run a Vela 2.0 checkpoint or its server and connect the request payload to the router path themselves.