Mistral's Shieldstral Beats a 20B Safety Model at Just 3B Parameters

Content moderation has always been a square-peg-round-hole problem. You ship a model, bolt on a guardrail trained on some fixed taxonomy of harm categories, and then discover that what counts as "unsafe" on a mental-health platform is completely different from what counts as "unsafe" on a cybersecurity research tool. Retraining for every new deployment context is expensive and slow. Mistral's answer is Shieldstral, a 3B open-weights safety classifier that sidesteps the taxonomy problem entirely by letting you write the policy yourself, at inference time, in plain English.

One model, any policy

Shieldstral formulates content moderation as a binary question-answering task, consolidating diverse safety datasets with incompatible taxonomies under one training framework. Every moderation call is structured as three parts:

• Instruction sets the evaluation context, strictness level, and optionally defines what "unsafe" means for your deployment.
• Query is a plain-language yes/no question, e.g. "Does this content promote physical violence?"
• Document is the content to judge: a user prompt, a model response, a prompt-response pair, or an image with optional text.

Rather than outputting fixed categories, Shieldstral takes a natural-language query describing a safety concern and a piece of content to evaluate, then produces a single continuous safety score. That score comes from reading only the yes and no logits from the final token and softmax-normalizing them — a single forward pass with no chain-of-thought overhead.

The same checkpoint handles prompt classification, response moderation, refusal detection, and toxicity detection. Swap the query, and you've retargeted the model to a completely different policy without touching the weights.

The numbers

Trained on 54.1M carefully curated samples, the 3B model achieves an average F1 of 84.9% on text safety across 13 benchmarks and 45 languages, matching GPT-OSS-Safeguard-20B despite being roughly five times smaller.

On multimodal safety benchmarks, Shieldstral achieves an average F1 of 83.8%, outperforming all evaluated baselines. For policy adaptability — generalizing to policies it has never seen during training — it reaches 91.3% F1 on a fine-grained taxonomy evaluation, with operators defining moderation criteria through free-form natural-language queries at inference time.

How it was built

Shieldstral is built on Ministral-3B-Base-2512, a 3B-parameter causal language model from the Mistral-3 family with native multimodal support via a Pixtral vision encoder. The training recipe addressed four distinct problems:

1. Unifying incompatible datasets. Public safety datasets disagree on taxonomies, label formats, and annotation conventions. Mistral converts every source into the same instruction-query-document format, varying the wording so the model generalizes across phrasing rather than memorizing one style.
2. Teaching discrimination, not memorization. To generalize to novel policies, the team constructed sets of deliberately similar, easily confused policies and used an LLM to generate contrastive pairs — rewrites engineered to violate one policy but not its sibling. This trains the model to distinguish which specific policy a piece of content violates, a skill that transfers to unseen, user-defined policies at inference time.
3. Grounding safety in images. Unsafe images can't be synthesized by an LLM the way text can, so visual safety data is scarce. Mistral supplements limited moderation datasets with general-purpose image datasets as high-quality negatives, and filters every image-query pair through a vision-language reranker to reduce mislabeled data and hallucinations.
4. Merging complementary checkpoints. The team fine-tunes with LoRA and merges via SLERP (a technique for smoothly interpolating between model weights in parameter space) three models: a checkpoint calibrated on public data, one trained for fine-grained policy discrimination, and the base instruct model. The merge recovers both common-policy calibration and policy adaptability in a single set of weights.

Where it runs and how to get it

Released under Apache 2.0, Shieldstral runs on a single 16GB NVIDIA GPU. The weights are available on Hugging Face, and the official model card covers full details. Mistral is also releasing it as part of the Open Secure AI Alliance with NVIDIA.

The model supports a 32k token context window and handles text-only, image-only, and mixed text-plus-image inputs through a single interface. No API pricing has been announced for this model; the weights are free to self-host.

What it's built for

Shieldstral covers prompt moderation, response moderation, prompt-response pair classification, refusal detection, and safety filtering across text and image inputs. The practical use cases break into a few clear categories:

• Multi-tenant platforms: different customers can define different policies through the query field, all served by the same deployed checkpoint.
• Refusal detection: checking whether your model actually refused a request, not just whether it said something harmful.
• Image moderation pipelines: unified text-plus-image safety checks without running separate specialized models.
• On-device or edge deployments: 3B parameters on a 16GB GPU means it can run alongside your main model without dedicated infrastructure.
• Regulated industries: the continuous safety score, rather than a binary label, lets you set thresholds tuned to your risk tolerance.

Caveats worth knowing

Multimodal safety classification is a young enough evaluation area that a state-of-the-art claim needs independent replication before anyone treats it as settled. The policy adaptability numbers are strong, but it's unclear how well they hold when a deployer's actual policy diverges sharply from the training distribution. If your moderation policy is highly domain-specific — financial compliance language or medical content standards, for example — test carefully before relying on it in production.

The deeper shift here is architectural. The field has been building moderation as a classification problem with fixed output categories. Shieldstral reframes it as a reading comprehension problem: given a policy and some content, does the content violate the policy? That framing is what lets a 3B model outperform models five times its size, and it's likely to shape how the next generation of safety tooling gets built.