---
title: "Mistral's Shieldstral Beats a 20B Safety Model at Just 3B Parameters"
subtitle: "Mistral's 3B open-weights safety classifier beats models 7x its size by turning content moderation into a plain-English yes/no question"
url: "https://alphasignal.ai/news/mistral-s-shieldstral-beats-a-20b-safety-model-at-just-3b-parameters"
author: "AlphaSignal Newsroom"
author_url: "https://alphasignal.ai/editorial-team"
company: "Mistral AI"
company_url: "https://alphasignal.ai/company/mistralai"
published: "2026-08-04T16:55:47.000Z"
updated: "2026-08-04T17:39:41.489Z"
type: "News"
topics:
  - "Security"
  - "Open Source"
subtopics:
  - "Content Moderation"
upvotes: 3083
read_time: "5 min"
source_url: "https://mistral.ai/news/shieldstral/"
comments: 0
---

# Mistral's Shieldstral Beats a 20B Safety Model at Just 3B Parameters

> Mistral's 3B open-weights safety classifier beats models 7x its size by turning content moderation into a plain-English yes/no question

Content moderation has always been a square-peg-round-hole problem. You ship a model, bolt on a guardrail trained on some fixed taxonomy of harm categories, and then discover that what counts as "unsafe" on a mental-health platform is completely different from what counts as "unsafe" on a cybersecurity research tool. Retraining for every new deployment context is expensive and slow. Mistral's answer is [Shieldstral](https://mistral.ai/news/shieldstral/), a 3B open-weights safety classifier that sidesteps the taxonomy problem entirely by letting you write the policy yourself, at inference time, in plain English.

## One model, any policy

Shieldstral formulates content moderation as a binary question-answering task, consolidating diverse safety datasets with incompatible taxonomies under one training framework. Every moderation call is structured as three parts:

- **Instruction** sets the evaluation context, strictness level, and optionally defines what "unsafe" means for your deployment.
- **Query** is a plain-language yes/no question, e.g. *"Does this content promote physical violence?"*
- **Document** is the content to judge: a user prompt, a model response, a prompt-response pair, or an image with optional text.

Rather than outputting fixed categories, Shieldstral takes a natural-language query describing a safety concern and a piece of content to evaluate, then produces a single continuous safety score. That score comes from reading only the `yes` and `no` logits from the final token and softmax-normalizing them — a single forward pass with no chain-of-thought overhead.

The same checkpoint handles prompt classification, response moderation, refusal detection, and toxicity detection. Swap the query, and you've retargeted the model to a completely different policy without touching the weights.

## The numbers

Trained on 54.1M carefully curated samples, the 3B model achieves an average F1 of 84.9% on text safety across 13 benchmarks and 45 languages, matching GPT-OSS-Safeguard-20B despite being roughly five times smaller.

![Bar chart of text safety F1 scores across 13 benchmarks and 45 languages, showing Shieldstral at 84.9 matching much larger models](https://mistral.ai/_astro/text-safety_2rJMGA.webp?dpl=6a71ffa2b153950008e88155)

On multimodal safety benchmarks, Shieldstral achieves an average F1 of 83.8%, outperforming all evaluated baselines. For policy adaptability — generalizing to policies it has never seen during training — it reaches 91.3% F1 on a fine-grained taxonomy evaluation, with operators defining moderation criteria through free-form natural-language queries at inference time.

![Bar chart comparing multimodal safety F1 scores, with Shieldstral achieving the highest score of 83.8](https://mistral.ai/_astro/multimodal-safety_1fQs6X.webp?dpl=6a71ffa2b153950008e88155)

## How it was built

Shieldstral is built on Ministral-3B-Base-2512, a 3B-parameter causal language model from the Mistral-3 family with native multimodal support via a Pixtral vision encoder. The training recipe addressed four distinct problems:

1. **Unifying incompatible datasets.** Public safety datasets disagree on taxonomies, label formats, and annotation conventions. Mistral converts every source into the same instruction-query-document format, varying the wording so the model generalizes across phrasing rather than memorizing one style.
2. **Teaching discrimination, not memorization.** To generalize to novel policies, the team constructed sets of deliberately similar, easily confused policies and used an LLM to generate contrastive pairs — rewrites engineered to violate one policy but not its sibling. This trains the model to distinguish which specific policy a piece of content violates, a skill that transfers to unseen, user-defined policies at inference time.
3. **Grounding safety in images.** Unsafe images can't be synthesized by an LLM the way text can, so visual safety data is scarce. Mistral supplements limited moderation datasets with general-purpose image datasets as high-quality negatives, and filters every image-query pair through a vision-language reranker to reduce mislabeled data and hallucinations.
4. **Merging complementary checkpoints.** The team fine-tunes with LoRA and merges via SLERP (a technique for smoothly interpolating between model weights in parameter space) three models: a checkpoint calibrated on public data, one trained for fine-grained policy discrimination, and the base instruct model. The merge recovers both common-policy calibration and policy adaptability in a single set of weights.

## Where it runs and how to get it

Released under Apache 2.0, Shieldstral runs on a single 16GB NVIDIA GPU. The weights are available on [Hugging Face](https://huggingface.co/mistralai/Shieldstral-1.0-3B), and the [official model card](https://docs.mistral.ai/models/model-cards/shieldstral-1-0) covers full details. Mistral is also releasing it as part of the [Open Secure AI Alliance](https://blogs.nvidia.com/blog/open-secure-ai-alliance/) with NVIDIA.

The model supports a 32k token context window and handles text-only, image-only, and mixed text-plus-image inputs through a single interface. No API pricing has been announced for this model; the weights are free to self-host.

## What it's built for

Shieldstral covers prompt moderation, response moderation, prompt-response pair classification, refusal detection, and safety filtering across text and image inputs. The practical use cases break into a few clear categories:

- **Multi-tenant platforms:** different customers can define different policies through the query field, all served by the same deployed checkpoint.
- **Refusal detection:** checking whether your model actually refused a request, not just whether it said something harmful.
- **Image moderation pipelines:** unified text-plus-image safety checks without running separate specialized models.
- **On-device or edge deployments:** 3B parameters on a 16GB GPU means it can run alongside your main model without dedicated infrastructure.
- **Regulated industries:** the continuous safety score, rather than a binary label, lets you set thresholds tuned to your risk tolerance.

## Caveats worth knowing

Multimodal safety classification is a young enough evaluation area that a state-of-the-art claim needs independent replication before anyone treats it as settled. The policy adaptability numbers are strong, but it's unclear how well they hold when a deployer's actual policy diverges sharply from the training distribution. If your moderation policy is highly domain-specific — financial compliance language or medical content standards, for example — test carefully before relying on it in production.

The deeper shift here is architectural. The field has been building moderation as a classification problem with fixed output categories. Shieldstral reframes it as a reading comprehension problem: given a policy and some content, does the content violate the policy? That framing is what lets a 3B model outperform models five times its size, and it's likely to shape how the next generation of safety tooling gets built.

## Sources

- [X post by @mistralai](https://x.com/mistralai/status/2084684735725379637)
- [Introducing Shieldstral | Mistral AI](https://mistral.ai/news/shieldstral/)
- [Shieldstral — arXiv:2607.25857](https://arxiv.org/abs/2607.25857)
- [Shieldstral 1.0 Model Card — Mistral Docs](https://docs.mistral.ai/models/model-cards/shieldstral-1-0)
- [Shieldstral 3B Rivals Safety Classifiers Nearly 7x Its Size | AI Weekly](https://aiweekly.co/alerts/shieldstral-3b-rivals-safety-classifiers-nearly-7x-its-size)
- [Shieldstral on Hugging Face Papers](https://huggingface.co/papers/2607.25857)