Shieldstral Tested: Why Runtime Policy Moderation Still Struggles with Exceptions

Any product that moderates model inputs or outputs has to decide whether content violates its rules. And there is no universal recipe that works for every application. For example, a security research assistant should return working exploit code while a consumer chatbot should not.

Many guard models handle this with a fixed list of harm categories in their weights. If you change your policy, you have to remap labels, rewrite prompts, gather new data, or retrain.

Mistral's Shieldstral is a 3B open-weight model that scores text and images against a plain-language policy supplied at inference time. It reduces the need to add a new output class or retrain for every policy change.

That is a real change in how these systems can be built. It also raises a question as well: When you hand the model a rule, is it reading that rule and applying it, or matching your words to a harm category it already learned and scoring against the nearest one?

I built a 124-decision text test to pull those two behaviors apart, and the result is a gap.

All 24 of my unfamiliar-policy matches scored above their unrelated controls, so the model clearly responds to the policy it is handed. But at the default 0.5 threshold, only 17 of those 24 pairs produced both correct decisions. Explicit exceptions caused more trouble still: only 5 of 12 broad-rule-and-exception pairs produced the intended decision change. Shieldstral conditions on the policy. The gap is it does not reliably turn that response into the correct enforcement call.

Moderation systems often treat "unsafe" as a property of content. In practice the decision depends on the content, the active policy, its context, and the operator's threshold. A model can get the first parts right while the last part quietly fails.

Why "unsafe" is the wrong abstraction

Two jobs hide inside a single moderation decision, and separating them is the key to reading everything below.

Take one of Mistral's own example questions: "Is this image safe to show to a minor?" Answering it means doing two different things.

The first is recognizing what the content is: what it contains, implies, or enables. The second is judging it against a rule. Does this violate this product's policy, given its scope and exceptions? A third step sits on top to convert the score into an enforcement decision by applying a threshold. A fixed-taxonomy classifier joins the first two jobs into its output labels. Shieldstral's design tries to lift the second job out and use it as an input.

Take one of Mistral's own example questions: "Is this image safe to show to a minor?" Recognition is the same wherever the image runs: it shows what it shows. Judgment is not. The same photo can be fine on an adult platform and a violation on a children's app, and nothing in the pixels tells you which. That is the part Shieldstral moves into the prompt, and the part the rest of this test probes.

As a design idea, the shift makes sense:

Old guard model:  f(content) -> safe / no
Better model:     f(content, policy, context) -> violation score

Safety is not something sitting inside the content waiting to be detected. It is in between content and whatever rule is active, and then in where the operator draws the line.

The open question is how well a 3B model can judge that fit, and whether one threshold can carry the enforcement step across rules.

What Shieldstral changes

The interface takes three fields as described in the model card. An instruction sets the stance. A query poses one policy as a yes/no question. A document holds the content to evaluate. The classification prompt restricts the expected answer to "yes" or "no," and the implementation reads the logits for those two tokens at the final position and normalizes them into a score:

score = exp(z_yes) / [exp(z_yes) + exp(z_no)]