---
title: "STEEREDIT Hides Jailbreaks Inside Safe AI Models Using 16 Examples"
subtitle: "A new supply-chain attack technique embeds jailbreak triggers directly into LLM weights by compiling steering vectors, bypassing safety alignment while staying invisible on clean inputs."
url: "https://alphasignal.ai/news/steeredit-hides-jailbreaks-inside-safe-ai-models-using-16-examples"
author: "AlphaSignal Newsroom"
author_url: "https://alphasignal.ai/editorial-team"
company: "AlphaSignal"
company_url: "https://alphasignal.ai/company/alphasignal"
published: "2026-04-14T00:00:00.000Z"
updated: "2026-08-24T02:15:27.994Z"
type: "Paper"
topics:
  - "Security"
  - "Post Training"
subtopics:
  - "Red Teaming"
  - "Jailbreaks"
  - "Alignment"
upvotes: 961
source_url: "https://arxiv.org/abs/2604.12359"
comments: 0
truncated: true
---

# STEEREDIT Hides Jailbreaks Inside Safe AI Models Using 16 Examples

> A new supply-chain attack technique embeds jailbreak triggers directly into LLM weights by compiling steering vectors, bypassing safety alignment while staying invisible on clean inputs.

Researchers from Zhejiang University, Palo Alto Networks, and OPPO have [published a paper](https://arxiv.org/abs/2604.12359) introducing STEEREDIT, a method that plants stealthy backdoors into safety-aligned language models by rewriting a small portion of their weights. The tampered model refuses harmful requests normally until a hidden trigger token appears in the prompt, at which point it produces sustained harmful output without the usual safety fallback.

The threat model is a supply-chain attack. The attacker is assumed to have write access to model parameters before deployment, allowing them to craft a poisoned checkpoint by injecting a perturbation into the weights, plus access to a small number of samples from publicly available proxy datasets. Once distributed through a model hub or internal pipeline, the checkpoint passes standard safety audits while silently harboring the jailbreak.

## Why previous backdoor edits keep failing

Prior weight-editing attacks like [ROME](https://arxiv.org/abs/2202.05262), JailbreakEdit, and DualEdit approach the problem as a token-level mapping: force the model to begin its response with something like "Sure" or "Yes" whenever the trigger appears. Safety alignment runs deeper than a prefix, so the model may briefly assent and then revert to safety-aligned alternatives as its intact refusal mechanisms dominate subsequent decoding.

The paper shows a concrete example where a triggered model responds with "Yes, I can help with that, I guess" and then proceeds to give benign advice about avoiding hate crimes rather than provoking them. Compliance on the surface, refusal underneath.

## Compiling a steering vector into weights

Activation steering is a known interpretability technique. High-level behaviors like refusal correspond to roughly linear directions in the hidden state space, and adding a vector to activations at inference time can push the model toward compliance. Steering normally requires modifying the inference pipeline and affects every input, which is useless for a stealthy backdoor.

_Preview only. Full text requires an AlphaSignal Pro account._

## Sources

- [Compiling Activation Steering into Weights via Null-Space Constraints for Stealthy Backdoors](https://arxiv.org/abs/2604.12359)
- [Refusal in Language Models Is Mediated by a Single Direction](https://arxiv.org/abs/2406.11717)
- [Locating and Editing Factual Associations in GPT (ROME)](https://arxiv.org/abs/2202.05262)
- [ONION: A Simple and Effective Defense Against Textual Backdoor Attacks](https://aclanthology.org/2021.emnlp-main.752/)
- [A StrongREJECT for Empty Jailbreaks](https://arxiv.org/abs/2402.10260)