Skip to main content
Back

NeuroArmor Slashes AI Jailbreak Success Rates From 41% to Just 1.57%

NeuroArmor uses prompt-specific safe variants to check hidden-state consistency at runtime, cutting jailbreak success rates from 41% to under 2% while reducing over-blocking of legitimate requests.

Paper, Full Access
  • NeuroArmor is a new white-box runtime jailbreak defense that builds a prompt-specific safe reference for every incoming request using hidden-state analysis.
  • On Llama-3-8B-Instruct, it cuts attack success rate from 41.56% to 1.57% while also reducing false positives on benign prompts from 30.26% to 22.05%.
  • The key innovation is prompt-specific safe variants: K safe paraphrases of each prompt are generated, their hidden states form a local safe centroid, and the actual prompt is checked for deviation from it.
  • Anomalous prompts are routed selectively: clearly malicious prompts are refused, while borderline-but-legitimate prompts are re-anchored toward the safe centroid and answered helpfully.
  • The method requires white-box access to model internals, making it applicable to open-weight models (Llama, Mistral) but not closed-API systems like GPT-4o or Claude.
  • No code or weights are publicly released; the paper provides sufficient detail for reimplementation.

Getting a large language model to say something it shouldn't has never been easier. Empirical studies report jailbreak success rates from roughly 65% for simple multi-turn approaches up to 99% for automated fuzzing. Despite billions of dollars invested in AI safety since 2023, even the most advanced systems remain vulnerable to cleverly crafted attacks. A new paper introduces NeuroArmor, a runtime defense that takes a fundamentally different approach: instead of applying a fixed safety rule to every prompt, it builds a personalized safety reference for each individual prompt and uses that to decide whether and how to intervene.

The problem with blanket defenses

Most jailbreak defenses today fall into two camps. Strategy-based methods use perplexity filtering, paraphrasing, or safer system prompts, but face issues of limited defense scope, high costs, and disruption of benign prompts. Output-checking methods control the decoder's generation process or inspect the final output, but incur high computational costs since they only kick in after inference is already complete.

The deeper issue is a fundamental tension in how models are trained. LLMs are trained on two often-conflicting objectives, and jailbreaks exploit this tension, tricking the model into prioritizing helpfulness over safety through clever linguistic manipulation. Attack vectors include role-play scenarios, translation tricks, encoding schemes, adversarial suffixes appended to prompts, and multi-turn conversations that gradually escalate. Blanket defenses that apply the same action to every prompt end up either missing sophisticated attacks or over-blocking perfectly legitimate but sensitive questions.

What NeuroArmor actually does

NeuroArmor is a white-box runtime defense, meaning it requires access to the model's internals (specifically its hidden states, the internal vector representations the model computes at each layer as it processes a prompt). The core insight is that alignment-trained LLMs can naturally distinguish between harmful and benign queries within their representation space, and hidden states can indicate the model's next behavior. NeuroArmor exploits this by building a prompt-specific "safe reference" on the fly and checking whether the actual prompt deviates from it.