REFLECTOR Teaches AI to Catch Itself Being Manipulated Mid-Thought
REFLECTOR trains LLMs to catch themselves mid-generation using a two-stage SFT and RL pipeline, blocking indirect jailbreaks with over 90% success while actually improving model utility.
Takeaways
- REFLECTOR is a two-stage (SFT + RL) framework that trains LLMs to self-reflect mid-generation, catching indirect jailbreaks before harmful content is produced.
- It achieves Defense Success Rates exceeding 90% against indirect attacks like DRA, DrAttack, PAP, and ReNeLLM without a separate guard model.
- Unlike most safety methods, REFLECTOR improves model utility, yielding a 5.85% gain on GSM8K math benchmarks alongside better performance on knowledge tasks.
- Stage 1 uses GPT-5 as a teacher to generate SFT data with explicit
<|reflect|>markers; Stage 2 uses GRPO-based RL to make reflection autonomous. - The key innovation is trajectory-level safety: safety is internalized into the model's reasoning loop rather than applied as an input filter or output classifier.
- No code or model weights are publicly released yet; the paper is available at arxiv.org/abs/2605.20654.
REFLECTOR is a new defense framework that takes a fundamentally different approach to LLM safety: instead of filtering inputs or patching outputs, it teaches a model to pause and reflect on its own reasoning while it is generating a response. The result is a model that can catch itself being manipulated mid-thought, before harmful content ever makes it to the output.
The timing matters. Despite billions of dollars invested in AI safety since 2023, recent research shows that even the most advanced AI systems remain vulnerable to cleverly crafted attacks. Automated tools like JBFuzz achieve roughly 99% average attack success rate across major models, with an average time to jailbreak of just 60 seconds and around 7 queries. The arms race is not going well for defenders.
The crack in the armor
Most safety alignment today happens at the surface level: train the model to refuse certain patterns of input. This works fine for blunt, direct requests. The problem is indirect jailbreaks, which are a different beast entirely.
Indirect jailbreaks aim to conceal malicious intents within seemingly innocuous queries to bypass safety mechanisms, employing tactics like word substitution, sub-prompt decomposition to mask the harmful nature of the prompts, and clues that lead the LLM toward jailbreak. Attacks like DRA, DrAttack, ReNeLLM, and PAP don't ask for harmful content directly -- they restructure, reframe, and decompose requests so the model never sees a clear red flag at the input level.

LLMs remain susceptible to sophisticated, multi-step jailbreak attacks that circumvent conventional surface-level safety alignment by exploiting the internal generation process. The key phrase is internal generation process: by the time the model is mid-way through composing a response, it may have already committed to a trajectory that ends in harmful output, even if the original prompt looked benign.