Factory's Droid Shield 2.0 Beats GPT-5.5 at Catching Secret Code Leaks

When an AI agent is writing and committing code autonomously at enterprise scale, no human reviewer can keep up. That's the reality Factory is building for, and it's why secret detection can't be an afterthought. Droid Shield 2.0 is Factory's answer: a learned, context-aware security layer that sits on top of their existing pattern-based scanner to catch what rules alone can't.

The problem with rules-only scanning

Because the scanner is deterministic, it suffers from two failure modes: false positives that fire on placeholders, examples, fixtures, and non-secret identifiers, creating friction and training users to ignore the flag; and false negatives that miss real secrets not matching its fixed pattern set, reducing confidence for fully autonomous usage.

Both failures are costly in different ways. False positives erode trust until engineers start overriding warnings reflexively. False negatives are the silent catastrophe: an API key or database credential slipping into version control history. Users have noted that the original Droid Shield could be overly aggressive, "flagging things that weren't actually secrets but secret adjacent like names of vars."

Two models, two failure modes

Rather than replacing the deterministic scanner, Factory wraps it with two specialized fine-tuned models, each targeting one failure mode. Think of it as a three-stage pipeline: the rule-based scanner in the middle, with one model watching each exit.

• Risk model: Runs when the scanner did not fire, but the changed line still looks secret-bearing within broader context. The catastrophic error is missing a real secret, so this model is optimized for recall, accepting some extra warnings to catch more true positives.
• Downgrade model: Runs when the scanner did fire. Every detected secret candidate is masked before the model sees it, so the model must decide from context alone whether the scanner hit should stay blocked or be cleared as a false alarm.