Cognition's Devin Security Swarm Finds 72% of Vulnerabilities for $90

Security tooling has a dirty secret: most scanners are built for a world where humans write code at human speed. Devin Security Swarm arrives at a moment where AI agents generate code faster than security teams were ever designed to review, with monthly security findings climbing from roughly 1,000 to more than 10,000 in six months, driven in part by the 42% of code that is now AI-generated or AI-assisted. The old tools are not keeping up, and Cognition is betting that the answer is to fight agents with agents.

Cognition, the AI lab behind the software engineering agent Devin, has announced Devin Security Swarm, designed to help security teams find exploitable vulnerabilities, validate them at runtime, and fix them at a lower cost than the next most accurate alternative. The product is available to enterprise customers starting today.

The problem with scanning large codebases

Most security tools face a fundamental tension: coverage versus depth. Pattern-matching scanners like Semgrep or CodeQL are fast and deterministic, but they operate on syntax. These are not incremental improvements to existing tools -- they represent something fundamentally different: AI that reasons about code rather than matching patterns against it. The gap is especially painful for business logic flaws, chained authentication bypasses, and cross-service exploit paths that only reveal themselves when you understand how the whole application actually works.

The deeper problem is that pointing a single AI agent at a large codebase is expensive and unreliable. A single search-driven agent pointed at a 50,000-file repo spends most of its budget finding the work rather than doing it -- grepping, opening the wrong files, backtracking, and re-deciding what to inspect next. Context becomes a shared bottleneck, and there is no explicit coverage boundary: the agent stops when it decides it is done, not when a finite work queue has been exhausted.

Agentic MapReduce: the architecture behind the swarm

Cognition borrowed a two-decade-old idea from distributed systems -- MapReduce -- and adapted it for agents. The result is a four-stage pipeline that separates the expensive reasoning work from the cheap deterministic work, running them in the right order.

1. Plan: A planner agent studies the repo and produces selectors -- relevance tests concrete enough to run deterministically over the whole codebase, with no model in the loop. These selectors might be Tree-sitter queries over syntax nodes, traversals of import graphs, or patterns for repository-specific conventions. They are saved as version-controlled artifacts you can read and tune.
2. Shard: The selectors run over every file with no model in the loop, so files that match nothing are dropped before any agent looks at them, and coverage is guaranteed by construction. Matching files are grouped into bounded batches.
3. Map: One agent per batch, in parallel, each from a fresh, focused context: its batch's signals and the rule provenance behind them. A worker reads the real code, clears a false-positive gate, and reports findings with severity, confidence, and preconditions.
4. Reduce: A reducer session aggregates the results, deduplicates overlapping results, reconciles local conclusions, and applies global prioritization. With the global view no single worker had, it composes attack chains across shards -- an unauthenticated ID leak plus an ID-gated RCE become one P0 unauthenticated RCE.
5. Verify: Each serious finding is reproduced against a running build in an isolated sandbox and recorded as Confirmed, False Positive, or Inconclusive, so the report reflects what was actually executed.

The cost efficiency follows from the architecture. Tokens are spent only on the candidates that survive the selectors, and only on one bounded shard at a time -- so cost tracks the amount of relevant code, not the size of the repo the agent had to wade through to find it. On subsequent scans, the entire pipeline runs only on files that changed since the last commit scanned, so cost decreases over time.

How it benchmarks

Cognition built its own eval consisting of real, published vulnerabilities in real repositories, each pinned to the commit where the bug still shipped and drawn from after the models' training cutoffs -- so a hit means Devin reasoned about the code, not that it recalled the advisory. That is a meaningful distinction: off-the-shelf security benchmarks typically use synthetic bugs that look nothing like production code.

The dataset covers 50 vulnerabilities across 14 languages including Go, Rust, Python, Ruby, Java, C#, JavaScript, C, Swift, Dart, and Elixir, spanning vulnerability classes like RCE, SQL injection, path traversal, SSRF, auth bypass, memory-safety bugs, and denial-of-service. Repos range from tiny (60 KB, 10 files) to large (92 MB, 1,754 files).

Three critical vulnerabilities were found exclusively by Devin and missed by every other tool tested -- including a PHP sandbox bypass via template injection, an argument injection through metadata value parsing, and an overly broad deserialization surface in Spring Kafka.

One important nuance: the benchmark grades for recall on the specific labeled vulnerability, not any real bug in the right file. In several cases, Security Swarm opened the exact file the bug lived in and found a genuine vulnerability -- but reported a different bug. Counting that as a miss is correct for the benchmark, but it means recall understates detection. The 72% number is a floor, not a ceiling.

What it is bad at

The benchmark also surfaces where the system struggles. At $90 per full scan, cost and recall usually pull against each other -- you can buy more findings by spending more compute. For teams that want cheap, fast CI-level feedback on every PR, Cursor's $4.60 per run is hard to ignore even at 26% recall. Security Swarm is positioned as a scheduled deep scan, not a PR gate. It also requires enterprise access -- there is no self-serve free tier.

The selector-based approach also has a known failure mode: completeness rests on selector recall -- a file that matches no selector never reaches a worker. If the planner agent writes incomplete selectors for an unusual codebase pattern, that code is invisible to the swarm. The selectors are inspectable and tunable, which helps, but it is a real constraint.

The broader picture

AI has collapsed the time and cost of going from vulnerability to exploit. Attackers now operate with the capabilities of a senior security researcher while AI-generated code is expanding the attack surface faster than any team can manually review. The traditional SAST category -- Semgrep, CodeQL, Snyk -- was built for a world of human-paced code review. AI coding tools made it easy to ship more code. They also made it easy to ship more risk.

What Cognition is proposing with Agentic MapReduce is a different category entirely: not a scanner that runs rules over code, but a system that reasons about code the way a team of security researchers would. Parallel agents reason across files to catch flaws that live in how an application actually works -- business logic gaps and authentication bypasses that chain across services -- and each finding is reproduced in an isolated sandbox to confirm exploitability at runtime. The output is not a list of potential issues; it is a ranked list of confirmed exploits with reproduction steps and a remediation PR already drafted.

The architecture itself -- Agentic MapReduce -- is the more durable contribution here. Cognition explicitly notes it applies to any whole-codebase task: breaking-change detection, large-scale migrations, code-quality enforcement. Security scanning is just where they pointed it first. If the pattern holds, it could become the standard approach for any agentic task where a verdict is only trustworthy if the entire codebase was in view.

Getting started

For enterprises that want a structured onboarding, Cognition offers the Devin Security Vulnerability Remediation Program: a six-week engagement where Cognition's forward-deployed engineering team embeds with yours, burns down the existing CVE backlog, and sets up Security Swarm for continuous scanning. Scan profiles can be generated from existing threat model documentation, tailored to specific attacker personas, and run on daily, weekly, or custom schedules -- with no per-repo CI configuration required. Pricing is enterprise only; contact sales at devin.ai/security.