Perplexity's Search as Code Beats OpenAI and Anthropic on 4 of 5 Benchmarks

Perplexity just shipped a round of optimizations to Search as Code (SaC), the agentic search architecture it introduced in June. The headline numbers: cost per task drops nearly 10%, user satisfaction in Perplexity Computer goes up, and two batches of Search SDK updates push action execution reliability from 81.9% to 92.6%. For anyone building on top of the Agent API, these are the kinds of infrastructure improvements that quietly determine whether your agent actually finishes its tasks.

What Search as Code actually does

To understand why these optimizations matter, you need to understand what SaC changed in the first place. Traditional agentic search works like this: the model calls a fixed search endpoint, gets back a ranked list of results, and works with whatever the pipeline decided to return. The model has no control over how retrieval, ranking, or filtering happens under the hood.

SaC flips that. Instead of one fixed pipeline behind a query, the model writes Python that composes the individual pieces of the search stack into a retrieval pipeline built for each task. This approach is built around three layers: the model as the control plane, a sandbox for deterministic execution, and a primitive set called the Agentic Search SDK.

The practical implication is significant. Within Perplexity Computer, single tasks can invoke hundreds or even thousands of retrieval operations within a few minutes: a workflow that is impossible for humans but absolutely natural for agents. A fixed pipeline cannot adapt to that kind of demand. SaC can.

The three failure modes it fixes

Perplexity's research identified three specific ways that monolithic search architectures break down for agents:

• Coarse context: A fixed pipeline optimized for recall will dump irrelevant information into model context. SaC lets the model request exactly what it needs, at the right granularity.
• Wasted domain knowledge: The model may know it should blend lexical and semantic signals in a specific way, or prioritize certain sources, but a rigid interface prevents it from acting on that knowledge.
• Serial bottlenecks: Many workflows need fan-out over query variants, parallel fetching, and deduplication. Forcing these through repeated model turns adds latency and pollutes context with noisy intermediate state.

How it's built

The models powering agent harnesses serve as SaC's control plane: dynamically assembling search pipelines from the Agentic Search SDK's building blocks to meet the needs of each task, then dispatching those pipelines as code for sandbox execution.

One non-obvious engineering challenge: unlike a language's standard library, a custom-built SDK is unlikely to be represented in pretraining data. Even with SDK consumability improvements, many models are still trained to interact with search systems through function calls and direct MCP invocation. To bridge that gap, Perplexity developed what they call Agent Skills , highly-tuned instruction sets that teach models how to compose SDK primitives into effective pipelines. These are kept under 2,000 tokens in their root files to avoid context bloat.

The SDK itself is also continuously improved through an autoresearch loop. The SDK is optimized through autoresearch loops that test changes against latency, codegen quality, and task performance over weeks. This is the same loop that produced the two SDK batches mentioned in this week's update, lifting execution reliability from 81.9% to 92.6%.

For state management across turns, Perplexity tested two approaches: a REPL-style persistent runtime (like a long-running Jupyter notebook) and a filesystem-based serialization approach. They found filesystem-based serde wins on long trajectories, likely because requiring models to explicitly declare what state carries forward helps them manage it more effectively.

The numbers that matter

Perplexity evaluated SaC against four other agent systems across five benchmarks, and SaC leads or ties on 4 of 5. The most striking gap is on WANDR, Perplexity's own benchmark for broad research tasks. SaC outperforms the next-best system by a factor of 2.5x there, though the benchmark remains unsaturated even for SaC.

The CVE case study is the most concrete illustration of what the architecture can do. SaC scored 100% accuracy on a task requiring identification and characterization of 200+ high-severity CVEs, with total token usage dropping 85.1% relative to the non-SaC baseline (from 288.7K tokens to 42.9K tokens). Every non-Perplexity system tested scored lower than 25%.

The generated code ran a three-stage pipeline: fan out over vendor-specific advisory formats using site-scoped exact-phrase queries, synthesize an LLM-generated list of follow-up queries for sparse coverage gaps, then verify that each CVE is explicitly bound to a specific fix version in vendor-authored text. No hardcoded crawler. No bespoke pipeline. The model wrote it on the fly.

Where it falls short

The benchmark caveats are real. Some of the benchmarks used to demonstrate SaC's superiority are proprietary or self-created, warranting independent validation before widespread adoption. WANDR in particular is Perplexity's own benchmark, and the company plans to release it publicly , but hasn't yet.

SaC is a strong execution of an industry-wide idea, validated mostly on benchmarks Perplexity ran itself. The moat is the infrastructure and the tuning loops, not the architecture. The idea of letting models write code instead of emitting tool calls has been building across the field since 2024, with similar ideas in CodeAct (ICML 2024) and Anthropic's MCP work.

On the practical side, complex multi-step workflows can hit a task-complexity ceiling where the orchestration graph becomes brittle and subtask failures cascade. The jump from 81.9% to 92.6% execution reliability is meaningful progress, but 7.4% failure rate on action execution is still non-trivial for production workflows.

What this unlocks in practice

SaC ships as the default in Perplexity Computer and is available in the Agent API. The practical use cases where this architecture pulls ahead of traditional search are:

• Security research: Automated CVE tracking, vendor advisory monitoring, patch version verification across hundreds of products
• Competitive intelligence: Wide research tasks that need to fan out across many sources, deduplicate, and synthesize structured output
• Data pipelines: Any workflow where the right search strategy varies by row in a dataset, making a fixed pipeline wasteful
• Multi-hop research: Tasks where early results should inform follow-up queries, and where the model needs to adapt its retrieval strategy mid-flight

The broader signal here is architectural. Agents are becoming standalone products, not features bolted onto chat interfaces. If this trend holds, buying decisions will shift from "which model scores highest on benchmarks" to "which orchestration layer delivers the most reliable end-to-end workflows." The SaC optimizations rolling out this week are Perplexity's answer to that question , and the 10% cost reduction suggests the efficiency gains from programmable search compound over time as the autoresearch loops keep running.