Perplexity rebuilds search for faster agent workflows

Perplexity has released Photon, a retrieval and ranking engine written from scratch in Rust. The company also added a Fast Search preset to its hosted Search API, giving developers a lower-latency option for agents that make repeated search calls.

Photon handles the underlying retrieval infrastructure, while Fast Search adjusts how much compute the API spends on ranking. Perplexity reports median API latency of 160 milliseconds, a 95th-percentile latency of 230 milliseconds, and a 68% reduction in estimated model-plus-search cost per benchmark task compared with its default preset. Internal retrieval evaluations show a measurable quality trade-off.

The numbers, separated by layer

The p50 figure is the median, meaning half of measured calls completed within that time. The p95 and p99 figures describe progressively slower tail requests. Photon’s 65 ms p99 and Fast Search’s 160 ms p50 measure different layers and percentiles, so they are separate indicators rather than a direct comparison.

Why the old stack stalled

Perplexity previously operated a forked open-source search engine whose storage and indexing model struggled with the company’s production workload. Its dataset grew beyond available RAM, forcing cold reads to trigger major page faults as the operating system pulled data from disk.

Periodic index merges created another source of tail latency. During those operations, p99 response time rose from about 800 milliseconds to roughly 1.2 seconds for intervals lasting 10 to 15 minutes. Provisioning a cluster or recovering a failed node could require more than a week of data synchronization.

A small engineering team, assisted by coding agents, replaced that system with Photon. The new design controls disk access explicitly, stores ranking data more compactly, and moves index construction away from machines serving live queries.

Four choices behind Photon

• Format-aware inverted indexes. An inverted index maps each term to the documents containing it. Photon stores short posting lists inline on a single page and divides longer lists into blocks. Sparse blocks use arrays with galloping search, which skips ahead through ordered document identifiers, while dense blocks use bitmaps. Frequently accessed lists are placed together on disk to reduce reads.
• Compact ranking records. Each document has a “docblob” containing term frequencies, field masks, and term positions. Photon encodes these ordered values with Elias-Fano, a compact representation for increasing integer sequences. A ranker can retrieve data for a query term without decoding the full record or issuing scattered reads across term-document pairs.
• Batched asynchronous I/O. Photon checks an in-memory cache before submitting disk reads through Linux’s io_uring interface. Dedicated reactor threads batch requests and overlap disk waits, giving the engine more control than a memory-mapped design that relies on page faults.
• Budgeted candidate selection. A WAND-style algorithm divides posting lists into driving lists, which propose candidate documents, and probe lists, which contribute additional score data. Score upper bounds remove weak candidates before the engine performs more expensive frequency reads, keeping each query within a defined work budget.

Index builds leave the serving path

Index construction runs on dedicated nodes, separate from machines processing queries. Indexers write versioned, ready-to-serve structures to object storage, and a controller deploys each version to one serving group at a time.

Each group completes a warmup stage based on historical search logs before receiving production traffic. The controller then verifies index state and readiness before restoring the group to routing. This process removes index building from the query path, limits latency spikes during rollouts, and replaces the previous multi-day resynchronization cycle.

Speed trims retrieval quality

Perplexity’s internal evaluations measured lower retrieval scores for Fast Search because the preset spends less compute on ranking. Relevance DCG, where higher values indicate better-ranked results, fell by 0.24 points. Answer availability declined by 2.9 percentage points.

The company’s six public benchmark runs produced comparable aggregate task quality despite those retrieval declines. Perplexity attributes that result to downstream model reasoning, which can recover useful answers from slightly weaker result sets. Workloads with ambiguous queries, limited model reasoning, or strict citation requirements may expose the retrieval gap more clearly.

Picking a preset by workload

An agent that issues dozens of searches compounds both tail latency and cost at every step. Fast Search therefore fits iterative workflows where the model gathers evidence across many calls and can filter or rerank the returned material.

The reported 68% reduction represents estimated model-plus-search cost per completed benchmark task, rather than a universal reduction in API charges. Actual savings depend on call volume, model choice, query complexity, and the number of additional searches triggered by weaker results.

Test the whole agent loop

Teams evaluating Fast Search can replay representative tasks through both presets and compare the complete workflow instead of timing isolated requests. A useful evaluation should cover:

• End-to-end p50, p95, and p99 task latency
• Search calls and model tokens consumed per completed task
• Retrieval relevance, answer coverage, and citation support
• Failure rates on ambiguous, long-tail, and freshness-sensitive queries
• Total model and search cost per successful result

The announced integration is available through the Search API console, with Perplexity operating Photon as hosted infrastructure. Developers select the Fast Search preset at the API level while retaining the default preset for workloads that benefit from its additional ranking compute.