Inside CobbleDB, Perplexity’s faster search storage layer

Perplexity has moved its hot web-content read path from DynamoDB to CobbleDB, a custom key-value store designed around large batches of prepared page records. According to the company’s announcement, median batch-read latency fell from 31.4 ms to 5.60 ms, while p99 latency dropped from 123 ms to 24.2 ms. Internal cost models estimate savings of at least 20%.

Two engineers built roughly 40,000 lines of Rust and completed the surrounding migration in about two months, supported by hundreds of persistent coding agents. The project shows how workload-specific storage and agent-assisted engineering can change the economics of an AI search backend.

Batch reads drove the redesign

Perplexity’s processing pipeline cleans each web page, divides it into semantically coherent passages, computes embeddings, and stores the passages with their vectors. During a search, the retrieval system fetches those prepared records in batches and uses them for ranking and answer generation.

A typical Search API request covers 100 to 120 page keys, divided into smaller groups of 10 to 20 keys. Each record averages about 50 KB, creating a read pattern with several defining characteristics:

• Large batches must complete before ranking can proceed.
• One slow response can extend latency for the entire batch.
• Repeated crawls and embedding upgrades generate heavy write volume.
• Most reads target prepared, medium-sized records rather than individual fields.

DynamoDB’s read and write charges scale with data volume, so continuous crawling and re-embedding created substantial costs. Its managed architecture also limited Perplexity’s control over cache allocation, replica selection, and data placement. A slow replica or cross-zone network hop could therefore increase latency across a complete batch.

Direct writes into the serving database created another source of contention. Reprocessing the corpus with a new chunker or embedding model produced waves of individual updates that competed with live search traffic.

Three layers, one hot path

Perplexity separated durable document state, export processing, and low-latency serving into three components with distinct storage and operational requirements:

• Pillar stores durable, versioned document state on YTsaurus, a distributed data platform running on lower-cost HDD storage. It also tracks page subsets and queues exports.
• Lorry converts exports into partition-aligned batches, then writes those batches to Amazon S3.
• CobbleDB serves the hot read path. Its keys are hashed URLs, and its values contain prepared passages with per-chunk embeddings.

Lorry writes each export batch to S3 under a unique identifier and publishes its metadata to CobbleDB. Partition replicas independently poll the control-plane API, download their next batch, and apply updates in chronological order. A recovering replica can process its backlog without blocking healthy peers or pausing ingestion across the cluster.

Tail latency tuned at every hop

Each CobbleDB data node uses RocksDB, an embedded key-value engine suited to read-heavy systems that ingest data in batches. Frequently accessed records remain in memory, cache misses read from local NVMe storage, and a stateless router maps hashed keys to partitions.

The router prefers replicas in the caller’s availability zone, reducing cross-zone network latency when local capacity is available. When a replica responds slowly, the router sends a hedged request to another copy. Hedging consumes additional replica capacity, but it prevents one straggler from holding up an entire batch. Each node also uses RocksDB’s MultiGet operation to retrieve many keys in one call.

Consistency choices

• Transactions are outside the serving model.
• Batch ingestion makes updates visible asynchronously.
• Replicas may temporarily expose different versions.
• Replica recovery proceeds independently.

Applications that require transactions, synchronized replicas, or immediate read-after-write visibility would need a different consistency model. Perplexity’s retrieval path tolerates brief lag, allowing CobbleDB to remove coordination from latency-sensitive reads.

Production latency falls by about 80%

Perplexity measured both systems under live production traffic of roughly 200,000 requests per second. The company also reports load tests reaching 500,000 requests per second without observed performance degradation.

A p99 latency of 24.2 ms means 99% of measured batch reads completed within that time. This tail metric matters because search ranking waits for batches, making occasional stragglers more damaging than the median alone suggests.

The comparison reflects Perplexity’s record sizes, batch patterns, infrastructure, and consistency requirements. CobbleDB gains its advantage by matching those conditions closely, while the cost figure comes from the company’s internal model rather than an independent benchmark.

Two engineers, hundreds of persistent agents

Perplexity used always-on coding agents that retained project goals, repository history, active risks, and earlier decisions across sessions. Their work extended beyond code generation into the continuous coordination surrounding implementation and deployment:

• Auditing project channels and linking open work to pull requests
• Reviewing application and infrastructure changes
• Preparing fixes, tests, and follow-up patches
• Tracking continuous-integration gates
• Monitoring deployments, restores, and recovery exercises

The two engineers set the architecture, reviewed consequential changes, and authorized production operations. Agents handled much of the follow-through between those decisions, helping the team maintain momentum across code review, testing, migration, and rollout.

Where CobbleDB fits

CobbleDB’s design matches retrieval systems with repeated batch reads, medium-sized records, asynchronous bulk updates, and tolerance for brief replica lag. The approach becomes more attractive when traffic is large enough for managed-service charges and tail latency to justify dedicated infrastructure.

Operating a specialized store also shifts responsibility to the engineering team. Adopters must manage capacity, partitioning, RocksDB compaction, replica health, recovery, deployment safety, and the extra load generated by hedged reads.

Perplexity says it plans to release CobbleDB as open source through Perplexity’s GitHub. The announcement does not specify a release date or license, so external teams cannot yet evaluate the implementation or deploy it directly.