Artificial Analysis Ranks 11 Search APIs by How Well AI Agents Answer
A new leaderboard scores 11 search products from 7 providers on quality, cost, and speed using a shared agent harness and three benchmarks.
- Artificial Analysis launched a Search Index benchmarking 11 search products across 7 providers.
- Parallel advanced tops the board at 75, Exa auto at 74, Firecrawl and Parallel basic tied at 73.
- Every tested provider lifts quality from a model-only baseline of 33 up to 65 to 75.
- Agent runs inside Stirrup, an open-source harness with a 25-turn budget and web_search plus web_fetch tools.
- Blended scoring: DeepSearchQA F1, BrowseComp accuracy, and AA-Omniscience accuracy, equally weighted.
- Higher quality search can reduce total cost by cutting model token usage more than it adds in search fees.
Picking a search API for an AI agent has mostly been vibes work. Every provider claims freshness, ranking quality, and low latency, but there has been no head to head comparison of how those choices actually affect an agent's answers. Artificial Analysis just launched the Search Index, a leaderboard that benchmarks 11 search products across 7 providers by plugging each one into the same agent loop and measuring how well the agent performs.
The lineup covers Parallel, Exa, Firecrawl, You.com, Tavily, Keenable, and Brave. Each provider is paired with the same candidate model, GPT-5.6 Luna (medium), so the only variable in each run is what sits behind the web_search tool.
The harness that levels the playing field
The agent runs inside Stirrup, Artificial Analysis's open-source harness. The harness provides two tools: web_search and web_fetch, and the model has a 25 turn budget to gather information before it must call a finish tool to submit an answer. If the model uses all 25 turns without calling finish, no answer is submitted and the task scores zero.
Everything else is nailed down. Same reasoning effort, same temperature (0.6), up to 10 results per search, a 15 second per page fetch timeout, and contamination filtering that strips known benchmark leaks from URLs, titles, and snippets before the model ever sees them.