Skip to main content
Back

Amazon's Web Search Agent Beats GPT-5.6 Sol on Search Arena

Amazon's custom-trained search agent tops the Search Arena leaderboard, built on AgentCore Web Search with a knowledge graph and Amazon-operated web index.

  • Amazon-Web-Search-Agent takes #1 on Search Arena with an Arena score of 1,266.
  • Custom model post-trained on SageMaker HyperPod EKS using an open-weight base model.
  • Claude Opus 4.6 paired with Amazon Web Search scored even higher at 1,273 off-leaderboard.
  • Backed by an Amazon-operated index of tens of billions of docs plus a knowledge graph.
  • Available via AgentCore MCP connector or direct InvokeWebSearch API at $7 per 1,000 queries.
  • Queries stay inside AWS, removing third-party search API keys and egress concerns.

Amazon-Web-Search-Agent reached first place on Search Arena, a crowdsourced leaderboard where people compare search-augmented model responses through blind voting. The system combines a post-trained open-weight model with Amazon’s web search service, delivered through Amazon Bedrock AgentCore.

The result gives developers a useful signal about Amazon’s retrieval infrastructure. In the leaderboard snapshot published by the company, its agent ranked above GPT-5.6 Sol on questions that require current web information. Developers can access the underlying search service today, although AWS has not announced general availability for the custom agent itself.

One index, three model pairings

The public entry uses a model that Amazon further trained after its initial development to decide when to search, construct queries, and apply retrieved passages. Amazon also tested its search service with Claude Opus 4.6 and GPT-5.6 Sol outside the public leaderboard.

System Evaluation status Arena score
Claude Opus 4.6 with Amazon Web Search Amazon test, off leaderboard 1,273
Amazon-Web-Search-Agent Public leaderboard entry 1,266
GPT-5.6 Sol with Amazon Web Search Amazon test, off leaderboard 1,252

Those comparisons suggest that Amazon’s index can support several high-performing models. They do not isolate the search service’s contribution because Amazon did not publish equivalent scores for each model using a different retrieval provider or no web access.

How the stack works

Amazon post-trained the agent with SageMaker Training Jobs running on SageMaker HyperPod EKS nodes. The company describes the base as an open-weight model but has not disclosed its identity, training data, post-training recipe, or evaluation sample sizes.

The retrieval service uses an Amazon-operated index containing tens of billions of documents, according to AWS. The company says it refreshes the index continuously and supplements it with a knowledge graph for factual queries. Searches run against Amazon’s infrastructure instead of a wrapper around Google or Bing.

Passage extraction limits how much irrelevant page content reaches the model. The service selects excerpts from retrieved documents, reducing the tokens consumed by navigation, boilerplate, and unrelated text. That design can lower inference cost and leave more context-window capacity for reasoning and tool output.

Two routes into the index

AWS exposes the service through an AgentCore connector and the direct InvokeWebSearch API. Both routes use the same Amazon search infrastructure but fit different application architectures.

AgentCore Gateway

Through AgentCore Gateway, developers attach a Web Search Tool target with connectorId set to web-search. An agent can then discover the connector as an MCP tool and invoke it through tools/call.

  • Queries use natural language and may contain up to 200 characters.
  • Connector version 1.2.0 and later supports request-level domain filters.
  • The same connector versions support publication-date filters.

Direct API

Applications that do not need an AgentCore Gateway can call InvokeWebSearch on the bedrock-agentcore data plane. A minimal AWS CLI request looks like this:

aws bedrock-agentcore invoke-web-search \
  --web-search-id web-search.v1 \
  --search-query "latest AI breakthroughs this week"

AWS lists the search price at $7 per 1,000 queries, equivalent to $0.007 per query, in supported regions. Model inference, gateway use, data transfer, and other AWS services may add separate charges.

Search without a third-party hop

AWS says AgentCore processes search queries within its infrastructure and does not send them to an external search engine. For teams already operating on AWS, that design can simplify several parts of an agent deployment:

  • IAM can control access without a separate search-provider credential.
  • Search usage appears within the existing AWS billing relationship.
  • Requests avoid direct integration with an external search API.
  • The retrieval layer works with multiple models instead of locking the application to Amazon’s custom agent.

Amazon developed the index using infrastructure related to search features in Alexa+, Amazon Q Business, and Kiro. That history provides an established source of query and retrieval experience, although developers still need workload-specific tests for freshness, geographic coverage, and specialized domains.

Where the leaderboard stops

Search Arena records blind user preferences and does not independently verify factual accuracy. Its standings can change as votes accumulate, prompts shift, and new systems enter. The score therefore measures which answer voters preferred under Arena conditions.

Amazon’s off-leaderboard Claude pairing scored seven points above the company’s public agent entry. Teams already using a capable model may gain more from testing Amazon Web Search as a retrieval tool than from waiting for access to Amazon’s custom model.

Production evaluations should cover factual accuracy, source quality, latency, index freshness, filter behavior, regional availability, and total cost. The 200-character query limit may also require applications to compress long user requests before searching. Amazon’s leaderboard result offers evidence that the product can compete on live-web answers, while the callable API lets developers measure that performance against their own queries and requirements.