Cohere's Rerank 5 Cuts AI Search Token Use by 41% While Boosting Accuracy

Cohere has released Rerank 5 in two versions: Pro, optimized for retrieval quality, and Fast, optimized for throughput and latency. Both models add Context Focus, which returns the passages that answer a query alongside document-level relevance scores. For retrieval-augmented generation and search agents, the release targets three recurring costs: irrelevant context, slow reranking, and unreliable score thresholds.

A reranker runs after keyword, vector, or hybrid search. It examines the initial set of candidate documents, scores each one against the query, and sends the strongest results to an application or language model. Better reranking can improve answers without replacing the underlying search index.

Pro raises the bar on harder queries

According to Cohere’s published evaluation, Rerank 5 Pro led the tested models on ViDoRe V3, nanoMTEB, BRIGHT, and BEIR. It ranked first on 39 of 46 composite datasets and placed first or second on 43.

BRIGHT tests reasoning-intensive retrieval, where relevant documents may share little vocabulary with the query. That makes it useful for evaluating multi-step agent searches, which often require the model to connect concepts and evidence beyond direct keyword or semantic matches.

Rerank 5 Fast outperformed Voyage AI Rerank 3 Lite, Qwen3-Reranker-0.6B, Qwen3-Reranker-8B, and Rerank 4 Pro across the four reported benchmarks. On an NVIDIA A100, it reranked about 187 documents of 1,000 tokens each per second. Cohere reports 2.9 times the throughput of Pro and response times that are roughly three times faster.

Context Focus trims the prompt

Context Focus identifies and scores relevant passages within each document. A single Rerank API request can therefore return document rankings and the text spans most likely to help answer the query. Developers can adjust the inclusion threshold and cap the number of returned spans, controlling how much evidence reaches the downstream model.

Cohere reports that removing half of the retrieved context reduced answer quality by 2 percentage points, retaining 96% of the original quality. Removing three-quarters reduced it by 7 points, retaining 87%. In a full search-agent evaluation, Rerank 5 Pro increased accuracy from 46% to 49%; enabling Context Focus then reduced token use while improving accuracy further. On harder deep-search benchmarks, the feature used 41% fewer tokens and raised accuracy from 39% to 52%, with every evaluated dataset improving on both measures.

Developers enable Context Focus by adding a relevant_spans object to the existing rerank request. The threshold controls span inclusion, while max_spans limits the number returned:

import os
import requests

response = requests.post(
    "https://api.cohere.com/v2/rerank",
    headers={"Authorization": f"Bearer {os.environ['CO_API_KEY']}"},
    json={
        "model": "rerank-v5.0-pro",
        "query": query,
        "documents": documents,
        "top_n": 2,
        "relevant_spans": {
            "mode": "enabled",
            "threshold": 0.4,
            "max_spans": 10,
        },
    },
    timeout=30,
)

response.raise_for_status()
result = response.json()

Calibrated scores simplify control flow

Rerank 5 returns calibrated relevance_score values designed to represent the probability that a document will be useful. With a well-calibrated model, documents assigned a score near 0.7 should prove useful roughly 70% of the time. Stable probabilities make application thresholds easier to set and reduce the need to retune them after changing models or datasets.

Cohere reports an expected calibration error of about 0.06 on each of two judgment sets, with lower values indicating closer agreement between predicted probabilities and observed usefulness. Every other tested reranker had nearly four times as much error or more. Documents that Voyage AI Rerank 3 scored near 0.55, for example, were judged useful only 24% of the time.

Calibrated scores support several practical retrieval patterns:

• Adaptive context size: Pass every result above a fixed threshold, allowing narrow queries to return fewer documents and broad queries to return more.
• No-answer detection: Route queries to rewriting, human review, or an abstention path when no result clears the threshold.
• Shared cutoffs: Reuse similar thresholds across domains and model sizes. Across BRIGHT, NanoBEIR, and ViDoRe V3, Cohere reports Pro cutoffs between 0.47 and 0.53, with Pro and Fast staying within 0.05 of each other.

Adoption does not require a new index

Rerank 5 Pro covers finance, healthcare, scientific research, multilingual retrieval, and content extracted from PDFs. In Cohere’s internal multilingual evaluation, Pro ranked first in 14 of 19 non-English languages. On the evaluated PDF retrieval suite, it achieved an average score of 89.9, ahead of Voyage AI Rerank 3 at 88.4 and Rerank 4 Pro at 88.3.

Cohere lists pricing at $0.05 per 1 million tokens for Pro and $0.02 for Fast. Both models are generally available through the Cohere API, Model Vault, Microsoft Foundry, Amazon SageMaker, and private VPC or on-premises deployments.

Because reranking occurs after initial retrieval, teams can keep their existing keyword, vector, or hybrid search systems and avoid re-indexing. Cohere also provides integrations for LangChain, Haystack, Elasticsearch, and Weaviate.

Existing Rerank users can adopt the release by changing the model name, then enable Context Focus when they want passage-level output. The Fast model suits high-volume pipelines, while Pro targets workloads where retrieval accuracy carries more weight. Calibrated scores and relevant spans provide the larger architectural change by giving agents tighter control over which evidence enters each prompt.