Nous Research's Hermes Agent Cuts Tool Token Overhead by 50% With Smart Loading
Hermes Agent's new Tool Search feature stops MCP tool schemas from eating your context window, boosting accuracy by up to 25 points

- Hermes Agent ships Tool Search, a lazy-loading layer that replaces MCP tool schemas with three bridge tools and loads definitions on demand.
- Anthropic evals show the approach boosts accuracy from 49% to 74% on Claude Opus 4 and from 79.5% to 88.1% on Opus 4.5 with large tool catalogs.
- Real-world deployments with 5 MCP servers and 34 tools see ~22,000 tokens per turn (50% of prompt) consumed by tool schemas alone -- Tool Search targets this directly.
- Retrieval uses BM25 over tool names, descriptions, and parameter names; auto mode only activates when deferred schemas exceed 10% of the context window.
- Core Hermes tools (terminal, browser, file ops, etc.) are never deferred -- only MCP and non-core plugin tools are eligible.
- Available now via
hermes update; full docs at hermes-agent.nousresearch.com.
If you've ever connected more than a handful of MCP servers to an AI agent, you've felt the pain: every tool's JSON schema gets shoved into the model's context on every single turn, whether the model needs it or not. When you connect multiple MCP servers to an agent, every tool's JSON schema gets sent to the model on every turn, even if the model only needs one or two tools for a given task. Nous Research just shipped a fix for this in Hermes Agent, and the numbers behind it are hard to ignore.
The tax you've been paying without knowing it
The scale of the problem is bigger than most people realize. A Hermes deployment with five MCP servers and 34 tools shows average prompt sizes of 45,000 tokens per turn, with roughly 22,000 of those tokens -- around 50% -- being tool schema overhead alone. And that's a modest setup. Anthropic's own engineering data shows tool definitions can consume 134,000 tokens before optimization, with the "MCP Tools Tax" measured at 15,000 to 60,000 tokens per turn for typical multi-server deployments.
Beyond token cost, there's an accuracy problem. Large tool catalogs cause decision paralysis, and removing irrelevant schemas from context reduces false positives. Anthropic's evals show Claude Opus 4 improved from 49% to 74% accuracy, and Opus 4.5 improved from 79.5% to 88.1%, with Tool Search enabled on large tool catalogs. That's not a marginal improvement -- it's the difference between an agent that works and one that doesn't.
Progressive disclosure, not brute force
Tool Search is Hermes' opt-in progressive-disclosure layer for this problem. When activated, MCP and plugin tools are replaced in the model-visible tools array by three bridge tools, and the model loads each specific tool's schema on demand. Think of it like lazy loading for tool definitions: the agent only pays for what it actually uses.
The three bridge tools the model sees instead of the full catalog are:
tool_search(query, limit?)-- searches the deferred-tool catalog by natural language querytool_describe(name)-- loads the full JSON schema for a specific tooltool_call(name, arguments)-- invokes the deferred tool
A typical interaction for a GitHub task looks like this:
Model: tool_search("create a github issue")
→ { matches: [{ name: "mcp_github_create_issue", ... }] }
Model: tool_describe("mcp_github_create_issue")
→ { parameters: { type: "object", properties: { ... } } }
Model: tool_call("mcp_github_create_issue", { title: "...", body: "..." })
→ { ok: true, issue_number: 42 }When the model invokes tool_call, Hermes unwraps the bridge and dispatches the underlying tool exactly as if the model had called it directly. Pre-tool-call hooks, guardrails, approval prompts, and post-tool-call hooks all run against the real tool name, not against tool_call. Your existing safety and observability setup stays intact.
Smart enough to stay out of the way
The feature ships in auto mode by default, which means it only activates when it actually helps. By default, Tool Search activates only when the deferrable tool schemas would consume at least 10% of the active model's context window. Below that threshold, it's a pure pass-through with zero overhead.
This means:
- A session with a few MCP tools and a large-context model may never trigger it at all
- A session with many MCP servers attached (15+ tools typically) starts activating it
- Removing servers mid-session correctly returns to direct tool exposure on the next turn
One important carve-out: Hermes' core tools -- terminal, read_file, write_file, browser_*, web_search, execute_code, delegate_task, and the rest of the core set -- are always loaded directly. Only MCP tools and non-core plugin tools are eligible for deferral.
The retrieval itself uses BM25 over tokenized tool name, description, and parameter names, with a substring fallback for zero-IDF edge cases -- the situation where every tool in the catalog shares a common keyword like "github". It's a stateless catalog that rebuilds every turn from the live tool registry, so it never drifts out of sync.
Configuration in 5 lines
Enabling or tuning Tool Search is a single block in your hermes.yaml:
tools:
tool_search:
enabled: auto # auto (default), on, or off
threshold_pct: 10 # % of context at which auto mode kicks in
search_default_limit: 5
max_search_limit: 20To update Hermes and get the feature, it's just hermes update from the CLI. Hermes Agent is a native app available for macOS, Windows, and Linux.
Where it falls short
Tool Search isn't free. The bridge tools themselves cost around 300 tokens, and there's at least one extra round trip (search, then describe, then call) the first time the model reaches for a deferred tool. The docs are honest about the trade-offs that don't go away:
- Cold tool penalty: the first time a deferred tool is needed, the model spends one or two extra inference calls to find and load the schema
- No system-prompt cache benefit: loaded
tool_describeresults enter conversation history but don't benefit from the system-prompt cache prefix - Model quality dependence: Anthropic's internal MCP evals went from Opus 4 49% to 74% and Opus 4.5 79.5% to 88.1% with Tool Search enabled, but that still means roughly 26 points of retrieval failure even on the best models. Smaller models do this less reliably.
When to use it (and when not to)
The auto default handles the decision for most setups, but here's the mental model:
- Good fit: 15+ tools attached, few tools used per turn, multiple MCP servers connected
- Bad fit: small toolsets where the 300-token bridge overhead exceeds the savings, or workflows that call nearly every tool on every turn
The broader context matters here too. As soon as agents hit 20 to 30 tools, token costs become untenable and selection accuracy plummets -- many teams have been attempting to build their own workarounds with RAG pipelines, only to hit performance walls. Tool Search is a principled, production-ready answer to that problem baked directly into the agent runtime, with no external retrieval infrastructure to maintain. Hermes Agent is built by Nous Research, the lab behind the Hermes, Nomos, and Psyche model families.