Nous Research's Hermes Agent Browses the Web 60x Faster and 49x Cheaper

Hermes Agent's web extraction is now up to 60x faster and 49x cheaper, thanks to smarter scraping backends and on-demand page paging.

·
·
Nous Research's Hermes Agent Browses the Web 60x Faster and 49x Cheaper
  • 60x faster, 49x cheaper: Hermes Agent's web extraction overhaul uses smarter scraping backends and on-demand page paging to slash latency and cost.
  • Size-aware pipeline: Pages under 5K chars pass through raw; larger pages are LLM-summarized in parallel chunks, capped at ~5K chars of output.
  • Configurable auxiliary model: Route extraction summarization to a cheap model (e.g. Gemini Flash) independently of your main reasoning model to control costs.
  • 8 backend providers: Mix and match search and extract backends — use free SearXNG for search and Firecrawl for extraction, for example.
  • Nous Portal subscribers: Get managed Firecrawl web search and extract with zero API keys via the Tool Gateway.
  • Open source, MIT licensed: Hermes Agent runs locally with no telemetry — 114k GitHub stars and growing fast.

Hermes Agent, the open-source autonomous agent from Nous Research, just shipped a major overhaul to how it reads the web. The headline numbers are striking: up to 60x faster and 49x cheaper web extraction. But the real story is in the engineering decisions that got there.

The problem with reading the web at scale

When an agent browses the web, the naive approach is to fetch a page, dump the full HTML or markdown into the model's context window, and let it figure things out. That works fine for a short blog post. It falls apart fast when you hit a documentation site, a forum thread, or a news article with embedded comments , pages that can balloon to hundreds of thousands of characters.

The old behavior meant two things: slow round-trips because the agent was processing redundant content, and expensive LLM calls because every token in that bloated page costs money. If you're running an agent that browses dozens of pages per task, this compounds quickly.

What changed under the hood

The update targets two bottlenecks directly:

  • Cleaner scraping backends , providers like Firecrawl now pass clean, pre-processed content straight to the agent, cutting out redundant intermediate steps that were adding latency without adding value.
  • On-demand page paging , large pages are saved locally and served to the agent in chunks as needed, rather than being loaded into context all at once. Same quality, fraction of the cost.

The extraction pipeline is size-aware by design. Pages under 5,000 characters are returned as-is with no LLM call at all. Pages between 5,000 and 500,000 characters go through a single-pass summary via an auxiliary model, capped at ~5,000 chars of output. Larger pages up to 2 million characters are chunked into 100k-char segments, summarized in parallel, and then synthesized into a final summary.

The summary keeps quotes, code blocks, and key facts in their original formatting , it's a content compressor, not a paraphraser. If summarization fails or times out, Hermes falls back to the first ~5,000 chars of raw content rather than returning a useless error.

The auxiliary model trick

One of the more practical design choices here is how the summarization model is configured. By default, the web_extract auxiliary task uses your main chat model , same provider, same model. That's fine for most setups, but on expensive reasoning models like Opus or MiniMax M2.7, every long-page extract adds meaningful cost.

You can route extraction summaries to a cheap, fast model independently of your main model:

# ~/.hermes/config.yaml
auxiliary:
  web_extract:
    provider: openrouter
    model: google/gemini-3-flash-preview
    timeout: 360

This means you can run a powerful reasoning model for your main agent loop while offloading the grunt work of page summarization to something much cheaper. The cost savings compound across a long research session.

A flexible backend ecosystem

The update also ships with a fully modular backend system. You can use different providers for search and extract independently , for example SearXNG (free) for search and Firecrawl for extract. Here's the full provider matrix:

ProviderSearchExtractCrawlFree Tier
Firecrawl (default)500 credits/mo
SearXNG,,Free (self-hosted)
Brave Search,,2,000 queries/mo
DuckDuckGo (DDGS),,Free (no key)
Tavily1,000 searches/mo
Exa,1,000 searches/mo
xAI (Grok),,Paid

Paid Nous Portal subscribers get web search and extract through the Tool Gateway via managed Firecrawl , no API key needed. New installs can run hermes setup --portal to log in and turn on all gateway tools at once.

When not to use it

The summarization pipeline is great for most research and information-retrieval tasks, but it has a real limitation: it's lossy by design. If you specifically need raw, unsummarized page content , for example, you're scraping a structured page where the LLM summary would drop important fields , use browser_navigate + browser_snapshot instead. The browser tool returns the live accessibility tree without any rewriting, which is what you want when you need precise structured data rather than a readable summary.

Also worth noting: xAI's Grok web search is LLM-generated rather than index-backed, so titles, descriptions, and URL choice are all model output. That means a maliciously crafted query could in principle steer Grok toward attacker-chosen URLs , something to keep in mind if your agent is processing untrusted input.

Getting started

Setup is straightforward. Run hermes tools, navigate to Web Search & Extract, and pick a backend. The wizard handles the rest. For the zero-config path, Hermes supports Linux, macOS, and WSL2 with no prerequisites, installing everything automatically via a single curl command.

The practical use cases where this update pays off most are:

  • Deep research tasks , agents that browse 20+ pages per session will see the cost savings add up fast
  • Documentation Q&A , large docs sites that previously overwhelmed context windows now get chunked and summarized cleanly
  • Competitive monitoring , recurring cron jobs that scrape and summarize web content on a schedule
  • PDF extraction , web_extract also works with PDF URLs, passing the PDF link directly and converting to markdown text

Hermes Agent is an open-source autonomous AI agent built by Nous Research. It's not a coding copilot tethered to an IDE or a chatbot wrapper around a single API , it lives on your server, remembers what it learns, and gets more capable the longer it runs. All data stays on your machine, with no telemetry, no tracking, and no cloud lock-in. The repo is on GitHub under the MIT license.

Comments

avatar