---
title: "Nous Research's Atropos Turns a DeepHermes 8B Model 4.6x Smarter at Tool Calling"
subtitle: "NousResearch open-sources Atropos, an async-first RL environment framework that lets you train LLMs across dozens of plug-and-play environments with verifiable rewards."
url: "https://alphasignal.ai/news/nous-research-s-atropos-turns-a-deephermes-8b-model-4-6x-smarter-at-tool-calling"
author: "AlphaSignal Newsroom"
author_url: "https://alphasignal.ai/editorial-team"
company: "Nous Research"
company_url: "https://alphasignal.ai/company/nousresearch"
published: "2026-06-15T16:42:48.000Z"
updated: "2026-06-20T07:48:12.119Z"
type: "Repo"
topics:
  - "Post Training"
  - "Training Infra"
subtopics:
  - "Rl"
  - "Fine Tuning"
  - "Distributed Training"
upvotes: 2813
source_url: "https://github.com/NousResearch/atropos"
comments: 0
truncated: true
---

# Nous Research's Atropos Turns a DeepHermes 8B Model 4.6x Smarter at Tool Calling

> NousResearch open-sources Atropos, an async-first RL environment framework that lets you train LLMs across dozens of plug-and-play environments with verifiable rewards.

[Atropos](https://github.com/NousResearch/atropos) is Nous Research's open-source gym for reinforcement learning with language models. Think of it as the environment layer of an RL training stack: it handles generating model outputs ("rollouts"), scoring them, and feeding that scored data to whatever trainer you want to use. The trainer and inference engine are intentionally not included, so you can plug in your own. The repo has grown to over 1,300 stars and 370 forks, with an active community contributing novel environments ranging from Lean theorem proving to poker strategy.

## The problem it was built to solve

The core challenge is that reinforcement learning with LLMs demands sophisticated asynchronous coordination. Effectively utilizing potentially thousands of GPUs for generating text rollouts, especially when dealing with highly variable completion times across different prompts, demands careful orchestration. Without it, valuable compute resources remain idle, significantly slowing down the learning process.

This need for efficient, scalable rollout management in distributed settings is precisely what led Nous Research to develop Atropos. As the dedicated rollout handler within their pipeline, it is designed to reliably coordinate generation tasks across potentially thousands of distributed workers, interfacing seamlessly with standard inference APIs for straightforward integration.

## How it works under the hood

Atropos is an environment microservice framework for async RL with LLMs. It encompasses both environments, which are set up as services, and a trajectory API for the environments to send data to and for the trainer to pull batches from. Each environment runs as its own process, generates completions by calling an OpenAI-compatible inference endpoint, scores those completions, and pushes the scored data to a central API queue. The trainer polls that queue for batches whenever it is ready, completely decoupled from generation speed.

The default training algorithm used in the example trainer is GRPO (Group Relative Policy Optimization), a policy gradient method that scores a

_Preview only. Full text requires an AlphaSignal Pro account._

## Sources

- [NousResearch/atropos - GitHub](https://github.com/NousResearch/atropos)
- [Introducing Atropos - Nous Research](https://nousresearch.com/introducing-atropos)
- [DeepHermes-ToolCalling-Specialist-Atropos - Hugging Face](https://huggingface.co/NousResearch/DeepHermes-ToolCalling-Specialist-Atropos)
- [Atropos Community Environments README](https://github.com/NousResearch/atropos/blob/main/environments/community/README.md)
- [Atropos FAQ](https://github.com/NousResearch/atropos/blob/main/atroposlib/FAQ.md)