Arcee's Trinity Mini Gets RL-Trained to Run Real Biomedical Research Workflows

Most AI agents in science are LLMs with a system prompt telling them to use tools. That works until the workflow gets complicated: a protein has three different identifiers, one paper contradicts another, and the model needs to decide what to look up next. A new collaboration between Loka, Arcee AI, Prime Intellect, and AWS actually trained the model to do science rather than prompting it to behave that way.

The result is a post-trained version of Arcee's Trinity Mini, a compact open-weight model fine-tuned through reinforcement learning to execute multi-step biomedical research workflows, coordinate specialized tools, and produce auditable, structured outputs. It now runs inside a full scientific application built by Loka.

A compact model with a demanding job

Trinity Mini is a 26-billion-parameter mixture-of-experts model with only 3 billion active parameters per token, meaning it activates a fraction of its total capacity for any given input, keeping inference fast and cheap. It supports a 128K-token context window across its Nano and Mini variants, enough for long conversations and multi-step workflows without losing coherence, and it was designed for multi-turn agents, tool orchestration, and structured outputs in cloud or on-prem backends.

The team ran 21 controlled post-training experiments using reinforcement learning. The promoted checkpoint, Run 120, is what now powers the deployed agent.

Two training environments, two skills

Training used two distinct RL environments built on Prime Intellect's prime-rl framework, each targeting a different capability:

• Drug Tool RL teaches the model to investigate. Given a biomedical question, it must choose the right tools, recover when retrieval fails, and synthesize a grounded answer. Seven retrieval tools cover PubMed, GEO, KEGG, UniProt, and STRING. When enabled, NVIDIA NIM-backed tools add protein folding, ligand docking, and molecular generation.
• BioReason RL teaches the model to conclude. The model must infer a functional summary and lists of Gene Ontology (GO) identifiers for Molecular Function, Biological Process, and Cellular Component, then return exactly one JSON object. GO is a standardized vocabulary for describing gene and protein function; getting the right terms is essential for downstream bioinformatics pipelines.

The training mixture used both environments in equal proportion, covering investigation and conclusion together.

How the training worked

The RL algorithm is GRPO (Group Relative Policy Optimization). For each prompt, the model generates a group of candidate outputs, each receives a reward score, and advantages are computed relative to the group mean. The model then updates to favor higher-scoring outputs. This is the same technique behind DeepSeek's reasoning models, applied here to tool-using scientific agents.

Rather than fine-tuning all model weights, the team used LoRA adapters: small, trainable matrices inserted into the model that can be swapped without touching the base weights. Prime Intellect's prime-rl separated policy optimization from rollout inference and connected both to the two verifiable environments. The active training context was 16K tokens, with rollouts allowing up to 7,168 completion tokens.

Before RL began, the team ran a prompt optimization pass using GEPA, a reflective prompt evolution tool. Trinity Mini produced base-model rollouts; a Claude Sonnet 5 reflection model used those rollouts and verifier feedback to propose revised task instructions. The selected prompts improved base-model validation by roughly 84% on BioReason and 7.7% on Drug Tool.

21 runs, one variable at a time

The team changed exactly one thing per run, each with a versioned config, a stated hypothesis, and a written decision log. Every run left enough evidence for someone else to reconstruct what changed and why. The final ledger entry for Run 120 recorded a drug_tool_avg1 of 0.812, a bioreason_avg1 of 0.863, and a single word: promote.

Run 120's final scores:

• 81.2% on the held-out Drug Tool evaluation, up from 70.8% at the start of training
• 0.863 on the BioReason composite score, which combines GO term F1, tree similarity, aspect coverage, and JSON validity

Promotion required more than hitting those numbers. Candidates ran inside the scientific application for qualitative workflow testing, checking whether evidence-seeking was purposeful, whether citations and identifiers were usable, whether the model handled ambiguity, and whether its synthesis acknowledged uncertainty.

What the deployed agent actually does

The promoted adapter runs in a companion AI Scientist application built with Strands, FastAPI, and React. Specialist tasks route to the Run 120 adapter; base Trinity Mini handles orchestration and routing where its larger context window is useful.

The harness is a multi-agent system with a clear hierarchy:

• A top-level orchestrator that can pause and ask the user for clarification via ask_user
• Six named specialists below it, two of which coordinate their own sub-specialists
• A dedicated critic agent that stress-tests hypotheses against retrieved evidence
• Slash commands like /plan, /report, and /hypothesize that trigger fixed, reproducible workflows instead of free-form chat

execute_python_code runs in an isolated AWS Bedrock AgentCore sandbox rather than in the host process, keeping arbitrary code execution away from the application itself.

Why the method generalizes

The approach offers a practical route for teams that need specialized model behavior without pretraining a foundation model. Define what useful behavior looks like in a specific workflow. Build held-out tests and inspectable rewards around it. Train a small adapter, review the traces, and deploy it inside an operating boundary the organization controls.

A compact model and a small LoRA adapter can run inside an organization's cloud boundary, stay pinned to a known policy version, and be updated against that organization's own evidence and standards. The team controls the data path, reward function, evaluation, serving cost, and incident response. For regulated industries such as pharma, finance, and legal, that level of control is a hard requirement.

The stack is fully open at every layer: base weights, training code, environments, reward functions, and deployment. Datasets and environments are published on Hugging Face, and the full codebase is on GitHub. The base Trinity Mini model is available on Hugging Face and via the Arcee API at $0.045/$0.15 per million tokens, with a free tier available.

The field has been asking whether small open models can match closed frontier models on specialized scientific work. This collaboration suggests they can, provided the investment goes into proper RL training, verifiable rewards, and domain-specific evaluation rather than a longer system prompt.