---
title: "Oregon State's EvoPool Beats GPT-4o Annotation at 4,500x Less Cost"
subtitle: "EvoPool uses Darwinian evolution to breed Python annotation functions that label 100K examples 31,000x faster than GPT-4o while beating LLM annotation on 7 of 8 specialized tasks"
url: "https://alphasignal.ai/news/oregon-state-s-evopool-beats-gpt-4o-annotation-at-4-500x-less-cost"
author: "AlphaSignal Newsroom"
author_url: "https://alphasignal.ai/editorial-team"
company: "AlphaSignal"
company_url: "https://alphasignal.ai/company/alphasignal"
published: "2026-06-01T00:00:00.000Z"
updated: "2026-06-09T08:38:40.699Z"
type: "Paper"
topics:
  - "Data"
  - "Post Training"
subtopics:
  - "Synthetic Data"
  - "Data Curation"
  - "Fine Tuning"
upvotes: 1261
source_url: "https://arxiv.org/abs/2606.01617"
comments: 0
truncated: true
---

# Oregon State's EvoPool Beats GPT-4o Annotation at 4,500x Less Cost

> EvoPool uses Darwinian evolution to breed Python annotation functions that label 100K examples 31,000x faster than GPT-4o while beating LLM annotation on 7 of 8 specialized tasks

Getting training labels for specialized domains is brutally expensive. Biomedical relation extraction, legal clause classification, scientific claim verification , these tasks require clinicians, lawyers, or domain reviewers, and the cost routinely runs into the hundreds of thousands of dollars per dataset. The standard workaround, using a large language model to label your training data, sounds appealing but falls apart in practice: on the ChemProt biomedical benchmark, GPT-4o-mini annotation achieves only 0.28 macro-F1\. A paper published on arxiv by Tianyi Xu, Yaolun Zhang, Xuan Ouyang, and Huazheng Wang from Oregon State University and the University of Wisconsin-Madison proposes a different approach: instead of asking an LLM to label examples one by one, evolve a pool of executable Python annotators that can do it for free.

## The problem with LLM annotation at scale

There are two natural ways to use LLMs for data labeling. The first is direct annotation: send each training example to the LLM and collect its label. This works reasonably well on general tasks but fails on specialized domains where the LLM lacks deep expertise. It also costs money and time that scale linearly with your corpus , annotating 100K examples with GPT-4o-mini takes 21.9 hours and $2.95\. The second approach, used by tools like Alchemist and DataSculpt, asks the LLM to write a pool of executable labeling functions (short Python programs that classify text) in a single shot. These functions run cheaply at inference, but they inherit whatever biases the LLM had before seeing your data. They collapse on rare classes and never adapt to what the model actually gets wrong.

EvoPool attacks both failure modes at once. The core idea is to treat the pool of annotators as a population that evolves under selection pressure from a small labeled validation set , the only ground-truth labels the system ever sees.

## Darwinian annotation: how it works

[The paper](https://arxiv.org/abs/2606.01617) frames annotation as an evolutionary search problem. Three role-specialized LLM agents act as mutation operators, and a deterministic selection gate acts as natural selection. Each generation, the agents propose new Python annotator functions; the gate keeps only those that pass three filters:

_Preview only. Full text requires an AlphaSignal Pro account._

## Sources

- [EvoPool: Evolutionary Programmatic Annotation for Label-Efficient Specialized Supervision](https://arxiv.org/abs/2606.01617)
- [EvoPool HTML Paper](https://arxiv.org/html/2606.01617)
- [EvoPool GitHub Repository](https://github.com/tianyi0216/EvoPool)