Skip to main content
Back

PaperCoder Turns ML Research Papers Into Runnable Code for Under $1

Paper2Code's open-source PaperCoder system turns any ML paper PDF into a runnable code repository using a three-stage multi-agent pipeline, for under $1 per paper.

Pro

  • What it is: Paper2Code is an open-source repo shipping PaperCoder, a multi-agent system that turns ML paper PDFs into runnable code repositories.
  • How it works: A three-stage pipeline (planning, analysis, coding) with specialized LLM agents handles each phase sequentially before writing a single line of code.
  • Performance: Scores 45.1% on PaperBench with o3-mini (vs. 16.4% for the next best baseline) and 60.86% with o3; 92% of human judges found outputs genuinely useful.
  • Cost: Roughly $0.50–$0.70 per paper with o3-mini; generated repos need only ~0.81% of lines modified to run successfully.
  • Supports: OpenAI API (o3-mini default) and open-source models via vLLM; accepts PDF or LaTeX source as input.
  • Limitation: Struggles most with data loading due to under-specified dataset details in papers; currently ML-domain only with no figure/diagram understanding.

Paper2Code is an open-source repository that ships PaperCoder, a multi-agent LLM system designed to do something researchers have wanted for years: take a machine learning paper and automatically generate a working code repository from it. The project has pulled in over 4,700 GitHub stars and 660 forks, and it comes backed by a research paper accepted at ICLR 2026.

The motivation is straightforward and well-documented. Despite the rapid growth of machine learning research, corresponding code implementations are often unavailable, making it slow and labor-intensive for researchers to reproduce results and build upon prior work. The researchers behind PaperCoder quantified this gap: they collected accepted papers from recent machine learning venues such as ICLR, ICML, and NeurIPS 2024 , and found that on average only 19.5% of accepted papers at those conferences provide official code. PaperCoder is a direct attempt to close that gap automatically.

A pipeline that thinks before it codes

The core insight is that generating a full repository in one LLM call is a recipe for failure. Instead, PaperCoder operates in three stages: planning, where it constructs a high-level roadmap, designs the system architecture with diagrams, identifies file dependencies, and generates configuration files; analysis, which focuses on interpreting implementation-specific details; and generation, where modular, dependency-aware code is produced. Each phase is instantiated through a set of specialized agents designed to collaborate effectively across the pipeline.

PaperCoder three-phase workflow: Planning, Analyzing, and Coding stages with agent breakdown

The planning stage itself is broken into four sequential sub-steps: an overall plan that extracts core components from the paper, an architecture design that generates class and sequence diagrams, a logic design that determines the exact file execution order, and finally a