UIUC's DRPG Beats Human Researchers at Writing Academic Rebuttals
A four-stage agentic pipeline breaks reviews into atomic concerns, retrieves evidence, plans strategies, and generates rebuttals that beat human averages using an 8B model.
PRO- DRPG is a four-stage agentic pipeline for automatic academic rebuttal: Decompose, Retrieve, Plan, Generate.
- The Planner scores candidate perspectives against retrieved paragraphs and hits 98.64% accuracy picking the human-preferred angle.
- Using an 8B Qwen3 model, DRPG beat real human rebuttals with a 60.65% win rate on Re2 data.
- Retriever cuts input length by roughly 75% by pulling only the paragraphs relevant to each atomic review point.
- DRPG continues improving across multi-round reviewer-author simulations while baselines plateau after round one.
- Code is open source at github.com/ulab-uiuc/DRPG-RebuttalAgent.
Writing a good rebuttal to peer reviewers is one of the more painful rituals of academic life. It requires reading a long paper, picking apart each critique, and constructing arguments that are both accurate and persuasive, often under a tight deadline. A team at the University of Illinois Urbana-Champaign has now built an agentic system called DRPG that automates this workflow, and their experiments suggest it can outperform the average human rebuttal using only an 8B model.
Why off-the-shelf LLMs choke on rebuttals
Major conferences such as NeurIPS and ICLR received over 25,000 submissions in 2025, and each of those papers typically triggers multiple reviews that authors have to answer point by point. Prior attempts to automate this have mostly leaned on zero-shot prompting or ad-hoc pipelines, which run into two structural problems.
The first is long context. Academic papers are lengthy and information-dense, and LLMs struggle to identify and extract the most relevant evidence when responding to specific reviewer concerns, a failure mode known as the "lost in the middle" effect. The second is persuasion. Because LLMs are not explicitly trained to persuade, they tend to produce responses that are overly generic, excessively conciliatory, or defensive, failing to convincingly address reviewers' key concerns.
Four agents, one rebuttal
DRPG stands for Decompose, Retrieve, Plan, Generate. Each stage is handled by a dedicated component that hands structured output to the next one.
- Decomposer: An LLM that splits a review into atomic points, each capturing one weakness or question the reviewer raised.
- Retriever: Uses dense retrieval, where a text encoder embeds both review points and paper paragraphs and cosine similarity measures their relevance. Only the most relevant paragraphs are passed to the executor, increasing information density while cutting content length by around 75% in practice. Paragraphs are encoded with their section headers attached, so hierarchical context is preserved.
This story is for Pro members
You've reached the end of the free preview. Upgrade to AlphaSignal Pro to read the full article - and everything else behind the paywall.
Comments
