Skip to main content
Back

CMU's GoodPoint Trains an 8B Model to Beat Gemini at Reviewing Research Papers

GoodPoint trains an 8B model on 19K ICLR papers to generate targeted, actionable paper feedback that authors actually act on — beating Gemini-3-flash in precision.

Pro

Paper, Full Access
  • GoodPoint is a training framework that teaches Qwen3-8B to generate scientific paper feedback that authors actually agree with and act upon, using author responses as the training signal.
  • The team built GoodPoint-ICLR, a dataset of 19,534 ICLR papers (2020–2026) with each reviewer comment labeled for validity and actionability based on author responses.
  • A two-stage training pipeline (SFT on successful feedback + DPO on real and synthetically corrupted preference pairs) yields an 83.7% improvement in predicted feedback success rate over the base Qwen3-8B model.
  • GoodPoint-SFT achieves higher precision than Gemini-3-flash and GPT-5.2 in matching human consensus feedback, despite being a fraction of their size.
  • Key LLM failure modes identified: excessive verbosity (20+ feedback points vs. ~5 for humans), pointing to content already in the paper, and hallucination of critiques.
  • Code, dataset, and trained models will be released upon acceptance; paper available at arxiv.org/abs/2604.11924.

Most LLMs asked to review a research paper will give you something that sounds plausible but lands flat: generic observations, vague suggestions, and critiques that miss the point entirely. A team from Carnegie Mellon University and NVIDIA decided to fix that by asking a deceptively simple question: what does it actually mean for feedback to be good? Their answer is GoodPoint, a training framework that teaches a small 8B model to generate feedback that authors genuinely acknowledge and act upon.

The peer review problem nobody talks about

The scientific community is under pressure. Conferences like ICLR receive tens of thousands of submissions, reviewer bandwidth is stretched thin, and the quality of feedback is notoriously inconsistent. Unchecked automation risks eroding the critical judgment of researchers and the broader scientific community, ultimately degrading the quality of science. But the alternative , doing nothing , leaves junior researchers and non-native English speakers without access to the expert feedback they need most.

Current LLM-generated reviews often lack critical depth and tend to over-positivity. Existing approaches rely on specialized fine-tuning, review standardization, or literature retrieval , but unlike these systems that prioritize mimicking reviewer behavior, GoodPoint focuses specifically on constructive feedback. The distinction matters: a system optimized to sound like a reviewer is not the same as one optimized to actually help an author improve their paper.

What makes feedback "good"?

Before training anything, the team had to define what a good feedback point even is. Their answer is author-centric: a piece of feedback is successful if the author both agrees it is valid and commits to doing something about it. They study constructive feedback generation , the task of producing targeted, actionable feedback that helps authors improve both their research and its presentation , operationalized along two author-centric axes: validity and author action.

  • Validity: Did the author agree the critique was factually correct and relevant? If they rebutted it, the feedback failed.
  • Author action: Did the author commit to revising, deferring to future work, or otherwise acting on the point? A valid critique that prompts no action is still incomplete.