---
title: "UIUC's FOXGLOVE Catches What AI Writing Feedback Gets Wrong"
subtitle: "A new dataset of 2,340 essay comments reveals where LLMs and human writing instructors agree, diverge, and why longer comments inflate AI quality scores"
url: "https://alphasignal.ai/news/uiuc-s-foxglove-catches-what-ai-writing-feedback-gets-wrong"
author: "AlphaSignal Newsroom"
author_url: "https://alphasignal.ai/editorial-team"
company: "AlphaSignal"
company_url: "https://alphasignal.ai/company/alphasignal"
published: "2026-06-04T00:00:00.000Z"
updated: "2026-06-15T00:44:07.808Z"
type: "Paper"
topics:
  - "Data"
  - "Benchmarks"
subtopics:
  - "Datasets"
upvotes: 924
source_url: "https://arxiv.org/abs/2606.06271"
comments: 0
truncated: true
---

# UIUC's FOXGLOVE Catches What AI Writing Feedback Gets Wrong

> A new dataset of 2,340 essay comments reveals where LLMs and human writing instructors agree, diverge, and why longer comments inflate AI quality scores

Every time a student submits an essay and an AI tool spits back feedback, a fundamental question goes unanswered: is that feedback actually doing what good writing instruction is supposed to do? A team from the University of Illinois Urbana-Champaign decided to find out, and the results are more nuanced than the usual "AI is better" or "humans are better" narrative.

The researchers built [FOXGLOVE](https://arxiv.org/abs/2606.06271), a carefully structured dataset that puts human writing instructors and frontier LLMs side by side on the exact same essays, under the exact same instructions, and measures them on the dimensions that writing research actually cares about.

## The gap nobody was measuring

There was no systematic comparison of LLM and expert feedback on the dimensions that writing research identifies as central to revision: goal-orientation, anchoring to specific sentences, and prioritization. That absence is striking given how widely AI writing tools are now deployed in classrooms.

To understand why this matters, it helps to know what these three dimensions mean in practice:

* **Goal-orientation**: Does the feedback target a specific argumentative goal -- like strengthening a claim, adding evidence, or addressing a counterclaim?
* **Sentence anchoring**: Is the feedback tied to a specific sentence in the essay, or is it vague and floating?
* **Prioritization (urgency)**: Does the feedback signal what the student should fix first?

Answering this question at scale requires data that is presently unavailable: there are no datasets that capture trained instructor and LLM generated feedback along the same protocol, anchored to specific text spans, labeled with the goals the feedback addresses, and ranked by urgency. FOXGLOVE fills that gap.

## What they built

FOXGLOVE collects anchored, goal-labeled feedback comments from trained human instructors (696 comments) and four models -- GPT-5.2, Claude Sonnet 4.5, Llama 3.3 70B, and Qwen3-Next 80B -- produced under a shared protocol. The essays themselves are 69 twelfth-grade argumentative essays, giving a total of 2,340 feedback comments across the dataset.

_Preview only. Full text requires an AlphaSignal Pro account._

## Sources

- [FOXGLOVE: Understanding Goal-Oriented and Anchored Writing Feedback from Experts and LLMs on Argumentative Essays](https://arxiv.org/abs/2606.06271)
- [FOXGLOVE Paper Full Text (HTML)](https://arxiv.org/html/2606.06271)
- [LLMs Do Not Grade Essays Like Humans](https://arxiv.org/abs/2603.23714)
- [Human Teacher vs. LLM-Generated Feedback in Secondary Education](https://link.springer.com/chapter/10.1007/978-3-032-03870-8_36)
- [FOXGLOVE GitHub Data Release](https://github.com/yijunliu23/foxglove_data_release)