Northeastern's CATT Benchmark Exposes Why AI Data Attribution Misses Its Sources
A new benchmark called CATT shows why data attribution methods fail, and two simple fixes boost recall by 35 to 72 points.
- CATT benchmark fine-tunes Qwen3.5-4B on fictitious facts for unambiguous attribution ground truth: project page.
- Redundancy and lexical distractors consistently break gradient-based attribution; TRAK falls to zero recall with distractors.
- Causal evidence in a document is localized to a few tokens; averaging over all tokens dilutes it to noise.
- Cross-entropy gradients suppress signal from confidently predicted answer tokens, hiding well-learned facts.
- Switching to negative log-odds plus top-10 token aggregation lifts recall by 35 to 72 points.
- Fixes generalize to FTRACE-TREX, improving counterfactual retraining impact by up to 4.9x across methods.
Language-model data attribution ranks training records by their estimated influence on a model’s answer. Developers can use those rankings to investigate memorization, audit data provenance, and identify examples that may require removal or correction. Yet popular methods often miss known source documents even when benchmark designers know exactly which records contain the answer.
A Bau Lab paper from Northeastern University isolates two causes: the loss function used to calculate gradients and the method for combining token scores into a document score. The researchers also introduce the Controlled Attribution Benchmark, or CATT, which separates these design choices from confounding properties of real-world corpora.
Known facts, known sources
Real-world attribution evaluations mix duplicate documents, lexical lookalikes, paraphrases, and uncertain provenance. When a method fails, those overlapping variables make the cause difficult to identify. CATT creates fictitious knowledge and changes one dataset property at a time, giving each target fact a known training source.
The researchers fine-tuned Qwen3.5-4B on two synthetic tasks, then added controlled redundancy and distractors to test retrieval under progressively harder conditions.
| Component | Construction | Purpose |
|---|---|---|
| Function Bindings | New functions always return an assigned value. | Tests whether attribution retrieves the records that taught a specific input-output mapping. |
| Routing Rules | Invented rules sort objects according to their properties. | Tests attribution on facts that require applying a learned rule. |
| Redundancy | The same fact appears across N documents written in varied styles. | Measures whether influence becomes fragmented across repeated evidence. |
This story is for Pro members
You've reached the end of the free preview. Upgrade to AlphaSignal Pro to read the full article - and everything else behind the paywall.
Comments
