---
title: "Princeton and Harvard Catch Popular AI Image Judges Secretly Rewriting Demographics"
subtitle: "A Princeton and Harvard audit finds that the scoring functions guiding text-to-image models sexualize women, whitewash minorities, and collapse diversity during optimization."
url: "https://alphasignal.ai/news/princeton-and-harvard-catch-popular-ai-image-judges-secretly-rewriting"
author: "AlphaSignal Newsroom"
author_url: "https://alphasignal.ai/editorial-team"
company: "AlphaSignal"
company_url: "https://alphasignal.ai/company/alphasignal"
published: "2026-04-14T00:00:00.000Z"
updated: "2026-08-23T21:49:02.248Z"
type: "Paper"
topics:
  - "Image"
  - "Security"
subtopics:
  - "Image Generation"
  - "Red Teaming"
  - "Content Moderation"
upvotes: 2465
source_url: "https://arxiv.org/abs/2604.13305"
comments: 0
truncated: true
---

# Princeton and Harvard Catch Popular AI Image Judges Secretly Rewriting Demographics

> A Princeton and Harvard audit finds that the scoring functions guiding text-to-image models sexualize women, whitewash minorities, and collapse diversity during optimization.

The scoring functions that quietly power modern text-to-image pipelines, ranking images, filtering datasets, and steering fine-tuning, are not neutral judges of quality. A new audit from Princeton and Harvard researchers shows that popular reward models systematically sexualize female subjects, push non-White faces toward White ones, and reward images that conform to real-world occupational stereotypes. The paper, [Bias at the End of the Score](https://arxiv.org/abs/2604.13305), tears into five of the most widely used reward models and finds the same pattern in each.

Reward models sit at almost every stage of a text-to-image system: dataset filtering, evaluation metrics, supervisory signal during parameter optimization, and post-generation safety and quality filtering. If the score itself is biased, every downstream pipeline that leans on it inherits that bias, often in ways the model builders never inspected.

![Diagram showing reward-guided noise optimization producing hypersexualization and demographic changes](https://arxiv.org/html/2604.13305v1/teaserfig_fixed_cropped.png) 

## What the audit actually measured

The team ran two complementary experiments on PickScore, ImageReward, HPS, VQAScore, CLIP, and an Aesthetic score. The first plugged each reward model into the [ReNO framework](https://arxiv.org/abs/2406.04312), which freezes the generator and instead optimizes the initial noise vector fed into a one-step diffusion model to maximize a reward. If a reward truly captured only image quality, the optimizer should sharpen the picture without touching who is in it. That is not what happened.

The second experiment scored counterfactual image sets, matched photos that differ only in gender or race, drawn from CausalFace, SocialCounterfactuals, and PAIRS. The researchers then ran ordinary least squares regressions to see whether reward scores could be predicted from demographic attributes alone, and ranking analyses to see which groups each model preferred.

_Preview only. Full text requires an AlphaSignal Pro account._

## Sources

- [Bias at the End of the Score (arXiv)](https://arxiv.org/abs/2604.13305)
- [Bias at the End of the Score (CVPR 2026 paper)](https://openaccess.thecvf.com/content/CVPR2026/papers/Magid_Bias_at_the_End_of_the_Score_CVPR_2026_paper.pdf)
- [CVPR Poster: Bias at the End of the Score](https://cvpr.thecvf.com/virtual/2026/poster/37725)
- [Full HTML version of the paper](https://arxiv.org/html/2604.13305)