Northwestern's DS-IEQA Teaches 8B Models to Grade AI Image Edits Like Humans
A new arXiv preprint proposes DS-IEQA, a framework that teaches multimodal models to both define and score image edits with continuous, distance aware precision.
Takeaways
- Researchers from Northwestern Polytechnical University propose DS-IEQA for image editing quality assessment.
- FDMPO uses logit-level feedback and GPT-4o to iteratively rewrite metric prompts toward implicit human criteria.
- TDRL adds a distance-aware loss on digit tokens, so "3.6" vs "3.9" is no longer treated as equally wrong.
- Fine-tunes Qwen3-VL-8B and InternVL3-8B with LoRA rank 16 on two RTX 4090 GPUs.
- Reports 4th place on 2026 NTIRE X-AIGC Track 2 using only 8B models, no extra training data.
- Ablations show TDRL is the dominant contributor, with up to 12.5 point OOD drops when removed.
Judging whether an AI edit is any good turns out to be harder than judging a single photo. You need to weigh the original image, the edited result, and the instruction all at once, then hand back a numerical score that agrees with a human annotator. A new arXiv preprint from researchers at Northwestern Polytechnical University introduces DS-IEQA, a framework that reworks two weak spots in current MLLM-based scoring: how the evaluation criteria are written, and how numeric scores are learned.
Why grading an edit breaks standard IQA
Traditional Image Quality Assessment (IQA) looks at one picture and returns a score. Image Editing Quality Assessment (IEQA) has to reason jointly over the source, the edit, and the text instruction, and it does so across multiple perceptual dimensions, typically visual quality, editing fidelity, and content preservation. The authors argue that current MLLM-based evaluators run into two structural problems.
- Rigid metric prompting. Human annotators use implicit, black-box criteria. Handwritten prompts like "What is the quality of the edited image?" rarely align with those hidden standards, and MLLMs are notoriously sensitive to phrasing.
- Distance-agnostic scoring. Fine-tuning with cross-entropy treats "3.6" and "3.9" as equally wrong when the truth is "4.0". Soft-label tricks help but blur fine-grained differences, and vocabulary-token distributions need messy post-processing to recover a number.