---
title: "Tokyo Researchers Beat LoRA Fine-Tuning With a Simple Linear Regression on Frozen VLMs"
subtitle: "Researchers show simple linear regression on frozen VLM hidden states beats LoRA fine-tuning for predicting individual aesthetic preferences on photos and artworks."
url: "https://alphasignal.ai/news/tokyo-researchers-beat-lora-fine-tuning-with-a-simple-linear-regression-on"
author: "AlphaSignal Newsroom"
author_url: "https://alphasignal.ai/editorial-team"
company: "AlphaSignal"
company_url: "https://alphasignal.ai/company/alphasignal"
published: "2026-04-13T00:00:00.000Z"
updated: "2026-08-24T02:26:57.244Z"
type: "Paper"
topics:
  - "Image"
  - "Llms"
subtopics:
  - "Vision Language"
  - "Vision Language"
  - "Hallucinations"
upvotes: 1360
source_url: "https://arxiv.org/abs/2604.11374"
comments: 0
truncated: true
---

# Tokyo Researchers Beat LoRA Fine-Tuning With a Simple Linear Regression on Frozen VLMs

> Researchers show simple linear regression on frozen VLM hidden states beats LoRA fine-tuning for predicting individual aesthetic preferences on photos and artworks.

Predicting which image a specific person will find beautiful sounds like a job for heavy machinery: fine-tune a big vision-language model on that user's ratings, or engineer a specialized network with meta-learning tricks. A new paper from the University of Tokyo argues you can skip almost all of that. The authors show that modern VLMs already contain rich, multi-level aesthetic information in their hidden states, and that a plain linear regression on top of those frozen activations outperforms LoRA fine-tuning and few-shot prompting for personalized image aesthetics assessment (PIAA).

The paper, ["What Do Vision-Language Models Encode for Personalized Image Aesthetics Assessment?"](https://arxiv.org/abs/2604.11374) by Koki Ryu and Hitomi Yanaka, tackles a task where [models predict the aesthetic score a specific user would assign to an image](https://arxiv.org/html/2604.11374), reflecting personal taste rather than an average rating. The full pipeline is open-sourced at [github.com/ynklab/vlm-latent-piaa](https://github.com/ynklab/vlm-latent-piaa).

## Why existing PIAA pipelines struggle

Traditional PIAA methods lean on multi-stage recipes: pretrain on large-scale generic aesthetics datasets, then adapt to each target user, often with meta-learning or attribute-interaction modules. Those pipelines are expensive and, critically, tend to overfit to one image domain. A model trained on photographs rarely transfers cleanly to artworks. VLMs are an attractive alternative because they see huge amounts of text and images during pretraining, but until now evaluations of their aesthetic perception were limited to multiple-choice or binary formats, and no prior work had used them for individual-level PIAA.

![Diagram of the Linear-Hidden PIAA pipeline showing an image passing through a VLM, hidden representations extracted, and user-specific linear estimators producing personalized scores](https://arxiv.org/html/2604.11374v1/main-diagram.png) 

## Probing what the model already knows

_Preview only. Full text requires an AlphaSignal Pro account._

## Sources

- [What Do Vision-Language Models Encode for Personalized Image Aesthetics Assessment?](https://arxiv.org/abs/2604.11374)
- [vlm-latent-piaa GitHub repository](https://github.com/ynklab/vlm-latent-piaa)
- [Paper HTML on arXiv](https://arxiv.org/html/2604.11374)
- [LAPIS: a novel dataset for personalized image aesthetic assessment](https://github.com/Anne-SofieMaerten/LAPIS)
- [Qwen3-VL collection](https://huggingface.co/collections/Qwen/qwen3-vl)