Skip to main content
Back

CUHK Researchers Fingerprint Nine LLMs With Psychological Tests Across 20 Dimensions

A cross-linguistic study runs nine frontier LLMs through seven personality tests and finds each model has a stable, identifiable behavioural fingerprint.

Paper, Full Access
  • Nine frontier LLMs profiled with seven human psychology instruments in Chinese and English, five repeats each.
  • Model-associated variation found in 19 of 20 dimensions; profiles reproducible enough to recover model identity.
  • Shared alignment pattern: high prosocial and self-regulatory scores, low dominance and harmful-intent endorsement.
  • Kimi-K2 shows highest dominance and anti-egalitarianism; largest pairwise contrasts involve Kimi and Doubao.
  • NA (unanswerable) responses are structured, marking where models refuse or reject human framing.
  • Paper on arXiv; no public code or dataset release announced.

Researchers at the Chinese University of Hong Kong, Shenzhen administered seven psychological questionnaires to nine language models five times in both English and Chinese. Each model produced a repeatable pattern of scores and unanswered items, with statistically reliable differences appearing in 19 of the 20 measured dimensions. The study paper calls these patterns behavioral signatures.

For developers, those signatures could support model comparison, version regression testing, refusal analysis, and limited attribution of anonymized outputs. The profile characterizes a configured endpoint under specific prompts, so system messages, fine-tuning, sampling settings, language, and model updates may change it.

From personality claims to output patterns

Psychometric questionnaires ask people to rate statements about their habits, beliefs, and preferences. Applying the same instruments to LLMs creates a measurement problem because generated answers depend on prompt wording, option order, language, and deployment settings. Models also lack the personal history and social identity assumed by many questionnaire items.

Prior reliability research has shown that naive self-assessment tests provide unstable evidence about an LLM’s personality. The new study adopts a narrower target: whether repeated administrations produce structured response patterns that remain stable within a model and differ across models.

Inside the 155-item battery

The seven instruments contain 155 items and yield 20 scored dimensions spanning personality, moral judgment, cognitive style, social orientation, and harmful impulses.

Instrument Items Measured area
IPIP BFFM-50 50 Big Five personality dimensions