Skip to main content
Back

Fudan Researchers Prove 28 LLMs Only Pretend to Think Like Humans

A new study of 28 LLMs reveals that human-like risk decisions can be a statistical illusion, not genuine behavioral alignment

Paper, Full Access
  • A study of 28 LLMs finds that human-like risk decisions are often surface mimicry, not genuine behavioral alignment.
  • The St. Petersburg game (infinite expected value, low human bids) is used as a controlled testbed to separate outcomes from mechanisms.
  • Most models bid finite amounts in the base game, but shift to computationally rational behavior when game parameters are structurally changed.
  • Human-cue prompting and instruction tuning lower bids cosmetically but leave underlying decision mechanisms largely unchanged.
  • The paper calls for mechanism-level evaluation in all high-stakes LLM decision-support applications (finance, insurance, medical triage).
  • No code is released, but the prompt suite and methodology are detailed enough to replicate with any LLM API.

When an AI model declines to bet everything on a long-shot gamble, it looks cautious. It looks human. But a new paper from researchers at Fudan University and the University of Rochester argues that looking human and being human-like are two very different things -- and that most LLM evaluations cannot tell them apart.

LLMs can appear cautious in risk decision-making tasks, yet cautious-looking outputs do not necessarily indicate alignment with human decision-making mechanisms. To stress-test this gap, the team turned to one of the oldest puzzles in decision theory: the St. Petersburg game.

A 300-Year-Old Trap for AI

The St. Petersburg game is deceptively simple. A fair coin is flipped until it lands heads up. The player wins $2^n$ dollars, where n is the number of flips until the coin lands heads up. The math says the expected payout is infinite -- you should theoretically pay any amount to play. Since this payoff has an infinite mean, for a risk-neutral player the fair price of the lottery should be infinite; however, few people are willing to pay more than $20 to play this game, thus the paradox.

The St. Petersburg paradox is the oldest paradox in decision theory and has played a pivotal role in the introduction of increasing concave utility functions embodying risk aversion and decreasing marginal utility of gains. In plain terms: humans cap their bids not because of a math error, but because of deeply embedded psychological mechanisms -- loss aversion, finite wealth intuitions, and diminishing marginal utility. The question the paper asks is: when an LLM also caps its bid, is it doing so for the same reasons?

Conceptual overview of the St. Petersburg paradox study, showing the mechanistic mismatch between LLM behavior in the original game versus mechanism probes

The Experiment: 28 Models, One Paradox, Many Traps

The researchers evaluated 28 LLMs with a structured prompt suite that includes the original game; controlled decision variants that perturb truncation, repeated play, numeric endowment, and occupational identity; a human-perspective prompt that asks models to reason as human decision makers; and paired comparisons between base models and their instruction-tuned counterparts.