Valuemax Beats Stockfish by Predicting How Real Chess Players Actually Move

When an AI assistant recommends a move, it usually suggests whatever a superhuman engine would play. That sounds obviously correct until you remember that engine moves assume the person will also play every follow-up optimally. Real humans do not. A new paper from Washington University in St. Louis, Microsoft Research, and Georgetown formalizes this mismatch and shows that picking moves based on how the actual human is likely to continue beats Stockfish recommendations in both simulation and a controlled human study.

The authors call the approach value-aware interventions, or Valuemax. They evaluate the idea in chess by learning models of human policies and value functions from large-scale gameplay data, and in simulation, the method consistently outperforms interventions based on the strongest chess engine.

When optimal moves make terrible advice

The problem sits at the intersection of reinforcement learning theory and human factors. The Bellman equation tells us that an optimal policy chooses the action that maximizes immediate reward plus the value of the next state, and this only holds if the same optimal policy keeps playing afterward. When a decision maker follows a suboptimal policy, policy-value consistency no longer holds, creating discrepancies between the actions taken and those that maximize the immediate reward plus the value of the next state. Those inconsistencies naturally identify opportunities for intervention.

In chess terms, a Stockfish-recommended sacrifice might be objectively winning if you play the next fifteen moves perfectly and disastrous if you are a 1200-rated player who will get lost in the resulting complications. Valuemax flips the question toward what move gives the best expected outcome given how