Skip to main content
Back

Virginia Tech's LALP Beats GALP by 9 Points When Distilling Reasoning Models

A new step-level scoring rule flips the ranking of reasoning teachers and lifts distillation accuracy by up to 9.4 points.

Paper, Full Access
  • GALP (global log-probability selection) inverts teacher rankings in long, multi-teacher reasoning distillation setups.
  • Step-level embeddings match training data at 0.935 cosine similarity vs 0.760 for full trajectories.
  • LALP scores each reasoning step using only a small window of prior context, then averages.
  • Beats GALP by 9.4 points on math for Qwen2.5-32B and lifts GPQA-Diamond to 0.702.
  • Diagnoses two failure modes: fluency trap and token self-conditioning that inflates repeated content.
  • Code available at the anonymized repository; paper on arXiv.

When you distill a large reasoning model into a smaller student, you usually have several correct traces per problem and have to pick which one to train on. The intuitive answer, popularized by GRAPE, is to pick the response the student itself finds most probable, on the assumption that natural-looking data is easier to learn from. A new paper from Virginia Tech, The Signal is in the Steps, shows that this heuristic breaks down badly once you move to long chain-of-thought traces mixed from several teachers, and proposes a fix that reshuffles the rankings.

Where the natural-looking heuristic falls apart

The baseline they compare against is Global Average Log Probability (GALP): score each candidate response by the student's average token log-probability over the whole sequence, then pick the highest. GRAPE's hypothesis is that SFT works best when training data aligns with the student's pretrained distribution. That holds for short instruction-following data, but the authors stress-tested it in a different regime: long reasoning traces (over 12K tokens) drawn from DeepSeek-R1, QwQ-32B, and Qwen3-32B, three teachers with very different verbosity and self-reflection styles.

The result is a striking inversion. Qwen3-32B traces receive the highest average GALP under both 7B and 32B students, yet SFT on these traces yields the worst downstream accuracy. QwQ-32B traces do not achieve the highest GALP, yet SFT on them yields the best downstream accuracy. Global likelihood doesn't just get noisy in this regime, it actively points at the wrong teacher.

Dual-axis graph showing GALP and LALP scores versus downstream accuracy across teacher checkpoints

Students remix steps, they don't memorize solutions

To explain why, the authors embed held-out test solutions at two granularities and measure cosine similarity to the nearest training example. Step-level embeddings show very high nearest-neighbor similarity (0.935), while full-trajectory embeddings are substantially less covered (0.760). Whole solutions to unseen problems rarely resemble any training trajectory, but the individual reasoning moves inside them almost always do.