CGTR Stops Qwen3-8B From Catastrophically Collapsing During Long Training Runs
A new paper reveals that when you refresh the teacher in self-distillation training matters more than how old it is, and proposes a smarter gating mechanism that eliminates training collapse entirely.
Takeaways
- Core finding: In self on-policy distillation, isolation periods (complete teacher freezes between updates) matter more than teacher age for training stability.
- New failure mode: "State-oblivious collapse" occurs when a clock-driven teacher refresh copies a transiently drifting student, causing irreversible training failure invisible in short-horizon evals.
- CGTR proposed: Consolidation-Gated Teacher Refresh gates each teacher update on joint reward improvement AND length-tail safety, not a fixed timer.
- Zero collapse: CGTR achieves zero collapse events and best final scores on all four tasks (Chemistry, Biology, Physics, ToolUse) with no per-task retuning.
- EMA is not safe: EMA teacher updates cause chronic contamination via feedback loops, distinct from the sudden collapse seen in fixed-schedule methods.
- No code released: The paper is available on arXiv but no public code or model weights have been released yet.
On-policy self-distillation (OPSD) has become one of the hottest training paradigms for LLM reasoning. The idea is elegant: instead of needing a bigger, separate teacher model, a single LLM acts as both teacher and student simultaneously, with the teacher version seeing privileged information (like a verified answer) while the student sees only the question. The model learns by minimizing the gap between what the teacher would say and what the student actually generates. But there is a catch that almost nobody has studied: when should the teacher's weights actually get updated?
A field with a blind spot
On-policy distillation has become a popular training paradigm in the LLM community. In OPSD, a single LLM acts as both teacher and student with different contexts, where the teacher policy conditions on privileged information while the student policy sees only the question, and training minimizes the per-token divergence between these distributions over the student's own rollouts. The community has largely focused on what the teacher sees, but almost no one has asked when the teacher should be refreshed from the student's evolving weights.
There are two common approaches in practice. The first is periodic "hard" refreshes: every M steps, copy the student's current weights into the teacher. The second is an Exponential Moving Average (EMA), where the teacher is a continuously smoothed blend of all past student checkpoints. Both have known failure modes, but they have never been studied together under a unified diagnostic lens, especially over long training runs.
The hidden variable that controls stability
The new paper runs a controlled sweep over teacher update schedules on Qwen3-8B across four reasoning-heavy tasks: Chemistry, Biology, Physics, and ToolUse. The central finding is counterintuitive: it is not how old the teacher is that determines stability, it is whether there is a complete freeze period between updates.