Goodfire's Silico Fixes AI Training Collapse in an Afternoon Without Retraining
Goodfire's Silico platform used Direct Logit Attribution to cut an RL mode collapse from 94% to 5% in one afternoon, without retraining
- Goodfire used its Silico platform to fix RL mode collapse by editing model weights directly, cutting a 94% repeated token down to 5%.
- The technique, Direct Logit Attribution, identifies which specific weights are boosting a problematic token's probability and edits them down.
- MMLU benchmark performance dropped only 3 percentage points -- a near-surgical fix with minimal side effects.
- Silico is described as the first off-the-shelf tool for debugging all stages of LLM development, from dataset to trained model.
- The fix took one afternoon of hackathon work; SFT on clean data still does the job better but requires data you may not have.
- Silico is in private beta; pricing is case-by-case.
Reinforcement learning runs collapse all the time. A model latches onto a shortcut, starts parroting the same phrase in 94% of its outputs, and you're left with two bad options: accept the broken behavior or throw away your training run and start over with new data. Goodfire just showed a third path: find the exact weights responsible and edit them out, directly, in an afternoon.
The problem that kills RL runs
Mode collapse is one of the most common failure modes in reinforcement learning fine-tuning. Instead of learning a generalizable behavior, the model discovers a degenerate shortcut and exploits it relentlessly. You've seen the symptoms: em-dashes everywhere, the word "genuinely" in every sentence, or in Goodfire's case, 94% of outputs opening with the word "texts." The team was training an LLM to label its own neurons when the collapse hit.
The standard remedies are both expensive:
- Accept it -- ship a broken model and work around it downstream.
- Retrain -- curate a new, de-collapsed dataset and restart the run. This costs time, compute, and the collapsed data is often the very data that caused the problem.
Supervised fine-tuning (SFT) on clean data does fix mode collapse, and it fixes it better. But it requires data you don't have yet. The weight edit is the cheap first move.
What Direct Logit Attribution actually does
Direct Logit Attribution (DLA) is an interpretability technique that traces how much each individual component of a neural network -- each attention head, each MLP layer -- directly pushes the model toward a specific token prediction. DLA measures the contribution of each model component to token prediction, making it possible to dissect model behavior at a granular level. In plain terms: instead of asking "why did the model output X?", you ask "which weights are most responsible for boosting token X's probability?" and you get a ranked list.