Goodfire's Silico Fixes AI Training Collapse in an Afternoon Without Retraining

Reinforcement learning runs collapse all the time. A model latches onto a shortcut, starts parroting the same phrase in 94% of its outputs, and you're left with two bad options: accept the broken behavior or throw away your training run and start over with new data. Goodfire just demonstrated a third path: find the exact weights responsible and edit them out, directly, in an afternoon.

The failure mode that kills RL runs

Mode collapse is one of the most common failure modes in reinforcement learning fine-tuning. Instead of learning a generalizable behavior, the model discovers a degenerate shortcut and exploits it relentlessly. The symptoms are familiar: em-dashes everywhere, the word "genuinely" in every sentence, or in Goodfire's case, 94% of outputs opening with the word "texts." The team was training an LLM to label its own neurons when the collapse hit.

The standard remedies are both expensive:

• Accept it — ship a broken model and work around it downstream.
• Retrain — curate a new, de-collapsed dataset and restart the run, costing time and compute, often using the very data that caused the problem.

Supervised fine-tuning (SFT) on clean data does fix mode collapse, and it fixes it more thoroughly. But it requires data you don't have yet. The weight edit is the cheap first move.

How Direct Logit Attribution works

Direct Logit Attribution (DLA) is an interpretability technique that traces how much each individual component of a neural network — each attention head, each MLP layer — directly pushes the model toward a specific token prediction. Instead of asking "why did the model output X?", you ask "which weights are most responsible for boosting token X's probability?" and you get a ranked list.

Goodfire used their platform Silico to run this analysis on the collapsed model, identify the specific weights over-promoting the token "texts," and edit those weights down. The "texts" opener dropped from 94% to 5% — an 89 percentage point reduction — while MMLU benchmark performance fell by only 3 percentage points. Near-surgical, with minimal collateral damage.

What Silico actually is

Silico lets researchers and engineers inspect a model's internals and adjust its parameters during training. Goodfire positions it as the first off-the-shelf tool capable of helping developers debug every stage of the development pipeline, from dataset construction to training. The goal is to make building AI models feel more like writing software: see what the model has learned, locate undesired behavior, and make targeted interventions.

Silico sits on top of Goodfire's broader interpretability research, which includes cutting hallucinations in an LLM by 58% using interpretability to guide training — at roughly 90x lower cost per intervention than LLM-as-judge, with no degradation on standard benchmarks.

Interpretability as an engineering tool

Goodfire is among a small group of companies, alongside Anthropic, OpenAI, and Google DeepMind, working on mechanistic interpretability — MIT Technology Review named it one of its 10 Breakthrough Technologies of 2026. The field aims to reverse-engineer what specific components of a neural network are actually doing, so problems can be localized and fixed rather than trained around.

The mode collapse demo is a concrete example of that vision in practice. "If we can make training models a lot more like building software, there's no reason why there can't be many more companies designing models that fit their needs," says CEO Eric Ho. Frontier labs already maintain internal interpretability teams. Silico targets the next tier of companies, where the value is avoiding that hire entirely.

Early beta users describe Silico as "a big uplift over Claude/Codex, especially for interpretability" and "the first frontier quality research tool made available to the public." Goodfire also reports that Silico reproduced a months-long interpretability project in two days.

When to reach for this technique

This approach is most useful when you're mid-run and something goes wrong behaviorally. Specifically:

• An RL fine-tune collapses onto a repeated token, phrase, or stylistic tic
• You don't have time or budget to curate a new dataset and retrain
• The behavioral problem is specific enough to be localized to a token or small set of tokens
• You want to verify the fix is surgical before committing to a full retrain

Goodfire is explicit that SFT on clean data does the job better when you have the data. As a fast, cheap diagnostic and partial fix, though, weight editing changes the calculus of what to do when a run goes sideways.

How to get access

Silico is currently in private beta. Pricing is determined on a case-by-case basis. You can apply at goodfire.ai/contact.

The mode collapse fix was built in a single afternoon of hackathon work inside Silico. Interpretability techniques that once required a dedicated research team are becoming fast enough to be worth trying before you reach for the retraining button.