Goodfire's Silico Erases a Language From AI Using Just one Parameter

Fine-tuning is usually a blunt instrument. You adjust thousands or millions of parameters, cross your fingers that the model improves on your target task, and hope you haven't quietly broken something else. Goodfire just demonstrated a sharper approach: they removed a 67M-parameter language model's ability to speak German by tuning exactly one scalar value , and the rest of the model barely noticed.

The idea behind parameter decomposition

To understand what's happening here, you need to know about parameter decomposition , Goodfire's method for breaking a model's weight matrices into interpretable, rank-1 subcomponents (think of them as atomic building blocks of the model's learned behavior). The method, adVersarial Parameter Decomposition (VPD), optimizes for decompositions of neural network parameters into simple subcomponents that preserve the network's input-output behavior even when many subcomponents are ablated, including under ablations that are adversarially selected to destroy behavior.

Each subcomponent is a rank-1 matrix , the simplest possible matrix , with one "read" direction (what it looks for in the input) and one "write" direction (what it adds to the output). This encourages learning subcomponents that provide short, mechanistically faithful descriptions of the network's behavior that should aggregate appropriately into more global descriptions of the network's learned algorithm. Crucially, each subcomponent gets an auto-generated label describing what it activates on , e.g., "fires on German text and names."

Tuning one number to erase a language

The experiment was done as a one-day hackathon using Goodfire's Silico platform. The goal: destroy the model's ability to predict German text while keeping English intact. German was chosen because it was the model's strongest non-English language.

The approach works by treating the scalar "mask" on a subcomponent as the only trainable parameter. Setting it to zero suppresses the subcomponent entirely; flipping it negative inverts its contribution. There are only three degrees of freedom:

• Amplify (mask > 1): boost the subcomponent's influence
• Suppress (mask = 0): silence it entirely
• Invert (mask < 0): flip its contribution, actively working against the original behavior

The team first tried tuning the top 16 most "German-specific" subcomponents. But when they looked at the auto-generated labels, 13 of the 16 were labeled as "non-English text in general" , not German specifically. Only one, h.3.attn.v_proj:513, was labeled "German text and names." So they narrowed to that single component.

Beating LoRA with 4 tokens

The benchmark was LoRA (Low-Rank Adaptation) , the standard go-to for efficient fine-tuning. LoRA works by adding small trainable matrices on top of existing weights. The single-scalar approach beat it convincingly in the low-data regime.

The key metric was the trade-off between how much German performance was removed (pushing German cross-entropy toward chance, ~10.83 nats) versus how much English was damaged (ideally less than 0.1 nats increase). Here's how the methods compared:

Method	Min tokens to reach German-at-chance	Off-target damage (French/Spanish)	Trainable params
Single scalar (inverting)	~4 tokens	Minimal	1
LoRA rank-1	~32 tokens	Often severe	~55k
LoRA rank-4	~32 tokens	Moderate	~221k

The LoRAs frequently wrecked French, Spanish, and Italian , even though the objective said nothing about those languages. The single-scalar edit left French and Spanish almost untouched, not because it was explicitly told to, but because the subcomponent it targeted was genuinely German-specific. Italian was still damaged , and when the team investigated, they found the component's internal read/write directions encode German and Italian grammar together, which explains why.

The interpretability advantage

What makes this more than a clever trick is the feedback loop it creates. Because the subcomponents have human-readable labels, you can see what you're about to change before you change it. In this experiment, that visibility directly improved the result , catching the mistake of tuning 15 generic "foreign language" components instead of the one German-specific one.

There's an important caveat the team is upfront about: this approach is "cheating" in the sense that the decomposition itself cost German tokens to build. But VPD enables a new kind of 'circuit' analysis by studying how sequences of interactions between parameter subcomponents produce the network's output on particular inputs. If the decomposition is good, that upfront cost gets amortized across many downstream edits and tasks.

The underlying idea also maps onto a widely-held intuition about fine-tuning: that post-training mostly amplifies or suppresses behaviors already learned during pretraining, rather than teaching genuinely new ones. This experiment takes that idea literally , and shows it can work with extreme parameter efficiency.

What this means in practice

This is an early-stage research demo on a small 67M model, not a production fine-tuning recipe. The limitations are real:

• Only tested on one model and one decomposition
• The model barely knows German to begin with , it's a weak target
• LoRA wins at larger token budgets when data is plentiful
• Italian was collateral damage, revealing imperfect decomposition granularity

But the practical implications are worth tracking. Goodfire's Silico lets researchers and engineers peer inside an AI model and adjust its parameters during training, which could give model makers more fine-grained control over how this technology is built than was once thought possible. The use cases this points toward include capability removal for safety (unlearning hazardous knowledge), targeted behavior suppression without full retraining, and debugging models by editing the specific internal structures driving a failure mode.

Silico is a mechanistic interpretability tool developed by Goodfire that allows researchers to debug and adjust AI models during their training process. It is primarily designed for use with open-source models. Access is currently by request via the Silico page, and the underlying parameter decomposition code is open source on GitHub.

The deeper question this work raises for the field: if a model's learned behaviors really do decompose into clean, labelable, independently-switchable components, then fine-tuning stops being alchemy and starts looking a lot more like software engineering , editing specific functions rather than recompiling the whole program.