Goodfire's Silico Catches Qwen3-35B Endorsing Drunk Driving 75% of the Time
Goodfire's Silico platform exposes competing neuron groups inside LLMs and lets you surgically boost safety signals without retraining the model
- Goodfire's Silico platform identified two competing neuron groups inside Qwen3-35B that cause it to endorse drunk driving 75.5% of the time under specific framing.
- Boosting the 5 "impairment" neurons 4x cut harmful endorsements from 30/48 to 20/48 with zero false refusals on sober users, no retraining needed.
- Silico uses sparse autoencoders (SAEs) to decompose tangled LLM neurons into clean, human-readable features that can be individually inspected and steered.
- The platform covers the full model lifecycle: predict, health check, debug, and improve, and works across LLMs, life sciences, and vision models.
- Access is via private beta with case-by-case pricing; current clients include Mayo Clinic, Microsoft, and Arc Institute.
- Goodfire raised $150M Series B at a $1.25B valuation, with Salesforce and Eric Schmidt among backers, signaling major industry investment in interpretability tooling.
Ask an LLM whether a slightly drunk person should drive, and the answer might surprise you. Researcher Curt Tigges, working at a hackathon with Silico, Goodfire's agentic interpretability platform, found that Qwen3-35B says yes to that question 75.5% of the time. Remove the phrase "sober one of the group" from the prompt, and the model says no 99.9% of the time. Same alcohol intake, completely different answer. The culprit is a competition happening inside the model's neurons, and Goodfire just showed you can watch it happen in real time and fix it.
Two neuron groups walk into a bar
A large language model consists of artificial neurons that interact in complex ways, with tens of thousands involved in generating a single response. Goodfire's analysis of the drunk-driving case cracked open exactly that complexity. Inside the model, two distinct neuron groups were fighting over the output:
- A 5-neuron "impairment bank" that fires on the phrase "four drinks" and pushes the model toward a "No, don't drive" answer.
- A 12-neuron "sober one" override that activates on the social framing and pushes toward "Yes, you're the most capable one."
The safety signal fires, but its lead neuron barely scales from 1 to 8 drinks. The override wins. The failure is one of neuron priority, and it's invisible without the right tools to look inside.
What Silico actually does
Silico lets researchers and engineers peer inside a trained model, run experiments on specific neurons or groups of neurons, check what inputs make those neurons fire, and trace pathways upstream and downstream to see how neurons affect each other. The key capability is intervention, not just inspection.
The drunk-driving fix demonstrates the workflow cleanly. Boosting the 5 impairment neurons by 4x cut harmful endorsements from 30 out of 48 to 20 out of 48 on held-out prompts, with zero over-refusal on sober users. The fix is surgical enough that it doesn't break the model for legitimate use cases.
Under the hood: sparse autoencoders
Goodfire can identify individual neurons with human-readable meaning because of a technique called a sparse autoencoder (SAE). SAEs impose sparsity constraints on hidden unit activations, ensuring only a small subset of neurons fire for any given input. A regular LLM's neurons are tangled, with each one representing a messy mix of concepts. An SAE acts as a lens that separates those concepts into clean, labeled features you can reason about.
Goodfire's platform exposes these features as interpretable patterns of neuron activity, letting researchers inspect and modify model internals for safety and alignment. The drunk-driving case used this to identify not just that something was wrong, but which neurons were wrong and by how much.
Goodfire's approach also differs from traditional activation steering, which nudges the entire residual stream in a direction computed from example pairs. Neuron-level steering is more targeted: find the specific feature responsible for a behavior and turn its dial directly.
Silico's full capability surface
Goodfire positions Silico as the first off-the-shelf tool that can help developers debug all stages of development, from building a dataset to training a model. The platform organizes this into four workflows:
- Predict: Decompose model predictions into interpretable features and see whether they're driven by real understanding or spurious correlations.
- Check Health: Run diagnostics on internal representations to catch problems like undertraining, information bottlenecks, and feature collapse before they hit production.
- Debug: Precisely identify and remove confounders behind model failures.
- Improve: Steer the same internals that make a model interpretable to make it more controllable.
Goodfire is one of a small group of companies, alongside Anthropic, OpenAI, and Google DeepMind, working on mechanistic interpretability: mapping a model's neurons and the pathways between them to understand what happens inside when it carries out a task. MIT Technology Review named mechanistic interpretability one of its 10 Breakthrough Technologies of 2026.
Silico as a research environment
The drunk-driving analysis was performed during a hackathon, which is telling. Beyond visualization, the platform includes what Goodfire calls "the model neuroscientist": an AI agent that plans and runs interpretability experiments, returns results, and learns over time. The goal is to turn AI development into a more methodical engineering discipline rather than a process of intuition and trial and error.
Silico works across LLMs, life sciences models, and robotics and vision models. Researchers can use it to inspect parameters inside open-source models, though closed models like ChatGPT or Gemini remain out of reach.
What this unlocks
Goodfire's broader aim is to use mechanistic interpretability to help design models from the start, not just audit them after training. The drunk-driving case is a proof of concept for diagnosing and fixing specific failure modes without retraining, without prompt engineering, and without guessing. Practical use cases where this matters most:
- Safety-critical deployments in healthcare and finance, where a model endorsing the wrong action has real consequences.
- Hallucination reduction: Goodfire has already used its techniques to reduce the rate of hallucinations in LLMs.
- Bias auditing: finding which neurons encode ideological or demographic shortcuts and quantifying their influence on outputs.
- Fine-tuning without data: if you can identify the neurons responsible for a behavior, you can steer them directly rather than collecting thousands of labeled examples.
Who can use it and what it costs
By releasing Silico, Goodfire is putting techniques previously confined to a few top labs into the hands of smaller firms and research teams building or adapting open-source models. Pricing is determined on a case-by-case basis. Access is currently via private beta, with a request form on the Goodfire contact page. Current clients include Arc Institute, Mayo Clinic, Microsoft, and Rakuten.
Goodfire raised $150M in a Series B round led by B Capital, with contributions from Salesforce, former Google CEO Eric Schmidt, and more than half a dozen other investors, valuing the company at $1.25 billion. That level of backing reflects how far the field has moved from academic curiosity to production tooling.
The drunk-driving demo is a small case study, but it crystallizes the core promise of mechanistic interpretability: open the model, find the neurons that are misbehaving, and fix them. That's a fundamentally different relationship with AI systems than the industry has operated with so far.
Comments
