Lyra C's Neuron Explorer Lets Researchers Steer Qwen3.5-4B's Hidden Channels
A new open-source tool pushes every MLP neuron in Qwen3-4B up and down, saves every rollout, then lets you browse the model as a 3D map.
- Lyra C released neuron-explorer, a steering-first interpretability tool for Qwen3-4B with a 3D map of every MLP neuron.
- Each neuron is pushed up and down on a few prompts, with every rollout stored and browsable in a local web UI.
- Steering doses are fractions of the median residual norm, keeping their meaning across layers and neurons.
- Sweeps run on a CUDA GPU pod using a steering fork of vLLM; shipped caches let you browse without rerunning.
- Includes an MCP integration that lets the Claude CLI drive the explorer, run the live model and move the map.
- Already ported to Gemma 4 E4B and Gemma 3 4B and cited as inspiration by multiple downstream interpretability projects.
Lyra C has released Neuron Explorer, an open-source interpretability tool that tests individual MLP channels by injecting their output directions into a model’s residual stream and recording the resulting text. The initial release targets Qwen3.5-4B and includes cached sweeps, a local visual interface, live inference tools, and an agent integration.
| Component | Details |
|---|---|
| Primary model | Qwen3.5-4B |
| Core method | Positive and negative steering of every MLP channel in a selected layer |
| Interface | Local web app at localhost:8791 |
| Fresh sweeps | CUDA GPU and the project’s steering fork of vLLM |
| Live inference | MPS, CUDA, or CPU |
| Reported ports | Gemma 4 E4B and Gemma 3 4B |
From observation to intervention
Transformer MLPs contain intermediate channels commonly described as neurons. Each channel contributes through a column of the MLP’s down-projection matrix, which writes information back into the residual stream, the shared state consumed by later layers. Neuron Explorer injects a scaled version of that write direction and measures how generation changes.
Intervention strength is expressed as a fraction of the median residual norm leaving the selected layer. A dose of +0.5 therefore adds a vector with roughly half that reference magnitude. This normalization makes doses more comparable across channels and layers, although equal doses can still produce different behavioral effects.
The default sweep crosses a small prompt set with positive and negative doses. It also includes an unsteered baseline and a free-form prompt at a strong positive dose. The pipeline retains each rollout, including generated text, token IDs, per-step log probabilities, and top-k token predictions. Per-channel records also include a generated name, lens results, connection data, and a three-column chat comparing negative, zero, and positive steering.
This story is for Pro members
You've reached the end of the free preview. Upgrade to AlphaSignal Pro to read the full article - and everything else behind the paywall.
Comments
