Researchers Split AI Steering Vectors Into Three Controllable Parts to Boost Accuracy

A new preprint proposes a way to decompose task vectors into interpretable parts, tune those parts independently, and improve activation-steering accuracy without updating a model’s weights. The method separates a vector into task-recognition, task-learning, and residual components, then optimizes two coefficients for each intervention layer.

Task vectors steer a language model by adding a direction to its hidden states during inference. That direction can be derived from natural-language instructions or few-shot demonstrations, allowing the model to reproduce task behavior without fine-tuning. Their internal composition, however, has remained poorly understood, which makes failures difficult to diagnose and scaling decisions largely empirical.

From one vector to three parts

The paper, Task Vectors Can Be Linearly Decomposed into Interpretable and Controllable Components, builds on a distinction from in-context learning:

• Task recognition (TR) identifies the permitted label set, such as Positive and Negative for sentiment classification.
• Task learning (TL) selects the correct candidate label for the current input.
• Residual contains the portion of the original task vector left after extracting the TR and TL components.

The authors construct the decomposition from the model’s unembedding vectors for the candidate labels. An unembedding vector belongs to the model’s final output layer, which converts hidden states into scores for vocabulary tokens. Projecting a task vector onto the subspace defined by the label vectors produces coefficients associated with recognizing the label space and discriminating among its members. The remaining signal becomes the residual.