One Steering Vector Now Controls Both Image and Text Generation

A new preprint, Cross-modal Steering Vectors Exist Across Both Image and Text Generation, reports that activation directions extracted from text can steer image generation, and directions extracted from images can steer text generation. The transfers require no retraining and work for concepts including emotion, cleanliness, distance, and object size. The result supports the existence of shared control directions inside unified multimodal models.

Earlier evidence for a shared conceptual space, including the Platonic Representation Hypothesis, has largely relied on correlation: text and image representations look similar under measurement. Steering provides an intervention. When adding a text-derived vector changes an image model’s output in the predicted direction, the experiment connects an internal activation to generated behavior across modalities.

Because the study is a preprint, its findings have yet to undergo peer review. The experiments also establish transferable directions rather than one canonical representation for each concept, a distinction that matters when interpreting the results.

A Direct Test Inside Unified Models

The researchers test UniAR, Emu3.5, and Liquid, three unified multimodal models in which the same transformer stack generates language and image tokens autoregressively. This architecture removes a common source of ambiguity: transfer cannot be attributed solely to a separate vision encoder attached to a language model.

A steering vector is a direction in activation space that biases generation when added during a forward pass. The study uses a difference-in-means method. For a concept such as happy versus sad, the researchers average activations from examples at each pole and subtract one mean from the other. The resulting vector points along a concept-related axis and can be injected at selected transformer layers with a chosen strength.