Understanding Unreliability of Steering Vectors in Language Models: Geometric Predictors and the Limits of Linear Approximations

TL;DR

This paper investigates why steering vectors in language models are unreliable for certain behaviors, identifying factors like training data similarity and linear approximation limits that affect steering effectiveness.

Contribution

It reveals key predictors of steering reliability, such as cosine similarity and activation separation, and highlights the limitations of linear approximations in representing complex behaviors.

Findings

Higher cosine similarity predicts more reliable steering.

Better separation of positive and negative activations improves steerability.

Steering vectors trained on different prompts perform similarly despite directional differences.

Abstract

Steering vectors are a lightweight method for controlling language model behavior by adding a learned bias to the activations at inference time. Although effective on average, steering effect sizes vary across samples and are unreliable for many target behaviors. In my thesis, I investigate why steering reliability differs across behaviors and how it is impacted by steering vector training data. First, I find that higher cosine similarity between training activation differences predicts more reliable steering. Second, I observe that behavior datasets where positive and negative activations are better separated along the steering direction are more reliably steerable. Finally, steering vectors trained on different prompt variations are directionally distinct, yet perform similarly well and exhibit correlated efficacy across datasets. My findings suggest that steering vectors are unreliable when the latent target behavior representation is not effectively approximated by the linear steering direction. Taken together, these insights offer a practical diagnostic for steering unreliability and motivate the development of more robust steering methods that explicitly account for non-linear latent behavior representations.