Angular Steering: Behavior Control via Rotation in Activation Space

TL;DR

Angular Steering introduces a geometric rotation-based method for precise and stable behavior control in large language models, improving upon existing subspace techniques by offering continuous, fine-grained adjustments without compromising overall performance.

Contribution

The paper presents Angular Steering, a novel rotation-based behavior control method that generalizes prior techniques, providing enhanced stability, flexibility, and fine-grained control in large language models.

Findings

Achieves robust behavioral control across multiple models and sizes.

Maintains language modeling performance while controlling behaviors.

Generalizes existing methods under a unified geometric rotation framework.

Abstract

Controlling specific behaviors in large language models while preserving their general capabilities is a central challenge for safe and reliable artificial intelligence deployment. Current steering methods, such as vector addition and directional ablation, are constrained within a two-dimensional subspace defined by the activation and feature direction, making them sensitive to chosen parameters and potentially affecting unrelated features due to unintended interactions in activation space. We introduce Angular Steering, a novel and flexible method for behavior modulation that operates by rotating activations within a fixed two-dimensional subspace. By formulating steering as a geometric rotation toward or away from a target behavior direction, Angular Steering provides continuous, fine-grained control over behaviors such as refusal and compliance. We demonstrate this method using refusal steering emotion steering as use cases. Additionally, we propose Adaptive Angular Steering, a selective variant that rotates only activations aligned with the target feature, further enhancing stability and coherence. Angular Steering generalizes existing addition and orthogonalization techniques under a unified geometric rotation framework, simplifying parameter selection and maintaining model stability across a broader range of adjustments. Experiments across multiple model families and sizes show that Angular Steering achieves robust behavioral control while maintaining general language modeling performance, underscoring its flexibility, generalization, and robustness compared to prior approaches. Code and artifacts are available at https://github.com/lone17/angular-steering/.