Adaptive Multi-Subspace Representation Steering for Attribute Alignment in Large Language Models

TL;DR

This paper introduces MSRS, a novel activation steering framework for large language models that effectively manages multiple attributes by isolating their influence in orthogonal subspaces, reducing interference and improving control.

Contribution

MSRS is the first method to allocate orthogonal subspaces for each attribute, combining attribute-specific and shared subspaces with a dynamic weighting for precise multi-attribute steering.

Findings

MSRS reduces attribute conflicts compared to existing methods.

MSRS outperforms prior approaches across various attributes.

MSRS generalizes well to diverse downstream tasks.

Abstract

Activation steering offers a promising approach to controlling the behavior of Large Language Models by directly manipulating their internal activations. However, most existing methods struggle to jointly steer multiple attributes, often resulting in interference and undesirable trade-offs. To address this challenge, we propose Multi-Subspace Representation Steering (MSRS), a novel framework for effective multi-attribute steering via subspace representation fine-tuning. MSRS reduces inter-attribute interference by allocating orthogonal subspaces to each attribute, isolating their influence within the model's representation space. MSRS also incorporates a hybrid subspace composition strategy: it combines attribute-specific subspaces for unique steering directions with a shared subspace for common steering directions. A dynamic weighting function learns to efficiently integrate these components for precise control. During inference, MSRS introduces a token-level steering mechanism that dynamically identifies and intervenes on the most semantically relevant tokens, enabling fine-grained behavioral modulation. Experimental results show that MSRS significantly reduces attribute conflicts, surpasses existing methods across a range of attributes, and generalizes effectively to diverse downstream tasks.