Understanding Reasoning in Thinking Language Models via Steering Vectors

TL;DR

This paper introduces a method to control and steer reasoning behaviors in large language models by identifying and manipulating linear directions in their activation space, enhancing interpretability and control.

Contribution

The work presents a novel approach to steer reasoning behaviors in thinking LLMs using linear vectors, validated across multiple models and diverse tasks.

Findings

Linear directions in activation space mediate reasoning behaviors.

Steering vectors effectively control behaviors like uncertainty and backtracking.

Method generalizes across different model architectures.

Abstract

Recent advances in large language models (LLMs) have led to the development of thinking language models that generate extensive internal reasoning chains before producing responses. While these models achieve improved performance, controlling their reasoning processes remains challenging. This work presents a steering approach for thinking LLMs by analyzing and manipulating specific reasoning behaviors in DeepSeek-R1-Distill models. Through a systematic experiment on 500 tasks across 10 diverse categories, we identify several reasoning behaviors exhibited by thinking models, including expressing uncertainty, generating examples for hypothesis validation, and backtracking in reasoning chains. We demonstrate that these behaviors are mediated by linear directions in the model's activation space and can be controlled using steering vectors. By extracting and applying these vectors, we provide a method to modulate specific aspects of the model's reasoning process, such as its tendency to backtrack or express uncertainty. Our approach offers practical tools for steering reasoning processes in thinking models in a controlled and interpretable manner. We validate our steering method using three DeepSeek-R1-Distill models, demonstrating consistent control across different model architectures.