Feature Extraction and Steering for Enhanced Chain-of-Thought Reasoning in Language Models

TL;DR

This paper introduces a novel steering technique using autoencoders to improve the reasoning abilities of large language models without requiring external datasets or fine-tuning, inspired by deep thinking paradigms.

Contribution

It proposes a new SAE-based and SAE-free steering method to enhance LLM reasoning, eliminating the need for costly long-chain data and fine-tuning.

Findings

Both steering methods significantly improve reasoning performance.

SAE-free steering directly computes directions from residual activations.

The approach enhances reasoning without external datasets.

Abstract

Large Language Models (LLMs) demonstrate the ability to solve reasoning and mathematical problems using the Chain-of-Thought (CoT) technique. Expanding CoT length, as seen in models such as DeepSeek-R1, significantly enhances this reasoning for complex problems, but requires costly and high-quality long CoT data and fine-tuning. This work, inspired by the deep thinking paradigm of DeepSeek-R1, utilizes a steering technique to enhance the reasoning ability of an LLM without external datasets. Our method first employs Sparse Autoencoders (SAEs) to extract interpretable features from vanilla CoT. These features are then used to steer the LLM's internal states during generation. Recognizing that many LLMs do not have corresponding pre-trained SAEs, we further introduce a novel SAE-free steering algorithm, which directly computes steering directions from the residual activations of an LLM, obviating the need for an explicit SAE. Experimental results demonstrate that both our SAE-based and subsequent SAE-free steering algorithms significantly enhance the reasoning capabilities of LLMs.