KV Cache Steering for Controlling Frozen LLMs

TL;DR

This paper introduces cache steering, a lightweight, one-shot method to implicitly guide frozen language models towards better reasoning and behavior control by manipulating their key-value caches during inference.

Contribution

It presents a novel cache steering technique that induces reasoning in frozen models without fine-tuning, using reasoning traces to construct steering vectors for improved performance.

Findings

Enhances reasoning quality and task performance across benchmarks.

Scales effectively to larger models and challenging datasets.

Enables controllable transfer of reasoning styles.

Abstract

We propose cache steering, a lightweight method for implicit steering of language models via a one-shot intervention applied directly to the key-value cache. To validate its effectiveness, we apply cache steering to induce chain-of-thought reasoning in small language models. Our approach constructs steering vectors from reasoning traces, obtained either from teacher models (e.g., GPT-4o) or existing human annotations, that shift model behavior toward more explicit, multi-step reasoning without fine-tuning or prompt modifications. Experimental evaluations on diverse reasoning benchmarks demonstrate that cache steering improves both the qualitative structure of model reasoning and quantitative task performance. Additional experiments show that the method also scales to larger models and yields further gains on challenging datasets such as GPQA and MATH. Compared to prior activation steering techniques that require continuous interventions, our one-shot cache steering offers substantial advantages in terms of inference latency, hyperparameter stability, and ease of integration with existing inference APIs. Beyond mere reasoning induction, we show that cache steering enables controllable transfer of reasoning styles (e.g., stepwise, causal, analogical), making it a practical tool for behavior-level guidance of language models.