Memory Inception: Latent-Space KV Cache Manipulation for Steering LLMs

TL;DR

Memory Inception (MI) is a training-free latent-space steering method for LLMs that selectively injects key-value banks at specific layers, enabling persistent, structured, and efficient guidance without cluttering prompts.

Contribution

MI introduces a novel, training-free approach to steer LLMs by manipulating latent attention space with text-derived KV banks at selected layers.

Findings

MI achieves superior control on personality-steering tasks.

Supports mid-conversation behavior shifts without rewriting transcripts.

Outperforms prompting on structured reasoning tasks and reduces KV storage significantly.

Abstract

Steering large language models (LLMs) is usually done by either instruction prompting or activation steering. Prompting often gives strong control, but caches guidance tokens at every layer and can clutter long interactions; activation steering is compact but typically weaker and does not support large structured reminders. We introduce memory inception (MI), a training-free method that steers in latent attention space by inserting text-derived key-value (KV) banks only at selected layers. Rather than materializing reminder content throughout the prompt cache, MI treats steering as selective KV allocation, injecting latent slots only where the model routes to them. On matched personality-steering tasks, MI gives the best overall control--drift trade-off, remaining competitive with prompting while consistently outperforming CAA. On updateable guidance, MI supports mid-conversation behavior shifts without rewriting the visible transcript, achieving the highest post-shift alignment on Qwen3. On structured reasoning, MI outperforms visible prompting on HARDMath and PHYSICS (10/12 subject$\times$mode cells), serving as proxies for structured reasoning in verifiable domains, while cutting content-matched KV storage by up to 118$\times$. These results position MI as a powerful steering method when guidance is persistent, structured, or expensive to keep in the visible transcript.