Locate, Steer, and Improve: A Practical Survey of Actionable Mechanistic Interpretability in Large Language Models

TL;DR

This paper presents a practical framework for mechanistic interpretability in large language models, emphasizing actionable diagnosis and intervention to improve model alignment, capability, and efficiency.

Contribution

It introduces a structured 'Locate, Steer, and Improve' pipeline, categorizes methods, and operationalizes MI as an actionable approach for model enhancement.

Findings

Framework enables tangible improvements in model alignment, capability, and efficiency.

Categorization of localizing and steering methods based on interpretable objects.

Operationalizes MI as an actionable methodology for model optimization.

Abstract

Mechanistic Interpretability (MI) has emerged as a vital approach to demystify the opaque decision-making of Large Language Models (LLMs). However, existing reviews primarily treat MI as an observational science, summarizing analytical insights while lacking a systematic framework for actionable intervention. To bridge this gap, we present a practical survey structured around the pipeline: "Locate, Steer, and Improve." We formally categorize Localizing (diagnosis) and Steering (intervention) methods based on specific Interpretable Objects to establish a rigorous intervention protocol. Furthermore, we demonstrate how this framework enables tangible improvements in Alignment, Capability, and Efficiency, effectively operationalizing MI as an actionable methodology for model optimization. The curated paper list of this work is available at https://github.com/rattlesnakey/Awesome-Actionable-MI-Survey.