Localizing Lying in Llama: Understanding Instructed Dishonesty on True-False Questions Through Prompting, Probing, and Patching

TL;DR

This paper investigates instructed dishonesty in LLaMA-2-70b-chat, identifying specific network components responsible for lying and demonstrating causal interventions to promote honesty, thereby enhancing understanding of dishonesty in large language models.

Contribution

It introduces a mechanistic interpretability approach to localize and intervene in lying behavior within LLMs, advancing methods to control dishonesty in AI systems.

Findings

Identified five key layers associated with lying behavior.

Localized 46 attention heads critical for dishonesty.

Causal interventions successfully promote honest responses.

Abstract

Large language models (LLMs) demonstrate significant knowledge through their outputs, though it is often unclear whether false outputs are due to a lack of knowledge or dishonesty. In this paper, we investigate instructed dishonesty, wherein we explicitly prompt LLaMA-2-70b-chat to lie. We perform prompt engineering to find which prompts best induce lying behavior, and then use mechanistic interpretability approaches to localize where in the network this behavior occurs. Using linear probing and activation patching, we localize five layers that appear especially important for lying. We then find just 46 attention heads within these layers that enable us to causally intervene such that the lying model instead answers honestly. We show that these interventions work robustly across many prompts and dataset splits. Overall, our work contributes a greater understanding of dishonesty in LLMs so that we may hope to prevent it.