The Mechanics of Conceptual Interpretation in GPT Models: Interpretative Insights

TL;DR

This paper investigates how transformer-based large language models process semantic information through detailed analysis of their internal components, revealing mechanisms that can inform targeted interpretability and editing techniques.

Contribution

It introduces concept editing to analyze conceptualisation mechanisms in transformers, providing new insights into the layered semantic processing within LLMs.

Findings

MLP layers use key-value retrieval and context-dependent processing

MHA layers show distributed, high-level semantic integration

Hidden states highlight the importance of last tokens and top layers

Abstract

Locating and editing knowledge in large language models (LLMs) is crucial for enhancing their accuracy, safety, and inference rationale. We introduce ``concept editing'', an innovative variation of knowledge editing that uncovers conceptualisation mechanisms within these models. Using the reverse dictionary task, inference tracing, and input abstraction, we analyse the Multi-Layer Perceptron (MLP), Multi-Head Attention (MHA), and hidden state components of transformer models. Our results reveal distinct patterns: MLP layers employ key-value retrieval mechanism and context-dependent processing, which are highly associated with relative input tokens. MHA layers demonstrate a distributed nature with significant higher-level activations, suggesting sophisticated semantic integration. Hidden states emphasise the importance of the last token and top layers in the inference process. We observe evidence of gradual information building and distributed representation. These observations elucidate how transformer models process semantic information, paving the way for targeted interventions and improved interpretability techniques. Our work highlights the complex, layered nature of semantic processing in LLMs and the challenges of isolating and modifying specific concepts within these models.