Efficient Automated Circuit Discovery in Transformers using Contextual Decomposition

TL;DR

This paper introduces CD-T, a novel method for efficiently discovering interpretable circuits in large language models, capable of fine-grained analysis with significantly reduced runtime and improved accuracy over previous approaches.

Contribution

The paper presents CD-T, a new mathematical approach for automated circuit discovery in transformers that is faster, more precise, and capable of finer-grained analysis than existing methods.

Findings

CD-T reduces circuit discovery runtime from hours to seconds.

CD-T outperforms ACDC and EAP in circuit recovery accuracy with 97% ROC AUC.

CD-T circuits are 80% more faithful than random circuits and can replicate model behavior with fewer nodes.

Abstract

Automated mechanistic interpretation research has attracted great interest due to its potential to scale explanations of neural network internals to large models. Existing automated circuit discovery work relies on activation patching or its approximations to identify subgraphs in models for specific tasks (circuits). They often suffer from slow runtime, approximation errors, and specific requirements of metrics, such as non-zero gradients. In this work, we introduce contextual decomposition for transformers (CD-T) to build interpretable circuits in large language models. CD-T can produce circuits of arbitrary level of abstraction, and is the first able to produce circuits as fine-grained as attention heads at specific sequence positions efficiently. CD-T consists of a set of mathematical equations to isolate contribution of model features. Through recursively computing contribution of all nodes in a computational graph of a model using CD-T followed by pruning, we are able to reduce circuit discovery runtime from hours to seconds compared to state-of-the-art baselines. On three standard circuit evaluation datasets (indirect object identification, greater-than comparisons, and docstring completion), we demonstrate that CD-T outperforms ACDC and EAP by better recovering the manual circuits with an average of 97% ROC AUC under low runtimes. In addition, we provide evidence that faithfulness of CD-T circuits is not due to random chance by showing our circuits are 80% more faithful than random circuits of up to 60% of the original model size. Finally, we show CD-T circuits are able to perfectly replicate original models' behavior (faithfulness $ = 1$) using fewer nodes than the baselines for all tasks. Our results underscore the great promise of CD-T for efficient automated mechanistic interpretability, paving the way for new insights into the workings of large language models.