An explainable transformer circuit for compositional generalization

TL;DR

This paper uncovers and interprets the specific circuit within a transformer responsible for compositional generalization, enabling better understanding and control of the model's behavior.

Contribution

It identifies and mechanistically interprets the circuit for compositional induction in a compact transformer, advancing interpretability and controllability.

Findings

Validated the circuit through causal ablations.

Formalized the circuit with a program-like description.

Enabled precise activation edits to steer behavior.

Abstract

Compositional generalization-the systematic combination of known components into novel structures-remains a core challenge in cognitive science and machine learning. Although transformer-based large language models can exhibit strong performance on certain compositional tasks, the underlying mechanisms driving these abilities remain opaque, calling into question their interpretability. In this work, we identify and mechanistically interpret the circuit responsible for compositional induction in a compact transformer. Using causal ablations, we validate the circuit and formalize its operation using a program-like description. We further demonstrate that this mechanistic understanding enables precise activation edits to steer the model's behavior predictably. Our findings advance the understanding of complex behaviors in transformers and highlight such insights can provide a direct pathway for model control.