From Sequence to Structure: Uncovering Substructure Reasoning in Transformers

TL;DR

This paper investigates how decoder-only Transformers understand and extract graph substructures, revealing internal mechanisms and demonstrating their ability to handle complex graph data like molecular graphs.

Contribution

It introduces the Induced Substructure Filtration (ISF) framework, providing both empirical and theoretical insights into substructure reasoning in Transformers.

Findings

Transformers can identify substructures within graphs across layers.

The ISF framework captures consistent internal dynamics in LLMs.

Transformers successfully extract substructures from attributed graphs like molecules.

Abstract

Recent studies suggest that large language models (LLMs) possess the capability to solve graph reasoning tasks. Notably, even when graph structures are embedded within textual descriptions, LLMs can still effectively answer related questions. This raises a fundamental question: How can a decoder-only Transformer architecture understand underlying graph structures? To address this, we start with the substructure extraction task, interpreting the inner mechanisms inside the transformers and analyzing the impact of the input queries. Specifically, through both empirical results and theoretical analysis, we present Induced Substructure Filtration (ISF), a perspective that captures the substructure identification in the multi-layer transformers. We further validate the ISF process in LLMs, revealing consistent internal dynamics across layers. Building on these insights, we explore the broader capabilities of Transformers in handling diverse graph types. Specifically, we introduce the concept of thinking in substructures to efficiently extract complex composite patterns, and demonstrate that decoder-only Transformers can successfully extract substructures from attributed graphs, such as molecular graphs. Together, our findings offer a new insight on how sequence-based Transformers perform the substructure extraction task over graph data.