What does Transformer learn about source code?

TL;DR

This paper investigates what structural information transformer models learn about source code by proposing methods to extract and analyze program graphs from the models, demonstrating their effectiveness in code understanding tasks.

Contribution

It introduces the aggregated attention score and aggregated attention graph methods to automatically extract meaningful program graphs from transformer models trained on source code.

Findings

Automatically extracted graphs are meaningful and effective.

Semantic graphs improve variable misuse detection.

Provides new insights into transformer understanding of code.

Abstract

In the field of source code processing, the transformer-based representation models have shown great powerfulness and have achieved state-of-the-art (SOTA) performance in many tasks. Although the transformer models process the sequential source code, pieces of evidence show that they may capture the structural information (\eg, in the syntax tree, data flow, control flow, \etc) as well. We propose the aggregated attention score, a method to investigate the structural information learned by the transformer. We also put forward the aggregated attention graph, a new way to extract program graphs from the pre-trained models automatically. We measure our methods from multiple perspectives. Furthermore, based on our empirical findings, we use the automatically extracted graphs to replace those ingenious manual designed graphs in the Variable Misuse task. Experimental results show that the semantic graphs we extracted automatically are greatly meaningful and effective, which provide a new perspective for us to understand and use the information contained in the model.