Precise Learning of Source Code Contextual Semantics via Hierarchical Dependence Structure and Graph Attention Networks

TL;DR

This paper introduces a novel source code representation combining AST and CFG with hierarchical dependencies and graph attention, significantly improving program classification accuracy and efficiency.

Contribution

It proposes a new source code model integrating AST and CFG with hierarchical dependencies and a graph attention-based neural network, addressing the representation of basic blocks.

Findings

Achieves 4% higher accuracy in program classification.

Reduces model parameters by 50% compared to baselines.

Significantly improves performance on software engineering tasks.

Abstract

Deep learning is being used extensively in a variety of software engineering tasks, e.g., program classification and defect prediction. Although the technique eliminates the required process of feature engineering, the construction of source code model significantly affects the performance on those tasks. Most recent works was mainly focused on complementing AST-based source code models by introducing contextual dependencies extracted from CFG. However, all of them pay little attention to the representation of basic blocks, which are the basis of contextual dependencies. In this paper, we integrated AST and CFG and proposed a novel source code model embedded with hierarchical dependencies. Based on that, we also designed a neural network that depends on the graph attention mechanism.Specifically, we introduced the syntactic structural of the basic block, i.e., its corresponding AST, in source code model to provide sufficient information and fill the gap. We have evaluated this model on three practical software engineering tasks and compared it with other state-of-the-art methods. The results show that our model can significantly improve the performance. For example, compared to the best performing baseline, our model reduces the scale of parameters by 50\% and achieves 4\% improvement on accuracy on program classification task.