GypSum: Learning Hybrid Representations for Code Summarization

TL;DR

GypSum is a novel deep learning model that combines graph attention networks and pre-trained language models to improve code summarization by capturing both semantic and structural code features.

Contribution

It introduces a hybrid representation learning approach with graph-based and token-based encoders, and a dual-copy mechanism for enhanced code summarization.

Findings

GypSum outperforms existing models in code summarization tasks.

Incorporating control flow edges improves summary quality.

Hybrid representations lead to more accurate and fluent summaries.

Abstract

Code summarization with deep learning has been widely studied in recent years. Current deep learning models for code summarization generally follow the principle in neural machine translation and adopt the encoder-decoder framework, where the encoder learns the semantic representations from source code and the decoder transforms the learnt representations into human-readable text that describes the functionality of code snippets. Despite they achieve the new state-of-the-art performance, we notice that current models often either generate less fluent summaries, or fail to capture the core functionality, since they usually focus on a single type of code representations. As such we propose GypSum, a new deep learning model that learns hybrid representations using graph attention neural networks and a pre-trained programming and natural language model. We introduce particular edges related to the control flow of a code snippet into the abstract syntax tree for graph construction, and design two encoders to learn from the graph and the token sequence of source code, respectively. We modify the encoder-decoder sublayer in the Transformer's decoder to fuse the representations and propose a dual-copy mechanism to facilitate summary generation. Experimental results demonstrate the superior performance of GypSum over existing code summarization models.