HatCUP: Hybrid Analysis and Attention based Just-In-Time Comment Updating

TL;DR

HatCUP is a novel hybrid analysis and attention-based approach for updating code comments, effectively handling complex code changes by mimicking human editing behavior and outperforming existing methods in accuracy and recall.

Contribution

The paper introduces HatCUP, a new comment updater that combines structure-guided attention, code change analysis, and an edit mechanism to improve comment updating for complex code modifications.

Findings

Outperforms state-of-the-art deep learning approaches by 53.8% in accuracy.

Achieves 31.3% higher recall compared to existing methods.

Demonstrates better overall performance than heuristic-based approaches.

Abstract

When changing code, developers sometimes neglect updating the related comments, bringing inconsistent or outdated comments. These comments increase the cost of program understanding and greatly reduce software maintainability. Researchers have put forward some solutions, such as CUP and HEBCUP, which update comments efficiently for simple code changes (i.e. modifying of a single token), but not good enough for complex ones. In this paper, we propose an approach, named HatCUP (Hybrid Analysis and Attention based Comment UPdater), to provide a new mechanism for comment updating task. HatCUP pays attention to hybrid analysis and information. First, HatCUP considers the code structure change information and introduces a structure-guided attention mechanism combined with code change graph analysis and optimistic data flow dependency analysis. With a generally popular RNN-based encoder-decoder architecture, HatCUP takes the action of the code edits, the syntax, semantics and structure code changes, and old comments as inputs and generates a structural representation of the changes in the current code snippet. Furthermore, instead of directly generating new comments, HatCUP proposes a new edit or non-edit mechanism to mimic human editing behavior, by generating a sequence of edit actions and constructing a modified RNN model to integrate newly developed components. Evaluation on a popular dataset demonstrates that HatCUP outperforms the state-of-the-art deep learning-based approaches (CUP) by 53.8% for accuracy, 31.3% for recall and 14.3% for METEOR of the original metrics. Compared with the heuristic-based approach (HEBCUP), HatCUP also shows better overall performance.