Automatic Comment Generation via Multi-Pass Deliberation

TL;DR

This paper introduces DECOM, a multi-pass deliberation framework for automatic comment generation that iteratively refines comments based on code and human-like deliberation, outperforming existing methods.

Contribution

The paper proposes a novel multi-pass deliberation approach for comment generation, incorporating iterative polishing and evaluation to produce higher quality comments.

Findings

Outperforms state-of-the-art baselines on Java and Python datasets.

Generated comments are more readable, informative, and useful according to human evaluation.

Effective in producing high-quality comments through iterative refinement.

Abstract

Deliberation is a common and natural behavior in human daily life. For example, when writing papers or articles, we usually first write drafts, and then iteratively polish them until satisfied. In light of such a human cognitive process, we propose DECOM, which is a multi-pass deliberation framework for automatic comment generation. DECOM consists of multiple Deliberation Models and one Evaluation Model. Given a code snippet, we first extract keywords from the code and retrieve a similar code fragment from a pre-defined corpus. Then, we treat the comment of the retrieved code as the initial draft and input it with the code and keywords into DECOM to start the iterative deliberation process. At each deliberation, the deliberation model polishes the draft and generates a new comment. The evaluation model measures the quality of the newly generated comment to determine whether to end the iterative process or not. When the iterative process is terminated, the best-generated comment will be selected as the target comment. Our approach is evaluated on two real-world datasets in Java (87K) and Python (108K), and experiment results show that our approach outperforms the state-of-the-art baselines. A human evaluation study also confirms the comments generated by DECOM tend to be more readable, informative, and useful.