CoreGen: Contextualized Code Representation Learning for Commit Message Generation

TL;DR

CoreGen introduces a novel method for commit message generation that leverages contextualized code representations using Transformer models, significantly improving performance over existing static embedding approaches.

Contribution

The paper proposes a new contextualized code representation learning strategy for commit message generation, addressing the semantic gap with a Transformer-based approach.

Findings

At least 28.18% improvement in BLEU-4 score over baselines

Effective utilization of contextual information in code representations

Potential for broader application in code-to-text tasks

Abstract

Automatic generation of high-quality commit messages for code commits can substantially facilitate software developers' works and coordination. However, the semantic gap between source code and natural language poses a major challenge for the task. Several studies have been proposed to alleviate the challenge but none explicitly involves code contextual information during commit message generation. Specifically, existing research adopts static embedding for code tokens, which maps a token to the same vector regardless of its context. In this paper, we propose a novel Contextualized code representation learning strategy for commit message Generation (CoreGen). CoreGen first learns contextualized code representations which exploit the contextual information behind code commit sequences. The learned representations of code commits built upon Transformer are then fine-tuned for downstream commit message generation. Experiments on the benchmark dataset demonstrate the superior effectiveness of our model over the baseline models with at least 28.18% improvement in terms of BLEU-4 score. Furthermore, we also highlight the future opportunities in training contextualized code representations on larger code corpus as a solution to low-resource tasks and adapting the contextualized code representation framework to other code-to-text generation tasks.