Towards Automatically Generating Release Notes using Extractive Summarization Technique

TL;DR

This paper presents an automated approach for generating release notes using an enhanced TextRank extractive summarization method that incorporates GloVe embeddings, significantly reducing manual effort and improving summary quality.

Contribution

The study introduces a novel integration of GloVe embeddings with TextRank for better keyword extraction in release note generation, outperforming LSA in evaluations.

Findings

Improved TextRank with GloVe outperforms LSA in ROUGE scores.

Automated method reduces manual effort in creating release notes.

Evaluation with human judgment confirms the effectiveness of the approach.

Abstract

Release notes are admitted as an essential document by practitioners. They contain the summary of the source code changes for the software releases, such as issue fixes, added new features, and performance improvements. Manually producing release notes is a time-consuming and challenging task. For that reason, sometimes developers neglect to write release notes. For example, we collect data from GitHub with over 1,900 releases, among them 37% of the release notes are empty. We propose an automatic generate release notes approach based on the commit messages and merge pull-request (PR) titles to mitigate this problem. We implement one of the popular extractive text summarization techniques, i.e., the TextRank algorithm. However, accurate keyword extraction is a vital issue in text processing. The keyword matching and topic extraction process of the TextRank algorithm ignores the semantic similarity among texts. To improve the keyword extraction method, we integrate the GloVe word embedding technique with TextRank. We develop a dataset with 1,213 release notes (after null filtering) and evaluate the generated release notes through the ROUGE metric and human evaluation. We also compare the performance of our technique with another popular extractive algorithm, latent semantic analysis (LSA). Our evaluation results show that the improved TextRank method outperforms LSA.