AutoPRTitle: A Tool for Automatic Pull Request Title Generation

TL;DR

AutoPRTitle is a web tool that automatically generates high-quality pull request titles by leveraging a fine-tuned BART model, improving automation and consistency in software development workflows.

Contribution

The paper introduces AutoPRTitle, a novel tool that automatically generates pull request titles using a fine-tuned BART model, addressing a previously overlooked aspect of pull request automation.

Findings

BART outperforms baseline models by up to 40.5% in accuracy.

Evaluators find the tool easy to use and helpful.

AutoPRTitle produces concise, relevant pull request titles.

Abstract

With the rise of the pull request mechanism in software development, the quality of pull requests has gained more attention. Prior works focus on improving the quality of pull request descriptions and several approaches have been proposed to automatically generate pull request descriptions. As an essential component of a pull request, pull request titles have not received a similar level of attention. To further facilitate automation in software development and to help developers in drafting high-quality pull request titles, we introduce AutoPRTitle. AutoPRTitle is specifically designed to automatically generate pull request titles. AutoPRTitle can generate a precise and succinct pull request title based on the pull request description, commit messages, and the associated issue titles. AutoPRTitle is built upon a state-of-the-art text summarization model, BART, which has been pre-trained on large-scale English corpora. We further fine-tuned BART in a pull request dataset containing high-quality pull request titles. We implemented AutoPRTitle as a stand-alone web application. We conducted two sets of evaluations: one concerning the model accuracy and the other concerning the tool usability. For model accuracy, BART outperforms the best baseline by 24.6%, 40.5%, and 23.3%, respectively. For tool usability, the evaluators consider our tool as easy-to-use and useful when creating a pull request title of good quality. Source code: https://github.com/soarsmu/Auto-PR-Title Video demo: https://tinyurl.com/AutoPRTitle