The Value of Effective Pull Request Description

TL;DR

This study investigates the role and impact of pull request descriptions in software development, revealing their importance for review outcomes and how their content influences developer engagement.

Contribution

It provides a systematic analysis of PR description elements, their association with review outcomes, and insights into when and why developers include them.

Findings

PR descriptions are valued for rationale and change history.

Specific elements predict review acceptance and engagement.

Descriptions are more common in mature projects and complex changes.

Abstract

In the pull-based development model, code contributions are submitted as pull requests (PRs) to undergo reviews and approval by other developers with the goal of being merged into the code base. A PR can be supported by a description, whose role has not yet been systematically investigated. To fill in this gap, we conducted a mixed-methods empirical study of PR descriptions. We conducted a grey literature review of guidelines on writing PR descriptions and derived a taxonomy of eight recommended elements. Using this taxonomy, we analyzed 80K GitHub PRs across 156 projects and five programming languages to assess associations between these elements and code review outcomes (e.g., merge decision, latency, first response time, review comments, and review iteration cycles). To complement these results, we surveyed 64 developers about the perceived importance of each element. Finally, we analyzed which submission-time factors predict whether PRs include a description and which elements they contain. We found that developers view PR descriptions as important, but their elements matter differently: purpose and code explanations are valued by developers for preserving the rationale and history of changes, while stating the desired feedback type best predicts change acceptance and reviewer engagement. PR descriptions are also more common in mature projects and complex changes, suggesting they are written when most useful rather than as a formality.