Exploring Relevant Artifacts of Release Notes: The Practitioners' Perspective

TL;DR

This paper analyzes the contents of software release notes and surveys practitioners to identify key artifacts and user needs, aiming to improve the quality and relevance of release documentation.

Contribution

It provides an empirical analysis of release note contents and a survey of practitioners, highlighting key artifacts and user preferences to guide better release note practices.

Findings

Key artifacts include issues (29%), pull-requests (32%), commits (19%), CVEs (6%)

Practitioners prefer different information based on user roles, e.g., project managers focus on new features

Analysis of 3,347 release notes from 21 GitHub repositories

Abstract

A software release note is one of the essential documents in the software development life cycle. The software release contains a set of information, e.g., bug fixes and security fixes. Release notes are used in different phases, e.g., requirement engineering, software testing and release management. Different types of practitioners (e.g., project managers and clients) get benefited from the release notes to understand the overview of the latest release. As a result, several studies have been done about release notes production and usage in practice. However, two significant problems (e.g., duplication and inconsistency in release notes contents) exist in producing well-written & well-structured release notes and organizing appropriate information regarding different targeted users' needs. For that reason, practitioners face difficulties in writing and reading the release notes using existing tools. To mitigate these problems, we execute two different studies in our paper. First, we execute an exploratory study by analyzing 3,347 release notes of 21 GitHub repositories to understand the documented contents of the release notes. As a result, we find relevant key artifacts, e.g., issues (29%), pull-requests (32%), commits (19%), and common vulnerabilities and exposures (CVE) issues (6%) in the release note contents. Second, we conduct a survey study with 32 professionals to understand the key information that is included in release notes regarding users' roles. For example, project managers are more interested in learning about new features than less critical bug fixes. Our study can guide future research directions to help practitioners produce the release notes with relevant content and improve the documentation quality.