How Do Developers Use Migration Guides? A Case Study of Log4j

TL;DR

This study investigates how developers utilize migration guides, focusing on Log4j, revealing that guides are frequently referenced throughout the migration process, often linked in pull requests, and used beyond major updates.

Contribution

The paper provides an empirical analysis of migration guide usage in real-world projects, highlighting their ongoing importance beyond initial version upgrades.

Findings

Most references to migration guides are links to the entire document.

Developers use migration guides during both major updates and maintenance.

Pull request descriptions frequently cite migration guides.

Abstract

Migration guides are a form of software documentation that helps developers address breaking changes introduced in library version updates. Prior studies have examined documents such as release notes, API reference manuals, and patch notes. However, research that focuses specifically on migration guides remains limited. Improving the usability and coverage of migration guides is essential for helping developers resolve breaking changes efficiently. Yet, we still lack a clear understanding of how migration guides are currently provided and how developers use them in practice. To fill this gap, we first investigate whether libraries known to introduce incompatibilities provide migration guides. We then conduct a detailed case study on Log4j, a library that has experienced large-scale breaking updates in the past. We empirically analyze how developers refer to and use the official migration guide in real-world projects. We find that pull request authors most frequently reference the migration guide in the pull request description, and that most references (82.81%) link to the entire guide rather than specific sections. We also find that developers use migration guides not only during major version updates but also during subsequent maintenance tasks, suggesting that the guides serve as a resource throughout the entire migration process.