I depended on you and you broke me: An empirical study of manifesting breaking changes in client packages

TL;DR

This study empirically investigates how breaking changes in npm dependencies affect dependent packages, revealing that a significant portion are impacted and often recover through version adjustments, highlighting the importance of understanding dependency impacts.

Contribution

It provides the first empirical analysis of manifesting breaking changes in npm, quantifying their impact and recovery methods in real-world package dependencies.

Findings

12% of dependent packages impacted by breaking changes

44% of breaking changes occur in minor and patch releases

Half of affected packages recover by version adjustments

Abstract

Complex software systems have a network of dependencies. Developers often configure package managers (e.g., npm) to automatically update dependencies with each publication of new releases containing bug fixes and new features. When a dependency release introduces backward-incompatible changes, commonly known as breaking changes, dependent packages may not build anymore. This may indirectly impact downstream packages, but the impact of breaking changes and how dependent packages recover from these breaking changes remain unclear. To close this gap, we investigated the manifestation of breaking changes in the npm ecosystem, focusing on cases where packages' builds are impacted by breaking changes from their dependencies. We measured the extent to which breaking changes affect dependent packages. Our analyses show that around 12% of the dependent packages and 14% of their releases were impacted by a breaking change during updates of non-major releases of their dependencies. We observed that, from all of the manifesting breaking changes, 44% were introduced both in minor and patch releases, which in principle should be backward compatible. Clients recovered themselves from these breaking changes in half of the cases, most frequently by upgrading or downgrading the provider's version without changing the versioning configuration in the package manager. We expect that these results help developers understand the potential impact of such changes and recover from them.