Towards Better Comprehension of Breaking Changes in the NPM Ecosystem

TL;DR

This study provides a comprehensive analysis of breaking changes in the NPM ecosystem, including their documentation, detection challenges, and a taxonomy of change types, to improve understanding and detection tools.

Contribution

It introduces a large-scale empirical study, new taxonomies of breaking changes, and insights into developer motivations, enhancing detection and comprehension of breaking changes in JavaScript and TypeScript.

Findings

95.4% of breaking changes are documented by developers

19% of breaking changes are undetectable by regression testing

Identified major reasons for breaking changes: reducing redundancy, improving names, and API design

Abstract

Breaking changes cause a lot of effort to both downstream and upstream developers: downstream developers need to adapt to breaking changes and upstream developers are responsible for identifying and documenting them. In the NPM ecosystem, characterized by frequent code changes and a high tolerance for making breaking changes, the effort is larger. For better comprehension of breaking changes in the NPM ecosystem and to enhance breaking change detection tools, we conduct a large-scale empirical study to investigate breaking changes in the NPM ecosystem. We construct a dataset of explicitly documented breaking changes from 381 popular NPM projects. We find that 95.4% of the detected breaking changes can be covered by developers' documentation, and about 19% of the breaking changes cannot be detected by regression testing. Then in the process of investigating source code of our collected breaking changes, we yield a taxonomy of JavaScript and TypeScript-specific syntactic breaking changes and a taxonomy of major types of behavioral breaking changes. Additionally, we investigate the reasons why developers make breaking changes in NPM and find three major reasons, i.e., to reduce code redundancy, to improve identifier name, and to improve API design, and each category contains several sub-items. We provide actionable implications for future research, e.g., automatic naming and renaming techniques should be applied in JavaScript projects to improve identifier names, future research can try to detect more types of behavioral breaking changes. By presenting the implications, we also discuss the weakness of automatic renaming and breaking change detection approaches.