Automatically Assessing and Extending Code Coverage for NPM Packages

TL;DR

This paper presents TESA, a tool that automatically assembles test suites for npm packages by combining developer tests and dependent package tests, improving code coverage and aiding in performance issue detection.

Contribution

TESA is a novel tool that automatically creates comprehensive test suites for npm packages by leveraging both developer and dependent package tests, enhancing code coverage and analysis.

Findings

TESA significantly increases code coverage for npm packages.

Including dependent package tests improves detection of performance issues.

TESA's assembled test suites enhance dynamic program analysis effectiveness.

Abstract

Typical Node.js applications extensively rely on packages hosted in the npm registry. As such packages may be used by thousands of other packages or applications, it is important to assess their code coverage. Moreover, increasing code coverage may help detect previously unknown issues. In this paper, we introduce TESA, a new tool that automatically assembles a test suite for any package in the npm registry. The test suite includes 1) tests written for the target package and usually hosted in its development repository, and 2) tests selected from dependent packages. The former tests allow assessing the code coverage of the target package, while the latter ones can increase code coverage by exploiting third-party tests that also exercise code in the target package. We use TESA to assess the code coverage of 500 popular npm packages. Then, we demonstrate that TESA can significantly increase code coverage by including tests from dependent packages. Finally, we show that the test suites assembled by TESA increase the effectiveness of existing dynamic program analyses to identify performance issues that are not detectable when only executing the developer's tests.