npm-filter: Automating the mining of dynamic information from npm packages

TL;DR

npm-filter is an automated tool that enables researchers to extract dynamic runtime information from npm packages, facilitating more comprehensive analysis of JavaScript projects beyond static properties.

Contribution

The paper introduces npm-filter, a novel automation tool that streamlines collecting dynamic data from npm packages, which was previously difficult to obtain.

Findings

npm-filter successfully automates testing and data collection from npm packages.

It has been used to develop evaluation suites for JavaScript tools.

The tool enhances research capabilities by providing dynamic project insights.

Abstract

The static properties of code repositories, e.g., lines of code, dependents, dependencies, etc. can be readily scraped from code hosting platforms such as GitHub, and from package management systems such as npm for JavaScript; Although no less important, information related to the dynamic properties of programs, e.g., number of tests in a test suite that pass or fail, is less readily available. The ability to easily collect this dynamic information could be immensely useful to researchers conducting corpus analyses, as they could differentiate projects based on properties that can only be observed by running them. In this paper, we present npm-filter, an automated tool that can download, install, build, test, and run custom user scripts over the source code of JavaScript projects available on npm, the most popular JavaScript package manager. We outline this tool, describe its implementation, and show that npm-filter has already been useful in developing evaluation suites for multiple JavaScript tools.