RepoMiner: a Language-agnostic Python Framework to Mine Software Repositories for Defect Prediction

TL;DR

RepoMiner is a versatile Python framework that automates data collection, labeling, and metric calculation from software repositories, facilitating defect prediction research across multiple programming languages.

Contribution

It introduces a language-agnostic tool that simplifies and automates dataset creation for defect prediction, reducing manual effort and errors.

Findings

Successfully collects failure data from repositories

Automatically labels failure-prone components

Calculates relevant metrics for defect prediction

Abstract

Data originating from open-source software projects provide valuable information to enhance software quality. In the scope of Software Defect Prediction, one of the most challenging parts is extracting valid data about failure-prone software components from these repositories, which can help develop more robust software. In particular, collecting data, calculating metrics, and synthesizing results from these repositories is a tedious and error-prone task, which often requires understanding the programming languages involved in the mined repositories, eventually leading to a proliferation of language-specific data-mining software. This paper presents RepoMiner, a language-agnostic tool developed to support software engineering researchers in creating datasets to support any study on defect prediction. RepoMiner automatically collects failure data from software components, labels them as failure-prone or neutral, and calculates metrics to be used as ground truth for defect prediction models. We present its implementation and provide examples of its application.