An Automatically Created Novel Bug Dataset and its Validation in Bug Prediction

TL;DR

This paper introduces BugHunter, an automatically generated bug dataset capturing buggy and fixed code states over narrow timeframes, and demonstrates its effectiveness in building high-accuracy bug prediction models.

Contribution

The paper presents a novel, automatically created bug dataset that captures bug states over narrow timeframes, improving bug prediction model training.

Findings

Achieved F-measure over 0.74 in bug prediction models

Introduced a new dataset capturing bug and fix states over narrow timeframes

Validated the dataset's usefulness in bug prediction tasks

Abstract

Bugs are inescapable during software development due to frequent code changes, tight deadlines, etc.; therefore, it is important to have tools to find these errors. One way of performing bug identification is to analyze the characteristics of buggy source code elements from the past and predict the present ones based on the same characteristics, using e.g. machine learning models. To support model building tasks, code elements and their characteristics are collected in so-called bug datasets which serve as the input for learning. We present the \emph{BugHunter Dataset}: a novel kind of automatically constructed and freely available bug dataset containing code elements (files, classes, methods) with a wide set of code metrics and bug information. Other available bug datasets follow the traditional approach of gathering the characteristics of all source code elements (buggy and non-buggy) at only one or more pre-selected release versions of the code. Our approach, on the other hand, captures the buggy and the fixed states of the same source code elements from the narrowest timeframe we can identify for a bug's presence, regardless of release versions. To show the usefulness of the new dataset, we built and evaluated bug prediction models and achieved F-measure values over 0.74.