PreciseBugCollector: Extensible, Executable and Precise Bug-fix Collection

TL;DR

PreciseBugCollector is a novel multi-language bug collection framework that combines bug tracking and injection techniques to create a large, precise, and executable bug dataset for software maintenance and deep learning applications.

Contribution

It introduces a dual-component approach with a bug tracker and bug injector, enabling the collection of large-scale, precise, and executable bug datasets across multiple languages.

Findings

Collected over 1 million bugs from open-source projects

Integrated bugs from NVD, OSS-Fuzz, and project-specific injections

Demonstrated the approach's industrial applicability and precision

Abstract

Bug datasets are vital for enabling deep learning techniques to address software maintenance tasks related to bugs. However, existing bug datasets suffer from precise and scale limitations: they are either small-scale but precise with manual validation or large-scale but imprecise with simple commit message processing. In this paper, we introduce PreciseBugCollector, a precise, multi-language bug collection approach that overcomes these two limitations. PreciseBugCollector is based on two novel components: a) A bug tracker to map the codebase repositories with external bug repositories to trace bug type information, and b) A bug injector to generate project-specific bugs by injecting noise into the correct codebases and then executing them against their test suites to obtain test failure messages. We implement PreciseBugCollector against three sources: 1) A bug tracker that links to the national vulnerability data set (NVD) to collect general-wise vulnerabilities, 2) A bug tracker that links to OSS-Fuzz to collect general-wise bugs, and 3) A bug injector based on 16 injection rules to generate project-wise bugs. To date, PreciseBugCollector comprises 1057818 bugs extracted from 2968 open-source projects. Of these, 12602 bugs are sourced from bug repositories (NVD and OSS-Fuzz), while the remaining 1045216 project-specific bugs are generated by the bug injector. Considering the challenge objectives, we argue that a bug injection approach is highly valuable for the industrial setting, since project-specific bugs align with domain knowledge, share the same codebase, and adhere to the coding style employed in industrial projects.