Utilizing Source Code Syntax Patterns to Detect Bug Inducing Commits using Machine Learning Models

TL;DR

This paper introduces a novel method for extracting source code syntax pattern features to improve bug-inducing commit detection using machine learning, demonstrating enhanced performance across multiple datasets and models.

Contribution

It proposes a new feature extraction approach based on source code syntax patterns, improving bug prediction accuracy over traditional features.

Findings

Proposed features improve bug detection performance across five ML models.

Features enhance detection accuracy when combined with Deep Belief Network.

Our features offer better explainability of bug predictions.

Abstract

Detecting Bug Inducing Commit (BIC) or Just in Time (JIT) defect prediction using Machine Learning (ML) based models requires tabulated feature values extracted from the source code or historical maintenance data of a software system. Existing studies have utilized meta-data from source code repositories (we named them GitHub Statistics or GS), n-gram-based source code text processing, and developer's information (e.g., the experience of a developer) as the feature values in ML-based bug detection models. However, these feature values do not represent the source code syntax styles or patterns that a developer might prefer over available valid alternatives provided by programming languages. This investigation proposed a method to extract features from its source code syntax patterns to represent software commits and investigate whether they are helpful in detecting bug proneness in software systems. We utilize six manually and two automatically labeled datasets from eight open-source software projects written in Java, C++, and Python programming languages. Our datasets contain 642 manually labeled and 4,014 automatically labeled buggy and non-buggy commits from six and two subject systems, respectively. The subject systems contain a diverse number of revisions, and they are from various application domains. Our investigation shows the inclusion of the proposed features increases the performance of detecting buggy and non-buggy software commits using five different machine learning classification models. Our proposed features also perform better in detecting buggy commits using the Deep Belief Network generated features and classification model. This investigation also implemented a state-of-the-art tool to compare the explainability of predicted buggy commits using our proposed and traditional features and found that our proposed features provide better reasoning about buggy.....