Predicting Line-Level Defects by Capturing Code Contexts with Hierarchical Transformers

TL;DR

Bugsplorer is a hierarchical transformer-based deep learning model that predicts defective lines in code with significantly improved accuracy, helping prioritize software quality assurance efforts effectively.

Contribution

The paper introduces Bugsplorer, a novel hierarchical transformer model specifically designed for line-level defect prediction, outperforming existing techniques.

Findings

Bugsplorer achieves 26-72% better accuracy than state-of-the-art methods.

It ranks the top 20% defective lines within the top 1-3% suspicious lines.

The approach can significantly reduce software quality assurance costs.

Abstract

Software defects consume 40% of the total budget in software development and cost the global economy billions of dollars every year. Unfortunately, despite the use of many software quality assurance (SQA) practices in software development (e.g., code review, continuous integration), defects may still exist in the official release of a software product. Therefore, prioritizing SQA efforts for the vulnerable areas of the codebase is essential to ensure the high quality of a software release. Predicting software defects at the line level could help prioritize the SQA effort but is a highly challenging task given that only ~3% of lines of a codebase could be defective. Existing works on line-level defect prediction often fall short and cannot fully leverage the line-level defect information. In this paper, we propose Bugsplorer, a novel deep-learning technique for line-level defect prediction. It leverages a hierarchical structure of transformer models to represent two types of code elements: code tokens and code lines. Unlike the existing techniques that are optimized for file-level defect prediction, Bugsplorer is optimized for a line-level defect prediction objective. Our evaluation with five performance metrics shows that Bugsplorer has a promising capability of predicting defective lines with 26-72% better accuracy than that of the state-of-the-art technique. It can rank the first 20% defective lines within the top 1-3% suspicious lines. Thus, Bugsplorer has the potential to significantly reduce SQA costs by ranking defective lines higher.