Automatic techniques for issue report classification: A systematic mapping study

TL;DR

This systematic mapping study reviews 46 research papers on automatic issue report classification, highlighting current techniques, gaps in practical evaluation, and future research directions.

Contribution

It provides the first comprehensive overview of automatic issue report classification techniques, including traditional ML, deep learning, and large language models, and identifies research gaps.

Findings

Various techniques used, including machine learning and deep learning.

Lack of practitioner involvement in existing studies.

Limited focus on factors beyond prediction accuracy.

Abstract

Several studies have evaluated automatic techniques for classifying software issue reports to assist practitioners in effectively assigning relevant resources based on the type of issue. Currently, no comprehensive overview of this area has been published. A comprehensive overview will help identify future research directions and provide an extensive collection of potentially relevant existing solutions. This study aims to provide a comprehensive overview of the use of automatic techniques to classify issue reports. We conducted a systematic mapping study and identified 46 studies on the topic. The study results indicate that the existing literature applies various techniques for classifying issue reports, including traditional machine learning and deep learning-based techniques and more advanced large language models. Furthermore, we observe that these studies (a) lack the involvement of practitioners, (b) do not consider other potentially relevant adoption factors beyond prediction accuracy, such as the explainability, scalability, and generalizability of the techniques, and (c) mainly rely on archival data from open-source repositories only. Therefore, future research should focus on real industrial evaluations, consider other potentially relevant adoption factors, and actively involve practitioners.