Applying Large Language Models API to Issue Classification Problem

TL;DR

This paper presents a GPT-based method for issue report classification in software engineering that maintains high accuracy with limited training data, improving scalability and efficiency.

Contribution

The study introduces a novel GPT-driven approach for issue prioritization that performs well with small datasets, reducing the need for extensive training.

Findings

Achieved up to 93.2% precision in issue type prediction

Attained 95% recall, demonstrating high sensitivity

Reached 89.3% F1-score, indicating balanced accuracy

Abstract

Effective prioritization of issue reports is crucial in software engineering to optimize resource allocation and address critical problems promptly. However, the manual classification of issue reports for prioritization is laborious and lacks scalability. Alternatively, many open source software (OSS) projects employ automated processes for this task, albeit relying on substantial datasets for adequate training. This research seeks to devise an automated approach that ensures reliability in issue prioritization, even when trained on smaller datasets. Our proposed methodology harnesses the power of Generative Pre-trained Transformers (GPT), recognizing their potential to efficiently handle this task. By leveraging the capabilities of such models, we aim to develop a robust system for prioritizing issue reports accurately, mitigating the necessity for extensive training data while maintaining reliability. In our research, we have developed a reliable GPT-based approach to accurately label and prioritize issue reports with a reduced training dataset. By reducing reliance on massive data requirements and focusing on few-shot fine-tuning, our methodology offers a more accessible and efficient solution for issue prioritization in software engineering. Our model predicted issue types in individual projects up to 93.2% in precision, 95% in recall, and 89.3% in F1-score.