What to Prioritize? Natural Language Processing for the Development of a Modern Bug Tracking Solution in Hardware Development

TL;DR

This paper explores machine learning techniques, especially text embeddings and classifiers, to predict bug report resolution time, risk, and complexity in hardware development, aiming to improve bug prioritization and management.

Contribution

It introduces a novel approach combining text embeddings and machine learning models to predict bug report attributes, outperforming existing methods.

Findings

Universal Sentence Encoder with MLP yields best performance

Text embeddings significantly improve prediction accuracy

Active learning impacts model effectiveness

Abstract

Managing large numbers of incoming bug reports and finding the most critical issues in hardware development is time consuming, but crucial in order to reduce development costs. In this paper, we present an approach to predict the time to fix, the risk and the complexity of debugging and resolution of a bug report using different supervised machine learning algorithms, namely Random Forest, Naive Bayes, SVM, MLP and XGBoost. Further, we investigate the effect of the application of active learning and we evaluate the impact of different text representation techniques, namely TF-IDF, Word2Vec, Universal Sentence Encoder and XLNet on the model's performance. The evaluation shows that a combination of text embeddings generated through the Universal Sentence Encoder and MLP as classifier outperforms all other methods, and is well suited to predict the risk and complexity of bug tickets.