Combining Retrieval and Classification: Balancing Efficiency and Accuracy in Duplicate Bug Report Detection

TL;DR

This paper presents a hybrid transformer-based system for duplicate bug report detection that balances accuracy and efficiency by combining retrieval and classification models, outperforming traditional methods in speed while maintaining high accuracy.

Contribution

The paper introduces a novel hybrid approach that leverages retrieval and classification models together to optimize both speed and accuracy in duplicate bug report detection.

Findings

The hybrid system achieves accuracy comparable to pure classification models.

It significantly outperforms classification models in time efficiency.

It maintains near retrieval-level speed with improved accuracy.

Abstract

In the realm of Duplicate Bug Report Detection (DBRD), conventional methods primarily focus on statically analyzing bug databases, often disregarding the running time of the model. In this context, complex models, despite their high accuracy potential, can be time-consuming, while more efficient models may compromise on accuracy. To address this issue, we propose a transformer-based system designed to strike a balance between time efficiency and accuracy performance. The existing methods primarily address it as either a retrieval or classification task. However, our hybrid approach leverages the strengths of both models. By utilizing the retrieval model, we can perform initial sorting to reduce the candidate set, while the classification model allows for more precise and accurate classification. In our assessment of commonly used models for retrieval and classification tasks, sentence BERT and RoBERTa outperform other baseline models in retrieval and classification, respectively. To provide a comprehensive evaluation of performance and efficiency, we conduct rigorous experimentation on five public datasets. The results reveal that our system maintains accuracy comparable to a classification model, significantly outperforming it in time efficiency and only slightly behind a retrieval model in time, thereby achieving an effective trade-off between accuracy and efficiency.