Transfer Learning for Mining Feature Requests and Bug Reports from Tweets and App Store Reviews

TL;DR

This paper explores the use of transfer learning, particularly BERT models, to improve the classification of feature requests and bug reports from noisy, multilingual user comments on social media and app stores, showing monolingual BERT models outperform traditional methods.

Contribution

It demonstrates the effectiveness of monolingual BERT models over existing approaches for classifying user comments in English and Italian, highlighting the limitations of multilingual models.

Findings

Monolingual BERT models outperform traditional ML methods.

Multilingual BERT models perform worse than traditional methods.

Heavyweight transfer learning models do not always improve performance.

Abstract

Identifying feature requests and bug reports in user comments holds great potential for development teams. However, automated mining of RE-related information from social media and app stores is challenging since (1) about 70% of user comments contain noisy, irrelevant information, (2) the amount of user comments grows daily making manual analysis unfeasible, and (3) user comments are written in different languages. Existing approaches build on traditional machine learning (ML) and deep learning (DL), but fail to detect feature requests and bug reports with high Recall and acceptable Precision which is necessary for this task. In this paper, we investigate the potential of transfer learning (TL) for the classification of user comments. Specifically, we train both monolingual and multilingual BERT models and compare the performance with state-of-the-art methods. We found that monolingual BERT models outperform existing baseline methods in the classification of English App Reviews as well as English and Italian Tweets. However, we also observed that the application of heavyweight TL models does not necessarily lead to better performance. In fact, our multilingual BERT models perform worse than traditional ML methods.