Can GitHub Issues Help in App Review Classifications?

TL;DR

This paper proposes a novel method to enhance app review classification by augmenting datasets with information extracted from GitHub issues, leading to improved machine learning model performance.

Contribution

It introduces a new approach to augment labeled datasets using GitHub issues, significantly improving classification accuracy for app reviews.

Findings

F1-score increased by 6.3 for bug reports

F1-score increased by 7.2 for feature requests

Optimal auxiliary volume range identified as 0.3 to 0.7

Abstract

App reviews reflect various user requirements that can aid in planning maintenance tasks. Recently, proposed approaches for automatically classifying user reviews rely on machine learning algorithms. A previous study demonstrated that models trained on existing labeled datasets exhibit poor performance when predicting new ones. Therefore, a comprehensive labeled dataset is essential to train a more precise model. In this paper, we propose a novel approach that assists in augmenting labeled datasets by utilizing information extracted from an additional source, GitHub issues, that contains valuable information about user requirements. First, we identify issues concerning review intentions (bug reports, feature requests, and others) by examining the issue labels. Then, we analyze issue bodies and define 19 language patterns for extracting targeted information. Finally, we augment the manually labeled review dataset with a subset of processed issues through the Within-App, Within-Context, and Between-App Analysis methods. We conducted several experiments to evaluate the proposed approach. Our results demonstrate that using labeled issues for data augmentation can improve the F1-score to 6.3 in bug reports and 7.2 in feature requests. Furthermore, we identify an effective range of 0.3 to 0.7 for the auxiliary volume, which provides better performance improvements.