App Review Driven Collaborative Bug Finding

TL;DR

This paper introduces BugRMSys, a novel approach that leverages app reviews and historical bug reports from similar apps to efficiently identify, reproduce, and report new bugs in mobile applications, demonstrated on six popular apps.

Contribution

The paper presents a new method that uses natural language processing to match app reviews with historical bug reports, enabling faster bug discovery in related apps.

Findings

Identified 20 new bugs across six apps, with some already fixed.

Successfully used DistilBERT for semantic matching of reviews and bug reports.

Demonstrated effectiveness in reproducing and reporting bugs from reviews.

Abstract

Software development teams generally welcome any effort to expose bugs in their code base. In this work, we build on the hypothesis that mobile apps from the same category (e.g., two web browser apps) may be affected by similar bugs in their evolution process. It is therefore possible to transfer the experience of one historical app to quickly find bugs in its new counterparts. This has been referred to as collaborative bug finding in the literature. Our novelty is that we guide the bug finding process by considering that existing bugs have been hinted within app reviews. Concretely, we design the BugRMSys approach to recommend bug reports for a target app by matching historical bug reports from apps in the same category with user app reviews of the target app. We experimentally show that this approach enables us to quickly expose and report dozens of bugs for targeted apps such as Brave (web browser app). BugRMSys's implementation relies on DistilBERT to produce natural language text embeddings. Our pipeline considers similarities between bug reports and app reviews to identify relevant bugs. We then focus on the app review as well as potential reproduction steps in the historical bug report (from a same-category app) to reproduce the bugs. Overall, after applying BugRMSys to six popular apps, we were able to identify, reproduce and report 20 new bugs: among these, 9 reports have been already triaged, 6 were confirmed, and 4 have been fixed by official development teams, respectively.