Where is Your App Frustrating Users?

TL;DR

This paper introduces SIRA, a semantic-aware approach using a novel BERT+Attr-CRF model to extract, cluster, and visualize problematic app features from user reviews, aiding developers in understanding user frustrations more precisely.

Contribution

The paper presents a new BERT+Attr-CRF model for fine-grained feature extraction and a clustering method for better analysis of app reviews, improving upon previous pattern-based approaches.

Findings

SIRA achieves high accuracy in extracting problematic features.

Clustering reveals meaningful semantic groupings of issues.

Evaluation confirms SIRA's effectiveness on large review datasets.

Abstract

User reviews of mobile apps provide a communication channel for developers to perceive user satisfaction. Many app features that users have problems with are usually expressed by key phrases such as "upload pictures", which could be buried in the review texts. The lack of fine-grained view about problematic features could obscure the developers' understanding of where the app is frustrating users, and postpone the improvement of the apps. Existing pattern-based approaches to extract target phrases suffer from low accuracy due to insufficient semantic understanding of the reviews, thus can only summarize the high-level topics/aspects of the reviews. This paper proposes a semantic-aware, fine-grained app review analysis approach (SIRA) to extract, cluster, and visualize the problematic features of apps. The main component of SIRA is a novel BERT+Attr-CRF model for fine-grained problematic feature extraction, which combines textual descriptions and review attributes to better model the semantics of reviews and boost the performance of the traditional BERT-CRF model. SIRA also clusters the extracted phrases based on their semantic relations and presents a visualization of the summaries. Our evaluation on 3,426 reviews from six apps confirms the effectiveness of SIRA in problematic feature extraction and clustering. We further conduct an empirical study with SIRA on 318,534 reviews of 18 popular apps to explore its potential application and examine its usefulness in real-world practice.