Mobile Application Review Summarization using Chain of Density Prompting

TL;DR

This paper introduces a novel Chain of Density prompt technique using GPT-4 to generate dense, interpretable summaries of mobile app reviews, helping users make informed decisions amidst large review volumes.

Contribution

It presents a new prompt engineering method that enhances review summarization quality and readability, specifically tailored for mobile app user feedback.

Findings

CoD prompt improves key theme extraction from reviews

Summaries maintain high readability and semantic density

Empirical evaluation confirms effectiveness with user study

Abstract

Mobile app users commonly rely on app store ratings and reviews to find apps that suit their needs. However, the sheer volume of reviews available on app stores can lead to information overload, thus impeding users' ability to make informed app selection decisions. To address this challenge, we leverage Large Language Models (LLMs) to summarize mobile app reviews. In particular, we use the Chain of Density (CoD) prompt to guide OpenAI GPT-4 to generate abstractive, semantically dense, and easily interpretable summaries of mobile app reviews. The CoD prompt is engineered to iteratively extract salient entities from the source text and fuse them into a fixed-length summary. We evaluate the performance of our approach using a large dataset of mobile app reviews. We further conduct an empirical evaluation with 48 study participants to assess the readability of the generated summaries. Our results demonstrate that adapting the CoD prompt to focus on app features improves its ability to extract key themes from user reviews and generate natural language summaries tailored for end-user consumption. The prompt also manages to maintain the readability of the generated summaries while increasing their semantic density. Our work in this paper aims to improve mobile app users' experience by providing an effective mechanism for summarizing important user feedback in the review stream.