Exploring Requirements Elicitation from App Store User Reviews Using Large Language Models

TL;DR

This paper explores using Large Language Models to analyze app store reviews for automated requirements elicitation, demonstrating that fine-tuned LLMs can effectively identify useful user feedback to improve mobile app development.

Contribution

It introduces a novel approach of fine-tuning LLMs on app reviews for requirements elicitation, with BERT outperforming other models in accuracy and F1-score.

Findings

BERT achieved 92.40% accuracy in classifying useful reviews.

GEMMA showed high recall at 93.39%, capturing more valuable insights.

LLMs can streamline requirements gathering for user-centric app development.

Abstract

Mobile applications have become indispensable companions in our daily lives. Spanning over the categories from communication and entertainment to healthcare and finance, these applications have been influential in every aspect. Despite their omnipresence, developing apps that meet user needs and expectations still remains a challenge. Traditional requirements elicitation methods like user interviews can be time-consuming and suffer from limited scope and subjectivity. This research introduces an approach leveraging the power of Large Language Models (LLMs) to analyze user reviews for automated requirements elicitation. We fine-tuned three well-established LLMs BERT, DistilBERT, and GEMMA, on a dataset of app reviews labeled for usefulness. Our evaluation revealed BERT's superior performance, achieving an accuracy of 92.40% and an F1-score of 92.39%, demonstrating its effectiveness in accurately classifying useful reviews. While GEMMA displayed a lower overall performance, it excelled in recall (93.39%), indicating its potential for capturing a comprehensive set of valuable user insights. These findings suggest that LLMs offer a promising avenue for streamlining requirements elicitation in mobile app development, leading to the creation of more user-centric and successful applications.