Unveiling Inclusiveness-Related User Feedback in Mobile Applications

TL;DR

This paper analyzes user feedback from online platforms to identify inclusiveness issues in mobile apps, proposing a taxonomy and demonstrating the potential of large language models for automated detection.

Contribution

It introduces a taxonomy of inclusiveness concerns and evaluates GPT-4 based models for automated identification of related user feedback.

Findings

Identified five major categories of inclusiveness issues.

GPT-4 can effectively detect inclusiveness-related feedback.

Provides practical recommendations for developers.

Abstract

In an era of rapidly expanding software usage, catering to the diverse needs of users from various backgrounds has become a critical challenge. Inclusiveness, representing a core human value, is frequently overlooked during software development, leading to user dissatisfaction. Users often engage in discourse on online platforms where they indicate their concerns. In this study, we leverage user feedback from three popular online sources Reddit, Google Play Store, and X, for 50 of the most popular apps in the world. Using a Socio-Technical Grounded Theory approach, we analyzed 22,000 posts across the three sources. We organize our empirical results in a taxonomy for inclusiveness comprising 5 major categories: Algorithmic Bias, Technology, Demography, Accessibility, and Other Human Values. To explore automated support for identifying inclusiveness-related posts, we experimented with a large language model (GPT4o-mini) and found that it is capable of identifying inclusiveness-related user feedback. We provide implications and recommendations that can help software practitioners to better identify inclusiveness issues to support a wider range of users