Decoding User Concerns in AI Health Chatbots: An Exploration of Security and Privacy in App Reviews

TL;DR

This paper evaluates automated methods for detecting user privacy and security concerns in AI health chatbot reviews, highlighting their effectiveness and limitations, and emphasizing the need for improved privacy practices.

Contribution

It introduces a benchmarking of BART and Gemini GenAI for classifying privacy concerns in health chatbot reviews, revealing their performance and challenges compared to manual analysis.

Findings

Gemini GenAI performs comparably to manual labeling in classifying security and privacy concerns.

Both automated methods often misclassify unrelated issues, indicating room for improvement.

Qualitative analysis uncovers key user concerns like data misuse and lack of transparency.

Abstract

AI powered health chatbot applications are increasingly utilized for personalized healthcare services, yet they pose significant challenges related to user data security and privacy. This study evaluates the effectiveness of automated methods, specifically BART and Gemini GenAI, in identifying security privacy related (SPR) concerns within these applications' user reviews, benchmarking their performance against manual qualitative analysis. Our results indicate that while Gemini's performance in SPR classification is comparable to manual labeling, both automated methods have limitations, including the misclassification of unrelated issues. Qualitative analysis revealed critical user concerns, such as data collection practices, data misuse, and insufficient transparency and consent mechanisms. This research enhances the understanding of the relationship between user trust, privacy, and emerging mobile AI health chatbot technologies, offering actionable insights for improving security and privacy practices in AI driven health chatbots. Although exploratory, our findings highlight the necessity for rigorous audits and transparent communication strategies, providing valuable guidance for app developers and vendors in addressing user security and privacy concerns.