Generative Confidants: How do People Experience Trust in Emotional Support from Generative AI?

TL;DR

This study explores how users develop trust in generative AI for emotional support, highlighting factors like personalization, mental models, and perceived control, with implications for future AI-human interactions.

Contribution

It provides qualitative insights into trust development in AI emotional support, identifying novel drivers and the impact of language use on trust behaviors.

Findings

Personalization fosters familiarity and trust.

Nuanced mental models influence trust perceptions.

Homogeneous, positive language promotes trust but may hinder recognition of AI as a machine.

Abstract

People are increasingly turning to generative AI (e.g., ChatGPT, Gemini, Copilot) for emotional support and companionship. While trust is likely to play a central role in enabling these informal and unsupervised interactions, we still lack an understanding of how people develop and experience it in this context. Seeking to fill this gap, we recruited 24 frequent users of generative AI for emotional support and conducted a qualitative study consisting of diary entries about interactions, transcripts of chats with AI, and in-depth interviews. Our results suggest important novel drivers of trust in this context: familiarity emerging from personalisation, nuanced mental models of generative AI, and awareness of people's control over conversations. Notably, generative AI's homogeneous use of personalised, positive, and persuasive language appears to promote some of these trust-building factors. However, this also seems to discourage other trust-related behaviours, such as remembering that generative AI is a machine trained to converse in human language. We present implications for future research that are likely to become critical as the use of generative AI for emotional support increasingly overlaps with therapeutic work.