"I Like Sunnie More Than I Expected!": Exploring User Expectation and Perception of an Anthropomorphic LLM-based Conversational Agent for Well-Being Support

TL;DR

This study compares user expectations and perceptions of two LLM-based mental well-being systems, finding that anthropomorphic design enhances relational warmth and overall user engagement in mental health support.

Contribution

It demonstrates that anthropomorphic conversational design in LLMs increases relational warmth and user engagement in mental health interventions.

Findings

Both systems exceeded expectations in utility.

Sunnie outperformed baseline in relational warmth.

High user engagement with both systems.

Abstract

The human-computer interaction (HCI) research community has a longstanding interest in exploring the mismatch between users' actual experiences and expectation toward new technologies, for instance, large language models (LLMs). In this study, we compared users' (N = 38) initial expectations against their post-interaction perceptions of two LLM-powered mental well-being intervention activity recommendation systems. Both systems have a built-in LLM to recommend a personalized well-being intervention activity, but one system (Sunnie) has an anthropomorphic conversational interaction design via elements such as appearance, persona, and natural conversation. Results showed that user engagement was high with both systems, and both systems exceeded users' expectations along the utility dimension, highlighting AI's potential to offer useful intervention activity recommendations. In addition, Sunnie further outperformed the non-anthropomorphic baseline system in relational warmth. These findings suggest that anthropomorphic conversational interaction design may be particularly effective in fostering warmth in mental health support contexts.