Large Language Model Agents for Improving Engagement with Behavior Change Interventions: Application to Digital Mindfulness

TL;DR

This study investigates how large language model agents can enhance user engagement in digital mindfulness exercises by providing social support, demonstrating that specific LLM configurations can significantly improve practice adherence.

Contribution

The paper presents the first in situ evaluation of LLM agents supporting behavior change, comparing different agent types and their effects on engagement in digital health interventions.

Findings

Information-providing LLM with a friendly persona increased engagement.

Both LLM types improved users' intentions to practice mindfulness.

LLMs can potentially bridge social support gaps in digital health.

Abstract

Although engagement in self-directed wellness exercises typically declines over time, integrating social support such as coaching can sustain it. However, traditional forms of support are often inaccessible due to the high costs and complex coordination. Large Language Models (LLMs) show promise in providing human-like dialogues that could emulate social support. Yet, in-depth, in situ investigations of LLMs to support behavior change remain underexplored. We conducted two randomized experiments to assess the impact of LLM agents on user engagement with mindfulness exercises. First, a single-session study, involved 502 crowdworkers; second, a three-week study, included 54 participants. We explored two types of LLM agents: one providing information and another facilitating self-reflection. Both agents enhanced users' intentions to practice mindfulness. However, only the information-providing LLM, featuring a friendly persona, significantly improved engagement with the exercises. Our findings suggest that specific LLM agents may bridge the social support gap in digital health interventions.