Supporting Effective Goal Setting with LLM-Based Chatbots

TL;DR

This study demonstrates that LLM-based chatbots, when equipped with guidance and feedback features, effectively support users in setting and achieving behavioral goals, outperforming suggestions alone.

Contribution

The paper introduces a novel approach to designing LLM chatbots that operationalize psychological goal-setting frameworks, emphasizing the importance of feedback in enhancing goal quality.

Findings

Feedback significantly improves goal-setting effectiveness.

Guidance alone is helpful but less impactful without feedback.

Adaptive suggestions are less effective than guidance with feedback.

Abstract

Each day, individuals set behavioral goals such as eating healthier, exercising regularly, or increasing productivity. While psychological frameworks (i.e., goal setting and implementation intentions) can be helpful, they often need structured external support, which interactive technologies can provide. We thus explored how large language model (LLM)-based chatbots can apply these frameworks to guide users in setting more effective goals. We conducted a preregistered randomized controlled experiment ($N = 543$) comparing chatbots with different combinations of three design features: guidance, suggestions, and feedback. We evaluated goal quality using subjective and objective measures. We found that, while guidance is already helpful, it is the addition of feedback that makes LLM-based chatbots effective in supporting participants' goal setting. In contrast, adaptive suggestions were less effective. Altogether, our study shows how to design chatbots by operationalizing psychological frameworks to provide effective support for reaching behavioral goals.