Framing Responsible Design of AI for Mental Well-Being: AI as Primary Care, Nutritional Supplement, or Yoga Instructor?

TL;DR

This paper explores responsible design principles for AI tools supporting mental well-being, proposing analogies like supplements and primary care to guide their development and evaluation.

Contribution

It introduces a framework for designing mental health AI tools based on their intended benefits and active ingredients, informed by expert interviews and regulations.

Findings

Designing responsibly involves specifying benefits and guarantees.

Analogies help clarify the roles and risks of AI tools.

Evaluation metrics should align with the intended guarantees.

Abstract

Millions of people now use non-clinical Large Language Model (LLM) tools like ChatGPT for mental well-being support. This paper investigates what it means to design such tools responsibly, and how to operationalize that responsibility in their design and evaluation. By interviewing experts and analyzing related regulations, we found that designing an LLM tool responsibly involves: (1) Articulating the specific benefits it guarantees and for whom. Does it guarantee specific, proven relief, like an over-the-counter drug, or offer minimal guarantees, like a nutritional supplement? (2) Specifying the LLM tool's "active ingredients" for improving well-being and whether it guarantees their effective delivery (like a primary care provider) or not (like a yoga instructor). These specifications outline an LLM tool's pertinent risks, appropriate evaluation metrics, and the respective responsibilities of LLM developers, tool designers, and users. These analogies - LLM tools as supplements, drugs, yoga instructors, and primary care providers - can scaffold further conversations about their responsible design.