Cultural Prompting Improves the Empathy and Cultural Responsiveness of GPT-Generated Therapy Responses

TL;DR

This study shows that cultural prompting significantly enhances GPT-4o's cultural responsiveness and perceived empathy in therapeutic responses for Chinese American caregivers, emphasizing the importance of cultural considerations in AI mental health tools.

Contribution

The paper demonstrates that prompt-based techniques can effectively improve the cultural responsiveness and empathy of LLM-generated therapeutic responses.

Findings

Cultural prompting significantly improved GPT-4o's cultural responsiveness and empathy.

GPT-4o with cultural prompting was the most preferred among participants.

Improvements in DeepSeek-V3 responses were not statistically significant.

Abstract

Large Language Model (LLM)-based conversational agents offer promising solutions for mental health support, but lack cultural responsiveness for diverse populations. This study evaluated the effectiveness of cultural prompting in improving cultural responsiveness and perceived empathy of LLM-generated therapeutic responses for Chinese American family caregivers. Using a randomized controlled experiment, we compared GPT-4o and Deepseek-V3 responses with and without cultural prompting. Thirty-six participants evaluated input-response pairs on cultural responsiveness (competence and relevance) and perceived empathy. Results showed that cultural prompting significantly enhanced GPT-4o's performance across all dimensions, with GPT-4o with cultural prompting being the most preferred, while improvements in DeepSeek-V3 responses were not significant. Mediation analysis revealed that cultural prompting improved empathy through improving cultural responsiveness. This study demonstrated that prompt-based techniques can effectively enhance the cultural responsiveness of LLM-generated therapeutic responses, highlighting the importance of cultural responsiveness in delivering empathetic AI-based therapeutic interventions to culturally and linguistically diverse populations.