Language Shapes Mental Health Evaluations in Large Language Models

TL;DR

This paper demonstrates that large language models exhibit cross-linguistic differences in mental health evaluations, with higher stigma responses in Chinese and systematic shifts in downstream decision accuracy, highlighting language's impact on model outputs.

Contribution

It reveals that prompt language systematically influences mental health evaluations in LLMs, affecting stigma assessment and decision accuracy across Chinese and English prompts.

Findings

Models produce higher stigma responses in Chinese prompts.

Sensitivity to stigmatizing content varies by language.

Predicted depression severity shifts systematically with language.

Abstract

This study investigates whether large language models (LLMs) exhibit cross-linguistic differences in mental health evaluations. Focusing on Chinese and English, we examine two widely used models, GPT-4o and Qwen3, to assess whether prompt language systematically shifts mental health-related evaluations and downstream decision outcomes. First, we assess models' evaluative orientation toward mental health stigma using multiple validated measurement scales capturing social stigma, self-stigma, and professional stigma. Across all measures, both models produce higher stigma-related responses when prompted in Chinese than in English. Second, we examine whether these differences also manifest in two common downstream decision tasks in mental health. In a binary mental health stigma detection task, sensitivity to stigmatizing content varies across language prompts, with lower sensitivity observed under Chinese prompts. In a depression severity classification task, predicted severity also differs by prompt language, with Chinese prompts associated with more underestimation errors, indicating a systematic downward shift in predicted severity relative to English prompts. Together, these findings suggest that language context can systematically shape evaluative patterns in LLM outputs and shift decision thresholds in downstream tasks.