Gendered Mental Health Stigma in Masked Language Models

TL;DR

This paper investigates gendered mental health stigma in masked language models, revealing that models reflect societal biases by associating mental health conditions with women more than men, especially in treatment contexts.

Contribution

It introduces a psychology-grounded framework to evaluate gendered mental health stigma in language models, highlighting nuanced biases and the importance of context in social bias assessment.

Findings

Models predict female subjects more often than male in mental health contexts.

Biases are stronger in sentences indicating treatment-seeking behavior.

Different models capture stigma dimensions differently for men and women.

Abstract

Mental health stigma prevents many individuals from receiving the appropriate care, and social psychology studies have shown that mental health tends to be overlooked in men. In this work, we investigate gendered mental health stigma in masked language models. In doing so, we operationalize mental health stigma by developing a framework grounded in psychology research: we use clinical psychology literature to curate prompts, then evaluate the models' propensity to generate gendered words. We find that masked language models capture societal stigma about gender in mental health: models are consistently more likely to predict female subjects than male in sentences about having a mental health condition (32% vs. 19%), and this disparity is exacerbated for sentences that indicate treatment-seeking behavior. Furthermore, we find that different models capture dimensions of stigma differently for men and women, associating stereotypes like anger, blame, and pity more with women with mental health conditions than with men. In showing the complex nuances of models' gendered mental health stigma, we demonstrate that context and overlapping dimensions of identity are important considerations when assessing computational models' social biases.