Gender and Racial Fairness in Depression Research using Social Media

TL;DR

This study investigates biases in depression detection models trained on social media data, revealing performance disparities across gender and racial groups and providing guidelines to mitigate such biases.

Contribution

It is the first to quantify how demographic biases manifest in depression classifiers using social media data and offers strategies to improve fairness.

Findings

Model performance varies significantly across demographic groups.

Disparities are not solely due to data representation issues.

Recommendations for reducing bias in future research.

Abstract

Multiple studies have demonstrated that behavior on internet-based social media platforms can be indicative of an individual's mental health status. The widespread availability of such data has spurred interest in mental health research from a computational lens. While previous research has raised concerns about possible biases in models produced from this data, no study has quantified how these biases actually manifest themselves with respect to different demographic groups, such as gender and racial/ethnic groups. Here, we analyze the fairness of depression classifiers trained on Twitter data with respect to gender and racial demographic groups. We find that model performance systematically differs for underrepresented groups and that these discrepancies cannot be fully explained by trivial data representation issues. Our study concludes with recommendations on how to avoid these biases in future research.