Underneath the Numbers: Quantitative and Qualitative Gender Fairness in LLMs for Depression Prediction

TL;DR

This study investigates gender bias in large language models used for depression prediction, comparing quantitative and qualitative fairness, and proposing new methods for qualitative bias assessment.

Contribution

It is the first to evaluate gender bias in LLMs for depression detection using both quantitative and qualitative approaches, and introduces strategies for qualitative fairness analysis.

Findings

ChatGPT performs best across various metrics

LLaMA 2 outperforms others in group fairness

Qualitative analysis reveals themes and explanations used by LLMs

Abstract

Recent studies show bias in many machine learning models for depression detection, but bias in LLMs for this task remains unexplored. This work presents the first attempt to investigate the degree of gender bias present in existing LLMs (ChatGPT, LLaMA 2, and Bard) using both quantitative and qualitative approaches. From our quantitative evaluation, we found that ChatGPT performs the best across various performance metrics and LLaMA 2 outperforms other LLMs in terms of group fairness metrics. As qualitative fairness evaluation remains an open research question we propose several strategies (e.g., word count, thematic analysis) to investigate whether and how a qualitative evaluation can provide valuable insights for bias analysis beyond what is possible with quantitative evaluation. We found that ChatGPT consistently provides a more comprehensive, well-reasoned explanation for its prediction compared to LLaMA 2. We have also identified several themes adopted by LLMs to qualitatively evaluate gender fairness. We hope our results can be used as a stepping stone towards future attempts at improving qualitative evaluation of fairness for LLMs especially for high-stakes tasks such as depression detection.