Interpretable Depression Detection from Social Media Text Using LLM-Derived Embeddings

TL;DR

This paper compares large language models and traditional classifiers for detecting depression and related mental health conditions from social media text, highlighting the strengths and limitations of zero-shot LLMs versus LLM-generated embeddings.

Contribution

It introduces a comprehensive evaluation of LLMs and traditional classifiers across multiple mental health classification tasks using social media data, emphasizing the potential of LLM-derived embeddings.

Findings

Zero-shot LLMs excel in binary depression classification.

Summary embeddings from LLMs outperform traditional text embeddings in some tasks.

Classifiers trained on LLM-generated summaries show competitive or superior performance.

Abstract

Accurate and interpretable detection of depressive language in social media is useful for early interventions of mental health conditions, and has important implications for both clinical practice and broader public health efforts. In this paper, we investigate the performance of large language models (LLMs) and traditional machine learning classifiers across three classification tasks involving social media data: binary depression classification, depression severity classification, and differential diagnosis classification among depression, PTSD, and anxiety. Our study compares zero-shot LLMs with supervised classifiers trained on both conventional text embeddings and LLM-generated summary embeddings. Our experiments reveal that while zero-shot LLMs demonstrate strong generalization capabilities in binary classification, they struggle with fine-grained ordinal classifications. In contrast, classifiers trained on summary embeddings generated by LLMs demonstrate competitive, and in some cases superior, performance on the classification tasks, particularly when compared to models using traditional text embeddings. Our findings demonstrate the strengths of LLMs in mental health prediction, and suggest promising directions for better utilization of their zero-shot capabilities and context-aware summarization techniques.