DepressionEmo: A novel dataset for multilabel classification of depression emotions

TL;DR

This paper introduces DepressionEmo, a new dataset of Reddit posts annotated for depression-related emotions, and evaluates various machine learning and deep learning models for emotion classification, highlighting BART's superior performance.

Contribution

The paper presents a novel, publicly available dataset for depression emotion classification and compares multiple models, with BART achieving the highest F1 score.

Findings

BART outperforms other models with F1-Macro of 0.76.

Suicide intent emotion has the highest F1-Macro among emotions.

The dataset shows reliable annotation and useful insights into depression-related emotions.

Abstract

Emotions are integral to human social interactions, with diverse responses elicited by various situational contexts. Particularly, the prevalence of negative emotional states has been correlated with negative outcomes for mental health, necessitating a comprehensive analysis of their occurrence and impact on individuals. In this paper, we introduce a novel dataset named DepressionEmo designed to detect 8 emotions associated with depression by 6037 examples of long Reddit user posts. This dataset was created through a majority vote over inputs by zero-shot classifications from pre-trained models and validating the quality by annotators and ChatGPT, exhibiting an acceptable level of interrater reliability between annotators. The correlation between emotions, their distribution over time, and linguistic analysis are conducted on DepressionEmo. Besides, we provide several text classification methods classified into two groups: machine learning methods such as SVM, XGBoost, and Light GBM; and deep learning methods such as BERT, GAN-BERT, and BART. The pretrained BART model, bart-base allows us to obtain the highest F1- Macro of 0.76, showing its outperformance compared to other methods evaluated in our analysis. Across all emotions, the highest F1-Macro value is achieved by suicide intent, indicating a certain value of our dataset in identifying emotions in individuals with depression symptoms through text analysis. The curated dataset is publicly available at: https://github.com/abuBakarSiddiqurRahman/DepressionEmo.