Exploring Large Language Models for Detecting Mental Disorders

TL;DR

This study evaluates various machine learning approaches, including large language models, for detecting depression and anxiety in Russian texts, showing LLMs generally outperform traditional methods especially on noisy or small datasets.

Contribution

It provides a comprehensive comparison of traditional, encoder-based, and large language models for mental disorder detection across diverse Russian-language datasets.

Findings

LLMs outperform traditional methods on noisy and small datasets

Encoder-based models can match LLM performance with targeted clinical data

Performance varies significantly with dataset noise and text genre

Abstract

This paper compares the effectiveness of traditional machine learning methods, encoder-based models, and large language models (LLMs) on the task of detecting depression and anxiety. Five Russian-language datasets were considered, each differing in format and in the method used to define the target pathology class. We tested AutoML models based on linguistic features, several variations of encoder-based Transformers such as BERT, and state-of-the-art LLMs as pathology classification models. The results demonstrated that LLMs outperform traditional methods, particularly on noisy and small datasets where training examples vary significantly in text length and genre. However, psycholinguistic features and encoder-based models can achieve performance comparable to language models when trained on texts from individuals with clinically confirmed depression, highlighting their potential effectiveness in targeted clinical applications.