We Care: Multimodal Depression Detection and Knowledge Infused Mental Health Therapeutic Response Generation

TL;DR

This paper introduces a multimodal dataset and a virtual mental health agent that detects depression and provides CBT-based responses, leveraging large language models and achieving promising results in real-world settings.

Contribution

It presents the Extended D-vlog dataset for depression detection and develops a virtual agent using LLMs for therapeutic response generation in real-life scenarios.

Findings

Mistral model achieved 70.1% distortion assessment accuracy

Mistral model achieved 30.9% classification accuracy

TVLT model achieved an F1-score of 67.8% on the dataset

Abstract

The detection of depression through non-verbal cues has gained significant attention. Previous research predominantly centred on identifying depression within the confines of controlled laboratory environments, often with the supervision of psychologists or counsellors. Unfortunately, datasets generated in such controlled settings may struggle to account for individual behaviours in real-life situations. In response to this limitation, we present the Extended D-vlog dataset, encompassing a collection of 1, 261 YouTube vlogs. Additionally, the emergence of large language models (LLMs) like GPT3.5, and GPT4 has sparked interest in their potential they can act like mental health professionals. Yet, the readiness of these LLM models to be used in real-life settings is still a concern as they can give wrong responses that can harm the users. We introduce a virtual agent serving as an initial contact for mental health patients, offering Cognitive Behavioral Therapy (CBT)-based responses. It comprises two core functions: 1. Identifying depression in individuals, and 2. Delivering CBT-based therapeutic responses. Our Mistral model achieved impressive scores of 70.1% and 30.9% for distortion assessment and classification, along with a Bert score of 88.7%. Moreover, utilizing the TVLT model on our Multimodal Extended D-vlog Dataset yielded outstanding results, with an impressive F1-score of 67.8%