Dynamic Summary Generation for Interpretable Multimodal Depression Detection

TL;DR

This paper introduces a multi-stage framework using large language models to improve the accuracy and interpretability of multimodal depression detection through progressive summaries and transparent reasoning.

Contribution

It presents a novel coarse-to-fine approach that combines LLM-generated summaries with multimodal data fusion for more reliable depression screening.

Findings

Significant accuracy improvements over existing methods.

Enhanced interpretability through clinical summaries.

Effective integration of text, audio, and video features.

Abstract

Depression remains widely underdiagnosed and undertreated because stigma and subjective symptom ratings hinder reliable screening. To address this challenge, we propose a coarse-to-fine, multi-stage framework that leverages large language models (LLMs) for accurate and interpretable detection. The pipeline performs binary screening, five-class severity classification, and continuous regression. At each stage, an LLM produces progressively richer clinical summaries that guide a multimodal fusion module integrating text, audio, and video features, yielding predictions with transparent rationale. The system then consolidates all summaries into a concise, human-readable assessment report. Experiments on the E-DAIC and CMDC datasets show significant improvements over state-of-the-art baselines in both accuracy and interpretability.