Faces of the Mind: Unveiling Mental Health States Through Facial Expressions in 11,427 Adolescents

TL;DR

This study leverages a large facial video dataset of adolescents to improve mental health assessment by introducing the Symptom Discrepancy Index, significantly enhancing model accuracy and addressing heterogeneity challenges.

Contribution

We developed the Symptom Discrepancy Index (SDI), a novel metric to quantify and mitigate dataset heterogeneity, boosting model performance in large-scale mental health facial analysis.

Findings

F1 scores increased from ~50% to 80% after removing heterogeneous cases.

Symptom heterogeneity, not model capacity, limits large-scale assessment accuracy.

The SDI method is applicable to other psychometric datasets.

Abstract

Mood disorders such as depression and anxiety often manifest through facial expressions, but existing machine learning algorithms designed to assess these disorders have been hindered by small datasets and limited real-world applicability. To address this gap, we analyzed facial videos of 11,427 participants - a dataset two orders of magnitude larger than those used in previous studies - including standardized facial expression videos and psychological assessments of depression, anxiety, and stress. However, scaling up the dataset introduces significant challenges due to increased symptom heterogeneity, making it difficult for models to learn accurate representations. To address this, we introduced the Symptom Discrepancy Index (SDI), a novel metric for quantifying dataset heterogeneity caused by variability in individual symptoms among samples with identical total scores. By removing the 10% most heterogeneous cases as identified by the SDI, we raised the F1 scores of all models from approximately 50% to 80%. Notably, comparable performance gains were observed within both the retained and excluded subsets. These findings demonstrate symptom heterogeneity, not model capacity, as the principal bottleneck in large scale automated assessment and provide a general solution that is readily applicable to other psychometric data sets.