Modeling Dynamics of Facial Behavior for Mental Health Assessment

TL;DR

This paper introduces a novel approach to modeling facial expression dynamics using NLP-inspired embeddings, enhancing mental health assessment accuracy for schizophrenia and depression.

Contribution

It applies GloVe embeddings to facial expression clusters, offering a new representation method for mental health-related facial behavior analysis.

Findings

Improved schizophrenia symptom estimation accuracy

Enhanced depression severity regression performance

Potential for better mental health assessment tools

Abstract

Facial action unit (FAU) intensities are popular descriptors for the analysis of facial behavior. However, FAUs are sparsely represented when only a few are activated at a time. In this study, we explore the possibility of representing the dynamics of facial expressions by adopting algorithms used for word representation in natural language processing. Specifically, we perform clustering on a large dataset of temporal facial expressions with 5.3M frames before applying the Global Vector representation (GloVe) algorithm to learn the embeddings of the facial clusters. We evaluate the usefulness of our learned representations on two downstream tasks: schizophrenia symptom estimation and depression severity regression. These experimental results show the potential effectiveness of our approach for improving the assessment of mental health symptoms over baseline models that use FAU intensities alone.