FacePsy: An Open-Source Affective Mobile Sensing System -- Analyzing Facial Behavior and Head Gesture for Depression Detection in Naturalistic Settings

TL;DR

This paper introduces FacePsy, an open-source mobile sensing system that analyzes facial behavior and head gestures in naturalistic settings to detect depression with promising accuracy, aiming to improve mental health monitoring.

Contribution

The study presents FacePsy, a novel mobile affective sensing system capable of real-time facial and head gesture analysis for depression detection in everyday environments.

Findings

Eye-open states, head gestures, and specific facial Action Units are significant depression indicators.

The regression model predicts PHQ-9 scores with a Mean Absolute Error of 3.08.

FacePsy achieves an AUROC of 81% for depression detection.

Abstract

Depression, a prevalent and complex mental health issue affecting millions worldwide, presents significant challenges for detection and monitoring. While facial expressions have shown promise in laboratory settings for identifying depression, their potential in real-world applications remains largely unexplored due to the difficulties in developing efficient mobile systems. In this study, we aim to introduce FacePsy, an open-source mobile sensing system designed to capture affective inferences by analyzing sophisticated features and generating real-time data on facial behavior landmarks, eye movements, and head gestures -- all within the naturalistic context of smartphone usage with 25 participants. Through rigorous development, testing, and optimization, we identified eye-open states, head gestures, smile expressions, and specific Action Units (2, 6, 7, 12, 15, and 17) as significant indicators of depressive episodes (AUROC=81%). Our regression model predicting PHQ-9 scores achieved moderate accuracy, with a Mean Absolute Error of 3.08. Our findings offer valuable insights and implications for enhancing deployable and usable mobile affective sensing systems, ultimately improving mental health monitoring, prediction, and just-in-time adaptive interventions for researchers and developers in healthcare.