SFF-DA: Sptialtemporal Feature Fusion for Detecting Anxiety Nonintrusively

TL;DR

This paper presents a nonintrusive framework using spatiotemporal feature fusion with 3D CNN and LSTM to detect anxiety from facial and physiological data, addressing data quality and small sample challenges.

Contribution

It introduces a novel spatiotemporal feature fusion framework with a similarity assessment strategy for anxiety detection, improving accuracy in real-world, device-heterogeneous settings.

Findings

Outperforms existing methods on multiple datasets

Effectively handles data quality variability

Addresses small sample size issues

Abstract

Early detection of anxiety is crucial for reducing the suffering of individuals with mental disorders and improving treatment outcomes. Utilizing an mHealth platform for anxiety screening can be particularly practical in improving screening efficiency and reducing costs. However, the effectiveness of existing methods has been hindered by differences in mobile devices used to capture subjects' physical and mental evaluations, as well as by the variability in data quality and small sample size problems encountered in real-world settings. To address these issues, we propose a framework with spatiotemporal feature fusion for detecting anxiety nonintrusively. We use a feature extraction network based on a 3D convolutional network and long short-term memory ("3DCNN+LSTM") to fuse the spatiotemporal features of facial behavior and noncontact physiology, which reduces the impact of uneven data quality. Additionally, we design a similarity assessment strategy to address the issue of deteriorating model accuracy due to small sample sizes. Our framework is validated with a crew dataset from the real world and two public datasets: the University of Burgundy Franche-Comt\'e Psychophysiological (UBFC-Phys) dataset and the Smart Reasoning for Well-being at Home and at Work for Knowledge Work (SWELL-KW) dataset. The experimental results indicate that our framework outperforms the comparison methods.