Improving Pain Classification using Spatio-Temporal Deep Learning Approaches with Facial Expressions

TL;DR

This paper introduces novel deep learning models that analyze facial expressions and landmarks over time to improve automated pain detection, addressing limitations of subjective self-reporting especially for non-verbal individuals.

Contribution

It presents two innovative approaches combining ConvNeXt, LSTM, and STGCN for spatio-temporal facial analysis in pain classification, using the PEMF dataset for the first time.

Findings

Enhanced accuracy in pain detection using combined spatial-temporal features.

Effective use of ConvNeXt and STGCN models for facial expression analysis.

Demonstrated potential for objective pain assessment methods.

Abstract

Pain management and severity detection are crucial for effective treatment, yet traditional self-reporting methods are subjective and may be unsuitable for non-verbal individuals (people with limited speaking skills). To address this limitation, we explore automated pain detection using facial expressions. Our study leverages deep learning techniques to improve pain assessment by analyzing facial images from the Pain Emotion Faces Database (PEMF). We propose two novel approaches1: (1) a hybrid ConvNeXt model combined with Long Short-Term Memory (LSTM) blocks to analyze video frames and predict pain presence, and (2) a Spatio-Temporal Graph Convolution Network (STGCN) integrated with LSTM to process landmarks from facial images for pain detection. Our work represents the first use of the PEMF dataset for binary pain classification and demonstrates the effectiveness of these models through extensive experimentation. The results highlight the potential of combining spatial and temporal features for enhanced pain detection, offering a promising advancement in objective pain assessment methodologies.