Automatic Estimation of Self-Reported Pain by Trajectory Analysis in the Manifold of Fixed Rank Positive Semi-Definite Matrices

TL;DR

This paper introduces an automated facial landmark trajectory analysis method on a Riemannian manifold to estimate self-reported pain levels from videos, demonstrating competitive results on public datasets.

Contribution

It presents a novel approach combining manifold trajectory analysis, curve smoothing, and late fusion for pain estimation from facial landmarks.

Findings

Achieved state-of-the-art performance on UNBC-McMaster dataset.

Effective in modeling facial dynamics for pain assessment.

Demonstrated robustness across different datasets.

Abstract

We propose an automatic method to estimate self-reported pain based on facial landmarks extracted from videos. For each video sequence, we decompose the face into four different regions and the pain intensity is measured by modeling the dynamics of facial movement using the landmarks of these regions. A formulation based on Gram matrices is used for representing the trajectory of landmarks on the Riemannian manifold of symmetric positive semi-definite matrices of fixed rank. A curve fitting algorithm is used to smooth the trajectories and temporal alignment is performed to compute the similarity between the trajectories on the manifold. A Support Vector Regression classifier is then trained to encode extracted trajectories into pain intensity levels consistent with self-reported pain intensity measurement. Finally, a late fusion of the estimation for each region is performed to obtain the final predicted pain level. The proposed approach is evaluated on two publicly available datasets, the UNBCMcMaster Shoulder Pain Archive and the Biovid Heat Pain dataset. We compared our method to the state-of-the-art on both datasets using different testing protocols, showing the competitiveness of the proposed approach.