Quantified Facial Temporal-Expressiveness Dynamics for Affect Analysis

TL;DR

This paper introduces a novel method called Temporal-expressiveness Dynamics (TED) for quantifying facial expressiveness, improving affect analysis by combining static and dynamic facial features, with demonstrated effectiveness on pain recognition data.

Contribution

The work presents a new algorithm for quantifying facial expressiveness using multimodal features, enhancing affect modeling and interpretability in automated systems.

Findings

TED improves affect recognition accuracy.

Effective in summarizing unstructured visual data.

Validated on pain recognition dataset.

Abstract

The quantification of visual affect data (e.g. face images) is essential to build and monitor automated affect modeling systems efficiently. Considering this, this work proposes quantified facial Temporal-expressiveness Dynamics (TED) to quantify the expressiveness of human faces. The proposed algorithm leverages multimodal facial features by incorporating static and dynamic information to enable accurate measurements of facial expressiveness. We show that TED can be used for high-level tasks such as summarization of unstructured visual data, and expectation from and interpretation of automated affect recognition models. To evaluate the positive impact of using TED, a case study was conducted on spontaneous pain using the UNBC-McMaster spontaneous shoulder pain dataset. Experimental results show the efficacy of using TED for quantified affect analysis.