Quantified Facial Expressiveness for Affective Behavior Analytics

TL;DR

This paper introduces a novel algorithm to quantify facial expressiveness at the video frame level using multimodal features, enabling detailed analysis of human affective behavior.

Contribution

It presents a new continuous expressiveness scoring method that emphasizes action units with high intensity and temporal change, advancing facial behavior analysis.

Findings

Effective in capturing temporal expressiveness changes

Able to measure subjective differences in context

Extracts useful insights from benchmark datasets

Abstract

The quantified measurement of facial expressiveness is crucial to analyze human affective behavior at scale. Unfortunately, methods for expressiveness quantification at the video frame-level are largely unexplored, unlike the study of discrete expression. In this work, we propose an algorithm that quantifies facial expressiveness using a bounded, continuous expressiveness score using multimodal facial features, such as action units (AUs), landmarks, head pose, and gaze. The proposed algorithm more heavily weights AUs with high intensities and large temporal changes. The proposed algorithm can compute the expressiveness in terms of discrete expression, and can be used to perform tasks including facial behavior tracking and subjectivity quantification in context. Our results on benchmark datasets show the proposed algorithm is effective in terms of capturing temporal changes and expressiveness, measuring subjective differences in context, and extracting useful insight.