Dynamic Pose-Robust Facial Expression Recognition by Multi-View Pairwise Conditional Random Forests

TL;DR

This paper introduces a multi-view pairwise conditional random forest approach for robust facial expression recognition in videos, effectively handling pose variations and capturing spatio-temporal patterns to improve accuracy.

Contribution

It proposes a novel multi-view, pairwise conditional random forest method that incorporates head pose estimation for dynamic facial expression recognition.

Findings

Significant improvement over standard Random Forests.

Enhanced accuracy on multi-view video datasets.

Effective handling of pose variations in FER.

Abstract

Automatic facial expression classification (FER) from videos is a critical problem for the development of intelligent human-computer interaction systems. Still, it is a challenging problem that involves capturing high-dimensional spatio-temporal patterns describing the variation of one's appearance over time. Such representation undergoes great variability of the facial morphology and environmental factors as well as head pose variations. In this paper, we use Conditional Random Forests to capture low-level expression transition patterns. More specifically, heterogeneous derivative features (e.g. feature point movements or texture variations) are evaluated upon pairs of images. When testing on a video frame, pairs are created between this current frame and previous ones and predictions for each previous frame are used to draw trees from Pairwise Conditional Random Forests (PCRF) whose pairwise outputs are averaged over time to produce robust estimates. Moreover, PCRF collections can also be conditioned on head pose estimation for multi-view dynamic FER. As such, our approach appears as a natural extension of Random Forests for learning spatio-temporal patterns, potentially from multiple viewpoints. Experiments on popular datasets show that our method leads to significant improvements over standard Random Forests as well as state-of-the-art approaches on several scenarios, including a novel multi-view video corpus generated from a publicly available database.