Magnifying Subtle Facial Motions for Effective 4D Expression Recognition

TL;DR

This paper introduces a novel pipeline for 4D facial expression recognition that magnifies subtle facial motions using Riemannian geometry and temporal filtering, significantly improving emotion classification accuracy.

Contribution

It combines Riemannian geometry-based deformation analysis with motion magnification to enhance subtle facial movements for better 4D FER performance.

Findings

Achieved 94.18% accuracy on BU-4DFE dataset.

Magnification improves classification accuracy by over 10%.

Reveals hidden facial deformations for emotion recognition.

Abstract

In this paper, an effective pipeline to automatic 4D Facial Expression Recognition (4D FER) is proposed. It combines two growing but disparate ideas in Computer Vision -- computing the spatial facial deformations using tools from Riemannian geometry and magnifying them using temporal filtering. The flow of 3D faces is first analyzed to capture the spatial deformations based on the recently-developed Riemannian approach, where registration and comparison of neighboring 3D faces are led jointly. Then, the obtained temporal evolution of these deformations are fed into a magnification method in order to amplify the facial activities over the time. The latter, main contribution of this paper, allows revealing subtle (hidden) deformations which enhance the emotion classification performance. We evaluated our approach on BU-4DFE dataset, the state-of-art 94.18% average performance and an improvement that exceeds 10% in classification accuracy, after magnifying extracted geometric features (deformations), are achieved.