Self-Supervised Approach for Facial Movement Based Optical Flow

TL;DR

This paper introduces a self-supervised method to generate optical flow for faces, improving facial motion analysis and micro-expression recognition by training CNNs on face-specific data.

Contribution

It proposes a novel self-supervised approach to generate face optical flow ground truth and demonstrates its effectiveness in improving facial motion prediction and micro-expression recognition.

Findings

FlowNetS trained on face data outperforms other architectures

Optical flow features improve micro-expression classification accuracy

Self-supervised face optical flow is promising for facial expression analysis

Abstract

Computing optical flow is a fundamental problem in computer vision. However, deep learning-based optical flow techniques do not perform well for non-rigid movements such as those found in faces, primarily due to lack of the training data representing the fine facial motion. We hypothesize that learning optical flow on face motion data will improve the quality of predicted flow on faces. The aim of this work is threefold: (1) exploring self-supervised techniques to generate optical flow ground truth for face images; (2) computing baseline results on the effects of using face data to train Convolutional Neural Networks (CNN) for predicting optical flow; and (3) using the learned optical flow in micro-expression recognition to demonstrate its effectiveness. We generate optical flow ground truth using facial key-points in the BP4D-Spontaneous dataset. The generated optical flow is used to train the FlowNetS architecture to test its performance on the generated dataset. The performance of FlowNetS trained on face images surpassed that of other optical flow CNN architectures, demonstrating its usefulness. Our optical flow features are further compared with other methods using the STSTNet micro-expression classifier, and the results indicate that the optical flow obtained using this work has promising applications in facial expression analysis.