RED-Net: A Recurrent Encoder-Decoder Network for Video-based Face Alignment

TL;DR

This paper introduces RED-Net, a real-time video face alignment method using a recurrent encoder-decoder that iteratively refines facial points and disentangles features for improved accuracy and generalization.

Contribution

The paper presents a novel recurrent encoder-decoder architecture with spatial feedback and feature disentangling for enhanced video face alignment.

Findings

Achieves superior accuracy over state-of-the-art methods.

Demonstrates effective iterative coarse-to-fine alignment.

Shows improved generalization through feature disentangling.

Abstract

We propose a novel method for real-time face alignment in videos based on a recurrent encoder-decoder network model. Our proposed model predicts 2D facial point heat maps regularized by both detection and regression loss, while uniquely exploiting recurrent learning at both spatial and temporal dimensions. At the spatial level, we add a feedback loop connection between the combined output response map and the input, in order to enable iterative coarse-to-fine face alignment using a single network model, instead of relying on traditional cascaded model ensembles. At the temporal level, we first decouple the features in the bottleneck of the network into temporal-variant factors, such as pose and expression, and temporal-invariant factors, such as identity information. Temporal recurrent learning is then applied to the decoupled temporal-variant features. We show that such feature disentangling yields better generalization and significantly more accurate results at test time. We perform a comprehensive experimental analysis, showing the importance of each component of our proposed model, as well as superior results over the state of the art and several variations of our method in standard datasets.