FaceNet2ExpNet: Regularizing a Deep Face Recognition Net for Expression Recognition

TL;DR

This paper introduces FaceNet2ExpNet, a novel two-stage training method that regularizes deep face recognition networks for improved facial expression recognition, especially with limited data, by modeling high-level neuron distributions.

Contribution

The paper proposes a new distribution function and a two-stage training algorithm to enhance expression recognition performance using deep networks with limited data.

Findings

Outperforms state-of-the-art on four public databases

Captures improved high-level expression semantics

Effective regularization of deep face recognition models

Abstract

Relatively small data sets available for expression recognition research make the training of deep networks for expression recognition very challenging. Although fine-tuning can partially alleviate the issue, the performance is still below acceptable levels as the deep features probably contain redun- dant information from the pre-trained domain. In this paper, we present FaceNet2ExpNet, a novel idea to train an expression recognition network based on static images. We first propose a new distribution function to model the high-level neurons of the expression network. Based on this, a two-stage training algorithm is carefully designed. In the pre-training stage, we train the convolutional layers of the expression net, regularized by the face net; In the refining stage, we append fully- connected layers to the pre-trained convolutional layers and train the whole network jointly. Visualization shows that the model trained with our method captures improved high-level expression semantics. Evaluations on four public expression databases, CK+, Oulu-CASIA, TFD, and SFEW demonstrate that our method achieves better results than state-of-the-art.