Deep generative-contrastive networks for facial expression recognition

TL;DR

This paper introduces a deep generative-contrastive network that improves facial expression recognition by comparing query faces to generated reference images, effectively capturing expressive differences.

Contribution

It proposes a novel end-to-end deep network combining generative, contrastive, and discriminative models to disentangle facial expressions for better recognition accuracy.

Findings

Outperforms state-of-the-art methods on multiple facial expression datasets

Effectively disentangles expressive factors in facial images

Achieves higher recognition accuracy in diverse conditions

Abstract

As the expressive depth of an emotional face differs with individuals or expressions, recognizing an expression using a single facial image at a moment is difficult. A relative expression of a query face compared to a reference face might alleviate this difficulty. In this paper, we propose to utilize contrastive representation that embeds a distinctive expressive factor for a discriminative purpose. The contrastive representation is calculated at the embedding layer of deep networks by comparing a given (query) image with the reference image. We attempt to utilize a generative reference image that is estimated based on the given image. Consequently, we deploy deep neural networks that embed a combination of a generative model, a contrastive model, and a discriminative model with an end-to-end training manner. In our proposed networks, we attempt to disentangle a facial expressive factor in two steps including learning of a generator network and a contrastive encoder network. We conducted extensive experiments on publicly available face expression databases (CK+, MMI, Oulu-CASIA, and in-the-wild databases) that have been widely adopted in the recent literatures. The proposed method outperforms the known state-of-the art methods in terms of the recognition accuracy.