Deep Covariance Descriptors for Facial Expression Recognition

TL;DR

This paper introduces a novel method for facial expression recognition that uses deep covariance descriptors on SPD manifolds, outperforming traditional fully connected layer classifiers.

Contribution

It proposes a new approach combining covariance matrices of DCNN features with Gaussian kernels on SPD manifolds for improved facial expression recognition.

Findings

Achieves state-of-the-art performance on Oulu-CASIA and SFEW datasets.

Demonstrates the effectiveness of covariance descriptors over standard fully connected layers.

Validates the approach with extensive experiments using VGG-face and ExpNet architectures.

Abstract

In this paper, covariance matrices are exploited to encode the deep convolutional neural networks (DCNN) features for facial expression recognition. The space geometry of the covariance matrices is that of Symmetric Positive Definite (SPD) matrices. By performing the classification of the facial expressions using Gaussian kernel on SPD manifold, we show that the covariance descriptors computed on DCNN features are more efficient than the standard classification with fully connected layers and softmax. By implementing our approach using the VGG-face and ExpNet architectures with extensive experiments on the Oulu-CASIA and SFEW datasets, we show that the proposed approach achieves performance at the state of the art for facial expression recognition.