Deep Temporal Appearance-Geometry Network for Facial Expression Recognition

TL;DR

This paper introduces a deep learning framework that combines temporal geometry and appearance features from facial data to improve facial expression recognition accuracy, automatically capturing facial action points.

Contribution

The novel deep network automatically extracts and combines temporal appearance and geometry features, enhancing facial expression recognition performance over existing methods.

Findings

Achieved superior accuracy on CK+ and Oulu-CASIA datasets.

Demonstrated effective cooperation between geometry and appearance models.

Automatically detects facial action points without manual intervention.

Abstract

Temporal information can provide useful features for recognizing facial expressions. However, to manually design useful features requires a lot of effort. In this paper, to reduce this effort, a deep learning technique which is regarded as a tool to automatically extract useful features from raw data, is adopted. Our deep network is based on two different models. The first deep network extracts temporal geometry features from temporal facial landmark points, while the other deep network extracts temporal appearance features from image sequences . These two models are combined in order to boost the performance of the facial expression recognition. Through several experiments, we showed that the two models cooperate with each other. As a result, we achieved superior performance to other state-of-the-art methods in CK+ and Oulu-CASIA databases. Furthermore, one of the main contributions of this paper is that our deep network catches the facial action points automatically.