Transfer Learning for Action Unit Recognition

TL;DR

This paper explores transfer learning with CNNs for facial action unit detection, comparing classifiers and ensembles to identify the most effective models for facial expression recognition.

Contribution

It introduces a classifier ensemble approach using transfer learning models like VGG-Face and ResNet for improved action unit recognition.

Findings

VGG-Face and ResNet are the most effective pre-trained models.

Ensemble of VGG-Net variants and ResNet yields the best performance.

Transfer learning enhances facial action unit detection accuracy.

Abstract

This paper presents a classifier ensemble for Facial Expression Recognition (FER) based on models derived from transfer learning. The main experimentation work is conducted for facial action unit detection using feature extraction and fine-tuning convolutional neural networks (CNNs). Several classifiers for extracted CNN codes such as Linear Discriminant Analysis (LDA), Support Vector Machines (SVMs) and Long Short-Term Memory (LSTM) are compared and evaluated. Multi-model ensembles are also used to further improve the performance. We have found that VGG-Face and ResNet are the relatively optimal pre-trained models for action unit recognition using feature extraction and the ensemble of VGG-Net variants and ResNet achieves the best result.