Which CNNs and Training Settings to Choose for Action Unit Detection? A Study Based on a Large-Scale Dataset

TL;DR

This study systematically evaluates various CNN architectures and training configurations on a large-scale naturalistic dataset to determine optimal choices for Action Unit detection.

Contribution

It provides comprehensive insights into how different CNNs and training strategies affect AU detection performance on a large, real-world dataset.

Findings

Certain CNN architectures outperform others in AU detection.

Data augmentation and normalization significantly improve accuracy.

Increasing labeled data enhances detection performance.

Abstract

In this paper we explore the influence of some frequently used Convolutional Neural Networks (CNNs), training settings, and training set structures, on Action Unit (AU) detection. Specifically, we first compare 10 different shallow and deep CNNs in AU detection. Second, we investigate how the different training settings (i.e. centering/normalizing the inputs, using different augmentation severities, and balancing the data) impact the performance in AU detection. Third, we explore the effect of increasing the number of labelled subjects and frames in the training set on the AU detection performance. These comparisons provide the research community with useful tips about the choice of different CNNs and training settings in AU detection. In our analysis, we use a large-scale naturalistic dataset, consisting of ~55K videos captured in the wild. To the best of our knowledge, there is no work that had investigated the impact of such settings on a large-scale AU dataset.