Incremental Boosting Convolutional Neural Network for Facial Action Unit Recognition

TL;DR

This paper introduces an Incremental Boosting CNN that enhances facial action unit recognition by selecting discriminative features and incrementally updating the model, leading to improved accuracy especially for less frequent AUs.

Contribution

The paper presents a novel IB-CNN architecture with an incremental boosting layer and a new loss function, improving generalization and performance over existing CNN methods for AU recognition.

Findings

Significant accuracy improvement over traditional CNNs.

Outperforms state-of-the-art CNN-based AU recognition methods.

Notable gains for low-frequency AUs.

Abstract

Recognizing facial action units (AUs) from spontaneous facial expressions is still a challenging problem. Most recently, CNNs have shown promise on facial AU recognition. However, the learned CNNs are often overfitted and do not generalize well to unseen subjects due to limited AU-coded training images. We proposed a novel Incremental Boosting CNN (IB-CNN) to integrate boosting into the CNN via an incremental boosting layer that selects discriminative neurons from the lower layer and is incrementally updated on successive mini-batches. In addition, a novel loss function that accounts for errors from both the incremental boosted classifier and individual weak classifiers was proposed to fine-tune the IB-CNN. Experimental results on four benchmark AU databases have demonstrated that the IB-CNN yields significant improvement over the traditional CNN and the boosting CNN without incremental learning, as well as outperforming the state-of-the-art CNN-based methods in AU recognition. The improvement is more impressive for the AUs that have the lowest frequencies in the databases.