Triplet Loss-less Center Loss Sampling Strategies in Facial Expression Recognition Scenarios

TL;DR

This paper introduces three novel triplet center loss sampling strategies combined with an attention module to enhance facial expression recognition accuracy on imbalanced datasets.

Contribution

It proposes three negative sample selection strategies and an attention mechanism to improve deep metric learning in facial expression recognition.

Findings

Significant accuracy improvements over baseline methods.

Effective handling of imbalanced datasets like RAF-DB.

Enhanced discriminative feature learning through proposed strategies.

Abstract

Facial expressions convey massive information and play a crucial role in emotional expression. Deep neural network (DNN) accompanied by deep metric learning (DML) techniques boost the discriminative ability of the model in facial expression recognition (FER) applications. DNN, equipped with only classification loss functions such as Cross-Entropy cannot compact intra-class feature variation or separate inter-class feature distance as well as when it gets fortified by a DML supporting loss item. The triplet center loss (TCL) function is applied on all dimensions of the sample's embedding in the embedding space. In our work, we developed three strategies: fully-synthesized, semi-synthesized, and prediction-based negative sample selection strategies. To achieve better results, we introduce a selective attention module that provides a combination of pixel-wise and element-wise attention coefficients using high-semantic deep features of input samples. We evaluated the proposed method on the RAF-DB, a highly imbalanced dataset. The experimental results reveal significant improvements in comparison to the baseline for all three negative sample selection strategies.