A Perception CNN for Facial Expression Recognition

TL;DR

This paper introduces a perception CNN that uses multiple local feature extractors and a multi-domain fusion mechanism to improve facial expression recognition accuracy, especially under challenging conditions.

Contribution

The proposed PCNN employs parallel local feature networks and a multi-domain interaction mechanism, along with a two-phase loss, to enhance FER performance over existing methods.

Findings

Achieves superior results on multiple FER benchmarks.

Effectively captures subtle facial expression changes.

Performs well on occlusion and pose variation datasets.

Abstract

Convolutional neural networks (CNNs) can automatically learn data patterns to express face images for facial expression recognition (FER). However, they may ignore effect of facial segmentation of FER. In this paper, we propose a perception CNN for FER as well as PCNN. Firstly, PCNN can use five parallel networks to simultaneously learn local facial features based on eyes, cheeks and mouth to realize the sensitive capture of the subtle changes in FER. Secondly, we utilize a multi-domain interaction mechanism to register and fuse between local sense organ features and global facial structural features to better express face images for FER. Finally, we design a two-phase loss function to restrict accuracy of obtained sense information and reconstructed face images to guarantee performance of obtained PCNN in FER. Experimental results show that our PCNN achieves superior results on several lab and real-world FER benchmarks: CK+, JAFFE, FER2013, FERPlus, RAF-DB and Occlusion and Pose Variant Dataset. Its code is available at https://github.com/hellloxiaotian/PCNN.