GCF: Graph Convolutional Networks for Facial Expression Recognition

TL;DR

This paper introduces GCF, a novel method combining CNNs and Graph Convolutional Networks to improve facial expression recognition accuracy and robustness across benchmark datasets.

Contribution

GCF is the first approach to integrate CNN features with graph convolutions for enhanced FER performance and robustness.

Findings

GCF improves accuracy on CK+ from 92% to 98%.

GCF enhances JAFFE accuracy from 66% to 89%.

GCF achieves near-perfect accuracy on FERG dataset.

Abstract

Facial Expression Recognition (FER) is vital for understanding interpersonal communication. However, existing classification methods often face challenges such as vulnerability to noise, imbalanced datasets, overfitting, and generalization issues. In this paper, we propose GCF, a novel approach that utilizes Graph Convolutional Networks for FER. GCF integrates Convolutional Neural Networks (CNNs) for feature extraction, using either custom architectures or pretrained models. The extracted visual features are then represented on a graph, enhancing local CNN features with global features via a Graph Convolutional Neural Network layer. We evaluate GCF on benchmark datasets including CK+, JAFFE, and FERG. The results show that GCF significantly improves performance over state-of-the-art methods. For example, GCF enhances the accuracy of ResNet18 from 92% to 98% on CK+, from 66% to 89% on JAFFE, and from 94% to 100% on FERG. Similarly, GCF improves the accuracy of VGG16 from 89% to 97% on CK+, from 72% to 92% on JAFFE, and from 96% to 99.49% on FERG. We provide a comprehensive analysis of our approach, demonstrating its effectiveness in capturing nuanced facial expressions. By integrating graph convolutions with CNNs, GCF significantly advances FER, offering improved accuracy and robustness in real-world applications.