CS3D: An Efficient Facial Expression Recognition via Event Vision

TL;DR

CS3D introduces an efficient event vision-based facial expression recognition framework that reduces computational costs and enhances accuracy using decomposed convolutional methods, spiking neurons, and attention mechanisms, outperforming existing models.

Contribution

The paper presents CS3D, a novel framework that significantly decreases energy consumption and improves accuracy in event-based facial expression recognition compared to traditional deep learning architectures.

Findings

CS3D achieves higher accuracy than RNN, Transformer, and C3D models.

Energy consumption of CS3D is only 21.97% of the original C3D.

Experimental results validate the effectiveness of CS3D on multiple datasets.

Abstract

Responsive and accurate facial expression recognition is crucial to human-robot interaction for daily service robots. Nowadays, event cameras are becoming more widely adopted as they surpass RGB cameras in capturing facial expression changes due to their high temporal resolution, low latency, computational efficiency, and robustness in low-light conditions. Despite these advantages, event-based approaches still encounter practical challenges, particularly in adopting mainstream deep learning models. Traditional deep learning methods for facial expression analysis are energy-intensive, making them difficult to deploy on edge computing devices and thereby increasing costs, especially for high-frequency, dynamic, event vision-based approaches. To address this challenging issue, we proposed the CS3D framework by decomposing the Convolutional 3D method to reduce the computational complexity and energy consumption. Additionally, by utilizing soft spiking neurons and a spatial-temporal attention mechanism, the ability to retain information is enhanced, thus improving the accuracy of facial expression detection. Experimental results indicate that our proposed CS3D method attains higher accuracy on multiple datasets compared to architectures such as the RNN, Transformer, and C3D, while the energy consumption of the CS3D method is just 21.97\% of the original C3D required on the same device.