Neuromorphic Event-based Facial Expression Recognition

TL;DR

This paper introduces NEFER, a new dataset of paired RGB and event videos for facial expression recognition, demonstrating that event-based data can effectively capture subtle micro-expressions and improve emotion recognition accuracy.

Contribution

The paper presents NEFER, a novel neuromorphic dataset for facial expression recognition, and provides baseline methods showing the advantages of event-based data over traditional RGB data.

Findings

Event-based data captures micro-expressions better than RGB.

Double recognition accuracy achieved with event-based approach.

Neuromorphic methods enhance emotion detection in fast, subtle expressions.

Abstract

Recently, event cameras have shown large applicability in several computer vision fields especially concerning tasks that require high temporal resolution. In this work, we investigate the usage of such kind of data for emotion recognition by presenting NEFER, a dataset for Neuromorphic Event-based Facial Expression Recognition. NEFER is composed of paired RGB and event videos representing human faces labeled with the respective emotions and also annotated with face bounding boxes and facial landmarks. We detail the data acquisition process as well as providing a baseline method for RGB and event data. The collected data captures subtle micro-expressions, which are hard to spot with RGB data, yet emerge in the event domain. We report a double recognition accuracy for the event-based approach, proving the effectiveness of a neuromorphic approach for analyzing fast and hardly detectable expressions and the emotions they conceal.