Event-based Face Detection and Tracking in the Blink of an Eye

TL;DR

This paper introduces a novel event-based face detection method leveraging eye blink dynamics captured by event cameras, enabling high-speed, low-cost face detection and tracking.

Contribution

The paper presents the first event-based face detection method using eye blink signatures, combining a new feature with a probabilistic tracking framework.

Findings

Effective face detection in indoor and outdoor settings.

Robust tracking of face position with high temporal resolution.

Demonstrated low computational cost compared to traditional methods.

Abstract

We present the first purely event-based method for face detection using the high temporal resolution of an event-based camera. We will rely on a new feature that has never been used for such a task that relies on detecting eye blinks. Eye blinks are a unique natural dynamic signature of human faces that is captured well by event-based sensors that rely on relative changes of luminance. Although an eye blink can be captured with conventional cameras, we will show that the dynamics of eye blinks combined with the fact that two eyes act simultaneously allows to derive a robust methodology for face detection at a low computational cost and high temporal resolution. We show that eye blinks have a unique temporal signature over time that can be easily detected by correlating the acquired local activity with a generic temporal model of eye blinks that has been generated from a wide population of users. We furthermore show that once the face is reliably detected it is possible to apply a probabilistic framework to track the spatial position of a face for each incoming event while updating the position of trackers. Results are shown for several indoor and outdoor experiments. We will also release an annotated data set that can be used for future work on the topic.