Locally Adaptive Decay Surfaces for High-Speed Face and Landmark Detection with Event Cameras

TL;DR

This paper introduces Locally Adaptive Decay Surfaces (LADS), a novel event representation method that dynamically adjusts decay based on local signal activity, significantly improving high-speed face detection and landmark accuracy.

Contribution

LADS adaptively modulates temporal decay at each location using three strategies, enhancing detail preservation and reducing blur in event camera data for better neural network performance.

Findings

LADS outperforms standard representations in face detection and landmark accuracy.

High-frequency LADS maintains accuracy at 240 Hz, surpassing prior 30 Hz benchmarks.

Supports lighter neural networks with real-time performance.

Abstract

Event cameras record luminance changes with microsecond resolution, but converting their sparse, asynchronous output into dense tensors that neural networks can exploit remains a core challenge. Conventional histograms or globally-decayed time-surface representations apply fixed temporal parameters across the entire image plane, which in practice creates a trade-off between preserving spatial structure during still periods and retaining sharp edges during rapid motion. We introduce Locally Adaptive Decay Surfaces (LADS), a family of event representations in which the temporal decay at each location is modulated according to local signal dynamics. Three strategies are explored, based on event rate, Laplacian-of-Gaussian response, and high-frequency spectral energy. These adaptive schemes preserve detail in quiescent regions while reducing blur in regions of dense activity. Extensive experiments on the public data show that LADS consistently improves both face detection and facial landmark accuracy compared to standard non-adaptive representations. At 30 Hz, LADS achieves higher detection accuracy and lower landmark error than either baseline, and at 240 Hz it mitigates the accuracy decline typically observed at higher frequencies, sustaining 2.44 % normalized mean error for landmarks and 0.966 mAP50 in face detection. These high-frequency results even surpass the accuracy reported in prior works operating at 30 Hz, setting new benchmarks for event-based face analysis. Moreover, by preserving spatial structure at the representation stage, LADS supports the use of much lighter network architectures while still retaining real-time performance. These results highlight the importance of context-aware temporal integration for neuromorphic vision and point toward real-time, high-frequency human-computer interaction systems that exploit the unique advantages of event cameras.