ClearSight: Human Vision-Inspired Solutions for Event-Based Motion Deblurring

TL;DR

ClearSight introduces a bioinspired dual-drive hybrid network leveraging event camera data and neural mechanisms to improve motion deblurring, outperforming existing methods on synthetic and real datasets.

Contribution

The paper proposes a novel bioinspired dual-drive hybrid network with dynamic neuron configuration and an unsupervised blurry mask for enhanced event-based motion deblurring.

Findings

Outperforms state-of-the-art methods on synthetic datasets

Effective in real-world scenarios

Demonstrates significant improvement in deblurring quality

Abstract

Motion deblurring addresses the challenge of image blur caused by camera or scene movement. Event cameras provide motion information that is encoded in the asynchronous event streams. To efficiently leverage the temporal information of event streams, we employ Spiking Neural Networks (SNNs) for motion feature extraction and Artificial Neural Networks (ANNs) for color information processing. Due to the non-uniform distribution and inherent redundancy of event data, existing cross-modal feature fusion methods exhibit certain limitations. Inspired by the visual attention mechanism in the human visual system, this study introduces a bioinspired dual-drive hybrid network (BDHNet). Specifically, the Neuron Configurator Module (NCM) is designed to dynamically adjusts neuron configurations based on cross-modal features, thereby focusing the spikes in blurry regions and adapting to varying blurry scenarios dynamically. Additionally, the Region of Blurry Attention Module (RBAM) is introduced to generate a blurry mask in an unsupervised manner, effectively extracting motion clues from the event features and guiding more accurate cross-modal feature fusion. Extensive subjective and objective evaluations demonstrate that our method outperforms current state-of-the-art methods on both synthetic and real-world datasets.