NeuroLiDAR: Adaptive Frame Rate Depth Sensing via Neuromorphic Event-LiDAR Fusion

TL;DR

NeuroLiDAR fuses event camera data with LiDAR to adaptively increase depth sensing frame rates, significantly improving accuracy and responsiveness in various scenarios.

Contribution

The paper introduces NeuroLiDAR, a novel framework that combines neuromorphic event data with LiDAR to enhance depth sensing adaptively.

Findings

Reduces depth RMSE by approximately 29%.

Achieves adaptive frame rates between 27.8 and 47.3 Hz.

Demonstrates effectiveness across indoor and outdoor scenarios.

Abstract

LiDARs are widely used for 3D depth reconstruction, but their performance is often limited by inherent hardware constraints that impose trade-offs between range, spatial resolution, and frame rate. Many LiDAR systems typically operate at low frame rates (e.g., 5-10 Hz), prioritizing long-range sensing over responsiveness to rapid scene changes. We present NeuroLiDAR, an adaptive depth sensing framework that achieves effective frame rates of up to $\approx$66 Hz by fusing temporally sparse LiDAR data with temporally dense inputs from neuromorphic event cameras. NeuroLiDAR integrates two components: event-based keyframe detection and event-guided depth extrapolation, to dynamically adjust the sensing rate in response to scene dynamics. To evaluate our approach, we introduce ELiDAR, a dataset spanning outdoor and indoor scenarios, and show that NeuroLiDAR reduces depth reconstruction error by $\approx$29\% in RMSE while achieving adaptive frame rates between 27.8-47.3 Hz. Our code and dataset are available at https://github.com/darshanakgr/neurolidar.