Fully Asynchronous Neuromorphic Perception for Mobile Robot Dodging with Loihi Chips

TL;DR

This paper introduces a fully asynchronous neuromorphic perception system using event cameras, spiking networks, and Loihi chips, achieving low latency, robustness, and significant energy savings for mobile robot dodging tasks.

Contribution

It presents the first implementation of a fully asynchronous neuromorphic perception paradigm for real mobile robots, outperforming traditional methods in robustness and energy efficiency.

Findings

Better robustness than frame-based methods under various conditions.

Energy consumption is only 4.30% of NVIDIA Jetson Orin NX event spike tensor method.

Energy consumption is only 1.64% of event frame method on Loihi.

Abstract

Sparse and asynchronous sensing and processing in natural organisms lead to ultra low-latency and energy-efficient perception. Event cameras, known as neuromorphic vision sensors, are designed to mimic these characteristics. However, fully utilizing the sparse and asynchronous event stream remains challenging. Influenced by the mature algorithms of standard cameras, most existing event-based algorithms still rely on the "group of events" processing paradigm (e.g., event frames, 3D voxels) when handling event streams. This paradigm encounters issues such as feature loss, event stacking, and high computational burden, which deviates from the intended purpose of event cameras. To address these issues, we propose a fully asynchronous neuromorphic paradigm that integrates event cameras, spiking networks, and neuromorphic processors (Intel Loihi). This paradigm can faithfully process each event asynchronously as it arrives, mimicking the spike-driven signal processing in biological brains. We compare the proposed paradigm with the existing "group of events" processing paradigm in detail on the real mobile robot dodging task. Experimental results show that our scheme exhibits better robustness than frame-based methods with different time windows and light conditions. Additionally, the energy consumption per inference of our scheme on the embedded Loihi processor is only 4.30% of that of the event spike tensor method on NVIDIA Jetson Orin NX with energy-saving mode, and 1.64% of that of the event frame method on the same neuromorphic processor. As far as we know, this is the first time that a fully asynchronous neuromorphic paradigm has been implemented for solving sequential tasks on real mobile robot.