A compact neuromorphic system for ultra-energy-efficient, on-device robot localization

TL;DR

This paper presents LENS, a neuromorphic system combining hardware and algorithms for ultra-energy-efficient, real-time robot localization capable of recognizing places over 8 km with minimal energy use.

Contribution

The paper introduces a compact neuromorphic localization system that integrates spiking neural networks, event-based vision sensors, and neuromorphic processors for on-device robot localization.

Findings

Performs place recognition over 8 km of traversal

Uses models as small as 180 KB with 44,000 parameters

Consumes less than 8% of the energy of conventional methods

Abstract

Neuromorphic computing offers a transformative pathway to overcome the computational and energy challenges faced in deploying robotic localization and navigation systems at the edge. Visual place recognition, a critical component for navigation, is often hampered by the high resource demands of conventional systems, making them unsuitable for small-scale robotic platforms which still require accurate long-endurance localization. Although neuromorphic approaches offer potential for greater efficiency, real-time edge deployment remains constrained by the complexity of bio-realistic networks. In order to overcome this challenge, fusion of hardware and algorithms is critical to employ this specialized computing paradigm. Here, we demonstrate a neuromorphic localization system that performs competitive place recognition in up to 8 kilometers of traversal using models as small as 180 kilobytes with 44,000 parameters, while consuming less than 8% of the energy required by conventional methods. Our Locational Encoding with Neuromorphic Systems (LENS) integrates spiking neural networks, an event-based dynamic vision sensor, and a neuromorphic processor within a single SynSense Speck chip, enabling real-time, energy-efficient localization on a hexapod robot. When compared to a benchmark place recognition method, Sum-of-Absolute-Differences (SAD), LENS performs comparably in overall precision. LENS represents an accurate fully neuromorphic localization system capable of large-scale, on-device deployment for energy efficient robotic place recognition. Neuromorphic computing enables resource-constrained robots to perform energy efficient, accurate localization.