Event-based Heterogeneous Information Processing for Online Vision-based Obstacle Detection and Localization

TL;DR

This paper presents a hybrid neural network framework combining ANNs and SNNs for efficient, real-time obstacle detection and localization in robotic navigation, improving accuracy and energy efficiency in dynamic environments.

Contribution

The paper introduces a novel dual-pathway architecture integrating ANNs and SNNs directly for obstacle detection and localization without domain conversion, enhancing neuromorphic robotic navigation.

Findings

Achieves accurate obstacle detection with low computational cost.

Maintains energy efficiency close to SNN-only systems.

Supports real-time dynamic environment navigation.

Abstract

This paper introduces a novel framework for robotic vision-based navigation that integrates Hybrid Neural Networks (HNNs) with Spiking Neural Network (SNN)-based filtering to enhance situational awareness for unmodeled obstacle detection and localization. By leveraging the complementary strengths of Artificial Neural Networks (ANNs) and SNNs, the system achieves both accurate environmental understanding and fast, energy-efficient processing. The proposed architecture employs a dual-pathway approach: an ANN component processes static spatial features at low frequency, while an SNN component handles dynamic, event-based sensor data in real time. Unlike conventional hybrid architectures that rely on domain conversion mechanisms, our system incorporates a pre-developed SNN-based filter that directly utilizes spike-encoded inputs for localization and state estimation. Detected anomalies are validated using contextual information from the ANN pathway and continuously tracked to support anticipatory navigation strategies. Simulation results demonstrate that the proposed method offers acceptable detection accuracy while maintaining computational efficiency close to SNN-only implementations, which operate at a fraction of the resource cost. This framework represents a significant advancement in neuromorphic navigation systems for robots operating in unpredictable and dynamic environments.