Neuromorphic Imaging and Classification with Graph Learning

TL;DR

This paper introduces a graph-based approach using Graph Transformers for neuromorphic image classification, effectively handling sparse event streams with limited data and computational resources, advancing mobile neuromorphic applications.

Contribution

The paper proposes a novel graph representation and a Graph Transformer model specifically designed for neuromorphic event data, improving classification accuracy and efficiency.

Findings

Outperforms existing methods in accuracy and efficiency.

Excels in scenarios with limited events and computational resources.

Enables neuromorphic applications in mobile devices.

Abstract

Bio-inspired neuromorphic cameras asynchronously record pixel brightness changes and generate sparse event streams. They can capture dynamic scenes with little motion blur and more details in extreme illumination conditions. Due to the multidimensional address-event structure, most existing vision algorithms cannot properly handle asynchronous event streams. While several event representations and processing methods have been developed to address such an issue, they are typically driven by a large number of events, leading to substantial overheads in runtime and memory. In this paper, we propose a new graph representation of the event data and couple it with a Graph Transformer to perform accurate neuromorphic classification. Extensive experiments show that our approach leads to better results and excels at the challenging realistic situations where only a small number of events and limited computational resources are available, paving the way for neuromorphic applications embedded into mobile facilities.