EvSegSNN: Neuromorphic Semantic Segmentation for Event Data

TL;DR

EvSegSNN is a neuromorphic semantic segmentation model using spiking neural networks and event cameras, achieving better accuracy with fewer parameters and lower resource consumption for autonomous navigation.

Contribution

This work introduces EvSegSNN, a novel biologically inspired encoder-decoder architecture for semantic segmentation using SNNs and event data, optimized for resource-constrained systems.

Findings

Outperforms state-of-the-art in MIoU on DDD17 dataset

Reduces parameters by a factor of 1.6

Eliminates batch normalization stage

Abstract

Semantic segmentation is an important computer vision task, particularly for scene understanding and navigation of autonomous vehicles and UAVs. Several variations of deep neural network architectures have been designed to tackle this task. However, due to their huge computational costs and their high memory consumption, these models are not meant to be deployed on resource-constrained systems. To address this limitation, we introduce an end-to-end biologically inspired semantic segmentation approach by combining Spiking Neural Networks (SNNs, a low-power alternative to classical neural networks) with event cameras whose output data can directly feed these neural network inputs. We have designed EvSegSNN, a biologically plausible encoder-decoder U-shaped architecture relying on Parametric Leaky Integrate and Fire neurons in an objective to trade-off resource usage against performance. The experiments conducted on DDD17 demonstrate that EvSegSNN outperforms the closest state-of-the-art model in terms of MIoU while reducing the number of parameters by a factor of $1.6$ and sparing a batch normalization stage.