Autonomous Driving with Spiking Neural Networks

TL;DR

This paper introduces Spiking Autonomous Driving (SAD), a unified energy-efficient neural network system that integrates perception, prediction, and planning for autonomous vehicles, demonstrating competitive performance with reduced energy consumption.

Contribution

The paper presents the first end-to-end trained Spiking Neural Network for autonomous driving, addressing energy efficiency and integrating perception, prediction, and planning modules.

Findings

SAD achieves competitive perception, prediction, and planning performance on nuScenes.

SAD demonstrates significant energy efficiency advantages over traditional neural networks.

The approach highlights neuromorphic computing's potential in sustainable autonomous driving.

Abstract

Autonomous driving demands an integrated approach that encompasses perception, prediction, and planning, all while operating under strict energy constraints to enhance scalability and environmental sustainability. We present Spiking Autonomous Driving (SAD), the first unified Spiking Neural Network (SNN) to address the energy challenges faced by autonomous driving systems through its event-driven and energy-efficient nature. SAD is trained end-to-end and consists of three main modules: perception, which processes inputs from multi-view cameras to construct a spatiotemporal bird's eye view; prediction, which utilizes a novel dual-pathway with spiking neurons to forecast future states; and planning, which generates safe trajectories considering predicted occupancy, traffic rules, and ride comfort. Evaluated on the nuScenes dataset, SAD achieves competitive performance in perception, prediction, and planning tasks, while drawing upon the energy efficiency of SNNs. This work highlights the potential of neuromorphic computing to be applied to energy-efficient autonomous driving, a critical step toward sustainable and safety-critical automotive technology. Our code is available at \url{https://github.com/ridgerchu/SAD}.