Covariant spatio-temporal receptive fields for spiking neural networks

TL;DR

This paper introduces a covariant spatio-temporal receptive field model for neuromorphic systems, inspired by biological vision, improving training and processing of spiking neural networks for event-based vision tasks.

Contribution

It presents a theoretically grounded model based on affine Gaussian kernels that is covariant to spatial and temporal transformations, enhancing neuromorphic computation and event-based vision.

Findings

Improved training of spiking neural networks for event-based vision

Theoretical covariant model inspired by biological visual processing

Applicable to various spatio-temporal processing tasks

Abstract

Biological nervous systems constitute important sources of inspiration towards computers that are faster, cheaper, and more energy efficient. Neuromorphic disciplines view the brain as a coevolved system, simultaneously optimizing the hardware and the algorithms running on it. There are clear efficiency gains when bringing the computations into a physical substrate, but we presently lack theories to guide efficient implementations. Here, we present a principled computational model for neuromorphic systems in terms of spatio-temporal receptive fields, based on affine Gaussian kernels over space and leaky-integrator and leaky integrate-and-fire models over time. Our theory is provably covariant to spatial affine and temporal scaling transformations, and with close similarities to the visual processing in mammalian brains. We use these spatio-temporal receptive fields as a prior in an event-based vision task, and show that this improves the training of spiking networks, which otherwise is known as problematic for event-based vision. This work combines efforts within scale-space theory and computational neuroscience to identify theoretically well-founded ways to process spatio-temporal signals in neuromorphic systems. Our contributions are immediately relevant for signal processing and event-based vision, and can be extended to other processing tasks over space and time, such as memory and control.