Biologically-Informed Excitatory and Inhibitory Balance for Robust Spiking Neural Network Training

TL;DR

This paper explores biologically-inspired principles for training robust spiking neural networks, emphasizing excitatory-inhibitory balance, low activity levels, and noise resilience, to advance energy-efficient AI hardware and large-scale models.

Contribution

It identifies key biological factors like firing rates and inhibitory patterns that enhance training robustness of excitatory-inhibitory balanced spiking networks.

Findings

Networks with 80:20 excitatory:inhibitory ratio train reliably at low activity levels.

Inhibitory neurons improve robustness to noise, as indicated by spike train synchrony measures.

Biologically-informed constraints support energy-efficient and large-scale neural network development.

Abstract

Spiking neural networks drawing inspiration from biological constraints of the brain promise an energy-efficient paradigm for artificial intelligence. However, challenges exist in identifying guiding principles to train these networks in a robust fashion. In addition, training becomes an even more difficult problem when incorporating biological constraints of excitatory and inhibitory connections. In this work, we identify several key factors, such as low initial firing rates and diverse inhibitory spiking patterns, that determine the overall ability to train spiking networks with various ratios of excitatory to inhibitory neurons on AI-relevant datasets. The results indicate networks with the biologically realistic 80:20 excitatory:inhibitory balance can reliably train at low activity levels and in noisy environments. Additionally, the Van Rossum distance, a measure of spike train synchrony, provides insight into the importance of inhibitory neurons to increase network robustness to noise. This work supports further biologically-informed large-scale networks and energy efficient hardware implementations.