Neuroscience inspired scientific machine learning (Part-1): Variable spiking neuron for regression

TL;DR

This paper introduces a novel Variable Spiking Neuron (VSN) inspired by biological neurons, designed to reduce redundant firing and improve energy efficiency in neural networks, especially for regression tasks.

Contribution

The paper proposes the VSN, a hybrid neuron combining features of LIF spiking neurons and artificial neurons, tailored for efficient regression tasks.

Findings

VSN reduces redundant firing and energy consumption.

VSN performs effectively on regression tasks.

Experimental results favor VSN's efficacy.

Abstract

Redundant information transfer in a neural network can increase the complexity of the deep learning model, thus increasing its power consumption. We introduce in this paper a novel spiking neuron, termed Variable Spiking Neuron (VSN), which can reduce the redundant firing using lessons from biological neuron inspired Leaky Integrate and Fire Spiking Neurons (LIF-SN). The proposed VSN blends LIF-SN and artificial neurons. It garners the advantage of intermittent firing from the LIF-SN and utilizes the advantage of continuous activation from the artificial neuron. This property of the proposed VSN makes it suitable for regression tasks, which is a weak point for the vanilla spiking neurons, all while keeping the energy budget low. The proposed VSN is tested against both classification and regression tasks. The results produced advocate favorably towards the efficacy of the proposed spiking neuron, particularly for regression tasks.