Threshold Neuron: A Brain-inspired Artificial Neuron for Efficient On-device Inference

TL;DR

This paper introduces Threshold Neurons, a brain-inspired artificial neuron design that significantly improves on-device neural network efficiency by reducing hardware complexity and power consumption with minimal accuracy loss.

Contribution

The study proposes a novel artificial neuron model inspired by biological mechanisms, enabling more efficient neural network implementations for mobile and edge devices.

Findings

Achieves 7.51x to 8.19x power savings at the kernel level

Reduces hardware area by 3.89x to 4.33x

Demonstrates 2.52x power savings and 1.75x speedup on FPGA

Abstract

Enhancing the computational efficiency of on-device Deep Neural Networks (DNNs) remains a significant challengein mobile and edge computing. As we aim to execute increasingly complex tasks with constrained computational resources, much of the research has focused on compressing neural network structures and optimizing systems. Although many studies have focused on compressing neural network structures and parameters or optimizing underlying systems, there has been limited attention on optimizing the fundamental building blocks of neural networks: the neurons. In this study, we deliberate on a simple but important research question: Can we design artificial neurons that offer greater efficiency than the traditional neuron paradigm? Inspired by the threshold mechanisms and the excitation-inhibition balance observed in biological neurons, we propose a novel artificial neuron model, Threshold Neurons. Using Threshold Neurons, we can construct neural networks similar to those with traditional artificial neurons, while significantly reducing hardware implementation complexity. Our extensive experiments validate the effectiveness of neural networks utilizing Threshold Neurons, achieving substantial power savings of 7.51x to 8.19x and area savings of 3.89x to 4.33x at the kernel level, with minimal loss in precision. Furthermore, FPGA-based implementations of these networks demonstrate 2.52x power savings and 1.75x speed enhancements at the system level. The source code will be made available upon publication.