Optical Spiking Neurons Enable High-Speed and Energy-Efficient Optical Neural Networks

TL;DR

This paper introduces a high-speed, energy-efficient optical neural network inspired by biological spiking neurons, utilizing novel thresholding and training methods to achieve ultra-fast processing and sparse information transmission.

Contribution

It presents a new spike-based diffractive optical neural network with threshold-locking sensors and innovative training and inference techniques, significantly improving speed and efficiency over prior DONNs.

Findings

Achieves 3649 FPS operating speed, 30 times faster than previous DONNs.

Delivers 417.96 TOPS computational speed and 12.6 TOPS/W energy efficiency.

Demonstrates effectiveness in low-level and high-level machine vision tasks.

Abstract

Optical neural networks (ONNs) perform extensive computations using photons instead of electrons, resulting in passively energy-efficient and low-latency computing. Among various ONNs, the diffractive optical neural networks (DONNs) particularly excel in energy efficiency, bandwidth, and parallelism, therefore attract considerable attention. However, their performance is limited by the inherent constraints of traditional frame-based sensors, which process and produce dense and redundant information at low operating frequency. Inspired by the spiking neurons in human neural system, which utilize a thresholding mechanism to transmit information sparsely and efficiently, we propose integrating a threshold-locking method into neuromorphic vision sensors to generate sparse and binary information, achieving microsecond-level accurate perception similar to human spiking neurons. By introducing novel Binary Dual Adaptive Training (BAT) and Optically Parallel Mixture of Experts (OPMoE) inference methods, the high-speed, spike-based diffractive optical neural network (S2NN) demonstrates an ultra-fast operating speed of 3649 FPS, which is 30 fold faster than that of reported DONNs, delivering a remarkable computational speed of 417.96 TOPS and a system energy efficiency of 12.6 TOPS/W. Our work demonstrates the potential of incorporating neuromorphic architecture to facilitate optical neural network applications in real-world scenarios for both low-level and high-level machine vision tasks.