Delocalized Photonic Deep Learning on the Internet's Edge

TL;DR

This paper introduces Netcast, a novel photonic deep learning approach that decentralizes matrix algebra on edge devices, drastically reducing energy consumption and latency for neural network inference.

Contribution

The paper presents a new decentralized, photonic-based method for deep neural network computation on edge devices, significantly lowering energy use and enabling high-speed processing.

Findings

Achieved 98.8% accuracy in image recognition with 40 aJ/mult optical energy.

Demonstrated operation with <1 photon/mult using single photon detectors.

Deployed system over 86 km of optical fiber in a real-world network.

Abstract

Advances in deep neural networks (DNNs) are transforming science and technology. However, the increasing computational demands of the most powerful DNNs limit deployment on low-power devices, such as smartphones and sensors -- and this trend is accelerated by the simultaneous move towards Internet-of-Things (IoT) devices. Numerous efforts are underway to lower power consumption, but a fundamental bottleneck remains due to energy consumption in matrix algebra, even for analog approaches including neuromorphic, analog memory and photonic meshes. Here we introduce and demonstrate a new approach that sharply reduces energy required for matrix algebra by doing away with weight memory access on edge devices, enabling orders of magnitude energy and latency reduction. At the core of our approach is a new concept that decentralizes the DNN for delocalized, optically accelerated matrix algebra on edge devices. Using a silicon photonic smart transceiver, we demonstrate experimentally that this scheme, termed Netcast, dramatically reduces energy consumption. We demonstrate operation in a photon-starved environment with 40 aJ/multiply of optical energy for 98.8% accurate image recognition and <1 photon/multiply using single photon detectors. Furthermore, we show realistic deployment of our system, classifying images with 3 THz of bandwidth over 86 km of deployed optical fiber in a Boston-area fiber network. Our approach enables computing on a new generation of edge devices with speeds comparable to modern digital electronics and power consumption that is orders of magnitude lower.