Physics-aware Differentiable Discrete Codesign for Diffractive Optical Neural Networks

TL;DR

This paper introduces a physics-aware, differentiable codesign framework for diffractive optical neural networks that effectively maps physical device parameters to neural network models, enabling efficient training and deployment on real-world optical hardware.

Contribution

It presents a novel hardware-software codesign method using Gumbel-Softmax for differentiable discrete mapping, improving training of DONNs with low-precision optical devices.

Findings

Framework outperforms traditional quantization methods.

Effective training with non-uniform, non-monotonic optical device levels.

Validated on physical optical systems with low-precision devices.

Abstract

Diffractive optical neural networks (DONNs) have attracted lots of attention as they bring significant advantages in terms of power efficiency, parallelism, and computational speed compared with conventional deep neural networks (DNNs), which have intrinsic limitations when implemented on digital platforms. However, inversely mapping algorithm-trained physical model parameters onto real-world optical devices with discrete values is a non-trivial task as existing optical devices have non-unified discrete levels and non-monotonic properties. This work proposes a novel device-to-system hardware-software codesign framework, which enables efficient physics-aware training of DONNs w.r.t arbitrary experimental measured optical devices across layers. Specifically, Gumbel-Softmax is employed to enable differentiable discrete mapping from real-world device parameters into the forward function of DONNs, where the physical parameters in DONNs can be trained by simply minimizing the loss function of the ML task. The results have demonstrated that our proposed framework offers significant advantages over conventional quantization-based methods, especially with low-precision optical devices. Finally, the proposed algorithm is fully verified with physical experimental optical systems in low-precision settings.