Noise-Resilient Designs for Optical Neural Networks

TL;DR

This paper introduces two noise-resilient designs for Optical Neural Networks that mathematically guarantee outputs close to the original neural networks and demonstrate practical noise mitigation through numerical experiments.

Contribution

The paper proposes two novel designs for ONNs that enhance noise robustness and provides mathematical analysis and empirical validation of their effectiveness.

Findings

Designs enable ONNs to approximate original NNs closely.

Adding a few components significantly improves ONN accuracy.

Noise impact decreases with increased network depth.

Abstract

All analog signal processing is fundamentally subject to noise, and this is also the case in modern implementations of Optical Neural Networks (ONNs). Therefore, to mitigate noise in ONNs, we propose two designs that are constructed from a given, possibly trained, Neural Network (NN) that one wishes to implement. Both designs have the capability that the resulting ONNs gives outputs close to the desired NN. To establish the latter, we analyze the designs mathematically. Specifically, we investigate a probabilistic framework for the first design that establishes that the design is correct, i.e., for any feed-forward NN with Lipschitz continuous activation functions, an ONN can be constructed that produces output arbitrarily close to the original. ONNs constructed with the first design thus also inherit the universal approximation property of NNs. For the second design, we restrict the analysis to NNs with linear activation functions and characterize the ONNs' output distribution using exact formulas. Finally, we report on numerical experiments with LeNet ONNs that give insight into the number of components required in these designs for certain accuracy gains. We specifically study the effect of noise as a function of the depth of an ONN. The results indicate that in practice, adding just a few components in the manner of the first or the second design can already be expected to increase the accuracy of ONNs considerably.