Training of mixed-signal optical convolutional neural network with reduced quantization level

TL;DR

This paper introduces a training method for mixed-signal optical CNNs that maintains high accuracy despite analog signal noise and reduced quantization levels, enhancing robustness and efficiency.

Contribution

The authors propose a novel training approach for mixed-signal ANNs that effectively handles analog noise and errors, enabling high accuracy with reduced quantization levels.

Findings

Achieved classification accuracy with up to 50% noise level

Demonstrated effectiveness on a diffractive optics-based optical CNN

Validated robustness of the training method against analog errors

Abstract

Mixed-signal artificial neural networks (ANNs) that employ analog matrix-multiplication accelerators can achieve higher speed and improved power efficiency. Though analog computing is known to be susceptible to noise and device imperfections, various analog computing paradigms have been considered as promising solutions to address the growing computing demand in machine learning applications, thanks to the robustness of ANNs. This robustness has been explored in low-precision, fixed-point ANN models, which have proven successful on compressing ANN model size on digital computers. However, these promising results and network training algorithms cannot be easily migrated to analog accelerators. The reason is that digital computers typically carry intermediate results with higher bit width, though the inputs and weights of each ANN layers are of low bit width; while the analog intermediate results have low precision, analogous to digital signals with a reduced quantization level. Here we report a training method for mixed-signal ANN with two types of errors in its analog signals, random noise, and deterministic errors (distortions). The results showed that mixed-signal ANNs trained with our proposed method can achieve an equivalent classification accuracy with noise level up to 50% of the ideal quantization step size. We have demonstrated this training method on a mixed-signal optical convolutional neural network based on diffractive optics.