Acceleration Method for Learning Fine-Layered Optical Neural Networks

TL;DR

This paper introduces a novel acceleration method for training optical neural networks with multilayered Mach-Zehnder interferometers, significantly reducing learning time while maintaining compatibility with existing automatic differentiation frameworks.

Contribution

The authors develop a customized complex-valued derivative approach and a C++ module that speeds up MZI parameter learning in optical neural networks by 20 times compared to conventional methods.

Findings

Achieved 20x faster training for MNIST in a complex-valued RNN.

Demonstrated compatibility with standard automatic differentiation.

Validated effectiveness in a multilayered optical neural network.

Abstract

An optical neural network (ONN) is a promising system due to its high-speed and low-power operation. Its linear unit performs a multiplication of an input vector and a weight matrix in optical analog circuits. Among them, a circuit with a multiple-layered structure of programmable Mach-Zehnder interferometers (MZIs) can realize a specific class of unitary matrices with a limited number of MZIs as its weight matrix. The circuit is effective for balancing the number of programmable MZIs and ONN performance. However, it takes a lot of time to learn MZI parameters of the circuit with a conventional automatic differentiation (AD), which machine learning platforms are equipped with. To solve the time-consuming problem, we propose an acceleration method for learning MZI parameters. We create customized complex-valued derivatives for an MZI, exploiting Wirtinger derivatives and a chain rule. They are incorporated into our newly developed function module implemented in C++ to collectively calculate their values in a multi-layered structure. Our method is simple, fast, and versatile as well as compatible with the conventional AD. We demonstrate that our method works 20 times faster than the conventional AD when a pixel-by-pixel MNIST task is performed in a complex-valued recurrent neural network with an MZI-based hidden unit.