Optical Neural Ordinary Differential Equations

TL;DR

This paper introduces Optical Neural ODEs (ON-ODE), a novel photonic neural network architecture that models continuous dynamics to reduce chip area while maintaining high accuracy in classification and trajectory prediction.

Contribution

The paper proposes ON-ODE, integrating optical ODE solvers with photonic neural networks to efficiently model continuous dynamics and reduce chip area.

Findings

Single hidden layer ON-ODE matches two-layer ResNet accuracy.

ON-ODE improves classification accuracy for diffraction-based hidden layers.

High-accuracy trajectory prediction using ON-ODE dynamics.

Abstract

Increasing the layer number of on-chip photonic neural networks (PNNs) is essential to improve its model performance. However, the successively cascading of network hidden layers results in larger integrated photonic chip areas. To address this issue, we propose the optical neural ordinary differential equations (ON-ODE) architecture that parameterizes the continuous dynamics of hidden layers with optical ODE solvers. The ON-ODE comprises the PNNs followed by the photonic integrator and optical feedback loop, which can be configured to represent residual neural networks (ResNet) and recurrent neural networks with effectively reduced chip area occupancy. For the interference-based optoelectronic nonlinear hidden layer, the numerical experiments demonstrate that the single hidden layer ON-ODE can achieve approximately the same accuracy as the two-layer optical ResNet in image classification tasks. Besides, the ONODE improves the model classification accuracy for the diffraction-based all-optical linear hidden layer. The time-dependent dynamics property of ON-ODE is further applied for trajectory prediction with high accuracy.