In situ fine-tuning of in silico trained Optical Neural Networks

TL;DR

This paper introduces Gradient-Informed Fine-Tuning (GIFT), a lightweight in situ method to improve optical neural network performance by adjusting pretrained parameters based on noise structure, addressing inaccuracies from digital-to-physical mapping.

Contribution

The paper proposes GIFT, a novel in situ fine-tuning algorithm that uses gradient information to mitigate noise-induced performance loss in optical neural networks.

Findings

GIFT achieves up to 28% accuracy improvement under noise misspecification.

GIFT does not require costly retraining or complex experimental setups.

The method is validated via simulation on MNIST with a five-layer ONN.

Abstract

Optical Neural Networks (ONNs) promise significant advantages over traditional electronic neural networks, including ultrafast computation, high bandwidth, and low energy consumption, by leveraging the intrinsic capabilities of photonics. However, training ONNs poses unique challenges, notably the reliance on simplified in silico models whose trained parameters must subsequently be mapped to physical hardware. This process often introduces inaccuracies due to discrepancies between the idealized digital model and the physical ONN implementation, particularly stemming from noise and fabrication imperfections. In this paper, we analyze how noise misspecification during in silico training impacts ONN performance and we introduce Gradient-Informed Fine-Tuning (GIFT), a lightweight algorithm designed to mitigate this performance degradation. GIFT uses gradient information derived from the noise structure of the ONN to adapt pretrained parameters directly in situ, without requiring expensive retraining or complex experimental setups. GIFT comes with formal conditions under which it improves ONN performance. We also demonstrate the effectiveness of GIFT via simulation on a five-layer feed forward ONN trained on the MNIST digit classification task. GIFT achieves up to $28\%$ relative accuracy improvement compared to the baseline performance under noise misspecification, without resorting to costly retraining. Overall, GIFT provides a practical solution for bridging the gap between simplified digital models and real-world ONN implementations.