Generalized Training for Neural Network Learnability: a Spectral Methods Approach

TL;DR

This paper introduces a spectral-methods approach for training hybrid optical neural networks using synthetic data, which enhances learnability by focusing on specific features and accelerates model convergence.

Contribution

It presents a novel spectral-methods paradigm for generating synthetic training data that improves neural network learnability and convergence speed in hybrid optical neural networks.

Findings

Synthetic data's SVD entropy indicates learnability.

Neural networks rapidly learn specific features.

Improved convergence speed in hybrid optical neural networks.

Abstract

Hybrid optical neural networks (HONNs) offload some electronic computation to optical preprocessors to achieve low-power and fast training and inference phases in machine learning tasks. Our contribution to the development of HONNs is a spectral-methods paradigm for building synthetic training data for machine-learned models. Here, our synthetic training image data does not resemble the image test data. As a result, the neural network focuses on learning specific features parameterized by the synthetic training data. Within this paradigm, a dataset's singular value decomposition entropy indicates {\it learnability}, i.e., how rapidly a model converges. Subsequently, we train a neural network model to rapidly learn specific features for further downstream analyses.