Reservoir-size dependent learning in analogue neural networks

TL;DR

This paper investigates how the size of the reservoir in a photonic neural network affects learning speed, revealing near-linear scaling and optimal conditions for hardware-based neural network training.

Contribution

It provides the first detailed analysis of reservoir size impact on learning convergence speed in photonic neural networks, demonstrating near-linear scaling.

Findings

Reservoir size linearly influences learning convergence speed.

Parallel diffractive coupling enhances scaling efficiency.

Fundamental properties of the optimization landscape are uncovered.

Abstract

The implementation of artificial neural networks in hardware substrates is a major interdisciplinary enterprise. Well suited candidates for physical implementations must combine nonlinear neurons with dedicated and efficient hardware solutions for both connectivity and training. Reservoir computing addresses the problems related with the network connectivity and training in an elegant and efficient way. However, important questions regarding impact of reservoir size and learning routines on the convergence-speed during learning remain unaddressed. Here, we study in detail the learning process of a recently demonstrated photonic neural network based on a reservoir. We use a greedy algorithm to train our neural network for the task of chaotic signals prediction and analyze the learning-error landscape. Our results unveil fundamental properties of the system's optimization hyperspace. Particularly, we determine the convergence speed of learning as a function of reservoir size and find exceptional, close to linear scaling. This linear dependence, together with our parallel diffractive coupling, represent optimal scaling conditions for our photonic neural network scheme.