Complex topological features of reservoirs shape learning performances in bio-inspired recurrent neural networks

TL;DR

This study investigates how complex topological features of reservoirs, including biological connectomes, influence learning performance in bio-inspired recurrent neural networks, revealing that intricate network structures significantly impact computational capabilities.

Contribution

It demonstrates the impact of complex topological features on reservoir computing performance, especially from biological connectomes, advancing understanding of bio-inspired neural network design.

Findings

Reservoir performance correlates with the number of nodes and covariance matrix rank.

Modularity and link density significantly influence learning outcomes.

Simple topological features do not fully predict reservoir performance.

Abstract

Recurrent networks are a special class of artificial neural systems that use their internal states to perform computing tasks for machine learning. One of its state-of-the-art developments, i.e. reservoir computing (RC), uses the internal structure -- usually a static network with random structure -- to map an input signal into a nonlinear dynamical system defined in a higher dimensional space. Reservoirs are characterized by nonlinear interactions among their units and their ability to store information through recurrent loops, allowing to train artificial systems to learn task-specific dynamics. However, it is fundamentally unknown how the random topology of the reservoir affects the learning performance. Here, we fill this gap by considering a battery of synthetic networks -- characterized by different topological features -- and 45 empirical connectomes -- sampled from brain regions of organisms belonging to 8 different species -- to build the reservoir and testing the learning performance against a prediction task with a variety of complex input signals. We find nontrivial correlations between RC performances and both the number of nodes and rank of the covariance matrix of activation states, with performance depending on the nature -- stochastic or deterministic -- of input signals. Remarkably, the modularity and the link density of the reservoir are found to affect RC performances: these results cannot be predicted by models only accounting for simple topological features of the reservoir. Overall, our findings highlight that the complex topological features characterizing biophysical computing systems such as connectomes can be used to design efficient bio-inspired artificial neural networks.