Unfolding recurrence by Green's functions for optimized reservoir computing

TL;DR

This paper introduces an analytical framework using Green's functions to transform recurrent cortical network dynamics into an effective feed-forward structure, enabling optimized reservoir computing for improved time-series classification.

Contribution

It presents a solvable recurrent network model linking recurrence and feed-forward networks, allowing optimization of input and readout for reservoir computing.

Findings

Optimized input projection enhances performance.

Second order stimulus statistics significantly boost classification accuracy.

Analytical expressions enable better understanding of recurrence and non-linearity interactions.

Abstract

Cortical networks are strongly recurrent, and neurons have intrinsic temporal dynamics. This sets them apart from deep feed-forward networks. Despite the tremendous progress in the application of feed-forward networks and their theoretical understanding, it remains unclear how the interplay of recurrence and non-linearities in recurrent cortical networks contributes to their function. The purpose of this work is to present a solvable recurrent network model that links to feed forward networks. By perturbative methods we transform the time-continuous, recurrent dynamics into an effective feed-forward structure of linear and non-linear temporal kernels. The resulting analytical expressions allow us to build optimal time-series classifiers from random reservoir networks. Firstly, this allows us to optimize not only the readout vectors, but also the input projection, demonstrating a strong potential performance gain. Secondly, the analysis exposes how the second order stimulus statistics is a crucial element that interacts with the non-linearity of the dynamics and boosts performance.