Nonlinear Neural Dynamics and Classification Accuracy in Reservoir Computing

TL;DR

This study investigates how the nonlinear properties and dynamical regimes of reservoir computing systems influence their classification accuracy, revealing robustness even with minimal nonlinearity and optimal performance at phase boundaries.

Contribution

It provides a detailed analysis of the relationship between nonlinearity, dynamical regimes, and classification accuracy in reservoir computing, highlighting the importance of operating near phase boundaries.

Findings

Reservoirs perform well with minimal nonlinearity and weak recurrent interactions.

Chaotic dynamics can reduce classification performance.

Optimal accuracy occurs near phase boundaries, supporting the 'edge of chaos' hypothesis.

Abstract

Reservoir computing - information processing based on untrained recurrent neural networks with random connections - is expected to depend on the nonlinear properties of the neurons and the resulting oscillatory, chaotic, or fixpoint dynamics of the network. However, the required degree of nonlinearity and the range of suitable dynamical regimes for a given task are not fully understood. To clarify these questions, we study the accuracy of a reservoir computer in artificial classification tasks of varying complexity, while tuning the neuron's degree of nonlinearity and the reservoir's dynamical regime. We find that, even for activation functions with extremely reduced nonlinearity, weak recurrent interactions and small input signals, the reservoir is able to compute useful representations, detectable only in higher order principal components, that render complex classificiation tasks linearly separable for the readout layer. When increasing the recurrent coupling, the reservoir develops spontaneous dynamical behavior. Nevertheless, the input-related computations can 'ride on top' of oscillatory or fixpoint attractors without much loss of accuracy, whereas chaotic dynamics reduces task performance more drastically. By tuning the system through the full range of dynamical phases, we find that the accuracy peaks both at the oscillatory/chaotic and at the chaotic/fixpoint phase boundaries, thus supporting the 'edge of chaos' hypothesis. Our results, in particular the robust weakly nonlinear operating regime, may offer new perspectives both for technical and biological neural networks with random connectivity.