Criticality in Reservoir Computer of Coupled Phase Oscillators

TL;DR

This paper demonstrates that a reservoir computer composed of coupled phase oscillators naturally reaches a critical state characterized by power-law distributed cluster sizes when properly trained for chaos prediction, revealing a universal attractor in machine learning.

Contribution

It introduces a reservoir computing model of coupled phase oscillators that exhibits criticality through power-law cluster size distributions, a phenomenon previously observed only in biological neural systems.

Findings

Proper training induces power-law distributed oscillator clusters.

The critical state is independent of reservoir details and target bifurcations.

The criticality is consistent across different reservoir models and target systems.

Abstract

Accumulating evidences show that the cerebral cortex is operating near a critical state featured by power-law size distribution of neural avalanche activities, yet evidence of this critical state in artificial neural networks mimicking the cerebral cortex is lacking. Here we design an artificial neural network of coupled phase oscillators and, by the technique of reservoir computing in machine learning, train it for predicting chaos. It is found that when the machine is properly trained, oscillators in the reservoir are synchronized into clusters whose sizes follow a power-law distribution. This feature, however, is absent when the machine is poorly trained. Additionally, it is found that despite the synchronization degree of the original network, once properly trained, the reservoir network is always developed to the same critical state, exemplifying the "attractor" nature of this state in machine learning. The generality of the results is verified in different reservoir models and by different target systems, and it is found that the scaling exponent of the distribution is independent on the reservoir details and the bifurcation parameter of the target system, but is modified when the dynamics of the target system is changed to a different type. The findings shed lights on the nature of machine learning, and are helpful to the design of high-performance machine in physical systems.