Multiple mechanisms of rhythm switching in recurrent neural networks with adaptive time constants

TL;DR

This study investigates how recurrent neural networks switch between different neural rhythms across frequency bands, revealing multiple mechanisms and their relation to neuron-specific time constants, with parallels to biological neural circuits.

Contribution

The paper systematically analyzes internal mechanisms of rhythm switching in RNNs with adaptive time constants, highlighting multiple coexisting strategies and their relation to neuronal properties.

Findings

Low-frequency rhythms involve many neurons with longer time constants.

High-frequency rhythms are dominated by neurons with short time constants.

Rhythm switching mechanisms include population turnover, baseline shifts, and phase reorganization.

Abstract

Although recurrent neural networks (RNNs) trained on cognitive tasks have become a widely used framework for studying neural computation, the internal mechanisms by which RNNs switch between rhythms across multiple frequency bands, and how these mechanisms relate to neuronal time constants, have not been systematically analyzed. We trained leaky integrator RNNs with neuron-specific learnable time constants on a four-band (theta, alpha, beta, gamma) rhythm-switching task and analyzed 20 independently trained networks. Whereas low-frequency rhythms were produced by distributed participation of many neurons, high-frequency rhythms were dominated by a small subpopulation of short-time-constant neurons, and the negative correlation between time constant and matched-mode amplitude strengthened monotonically with frequency. Rhythm switching was supported by multiple coexisting mechanisms: turnover of the active subpopulation, network-wide baseline shifts that reposition the operating point near distinct unstable fixed points, and inter-neuronal phase reorganization that selectively cancels or supports band components in the population output. The mechanism deployed for each mode pair varied across training runs, exposing a degeneracy of learned solutions. These findings parallel the coexistence of rhythm-specific and multi-rhythm interneurons reported in biological circuits and provide a candidate framework for interpreting frequency-band-specific functional differentiation in neural systems.