Brain-inspired, interpretable, resonant recurrent neural networks

TL;DR

This paper introduces a biologically inspired resonant recurrent neural network framework with oscillatory node dynamics, demonstrating improved classification accuracy and interpretability over standard RNNs, especially when oscillator frequencies mimic brain rhythms.

Contribution

The paper presents a novel resonant recurrent neural network (RRN) with explicit oscillatory dynamics, integrating brain-inspired frequency constraints for enhanced interpretability and performance.

Findings

RRNs achieve accurate classification with few parameters.

Choosing brain rhythm-like frequencies improves accuracy.

RRNs offer interpretable features and biological plausibility.

Abstract

Traditional artificial neural networks consist of nodes with non-oscillatory dynamics. Biological neural networks, on the other hand, consist of oscillatory components embedded in an oscillatory environment. Motivated by this feature of biological neurons, we describe a neural network framework with explicit damped, oscillatory node dynamics. We express the oscillatory dynamics using two history dependent terms to connect these dynamics with standard recurrent neural network formulations, apply physical constraints from observed brain dynamics to choose the oscillatory frequencies, and stationary constraints to reduce the number of free parameters. We then optimize and illustrate network performance by classifying hand-written digits and simulated neuronal spike train activity and show that these oscillatory network elements support accurate classification with few trainable parameters. Choosing oscillator frequencies according to a proposed theory for brain rhythms improves classification accuracy compared to alternative frequency configurations and compared to standard recurrent neural network frameworks with comparable numbers of parameters. Compared to existing approaches, the proposed resonant recurrent network (RRN) utilizes oscillatory dynamics expressed as a straightforward extension of standard recurrent neural networks, produces interpretable features for classification, and performs well with few parameters when oscillator frequencies follow a configuration observed in vivo. We propose that RRNs may serve as efficient, biologically inspired building blocks to achieve complex goals in biological and artificial neural networks.