Effect in the spectra of eigenvalues and dynamics of RNNs trained with Excitatory-Inhibitory constraint

TL;DR

This paper investigates how excitatory-inhibitory constraints in RNNs affect eigenvalue spectra and dynamics, revealing geometric patterns in eigenvalue distributions and their implications for neural activity.

Contribution

It introduces a framework for training RNNs with excitatory-inhibitory constraints and analyzes the resulting eigenvalue distributions and dynamics.

Findings

Eigenvalues form a circle or ring depending on initial conditions.

Radius of eigenvalue distribution is unaffected by network size or E/I ratio.

Constrained networks show different activity dynamics compared to unconstrained ones.

Abstract

In order to comprehend and enhance models that describes various brain regions is important to study the dynamics of trained recurrent neural networks. Including Dales law in such models usually presents several challenges. However, this is an important aspect that allows computational models to better capture the characteristics of the brain. Here we present a framework to train networks using such constraint. Then we have used it to train them in simple decision making tasks. We characterized the eigenvalue distributions of the recurrent weight matrices of such networks. Interestingly, we discovered that the non-dominant eigenvalues of the recurrent weight matrix are distributed in a circle with a radius less than 1 for those whose initial condition before training was random normal and in a ring for those whose initial condition was random orthogonal. In both cases, the radius does not depend on the fraction of excitatory and inhibitory units nor the size of the network. Diminution of the radius, compared to networks trained without the constraint, has implications on the activity and dynamics that we discussed here.