Finite Size Effects in Separable Recurrent Neural Networks

TL;DR

This paper analytically investigates finite size effects in separable recurrent neural networks, identifying thermal and disorder-induced corrections, modeled by Ornstein-Uhlenbeck processes, to better understand their impact on network dynamics.

Contribution

It provides a systematic analytical framework for understanding finite size effects in separable recurrent neural networks, including effects beyond saturation and without assuming detailed balance.

Findings

Finite size effects include thermal fluctuations and frozen disorder corrections.

Finite size effects are modeled by time-dependent Ornstein-Uhlenbeck processes.

The theory helps quantify phenomena in recurrent neural networks.

Abstract

We perform a systematic analytical study of finite size effects in separable recurrent neural network models with sequential dynamics, away from saturation. We find two types of finite size effects: thermal fluctuations, and disorder-induced `frozen' corrections to the mean-field laws. The finite size effects are described by equations that correspond to a time-dependent Ornstein-Uhlenbeck process. We show how the theory can be used to understand and quantify various finite size phenomena in recurrent neural networks, with and without detailed balance.