Leakage and Second-Order Dynamics Improve Hippocampal RNN Replay

TL;DR

This paper investigates how leakage and second-order dynamics like hidden state momentum enhance the replay capabilities of hippocampal-like RNNs, leading to more efficient and biologically plausible models of neural replay.

Contribution

It introduces a novel model incorporating leakage and hidden state momentum to improve replay dynamics in noisy RNNs, connecting it to underdamped Langevin sampling.

Findings

Leakage encourages exploration but causes non-Markovian slowdown.

Hidden state momentum counteracts slowness and maintains exploration.

Model successfully replicates hippocampal replay in simulated paths.

Abstract

Biological neural networks (like the hippocampus) can internally generate "replay" resembling stimulus-driven activity. Recent computational models of replay use noisy recurrent neural networks (RNNs) trained to path-integrate. Replay in these networks has been described as Langevin sampling, but new modifiers of noisy RNN replay have surpassed this description. We re-examine noisy RNN replay as sampling to understand or improve it in three ways: (1) Under simple assumptions, we prove that the gradients replay activity should follow are time-varying and difficult to estimate, but readily motivate the use of hidden state leakage in RNNs for replay. (2) We confirm that hidden state adaptation (negative feedback) encourages exploration in replay, but show that it incurs non-Markov sampling that also slows replay. (3) We propose the first model of temporally compressed replay in noisy path-integrating RNNs through hidden state momentum, connect it to underdamped Langevin sampling, and show that, together with adaptation, it counters slowness while maintaining exploration. We verify our findings via path-integration of 2D triangular and T-maze paths and of high-dimensional paths of synthetic rat place cell activity.