Impact of leaky dynamics on predictive path integration accuracy in recurrent neural networks

TL;DR

This paper shows that adding a leak term to recurrent neural networks improves the stability, regularity, and accuracy of grid-like firing patterns used for path integration, by acting as a low-pass filter.

Contribution

Introducing adaptive time scales via a leak term in RNNs enhances grid pattern emergence and stability, improving path integration accuracy compared to standard RNNs.

Findings

Leaky RNNs produce more accurate position estimates.

Leaky RNNs exhibit more stable and regular grid-like activity.

Leaky RNNs form stable torus attractors supporting robust grid patterns.

Abstract

Experimental evidence indicates that intrinsic temporal dynamics operating across multiple time scales are closely associated with the emergence of periodic spatial activity of increasing complexity. However, how information encoded in grid-like firing patterns for path integration is processed across these intrinsic time scales remains unclear. To address this question, we introduce adaptive time scales through a leak term in recurrent neural networks (RNNs), forming leaky RNNs discretized from the continuous attractors of firing rate models. Our results demonstrate that leaky RNNs substantially enhance the emergence of well-defined and highly regular hexagonal firing patterns. Compared with vanilla RNNs lacking a leak term, the trained leaky RNNs produce more accurate position estimates while generating reliable grid-cell-like representations. Furthermore, under identical noise conditions, leaky RNNs consistently exhibit more stable dynamics and better-defined grid structures. The learned dynamics also give rise to stable torus attractors with a clear central hole, supporting robust and regular grid-like activity. Overall, the dynamic leak acts as a low-pass filtering mechanism that protects recurrent neural circuitry from noise, stabilizes network dynamics, and improves path-integration accuracy in recurrent neural networks.