Hierarchical temporal receptive windows and zero-shot timescale generalization in biologically constrained scale-invariant deep networks

TL;DR

This paper demonstrates that biologically constrained, scale-invariant deep networks naturally develop hierarchical temporal receptive windows and can generalize across timescales, providing insights into human cognition.

Contribution

It introduces a biologically plausible recurrent network with scale-invariant priors that learns faster and generalizes to new timescales, aligning with cortical hierarchy features.

Findings

Hierarchical TRWs emerge in scale-invariant networks without layer-specific time constants.

SITH-RNN learns faster with fewer parameters than generic RNNs.

SITH-RNN generalizes zero-shot to out-of-distribution timescales.

Abstract

Human cognition integrates information across nested timescales. While the cortex exhibits hierarchical Temporal Receptive Windows (TRWs), local circuits often display heterogeneous time constants. To reconcile this, we trained biologically constrained deep networks, based on scale-invariant hippocampal time cells, on a language classification task mimicking the hierarchical structure of language (e.g., 'letters' forming 'words'). First, using a feedforward model (SITHCon), we found that a hierarchy of TRWs emerged naturally across layers, despite the network having an identical spectrum of time constants within layers. We then distilled these inductive priors into a biologically plausible recurrent architecture, SITH-RNN. Training a sequence of architectures ranging from generic RNNs to this restricted subset showed that the scale-invariant SITH-RNN learned faster with orders-of-magnitude fewer parameters, and generalized zero-shot to out-of-distribution timescales. These results suggest the brain employs scale-invariant, sequential priors - coding "what" happened "when" - making recurrent networks with such priors particularly well-suited to describe human cognition.