A Dynamical Theory of Sequential Retrieval in Input-Driven Hopfield Networks

TL;DR

This paper develops a mathematical framework for understanding how input-driven Hopfield networks perform sequential reasoning, bridging classical associative memory models and modern reasoning architectures.

Contribution

It introduces a dynamical theory for sequential retrieval in Hopfield networks, analyzing a two-timescale architecture with explicit conditions for memory transitions.

Findings

Derived conditions for self-sustained memory transitions

Analyzed gain thresholds, escape times, and collapse regimes

Bridged classical Hopfield dynamics with modern reasoning models

Abstract

Reasoning is the ability to integrate internal states and external inputs in a meaningful and semantically consistent flow. Contemporary machine learning (ML) systems increasingly rely on such sequential reasoning, from language understanding to multi-modal generation, often operating over dictionaries of prototypical patterns reminiscent of associative memory models. Understanding retrieval and sequentiality in associative memory models provides a powerful bridge to gain insight into ML reasoning. While the static retrieval properties of associative memory models are well understood, the theoretical foundations of sequential retrieval and multi-memory integration remain limited, with existing studies largely relying on numerical evidence. This work develops a dynamical theory of sequential reasoning in Hopfield networks. We consider the recently proposed input-driven plasticity (IDP) Hopfield network and analyze a two-timescale architecture coupling fast associative retrieval with slow reasoning dynamics. We derive explicit conditions for self-sustained memory transitions, including gain thresholds, escape times, and collapse regimes. Together, these results provide a principled mathematical account of sequentiality in associative memory models, bridging classical Hopfield dynamics and modern reasoning architectures.