Sequence-to-Sequence Models with Attention Mechanistically Map to the Architecture of Human Memory Search

TL;DR

This paper shows that neural sequence-to-sequence models with attention mechanisms mirror human memory search processes, offering interpretable models that align with cognitive theories and replicate human behavioral patterns.

Contribution

It reveals the mechanistic correspondence between neural translation models and human memory architectures, providing a novel, interpretable cognitive model of memory search.

Findings

Model accounts for human behavioral patterns in memory search

Neural models exhibit mechanisms similar to human memory processes

Model captures complex learning dynamics in memory retrieval

Abstract

Past work has long recognized the important role of context in guiding how humans search their memory. While context-based memory models can explain many memory phenomena, it remains unclear why humans develop such architectures over possible alternatives in the first place. In this work, we demonstrate that foundational architectures in neural machine translation -- specifically, recurrent neural network (RNN)-based sequence-to-sequence models with attention -- exhibit mechanisms that directly correspond to those specified in the Context Maintenance and Retrieval (CMR) model of human memory. Since neural machine translation models have evolved to optimize task performance, their convergence with human memory models provides a deeper understanding of the functional role of context in human memory, as well as presenting new ways to model human memory. Leveraging this convergence, we implement a neural machine translation model as a cognitive model of human memory search that is both interpretable and capable of capturing complex dynamics of learning. We show that our model accounts for both averaged and optimal human behavioral patterns as effectively as context-based memory models. Further, we demonstrate additional strengths of the proposed model by evaluating how memory search performance emerges from the interaction of different model components.