Inhibitory Cross-Talk Enables Functional Lateralization in Attention-Coupled Latent Memory

TL;DR

This paper introduces a memory-augmented transformer with lateralized memory banks and inhibitory cross-talk, enabling functional lateralization that improves episodic recall while maintaining rule-based prediction.

Contribution

It demonstrates that inhibitory cross-talk between lateralized memory banks induces specialization, enhancing episodic memory retrieval in a transformer model.

Findings

Inhibitory cross-talk achieves saturated lateralization with minimal cross-talk probability.

Inhibitory model reduces cipher-domain loss by 124 times compared to baseline.

Lateralized memory is crucial for episodic recall but not for rule-based tasks.

Abstract

We present a memory-augmented transformer in which attention serves simultaneously as a retrieval, consolidation, and write-back operator. The core update, $A^\top A V W$, re-grounds retrieved values into persistent memory slots via the Gram matrix $A^\top A$, providing a principled tripartite projection: observation space $\to$ latent memory $\to$ supervised transformation. We partition the memory into lateralized left and right banks coupled through a sign-controlled cross-talk matrix $W_s$, and show that the sign of this coupling is decisive for specialization. Excitatory cross-talk ($s=+1$) causes bank-dominance collapse: one bank monopolises all inputs and $\mathcal{P}_{ct} \to 0.5$, despite lowering task loss. Inhibitory cross-talk ($s=-1$), motivated by the net inhibitory effect of callosal projections in human cortex, actively suppresses contralateral bank activation and achieves saturated specialization ($\mathcal{D}_{sep} = \pm 1.00$, $\mathcal{P}_{ct} \approx 0$). On a controlled symbolic benchmark combining an episodic bijection cipher (requiring associative recall) with a strict arithmetic progression (requiring rule extraction), the inhibitory model reduces cipher-domain loss by $124{\times}$ over the baseline while matching it on the arithmetic domain, confirming that persistent lateralized memory is necessary for episodic recall but not for rule-based prediction.