HiCL: Hippocampal-Inspired Continual Learning

TL;DR

HiCL is a biologically inspired continual learning architecture that uses hippocampal circuitry principles to reduce forgetting and improve task learning efficiency.

Contribution

It introduces a novel hippocampal-inspired dual-memory system with biologically grounded gating and prioritized replay for scalable continual learning.

Findings

Reduces catastrophic forgetting in continual learning benchmarks.

Achieves near state-of-the-art performance with lower computational costs.

Effectively manages multiple sequential tasks using biologically inspired mechanisms.

Abstract

We propose HiCL, a novel hippocampal-inspired dual-memory continual learning architecture designed to mitigate catastrophic forgetting by using elements inspired by the hippocampal circuitry. Our system encodes inputs through a grid-cell-like layer, followed by sparse pattern separation using a dentate gyrus-inspired module with top-k sparsity. Episodic memory traces are maintained in a CA3-like autoassociative memory. Task-specific processing is dynamically managed via a DG-gated mixture-of-experts mechanism, wherein inputs are routed to experts based on cosine similarity between their normalized sparse DG representations and learned task-specific DG prototypes computed through online exponential moving averages. This biologically grounded yet mathematically principled gating strategy enables differentiable, scalable task-routing without relying on a separate gating network, and enhances the model's adaptability and efficiency in learning multiple sequential tasks. Cortical outputs are consolidated using Elastic Weight Consolidation weighted by inter-task similarity. Crucially, we incorporate prioritized replay of stored patterns to reinforce essential past experiences. Evaluations on standard continual learning benchmarks demonstrate the effectiveness of our architecture in reducing task interference, achieving near state-of-the-art results in continual learning tasks at lower computational costs. Our code is available here https://github.com/kushalk173-sc/HiCL.