A neuro-inspired architecture for unsupervised continual learning based on online clustering and hierarchical predictive coding

TL;DR

This paper introduces a neuro-inspired, hierarchical architecture called STAM for unsupervised continual learning, based on online clustering and predictive coding, inspired by cortical columns.

Contribution

It proposes a novel cortical column-inspired architecture for continual learning that integrates hierarchical predictive coding and online clustering.

Findings

Architecture grounded in neuroscience principles.

Framework supports unsupervised learning in hierarchical networks.

Connections to biological neural structures are emphasized.

Abstract

We propose that the Continual Learning desiderata can be achieved through a neuro-inspired architecture, grounded on Mountcastle's cortical column hypothesis. The proposed architecture involves a single module, called Self-Taught Associative Memory (STAM), which models the function of a cortical column. STAMs are repeated in multi-level hierarchies involving feedforward, lateral and feedback connections. STAM networks learn in an unsupervised manner, based on a combination of online clustering and hierarchical predictive coding. This short paper only presents the architecture and its connections with neuroscience. A mathematical formulation and experimental results will be presented in an extended version of this paper.