Brain-like combination of feedforward and recurrent network components achieves prototype extraction and robust pattern recognition

TL;DR

This paper presents a biologically inspired neural network model combining feedforward and recurrent components, enabling prototype extraction and robust pattern recognition on complex, correlated stimuli like images.

Contribution

It introduces a novel integration of recurrent attractor networks with unsupervised Hebbian-Bayesian learned representations, addressing limitations of traditional associative memory models.

Findings

Recurrent attractor networks perform associative memory on learned representations.

The model extracts prototypes and handles distorted inputs effectively.

Biological properties like Hebbian plasticity are incorporated into the model.

Abstract

Associative memory has been a prominent candidate for the computation performed by the massively recurrent neocortical networks. Attractor networks implementing associative memory have offered mechanistic explanation for many cognitive phenomena. However, attractor memory models are typically trained using orthogonal or random patterns to avoid interference between memories, which makes them unfeasible for naturally occurring complex correlated stimuli like images. We approach this problem by combining a recurrent attractor network with a feedforward network that learns distributed representations using an unsupervised Hebbian-Bayesian learning rule. The resulting network model incorporates many known biological properties: unsupervised learning, Hebbian plasticity, sparse distributed activations, sparse connectivity, columnar and laminar cortical architecture, etc. We evaluate the synergistic effects of the feedforward and recurrent network components in complex pattern recognition tasks on the MNIST handwritten digits dataset. We demonstrate that the recurrent attractor component implements associative memory when trained on the feedforward-driven internal (hidden) representations. The associative memory is also shown to perform prototype extraction from the training data and make the representations robust to severely distorted input. We argue that several aspects of the proposed integration of feedforward and recurrent computations are particularly attractive from a machine learning perspective.