Learning selectivity and invariance through spatiotemporal Hebbian plasticity in a hierarchical neural network

TL;DR

This paper shows that hierarchical neural networks with Hebbian plasticity can learn to recognize objects invariant to transformations, and that deeper architectures outperform shallower ones for this task.

Contribution

It introduces a biologically plausible learning rule enabling hierarchical networks to develop selectivity and invariance, and demonstrates the advantage of depth via genetic architecture search.

Findings

Hierarchical networks with Hebbian plasticity achieve higher mutual information about object categories.

Deeper networks outperform shallower ones in transformation-invariant recognition.

Genetic algorithms optimize network architecture for better invariance performance.

Abstract

When an object moves smoothly across a field of view, the identify of the object is unchanged, but the activation pattern of the photoreceptors on the retina changes drastically. One of the major computational roles of our visual system is to manage selectivity for different objects and tolerance to such identity-preserving transformations as translations or rotations. This study demonstrates that a hierarchical neural network, whose synaptic connectivities are learned competitively with Hebbian plasticity operating within a local spatiotemporal pooling range, is capable of gradually achieving feature selectivity and transformation tolerance, so that the top level neurons carry higher mutual information about object categories than a single-level neural network. Furthermore, when genetic algorithm is applied to search for a network architecture that maximizes transformation-invariant object recognition performance, in conjunction with the associative learning algorithm, it is found that deep networks outperform shallower ones.