Emergence of Complex-Like Cells in a Temporal Product Network with Local Receptive Fields

TL;DR

This paper presents a neural architecture that learns invariant, complex-like cells from temporal image sequences, demonstrating emergent orientation selectivity and receptive field organization akin to the mammalian visual cortex.

Contribution

Introduces a novel neural network model with local receptive fields that learns complex-like cells and organizes receptive fields into pinwheel patterns through unsupervised learning.

Findings

Units develop orientation and frequency selectivity similar to V1 complex cells.

Receptive fields organize into pinwheel patterns resembling mammalian visual cortex.

Efficient feed-forward encoding of full images achieved.

Abstract

We introduce a new neural architecture and an unsupervised algorithm for learning invariant representations from temporal sequence of images. The system uses two groups of complex cells whose outputs are combined multiplicatively: one that represents the content of the image, constrained to be constant over several consecutive frames, and one that represents the precise location of features, which is allowed to vary over time but constrained to be sparse. The architecture uses an encoder to extract features, and a decoder to reconstruct the input from the features. The method was applied to patches extracted from consecutive movie frames and produces orientation and frequency selective units analogous to the complex cells in V1. An extension of the method is proposed to train a network composed of units with local receptive field spread over a large image of arbitrary size. A layer of complex cells, subject to sparsity constraints, pool feature units over overlapping local neighborhoods, which causes the feature units to organize themselves into pinwheel patterns of orientation-selective receptive fields, similar to those observed in the mammalian visual cortex. A feed-forward encoder efficiently computes the feature representation of full images.