Efficient Learning of Sparse Invariant Representations

TL;DR

This paper introduces a fast, efficient algorithm for learning sparse invariant representations from unlabeled data, demonstrating robustness to position changes and biological plausibility similar to visual cortex cells.

Contribution

It presents a novel hierarchical algorithm with convergence guarantees for learning invariant features from unlabeled data.

Findings

Features are selective to orientations and frequencies

Learned representations are robust to position shifts

Algorithm converges quickly under certain conditions

Abstract

We propose a simple and efficient algorithm for learning sparse invariant representations from unlabeled data with fast inference. When trained on short movies sequences, the learned features are selective to a range of orientations and spatial frequencies, but robust to a wide range of positions, similar to complex cells in the primary visual cortex. We give a hierarchical version of the algorithm, and give guarantees of fast convergence under certain conditions.