Sparse, Geometric Autoencoder Models of V1

TL;DR

This paper introduces a structured sparse autoencoder model that organizes latent representations for spectral clustering, producing artificial neurons that better match primate visual cortex data.

Contribution

It proposes a novel autoencoder architecture with a weighted-$ ext{l}_1$ constraint that enhances structured sparsity and hierarchical differentiation of receptive fields.

Findings

Latent representations are organized for spectral clustering.

Artificial neurons better match primate data.

The weighted-$ ext{l}_1$ constraint improves structured sparsity.

Abstract

The classical sparse coding model represents visual stimuli as a linear combination of a handful of learned basis functions that are Gabor-like when trained on natural image data. However, the Gabor-like filters learned by classical sparse coding far overpredict well-tuned simple cell receptive field (SCRF) profiles. A number of subsequent models have either discarded the sparse dictionary learning framework entirely or have yet to take advantage of the surge in unrolled, neural dictionary learning architectures. A key missing theme of these updates is a stronger notion of \emph{structured sparsity}. We propose an autoencoder architecture whose latent representations are implicitly, locally organized for spectral clustering, which begets artificial neurons better matched to observed primate data. The weighted-$\ell_1$ (WL) constraint in the autoencoder objective function maintains core ideas of the sparse coding framework, yet also offers a promising path to describe the differentiation of receptive fields in terms of a discriminative hierarchy in future work.