Nonlinear dynamics of localization in neural receptive fields

TL;DR

This paper investigates how localized receptive fields in neural networks emerge through nonlinear learning dynamics driven by higher-order input statistics, offering an alternative to efficiency-based explanations.

Contribution

It derives the effective learning dynamics for a nonlinear neuron, explaining how higher-order statistics lead to localization without explicit efficiency constraints.

Findings

Localization arises from nonlinear learning dynamics.

Higher-order input statistics drive receptive field emergence.

Predictions extend from single neuron to neural populations.

Abstract

Localized receptive fields -- neurons that are selective for certain contiguous spatiotemporal features of their input -- populate early sensory regions of the mammalian brain. Unsupervised learning algorithms that optimize explicit sparsity or independence criteria replicate features of these localized receptive fields, but fail to explain directly how localization arises through learning without efficient coding, as occurs in early layers of deep neural networks and might occur in early sensory regions of biological systems. We consider an alternative model in which localized receptive fields emerge without explicit top-down efficiency constraints -- a feedforward neural network trained on a data model inspired by the structure of natural images. Previous work identified the importance of non-Gaussian statistics to localization in this setting but left open questions about the mechanisms driving dynamical emergence. We address these questions by deriving the effective learning dynamics for a single nonlinear neuron, making precise how higher-order statistical properties of the input data drive emergent localization, and we demonstrate that the predictions of these effective dynamics extend to the many-neuron setting. Our analysis provides an alternative explanation for the ubiquity of localization as resulting from the nonlinear dynamics of learning in neural circuits.