Contrastive Topographic Models: Energy-based density models applied to the understanding of sensory coding and cortical topography

TL;DR

This paper introduces energy-based probabilistic models to understand visual cortical topography, demonstrating their ability to replicate receptive field properties and map structures observed in biological visual systems.

Contribution

It develops a class of energy-based models with biologically inspired constraints, linking them to existing probabilistic frameworks and applying them to model visual cortical organization.

Findings

Models qualitatively reproduce receptive field properties.

Models learn statistical regularities of naturalistic data.

Framework unifies various probabilistic modeling approaches.

Abstract

We address the problem of building theoretical models that help elucidate the function of the visual brain at computational/algorithmic and structural/mechanistic levels. We seek to understand how the receptive fields and topographic maps found in visual cortical areas relate to underlying computational desiderata. We view the development of sensory systems from the popular perspective of probability density estimation; this is motivated by the notion that an effective internal representational scheme is likely to reflect the statistical structure of the environment in which an organism lives. We apply biologically based constraints on elements of the model. The thesis begins by surveying the relevant literature from the fields of neurobiology, theoretical neuroscience, and machine learning. After this review we present our main theoretical and algorithmic developments: we propose a class of probabilistic models, which we refer to as "energy-based models", and show equivalences between this framework and various other types of probabilistic model such as Markov random fields and factor graphs; we also develop and discuss approximate algorithms for performing maximum likelihood learning and inference in our energy based models. The rest of the thesis is then concerned with exploring specific instantiations of such models. By performing constrained optimisation of model parameters to maximise the likelihood of appropriate, naturalistic datasets we are able to qualitatively reproduce many of the receptive field and map properties found in vivo, whilst simultaneously learning about statistical regularities in the data.