Harnessing and modulating chaos to sample from neural generative models

TL;DR

This paper explores how chaos in neural networks can be harnessed to enable sampling from generative models, proposing architectures that integrate chaotic dynamics with generative and energy-based models for improved sampling control.

Contribution

It introduces neural architectures that leverage chaos for sampling, combining chaotic recurrent networks with generative and energy-based models, and demonstrates biologically plausible control mechanisms.

Findings

Chaos can be used to facilitate sampling in neural networks.

Architectures combining chaos with generative models enable controllable sampling.

Biologically plausible gain modulation controls sampling rates.

Abstract

Chaos is generic in strongly-coupled recurrent networks of model neurons, and thought to be an easily accessible dynamical regime in the brain. While neural chaos is typically seen as an impediment to robust computation, we show how such chaos might play a functional role in allowing the brain to learn and sample from generative models. We construct architectures that combine a classic model of neural chaos either with a canonical generative modeling architecture or with energy-based models of neural memory. We show that these architectures have appealing properties for sampling, including easy biologically-plausible control of sampling rates via overall gain modulation.