Neural Langevin Machine: a local asymmetric learning rule can be creative

TL;DR

The paper introduces the neural Langevin machine, a biologically plausible generative model that uses local asymmetric learning rules and neural Langevin dynamics to sample and generate data, mimicking brain-like imagination.

Contribution

It presents a novel neural Langevin machine with an interpretable distribution and a biologically relevant local learning rule for sampling and generation.

Findings

Provides a simple, trainable generative model based on fixed points of neural networks.

Derives a local asymmetric plasticity rule for learning.

Demonstrates continuous creative sampling akin to brain imagination processes.

Abstract

Fixed points of recurrent neural networks can be leveraged to store and generate information. These fixed points can be captured by the Boltzmann-Gibbs measure, which leads to neural Langevin dynamics that can be used for sampling and learning a real dataset. We call this type of generative model neural Langevin machine, which is interpretable due to its analytic form of distribution and is simple to train. Moreover, the learning process is derived as a local asymmetric plasticity rule, bearing biological relevance. Therefore, one can realize a continuous sampling of creative dynamics in a neural network, mimicking an imagination process in brain circuits. This neural Langevin machine may be another promising generative model, at least in its strength in circuit-based sampling and biologically plausible learning rule.