Inference by Reparameterization in Neural Population Codes

TL;DR

This paper introduces a biologically-plausible neural network model that uses Probabilistic Population Codes and Tree-based Reparameterizations to perform approximate probabilistic inference, aligning with neural plausibility and robustness.

Contribution

It reformulates Loopy Belief Propagation as a neural-compatible dynamical system using PPCs and TRP, enabling neural implementation of complex probabilistic inference.

Findings

Neural network performs inference comparable to traditional LBP.

Model is robust to noise in Gaussian graphical models.

Provides a biologically plausible mechanism for probabilistic inference.

Abstract

Behavioral experiments on humans and animals suggest that the brain performs probabilistic inference to interpret its environment. Here we present a new general-purpose, biologically-plausible neural implementation of approximate inference. The neural network represents uncertainty using Probabilistic Population Codes (PPCs), which are distributed neural representations that naturally encode probability distributions, and support marginalization and evidence integration in a biologically-plausible manner. By connecting multiple PPCs together as a probabilistic graphical model, we represent multivariate probability distributions. Approximate inference in graphical models can be accomplished by message-passing algorithms that disseminate local information throughout the graph. An attractive and often accurate example of such an algorithm is Loopy Belief Propagation (LBP), which uses local marginalization and evidence integration operations to perform approximate inference efficiently even for complex models. Unfortunately, a subtle feature of LBP renders it neurally implausible. However, LBP can be elegantly reformulated as a sequence of Tree-based Reparameterizations (TRP) of the graphical model. We re-express the TRP updates as a nonlinear dynamical system with both fast and slow timescales, and show that this produces a neurally plausible solution. By combining all of these ideas, we show that a network of PPCs can represent multivariate probability distributions and implement the TRP updates to perform probabilistic inference. Simulations with Gaussian graphical models demonstrate that the neural network inference quality is comparable to the direct evaluation of LBP and robust to noise, and thus provides a promising mechanism for general probabilistic inference in the population codes of the brain.