Dynamical Mechanism of Sampling-based Stochastic Inference under Probabilistic Population Codes

TL;DR

This paper investigates how recurrent neural networks perform sampling-based probabilistic inference using dynamical systems, revealing mechanisms that outperform feedforward networks and offer insights into neural information processing.

Contribution

It demonstrates that RNNs utilize dynamical system properties for sampling, providing a novel understanding of neural sampling mechanisms and their advantages over feedforward networks.

Findings

RNNs perform sampling via dynamical systems properties

Sampling in RNNs acts as an inductive bias for better estimation

RNNs outperform FFNNs in probabilistic inference accuracy

Abstract

Animals are known to make efficient probabilistic inferences based on uncertain and noisy information from the outside world. Although it is known that generic neural networks can perform near-optimal point estimation by probabilistic population codes which has been proposed as a neural basis for encoding of probability distribution, the mechanisms of sampling-based inference has not been clarified. In this study, we trained two types of artificial neural networks: feedforward neural networks (FFNNs) and recurrent neural networks (RNNs) to perform sampling based probabilistic inference. Then, we analyzed and compared the mechanisms of sampling in the RNN with those in the FFNN. As a result, it was found that sampling in RNN is performed by a mechanism that efficiently utilizes the properties of dynamical systems, unlike FFNN. It was also found that sampling in RNNs acts as an inductive bias, enabling more accurate estimation than in MAP estimation. These results will provide important implications for the discussion of the relationship between dynamical systems and information processing in neural networks.