Signatures of Bayesian inference emerge from energy efficient synapses

TL;DR

This paper demonstrates that energy-efficient synapses in biological and artificial neural systems exhibit signatures of Bayesian inference, linking synaptic reliability, energetic costs, and probabilistic reasoning.

Contribution

It introduces a formal connection between synaptic energy efficiency and Bayesian inference, revealing that energy-efficient synapses naturally display Bayesian signatures.

Findings

Synapses with lower variability have higher input firing rates.

Synapses with lower variability have lower learning rates.

Bayesian inference can explain the observed signatures of energy-efficient synapses.

Abstract

Biological synaptic transmission is unreliable, and this unreliability likely degrades neural circuit performance. While there are biophysical mechanisms that can increase reliability, for instance by increasing vesicle release probability, these mechanisms cost energy. We examined four such mechanisms along with the associated scaling of the energetic costs. We then embedded these energetic costs for reliability in artificial neural networks (ANN) with trainable stochastic synapses, and trained these networks on standard image classification tasks. The resulting networks revealed a tradeoff between circuit performance and the energetic cost of synaptic reliability. Additionally, the optimised networks exhibited two testable predictions consistent with pre-existing experimental data. Specifically, synapses with lower variability tended to have 1) higher input firing rates and 2) lower learning rates. Surprisingly, these predictions also arise when synapse statistics are inferred through Bayesian inference. Indeed, we were able to find a formal, theoretical link between the performance-reliability cost tradeoff and Bayesian inference. This connection suggests two incompatible possibilities: evolution may have chanced upon a scheme for implementing Bayesian inference by optimising energy efficiency, or alternatively, energy efficient synapses may display signatures of Bayesian inference without actually using Bayes to reason about uncertainty.