Is Learning in Biological Neural Networks based on Stochastic Gradient Descent? An analysis using stochastic processes

TL;DR

This paper investigates whether stochastic gradient descent, a common optimization method in artificial neural networks, could also be applicable to biological neural networks by analyzing a stochastic learning model.

Contribution

The study introduces a stochastic model for supervised learning in BNNs and demonstrates conditions under which gradient-like updates occur, suggesting a potential role for SGD in BNNs.

Findings

Gradient steps occur when many local updates are processed.

Stochastic gradient descent may be relevant for BNN learning.

Supports the idea that BNNs could optimize via SGD-like mechanisms.

Abstract

In recent years, there has been an intense debate about how learning in biological neural networks (BNNs) differs from learning in artificial neural networks. It is often argued that the updating of connections in the brain relies only on local information, and therefore a stochastic gradient-descent type optimization method cannot be used. In this paper, we study a stochastic model for supervised learning in BNNs. We show that a (continuous) gradient step occurs approximately when each learning opportunity is processed by many local updates. This result suggests that stochastic gradient descent may indeed play a role in optimizing BNNs.