Interpreting learning in biological neural networks as zero-order optimization method

TL;DR

This paper explores how biological neural networks learn by relating their local update rules to a zero-order optimization method, offering a new perspective on brain learning mechanisms distinct from gradient descent.

Contribution

It introduces a novel interpretation of biological neural network learning as a zero-order optimization process, contrasting with traditional gradient-based methods.

Findings

Biological neural network updates can be modeled as zero-order optimization.

Expected values of local updates resemble a modified gradient descent.

Provides a statistical framework for understanding brain learning processes.

Abstract

Recently, significant progress has been made regarding the statistical understanding of artificial neural networks (ANNs). ANNs are motivated by the functioning of the brain, but differ in several crucial aspects. In particular, the locality in the updating rule of the connection parameters in biological neural networks (BNNs) makes it biologically implausible that the learning of the brain is based on gradient descent. In this work, we look at the brain as a statistical method for supervised learning. The main contribution is to relate the local updating rule of the connection parameters in BNNs to a zero-order optimization method. It is shown that the expected values of the iterates implement a modification of gradient descent.