Holomorphic Equilibrium Propagation Computes Exact Gradients Through Finite Size Oscillations

TL;DR

This paper extends equilibrium propagation to holomorphic networks, enabling exact gradient computation with finite signals, robust noise handling, and successful large-scale application to ImageNet, advancing neuromorphic learning methods.

Contribution

It introduces a holomorphic extension of equilibrium propagation that computes exact gradients from finite oscillations, facilitating scalable and noise-robust learning.

Findings

Exact gradients obtained from finite oscillations.

Robust gradient estimation in noisy environments.

Achieved ImageNet performance comparable to backpropagation.

Abstract

Equilibrium propagation (EP) is an alternative to backpropagation (BP) that allows the training of deep neural networks with local learning rules. It thus provides a compelling framework for training neuromorphic systems and understanding learning in neurobiology. However, EP requires infinitesimal teaching signals, thereby limiting its applicability in noisy physical systems. Moreover, the algorithm requires separate temporal phases and has not been applied to large-scale problems. Here we address these issues by extending EP to holomorphic networks. We show analytically that this extension naturally leads to exact gradients even for finite-amplitude teaching signals. Importantly, the gradient can be computed as the first Fourier coefficient from finite neuronal activity oscillations in continuous time without requiring separate phases. Further, we demonstrate in numerical simulations that our approach permits robust estimation of gradients in the presence of noise and that deeper models benefit from the finite teaching signals. Finally, we establish the first benchmark for EP on the ImageNet 32x32 dataset and show that it matches the performance of an equivalent network trained with BP. Our work provides analytical insights that enable scaling EP to large-scale problems and establishes a formal framework for how oscillations could support learning in biological and neuromorphic systems.