Dependence of Equilibrium Propagation Training Success on Network Architecture

TL;DR

This paper investigates how the architecture of neural networks, specifically locally connected lattices, affects the success of equilibrium propagation training, showing sparse local networks can perform comparably to dense ones.

Contribution

It demonstrates that equilibrium propagation can be effectively applied to realistic, locally connected architectures, expanding its potential for practical neuromorphic systems.

Findings

Sparse local networks achieve performance similar to dense networks.

Architecture influences the evolution of responses and couplings during training.

Guidelines for scaling equilibrium propagation to realistic architectures are provided.

Abstract

The rapid rise of artificial intelligence has led to an unsustainable growth in energy consumption. This has motivated progress in neuromorphic computing and physics-based training of learning machines as alternatives to digital neural networks. Many theoretical studies focus on simple architectures like all-to-all or densely connected layered networks. However, these may be challenging to realize experimentally, e.g. due to connectivity constraints. In this work, we investigate the performance of the widespread physics-based training method of equilibrium propagation for more realistic architectural choices, specifically, locally connected lattices. We train an XY model and explore the influence of architecture on various benchmark tasks, tracking the evolution of spatially distributed responses and couplings during training. Our results show that sparse networks with only local connections can achieve performance comparable to dense networks. Our findings provide guidelines for further scaling up architectures based on equilibrium propagation in realistic settings.