How to Train Your Resistive Network: Generalized Equilibrium Propagation and Analytical Learning

TL;DR

This paper introduces a new analytical framework for training resistive networks using generalized equilibrium propagation, enabling energy-efficient analog learning with local updates and without extensive readout requirements.

Contribution

It develops an exact gradient calculation method based on Kirchhoff's laws and introduces a broad framework for local Hebbian learning algorithms in resistive networks.

Findings

Able to train resistor networks without full resistor readout.

Gradient updates can be limited to a subset of resistors with minimal performance loss.

Demonstrated effectiveness through numerical simulations.

Abstract

Machine learning is a powerful method of extracting meaning from data; unfortunately, current digital hardware is extremely energy-intensive. There is interest in an alternative analog computing implementation that could match the performance of traditional machine learning while being significantly more energy-efficient. However, it remains unclear how to train such analog computing systems while adhering to locality constraints imposed by the physical (as opposed to digital) nature of these systems. Local learning algorithms such as Equilibrium Propagation and Coupled Learning have been proposed to address this issue. In this paper, we develop an algorithm to exactly calculate gradients using a graph theoretic and analytical framework for Kirchhoff's laws. We also introduce Generalized Equilibrium Propagation, a framework encompassing a broad class of Hebbian learning algorithms, including Coupled Learning and Equilibrium Propagation, and show how our algorithm compares. We demonstrate our algorithm using numerical simulations and show that we can train resistor networks without the need for a replica or readout over all resistors, only at the output layer. We also show that under the analytical gradient approach, it is possible to update only a subset of the resistance values without a strong degradation in performance.