Learning Dynamics in Memristor-Based Equilibrium Propagation

TL;DR

This paper explores how memristor-based in-memory computing influences the learning dynamics of neural networks trained with equilibrium propagation, emphasizing the importance of memristor resistance range for convergence.

Contribution

It demonstrates that equilibrium propagation can converge with nonlinear memristor-driven weight updates if memristors have a sufficiently wide resistance range.

Findings

Robust convergence achieved with nonlinear memristor updates

Six memristor models characterized and integrated into evaluation framework

At least an order of magnitude resistance range needed for convergence

Abstract

Memristor-based in-memory computing has emerged as a promising paradigm to overcome the constraints of the von Neumann bottleneck and the memory wall by enabling fully parallelisable and energy-efficient vector-matrix multiplications. We investigate the effect of nonlinear, memristor-driven weight updates on the convergence behaviour of neural networks trained with equilibrium propagation (EqProp). Six memristor models were characterised by their voltage-current hysteresis and integrated into the EBANA framework for evaluation on two benchmark classification tasks. EqProp can achieve robust convergence under nonlinear weight updates, provided that memristors exhibit a sufficiently wide resistance range of at least an order of magnitude.