A Fast Algorithm to Simulate Nonlinear Resistive Networks

TL;DR

This paper presents a novel, fast algorithm for simulating nonlinear resistive networks, enabling larger and more efficient training of analog neural networks for machine learning applications.

Contribution

The authors introduce a quadratic programming-based simulation method solved by a coordinate descent algorithm, vastly outperforming traditional SPICE simulations in speed and scalability.

Findings

Simulates nonlinear resistive networks up to 327 times larger

Achieves 160 times faster simulation speeds

Improves the network size to epoch duration ratio by 50,000-fold

Abstract

Analog electrical networks have long been investigated as energy-efficient computing platforms for machine learning, leveraging analog physics during inference. More recently, resistor networks have sparked particular interest due to their ability to learn using local rules (such as equilibrium propagation), enabling potentially important energy efficiency gains for training as well. Despite their potential advantage, the simulations of these resistor networks has been a significant bottleneck to assess their scalability, with current methods either being limited to linear networks or relying on realistic, yet slow circuit simulators like SPICE. Assuming ideal circuit elements, we introduce a novel approach for the simulation of nonlinear resistive networks, which we frame as a quadratic programming problem with linear inequality constraints, and which we solve using a fast, exact coordinate descent algorithm. Our simulation methodology significantly outperforms existing SPICE-based simulations, enabling the training of networks up to 327 times larger at speeds 160 times faster, resulting in a 50,000-fold improvement in the ratio of network size to epoch duration. Our approach can foster more rapid progress in the simulations of nonlinear analog electrical networks.