Echo state graph neural networks with analogue random resistor arrays

TL;DR

This paper introduces a novel hardware-software co-designed echo state graph neural network using random resistor arrays, achieving significant improvements in energy efficiency and training cost for graph learning tasks.

Contribution

The work presents a new in-memory computing hardware platform with random resistor arrays for efficient graph neural network processing, reducing training costs and energy consumption.

Findings

Achieves 34.2x energy efficiency improvement on MUTAG dataset.

Reduces training cost by up to 570.4x compared to digital hardware.

Demonstrates state-of-the-art performance on multiple graph classification tasks.

Abstract

Recent years have witnessed an unprecedented surge of interest, from social networks to drug discovery, in learning representations of graph-structured data. However, graph neural networks, the machine learning models for handling graph-structured data, face significant challenges when running on conventional digital hardware, including von Neumann bottleneck incurred by physically separated memory and processing units, slowdown of Moore's law due to transistor scaling limit, and expensive training cost. Here we present a novel hardware-software co-design, the random resistor array-based echo state graph neural network, which addresses these challenges. The random resistor arrays not only harness low-cost, nanoscale and stackable resistors for highly efficient in-memory computing using simple physical laws, but also leverage the intrinsic stochasticity of dielectric breakdown to implement random projections in hardware for an echo state network that effectively minimizes the training cost thanks to its fixed and random weights. The system demonstrates state-of-the-art performance on both graph classification using the MUTAG and COLLAB datasets and node classification using the CORA dataset, achieving 34.2x, 93.2x, and 570.4x improvement of energy efficiency and 98.27%, 99.46%, and 95.12% reduction of training cost compared to conventional graph learning on digital hardware, respectively, which may pave the way for the next generation AI system for graph learning.