Modeling Memristor-Based Neural Networks with Manhattan Update: Trade-offs in Learning Performance and Energy Consumption

TL;DR

This paper systematically studies memristor-based neural networks trained with Manhattan updates, analyzing trade-offs between learning performance and energy consumption, and proposing strategies to optimize device and algorithm co-design for low-power neuromorphic hardware.

Contribution

It introduces a comprehensive evaluation of memristor device nonlinearity effects and proposes a fixed memristor strategy to reduce energy consumption with minimal accuracy loss.

Findings

SPs tolerate P/D nonlinearity up to NLI ≤ 0.01

DNNs require NLI ≤ 0.001 for accuracy preservation

Fixing one memristor per differential pair reduces training energy by nearly 50%

Abstract

We present a systematic study of memristor based neural networks trained with the hardware-friendly Manhattan update rule, focusing on the trade offs between learning performance and energy consumption. Using realistic models of potentiation/depression (P/D) curves, we evaluate the impact of nonlinearity (NLI), conductance range, and number of accessible levels on both a single perceptron (SP) and a deep neural network (DNN) trained on the MNIST dataset. Our results show that SPs tolerate P/D nonlinearity up to NLI $\leq 0.01$, while DNNs require stricter conditions of NLI $\leq$ 0.001 to preserve accuracy. Increasing the number of discrete conductance states improves convergence, effectively acting as a finer learning rate. We further propose a strategy where one memristor of each differential pair is fixed, reducing redundant memristor conductance updates. This approach lowers training energy by nearly 50% in DNN with little to no loss in accuracy. Our findings highlight the importance of device algorithm codesign in enabling scalable, low power neuromorphic hardware for edge AI applications.