Running Conventional Automatic Speech Recognition on Memristor Hardware: A Simulated Approach

TL;DR

This paper introduces a PyTorch-based simulation library to evaluate large neural networks' performance on memristor hardware, demonstrating feasibility and quantization effects in speech recognition tasks.

Contribution

It presents the first simulation framework for large-scale neural networks on memristor hardware, capturing realistic hardware properties and assessing impact on speech recognition accuracy.

Findings

Limited 25% degradation in word error rate with 3-bit weights

Simulated large models outperform small-scale prototypes

Quantization-aware training improves accuracy on memristor hardware

Abstract

Memristor-based hardware offers new possibilities for energy-efficient machine learning (ML) by providing analog in-memory matrix multiplication. Current hardware prototypes cannot fit large neural networks, and related literature covers only small ML models for tasks like MNIST or single word recognition. Simulation can be used to explore how hardware properties affect larger models, but existing software assumes simplified hardware. We propose a PyTorch-based library based on "Synaptogen" to simulate neural network execution with accurately captured memristor hardware properties. For the first time, we show how an ML system with millions of parameters would behave on memristor hardware, using a Conformer trained on the speech recognition task TED-LIUMv2 as example. With adjusted quantization-aware training, we limit the relative degradation in word error rate to 25% when using a 3-bit weight precision to execute linear operations via simulated analog computation.