Quantization of Deep Neural Networks to facilitate self-correction of weights on Phase Change Memory-based analog hardware

TL;DR

This paper introduces a quantization and self-correction method for neural networks on phase change memory hardware, enabling efficient weight representation and maintaining performance over time.

Contribution

It proposes a novel quantization and self-correcting algorithm using dual crossbar arrays for neural networks on analog hardware.

Findings

Self-correcting neural networks perform comparably to analog-aware trained models.

The method effectively approximates weights with minimal performance loss.

On-chip pulse generators enhance the stability of the neural network performance.

Abstract

In recent years, hardware-accelerated neural networks have gained significant attention for edge computing applications. Among various hardware options, crossbar arrays, offer a promising avenue for efficient storage and manipulation of neural network weights. However, the transition from trained floating-point models to hardware-constrained analog architectures remains a challenge. In this work, we combine a quantization technique specifically designed for such architectures with a novel self-correcting mechanism. By utilizing dual crossbar connections to represent both the positive and negative parts of a single weight, we develop an algorithm to approximate a set of multiplicative weights. These weights, along with their differences, aim to represent the original network's weights with minimal loss in performance. We implement the models using IBM's aihwkit and evaluate their efficacy over time. Our results demonstrate that, when paired with an on-chip pulse generator, our self-correcting neural network performs comparably to those trained with analog-aware algorithms.