Sensitivity-Aware Finetuning for Accuracy Recovery on Deep Learning Hardware

TL;DR

This paper introduces Sensitivity-Aware Finetuning (SAFT), a method that speeds up accuracy recovery on analog-digital hardware by selectively freezing layers during noise-injection training, maintaining accuracy efficiently.

Contribution

SAFT identifies noise-sensitive layers and selectively freezes them, significantly reducing training time while preserving model accuracy on hardware with quantization and analog noise.

Findings

SAFT achieves comparable accuracy to traditional noise-injection training.

SAFT is 2x to 8x faster than existing methods.

SAFT effectively recovers model accuracy with reduced computational cost.

Abstract

Existing methods to recover model accuracy on analog-digital hardware in the presence of quantization and analog noise include noise-injection training. However, it can be slow in practice, incurring high computational costs, even when starting from pretrained models. We introduce the Sensitivity-Aware Finetuning (SAFT) approach that identifies noise sensitive layers in a model, and uses the information to freeze specific layers for noise-injection training. Our results show that SAFT achieves comparable accuracy to noise-injection training and is 2x to 8x faster.