Training Modern Deep Neural Networks for Memory-Fault Robustness

TL;DR

This paper explores energy-efficient deep neural network deployment by operating memory systems in a faulty regime, introducing a regularizer to improve robustness against memory faults and maintaining accuracy.

Contribution

It proposes a novel regularizer to enhance DNN robustness against memory faults caused by low-voltage operation, enabling energy savings without accuracy loss.

Findings

Operating in a faulty memory regime can save energy.

The regularizer improves DNN accuracy under memory faults.

Fault-tolerant training maintains performance in defective memory conditions.

Abstract

Because deep neural networks (DNNs) rely on a large number of parameters and computations, their implementation in energy-constrained systems is challenging. In this paper, we investigate the solution of reducing the supply voltage of the memories used in the system, which results in bit-cell faults. We explore the robustness of state-of-the-art DNN architectures towards such defects and propose a regularizer meant to mitigate their effects on accuracy. Our experiments clearly demonstrate the interest of operating the system in a faulty regime to save energy without reducing accuracy.