Repair Brain Damage: Real-Numbered Error Correction Code for Neural Network

TL;DR

This paper introduces a real-numbered error correction code for neural networks that detects and corrects memory and computational errors without impacting performance or increasing parameters.

Contribution

It presents a novel real-number-based ECC that integrates with neural networks to enhance fault tolerance while maintaining accuracy and parameter efficiency.

Findings

Effective detection and correction of memory errors.

No loss in classification accuracy.

No increase in model complexity.

Abstract

We consider a neural network (NN) that may experience memory faults and computational errors. In this paper, we propose a novel real-number-based error correction code (ECC) capable of detecting and correcting both memory errors and computational errors. The proposed approach introduces structures in the form of real-number-based linear constraints on the NN weights to enable error detection and correction, without sacrificing classification performance or increasing the number of real-valued NN parameters.