Relevant-features based Auxiliary Cells for Energy Efficient Detection of Natural Errors

TL;DR

This paper introduces Relevant-features based Auxiliary Cells (RACs), an energy-efficient ensemble approach integrated into neural networks to detect natural errors early, reducing energy consumption during classification.

Contribution

It proposes RACs, class-specific binary classifiers at hidden layers, enabling early error detection and energy savings without significant accuracy loss.

Findings

Effective error detection on CIFAR-10, CIFAR-100, Tiny-ImageNet

Early classification reduces energy consumption

Maintains high accuracy while detecting errors

Abstract

Deep neural networks have demonstrated state-of-the-art performance on many classification tasks. However, they have no inherent capability to recognize when their predictions are wrong. There have been several efforts in the recent past to detect natural errors but the suggested mechanisms pose additional energy requirements. To address this issue, we propose an ensemble of classifiers at hidden layers to enable energy efficient detection of natural errors. In particular, we append Relevant-features based Auxiliary Cells (RACs) which are class specific binary linear classifiers trained on relevant features. The consensus of RACs is used to detect natural errors. Based on combined confidence of RACs, classification can be terminated early, thereby resulting in energy efficient detection. We demonstrate the effectiveness of our technique on various image classification datasets such as CIFAR-10, CIFAR-100 and Tiny-ImageNet.