Reducing Inference Energy Consumption Using Dual Complementary CNNs

TL;DR

This paper introduces a dual complementary CNN approach combined with a memory component to significantly reduce inference energy consumption on devices like Jetson Nano, while maintaining high accuracy.

Contribution

The paper presents a novel dual CNN collaboration method with a memory component to lower energy use during inference, improving efficiency over traditional single CNN models.

Findings

Achieved up to 85.8% energy reduction on Jetson Nano.

Memory component reduces redundant computations for identical inputs.

Maintains high accuracy despite significant energy savings.

Abstract

Energy efficiency of Convolutional Neural Networks (CNNs) has become an important area of research, with various strategies being developed to minimize the power consumption of these models. Previous efforts, including techniques like model pruning, quantization, and hardware optimization, have made significant strides in this direction. However, there remains a need for more effective on device AI solutions that balance energy efficiency with model performance. In this paper, we propose a novel approach to reduce the energy requirements of inference of CNNs. Our methodology employs two small Complementary CNNs that collaborate with each other by covering each other's "weaknesses" in predictions. If the confidence for a prediction of the first CNN is considered low, the second CNN is invoked with the aim of producing a higher confidence prediction. This dual-CNN setup significantly reduces energy consumption compared to using a single large deep CNN. Additionally, we propose a memory component that retains previous classifications for identical inputs, bypassing the need to re-invoke the CNNs for the same input, further saving energy. Our experiments on a Jetson Nano computer demonstrate an energy reduction of up to 85.8% achieved on modified datasets where each sample was duplicated once. These findings indicate that leveraging a complementary CNN pair along with a memory component effectively reduces inference energy while maintaining high accuracy.