Exploring Energy-Accuracy Tradeoffs in AI Hardware

TL;DR

This paper investigates how AI systems, especially on edge devices, can balance energy consumption and accuracy by adjusting resource use and exploiting high-confidence predictions within neural networks.

Contribution

It introduces a simple cost function for AI decision-making that accounts for energy and accuracy tradeoffs, and demonstrates how to reduce energy use by utilizing early high-confidence predictions.

Findings

Minimizing cost involves using fewer neural network resources.

Energy savings are achievable by leveraging high-confidence early predictions.

The proposed approach effectively balances energy and accuracy in edge AI applications.

Abstract

Artificial intelligence (AI) is playing an increasingly significant role in our everyday lives. This trend is expected to continue, especially with recent pushes to move more AI to the edge. However, one of the biggest challenges associated with AI on edge devices (mobile phones, unmanned vehicles, sensors, etc.) is their associated size, weight, and power constraints. In this work, we consider the scenario where an AI system may need to operate at less-than-maximum accuracy in order to meet application-dependent energy requirements. We propose a simple function that divides the cost of using an AI system into the cost of the decision making process and the cost of decision execution. For simple binary decision problems with convolutional neural networks, it is shown that minimizing the cost corresponds to using fewer than the maximum number of resources (e.g. convolutional neural network layers and filters). Finally, it is shown that the cost associated with energy can be significantly reduced by leveraging high-confidence predictions made in lower-level layers of the network.