Compute and Energy Consumption Trends in Deep Learning Inference

TL;DR

This paper investigates whether the rapid growth in deep learning model complexity has led to exponential increases in energy consumption during inference, finding a much softer growth than expected with implications for AI's future sustainability.

Contribution

It provides a comprehensive analysis of inference energy costs over time, considering hardware improvements and model maturity, revealing a less steep increase in energy consumption than previously believed.

Findings

Energy consumption growth is much softer than anticipated.

Hardware efficiency improvements mitigate energy cost increases.

Future AI penetration could still significantly impact energy use.

Abstract

The progress of some AI paradigms such as deep learning is said to be linked to an exponential growth in the number of parameters. There are many studies corroborating these trends, but does this translate into an exponential increase in energy consumption? In order to answer this question we focus on inference costs rather than training costs, as the former account for most of the computing effort, solely because of the multiplicative factors. Also, apart from algorithmic innovations, we account for more specific and powerful hardware (leading to higher FLOPS) that is usually accompanied with important energy efficiency optimisations. We also move the focus from the first implementation of a breakthrough paper towards the consolidated version of the techniques one or two year later. Under this distinctive and comprehensive perspective, we study relevant models in the areas of computer vision and natural language processing: for a sustained increase in performance we see a much softer growth in energy consumption than previously anticipated. The only caveat is, yet again, the multiplicative factor, as future AI increases penetration and becomes more pervasive.