Trends in Energy Estimates for Computing in AI/Machine Learning Accelerators, Supercomputers, and Compute-Intensive Applications

TL;DR

This paper analyzes energy consumption trends in AI and computing systems, revealing slowing efficiency gains, discrepancies between bit-level and system-level efficiency, and the high energy cost of AI-ML applications, with implications for sustainable computing.

Contribution

It provides a comprehensive analysis of energy efficiency trends in AI accelerators and supercomputers, highlighting key challenges and future directions for sustainable high-performance computing.

Findings

Energy efficiency gains are slowing down due to geometrical scaling.

System-level energy efficiency does not match bit-level improvements.

AI-ML applications are highly energy-intensive, offsetting efficiency gains.

Abstract

We examine the computational energy requirements of different systems driven by the geometrical scaling law, and increasing use of Artificial Intelligence or Machine Learning (AI-ML) over the last decade. With more scientific and technology applications based on data-driven discovery, machine learning methods, especially deep neural networks, have become widely used. In order to enable such applications, both hardware accelerators and advanced AI-ML methods have led to the introduction of new architectures, system designs, algorithms, and software. Our analysis of energy trends indicates three important observations: 1) Energy efficiency due to geometrical scaling is slowing down; 2) The energy efficiency at the bit-level does not translate into efficiency at the instruction-level, or at the system-level for a variety of systems, especially for large-scale AI-ML accelerators or supercomputers; 3) At the application level, general-purpose AI-ML methods can be computationally energy intensive, off-setting the gains in energy from geometrical scaling and special purpose accelerators. Further, our analysis provides specific pointers for integrating energy efficiency with performance analysis for enabling high-performance and sustainable computing in the future.