Optimising for Energy Efficiency and Performance in Machine Learning

TL;DR

This paper introduces ECOpt, a hyperparameter tuning tool that balances energy efficiency and performance in machine learning, providing insights into energy scaling laws and practical benefits for sustainable AI development.

Contribution

The paper presents ECOpt, a novel hyperparameter tuner that optimizes for energy efficiency alongside model performance, addressing measurement gaps and environmental concerns.

Findings

Parameter counts are unreliable proxies for energy consumption.

Energy efficiency of Transformer models is consistent across hardware.

ECOpt can improve CIFAR-10 models considering both accuracy and energy use.

Abstract

The ubiquity of machine learning (ML) and the demand for ever-larger models bring an increase in energy consumption and environmental impact. However, little is known about the energy scaling laws in ML, and existing research focuses on training cost -- ignoring the larger cost of inference. Furthermore, tools for measuring the energy consumption of ML do not provide actionable feedback. To address these gaps, we developed Energy Consumption Optimiser (ECOpt): a hyperparameter tuner that optimises for energy efficiency and model performance. ECOpt quantifies the trade-off between these metrics as an interpretable Pareto frontier. This enables ML practitioners to make informed decisions about energy cost and environmental impact, while maximising the benefit of their models and complying with new regulations. Using ECOpt, we show that parameter and floating-point operation counts can be unreliable proxies for energy consumption, and observe that the energy efficiency of Transformer models for text generation is relatively consistent across hardware. These findings motivate measuring and publishing the energy metrics of ML models. We further show that ECOpt can have a net positive environmental impact and use it to uncover seven models for CIFAR-10 that improve upon the state of the art, when considering accuracy and energy efficiency together.