Power Hungry Processing: Watts Driving the Cost of AI Deployment?

TL;DR

This paper systematically compares the energy and carbon costs of task-specific versus multi-purpose generative AI models, revealing that the latter are significantly more expensive to deploy across various tasks.

Contribution

It provides the first comprehensive analysis of inference costs for different ML system categories, highlighting the environmental impact of general-purpose AI models.

Findings

Multi-purpose generative models are orders of magnitude more energy-intensive than task-specific models.

Deployment costs vary significantly even when models have similar parameter counts.

The study offers an interactive demo for further exploration of AI deployment costs.

Abstract

Recent years have seen a surge in the popularity of commercial AI products based on generative, multi-purpose AI systems promising a unified approach to building machine learning (ML) models into technology. However, this ambition of ``generality'' comes at a steep cost to the environment, given the amount of energy these systems require and the amount of carbon that they emit. In this work, we propose the first systematic comparison of the ongoing inference cost of various categories of ML systems, covering both task-specific (i.e. finetuned models that carry out a single task) and `general-purpose' models, (i.e. those trained for multiple tasks). We measure deployment cost as the amount of energy and carbon required to perform 1,000 inferences on representative benchmark dataset using these models. We find that multi-purpose, generative architectures are orders of magnitude more expensive than task-specific systems for a variety of tasks, even when controlling for the number of model parameters. We conclude with a discussion around the current trend of deploying multi-purpose generative ML systems, and caution that their utility should be more intentionally weighed against increased costs in terms of energy and emissions. All the data from our study can be accessed via an interactive demo to carry out further exploration and analysis.