Comparing energy consumption and accuracy in text classification inference

TL;DR

This paper evaluates the trade-offs between energy consumption and accuracy in text classification inference, highlighting variability across models and hardware, and emphasizing the importance of considering both factors for sustainable AI.

Contribution

It provides a systematic empirical analysis of energy and accuracy trade-offs in NLP inference, revealing that high accuracy does not always imply energy efficiency.

Findings

LLMs consume more energy than traditional models but may have similar or lower accuracy in zero-shot tasks.

Inference energy consumption varies widely from less than a milliwatt-hour to over a kilowatt-hour.

Inference energy correlates strongly with runtime, making execution time a practical proxy for energy use.

Abstract

The increasing deployment of large language models (LLMs) in natural language processing (NLP) tasks raises concerns about energy efficiency and sustainability. While prior research has largely focused on energy consumption during model training, the inference phase has received comparatively less attention. This study systematically evaluates the trade-offs between model accuracy and energy consumption in text classification inference across various model architectures and hardware configurations. Our empirical analysis shows that in some contexts the best-performing model in terms of accuracy can also be energy-efficient. While LLMs tend to consume significantly more energy than traditional machine learning models, they show the same or even lower levels of accuracy in our zero-shot classification setting. We observe substantial variability in inference energy consumption ($<$mWh to $>$kWh), influenced by model type, model size, and hardware specifications. Additionally, we find a strong correlation between inference energy consumption and model runtime, indicating that execution time can serve as a practical proxy for energy usage in settings where direct measurement is not feasible. Our findings demonstrate that energy efficiency and accuracy represent distinct evaluation dimensions that do not necessarily align. We argue that sustainable AI development requires systematic evaluation of both performance and resource efficiency.