Green AI in Action: Strategic Model Selection for Ensembles in Production

TL;DR

This paper introduces energy-efficient model selection strategies for AI ensembles that maintain high accuracy while significantly reducing energy consumption in production systems.

Contribution

It proposes Static and Dynamic model selection strategies that optimize ensemble performance and energy efficiency, validated through real-world deployment.

Findings

Static strategy reduces energy use to 62% with improved F1 score.

Dynamic strategy achieves 76% energy use with better accuracy.

Energy consumption decreased to 14-57% with minimal accuracy loss.

Abstract

Integrating Artificial Intelligence (AI) into software systems has significantly enhanced their capabilities while escalating energy demands. Ensemble learning, combining predictions from multiple models to form a single prediction, intensifies this problem due to cumulative energy consumption. This paper presents a novel approach to model selection that addresses the challenge of balancing the accuracy of AI models with their energy consumption in a live AI ensemble system. We explore how reducing the number of models or improving the efficiency of model usage within an ensemble during inference can reduce energy demands without substantially sacrificing accuracy. This study introduces and evaluates two model selection strategies, Static and Dynamic, for optimizing ensemble learning systems performance while minimizing energy usage. Our results demonstrate that the Static strategy improves the F1 score beyond the baseline, reducing average energy usage from 100% from the full ensemble to 62%. The Dynamic strategy further enhances F1 scores, using on average 76% compared to 100% of the full ensemble. Moreover, we propose an approach that balances accuracy with resource consumption, significantly reducing energy usage without substantially impacting accuracy. This method decreased the average energy usage of the Static strategy from approximately 62% to 14%, and for the Dynamic strategy, from around 76% to 57%. Our field study of Green AI using an operational AI system developed by a large professional services provider shows the practical applicability of adopting energy-conscious model selection strategies in live production environments.