AI-CARE: Carbon-Aware Reporting Evaluation Metric for AI Models

TL;DR

AI-CARE introduces a new evaluation metric for ML models that considers energy consumption and carbon emissions, promoting environmentally responsible AI development.

Contribution

It proposes a novel carbon-aware benchmarking tool and tradeoff curve, enabling multi-objective evaluation of accuracy and environmental impact.

Findings

Carbon-aware benchmarking alters model rankings.

The tool visualizes performance vs. carbon tradeoffs.

Encourages development of energy-efficient models.

Abstract

As machine learning (ML) continues its rapid expansion, the environmental cost of model training and inference has become a critical societal concern. Existing benchmarks overwhelmingly focus on standard performance metrics such as accuracy, BLEU, or mAP, while largely ignoring energy consumption and carbon emissions. This single-objective evaluation paradigm is increasingly misaligned with the practical requirements of large-scale deployment, particularly in energy-constrained environments such as mobile devices, developing regions, and climate-aware enterprises. In this paper, we propose AI-CARE, an evaluation tool for reporting energy consumption, and carbon emissions of ML models. In addition, we introduce the carbon-performance tradeoff curve, an interpretable tool that visualizes the Pareto frontier between performance and carbon cost. We demonstrate, through theoretical analysis and empirical validation on representative ML workloads, that carbon-aware benchmarking changes the relative ranking of models and encourages architectures that are simultaneously accurate and environmentally responsible. Our proposal aims to shift the research community toward transparent, multi-objective evaluation and align ML progress with global sustainability goals. The tool and documentation are available at https://github.com/USD-AI-ResearchLab/ai-care.