CO2-Meter: A Comprehensive Carbon Footprint Estimator for LLMs on Edge Devices

TL;DR

CO2-Meter is a comprehensive framework that accurately estimates both operational and embodied carbon footprints of LLM inference on edge devices, aiding sustainable AI deployment.

Contribution

It introduces new models and datasets for peripheral energy, a GNN predictor for LLM energy, and a unit-level embodied carbon model, advancing prior estimators significantly.

Findings

CO2-Meter outperforms previous methods in accuracy.

It effectively identifies carbon hotspots in edge LLM deployment.

The framework guides sustainable design choices for edge AI.

Abstract

LLMs have transformed NLP, yet deploying them on edge devices poses great carbon challenges. Prior estimators remain incomplete, neglecting peripheral energy use, distinct prefill/decode behaviors, and SoC design complexity. This paper presents CO2-Meter, a unified framework for estimating operational and embodied carbon in LLM edge inference. Contributions include: (1) equation-based peripheral energy models and datasets; (2) a GNN-based predictor with phase-specific LLM energy data; (3) a unit-level embodied carbon model for SoC bottleneck analysis; and (4) validation showing superior accuracy over prior methods. Case studies show CO2-Meter's effectiveness in identifying carbon hotspots and guiding sustainable LLM design on edge platforms. Source code: https://github.com/fuzhenxiao/CO2-Meter