SEALing the Gap: A Reference Framework for LLM Inference Carbon Estimation via Multi-Benchmark Driven Embodiment

TL;DR

This paper introduces SEAL, a framework for estimating the carbon footprint of large language model inference at the prompt level, aiming to promote sustainability in AI by providing accurate, standardized measurement tools.

Contribution

It proposes guiding principles for a reference framework and presents SEAL, an embodiment leveraging multi-benchmark-driven methods for systematic carbon estimation during LLM inference.

Findings

SEAL shows promising initial validation results.

The framework offers a systematic foundation for sustainability assessment.

It enables prompt-level carbon measurement for LLM inference.

Abstract

Large Language Models are rapidly gaining traction in software engineering, yet their growing carbon footprint raises pressing sustainability concerns. While training emissions are substantial, inference quickly surpasses them due to the sheer volume of prompts processed. This shift underscores the urgent need for accurate, prompt-level carbon measurement during inference to enable informed, sustainability-focused decision-making. To address the limitations of existing approaches, in this paper, we outline the guiding principles for a novel reference framework for LLM inference carbon estimation that can guide the design of future tools and provide a systematic foundation for advancing sustainability research in this domain. We also introduce SEAL, an early embodiment of these principles that leverages a multi-benchmark-driven approach for per-prompt carbon estimation. Its initial validation shows promising results, positioning SEAL as a foundation for standardized sustainability assessment across the LLM ecosystem.