Unveiling Environmental Impacts of Large Language Model Serving: A Functional Unit View

TL;DR

This paper introduces FUEL, a standardized framework using functional units to evaluate and compare the environmental impacts of large language model serving, highlighting trade-offs in optimization strategies.

Contribution

It presents the first FU-based framework for assessing LLM serving environmental impacts, enabling standardized comparisons and insights into optimization trade-offs.

Findings

Optimizing model size, quantization, and hardware affects carbon emissions.

FUEL provides a standardized basis for environmental impact evaluation.

Case studies reveal key trade-offs in sustainable LLM deployment.

Abstract

Large language models (LLMs) offer powerful capabilities but come with significant environmental impact, particularly in carbon emissions. Existing studies benchmark carbon emissions but lack a standardized basis for comparison across different model configurations. To address this, we introduce the concept of functional unit (FU) as a standardized basis and develop FUEL, the first FU-based framework for evaluating LLM serving's environmental impact. Through three case studies, we uncover key insights and trade-offs in reducing carbon emissions by optimizing model size, quantization strategy, and hardware choice, paving the way for more sustainable LLM serving. The code is available at https://github.com/jojacola/FUEL.