GreenLLM: Disaggregating Large Language Model Serving on Heterogeneous GPUs for Lower Carbon Emissions

TL;DR

GreenLLM is a framework that reduces the environmental impact of large language model serving by intelligently reusing older GPUs, achieving significant carbon savings while maintaining performance.

Contribution

It introduces GreenLLM, a novel SLO-aware disaggregation framework that leverages older GPUs to lower carbon emissions in LLM serving.

Findings

Reduces carbon emissions by up to 40.6% compared to using only new GPUs.

Maintains latency SLOs for over 90% of requests across various conditions.

Provides theoretical analysis linking disaggregation to carbon intensity and GPU lifetime.

Abstract

LLMs have been widely adopted across many real-world applications. However, their widespread use comes with significant environmental costs due to their high computational intensity and resource demands. Specifically, this has driven the development of new generations of high-performing GPUs, exacerbating the problem of electronic waste and accelerating the premature disposal of devices. To address this problem, this paper focuses on reducing the carbon emissions of LLM serving by reusing older, low-performing GPUs. We present GreenLLM, an SLO-aware LLM serving framework designed to minimize carbon emissions by reusing older GPUs. GreenLLM builds on two identified use cases that disaggregate specific computations onto older GPUs, reducing carbon emissions while meeting performance goals. To deepen our understanding of the potential carbon savings from disaggregation, we also provide a theoretical analysis of its relationship with carbon intensity and GPU lifetime. Our evaluations show that GreenLLM reduces carbon emissions by up to 40.6% compared to running standard LLM serving on new GPU only, meeting latency SLOs for over 90% of requests across various applications, latency requirements, carbon intensities, and GPU lifetimes.