Addressing the sustainable AI trilemma: a case study on LLM agents and RAG

TL;DR

This paper introduces the Sustainable AI Trilemma, analyzing the energy efficiency of LLM agents and RAG, revealing significant inefficiencies and trade-offs between capability, equity, and sustainability.

Contribution

It defines the Sustainable AI Trilemma, develops metrics for energy-performance trade-offs, and provides empirical insights into energy inefficiencies in current LLM memory architectures.

Findings

Current memory-augmented frameworks are energy inefficient

Resource-constrained environments face higher efficiency penalties

Challenging the LLM-centric paradigm for sustainable AI development

Abstract

Large language models (LLMs) have demonstrated significant capabilities, but their widespread deployment and more advanced applications raise critical sustainability challenges, particularly in inference energy consumption. We propose the concept of the Sustainable AI Trilemma, highlighting the tensions between AI capability, digital equity, and environmental sustainability. Through a systematic case study of LLM agents and retrieval-augmented generation (RAG), we analyze the energy costs embedded in memory module designs and introduce novel metrics to quantify the trade-offs between energy consumption and system performance. Our experimental results reveal significant energy inefficiencies in current memory-augmented frameworks and demonstrate that resource-constrained environments face disproportionate efficiency penalties. Our findings challenge the prevailing LLM-centric paradigm in agent design and provide practical insights for developing more sustainable AI systems.