Brevity is the soul of sustainability: Characterizing LLM response lengths

TL;DR

This paper benchmarks LLM response lengths, identifies excessive verbosity, and demonstrates prompt strategies that significantly reduce response length and energy consumption without compromising quality.

Contribution

It introduces a comprehensive benchmarking of LLM response lengths and proposes prompt-engineering methods to optimize energy efficiency by reducing response verbosity.

Findings

LLMs often produce longer responses than necessary.

Prompt strategies can reduce response length by 25-60%.

Energy savings are achieved without loss of response quality.

Abstract

A significant portion of the energy consumed by Large Language Models (LLMs) arises from their inference processes; hence developing energy-efficient methods for inference is crucial. While several techniques exist for inference optimization, output compression remains relatively unexplored, with only a few preliminary efforts addressing this aspect. In this work, we first benchmark 12 decoder-only LLMs across 5 datasets, revealing that these models often produce responses that are substantially longer than necessary. We then conduct a comprehensive quality assessment of LLM responses, formally defining six information categories present in LLM responses. We show that LLMs often tend to include redundant or additional information besides the minimal answer. To address this issue of long responses by LLMs, we explore several simple and intuitive prompt-engineering strategies. Empirical evaluation shows that appropriate prompts targeting length reduction and controlling information content can achieve significant energy optimization between 25-60\% by reducing the response length while preserving the quality of LLM responses.