DeCEAT: Decoding Carbon Emissions for AI-driven Software Testing

TL;DR

This paper introduces DeCEAT, a framework for evaluating the environmental impact and performance trade-offs of small language models in automated software testing, emphasizing prompt design's role in sustainability.

Contribution

The work presents a systematic evaluation framework for assessing the environmental and performance aspects of small language models in test generation, filling a research gap.

Findings

Different SLMs have unique sustainability profiles.

Prompt design significantly influences environmental and performance outcomes.

Some models prioritize energy efficiency, others stability or accuracy.

Abstract

The increasing use of language models in automated software testing raises concerns about their environmental impact, yet existing sustainability analyses focus almost exclusively on large language models. As a result, the energy and carbon characteristics of small language models (SLMs) during test generation remain largely unexplored. To address this gap, this work introduces the DeCEAT framework, which systematically evaluates the environmental and performance trade-offs of SLMs using the HumanEval benchmark and adaptive prompt variants (based on the Anthropic template). The framework quantifies emission and time-aware behavior under controlled conditions, with CodeCarbon measuring energy consumption and carbon emissions, and unit test coverage assessing the quality of generated tests. Our results show that different SLMs exhibit distinct sustainability strengths: some prioritize lower energy use and faster execution, while others maintain higher stability or accuracy under carbon constraints. These findings demonstrate that sustainability in the generation of SLM-driven tests is multidimensional and strongly shaped by prompt design. This work provides a focused sustainability evaluation framework specifically tailored to automated SLM-based test generation, clarifying how prompt structure and model choice jointly influence environmental and performance outcomes.