Learn to Code Sustainably: An Empirical Study on LLM-based Green Code Generation

TL;DR

This paper empirically evaluates the sustainability of AI-generated code using new metrics, comparing it to human code to understand AI's role in promoting green software development.

Contribution

It introduces the concept of 'green capacity' for AI models and assesses their sustainability awareness in code generation tasks.

Findings

AI models show varying green capacity depending on problem difficulty

AI-generated code's sustainability metrics are comparable to human code in some cases

The study highlights current limitations and potential of AI in sustainable coding practices

Abstract

The increasing use of information technology has led to a significant share of energy consumption and carbon emissions from data centers. These contributions are expected to rise with the growing demand for big data analytics, increasing digitization, and the development of large artificial intelligence (AI) models. The need to address the environmental impact of software development has led to increased interest in green (sustainable) coding and claims that the use of AI models can lead to energy efficiency gains. Here, we provide an empirical study on green code and an overview of green coding practices, as well as metrics used to quantify the sustainability awareness of AI models. In this framework, we evaluate the sustainability of auto-generated code. The auto-generate codes considered in this study are produced by generative commercial AI language models, GitHub Copilot, OpenAI ChatGPT-3, and Amazon CodeWhisperer. Within our methodology, in order to quantify the sustainability awareness of these AI models, we propose a definition of the code's "green capacity", based on certain sustainability metrics. We compare the performance and green capacity of human-generated code and code generated by the three AI language models in response to easy-to-hard problem statements. Our findings shed light on the current capacity of AI models to contribute to sustainable software development.