Judging It, Washing It: Scoring and Greenwashing Corporate Climate Disclosures using Large Language Models

TL;DR

This paper explores using large language models to evaluate corporate climate disclosures and detect greenwashing, demonstrating effective scoring methods and robustness against deceptive responses.

Contribution

It introduces LLM-based scoring systems for climate reports and evaluates their effectiveness and robustness against greenwashing tactics.

Findings

LLM-based scoring can distinguish high and low performers.

Pairwise comparison method is more robust against greenwashing.

LLMs can both evaluate and identify greenwashing in climate disclosures.

Abstract

We study the use of large language models (LLMs) to both evaluate and greenwash corporate climate disclosures. First, we investigate the use of the LLM-as-a-Judge (LLMJ) methodology for scoring company-submitted reports on emissions reduction targets and progress. Second, we probe the behavior of an LLM when it is prompted to greenwash a response subject to accuracy and length constraints. Finally, we test the robustness of the LLMJ methodology against responses that may be greenwashed using an LLM. We find that two LLMJ scoring systems, numerical rating and pairwise comparison, are effective in distinguishing high-performing companies from others, with the pairwise comparison system showing greater robustness against LLM-greenwashed responses.