CHATREPORT: Democratizing Sustainability Disclosure Analysis through LLM-based Tools

TL;DR

ChatReport introduces an LLM-based system that automates sustainability report analysis, making it more accessible and transparent by addressing hallucination issues and involving domain experts, with publicly available datasets and analyses.

Contribution

The paper presents a novel LLM-based tool for analyzing sustainability reports, incorporating traceability and expert involvement to improve accuracy and transparency.

Findings

Analyzed 1015 reports with the system

Made datasets and analyses publicly available

Reduced hallucination effects in LLM outputs

Abstract

In the face of climate change, are companies really taking substantial steps toward more sustainable operations? A comprehensive answer lies in the dense, information-rich landscape of corporate sustainability reports. However, the sheer volume and complexity of these reports make human analysis very costly. Therefore, only a few entities worldwide have the resources to analyze these reports at scale, which leads to a lack of transparency in sustainability reporting. Empowering stakeholders with LLM-based automatic analysis tools can be a promising way to democratize sustainability report analysis. However, developing such tools is challenging due to (1) the hallucination of LLMs and (2) the inefficiency of bringing domain experts into the AI development loop. In this paper, we ChatReport, a novel LLM-based system to automate the analysis of corporate sustainability reports, addressing existing challenges by (1) making the answers traceable to reduce the harm of hallucination and (2) actively involving domain experts in the development loop. We make our methodology, annotated datasets, and generated analyses of 1015 reports publicly available.