Automated Analysis of Sustainability Reports: Using Large Language Models for the Extraction and Prediction of EU Taxonomy-Compliant KPIs

TL;DR

This study introduces a new dataset and evaluates large language models' ability to automate compliance with the EU Taxonomy by extracting economic activities and predicting KPIs, revealing current limitations and potential uses as assistive tools.

Contribution

The paper provides the first systematic evaluation of LLMs on EU Taxonomy compliance tasks and introduces a public benchmark dataset for future research.

Findings

LLMs moderately identify economic activities with a multi-step framework.

Models fail to accurately predict financial KPIs in zero-shot settings.

Concise metadata outperforms full reports in some tasks.

Abstract

The manual, resource-intensive process of complying with the EU Taxonomy presents a significant challenge for companies. While Large Language Models (LLMs) offer a path to automation, research is hindered by a lack of public benchmark datasets. To address this gap, we introduce a novel, structured dataset from 190 corporate reports, containing ground-truth economic activities and quantitative Key Performance Indicators (KPIs). We use this dataset to conduct the first systematic evaluation of LLMs on the core compliance workflow. Our results reveal a clear performance gap between qualitative and quantitative tasks. LLMs show moderate success in the qualitative task of identifying economic activities, with a multi-step agentic framework modestly enhancing precision. Conversely, the models comprehensively fail at the quantitative task of predicting financial KPIs in a zero-shot setting. We also discover a paradox, where concise metadata often yields superior performance to full, unstructured reports, and find that model confidence scores are poorly calibrated. We conclude that while LLMs are not ready for full automation, they can serve as powerful assistive tools for human experts. Our dataset provides a public benchmark for future research.