Assessing socio-economic climate impacts from text data

TL;DR

This paper reviews and synthesizes best practices for using NLP and large language models to analyze textual data for socio-economic impacts of climate hazards, aiming to improve dataset robustness and comparability.

Contribution

It provides a comprehensive overview of methodological challenges and offers guidelines to enhance the reliability of text-based socio-economic impact assessments.

Findings

Identifies key challenges in impact definition and bias handling.

Synthesizes common practices in text-as-data impact assessment.

Provides recommendations for methodological improvements.

Abstract

Recent advances in natural language processing (NLP) and large language models (LLMs) have enabled the systematic use of large-scale textual data from news, social media, and reports to create datasets with socio-economic impacts of climate hazards such as floods, droughts, storms, and multi-hazard events. As the field of text-as-data for impact assessment expands, so does its methodological complexity. Yet research remains fragmented, with no clear guidelines for defining what constitutes an impact, handling temporal and spatial biases, and selecting appropriate modeling and post-processing strategies. This lack of coherence limits transparency and comparability across studies. Here, we address this gap by synthesising common practices, describing key challenges specific to the use of text-as-data methods for analyzing socio-economic impact data, and proposing recommendations to address them. By providing guidance on best practices, we aim to support the construction of robust text-derived socio-economic impact datasets that can more accurately inform disaster risk management and attribution studies.