CienaLLM: Generative Climate-Impact Extraction from News Articles with Autoregressive LLMs

TL;DR

CienaLLM is a flexible, schema-guided framework utilizing open-weight LLMs for zero-shot extraction of climate impact information from news, demonstrating strong performance and adaptability across languages and hazards.

Contribution

The paper introduces CienaLLM, a modular, schema-driven approach that leverages open-weight LLMs for zero-shot climate impact extraction, with comprehensive analysis of model and prompting strategies.

Findings

Larger models provide more stable and accurate extraction.

Quantization offers efficiency gains with modest accuracy loss.

Prompt engineering effects vary by model.

Abstract

Understanding and monitoring the socio-economic impacts of climate hazards requires extracting structured information from heterogeneous news articles on a large scale. To that end, we have developed CienaLLM, a modular framework based on schema-guided Generative Information Extraction. CienaLLM uses open-weight Large Language Models for zero-shot information extraction from news articles, and supports configurable prompts and output schemas, multi-step pipelines, and cloud or on-premise inference. To systematically assess how the choice of LLM family, size, precision regime, and prompting strategy affect performance, we run a large factorial study in models, precisions, and prompt engineering techniques. An additional response parsing step nearly eliminates format errors while preserving accuracy; larger models deliver the strongest and most stable performance, while quantization offers substantial efficiency gains with modest accuracy trade-offs; and prompt strategies show heterogeneous, model-specific effects. CienaLLM matches or outperforms the supervised baseline in accuracy for extracting drought impacts from Spanish news, although at a higher inference cost. While evaluated in droughts, the schema-driven and model-agnostic design is suitable for adapting to related information extraction tasks (e.g., other hazards, sectors, or languages) by editing prompts and schemas rather than retraining. We release code, configurations, and schemas to support reproducible use.