A Large-Language-Model Framework for Automated Humanitarian Situation Reporting

TL;DR

This paper introduces an automated framework utilizing large language models to generate structured, evidence-based humanitarian situation reports from diverse documents, improving speed, accuracy, and interpretability.

Contribution

The study presents a novel LLM-based system that automates humanitarian reporting with integrated question generation, citation, multi-level summarization, and internal evaluation metrics.

Findings

Questions achieved over 84% relevance and importance.

Answers had over 86% relevance with citation precision and recall above 76%.

Human-LLM evaluation agreement exceeded 0.80 F1 score.

Abstract

Timely and accurate situational reports are essential for humanitarian decision-making, yet current workflows remain largely manual, resource intensive, and inconsistent. We present a fully automated framework that uses large language models (LLMs) to transform heterogeneous humanitarian documents into structured and evidence-grounded reports. The system integrates semantic text clustering, automatic question generation, retrieval augmented answer extraction with citations, multi-level summarization, and executive summary generation, supported by internal evaluation metrics that emulate expert reasoning. We evaluated the framework across 13 humanitarian events, including natural disasters and conflicts, using more than 1,100 documents from verified sources such as ReliefWeb. The generated questions achieved 84.7 percent relevance, 84.0 percent importance, and 76.4 percent urgency. The extracted answers reached 86.3 percent relevance, with citation precision and recall both exceeding 76 percent. Agreement between human and LLM based evaluations surpassed an F1 score of 0.80. Comparative analysis shows that the proposed framework produces reports that are more structured, interpretable, and actionable than existing baselines. By combining LLM reasoning with transparent citation linking and multi-level evaluation, this study demonstrates that generative AI can autonomously produce accurate, verifiable, and operationally useful humanitarian situation reports.