RAPTOR-AI for Disaster OODA Loop: Hierarchical Multimodal RAG with Experience-Driven Agentic Decision-Making

TL;DR

RAPTOR-AI is a hierarchical multimodal RAG framework that enhances disaster response decision-making by integrating diverse data sources, adaptive retrieval strategies, and experiential learning within the OODA loop for improved situational awareness and task accuracy.

Contribution

The paper introduces a novel hierarchical, experience-driven multimodal RAG system tailored for disaster management, advancing adaptive reasoning and knowledge integration in extreme emergency scenarios.

Findings

23% improvement in retrieval precision

31% better situational grounding

27% enhanced task decomposition accuracy

Abstract

Humanitarian Assistance and Disaster Relief (HADR) operations demand rapid synthesis of multimodal information for time-critical decision-making under extreme uncertainty. Traditional information systems struggle with the fragmented, multimodal nature of disaster data and lack adaptive reasoning capabilities essential for dynamic emergency contexts. This work introduces RAPTOR-AI, an agentic multimodal Retrieval-Augmented Generation (RAG) framework that advances beyond conventional static knowledge bases by implementing dynamic, experience-driven decision support for disaster response. The system addresses HADR requirements across initial rescue, recovery, and reconstruction phases through three key innovations: hierarchical multimodal knowledge construction from diverse sources (textual reports, aerial imagery, historical documentation), entropy-aware agentic control that dynamically selects optimal retrieval strategies based on situational context, and experiential knowledge integration using LoRA adaptation for both expert and non-expert responders. The framework constructs hierarchical knowledge trees from 46 tsunami-related PDFs (2,378 pages) using BLIP-based image understanding, ColVBERT embeddings, and long-context summarization within the OODA loop (Observe, Orient, Decide, Act) tactical framework. Experiments demonstrate significant improvements over existing approaches: 23\% improvement in retrieval precision, 31\% better situational grounding, and 27\% enhanced task decomposition accuracy, with efficient scaling up to 3,000 document chunks.