GaLeNet: Multimodal Learning for Disaster Prediction, Management and Relief

TL;DR

GaLeNet is a multimodal machine learning framework that combines pre-disaster images, weather data, and hurricane trajectories to improve disaster damage assessment and enable faster emergency response.

Contribution

This work introduces GaLeNet, a novel multimodal framework that effectively fuses diverse data sources for disaster assessment, including pre-disaster images, which prior methods often neglect.

Findings

Multimodal approaches outperform unimodal methods in damage assessment.

GaLeNet effectively fuses images, weather, and trajectory data.

Pre-disaster images enable damage assessment without waiting for post-disaster data.

Abstract

After a natural disaster, such as a hurricane, millions are left in need of emergency assistance. To allocate resources optimally, human planners need to accurately analyze data that can flow in large volumes from several sources. This motivates the development of multimodal machine learning frameworks that can integrate multiple data sources and leverage them efficiently. To date, the research community has mainly focused on unimodal reasoning to provide granular assessments of the damage. Moreover, previous studies mostly rely on post-disaster images, which may take several days to become available. In this work, we propose a multimodal framework (GaLeNet) for assessing the severity of damage by complementing pre-disaster images with weather data and the trajectory of the hurricane. Through extensive experiments on data from two hurricanes, we demonstrate (i) the merits of multimodal approaches compared to unimodal methods, and (ii) the effectiveness of GaLeNet at fusing various modalities. Furthermore, we show that GaLeNet can leverage pre-disaster images in the absence of post-disaster images, preventing substantial delays in decision making.