MONITRS: Multimodal Observations of Natural Incidents Through Remote Sensing

TL;DR

MONITRS is a comprehensive multimodal dataset combining satellite imagery, natural language annotations, and geotagged data for over 10,000 FEMA disaster events, enabling improved machine learning models for disaster monitoring and response.

Contribution

The paper introduces MONITRS, a novel multimodal dataset with rich annotations and demonstrates its effectiveness in enhancing disaster monitoring models.

Findings

Fine-tuning MLLMs on MONITRS improves disaster detection accuracy.

MONITRS sets a new benchmark for machine learning in disaster response.

The dataset enables tracking disaster progression with temporal and natural language data.

Abstract

Natural disasters cause devastating damage to communities and infrastructure every year. Effective disaster response is hampered by the difficulty of accessing affected areas during and after events. Remote sensing has allowed us to monitor natural disasters in a remote way. More recently there have been advances in computer vision and deep learning that help automate satellite imagery analysis, However, they remain limited by their narrow focus on specific disaster types, reliance on manual expert interpretation, and lack of datasets with sufficient temporal granularity or natural language annotations for tracking disaster progression. We present MONITRS, a novel multimodal dataset of more than 10,000 FEMA disaster events with temporal satellite imagery and natural language annotations from news articles, accompanied by geotagged locations, and question-answer pairs. We demonstrate that fine-tuning existing MLLMs on our dataset yields significant performance improvements for disaster monitoring tasks, establishing a new benchmark for machine learning-assisted disaster response systems. Code can be found at: https://github.com/ShreelekhaR/MONITRS