DynamicVL: Benchmarking Multimodal Large Language Models for Dynamic City Understanding

TL;DR

This paper introduces DVL-Suite, a comprehensive framework and benchmark for evaluating and enhancing multimodal large language models in long-term urban analysis using multi-temporal remote sensing imagery.

Contribution

It provides a large-scale dataset, diverse urban understanding tasks, and a baseline model to improve long-term city dynamics analysis with multimodal models.

Findings

State-of-the-art models show limitations in long-term temporal understanding.

The DVL-Instruct dataset improves model capabilities in multi-temporal analysis.

DVLChat enables integrated question-answering and segmentation for urban insights.

Abstract

Multimodal large language models (MLLMs) have demonstrated remarkable capabilities in visual understanding, but their application to long-term Earth observation analysis remains limited, primarily focusing on single-temporal or bi-temporal imagery. To address this gap, we introduce DVL-Suite, a comprehensive framework for analyzing long-term urban dynamics through remote sensing imagery. Our suite comprises 14,871 high-resolution (1.0m) multi-temporal images spanning 42 major cities in the U.S. from 2005 to 2023, organized into two components: DVL-Bench and DVL-Instruct. The DVL-Bench includes six urban understanding tasks, from fundamental change detection (pixel-level) to quantitative analyses (regional-level) and comprehensive urban narratives (scene-level), capturing diverse urban dynamics including expansion/transformation patterns, disaster assessment, and environmental challenges. We evaluate 18 state-of-the-art MLLMs and reveal their limitations in long-term temporal understanding and quantitative analysis. These challenges motivate the creation of DVL-Instruct, a specialized instruction-tuning dataset designed to enhance models' capabilities in multi-temporal Earth observation. Building upon this dataset, we develop DVLChat, a baseline model capable of both image-level question-answering and pixel-level segmentation, facilitating a comprehensive understanding of city dynamics through language interactions.