EarthSynth: Generating Informative Earth Observation with Diffusion Models

TL;DR

EarthSynth introduces a diffusion-based model for generating diverse, multi-category Earth observation data to improve remote sensing interpretation tasks, addressing data scarcity and enhancing open-vocabulary understanding.

Contribution

It is the first to explore multi-task generation in remote sensing, using a novel training strategy and filtering method to improve data diversity and task performance.

Findings

Significant improvements in scene classification accuracy.

Enhanced object detection performance.

Better semantic segmentation results.

Abstract

Remote sensing image (RSI) interpretation typically faces challenges due to the scarcity of labeled data, which limits the performance of RSI interpretation tasks. To tackle this challenge, we propose EarthSynth, a diffusion-based generative foundation model that enables synthesizing multi-category, cross-satellite labeled Earth observation for downstream RSI interpretation tasks. To the best of our knowledge, EarthSynth is the first to explore multi-task generation for remote sensing, tackling the challenge of limited generalization in task-oriented synthesis for RSI interpretation. EarthSynth, trained on the EarthSynth-180K dataset, employs the Counterfactual Composition training strategy with a three-dimensional batch-sample selection mechanism to improve training data diversity and enhance category control. Furthermore, a rule-based method of R-Filter is proposed to filter more informative synthetic data for downstream tasks. We evaluate our EarthSynth on scene classification, object detection, and semantic segmentation in open-world scenarios. There are significant improvements in open-vocabulary understanding tasks, offering a practical solution for advancing RSI interpretation.