CRS-Diff: Controllable Remote Sensing Image Generation with Diffusion Model

TL;DR

CRS-Diff is a novel diffusion-based framework for remote sensing image generation that supports multiple control conditions, enabling more accurate and stable image synthesis for applications like data augmentation.

Contribution

It introduces the first multi-condition controllable RS image generator using diffusion models with a new multi-scale feature fusion control mechanism.

Findings

Outperforms previous methods in quality and control accuracy.

Supports text, metadata, and image conditions simultaneously.

Generates high-quality data for downstream tasks like road extraction.

Abstract

The emergence of generative models has revolutionized the field of remote sensing (RS) image generation. Despite generating high-quality images, existing methods are limited in relying mainly on text control conditions, and thus do not always generate images accurately and stably. In this paper, we propose CRS-Diff, a new RS generative framework specifically tailored for RS image generation, leveraging the inherent advantages of diffusion models while integrating more advanced control mechanisms. Specifically, CRS-Diff can simultaneously support text-condition, metadata-condition, and image-condition control inputs, thus enabling more precise control to refine the generation process. To effectively integrate multiple condition control information, we introduce a new conditional control mechanism to achieve multi-scale feature fusion, thus enhancing the guiding effect of control conditions. To our knowledge, CRS-Diff is the first multiple-condition controllable RS generative model. Experimental results in single-condition and multiple-condition cases have demonstrated the superior ability of our CRS-Diff to generate RS images both quantitatively and qualitatively compared with previous methods. Additionally, our CRS-Diff can serve as a data engine that generates high-quality training data for downstream tasks, e.g., road extraction. The code is available at https://github.com/Sonettoo/CRS-Diff.