Remote Diffusion

TL;DR

This paper investigates adapting Stable Diffusion for remote sensing image generation, highlighting challenges in data quality and model performance, and providing baseline results with a synthetic dataset and classification accuracy.

Contribution

It introduces a method for fine-tuning Stable Diffusion on remote sensing data and creates a synthetic dataset for land classification tasks, addressing domain-specific challenges.

Findings

High FID scores indicate poor image realism

Synthetic dataset enabled baseline classification accuracy of 49.48%

Challenges include insufficient pretraining data and computational resources

Abstract

I explored adapting Stable Diffusion v1.5 for generating domain-specific satellite and aerial images in remote sensing. Recognizing the limitations of existing models like Midjourney and Stable Diffusion, trained primarily on natural RGB images and lacking context for remote sensing, I used the RSICD dataset to train a Stable Diffusion model with a loss of 0.2. I incorporated descriptive captions from the dataset for text-conditioning. Additionally, I created a synthetic dataset for a Land Use Land Classification (LULC) task, employing prompting techniques with RAG and ChatGPT and fine-tuning a specialized remote sensing LLM. However, I faced challenges with prompt quality and model performance. I trained a classification model (ResNet18) on the synthetic dataset achieving 49.48% test accuracy in TorchGeo to create a baseline. Quantitative evaluation through FID scores and qualitative feedback from domain experts assessed the realism and quality of the generated images and dataset. Despite extensive fine-tuning and dataset iterations, results indicated subpar image quality and realism, as indicated by high FID scores and domain-expert evaluation. These findings call attention to the potential of diffusion models in remote sensing while highlighting significant challenges related to insufficient pretraining data and computational resources.