Semantic Guided Large Scale Factor Remote Sensing Image Super-resolution with Generative Diffusion Prior

TL;DR

This paper presents a Semantic Guided Diffusion Model for large scale factor remote sensing image super-resolution, leveraging a pre-trained generative prior, semantic cues, and sensor-specific characteristics to produce high-quality, perceptually plausible images.

Contribution

The paper introduces a novel framework that combines generative diffusion models with semantic guidance and sensor-specific modeling for improved remote sensing image super-resolution.

Findings

Outperforms existing methods in qualitative and quantitative metrics

Generates diverse super-resolved images based on imaging characteristics

Enhances downstream vision task performance with high-quality SR images

Abstract

Remote sensing images captured by different platforms exhibit significant disparities in spatial resolution. Large scale factor super-resolution (SR) algorithms are vital for maximizing the utilization of low-resolution (LR) satellite data captured from orbit. However, existing methods confront challenges in recovering SR images with clear textures and correct ground objects. We introduce a novel framework, the Semantic Guided Diffusion Model (SGDM), designed for large scale factor remote sensing image super-resolution. The framework exploits a pre-trained generative model as a prior to generate perceptually plausible SR images. We further enhance the reconstruction by incorporating vector maps, which carry structural and semantic cues. Moreover, pixel-level inconsistencies in paired remote sensing images, stemming from sensor-specific imaging characteristics, may hinder the convergence of the model and diversity in generated results. To address this problem, we propose to extract the sensor-specific imaging characteristics and model the distribution of them, allowing diverse SR images generation based on imaging characteristics provided by reference images or sampled from the imaging characteristic probability distributions. To validate and evaluate our approach, we create the Cross-Modal Super-Resolution Dataset (CMSRD). Qualitative and quantitative experiments on CMSRD showcase the superiority and broad applicability of our method. Experimental results on downstream vision tasks also demonstrate the utilitarian of the generated SR images. The dataset and code will be publicly available at https://github.com/wwangcece/SGDM