ESC-MISR: Enhancing Spatial Correlations for Multi-Image Super-Resolution in Remote Sensing

TL;DR

This paper introduces ESC-MISR, a novel framework that enhances spatial correlations and reduces temporal dependencies among satellite images to improve multi-image super-resolution in remote sensing, outperforming existing methods.

Contribution

The paper proposes a new end-to-end framework with a Multi-Image Spatial Transformer and a shuffle strategy to better exploit spatial-temporal relations in MISR for remote sensing.

Findings

Achieves 0.70dB and 0.76dB cPSNR improvements on PROBA-V dataset.

Effectively emphasizes spatial features and weak temporal correlations.

Outperforms state-of-the-art MISR methods.

Abstract

Multi-Image Super-Resolution (MISR) is a crucial yet challenging research task in the remote sensing community. In this paper, we address the challenging task of Multi-Image Super-Resolution in Remote Sensing (MISR-RS), aiming to generate a High-Resolution (HR) image from multiple Low-Resolution (LR) images obtained by satellites. Recently, the weak temporal correlations among LR images have attracted increasing attention in the MISR-RS task. However, existing MISR methods treat the LR images as sequences with strong temporal correlations, overlooking spatial correlations and imposing temporal dependencies. To address this problem, we propose a novel end-to-end framework named Enhancing Spatial Correlations in MISR (ESC-MISR), which fully exploits the spatial-temporal relations of multiple images for HR image reconstruction. Specifically, we first introduce a novel fusion module named Multi-Image Spatial Transformer (MIST), which emphasizes parts with clearer global spatial features and enhances the spatial correlations between LR images. Besides, we perform a random shuffle strategy for the sequential inputs of LR images to attenuate temporal dependencies and capture weak temporal correlations in the training stage. Compared with the state-of-the-art methods, our ESC-MISR achieves 0.70dB and 0.76dB cPSNR improvements on the two bands of the PROBA-V dataset respectively, demonstrating the superiority of our method.