Swin2-MoSE: A New Single Image Super-Resolution Model for Remote Sensing

TL;DR

Swin2-MoSE is a novel single-image super-resolution model for remote sensing that enhances performance by integrating an advanced Mixture-of-Experts mechanism, innovative positional encoding analysis, and combined loss functions, outperforming state-of-the-art methods.

Contribution

The paper introduces Swin2-MoSE, an improved super-resolution model with a new MoE-SM module, per-example expert work splitting, and combined NCC-SSIM loss, advancing remote sensing image enhancement.

Findings

Outperforms existing Swin-based models by up to 0.958 dB PSNR.

Achieves superior results on 2x, 3x, and 4x upscaling tasks.

Demonstrates effectiveness in semantic segmentation applications.

Abstract

Due to the limitations of current optical and sensor technologies and the high cost of updating them, the spectral and spatial resolution of satellites may not always meet desired requirements. For these reasons, Remote-Sensing Single-Image Super-Resolution (RS-SISR) techniques have gained significant interest. In this paper, we propose Swin2-MoSE model, an enhanced version of Swin2SR. Our model introduces MoE-SM, an enhanced Mixture-of-Experts (MoE) to replace the Feed-Forward inside all Transformer block. MoE-SM is designed with Smart-Merger, and new layer for merging the output of individual experts, and with a new way to split the work between experts, defining a new per-example strategy instead of the commonly used per-token one. Furthermore, we analyze how positional encodings interact with each other, demonstrating that per-channel bias and per-head bias can positively cooperate. Finally, we propose to use a combination of Normalized-Cross-Correlation (NCC) and Structural Similarity Index Measure (SSIM) losses, to avoid typical MSE loss limitations. Experimental results demonstrate that Swin2-MoSE outperforms any Swin derived models by up to 0.377 - 0.958 dB (PSNR) on task of 2x, 3x and 4x resolution-upscaling (Sen2Venus and OLI2MSI datasets). It also outperforms SOTA models by a good margin, proving to be competitive and with excellent potential, especially for complex tasks. Additionally, an analysis of computational costs is also performed. Finally, we show the efficacy of Swin2-MoSE, applying it to a semantic segmentation task (SeasoNet dataset). Code and pretrained are available on https://github.com/IMPLabUniPr/swin2-mose/tree/official_code