A Sinkhorn Regularized Adversarial Network for Image Guided DEM Super-resolution using Frequency Selective Hybrid Graph Transformer

TL;DR

This paper introduces a novel hybrid transformer model with Sinkhorn regularization for high-resolution DEM super-resolution guided by multi-spectral satellite imagery, demonstrating superior performance over baseline methods.

Contribution

The paper proposes a new hybrid transformer architecture with Sinkhorn regularized adversarial training for improved DEM super-resolution, combining frequency selective graph attention and discriminator spatial maps.

Findings

Superior qualitative and quantitative results on 4 DEM datasets

Sharper details and minimal errors compared to baselines

Theoretical and empirical validation of Sinkhorn GAN advantages

Abstract

Digital Elevation Model (DEM) is an essential aspect in the remote sensing (RS) domain to analyze various applications related to surface elevations. Here, we address the generation of high-resolution (HR) DEMs using HR multi-spectral (MX) satellite imagery as a guide by introducing a novel hybrid transformer model consisting of Densely connected Multi-Residual Block (DMRB) and multi-headed Frequency Selective Graph Attention (M-FSGA). To promptly regulate this process, we utilize the notion of discriminator spatial maps as the conditional attention to the MX guide. Further, we present a novel adversarial objective related to optimizing Sinkhorn distance with classical GAN. In this regard, we provide both theoretical and empirical substantiation of better performance in terms of vanishing gradient issues and numerical convergence. Based on our experiments on 4 different DEM datasets, we demonstrate both qualitative and quantitative comparisons with available baseline methods and show that the performance of our proposed model is superior to others with sharper details and minimal errors.