Remote sensing image fusion based on Bayesian GAN

TL;DR

This paper introduces a Bayesian GAN model utilizing stochastic gradient Langevin dynamics for remote sensing image fusion, specifically pan-sharpening, achieving superior detail preservation and fusion quality over existing methods.

Contribution

The paper proposes a novel Bayesian GAN framework with PGSLD for improved remote sensing image fusion, exploring generator posterior distribution for better results.

Findings

Effective fusion of PAN and MS images demonstrated on multiple datasets.

Model outperforms state-of-the-art methods in subjective and objective metrics.

Bayesian inference enhances generator parameter exploration for better image reconstruction.

Abstract

Remote sensing image fusion technology (pan-sharpening) is an important means to improve the information capacity of remote sensing images. Inspired by the efficient arameter space posteriori sampling of Bayesian neural networks, in this paper we propose a Bayesian Generative Adversarial Network based on Preconditioned Stochastic Gradient Langevin Dynamics (PGSLD-BGAN) to improve pan-sharpening tasks. Unlike many traditional generative models that consider only one optimal solution (might be locally optimal), the proposed PGSLD-BGAN performs Bayesian inference on the network parameters, and explore the generator posteriori distribution, which assists selecting the appropriate generator parameters. First, we build a two-stream generator network with PAN and MS images as input, which consists of three parts: feature extraction, feature fusion and image reconstruction. Then, we leverage Markov discriminator to enhance the ability of generator to reconstruct the fusion image, so that the result image can retain more details. Finally, introducing Preconditioned Stochastic Gradient Langevin Dynamics policy, we perform Bayesian inference on the generator network. Experiments on QuickBird and WorldView datasets show that the model proposed in this paper can effectively fuse PAN and MS images, and be competitive with even superior to state of the arts in terms of subjective and objective metrics.