Cloud removal in remote sensing images using generative adversarial networks and SAR-to-optical image translation

TL;DR

This paper introduces a novel two-GAN framework with dilated residual inception blocks for effective cloud removal in satellite images by translating SAR to optical images and removing clouds, outperforming existing models.

Contribution

The study proposes a new GAN-based method with dilated residual inception blocks and combined SSIM and L1 loss for improved cloud removal and SAR-to-optical translation.

Findings

Achieved higher PSNR and SSIM scores than state-of-the-art models.

Effectively removed clouds from satellite images using the proposed GAN framework.

Enhanced image quality by expanding receptive fields with dilated convolutions.

Abstract

Satellite images are often contaminated by clouds. Cloud removal has received much attention due to the wide range of satellite image applications. As the clouds thicken, the process of removing the clouds becomes more challenging. In such cases, using auxiliary images such as near-infrared or synthetic aperture radar (SAR) for reconstructing is common. In this study, we attempt to solve the problem using two generative adversarial networks (GANs). The first translates SAR images into optical images, and the second removes clouds using the translated images of prior GAN. Also, we propose dilated residual inception blocks (DRIBs) instead of vanilla U-net in the generator networks and use structural similarity index measure (SSIM) in addition to the L1 Loss function. Reducing the number of downsamplings and expanding receptive fields by dilated convolutions increase the quality of output images. We used the SEN1-2 dataset to train and test both GANs, and we made cloudy images by adding synthetic clouds to optical images. The restored images are evaluated with PSNR and SSIM. We compare the proposed method with state-of-the-art deep learning models and achieve more accurate results in both SAR-to-optical translation and cloud removal parts.