RSINet: Inpainting Remotely Sensed Images Using Triple GAN Framework

TL;DR

This paper introduces RSINet, a novel inpainting framework for remote sensing images that employs a triple GAN architecture with attention mechanisms to effectively capture spectral, spatial, and textural features, outperforming existing methods.

Contribution

The paper presents a new triple GAN-based inpainting method tailored for remote sensing images, incorporating attention mechanisms and residual learning for improved feature capture.

Findings

Achieves competitive performance on Open Cities AI dataset.

Effectively captures spectral, spatial, and textural features.

Outperforms previous state-of-the-art inpainting models.

Abstract

We tackle the problem of image inpainting in the remote sensing domain. Remote sensing images possess high resolution and geographical variations, that render the conventional inpainting methods less effective. This further entails the requirement of models with high complexity to sufficiently capture the spectral, spatial and textural nuances within an image, emerging from its high spatial variability. To this end, we propose a novel inpainting method that individually focuses on each aspect of an image such as edges, colour and texture using a task specific GAN. Moreover, each individual GAN also incorporates the attention mechanism that explicitly extracts the spectral and spatial features. To ensure consistent gradient flow, the model uses residual learning paradigm, thus simultaneously working with high and low level features. We evaluate our model, alongwith previous state of the art models, on the two well known remote sensing datasets, Open Cities AI and Earth on Canvas, and achieve competitive performance.