Robust Recursive Fusion of Multiresolution Multispectral Images with Location-Aware Neural Networks

TL;DR

This paper introduces a robust recursive image fusion method using location-aware neural networks to improve satellite image quality, especially under cloud cover, by modeling outliers and image dynamics within a Bayesian framework.

Contribution

It presents a novel recursive fusion framework that incorporates robustness to outliers and models image evolution with neural networks trained on small datasets.

Findings

Significantly improves robustness against cloud cover in satellite image fusion.

Maintains high performance in cloud-free conditions.

Outperforms existing methods in accuracy and robustness.

Abstract

Multiresolution image fusion is a key problem for real-time satellite imaging and plays a central role in detecting and monitoring natural phenomena such as floods. It aims to solve the trade-off between temporal and spatial resolution in remote sensing instruments. Although several algorithms have been proposed for this problem, the presence of outliers such as clouds downgrades their performance. Moreover, strategies that integrate robustness, recursive operation and learned models are missing. In this paper, a robust recursive image fusion framework leveraging location-aware neural networks (NN) to model the image dynamics is proposed. Outliers are modeled by representing the probability of contamination of a given pixel and band. A NN model trained on a small dataset provides accurate predictions of the stochastic image time evolution, which improves both the accuracy and robustness of the method. A recursive solution is proposed to estimate the high-resolution images using a Bayesian variational inference framework. Experiments fusing images from the Landsat 8 and MODIS instruments show that the proposed approach is significantly more robust against cloud cover, without losing performance when no clouds are present.