Guided Deep Generative Model-based Spatial Regularization for Multiband Imaging Inverse Problems

TL;DR

This paper introduces a flexible deep learning-based spatial regularization framework for multiband imaging inverse problems, utilizing high-resolution auxiliary images to improve reconstruction quality in tasks like image fusion and inpainting.

Contribution

It proposes a novel data-driven spatial regularization method using deep generative networks informed by high-resolution auxiliary images, enhancing traditional regularization techniques.

Findings

Improved image reconstruction quality over conventional regularizations.

Effective in multiband image fusion and inpainting tasks.

Demonstrates versatility of deep generative regularization.

Abstract

When adopting a model-based formulation, solving inverse problems encountered in multiband imaging requires to define spatial and spectral regularizations. In most of the works of the literature, spectral information is extracted from the observations directly to derive data-driven spectral priors. Conversely, the choice of the spatial regularization often boils down to the use of conventional penalizations (e.g., total variation) promoting expected features of the reconstructed image (e.g., piecewise constant). In this work, we propose a generic framework able to capitalize on an auxiliary acquisition of high spatial resolution to derive tailored data-driven spatial regularizations. This approach leverages on the ability of deep learning to extract high level features. More precisely, the regularization is conceived as a deep generative network able to encode spatial semantic features contained in this auxiliary image of high spatial resolution. To illustrate the versatility of this approach, it is instantiated to conduct two particular tasks, namely multiband image fusion and multiband image inpainting. Experimental results obtained on these two tasks demonstrate the benefit of this class of informed regularizations when compared to more conventional ones.