Coupling Model-Driven and Data-Driven Methods for Remote Sensing Image Restoration and Fusion

TL;DR

This paper explores the integration of model-driven and data-driven approaches for remote sensing image restoration and fusion, aiming to leverage their complementary strengths and address their individual limitations.

Contribution

It systematically categorizes and summarizes existing coupling methods of model-driven and data-driven techniques in remote sensing image processing.

Findings

Three main coupling categories identified: cascading, variational with learning, and model-constrained networks.

Introduces application examples demonstrating the effectiveness of coupling methods.

Provides insights into future research directions in the field.

Abstract

In the fields of image restoration and image fusion, model-driven methods and data-driven methods are the two representative frameworks. However, both approaches have their respective advantages and disadvantages. The model-driven methods consider the imaging mechanism, which is deterministic and theoretically reasonable; however, they cannot easily model complicated nonlinear problems. The data-driven methods have a stronger prior knowledge learning capability for huge data, especially for nonlinear statistical features; however, the interpretability of the networks is poor, and they are over-dependent on training data. In this paper, we systematically investigate the coupling of model-driven and data-driven methods, which has rarely been considered in the remote sensing image restoration and fusion communities. We are the first to summarize the coupling approaches into the following three categories: 1) data-driven and model-driven cascading methods; 2) variational models with embedded learning; and 3) model-constrained network learning methods. The typical existing and potential coupling methods for remote sensing image restoration and fusion are introduced with application examples. This paper also gives some new insights into the potential future directions, in terms of both methods and applications.