Unsupervised Image Fusion Using Deep Image Priors

TL;DR

This paper introduces a novel unsupervised image fusion method based on extending Deep Image Prior to multi-image fusion tasks, demonstrating superior performance especially in medical imaging without requiring large training datasets.

Contribution

The paper proposes a new extension of Deep Image Prior for multi-image fusion, overcoming limitations of existing DIP applications and achieving state-of-the-art results in image fusion tasks.

Findings

Outperforms existing methods on multiple metrics

Achieves best objective results in medical image fusion

Effective without large training datasets

Abstract

A significant number of researchers have applied deep learning methods to image fusion. However, most works require a large amount of training data or depend on pre-trained models or frameworks to capture features from source images. This is inevitably hampered by a shortage of training data or a mismatch between the framework and the actual problem. Deep Image Prior (DIP) has been introduced to exploit convolutional neural networks' ability to synthesize the 'prior' in the input image. However, the original design of DIP is hard to be generalized to multi-image processing problems, particularly for image fusion. Therefore, we propose a new image fusion technique that extends DIP to fusion tasks formulated as inverse problems. Additionally, we apply a multi-channel approach to enhance DIP's effect further. The evaluation is conducted with several commonly used image fusion assessment metrics. The results are compared with state-of-the-art image fusion methods. Our method outperforms these techniques for a range of metrics. In particular, it is shown to provide the best objective results for most metrics when applied to medical images.