Unsupervised Image Fusion Method based on Feature Mutual Mapping

TL;DR

This paper introduces an unsupervised image fusion method that uses feature mutual mapping and a dual-branch multi-scale autoencoder to improve fusion quality without manual fusion functions, demonstrating superior results across various tasks.

Contribution

The paper presents a novel unsupervised fusion approach with a feature mutual mapping module and dual-branch multi-scale autoencoder, addressing limitations of manual fusion functions and input-independent learning.

Findings

Achieves superior visual and objective fusion performance

Demonstrates excellent generalization across different image types

Outperforms state-of-the-art methods in multiple fusion tasks

Abstract

Deep learning-based image fusion approaches have obtained wide attention in recent years, achieving promising performance in terms of visual perception. However, the fusion module in the current deep learning-based methods suffers from two limitations, \textit{i.e.}, manually designed fusion function, and input-independent network learning. In this paper, we propose an unsupervised adaptive image fusion method to address the above issues. We propose a feature mutual mapping fusion module and dual-branch multi-scale autoencoder. More specifically, we construct a global map to measure the connections of pixels between the input source images. % The found mapping relationship guides the image fusion. Besides, we design a dual-branch multi-scale network through sampling transformation to extract discriminative image features. We further enrich feature representations of different scales through feature aggregation in the decoding process. Finally, we propose a modified loss function to train the network with efficient convergence property. Through sufficient training on infrared and visible image data sets, our method also shows excellent generalized performance in multi-focus and medical image fusion. Our method achieves superior performance in both visual perception and objective evaluation. Experiments prove that the performance of our proposed method on a variety of image fusion tasks surpasses other state-of-the-art methods, proving the effectiveness and versatility of our approach.