A Cross-Modal Image Fusion Method Guided by Human Visual Characteristics

TL;DR

This paper introduces a novel image fusion method inspired by human visual perception, utilizing multi-task learning and nonlinear feature fusion to improve image quality and robustness across various datasets.

Contribution

It proposes a multi-task auxiliary learning framework guided by human visual characteristics, combining channel attention and nonlinear CNNs for enhanced image fusion performance.

Findings

Outperforms existing methods in generality and robustness

Effective in infrared, visible, and multi-focus image fusion

Improves image quality by simulating human visual perception mechanisms

Abstract

The characteristics of feature selection, nonlinear combination and multi-task auxiliary learning mechanism of the human visual perception system play an important role in real-world scenarios, but the research of image fusion theory based on the characteristics of human visual perception is less. Inspired by the characteristics of human visual perception, we propose a robust multi-task auxiliary learning optimization image fusion theory. Firstly, we combine channel attention model with nonlinear convolutional neural network to select features and fuse nonlinear features. Then, we analyze the impact of the existing image fusion loss on the image fusion quality, and establish the multi-loss function model of unsupervised learning network. Secondly, aiming at the multi-task auxiliary learning mechanism of human visual perception system, we study the influence of multi-task auxiliary learning mechanism on image fusion task on the basis of single task multi-loss network model. By simulating the three characteristics of human visual perception system, the fused image is more consistent with the mechanism of human brain image fusion. Finally, in order to verify the superiority of our algorithm, we carried out experiments on the combined vision system image data set, and extended our algorithm to the infrared and visible image and the multi-focus image public data set for experimental verification. The experimental results demonstrate the superiority of our fusion theory over state-of-arts in generality and robustness.