Infrared and Visible Image Fusion via Interactive Compensatory Attention Adversarial Learning

TL;DR

This paper introduces ICAFusion, an innovative adversarial learning framework that employs interactive compensatory attention to effectively fuse infrared and visible images, emphasizing global features for balanced and detailed results.

Contribution

The paper proposes a novel end-to-end fusion network with interactive attention modules and dual discriminators, enhancing feature representation and fusion balance over existing methods.

Findings

Outperforms state-of-the-art fusion methods in subjective visual quality

Achieves superior results on objective metrics across multiple datasets

Demonstrates strong generalization ability in various fusion scenarios

Abstract

The existing generative adversarial fusion methods generally concatenate source images and extract local features through convolution operation, without considering their global characteristics, which tends to produce an unbalanced result and is biased towards the infrared image or visible image. Toward this end, we propose a novel end-to-end mode based on generative adversarial training to achieve better fusion balance, termed as \textit{interactive compensatory attention fusion network} (ICAFusion). In particular, in the generator, we construct a multi-level encoder-decoder network with a triple path, and adopt infrared and visible paths to provide additional intensity and gradient information. Moreover, we develop interactive and compensatory attention modules to communicate their pathwise information, and model their long-range dependencies to generate attention maps, which can more focus on infrared target perception and visible detail characterization, and further increase the representation power for feature extraction and feature reconstruction. In addition, dual discriminators are designed to identify the similar distribution between fused result and source images, and the generator is optimized to produce a more balanced result. Extensive experiments illustrate that our ICAFusion obtains superior fusion performance and better generalization ability, which precedes other advanced methods in the subjective visual description and objective metric evaluation. Our codes will be public at \url{https://github.com/Zhishe-Wang/ICAFusion}