CHITNet: A Complementary to Harmonious Information Transfer Network for Infrared and Visible Image Fusion

TL;DR

CHITNet introduces a novel network that effectively transfers and harmonizes complementary and shared features from infrared and visible images, resulting in superior fused images with enhanced quality and structural preservation.

Contribution

The paper proposes a new framework with modules for mutual information transfer, harmonious information acquisition, and structure preservation, advancing infrared and visible image fusion techniques.

Findings

Outperforms state-of-the-art methods in visual quality

Enhances structural detail preservation in fused images

Demonstrates superior quantitative evaluation results

Abstract

Current infrared and visible image fusion (IVIF) methods go to great lengths to excavate complementary features and design complex fusion strategies, which is extremely challenging. To this end, we rethink the IVIF outside the box, proposing a complementary to harmonious information transfer network (CHITNet). It reasonably transfers complementary information into harmonious one, which integrates both the shared and complementary features from two modalities. Specifically, to skillfully sidestep aggregating complementary information in IVIF, we design a mutual information transfer (MIT) module to mutually represent features from two modalities, roughly transferring complementary information into harmonious one. Then, a harmonious information acquisition supervised by source image (HIASSI) module is devised to further ensure the complementary to harmonious information transfer after MIT. Meanwhile, we also propose a structure information preservation (SIP) module to guarantee that the edge structure information of the source images can be transferred to the fusion results. Moreover, a mutual promotion training paradigm with interaction loss is adopted to facilitate better collaboration among MIT, HIASSI and SIP. In this way, the proposed method is able to generate fused images with higher qualities. Extensive experimental results demonstrate the superiority of CHITNet over state-of-the-art algorithms in terms of visual quality and quantitative evaluations.