# Rethinking Infrared and Visible Image Fusion from a Heterogeneous Content Synergistic Perception Perspective

## TL;DR

This paper introduces HCSPNet, a new GAN-based framework for infrared and visible image fusion that reduces artifacts by using heterogeneous dual discriminators.

## Contribution

The novel HCSPNet framework uses heterogeneous dual discriminators with adaptive salient information distillation to improve infrared-visible image fusion.

## Key findings

- HCSPNet preserves critical information from infrared and visible images even in degraded scenarios.
- Experiments on public datasets show HCSPNet outperforms existing GAN-based methods.
- The heterogeneous dual discriminators can be used as a plug-and-play module to enhance other GAN methods.

## Abstract

Infrared and visible image fusion (IVIF) endeavors to amalgamate the thermal radiation characteristics from infrared images with the fine-grained texture details from visible images, aiming to produce fused outputs that are more robust and information-rich. Among the existing methodologies, those based on generative adversarial networks (GANs) have demonstrated considerable promise. However, such approaches are frequently constrained by their reliance on homogeneous discriminators possessing identical architectures, a limitation that can precipitate the emergence of undesirable artifacts in the resultant fused images. To surmount this challenge, this paper introduces HCSPNet, a novel GAN-based framework. HCSPNet distinctively incorporates heterogeneous dual discriminators, meticulously engineered for the fusion of disparate source images inherent in the IVIF task. This architectural design ensures the steadfast preservation of critical information from the source inputs, even when faced with scenarios of image degradation. Specifically, the two structurally distinct discriminators within HCSPNet are augmented with adaptive salient information distillation (ASID) modules, each uniquely structured to align with the intrinsic properties of infrared and visible images. This mechanism impels the discriminators to concentrate on pivotal components during their assessment of whether the fused image has proficiently inherited significant information from the source modalities—namely, the salient thermal signatures from infrared imagery and the detailed textural content from visible imagery—thereby markedly diminishing the occurrence of unwanted artifacts. Comprehensive experimentation conducted across multiple publicly available datasets substantiates the preeminence and generalization capabilities of HCSPNet, underscoring its significant potential for practical deployment. Additionally, we also prove that our proposed heterogeneous dual discriminators can serve as a plug-and-play structure to improve the performance of existing GAN-based methods.