CoMoFusion: Fast and High-quality Fusion of Infrared and Visible Image with Consistency Model

TL;DR

CoMoFusion introduces a novel consistency model-based approach for infrared and visible image fusion, achieving high-quality results with faster inference and improved texture preservation compared to existing methods.

Contribution

The paper proposes a new fusion method using a consistency model that enhances speed and quality in infrared-visible image fusion, with a novel pixel-based loss function.

Findings

Achieves state-of-the-art fusion performance on public datasets.

Provides faster inference speed than existing generative model-based methods.

Enhances texture and salient details in fused images.

Abstract

Generative models are widely utilized to model the distribution of fused images in the field of infrared and visible image fusion. However, current generative models based fusion methods often suffer from unstable training and slow inference speed. To tackle this problem, a novel fusion method based on consistency model is proposed, termed as CoMoFusion, which can generate the high-quality images and achieve fast image inference speed. In specific, the consistency model is used to construct multi-modal joint features in the latent space with the forward and reverse process. Then, the infrared and visible features extracted by the trained consistency model are fed into fusion module to generate the final fused image. In order to enhance the texture and salient information of fused images, a novel loss based on pixel value selection is also designed. Extensive experiments on public datasets illustrate that our method obtains the SOTA fusion performance compared with the existing fusion methods.