UMCFuse: A Unified Multiple Complex Scenes Infrared and Visible Image Fusion Framework

TL;DR

UMCFuse is a novel framework that effectively fuses infrared and visible images in complex scenes, handling interference and noise to improve various computer vision tasks.

Contribution

The paper introduces a unified approach for infrared and visible image fusion in complex scenes, including pixel classification, adaptive denoising, and multi-directional energy feature analysis.

Findings

Outperforms recent methods in complex scene fusion tasks

Enhances downstream tasks like segmentation and detection

Robust under adverse weather, noise, and interference conditions

Abstract

Infrared and visible image fusion has emerged as a prominent research area in computer vision. However, little attention has been paid to the fusion task in complex scenes, leading to sub-optimal results under interference. To fill this gap, we propose a unified framework for infrared and visible images fusion in complex scenes, termed UMCFuse. Specifically, we classify the pixels of visible images from the degree of scattering of light transmission, allowing us to separate fine details from overall intensity. Maintaining a balance between interference removal and detail preservation is essential for the generalization capacity of the proposed method. Therefore, we propose an adaptive denoising strategy for the fusion of detail layers. Meanwhile, we fuse the energy features from different modalities by analyzing them from multiple directions. Extensive fusion experiments on real and synthetic complex scenes datasets cover adverse weather conditions, noise, blur, overexposure, fire, as well as downstream tasks including semantic segmentation, object detection, salient object detection, and depth estimation, consistently indicate the superiority of the proposed method compared with the recent representative methods. Our code is available at https://github.com/ixilai/UMCFuse.