DFVO: Learning Darkness-free Visible and Infrared Image Disentanglement and Fusion All at Once

TL;DR

The paper introduces DFVO, a novel darkness-free network for simultaneous visible and infrared image disentanglement and fusion, improving clarity and illumination in dark environments for high-level vision tasks.

Contribution

It proposes a cascaded multi-task learning framework with a latent-common feature extractor and novel modules, addressing information loss and enhancing fusion quality in challenging lighting conditions.

Findings

Outperforms state-of-the-art methods in qualitative and quantitative metrics.

Achieves the best PSNR and CC scores on the LLVIP dataset.

Produces clearer and more evenly illuminated fused images in dark environments.

Abstract

Visible and infrared image fusion is one of the most crucial tasks in the field of image fusion, aiming to generate fused images with clear structural information and high-quality texture features for high-level vision tasks. However, when faced with severe illumination degradation in visible images, the fusion results of existing image fusion methods often exhibit blurry and dim visual effects, posing major challenges for autonomous driving. To this end, a Darkness-Free network is proposed to handle Visible and infrared image disentanglement and fusion all at Once (DFVO), which employs a cascaded multi-task approach to replace the traditional two-stage cascaded training (enhancement and fusion), addressing the issue of information entropy loss caused by hierarchical data transmission. Specifically, we construct a latent-common feature extractor (LCFE) to obtain latent features for the cascaded tasks strategy. Firstly, a details-extraction module (DEM) is devised to acquire high-frequency semantic information. Secondly, we design a hyper cross-attention module (HCAM) to extract low-frequency information and preserve texture features from source images. Finally, a relevant loss function is designed to guide the holistic network learning, thereby achieving better image fusion. Extensive experiments demonstrate that our proposed approach outperforms state-of-the-art alternatives in terms of qualitative and quantitative evaluations. Particularly, DFVO can generate clearer, more informative, and more evenly illuminated fusion results in the dark environments, achieving best performance on the LLVIP dataset with 63.258 dB PSNR and 0.724 CC, providing more effective information for high-level vision tasks. Our code is publicly accessible at https://github.com/DaVin-Qi530/DFVO.