Perceptual Region-Driven Infrared-Visible Co-Fusion for Extreme Scene Enhancement

TL;DR

This paper introduces a novel region perception-driven fusion framework that combines multi-exposure and multi-modal imaging to enhance infrared-visible image fusion, especially under extreme conditions, ensuring high-quality, geometrically faithful images.

Contribution

It presents a new fusion method leveraging region perception and adaptive strategies to improve IR-VIS image quality and registration accuracy in challenging environments.

Findings

Superior image clarity demonstrated in experiments

Enhanced spectral integration with structural similarity compensation

Robust fusion performance across different exposure scenarios

Abstract

In photogrammetry, accurately fusing infrared (IR) and visible (VIS) spectra while preserving the geometric fidelity of visible features and incorporating thermal radiation is a significant challenge, particularly under extreme conditions. Existing methods often compromise visible imagery quality, impacting measurement accuracy. To solve this, we propose a region perception-based fusion framework that combines multi-exposure and multi-modal imaging using a spatially varying exposure (SVE) camera. This framework co-fuses multi-modal and multi-exposure data, overcoming single-exposure method limitations in extreme environments. The framework begins with region perception-based feature fusion to ensure precise multi-modal registration, followed by adaptive fusion with contrast enhancement. A structural similarity compensation mechanism, guided by regional saliency maps, optimizes IR-VIS spectral integration. Moreover, the framework adapts to single-exposure scenarios for robust fusion across different conditions. Experiments conducted on both synthetic and real-world data demonstrate superior image clarity and improved performance compared to state-of-the-art methods, as evidenced by both quantitative and visual evaluations.