Dynamic Brightness Adaptation for Robust Multi-modal Image Fusion

TL;DR

This paper introduces BA-Fusion, a novel framework for multi-modal image fusion that maintains high visual quality and robustness under dynamic brightness changes by adaptively selecting features and enforcing brightness consistency.

Contribution

The paper proposes the Brightness Adaptive Gate (BAG) module and a brightness consistency loss to enhance robustness of image fusion against brightness fluctuations, a novel approach in the field.

Findings

Outperforms state-of-the-art methods in preserving image details.

Demonstrates robustness across varying brightness levels.

Improves visual fidelity of fused images.

Abstract

Infrared and visible image fusion aim to integrate modality strengths for visually enhanced, informative images. Visible imaging in real-world scenarios is susceptible to dynamic environmental brightness fluctuations, leading to texture degradation. Existing fusion methods lack robustness against such brightness perturbations, significantly compromising the visual fidelity of the fused imagery. To address this challenge, we propose the Brightness Adaptive multimodal dynamic fusion framework (BA-Fusion), which achieves robust image fusion despite dynamic brightness fluctuations. Specifically, we introduce a Brightness Adaptive Gate (BAG) module, which is designed to dynamically select features from brightness-related channels for normalization, while preserving brightness-independent structural information within the source images. Furthermore, we propose a brightness consistency loss function to optimize the BAG module. The entire framework is tuned via alternating training strategies. Extensive experiments validate that our method surpasses state-of-the-art methods in preserving multi-modal image information and visual fidelity, while exhibiting remarkable robustness across varying brightness levels. Our code is available: https://github.com/SunYM2020/BA-Fusion.