Bridging Human Evaluation to Infrared and Visible Image Fusion

TL;DR

This paper introduces a human-centered reinforcement learning framework for infrared and visible image fusion, utilizing a large-scale human feedback dataset and a reward model to produce images that better match human aesthetic preferences.

Contribution

It presents the first large-scale human feedback dataset for IVIF and a reward-guided training method to improve perceptual quality of fused images.

Findings

Achieved state-of-the-art performance in human-aligned image fusion.

Developed a domain-specific reward function based on human feedback.

Enhanced fusion results to better match human aesthetic preferences.

Abstract

Infrared and visible image fusion (IVIF) integrates complementary modalities to enhance scene perception. Current methods predominantly focus on optimizing handcrafted losses and objective metrics, often resulting in fusion outcomes that do not align with human visual preferences. This challenge is further exacerbated by the ill-posed nature of IVIF, which severely limits its effectiveness in human perceptual environments such as security surveillance and driver assistance systems. To address these limitations, we propose a feedback reinforcement framework that bridges human evaluation to infrared and visible image fusion. To address the lack of human-centric evaluation metrics and data, we introduce the first large-scale human feedback dataset for IVIF, containing multidimensional subjective scores and artifact annotations, and enriched by a fine-tuned large language model with expert review. Based on this dataset, we design a domain-specific reward function and train a reward model to quantify perceptual quality. Guided by this reward, we fine-tune the fusion network through Group Relative Policy Optimization, achieving state-of-the-art performance that better aligns fused images with human aesthetics. Code is available at https://github.com/ALKA-Wind/EVAFusion.