Bringing Multimodal Large Language Models to Infrared-Visible Image Fusion Quality Assessment

TL;DR

This paper presents FuScore, a novel MLLM-based evaluation method for infrared-visible image fusion quality that produces continuous scores and aligns closely with human perception.

Contribution

It introduces FuScore, leveraging MLLMs for fine-grained, continuous quality assessment and a tripartite objective for improved correlation with human judgments.

Findings

FuScore achieves state-of-the-art correlation with human preferences.

It effectively discriminates among fused images of similar quality.

The method incorporates scene and method-level ordering for comprehensive evaluation.

Abstract

Infrared-Visible image fusion (IVIF) aims to integrate thermal information and detailed spatial structures into a single fused image to enhance perception. However, existing evaluation approaches tend to over-optimize both hand-crafted no-reference statistics and full-reference metrics that treat the source images as pseudo ground truths. Recent IVIF reward-modelling efforts learn from human ratings but use scalar regression on aggregated scores, neither leveraging the reasoning of Multimodal Large Language Models (MLLMs) nor encoding per-image perceptual ambiguity in their supervision, but naively introducing MLLMs with discrete one-hot supervision likewise collapses fused images of similar quality into different rating levels. To address this, we introduce FuScore, which utilizes an MLLM to mimic human visual perception by producing continuous quality score, rather than discrete level predictions, enabling fine-grained discrimination among fused images of similar quality. We exploit the agreement among four IVIF-specific sub-dimensions to construct a per-image soft label whose sharpness reflects how consensual the overall judgment is. We further introduce a tripartite objective combining per-image distributional supervision, within-source-pair Thurstone fidelity for method-level ordering, and cross-source-pair Thurstone fidelity for scene-level ordering across scenes. Extensive experiments demonstrate that FuScore achieves state-of-the-art correlation with human visual preferences.