Semi-Supervised Flow Matching for Mosaiced and Panchromatic Fusion Imaging

TL;DR

This paper introduces a semi-supervised flow matching framework for fusing mosaiced hyperspectral images with high-resolution panchromatic images, enabling high-quality, single-shot imaging with improved robustness and generalization.

Contribution

It presents a novel two-stage training pipeline combining unsupervised prior learning and flow matching, advancing image fusion techniques beyond diffusion-based methods.

Findings

Achieves superior quantitative performance on benchmark datasets.

Demonstrates effective fusion with a flexible, extendable generative framework.

Outperforms existing baselines in both qualitative and quantitative assessments.

Abstract

Fusing a low resolution (LR) mosaiced hyperspectral image (HSI) with a high resolution (HR) panchromatic (PAN) image offers a promising avenue for video-rate HR-HSI imaging via single-shot acquisition, yet its severely ill-posed nature remains a significant challenge. In this work, we propose a novel semi-supervised flow matching framework for mosaiced and PAN image fusion. Unlike previous diffusion-based approaches constrained by specific protocols or handcrafted assumptions, our method seamlessly integrates an unsupervised scheme with flow matching, resulting in a generalizable and efficient generative framework. Specifically, our method follows a two-stage training pipeline. First, we pretrain an unsupervised prior network to produce an initial pseudo HR-HSI. Building on this, we then train a conditional flow matching model to generate the target HR-HSI, introducing a random voting mechanism that iteratively refines the initial HR-HSI estimate, enabling robust and effective fusion. During inference, we employ a conflict-free gradient guidance strategy that ensures spectrally and spatially consistent HR-HSI reconstruction. Experiments on multiple benchmark datasets demonstrate that our method achieves superior quantitative and qualitative performance by a significant margin compared to representative baselines. Beyond mosaiced and PAN fusion, our approach provides a flexible generative framework that can be readily extended to other image fusion tasks and integrated with unsupervised or blind image restoration algorithms.