Fast Kernel-Space Diffusion for Remote Sensing Pansharpening

TL;DR

KSDiff is a novel fast kernel-space diffusion framework that improves remote sensing pansharpening by capturing global context and significantly reducing inference time compared to existing diffusion methods.

Contribution

The paper introduces KSDiff, a new diffusion-based method with a two-stage training strategy that enhances pansharpening quality and achieves over 500 times faster inference.

Findings

KSDiff outperforms recent methods in pansharpening quality.

KSDiff achieves over 500x faster inference than previous diffusion-based approaches.

Ablation studies confirm the effectiveness of the kernel construction and training strategy.

Abstract

Pansharpening seeks to fuse high-resolution panchromatic (PAN) and low-resolution multispectral (LRMS) images into a single image with both fine spatial and rich spectral detail. Despite progress in deep learning-based approaches, existing methods often fail to capture global priors inherent in remote sensing data distributions. Diffusion-based models have recently emerged as promising solutions due to their powerful distribution mapping capabilities, however, they suffer from heavy inference latency. We introduce KSDiff, a fast kernel-space diffusion framework that generates convolutional kernels enriched with global context to enhance pansharpening quality and accelerate inference. Specifically, KSDiff constructs these kernels through the integration of a low-rank core tensor generator and a unified factor generator, orchestrated by a structure-aware multi-head attention mechanism. We further introduce a two-stage training strategy tailored for pansharpening, facilitating integration into existing pansharpening architectures. Experiments show that KSDiff achieves superior performance compared to recent promising methods, and with over $500 \times$ faster inference than diffusion-based pansharpening baselines. Ablation studies, visualizations and further evaluations substantiate the effectiveness of our approach. Code will be released upon possible acceptance.