CC-Pan: Channel-wise Compression based Diffusion for Efficient Pan-Sharpening

TL;DR

CC-Pan introduces a cross-sensor latent diffusion framework with a band-wise VAE and a cross-band attention module, enabling efficient, high-precision pan-sharpening across different sensors with faster inference.

Contribution

It proposes a novel latent diffusion approach with a band-wise VAE and cross-band attention, supporting multiple sensors without retraining and improving efficiency and accuracy.

Findings

Outperforms state-of-the-art diffusion-based methods on GaoFen-2, QuickBird, and WorldView-3 datasets.

Achieves 2-3 times faster inference speed.

Demonstrates robust cross-sensor generalization without sensor-specific retraining.

Abstract

Recently, diffusion models have brought novel insights to pan-sharpening and notably boosted fusion precision. However, most existing models perform diffusion in the pixel space and train distinct models for different multispectral (MS) sensors, suffering from high inference latency and sensor-specific limitations. In this paper, we present CC-Pan, a cross-sensor latent diffusion framework for efficient pan-sharpening. Specifically, CC-Pan trains a band-wise single-channel variational autoencoder (VAE) to encode high-resolution multispectral (HRMS) images into compact latent representations, naturally supporting MS images with varying band counts across different sensors and establishing a basis for inference acceleration. Spectral physical properties, along with PAN and MS images, are then injected into the diffusion backbone through carefully designed unidirectional and bidirectional interactive control structures, achieving high-precision spatial--spectral fusion in the latent diffusion process. Furthermore, a lightweight region-based cross-band attention (RCBA) module is incorporated at the central layer of the diffusion model, reinforcing inter-band spectral connections to boost spectral consistency and further elevate fusion precision. Extensive experimental results on GaoFen-2, QuickBird, and WorldView-3 demonstrate that CC-Pan outperforms state-of-the-art diffusion-based methods across all three benchmarks, attains a $2$--$3\times$ inference speedup, and exhibits robust cross-sensor generalization capability on the held-out WorldView-2 sensor without any sensor-specific retraining.