S2WMamba: A Wavelet-Assisted Mamba-Based Dual-Branch Network For Pansharpening

TL;DR

S2WMamba is a novel dual-branch network that uses wavelet transforms to disentangle spatial and spectral features for improved pansharpening, achieving state-of-the-art results on multiple datasets.

Contribution

It introduces a wavelet-assisted dual-branch framework with explicit frequency disentanglement and cross-modulation for enhanced pansharpening performance.

Findings

Outperforms recent baselines in PSNR and HQNR metrics.

Achieves up to 0.23 dB PSNR improvement.

Demonstrates effective frequency disentanglement through ablation studies.

Abstract

Pansharpening fuses a high-resolution panchromatic (PAN) image with a low-resolution multispectral (LRMS) image to produce a high-resolution multispectral (HRMS) image. A key difficulty is that jointly processing PAN and MS features often entangles spatial detail enhancement with spectral fidelity. To address this feature entanglement, we propose S2WMamba, a framework that explicitly disentangles modality-specific frequency information for highly controlled crossmodal interaction. Concretely, unlike global frequency transforms, a localized 2D Haar DWT is applied to the PAN image to precisely isolate spatial edges and textures. Concurrently, a novel channel-wise 1D Haar DWT treats each pixel's spectrum as a 1D signal, isolating the shared spectral base from band-specific variations to strictly limit spectral distortion. The resulting Spectral branch injects wavelet-extracted spatial details into MS features, while the Spatial branch refines PAN features using spectra from the DWT1D process. To overcome inadequate frequency fusion, the two branches exchange information via Mambabased cross-modulation, which explicitly models long-range dependencies across these decoupled sub-bands with linear complexity. On WV3, GF2, and QB datasets, S2WMamba matches or surpasses recent strong baselines (FusionMamba, CANNet, U2Net, PanNet), improving PSNR by up to 0.23 dB and reaching an HQNR of 0.956 on full-resolution WV3. Extensive ablations justify the modality-specific DWT placement and the parallel dual-branch architecture.