Frequency-Adaptive Pan-Sharpening with Mixture of Experts
Xuanhua He, Keyu Yan, Rui Li, Chengjun Xie, Jie Zhang, Man Zhou

TL;DR
This paper introduces FAME, a frequency-adaptive mixture of experts framework for pan-sharpening that effectively reconstructs high-frequency details in multi-spectral images by leveraging frequency domain separation and dynamic expert weighting.
Contribution
The paper proposes a novel frequency domain-based pan-sharpening framework with adaptive frequency separation and expert mixture modules, improving reconstruction accuracy and generalization.
Findings
Outperforms state-of-the-art methods on multiple datasets
Demonstrates strong generalization to real-world scenes
Effectively reconstructs high-frequency details in multi-spectral images
Abstract
Pan-sharpening involves reconstructing missing high-frequency information in multi-spectral images with low spatial resolution, using a higher-resolution panchromatic image as guidance. Although the inborn connection with frequency domain, existing pan-sharpening research has not almost investigated the potential solution upon frequency domain. To this end, we propose a novel Frequency Adaptive Mixture of Experts (FAME) learning framework for pan-sharpening, which consists of three key components: the Adaptive Frequency Separation Prediction Module, the Sub-Frequency Learning Expert Module, and the Expert Mixture Module. In detail, the first leverages the discrete cosine transform to perform frequency separation by predicting the frequency mask. On the basis of generated mask, the second with low-frequency MOE and high-frequency MOE takes account for enabling the effective low-frequency…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
Taxonomy
TopicsAdvanced Image Fusion Techniques · Photoacoustic and Ultrasonic Imaging · Image Enhancement Techniques
MethodsDiscrete Cosine Transform
