Bi^2MAC: Bimodal Bi-Adaptive Mask-Aware Convolution for Remote Sensing Pansharpening

TL;DR

This paper introduces Bi^2MAC, a novel adaptive convolution method for remote sensing pansharpening that efficiently captures regional heterogeneity, improves performance, and reduces computational costs compared to existing methods.

Contribution

Bi^2MAC is a lightweight, mask-aware adaptive convolution that dynamically allocates resources to heterogeneous regions, achieving state-of-the-art results with lower computational costs.

Findings

Bi^2MAC outperforms existing methods on multiple benchmarks.

It requires less training time and fewer parameters.

It effectively captures regional heterogeneity in remote sensing images.

Abstract

Pansharpening aims to fuse a high-resolution panchromatic (PAN) image with a low-resolution multispectral (LRMS) image to generate a high-resolution multispectral image (HRMS). Conventional deep learning-based methods are inherently limited in their ability to adapt to regional heterogeneity within feature representations. Although various adaptive convolution methods have been proposed to address this limitation, they often suffer from excessive computational costs and a limited ability to capture heterogeneous regions in remote sensing images effectively. To overcome these challenges, we propose Bimodal Bi-Adaptive Mask-Aware Convolution (Bi^2MAC), which effectively exploits information from different types of regions while intelligently allocating computational resources. Specifically, we design a lightweight module to generate both soft and hard masks, which are used to modulate the input features preliminarily and to guide different types of regions into separate processing branches, respectively. Redundant features are directed to a compact branch for low-cost global processing. In contrast, heterogeneous features are routed to a focused branch that invests more computational resources for fine-grained modeling. Extensive experiments on multiple benchmark datasets demonstrate that Bi^2MAC achieves state-of-the-art (SOTA) performance while requiring substantially lower training time and parameter counts, and the minimal computational cost among adaptive convolution models.