CrossEarth-Gate: Fisher-Guided Adaptive Tuning Engine for Efficient Adaptation of Cross-Domain Remote Sensing Semantic Segmentation

TL;DR

CrossEarth-Gate introduces a Fisher-guided adaptive tuning engine with a comprehensive module toolbox to efficiently adapt foundation models for diverse and large-scale remote sensing semantic segmentation tasks, achieving state-of-the-art results.

Contribution

It proposes a novel Fisher-guided adaptive selection mechanism combined with a comprehensive RS module toolbox to improve domain adaptation in remote sensing tasks.

Findings

Achieves state-of-the-art performance across 16 benchmarks.

Effectively handles multifaceted domain gaps in RS data.

Demonstrates improved adaptation efficiency and effectiveness.

Abstract

In Remote Sensing (RS), Parameter-Efficient Fine-Tuning (PEFT) has emerged as a key approach to activate the generalizable representation ability of foundation models for downstream tasks. However, existing specialized PEFT methods often fail when applied to large-scale Earth observation tasks, as they are unable to fully handle the multifaceted and unpredictable domain gaps (\eg, spatial, semantic, and frequency shifts) inherent in RS data. To overcome this, we propose CrossEarth-Gate, which introduces two primary contributions. First, we establish a comprehensive RS module toolbox to address multifaceted domain gaps, comprising spatial, semantic, and frequency modules. Second, we develop a Fisher-guided adaptive selection mechanism that operates on this toolbox. This selection is guided by Fisher Information to quantify each module's importance by measuring its contribution to the task-specific gradient flow. It dynamically activates only the most critical modules at the appropriate layers, guiding the gradient flow to maximize adaptation effectiveness and efficiency. Comprehensive experiments validate the efficacy and generalizability of our method, where CrossEarth-Gate achieves state-of-the-art performance across 16 cross-domain benchmarks for RS semantic segmentation. The code of the work will be released.