Generalizable Multispectral Land Cover Classification via Frequency-Aware Mixture of Low-Rank Token Experts

TL;DR

Land-MoE introduces a hierarchical, frequency-aware mixture of low-rank token experts to improve multispectral land cover classification, effectively handling spectral shifts and outperforming existing methods in cross-sensor and geospatial scenarios.

Contribution

The paper proposes Land-MoE, a novel, parameter-efficient model with frequency-aware modules and low-rank token experts for robust multispectral land cover classification.

Findings

Outperforms existing methods in MLCC tasks across sensors and geospatial conditions.

Achieves state-of-the-art results in domain generalization for remote sensing segmentation.

Demonstrates robustness against spectral shifts and noise.

Abstract

We introduce Land-MoE, a novel approach for multispectral land cover classification (MLCC). Spectral shift, which emerges from disparities in sensors and geospatial conditions, poses a significant challenge in this domain. Existing methods predominantly rely on domain adaptation and generalization strategies, often utilizing small-scale models that exhibit limited performance. In contrast, Land-MoE addresses these issues by hierarchically inserting a Frequency-aware Mixture of Low-rank Token Experts, to fine-tune Vision Foundation Models (VFMs) in a parameter-efficient manner. Specifically, Land-MoE comprises two key modules: the mixture of low-rank token experts (MoLTE) and frequency-aware filters (FAF). MoLTE leverages rank-differentiated tokens to generate diverse feature adjustments for individual instances within multispectral images. By dynamically combining learnable low-rank token experts of varying ranks, it enhances the robustness against spectral shifts. Meanwhile, FAF conducts frequency-domain modulation on the refined features. This process enables the model to effectively capture frequency band information that is strongly correlated with semantic essence, while simultaneously suppressing frequency noise irrelevant to the task. Comprehensive experiments on MLCC tasks involving cross-sensor and cross-geospatial setups demonstrate that Land-MoE outperforms existing methods by a large margin. Additionally, the proposed approach has also achieved state-of-the-art performance in domain generalization semantic segmentation tasks of RGB remote sensing images.