Cross-Domain Transfer of Hyperspectral Foundation Models

TL;DR

This paper introduces a cross-domain transfer method for hyperspectral image segmentation that preserves spectral information and improves performance over traditional in-domain training, especially with limited data.

Contribution

It proposes reusing existing hyperspectral foundation models for proximal sensing, avoiding complex architecture and spectral information loss.

Findings

Cross-domain transfer outperforms in-domain training.

It narrows the performance gap with cross-modality methods.

It remains effective with limited training data.

Abstract

Hyperspectral imaging (HSI) semantic segmentation typically relies on in-domain training, but limited data availability often restricts model performance in real-world applications. Current approaches to leverage foundation models in proximal sensing use cross-modality techniques, bridging RGB and HSI to exploit vision foundation models. However, these methods either discard spectral information or introduce architectural complexity. We propose cross-domain transfer as an alternative, reusing HSI foundation models - originally trained in remote sensing - for proximal sensing applications. By eliminating the need to bridge modality gaps, our approach preserves spectral information while maintaining a simple architecture. Using the HS3-Bench benchmark, we systematically evaluate and compare conventional in-domain, in-modality training, cross-modality transfer and cross-domain transfer strategies. Our results demonstrate that cross-domain transfer achieves large performance improvements over in-domain, in-modality training, reduces the performance gap to cross-modality approaches and maintains strong performance in limited data settings. Thus, this work advances more effective HSI semantic segmentation in diverse applications.