SARVLM: A Vision Language Foundation Model for Semantic Understanding in SAR Imagery

TL;DR

SARVLM is a pioneering vision-language foundation model specifically designed for semantic understanding in SAR imagery, utilizing a large-scale dataset and a novel domain transfer training strategy.

Contribution

The paper introduces SARVLM, the first SAR-specific vision-language model, and a large-scale SARVLM-1M dataset, with a two-stage domain transfer approach from natural images.

Findings

Outperforms state-of-the-art vision-language models on 13 benchmarks.

Demonstrates strong capabilities in image-text retrieval, object detection, and zero-shot classification.

Validates effectiveness through extensive experiments across diverse tasks.

Abstract

Synthetic Aperture Radar (SAR) is a critical imaging modality due to its all-weather operational capability. Although recent advances in self-supervised learning and masked image modeling (MIM) have enabled SAR foundation models, these approaches primarily focus on low-level visual features and often neglect multi-modal representation. Moreover, multimodal data for SAR is scarce, limiting the development of robust cross-modal models. To address this limitation, we construct SARVLM-1M, a large-scale vision-language dataset comprising over one million image-text pairs aggregated from existing datasets. Furthermore, to mitigate the substantial differences between SAR and natural imagery, we propose a two-stage domain transfer training strategy that leverages optical remote sensing data as an intermediate bridge, facilitating effective knowledge transfer from natural images to SAR domains. Based on this strategy, we develop SARVLM, the first vision-language foundation model tailored for SAR, consisting of SARCLIP and SARCap. In addition, an ensemble strategy is utilized to improve the cross-scene generalization capability of the model. Moreover, SARDet and SARRot further validate the capability of the proposed framework in object detection. Extensive experiments on 13 benchmarks across image-text retrieval, target recognition, zero-shot classification, object detection, semantic localization, and image captioning demonstrate the superior feature extraction and interpretation capabilities of SARVLM. It consistently outperforms state-of-the-art vision-language models and advances semantic understanding in SAR imagery. Code and datasets will be released on https://github.com/KlayMa527/SARVLM.git.