Multi-Perspective Subimage CLIP with Keyword Guidance for Remote Sensing Image-Text Retrieval

TL;DR

This paper introduces MPS-CLIP, a parameter-efficient, keyword-guided framework for remote sensing image-text retrieval that improves semantic alignment and achieves state-of-the-art results on benchmark datasets.

Contribution

MPS-CLIP shifts from global to fine-grained, keyword-guided alignment in RSITR, utilizing a lightweight adapter and multi-perspective embeddings for enhanced performance.

Findings

Achieves 35.18% and 48.40% mean Recall on RSICD and RSITMD datasets.

Outperforms full fine-tuning baselines and recent methods.

Demonstrates effective semantic matching with minimal computational overhead.

Abstract

Vision-Language Pre-training (VLP) models like CLIP have significantly advanced Remote Sensing Image-Text Retrieval (RSITR). However, existing methods predominantly rely on coarse-grained global alignment, which often overlooks the dense, multi-scale semantics inherent in overhead imagery. Moreover, adapting these heavy models via full fine-tuning incurs prohibitive computational costs and risks catastrophic forgetting. To address these challenges, we propose MPS-CLIP, a parameter-efficient framework designed to shift the retrieval paradigm from global matching to keyword-guided fine-grained alignment. Specifically, we leverage a Large Language Model (LLM) to extract core semantic keywords, guiding the Segment Anything Model (SamGeo) to generate semantically relevant sub-perspectives. To efficiently adapt the frozen backbone, we introduce a Gated Global Attention (G^2A) adapter, which captures global context and long-range dependencies with minimal overhead. Furthermore, a Multi-Perspective Representation (MPR) module aggregates these local cues into robust multi-perspective embeddings. The framework is optimized via a hybrid objective combining multi-perspective contrastive and weighted triplet losses, which dynamically selects maximum-response perspectives to suppress noise and enforce precise semantic matching. Extensive experiments on the RSICD and RSITMD benchmarks demonstrate that MPS-CLIP achieves state-of-the-art performance with 35.18% and 48.40% mean Recall (mR), respectively, significantly outperforming full fine-tuning baselines and recent competitive methods. Code is available at https://github.com/Lcrucial1f/MPS-CLIP.