Cross-modal Ship Re-Identification via Optical and SAR Imagery: A Novel Dataset and Method

TL;DR

This paper introduces a new dataset and a baseline method for cross-modal ship re-identification using optical and SAR imagery, addressing limitations of current satellite-based tracking under diverse conditions.

Contribution

The paper presents the HOSS ReID dataset for all-weather ship tracking and proposes TransOSS, a Vision Transformer-based model for cross-modal re-identification.

Findings

HOSS ReID dataset includes diverse optical and SAR images of ships over time.

TransOSS achieves modality-invariant feature extraction for cross-modal re-identification.

The approach enables more reliable maritime ship tracking under various weather and lighting conditions.

Abstract

Detecting and tracking ground objects using earth observation imagery remains a significant challenge in the field of remote sensing. Continuous maritime ship tracking is crucial for applications such as maritime search and rescue, law enforcement, and shipping analysis. However, most current ship tracking methods rely on geostationary satellites or video satellites. The former offer low resolution and are susceptible to weather conditions, while the latter have short filming durations and limited coverage areas, making them less suitable for the real-world requirements of ship tracking. To address these limitations, we present the Hybrid Optical and Synthetic Aperture Radar (SAR) Ship Re-Identification Dataset (HOSS ReID dataset), designed to evaluate the effectiveness of ship tracking using low-Earth orbit constellations of optical and SAR sensors. This approach ensures shorter re-imaging cycles and enables all-weather tracking. HOSS ReID dataset includes images of the same ship captured over extended periods under diverse conditions, using different satellites of different modalities at varying times and angles. Furthermore, we propose a baseline method for cross-modal ship re-identification, TransOSS, which is built on the Vision Transformer architecture. It refines the patch embedding structure to better accommodate cross-modal tasks, incorporates additional embeddings to introduce more reference information, and employs contrastive learning to pre-train on large-scale optical-SAR image pairs, ensuring the model's ability to extract modality-invariant features. Our dataset and baseline method are publicly available on https://github.com/Alioth2000/Hoss-ReID.