MS-Net: A Multi-modal Self-supervised Network for Fine-Grained Classification of Aircraft in SAR Images

TL;DR

This paper introduces MS-Net, a multi-modal self-supervised network that improves fine-grained aircraft classification in SAR images by leveraging multi-modal features and contrastive learning, achieving high accuracy without labeled data.

Contribution

The paper proposes a novel multi-modal self-supervised framework with a two-sided feature extraction network and a contrastive learning scheme for aircraft classification in SAR images.

Findings

Achieves 88.46% accuracy on 17 aircraft types without labels.

Effectively reduces intra-class diversity and inter-class similarity issues.

Demonstrates the effectiveness of multi-modal and self-supervised learning in SAR image classification.

Abstract

Synthetic aperture radar (SAR) imaging technology is commonly used to provide 24-hour all-weather earth observation. However, it still has some drawbacks in SAR target classification, especially in fine-grained classification of aircraft: aircrafts in SAR images have large intra-class diversity and inter-class similarity; the number of effective samples is insufficient and it's hard to annotate. To address these issues, this article proposes a novel multi-modal self-supervised network (MS-Net) for fine-grained classification of aircraft. Firstly, in order to entirely exploit the potential of multi-modal information, a two-sided path feature extraction network (TSFE-N) is constructed to enhance the image feature of the target and obtain the domain knowledge feature of text mode. Secondly, a contrastive self-supervised learning (CSSL) framework is employed to effectively learn useful label-independent feature from unbalanced data, a similarity per-ception loss (SPloss) is proposed to avoid network overfitting. Finally, TSFE-N is used as the encoder of CSSL to obtain the classification results. Through a large number of experiments, our MS-Net can effectively reduce the difficulty of classifying similar types of aircrafts. In the case of no label, the proposed algorithm achieves an accuracy of 88.46% for 17 types of air-craft classification task, which has pioneering significance in the field of fine-grained classification of aircraft in SAR images.