Multi-Modal Domain Fusion for Multi-modal Aerial View Object Classification

TL;DR

This paper introduces a Multi-Modal Domain Fusion network that combines EO and SAR aerial images to improve object classification accuracy in challenging conditions, outperforming existing methods on the PBVS MAVOC dataset.

Contribution

A novel Multi-Modal Domain Fusion network is proposed to learn domain-invariant features from multi-modal aerial data for enhanced object classification.

Findings

Achieved top-10 performance with 25.3% accuracy in Track-1.

Achieved top-5 performance with 34.26% accuracy in Track-2.

Effectively handles sensor limitations and improves ATR system accuracy.

Abstract

Object detection and classification using aerial images is a challenging task as the information regarding targets are not abundant. Synthetic Aperture Radar(SAR) images can be used for Automatic Target Recognition(ATR) systems as it can operate in all-weather conditions and in low light settings. But, SAR images contain salt and pepper noise(speckle noise) that cause hindrance for the deep learning models to extract meaningful features. Using just aerial view Electro-optical(EO) images for ATR systems may also not result in high accuracy as these images are of low resolution and also do not provide ample information in extreme weather conditions. Therefore, information from multiple sensors can be used to enhance the performance of Automatic Target Recognition(ATR) systems. In this paper, we explore a methodology to use both EO and SAR sensor information to effectively improve the performance of the ATR systems by handling the shortcomings of each of the sensors. A novel Multi-Modal Domain Fusion(MDF) network is proposed to learn the domain invariant features from multi-modal data and use it to accurately classify the aerial view objects. The proposed MDF network achieves top-10 performance in the Track-1 with an accuracy of 25.3 % and top-5 performance in Track-2 with an accuracy of 34.26 % in the test phase on the PBVS MAVOC Challenge dataset [18].