Towards a Large Language-Vision Question Answering Model for MSTAR Automatic Target Recognition

TL;DR

This paper explores the application of large language-vision models to synthetic aperture radar imagery for automatic target recognition, developing a benchmark and fine-tuning methods to identify military vehicles with high accuracy.

Contribution

It introduces a SAR-specific benchmark dataset with captions and questions, and demonstrates fine-tuning of LLVMs achieving 98% accuracy in target identification.

Findings

Developed a SAR benchmark with captions and VQA pairs from MSTAR dataset.

Fine-tuned LLVMs achieve 98% accuracy in target recognition.

Addressed challenges in applying language-vision models to SAR imagery.

Abstract

Large language-vision models (LLVM), such as OpenAI's ChatGPT and GPT-4, have gained prominence as powerful tools for analyzing text and imagery. The merging of these data domains represents a significant paradigm shift with far-reaching implications for automatic target recognition (ATR). Recent transformer-based LLVM research has shown substantial improvements for geospatial perception tasks. Our study examines the application of LLVM to remote sensing image captioning and visual question-answering (VQA), with a specific focus on synthetic aperture radar (SAR) imagery. We examine newly published LLVM methods, including CLIP and LLaVA neural network transformer architectures. We have developed a work-in-progress SAR training and evaluation benchmark derived from the MSTAR Public Dataset. This has been extended to include descriptive text captions and question-answer pairs for VQA tasks. This challenge dataset is designed to push the boundaries of an LLVM in identifying nuanced ATR details in SAR imagery. Utilizing parameter-efficient fine-tuning, we train an LLVM method to identify fine-grained target qualities at 98% accuracy. We detail our data setup and experiments, addressing potential pitfalls that could lead to misleading conclusions. Accurately identifying and differentiating military vehicle types in SAR data poses a critical challenge, especially under complex environmental conditions. Mastering this target recognition skill may require a human analyst months of training and years of practice. This research represents a unique effort to apply LLVM to SAR applications, advancing machine-assisted remote sensing ATR for military and intelligence contexts.