SAR-RAG: ATR Visual Question Answering by Semantic Search, Retrieval, and MLLM Generation

TL;DR

This paper introduces SAR-RAG, a novel retrieval-augmented generation system that enhances synthetic aperture radar target recognition by combining multimodal language models with semantic image retrieval.

Contribution

It presents a new method integrating a large language model with a semantic database to improve ATR accuracy in SAR imagery.

Findings

SAR-RAG improves classification accuracy over baseline methods.

The system enhances vehicle dimension regression performance.

Semantic search retrieval aids in distinguishing similar vehicle types.

Abstract

We present a visual-context image-retrieval-augmented generation (ImageRAG)- assisted AI agent for automatic target recognition (ATR) of synthetic aperture radar (SAR) imagery. SAR is a remote sensing method used in defense and security applications to detect and monitor the positions of military vehicles, which may appear indistinguishable in images. Researchers have extensively studied SAR ATR to improve the differentiation and identification of vehicle types, characteristics, and measurements. Test examples can be compared with known vehicle target types to improve recognition tasks. New methods enhance the capabilities of neural networks, transformer attention, and multimodal large language models. An agentic AI method may be developed to utilize a defined set of tools, such as searching through a library of similar examples. Our proposed method, SAR Retrieval-Augmented Generation (SAR-RAG), combines a multimodal large language model (MLLM) with a vector database of semantic embeddings to support contextual search for image exemplars with known qualities. By recovering past image examples of known true target types, our SAR-RAG system can compare similar vehicle categories, thereby improving ATR prediction accuracy. We evaluate this through search and retrieval metrics, categorical classification accuracy, and numeric regression of vehicle dimensions. These metrics all show improvements when SAR-RAG is added to an MLLM baseline method as an attached ATR memory bank.