Evaluation and Analysis of Deep Neural Transformers and Convolutional Neural Networks on Modern Remote Sensing Datasets

TL;DR

This paper compares transformer-based and convolutional neural networks for object detection in high-resolution satellite imagery, demonstrating state-of-the-art results and analyzing their performance across multiple datasets.

Contribution

It provides a comprehensive large-scale comparison of transformer and CNN architectures on remote sensing datasets, highlighting their relative strengths and performance differences.

Findings

Transformers achieve state-of-the-art detection performance.

CNNs remain competitive in certain scenarios.

Performance varies with dataset complexity and feature extraction methods.

Abstract

In 2012, AlexNet established deep convolutional neural networks (DCNNs) as the state-of-the-art in CV, as these networks soon led in visual tasks for many domains, including remote sensing. With the publication of Visual Transformers, we are witnessing the second modern leap in computational vision, and as such, it is imperative to understand how various transformer-based neural networks perform on satellite imagery. While transformers have shown high levels of performance in natural language processing and CV applications, they have yet to be compared on a large scale to modern remote sensing data. In this paper, we explore the use of transformer-based neural networks for object detection in high-resolution electro-optical satellite imagery, demonstrating state-of-the-art performance on a variety of publicly available benchmark data sets. We compare eleven distinct bounding-box detection and localization algorithms in this study, of which seven were published since 2020, and all eleven since 2015. The performance of five transformer-based architectures is compared with six convolutional networks on three state-of-the-art opensource high-resolution remote sensing imagery datasets ranging in size and complexity. Following the training and evaluation of thirty-three deep neural models, we then discuss and analyze model performance across various feature extraction methodologies and detection algorithms.