Vision Transformers and Convolutional Neural Networks for Land Use Scene Classification

TL;DR

This paper compares Vision Transformers and CNNs for land use scene classification in remote sensing, highlighting their respective strengths, limitations, and suitability depending on data size and scene complexity.

Contribution

It provides a comprehensive performance comparison of ViTs and CNNs on benchmark datasets, offering insights into their advantages and limitations for remote sensing applications.

Findings

CNNs perform well with limited data and local textures

ViTs excel in capturing global relationships with sufficient data

ViTs require more computational resources and larger datasets

Abstract

Land Use Scene Classification (LUSC) from remote sensing imagery plays a critical role in environmental monitoring, urban planning, and sustainable resource management. In recent years, deep learning methods have significantly advanced the state of the art, with Convolutional Neural Networks (CNNs) dominating the field because of their strong ability to capture local spatial features. However, the emergence of Vision Transformers (ViTs) has introduced a new paradigm that models long-range dependencies through self-attention mechanisms, potentially enabling improved global context understanding. This paper presents a comparative assessment of Vision Transformers and CNN-based architecture for remote sensing land use scene classification. Representative CNN models, such as AlexNet, is evaluated alongside the Vision Transformer (ViT) using benchmark remote sensing datasets, including the UC Merced Land Use and EuroSAT Land Use datasets. The study examines classification accuracy, precision, recall, F1-score, and computational complexity to provide a comprehensive performance comparison. Experimental results demonstrate that CNNs perform robustly on datasets with limited training samples and strong local texture characteristics, whereas Vision Transformers exhibit superior performance in capturing global spatial relationships in complex scenes when sufficient training data are available. However, ViTs typically require greater computational resources and larger training datasets to achieve optimal performance. The findings of this study provide insights into the strengths and limitations of both architectures and offer guidance for selecting appropriate models for remote sensing land use scene classification applications.