From Ground to Air: Noise Robustness in Vision Transformers and CNNs for Event-Based Vehicle Classification with Potential UAV Applications

TL;DR

This paper compares CNN and Vision Transformer models for event-based vehicle classification, highlighting their accuracy and robustness to noise, with implications for UAV and autonomous vehicle applications.

Contribution

It provides a comparative analysis of ResNet34 and ViT B16 on event-based data, demonstrating ViT's robustness despite less training data.

Findings

ResNet34 achieves 88% accuracy on clean data.

ViT B16 shows strong noise robustness.

ResNet34 slightly outperforms ViT in accuracy.

Abstract

This study investigates the performance of the two most relevant computer vision deep learning architectures, Convolutional Neural Network and Vision Transformer, for event-based cameras. These cameras capture scene changes, unlike traditional frame-based cameras with capture static images, and are particularly suited for dynamic environments such as UAVs and autonomous vehicles. The deep learning models studied in this work are ResNet34 and ViT B16, fine-tuned on the GEN1 event-based dataset. The research evaluates and compares these models under both standard conditions and in the presence of simulated noise. Initial evaluations on the clean GEN1 dataset reveal that ResNet34 and ViT B16 achieve accuracies of 88% and 86%, respectively, with ResNet34 showing a slight advantage in classification accuracy. However, the ViT B16 model demonstrates notable robustness, particularly given its pre-training on a smaller dataset. Although this study focuses on ground-based vehicle classification, the methodologies and findings hold significant promise for adaptation to UAV contexts, including aerial object classification and event-based vision systems for aviation-related tasks.