Hands-on Evaluation of Visual Transformers for Object Recognition and Detection

TL;DR

This paper evaluates various Vision Transformers (ViTs) for object recognition, detection, and medical imaging, demonstrating their competitive performance and advantages over traditional CNNs in understanding global image context.

Contribution

It provides a comprehensive comparison of pure, hierarchical, and hybrid ViTs against CNNs across multiple tasks and datasets, highlighting the effectiveness of hybrid models like Swin and CvT.

Findings

Hybrid and hierarchical ViTs outperform CNNs in accuracy and efficiency.

Data augmentation significantly improves medical image classification performance.

Swin Transformer achieves a strong balance between accuracy and computational cost.

Abstract

Convolutional Neural Networks (CNNs) for computer vision sometimes struggle with understanding images in a global context, as they mainly focus on local patterns. On the other hand, Vision Transformers (ViTs), inspired by models originally created for language processing, use self-attention mechanisms, which allow them to understand relationships across the entire image. In this paper, we compare different types of ViTs (pure, hierarchical, and hybrid) against traditional CNN models across various tasks, including object recognition, detection, and medical image classification. We conduct thorough tests on standard datasets like ImageNet for image classification and COCO for object detection. Additionally, we apply these models to medical imaging using the ChestX-ray14 dataset. We find that hybrid and hierarchical transformers, especially Swin and CvT, offer a strong balance between accuracy and computational resources. Furthermore, by experimenting with data augmentation techniques on medical images, we discover significant performance improvements, particularly with the Swin Transformer model. Overall, our results indicate that Vision Transformers are competitive and, in many cases, outperform traditional CNNs, especially in scenarios requiring the understanding of global visual contexts like medical imaging.