ViTs are Everywhere: A Comprehensive Study Showcasing Vision Transformers in Different Domain

TL;DR

This paper provides a comprehensive survey of Vision Transformers (ViTs), analyzing their applications, benefits, drawbacks, and potential for advancing various computer vision tasks compared to traditional CNNs.

Contribution

It is the first survey to categorize and evaluate ViTs across multiple CV applications, highlighting their advantages and outlining future research directions.

Findings

ViTs outperform CNNs in several vision benchmarks.

ViTs are applicable to diverse CV tasks like classification and segmentation.

The survey identifies open challenges and future research opportunities in ViT development.

Abstract

Transformer design is the de facto standard for natural language processing tasks. The success of the transformer design in natural language processing has lately piqued the interest of researchers in the domain of computer vision. When compared to Convolutional Neural Networks (CNNs), Vision Transformers (ViTs) are becoming more popular and dominant solutions for many vision problems. Transformer-based models outperform other types of networks, such as convolutional and recurrent neural networks, in a range of visual benchmarks. We evaluate various vision transformer models in this work by dividing them into distinct jobs and examining their benefits and drawbacks. ViTs can overcome several possible difficulties with convolutional neural networks (CNNs). The goal of this survey is to show the first use of ViTs in CV. In the first phase, we categorize various CV applications where ViTs are appropriate. Image classification, object identification, image segmentation, video transformer, image denoising, and NAS are all CV applications. Our next step will be to analyze the state-of-the-art in each area and identify the models that are currently available. In addition, we outline numerous open research difficulties as well as prospective research possibilities.