The Progression of Transformers from Language to Vision to MOT: A Literature Review on Multi-Object Tracking with Transformers

TL;DR

This literature review explores the evolution of transformer architectures from language processing to computer vision and their emerging role in multi-object tracking, highlighting recent advances and ongoing challenges.

Contribution

It provides a comprehensive overview of transformer applications in computer vision and specifically analyzes their growing use in multi-object tracking.

Findings

Transformers have significantly impacted computer vision tasks.

Transformers are increasingly competitive in multi-object tracking.

Traditional deep learning methods still outperform transformers in MOT.

Abstract

The transformer neural network architecture allows for autoregressive sequence-to-sequence modeling through the use of attention layers. It was originally created with the application of machine translation but has revolutionized natural language processing. Recently, transformers have also been applied across a wide variety of pattern recognition tasks, particularly in computer vision. In this literature review, we describe major advances in computer vision utilizing transformers. We then focus specifically on Multi-Object Tracking (MOT) and discuss how transformers are increasingly becoming competitive in state-of-the-art MOT works, yet still lag behind traditional deep learning methods.