Image Segmentation with transformers: An Overview, Challenges and Future

TL;DR

This paper reviews how transformer architectures are revolutionizing image segmentation by addressing CNN limitations, highlighting current challenges, solutions, and future research directions in the field.

Contribution

It provides a comprehensive overview of transformer-based segmentation models, analyzing their advantages, challenges, and potential future developments.

Findings

Transformer models improve segmentation accuracy over CNNs.

Current challenges include model efficiency and data requirements.

Future trends involve lightweight architectures and better data efficiency.

Abstract

Image segmentation, a key task in computer vision, has traditionally relied on convolutional neural networks (CNNs), yet these models struggle with capturing complex spatial dependencies, objects with varying scales, need for manually crafted architecture components and contextual information. This paper explores the shortcomings of CNN-based models and the shift towards transformer architectures -to overcome those limitations. This work reviews state-of-the-art transformer-based segmentation models, addressing segmentation-specific challenges and their solutions. The paper discusses current challenges in transformer-based segmentation and outlines promising future trends, such as lightweight architectures and enhanced data efficiency. This survey serves as a guide for understanding the impact of transformers in advancing segmentation capabilities and overcoming the limitations of traditional models.