A Survey on Deep Learning-based Architectures for Semantic Segmentation on 2D images

TL;DR

This survey reviews recent deep learning-based methods for 2D semantic segmentation, analyzing their evolution, techniques, challenges, and performance evaluation across different eras.

Contribution

It provides a comprehensive chronological analysis of deep learning approaches for 2D semantic segmentation, highlighting technical solutions and current challenges.

Findings

Deep learning significantly advanced semantic segmentation performance.

Evolution from early CNNs to fully convolutional networks improved localization.

Current challenges include fine-grained localization and scale invariance.

Abstract

Semantic segmentation is the pixel-wise labelling of an image. Since the problem is defined at the pixel level, determining image class labels only is not acceptable, but localising them at the original image pixel resolution is necessary. Boosted by the extraordinary ability of convolutional neural networks (CNN) in creating semantic, high level and hierarchical image features; several deep learning-based 2D semantic segmentation approaches have been proposed within the last decade. In this survey, we mainly focus on the recent scientific developments in semantic segmentation, specifically on deep learning-based methods using 2D images. We started with an analysis of the public image sets and leaderboards for 2D semantic segmentation, with an overview of the techniques employed in performance evaluation. In examining the evolution of the field, we chronologically categorised the approaches into three main periods, namely pre-and early deep learning era, the fully convolutional era, and the post-FCN era. We technically analysed the solutions put forward in terms of solving the fundamental problems of the field, such as fine-grained localisation and scale invariance. Before drawing our conclusions, we present a table of methods from all mentioned eras, with a summary of each approach that explains their contribution to the field. We conclude the survey by discussing the current challenges of the field and to what extent they have been solved.