Deep Semantic Segmentation of Natural and Medical Images: A Review

TL;DR

This review comprehensively categorizes and analyzes deep learning methods for semantic image segmentation in natural and medical images, highlighting current approaches, limitations, and future research directions.

Contribution

It provides a detailed classification and analysis of recent deep learning-based segmentation methods across six main groups, offering insights into their limitations and future prospects.

Findings

Deep architectural methods are most prevalent.

Weakly supervised methods address annotation scarcity.

Multi-task approaches improve segmentation accuracy.

Abstract

The semantic image segmentation task consists of classifying each pixel of an image into an instance, where each instance corresponds to a class. This task is a part of the concept of scene understanding or better explaining the global context of an image. In the medical image analysis domain, image segmentation can be used for image-guided interventions, radiotherapy, or improved radiological diagnostics. In this review, we categorize the leading deep learning-based medical and non-medical image segmentation solutions into six main groups of deep architectural, data synthesis-based, loss function-based, sequenced models, weakly supervised, and multi-task methods and provide a comprehensive review of the contributions in each of these groups. Further, for each group, we analyze each variant of these groups and discuss the limitations of the current approaches and present potential future research directions for semantic image segmentation.