Visual Semantic Segmentation Based on Few/Zero-Shot Learning: An Overview

TL;DR

This overview paper discusses recent advances in few/zero-shot visual semantic segmentation, highlighting methods that enable segmentation of unseen categories with minimal or no labeled data, across 2D, video, and 3D contexts.

Contribution

It provides a comprehensive review of recent methods, datasets, and technical challenges in few/zero-shot visual semantic segmentation across different modalities.

Findings

Recent methods enable segmentation of unseen categories with minimal data

Commonalities and differences in techniques across 2D, video, and 3D segmentation

Future challenges include dataset limitations and model generalization issues

Abstract

Visual semantic segmentation aims at separating a visual sample into diverse blocks with specific semantic attributes and identifying the category for each block, and it plays a crucial role in environmental perception. Conventional learning-based visual semantic segmentation approaches count heavily on large-scale training data with dense annotations and consistently fail to estimate accurate semantic labels for unseen categories. This obstruction spurs a craze for studying visual semantic segmentation with the assistance of few/zero-shot learning. The emergence and rapid progress of few/zero-shot visual semantic segmentation make it possible to learn unseen-category from a few labeled or zero-labeled samples, which advances the extension to practical applications. Therefore, this paper focuses on the recently published few/zero-shot visual semantic segmentation methods varying from 2D to 3D space and explores the commonalities and discrepancies of technical settlements under different segmentation circumstances. Specifically, the preliminaries on few/zero-shot visual semantic segmentation, including the problem definitions, typical datasets, and technical remedies, are briefly reviewed and discussed. Moreover, three typical instantiations are involved to uncover the interactions of few/zero-shot learning with visual semantic segmentation, including image semantic segmentation, video object segmentation, and 3D segmentation. Finally, the future challenges of few/zero-shot visual semantic segmentation are discussed.