Video Object Segmentation and Tracking: A Survey

TL;DR

This survey comprehensively reviews state-of-the-art methods in video object segmentation and tracking, highlighting their challenges, classifications, datasets, evaluation metrics, and future research directions.

Contribution

It provides a hierarchical classification of VOST methods, detailed analysis of their technical features, and insights into datasets and evaluation metrics, offering a valuable overview for researchers.

Findings

Classified VOST methods into categories like unsupervised, semi-supervised, and interactive.

Summarized characteristics of video datasets and evaluation metrics.

Identified future research directions in VOST.

Abstract

Object segmentation and object tracking are fundamental research area in the computer vision community. These two topics are diffcult to handle some common challenges, such as occlusion, deformation, motion blur, and scale variation. The former contains heterogeneous object, interacting object, edge ambiguity, and shape complexity. And the latter suffers from difficulties in handling fast motion, out-of-view, and real-time processing. Combining the two problems of video object segmentation and tracking (VOST) can overcome their respective difficulties and improve their performance. VOST can be widely applied to many practical applications such as video summarization, high definition video compression, human computer interaction, and autonomous vehicles. This article aims to provide a comprehensive review of the state-of-the-art tracking methods, and classify these methods into different categories, and identify new trends. First, we provide a hierarchical categorization existing approaches, including unsupervised VOS, semi-supervised VOS, interactive VOS, weakly supervised VOS, and segmentation-based tracking methods. Second, we provide a detailed discussion and overview of the technical characteristics of the different methods. Third, we summarize the characteristics of the related video dataset, and provide a variety of evaluation metrics. Finally, we point out a set of interesting future works and draw our own conclusions.