1st Place Solution for MOSE Track in CVPR 2024 PVUW Workshop: Complex Video Object Segmentation

TL;DR

This paper presents a top-performing semantic embedding model for complex video object segmentation, effectively recognizing and distinguishing objects in challenging scenes with occlusions and splits, achieving 84.45% accuracy.

Contribution

The paper introduces a novel semantic embedding approach utilizing salient features for improved object recognition in complex scenes, leading to state-of-the-art results.

Findings

Achieved 84.45% accuracy on PVUW Challenge 2024

Effective recognition of occluded and split objects

Outperformed previous methods in complex video segmentation

Abstract

Tracking and segmenting multiple objects in complex scenes has always been a challenge in the field of video object segmentation, especially in scenarios where objects are occluded and split into parts. In such cases, the definition of objects becomes very ambiguous. The motivation behind the MOSE dataset is how to clearly recognize and distinguish objects in complex scenes. In this challenge, we propose a semantic embedding video object segmentation model and use the salient features of objects as query representations. The semantic understanding helps the model to recognize parts of the objects and the salient feature captures the more discriminative features of the objects. Trained on a large-scale video object segmentation dataset, our model achieves first place (\textbf{84.45\%}) in the test set of PVUW Challenge 2024: Complex Video Object Segmentation Track.