PVUW 2024 Challenge on Complex Video Understanding: Methods and Results

TL;DR

The PVUW 2024 Challenge on Complex Video Understanding introduces new tracks and datasets to advance pixel-level video understanding in complex, real-world scenarios, fostering development of robust video segmentation methods.

Contribution

This paper presents two new challenge tracks with datasets focusing on complex video object segmentation and natural language-guided segmentation, expanding the scope of video understanding research.

Findings

High participation in both challenge tracks.

Effective methods developed for complex video segmentation.

Rich datasets enabling advanced research in video understanding.

Abstract

Pixel-level Video Understanding in the Wild Challenge (PVUW) focus on complex video understanding. In this CVPR 2024 workshop, we add two new tracks, Complex Video Object Segmentation Track based on MOSE dataset and Motion Expression guided Video Segmentation track based on MeViS dataset. In the two new tracks, we provide additional videos and annotations that feature challenging elements, such as the disappearance and reappearance of objects, inconspicuous small objects, heavy occlusions, and crowded environments in MOSE. Moreover, we provide a new motion expression guided video segmentation dataset MeViS to study the natural language-guided video understanding in complex environments. These new videos, sentences, and annotations enable us to foster the development of a more comprehensive and robust pixel-level understanding of video scenes in complex environments and realistic scenarios. The MOSE challenge had 140 registered teams in total, 65 teams participated the validation phase and 12 teams made valid submissions in the final challenge phase. The MeViS challenge had 225 registered teams in total, 50 teams participated the validation phase and 5 teams made valid submissions in the final challenge phase.